183 IX: IX/IXc - A General Purpose Macro Processor (183.html)

Keywords

ICH180RR "IX Macro Processor" IX Macro Processor Python html entity "&entityName|" "hash total" "hash totals" hashIX_ix.py Unix hash unix "unix date" "unix timestamp" Linux Mousepad Geany Thonny sh shell "proper Python dictionary" syntax parse DC PHP HTML .py _ix.py _ix.html _ix.sh _ix.txt _ix.php IXc "source code" source code parameter "initial value" ICH180R2 version author "null entity" "&|" LF EOL run Terminal "Terminal Window" Raspberry Pi "Raspberry Pi" Pico MicroPython "End Of Line" alphanumeric a-z A-Z 0-9 doubleQuote "double quotes" caveat "&nbsp|" CacheIX buIX Sardana "Sardana Macros" "Sardana Controls" "entity name" entityName hashIX listEntityRefsIX viewInititalEntitiesIX textPackIX macroIX marker #::::::::::&ix_textpak|=> &ix_textpak| :ix_textpak|=> ">IX:| $" IXp -python3 IXp_v1i.py ix_mLibr.txt mLibrListIX.py "nested macro" self-invocation mLibrContentsIX.py concatenate IXf pyix .pyix "IX Cache" showIt

/EndKeywords

(To enlarge .....Click it)
thumb: IXimage.jpg

IX or DC (IX by DC) or "|><"

The IX.py and IXc.py programs are part of the IX family of software.

Introduction

WARNING: Only v1i of IXp has been written as of 2023ISep06.

WARNING: IX input code (macros) must not contain any datastring strings containing "&ix_"

WARNING: Two new file extensions will be used by IX Software. They are ".bix" for IX batches and ".pyix" for definitions of entities, macro definitions and stub files that need to be preprocessed because some of their statements are NOT proper python code.

Source 10 describes IXp, a very simple version of IX for Python. Article 217 describes IXf, the latest definition of the IX macro preprocessor. IXf is not yet as comprehensive as IXp, but will become the best version of the IX macro preprocessor.

Source 03 actually contains IXp_v1i.py. It can process a single IX macro containing multiple occurances of multiple entities. To this extent, it is fully usable. It accepts a CLI statement that defines the instance of a macro invocation with multiple entity values defined.

The IX and IXc macroprocessors (and IXf) have the distinctive prompt of:

>IX:| $

IXp_v1i.py makes use of the following Python functions that were created for it:

	forcedList()		creates a List of elements (count can be any number, even 1 or 0)
	ixp_Parse()		parses a macro text file invocation
	strToAsc()		converts a string into a List of ascii values
	replaceNextEntity()	replaces one entity name with it's string value
	
	os.system()             processes a Terminal command (requires "import os")

One known issue exists: the "cd ..." command fails.

The IXp_v1i program accepts the following control phrases:

	-python3 macroinvocation	submits the resulting text file to python3 for execution
	-python  macroinvocation	submits the resulting text file to python  for execution
	-dir				displays the contents of the current folder
	-help				displays a help file for IXp
	-cd ...				fails to change directory

The IX macro processor is designed to be used with small Python processors. Initially, the Raspberry Pi was the target processor. The release of the Raspberry RP2040 (initially in the Raspberry Pico) but soon afterwards in the Tiny2040 and the Wio Terminal has made Python even more popular. The intent is for the IX macro processor porting to be extended beyond Python to MicroPython, Circuit Python, Thonny and CPython.

The macro processor named IX (whose icon is shown in the above photo) is not a text editor. It is merely a processor that processes one or more text files and then submits the modified file to be run by another program in the Terminal window on a Raspberry Pi. Perhaps it should be called a macro pre-processor. If possible, IX will be ported to the Raspberry Pico microprocessor (and/or other computers). Two versions of IX exist in the IX macro system: IX which is an interactive version and IXc which is a command line version.

IX, IXc and IXf are copyrights that belong to ICH180RR.

To be more precise, the IX Macro Processor is a text processor that reads in a text file, modifies it, writes it out with a slightly revised name, and then optionally runs a related program that uses it. The user will usually provide a list of entities (text strings) that will replace other character strings in the file that is written out. For example, presume that a text file named test1_ix.py exists. The user wishes that the character string (called an entity) "&count|" be automatically changed to "10" in test1_ix.py and be written out as a new file named test1.py. For some other programs, entity names do not have a terminator such as "|". But to be processed by IX, each entity name (that is mentioned in a macro file) must always start with "&" and always end with "|". A simple entity element in the IX command line can even insert multiple lines of code into the final output file. Because a simple element can produce multiple lines of code, the "_ix.any" files being processed are often called macros or "_ix" macro files. Finally, because Python3 will often be used to run the new file, the IXc command line to do this will often be:

	   >IX:| $ -python3 test1_ix.py?count=10 parmLast
	
	   [Note: The above prompt should be used in the IX home folder of "/home/pi/Desktop" ]

The syntax for this is similar to the syntax used when invoking PHP code.

For IX, the suffix of ".py" in the fileName tells IX that the file to be created will be a file containing Python source code. This permits the file that is created to be assigned the correct extension (e.g. ".py"). Before quitting, in this case, the IXc program will automatically invoke the following command within Terminal:

	   python3 test1.py parmLast

Within the Terminal program in a Raspberry Pi computer that is running Linux, the shell program (called a Bourne shell) is run by a command such as "sh program.sh parmLast". The resulting file produced by IX can be run by using an initial character string of "sh" followed by the remainder of the IX command. The "shebang" syntax is applicable.

An example of an IX command with an "sh" file is:

	   >IX:| $ -sh Environ_ix.sh?date=2022FJun21 parmLast

Some files (other than Python e.g. shells such as "sh"), when run, are followed by a parameter or parameter string. In the above case, the parameter is "parmLast". This final parameter is allowed in IX, so that IX will function correctly for such non-Python programs. In the case of Python, the parameter "parmLast", if present, will usually be ignored by the Python program.

It is recommended that entity names be composed of a lower case alphanumeric character string, often being only one word long beginning with a lower-case letter of the alphabet. If the user prefers to include second or third words in an entity name, the first letter of each of these two words should be capitalized. Of course, neither spaces nor "|" are allowed within entity names. Entity names beginning with "ix_" are reserved by IX and should not be used in "_ix." source files.

It is intended that the IXc command be used with a small (less than 10) number of entityNames values being specified and a small (less than 5) number of "_ix.any" files being concatenated into the resulting output file that will eventually be processed. IX will accept a list of input files (each with defined entity values) and will process them and concatenate them together in a similar manner to the Bash "cat" command. The single result file will then be passed to the appropriate program for processing. The input files should have identical extensions. Their common extension will be used to decide which standard program will be used to process the result. If the IX user wishes to specify that they be processed by a different program, the name of that program should be specified at the beginning of the IX control command. For example, a python program (with extension ".py" can be sent to Thonny for processing. This is done by beginning the IX command with -Thonny or Thonny followed by a space then by the remainder of the IX command.

The IX macro processor was created for two main reasons. The first reason was to run Python programs with simple entityName substitutions. The second reason was to permit the inclusion of macros (and entityName substitutions) when creating other files ( e.g. ".html" files). An example of the IX macro facility is described below under the heading: "Using IX to include clickableImageA_ix.html code in an HTML file".

dir or cd is allowed as an IX command

The IX commands "dir" and "cd" are permitted. But no other bash commands are allowed in IX nor IXc. The dash ("-") in front of them is optional. For example, the following command is permitted:

	   >IX:| $ dir

which (if the Desktop folder is empty) displays:

>IX:| $ dir
>IX:| $

Normally the Desktop will contain at least one folder: eg _Fla99S032G, which will be displayed as follows:

>IX:| $ dir
_Fla99S032G
>IX:| $

The IX command of "cd .." is:

>IX:| $ cd ..

It supposedly changes the current folder from "/home/pi/Desktop" to become "/home/pi". The resulting prompt is:

>IX:| $ cd ..
>IX:|/home/pi $

If the user wishes to run IX from a folder on the Desktop eg "Desktop/macros", "cd" can be used to do this:

>IX:| $ cd macros
>IX:|Desktop/macros $

WARNING: the "cd" command does not yet function correctly (and might never. . . )

The prompt includes the "Desktop/" characters so as to not confuse the user into thinking that no "Desktop" folder exists. However note that the characters "Desktop" do not appear in the simple IX prompt. This is to simplify the prompt.

Caveats

Syntax Caveat Re: Entity Names

Many programming languages permit the use of the "&" for special purposes. Furthermore "&" can be used in normal language text files and in data strings. To minimize the occurances of issues concerning the "&" in entity names, IX requires that the last character of each use of each entity name be a "|". Unfortunately some programming languages also use constructs that also begin with "&". Some even permit these constructs to terminate when a "special character" is encountered. One example is where HTML uses the entity "&nbsp". In cases where "&nbsp" is followed by (terminated by) a " " or an unprinted ";", there is no issue because an IX entity MUST be terminated by a "|" in the ".ix" file. In some PHP code, "&nbsp" can be immediately followed by a "|", a situation that might cause issues with IX. Of course, to avoid such issues, the user of IX can simply change the IX entityName to one that does not conflict. Before using IX, the prudent programmer will search the code to identify potential issues. Such issues might include all cases (references or uses) of "&" being followed by pure alphanumeric characters that are followed by a "|". A list of these cases should be consulted to avoid the use of these strings as entityNames. The IX system provides the command "listEntityRefsIX" that can also be used for this purpose.

The list of such problem cases has been greatly minimized because IX requires that every entity name begin with a "&" and end with a "|". But Beware!

When a character string defines the replacement value of an entity, it must NOT include any string beginning with "&" and ending with a "|". This will distinguish such a character string from an entity name.

Avoid These when defining Entity Values

	double quotes (eg '"')
	single quotes (eg "'")
	single space
	multiple spaces

In IX, actual spaces are used to separate elements in a list of macro invocations. Therefore spaces must be avoided when defining entity values. In IX, to insert a space in an entity value, use " " (without the double quotes). This convention also means that neither single quotes nor double quotes are needed when defining entity values.

Caveat: Use   for a space When Defining Entity Values

Examples of valid entity values are:

	10
	Hello&nbsp;World
	2023HSep10
	11:30
	three&nbsp;blind&nbsp;mice
	count=15

Repositories For IX Macro Definitions

When an IX macro is invoked on a Raspberry Pi computer, the IX program searches for the macro definition in three locations:

in the current folder
Code being developed is usually stored in the Desktop which is in "/home/pi/Desktop". A macro that is intended to be used by one, two or a small number of programs should be stored on the Desktop. This is one of the first places that IX will look for the macro definition when a macro is invoked.
in the ix_mLib.txt library
Some macros are written for multiple uses or frequent usage (invoked in many programs). These macros should be stored in the general-purpose IX macro library called "ix_mLibr.txt" which is stored in the /home/pi folder. Only IX macro definitions should be stored in this library. When IX is installed, an empty text file named "ix_mlibr.txt" is created in the /home/pi folder. The IX user can add macro definitions to this library. An IX user can choose to never use the ix_mLibr.txt library.
in the ix_mLibList
Often, a macro definition will be used in only one program, for single use. Such a macro should be defined "in-line" within the program that invokes it. It should be defined immediately before its invocation. In fact, the invocation of the defined macro will serve to define the "end" of the macro definition. A user can choose to never use in-line macro definitions. Note that this ix_mLibList list is an internal list and is NOT saved anywhere outside of this program.

Some users might choose to store all of their macros in the Desktop folder, however this can cause "macro-clutter" in that the Desktop might eventually contain too many files. If this occurs, the IX user should move some macros either into the IX macro library or as in-line macro definitions. It is recommended against having more than one macro with the same name. This is especially problematic if the same macro is defined in more than one of these three locations. A simple way to correct this is to insert a distinguishing letter (eg "A" or "B") to the name of each of the macro definitions to make their names become different. If the two macros of the same name are different versions of the same macro, delete the older deprecated macro definitions. Users of IX who have few macro definitions should simply store them in the Desktop folder. It is a simple matter to copy the ix_mLibr text file from one Raspberry Pi system to another. But IX users who do this might eventually be unable to recall on which Raspberyy Pi microSD card each IX macro is stored. Regular backups can help to address this issue. A simple way to backup the ix_mLibr.txt library is to append it to a self-addressed email. The python program named "mLibrContentsIX.py" will sort and list all of the macros in an ix_mLibr.txt library. Grep can also produce (an unsorted) list of macros in the "ix_mLibr.txt". The IX user should NOT allow a file named "ix_mLibr.txt" to exist elsewhere than in the "/home/pi" folder.

A simple example of an in-line IX macro definition named show_ix.py" follows:

	line 1 #progD.py
	line 2 #by D@CC on 2023ISep06
	line 3 #Purpose: define and invoke the "show" function
                         using a simple show_ix.py macro
	line 4 isP = 1 # cause show to list the contents of a variable
	line 5 count = 9
	
	line 6 #&| &macroIX|:show_ix.py &|
	line 7 show('"&v|"',&v|,isP)
	line 8 show_ix.py?v=count
	line 9 show_ix.py?v=isP
	line 10 print("end of progD.py")

When the author mentally reviews the simple program shown above, he thinks of each line as meaning:

	line 1 assigns a name to the python program
	line 2 states the author and date written
	line 3 states the purpose of the program named progD.py
	line 4 The isP = 1 boolean switch will "turn on" show so it willl list the name and contents for debugging
	line 5 count = 9  simply assigns the value 9 to the variable "count"
	
	line 6 assigns a name to the macro defined by the next line
	line 7 is the definition of the contents of the show_ix.py macro
	line 8 invokes the show_ix.py macro with v=count to show the value of the variable "count"
	line 9 invokes the show_ix.py macro with v=isP to show the value of the variable "isP"
	line 10 indicates the end of this program

Unfortunately, the "show" macro should have been named "showIt" to avoid confusion between the show macro and the show() function. Elsewhere in his articles, the author will refer to the deprecated show macro as its replacement, the showIt macro. A simple freefind search of the author's articles will discover mention of the showIt macro elsewhere. Readers should also note that the IXp.py program will coexist with Athe IXf described in article 217. Note that (as of 2024CMar17) the only articles that make mention of showIt() are Articles 207, 209, 210, 215, 217 and this article 183.

To minimize the keystrokes to invoke the show() function, the author will begin using the more concise version of the s() macro below:

	line 1 #progD.py
	line 2 #by D@CC on 2023ISep08
	line 3 #Purpose: define and invoke the "show" function
                     #  using a concise s.py macro
	line 4 isP = 1 # cause show to list the contents of a variable
	line 5 count = 9
	
	line 6 #&| &macroIX|:s.py &|
	line 7 show('"&v|"',&v|,isP)
	line 8 s.py?v=count
	line 9 s.py?v=isP
	line 10 print("end of progD.py")

It is interesting to note that while processing the lines that define the macro, IX is looking for the first line that contains a "?". Line 8 is the first line that contains the "?" that signifies the invocation of the macro "s.py". It is important that IX also checks the name of the macro that is ending. If another macro is nested within the macro "s.py", then IX must NOT consider this "?" as being the invocation of the macro. This would be because the "?" is the invocation of a nested macro.

The IX macro processor does not support the self-invocation of the macro within the macro definition. The previous paragraph shows that the macro will be terminated with the statement that precedes the self-invocation of the macro.

The above IX macro definition and invocation produces the Python program below:

	line 1 #progD.py
	line 2 #by D@CC on 2023ISep06
	line 3 #Purpose: define and invoke the "show" function
                         using a simple show_ix.py macro
	line 4 isP = 1 # cause show to list the contents of a variable
	line 5 count = 9
	line 6 show("count",count,isP)	
	line 7 show("isP",isP,isP)
	line 8 print("end of progD.py")

Note that "s.py" does NOT follow the IX macro naming convention. This is done to make it more concise when coding.
The resulting program shown above outputs the following:

	count: 9 is of type <class 'int'>
	isP: 1 is of type <class 'int'>
	end of progD.py

The author frequently makes use of the above s.py macro. After the program is completely debugged, the variable isP should be set to 0 which makes the show() function print nothing. Note that the "show()" function must also be defined for the above program to work correctly. It is interesting to note that the definition of "show()" function can be inserted once immediately before the definition of the "show_ix.py" macro. The author intends to add a "request show()" capability to the IX macro processor which will provide a rudimentary "include" function to the IX macro processor. The definition of the show() function can be found in Source 12. But a simplified version of the show() function can be found in Source 13. It is the simplified version that was used to create the output shown above.

An IX macro example in Python Code

	   >IX:| $ -python3 anyPreTest1_ix.py?steps=10 test1_ix.py?count=5 parmLast

The above command will cause IXc to process the (macro) text file named anyPreTest1_ix.py and then process the (macro) text file named test1_ix.py. Then it will write out the two processed files (concatenated together) into the single file named test1.py whose first name is taken from the last "_ix.py" text file (by removing the suffix "_ix" from the name. The extension (e.g. ".py") is used to create the output file extension (e.g. ".py" in this case). Then the following command will be automatically run within Terminal:

	   python3 test1.py parmLast

Note that the optional dash ("-") preceding the python3 command is dropped before the statement is executed.

test1_ix.py and test1.py (Simple IX macro Source Code Examples)

# program test1_ix.py
# becomes
# program test1.py
for i in range(&count|):
    print("line",i+.1)
#for end

becomes

# program test1_ix.py
# becomes
# program test1.py
for i in range(10):
    print("line",i+.1)
#for end

The first 6 lines of the code (above) can be converted by IXc into the next 6 lines of code using the IXc command:

	   >IX:| $ -python3 test1_ix.py?count=10

The resulting program will be run by python3 to print:

line 0.1
line 1.1
line 2.1
line 3.1
line 4.1
line 5.1
line 6.1
line 7.1
line 8.1
line 9.1

Assigning Initial Values to Entities by IX (or IXc or IXp)

Some initial values will be permitted to be assigned to entities (within the IX program) by preceding the text file by the definition of one entity on each line. This permits some IX macro programs to be run without need for their internally-referenced entities to be defined in the commandLine. For example:

#&| &count|=5 &|
# program test1_ix.py
# becomes
# program test1.py
for i in range(&count|):
    print("line",i+.1)
#for end

Each such initial value line must begin and end with the null entity which is "&|" and it must be surrounded by the comment tags. For python, this means that the statement must be preceded by an "#".

Two types of comments are permitted. Normal python comments begin with "#". But IX comments for Python are available. They will be suppressed (not included in the output file). These Python IX comments are preceded by "#&|" Some examples of recommended initial values for entities and a few IX Python comments are:

#&| &hashMD5_IX|= 7f9f3e0a256c07b876b6d31dc295f1ab &|
#&| &hashAsOfUnixDate|="016754534079" &|
#&|   the preceding 2 lines define a dated hash total for the text in the &|
#&|   remainder of this text file (the hash excludes the first line) &|
#&| &unixDateCreated|="016754534079"  # date when initially created &|
#&| &unixDateModified|="016754534079" # date when it was last modified &|
#&| &version|="024," # defines the version number (024 in this case) of this ".ix" text file &|
#&| &author|="David@ColeCanada.com" &| 
&|  This is an ix comment that will NOT be included in the resulting text file. &|
&|  another comment that will NOT be included in the resulting text file. &|

Note that lines defining initial values for entities must begin with "#&| &" and must end with "&|". Because they are in Python code, these initial values must be immediately preceded by a "#" to be treated as Python comment lines. Furthermore, comment lines in IX programs are allowed but they must begin with "&| " and must end with "&|".

The first line (above) begins with the prefix "#&| &hashMD5_IX|= ". This defines the MD5 hash total of the remainder of this IX macro. The md5_IX.py program in Article 206 has not yet (as of 2023JOct09) been updated to use this format of the MD5 hash total in the first record of files that it has processed.

In all cases, any lines beginning with "&| " will NOT be copied into the Python file that is written out.

Hash Totals of IX Functions (using the future hashIX.py program)

A novel use of hash totals will be incorporated into a future version (version 3) of the IXc preprocessor. To avoid unintentional edits (or errors or changes) to an IX function, the hash total and the unix date of a function in a folder will be saved in the first two lines of each macro function. IXc will always verify this hash total to ensure that the IX function source code has not been altered. See Source 09 for an example of the standard Python hash function that will be used.

Special Lines in a "_ix" Text File

In a text file (called a macro) that is to be processed by IX (or IXc), the following special line types can be used:

The first two lines of a ".ix" file can contain a dated hash total of the remainder of the text file. They appear with the syntax of an initial Entity value definition, therefore they must be surrounded by comment tags.


1. The first line lists the hash total for the remainder of the text file.   
    e.g. #&| &hash|="jkjlklkjljfdsasdfkj;dfsal;jsdflkjfd" &|
2. The second line is the Unix Date when the hash total was calculated.        
    e.g. #&| &hashAsOfUnixDate|="016754534079" &|
3. Any line can contain a comment for that file type (except in ".txt" files). 
    e.g. 
4. Any line can include another "_ix" file with entity substitutions.       
    e.g. #&| &macroIX|readFileRecords_ix.py?count=10 &|
      by using the special &macroIX| entity to include a "_ix" file. 
    This future construct may be deprecated if serious issues arise.
5. The easiest way to optionally delete multiple lines of "_ix" code is 
    to define each whole line as an ix comment.        
6. The IXc command can be used to concatenate all of the files in a 
      folder together into a single file named fileName.pkg 
      using the "textPackIX" command shown below:    ########## to verify #########     
    e.g. textPackIX all_ix.pkg?ix_eachMarkedFile
                                                                        
      where ix_eachMarkedFile = ix_marker>ix_fileNameAndInfo
      Note that "all.ix" means: for each file with extension "_ix.lib"         
      where &ix_marker| = "#::::::::::&ix_textpak=>"+"="  ?????
                                                                          
      where &ix_fileNameAndInfo| = &ix_fileInName|LF&ix_fileText|

      Note that the marker (followed by the file name) will precede           
      each file in the resulting file. For more info about the original
      textpack process see Source 02. If &marker| is null, marker lines will
      be absent resulting in a simple concatenation of all the "_ix" files 
      that were in the directory.  

      For example the "testPackIX_pkg test1_ix.py" command will create
      and use the following entities:                                       
       &ix_fileText| = the seven lines of file test1_ix.py
                                                                        
       &ix_fileNameAndInfo| = &ix_fileInName|LF&ix_fileText|
                                                                          
       &ix_fileName| = "test1_ix.py"
                                                                          
       &ix_fileNameAndInfo| = "test1_ix.py" 

                    plus the seven lines of file test1_py.ix resulting in: 

       &ix_eachMarkedFile = "#::::::::::&ix_textpak=>test1_ix.py"
                    plus LF and the seven lines of file test1_ix.py
      
      Note that the marker includes an entity viz &ix_textpak| which
      will be discovered if the command "listEntityRefsIX" is run.  This
      situation will occur if a ".pkg" file is mistakenly processed as
      a "_ix.any" file.

      Before using the textPackIX command
      any "_ix.any" files to not include in the package should have their
      extensions temporarily converted to another extension (such as
      ".nix".)  But be sure to remember to change the ".nix" back to
      "_ix.any" after the "textPackIX" operation is complete.

Using IXc to include clickableImageA_ix.html in an HTML file

The "_ix.html" file named "clickableImageA_ix.html" (shown below) can be inserted into a "_ix.html" file with the following statement:

     <!-- &| &macroIX|:clickableImageA_ix.html?imageNameA="USASCII-8bit.jpg" &| -->

that invokes the IX processor's macroIX "include" functionality. The first line is an "html" comment containing &macroIX| followed by a ":" followed by the macroName which is "clickableImageA_ix.html". The name of the macro is sufficient if no entity values are needed. In this case, an entity value must be supplied. The macroName can be optionally followed by a "?" followed by a definition of the value assigned to each entityName for use in the macroName. In this case, only one entityName: imageNameA is needed. Source 11 contains the text file that defines the macro definition named "clickableImageA_ix.html". Such macro definitions are normally placed in the folder where the final version of the code will be written.

                        <!-- &| &macroIX|:clickableImageA_ix.html?imageNameA="USASCII-8bit.jpg" &| -->
						(To enlarge .....Click it)
			<a href="USASCII-8bit.jpg" title="USASCII-8bit.jpg">
			<br>
			<img src="USASCII-8bit.jpg"
			alt="thumb: USASCII-8bit.jpg" height ="200" border=0><br>
			USASCII-8bit.jpg </a><br>	<br>

           The above "_ix.html" file will display any image when the html file (using it) is browsed using 
           an internet browser such as Chromium.  In fact, this exact syntax was used to display the 
           "USASCII-8bit.jpg" image shown a little later below. To verify this, closely examine the source code 
           of this web page.  This same IX macroIX invocation can be used to display many different images 
           in the resulting ".html" code.  Note that it would make sense to also use an IX entity to define the
           image height in this macro definition. It would also be better if the image description could be 
           something other than the image file name.  Another improvement would be to include a Figure 
           number or Reference number in the IX macro caption.

The 8-bit USASCII Codes

The 8-bit USASCII code definitions are displayed in the image below:
(To enlarge .....Click it)
thumb: USASCII-8bit.jpg

USASCII-8bit.jpg

Perhaps a more knowledgeable reader can explain why rows "80" and "90" display such strange characters. Most of the boxes in other rows have a character above a 4 digit hexadecimal number. Why do many of the boxes in rows 80 and 90 have 4 digit hexadecimal numbers of the form "20xx" and not "00xx"? Doh!

Actions Taken By The IXc macro Processor

The IXc.py macro program performs the following actions:

1. Creates (and defines some of) the following entities:

entityName         value
----------         -----
&ix_runDict|       ( ("hashIX,ix":"#,"), ("python,py":"#,"), 
                     ("Mousepad,html":"<!--,-->"), ("Mousepad,php":"//,"), 
                     ("Mousepad,txt":","), ("sh,sh":"#,"), 
                     ("Geany,txt":","), ("Thonny,py":"#,"), 
                     ("python3,py":"#,") )
&ix_buIX|          "/home/pi/buIX/"
&ix_ix|            "/home/pi/ix/"
&ix_Cache|         "/home/pi/ixCache/"
&ix_extEmpty|      "txt"
&ix_delimiterDef|  "?"
&ix_EOL|           LF (0x0A which is ctrl-J)
&ix_doubleQuote|   '"' &#34; (0x22)
&ix_null|          "&|"
&ix_upKey|         "^" (0x5E)  Note: support for the actual upKey is deferred 
                                     because it is very complex to implement
&ix_nbsp|          "&nbsp;" non-breakingSpace (0xA0)
&ix_breaks|        "&|=#^?" or a &ix_doubleQuote| or &ix_EOL| or a space
&ix_textpak|	   ":textpak=>" marks the start of a "new" module in a package
&ix_controls_P|	   ("hashIX","Python,"Mousepad","sh","Geany","Thonny","MicroPython",
			"Micro Python","Micro-Python","Python3","CPython","Circuit Python")
&ix_controls_I|    ("CLi","I")
&ix_version        "01"    numeric cannot contain "v1i"
&ix_extIn|         variable
&ix_runIn|         variable
&ix_extNew|        variable
&ix_commandIn|     variable
&ix_objectIn|      variable
&ix_objectOut|     variable
&ix_delimiterIn|   variable
&ix_parametersIn|  variable


Many of the entities listed above are included in the ix_eDict.txt file.
It is recommended that a user increment its &version| entity ix_runDictdefinition whenever a "_ix" file is modified.
It is intended that the IX processor be used with the many programs listed in the &ix_runDict| entity
     shown above. Note that the &ix_runDict| is not a variant of a proper Python dictionary because 
     the program names are not unique.  Note also that the extension names are also not unique. All 
     combinations appearing in the &ix_runDict| can be used.
The &ix_runDict| is a list of special character strings for each language that can be editted using IX.
     Each entry in &ix_runDict defines the following 4 character strings:

        program used to process a specific type of text file                             e.g. "python3"
        an extension for the type of text file that is processed by the program          e.g. "py"
        character (or string of characters) used at beginning of line to start a comment e.g. "#"
        character (or string of characters or null) used to end a comment                e.g. ""

Note that the IXc program itself does not modify a file with an extension of "_ix".
     One possible exception is the "hashIX" command that might place the hash information in the first line of a "_ix" file
     See Source 09 for an example of the simple MD5 and SHA1 hash functions that exist in Python.
Note that all entity names beginning with "ix_" should be reserved only for use by the IX program.
Note that spaces should not appear in unquoted character strings that are substituted for an entity.
     Instead, follow this example:
        e.g. "Hello World" or Hello&nbsp;World where &nbsp; will result in a non-breaking space
Note that the hashIX program command line is not permitted to include the &ix_delimiterDef| ("?").  The hashIX program
     is only permitted to modify (or insert) the first line of an ".ix" text file.  This is because
     the hash total and the date are stored in these first two lines.  It makes no sense for the hashIX
     command line to operate on more than one "_ix" file each time it is run.
Note that the hashIX program lists the hash and date entities after calculating the hash of the 
     remainder of the "_ix" text file. It prints out, then replaces any hash total (and maybe the date) 
     entities that previously existed.  If the hash total remains unchanged, the previous date and a notification 
     are printed out but they are not changed in the "_ix" file.  The IX command to do this is simply:


>IX:| $ -hashIX test1_ix.py


2. Reads a single command line from the user
    it is analyzed according to the following syntax:

   &ix_runIn| &ix_objectIn|&ix_delimiterIn|&entity1|="string1",&entity2|="string2" etc &ix_parametersIn| &ix_EOL|

   An example is shown below:

   
      >IX:| $ -python3 test1_py.ix?count1=3,count2=7 LF
   
   Note that the doubleQuotes enclosing the strings are only necessary if spaces or characters in &ix_breaks| are in the strings.


3. Processes the command line as follows


   a) if &ix_run| is not in the runDict &ix_runDict|          an ix_runDict warning is raised
        and the entity value is set to &ix_undefinedEntityValue
   b) if &ix_run| =&ix_run1| and &ix_extIn| is not &ix_ext1|   a warning is raised
   c) if &ix_ext| is empty 
       and &ix_runIn| is in &ix_runDict| then
       &ix_extNew| = &ix_ext1|
   d) if &ix_ext| is empty 
       and &ix_run| is not in &ix_runDict| then
       &ix_extNew| = &ix_extIn| if it is not null
   e) if &ix_ext| is empty 
       and &ix_run| is not in &ix_runDict| then
       &ix_extNew| = &ix_extEmpty| if it is null
   f) if &ix_ext| is empty 
       and &ix_run| is in &ix_runDict| then
       &ix_extNew| = &ix_ext| from the runDict
   g) &ix_extNew| =&ix_extIn|
   h) if &ix_objectNameIn| is null                             an error is raised
   i) &ix_objectName| = &ix_objectNameIn|.&ix_extNew| 
   j) &ix_objectOut|  = &ix_objectNameIn|.&ix_ext|
   k) if &ix_delimiterIn| is not &ix_delimiterDef|             an error is raised
   l) if &entityDef1| is not of format a=b,                    an error is raised
   m) parse &entityDef1| to define &characterString1|
   n) parse &entityDef2| to define &characterString2|
      etc
   o) the last entity definition is followed 
         by at least one space and an optional parameter string
   p) &ix_parameters| is the parameter string followed by
         zero or more spaces and an EndOfLine character

   Note that each entity name is limited to only include alphanumeric characters
       (a-z, A-Z, 0-9) and must begin with "&" and finish with "|".  The first
     character of an entity name must be a lower-case alphabetic character.

   Note that if any break character (in &ix_breaks| or a doubleQuote or a LF)
     is in &characterString1,
     then &characterString1 must be enclosed in a pair of doubleQuotes.  
     Each single doubleQuote that is in the original &characterString1, 
     must be cited as ".

              **to verify**

   Note that an entity value cannot include any EndOfLine characters, such as
     LF.  Instead
     the &macroIX| must appear in the "_ix" file. In
     this case, the entity in the input file should be at the beginning
     of a line and should be immediately followed by an EndOfLine character. A
     macroIX statement must be surrounded by null entities and the type of
     comment tags for the type of file beging processed.  For Python, the
     initial comment tag is "#" and the final comment tag is null.  For HTML
     the initial comment tag is "&!--" and the final comment tag is "-->".
     

4. Opens the input file named &ix_objectIn| and the output file named
      &ix_objectOut|.
   Opens and reads each line of &ix_objectIn|, processing each line as follows:

   a) search for any string matching &entityDef1|
   b) replace this entityName by the character string defined for it
   c) writes the (possibly modified) line to the output file
   d) after processing the last line, closes both files.

5 Prepares and executes the following command line automatically within Terminal:

  &ix_run| &ix_objectOut| &ix_parameters|
   
6. Quits

The IX and IXc programs will (initially) be written in Python3. This is because Python3 has strong parsing capabilities and hash functions.

View Initial Entity Values in a "_ix" File

The following command lists all of the lines of a "_ix" file that begin with the null entity: "&|". This permits the user to view all of the entities that have initial values defined in the ".ix" file. The initial entity values must only appear at the beginning of the "_ix" file. The string "any" can be any text string. A typical value for "any" would be "py". The command is:

>IX:| $viewInitialEntitiesIX progName_any.ix

List Entity Refs in an IX File

The following command lists all of the lines of a ".ix" file that use (i.e. reference) any IX entity of the form: "&entityName|". Entities that are not assigned a value during IX (or IXc processing) will be converted to a null string. This permits the user to view all of the entities that can be changed in the ".ix" file. An example is:

>IX:| $listEntityRefsIX progName_py.ix

IX With Prompts For All Entities

IX is an important variant of IXc. IX prompts the user for each entityName that exists in each ".ix" file that is processed by IX. An example is shown below:

 >IX:| $
python3 anyPreTest1_py.ix?steps=10
       test3_ix.py?count=5,
                   date="January 7, 2022"
       lastParm

Note that every entityName (except Initial Value entities) mentioned in each "_ix" file will be automatically typed as a prompt followed by the value (character string) most recently used for that entityName. IX does not prompt the user for entity Names with an initial value but which are not referenced in the "_ix" file. The user can then type in any of the following options:

     -"," to accept that value for the entityName.
     -"upKey" to see the next most recently used value for that entityName.
     -LF (Return key) to accept this value and not be prompted for any more entityNames for this "_ix" file
     -enough "backSpaces" to replace the "=" by "0=" followed by the LF (Return key).  This
           will cause the initialValue of the entityName (from the "_ix" file) to be displayed as a prompt.
     -"backSpaces" then a character string to be used as the current value for this entityName
           followed by typing a "," to be prompted for the next entity name or
           followed by a LF (Return Key) to allow the user to type in the name of the next "_ix" file

In all cases, "what you see" displayed is "what you get" processed by IX.

Note that the name of an "_ix" file must be followed by the "_ix" and the extension. When using IXc, instead of typing in the name of another "_ix" file, the user can stop mentioning "_ix" files by typing in the parameters to be appended to the run command (if any). If any spaces are contained in the character string that defines an entity value (or defines the parameter string), the whole character string must be enclosed in doubleQuotes. Furthermore, if the parameter string ends in "_ix", the whole parameter string should be enclosed in doubleQuotes. This is to distinguish the parameter string from another "_ix" file name.

The IX command is the most efficient method of using IX (compared to IXc) simply because it usually involves the least typing. It is intended that the first line of the previous IX (or IXc) command (in this session or in previous sessions) will be typed by IXc as a prompt if the user enters the "backSpace", "upKey" or "Return" key instead of a command for the IX program.

IX Cache

The IX (and IXc) programs normally keep a record of all commands that are processed, all "_ix" files used by each command and a copy of every file that is created by IX (and IXc). All three file groups are stored in a folder named &ix_buIX| which is initially defined as "/home/pi/buIX/" presuming that IX (or IXc) is being run on a Raspberry Pi. Unless changed, "/home/pi/buIX/" will unfortunately be on an SDcard, which might fill up quickly. The purpose of the first group of records is so that IX can make use of the command (and entityName definitions) used in each previous IX session. The purpose of the second group of records is so the exact "_ix" files used can be found and used if the process needs to be repeated. The purpose of the third group is so that a user can always find the exact file previously automatically created by IX (or IXc). The Unix time portion of the 3 file names that were used in each session will be identical. Because of this, the 3 files defining each session will be grouped together in the IX Cache directory lists.

The names of the files in the first group are of the format "016754534079_C_IX.txt" or "016754534079_C_IXc.txt". The names of the files in the second group are of the format "016754534079_I_ix.pkg". The names of the files in the third group are of the format "016754534079_F_test1.py". The first 12 characters define the Unix time (date and time) when the file was created. The next two characters of these filenames define the group. For the first group, the remaining characters define the program (IX or IXc) and command (line) that created the cached file. The names of the "_ix" files in the second group follow the &ix_marker| that precedes each "_ix" text file in the package. Even the "_ix" macros cited within the "_ix" files are included in the package in the second group. For the third group, the remaining characters define the name of the file that was created by IX (or IXc). The third group also serves to verify that the initial result is identical to the result produced if the IX Cache is used to redo the IX procedure. The only reason why they might not be identical is if the version of IX (or IXc) differs.

If &ix_buIX| is "" (i.e. null) or invalid, nothing is recorded in any folder, because the IX Cache folder name is unusable. In this case, IX will be less efficient than IXc. To avoid problems associated with the use of SDcards, it is recommended that the IX Cache be relocated to be stored on removeable media such as an external hard drive, SSD (SolidState Drive) or a removeable flash memory drive. Beware that the IX Cache will grow and will eventually occupy a large quantity of storage space. Furthermore, regular backups of the IX Cache folder are also recommended.

LF Issue With Python's Sequential Files

Sequential Python files are stored with a LF at the end of each record. When its records are read, the string of characters that is read ends with a LF. The write command appends a LF to each record that is written (unless the write command is instructed otherwise.)

An Example That Can Be Processed By Python or IX

Web Source 06 shows that Python code can be processed by either Python or IX if it is coded with this in mind. Of course, entityName substitutions will NOT occur if the code is not processed by IX. The article that you are reading should be updated so as to be compatible with the syntax shown in Source 06.

Future Issues To Be Addressed By IX (as of 2023ISep09)

Issue 1: If a macro can be defined and imbedded in another macro, how is the "end of macro" defined?
Solved by a statement that invokes the defined macro
Issue 2: If a macro is defined more than once and/or is available in another folder, which one is used?
Order is 1) first occurence in the in-line macro
2) first occurence of the macro defined in the current folder
3) first occurence of the macro defined in /home/pi/mLibr.txt
Issue 3: Only external macros (those in folders, not defined in-line) have embedded hash totals.
Issue 4: In-line macros are NOT assigned a hash total.
Issue 5: When a macro is invoked, which ix_entity value is used?
Order is 1) first value defined in the invocation
2) first value defined in the macro definition
3) most-recently value defined in another macro efinition.
Issue 6: Are in-line macros saved for future runs of IX?
Issue 7: How is a clashing hash total handled?
(The html of this article has been updated, but Article 182 is still deprecated.)

Planned Roll Out of the IX Program

IXc (and IX) will be programmed and released in the following versions / order:

version  0c: IXc.py that only processes one specific simple command: python test1_py.ix?count=3
             and "_ix" files can have entities of the form &entityName|
version  1i: IXc, IXp or IXf command: 
                           >IX:| $ -python3 progName_ix.py?entityName="seconds=10"
version  2 : supports control parameters (beginning with an optional "-")
version  3 : IX python concatenation:  
                           >IX:| $ pName1_ix.py progA_ix.py . . . program_ix.py
                  or       >IX:| $ pName1.py progA.py . . . . . . program_main.py
                             The latter outputs and runs a file with name: program.py
version  4 : supports hashIX.ix progName_ix.py
version  5 : supports textPackIX fileName_ix.py?marker="#::::::::::&ix_textpak|=>"
version  6 : "_ix" files can define entities with initial values
version  7 : "_ix" files can house macro definitions of the form: &macroIX|:fileName_ix.py?imageName=a.jpg
             (version 7 might be deprecated if it causes too many issues.)
version  8 : supports various runTime programs/formats (eg Python3, html, php, Geany, sh)
version  9 : supports viewInitialEntitiesIX pName_ix.py
version 10 : supports listEntityRefsIX pName_ix.py
version 11 : incorporates the IX Cache in the IX system
version 12 : supports viewing the contents of the mLibr.txt
                 >IX:| $ mLibrContentsIX.py
version 13 : IX with prompts for all undefined entity values
version 14 : >IXf t02_batch.py where t02_batch.py is a batch file invoking FRUM statements (see Articles 209 and 217 [Source 15] ).

Version 1i of IXp can be found in Source 03 below. It has no known issues. It adequately demonstrates the use of the IX (or IXc) system. It can be used to process test1_ix.py (in Source 04) to convert it into test1.py and then to automatically run it using python3 in the Raspberry Pi Terminal window. Note that Sources 03 and 04 are stored as ".txt" files that should have their extensions changed to ".py" after being downloaded. Any IX Python macro can be expanded using IXp. This version corresponds to rollout version 1.

Version 2 supports concatenation of "_ix.py" or ".py" code files. If a list of ".py" files is provided where the name of the last ".py" file has a suffix of "_main", the file output and run will have a suffix ".py" and not have the name suffix "_main". This permits the required functions to be listed so that they will be concatenated together and run as a single Python program. This is an alternative to using many import statements in a main python source code file. Note that the file output will contain all of the code in all of the files mentioned in the command. This facilitates the development of a single replacement routine or function, because the IX Cache retains a copy of all of the source code for every trial run. The user should specify that the IX Cache be changed from being on an SDcard to be on a removable medium when doing this. (Note that the IX Cache is initially defined to be "/home/pi/buIX/" by the IX entity named "&ix_buIX|". The size of the IX Cache is not an issue if the bootable volume is an SSD. Using a bootable SSD such as "_KIOXIA_GTLL" (even via USB-3) instead of a uSD card is recommended and can accomodate a huge IX Cache folder. Note that the IX Cache is initially defined to be in the "/home/pi/" folder. It is placed here so that it will NOT be backed up when the Desktop is regularly backed up.

In Source 14 (Article 209), as of 2024CMar15, the author presents IXf which is a revised description of the IX processor. The IXf will be the first program that will be used to produce IX python code and run the resulting program. Such incoming code is called "IX python code" (not python code) because it contains unexpanded macros. The author has decided on (the use of an ".pyix" extension for python code that contains IX macros and/or requires statements. This is because such code modules cannot be processed directly by python.

Sardana Macros

Sardana macros process Python code in a similar manner and have quite complex syntax. They can cause code to run in other computers. The reader should not mix IX macros with Sardana macros unless he/she understands both of them very well. For more information refer to Source 05.

Sources

Video Sources

Video Source V183:01: First Look at the Tiny2040 (3:14 min)

by Learn Embedded Systems c2021DApr

Web Sources

Web Source S183:01: US ASCII Chart 8-bit

at charset.org/charsets/us-ascii on 2022FJun15
Web Source S183:02: Pi: Creating Python Packages of Functions (174.html)

by D@CC on 2022BFeb26
Web Source S183:03: IXp_v1i.py (a Python file with txt extension)

by D@CC on 2023ISep06
Web Source S183:04: test1_ix.py (a Python _ix.py macro with txt extension)

by D@CC on 2022FJun15
Web Source S183:05: "more complex" Sardana macros (used in other computers)

by Sardana Controls discovered on 2022FJun21
Web Source S183:06: Example processed by Python or IX

by D@CC on 2022FJun22
Web Source S183:07: A&C: ixmRPC (eRPC lite) Remote Procedure Calls (193.html)

by D@CC on 2023EMay03
Web Source S183:08: www

Wio Terminal Battery Chassis Status Display Program

by Seeed Studio as of 2023DApr06
Web Source S183:09: www

Python Functions to create hash totals

by Randall Hunt as of 2014BFeb27
Web Source S183:10: www

IT: IXp - A Simple Macro Processor for Python (182.html)

by D@CC as of 2023HAug28
Web Source S183:11: clickableImageA_ix.html (an _ix.html macro with txt extension)

by D@CC on 2022FJun15
Web Source S183:12: www

Pi: ix_all Library (My Python Functions) (155.html)

by D@CC as of 2023HAug28
Web Source S183:13: minimal show.py (a Python file with txt extension)

by D@CC on 2023ISep06
Web Source S183:14: www

IX Family of Software: requires() Phase I (209.html)

by D@CC as of 2024CMar15
Web Source S183:15: www

IX bash utilities and IX macros (217.html)

by D@CC as of 2024CMar19

/EndSources

WebMaster: Ye Old King Cole

There is a way to "google" each of the part-numbers, words and phrases in all my articles. This "google-like" search limits itself ONLY to the author's articles. Just go to the top of "ePC Articles by Old King Cole" and look for the "search" input box named "freefind".

Click here to return to Articles by Old King Cole

/183.html