RELEASE NOTES
=============

This imp77 implementation is a prototype release with some known limitations.

It is an experimental back-end for the Imp77 portable compiler's intermediate code, and
generates C code rather than architecture-specific machine code.

This document lists the known issues with the current implementation, to allow a small group of trusted
alpha-testers to explore the limitations of this method and steer the course of the reimplementation.
This code is not intended for public release other than to these trusted users.

(A refactoring is underway which will smooth out some of the 'rough edges' of this
prototype, which was written to gain experience in using C as a portable back-end for Imp77.)

The compiler is accessed by the "i77" command.  Although at the moment "i77" invokes a second
executable, "i2c", to do the actual compilation, "i2c" is a hold-over from an earlier release
and will ultimately be removed.  Please don't use "i2c" directly other than at the request of
the author if you are helping debug some specific issue.


Here are some of the limitations of this pre-release code that you should be aware of:


* The associated built-in support routines (sometimes known as 'perms') are not fully developed; they are
a placeholder, so errors caused by incomplete or faulty library calls are to be expected.  Feedback on
specifically which calls require improvement is welcome, moreso if accompanied by corrected code.  I'll
prioritize adding calls that you need for your own programs.


* The implementation relies heavily on Imp77 being case-insensitive but Linux external linking being
case-sensitive, so external procedures written in Imp77 appear by default as upper case to non-Imp77 code.
If linking with system libraries, external references to procedures in other languages (such as C) should
be specified using Imp77's '%alias' mechanism, giving the C-compatible name in the alias, in lower-case.
For example, "%externalroutinespec sleep %alias "sleep" (%integer seconds)" would be necessary
to access the linux system call by that name, whereas "%externalroutine sleep (%integer seconds)"
would define a routine written in Imp77 which a C program (which is case-sensitive) would have to
specify as "extern void SLEEP(int seconds);"

* Be aware that C code evaluates parameters to procedures from right to left; Imp77 is defined to
evaluate parameters from left to right.  This does not affect the way that external procedures are
specified in Imp77, it is just a difference in the way the languages work which C programmers writing
in Imp77 should be aware of.  External procedures are specified in the same left to right order in
both languages, for example, the C call:

   extern int rename(const char *oldpath, const char *newpath);

might be declared in an Imp77 header file as:

   %external %integer %fn %spec rename %alias "rename" (%byte %array %name oldpath, %byte %array %name newpath)"

Note that Linux library procedures do not know about Imp77's string format, so any linux library calls
where strings are passed will usually need an interface procedure (sometimes called a 'shim') to convert
any strings from Imp77 style to C style.


* To make interfacing to operating systems calls easier to write, they may be specified using "%system"
rather than "%external".  This will handle the case issues for you and avoid the need to use "%alias",
for example, the above becomes:

   %system %integer %fn %spec rename(%byte %array %name oldpath, %byte %array %name newpath)

Here's an example of a complete program which calls a linux routine:

%system %integer %fn %spec  getloadavg (%long %real %array %name   loadavg, %integer   nelem)
%begin
  %integer rc
  %long %real %array LA(0:2)
  rc = getloadavg (LA, 3)
  print string("Load average:  1 min: ")
  print(LA(0), 4, 4)
  print string("  5 mins: ")
  print(LA(1), 4, 4)
  print string("  15 mins: ")
  print(LA(2), 4, 4)
  newline
%endofprogram


* This implementation does not pass dope-vectors for own/const/external arrays (which are always 1-D)
- the address of the notional element(0) is passed as the apparent base of the array, even if declared
as an array with a non-zero lower bound.  This means that within the called procedure, indexing works
correctly as expected (eg if %integerarray letter('A':'Z') is passed to "DO SOMETHING(letter)", then
within the code of "DO SOMETHING", letter('A') will access the correct element as expected.  However
what will *not* be available is run-time checking that an array index is within the declared range.
This is a shortcoming that should be corrected in a later release.

* It's possible that arrays of higher dimensions (eg 2D arrays) may have some issues, but for now
they seem to be working albeit with similar restrictions to the 1D case above (although the mechanism
used is somewhat different in the C translation).  This too is a shortcoming that should be corrected
in a later release.  Unlike ERCC versions of Imp which required a runtime dope vector (and which supported
the ARRAY() map), Imp77 was designed, if I remember rightly, to be implementable without runtime dope-vectors.
Unfortunately this meant that ARRAY() maps were not possible in Imp77, which restricted some of the
things that could be expressed in Imp77, although I believe that like other Imp features from earlier
implementations which were dropped from the language, they were considered not worth preserving because
they were seldom if ever used. (dynamic array bounds within records being another such feature, for example).

* routines and functions passed as parameters to other routines or functions are not yet implemented.
They will fail at compile time.  This code is currently being worked on.

* variant records (record formats using "%or" to overlay fields in store) are not fully supported.
For now, the overlaid fields are accepted, but are treated as separate variables and not overlaid.
If these were used in an Imp77 program simply to save space at runtime, the program should behave
correctly; if they were used to deliberately pun on the storage (for example by writing to a %real
field and reading the internal representation of that %real variable back as an integer by reading
from an overlaid %integer field) then the code will not behave as intended.  This is not just from
the feature not having been implemented yet; it is a subtle issue with the modern definition of the
C language that the code is translated into.  This will eventually be implemented.

* a %label declaration within a routine or function will cause a compile-time error in the generated
C code.  This unfortunately requires a disproportionate amount of coding to fix.  (It has to do with
the order of code vs declarations in C, and the fix requires a major restructuring of the C code
generated by Imp77 - which currently is handled in a single phase or pass, but which requires code
shuffling to correct, i.e. two-passes or a second phase with the entire program being held in memory).
For now this is being worked around by a *very temporary* hack to the generated C that is implemented
using the "ecce" text editor.  (It could also be worked around by using the "--noline" option to the
deprecated "i2c" commandline, but that route is very much not recommended)

* The translation of Imp variables to C in the current implementation was not originally centralised
in a single place as it should have been, so several edge cases which worked in the general case of
expressions failed in some specific circumstances.  The cause of these failures is well understood,
and the necessary restructuring of code is already underway, but that rewrite is a big task and may
take some weeks at the time of writing to complete.  Fortunately the few times these errors occur were
all in places where the generated C code caused compile-time errors, so they are usually easily found.
These errors are mostly C translation problems where indirection is involved, either "*" or "&",
in addition to a few cases where data is cast inappropriately.

* %string variables currently always have capacity for 255 characters.  Proper string length enforcement
is pending.  Most string handling works as expected at the cost of using excessive space, which is
not really an issue in a modern environment given the relatively small size of all remaining historical
Imp77 programs.

* As mentioned above, be aware that Imp77 does not add a terminating '\0' to strings in the style of C,
so interfacing with external C code will often require a 'shim' procedure to be written to append a '\0'
to a string, as well as having to adjust the address of a string (to skip the preceding length byte) when
passing strings to C procedures,

* Imp I/O tries to follow the Linux model while retaining compatibility with original era behaviour.
stream 0 is predefined to be the console and stream 1 is standard input and standard output.

* The common Imp "CLI PARAM" is supported but linux-style 'ARGC' and 'ARGV(n)' are also available as
built-in functions.  Users should explicitly SELECT INPUT 0 or 1 at the start of a program, and of
course explicit opening and selection of other streams with "OPEN INPUT" etc should work too.
SELECT INPUT(1) and SELECT OUTPUT(1) will access stdin and stdout for programs invoked from the
command line, e.g: ./myprog < infile > outfile

* The perms supplied are a bit of a mish-mash of various Imp systems (not just Imp77) to improve
the ability to compile old code.  Let me know if there is something missing which you think should
be present.  Perms being present which are not wanted should not be a problem because they are
declared as "%perm" which is a level of scope above even that of "%external", so any external
procedures you may write which happen to have the same name as a perm routine should not cause a
linker clash.  (Although until I finish writing the final perms code, you may find a few C library
tokens that are uppercase in C - such as FILE * - causing clashes with the translated Imp77 code.)

* As with historical Imp implementations, perm routines (which can potentially be implemented by the
compiler using open inline code) do not necessarily have an actual callable procedure body, and
therefore cannot be passed as parameters to other routines (when that facility is implemented!);
however the standard Imp solution of writing a shim procedure to call a perm routine does work,
and adds no runtime overhead.  For example:

   %routine my write(%integer n, p)
     write(n, p)
   %end

* An approximation to Imp77's "%on %event" mechanism is supplied.  It is not as extensive as
traditional implementations, and although it will mostly work, there are some corner cases where
we know it will not work (such as resignalling a caught exception and working back up a chain
of nested procedure calls.)  It almost certainly does not currently undo event blocks correctly
when the enclosing procedure returns.  That code has not yet been written.

* Imp77's "%alias" mechanism is in the process of being implemented, however aliased entrypoint
names will be restricted to a limited set of characters which should include '$' but will not
include '#'.  So "IMP$RTL" will be possible but not "S#GO"...

* On 32 bit ARM you'll see a warning about:  void *caller_addr = __builtin_return_address(1);
- ignore it.  I know what it is.



CALLING I77
===========

The user interface to the i77 command is under development - the initial command that was supplied
is "i2c" but the command-line options of that command were somwhat limited.  A replacement driver
for the compiler, which behaves much more like the C compiler's "cc" command, is now preferred.
That newer version is currently called "i77" and is included in the distribution, and it calls
"i2c" to do the translation from Imp to C before invoking gcc to compile to an object or an
executable file.

Users should understand that i77 is an Imp77 compiler which uses C as an intermediate step in
producing a binary - it is *not* a source-to-source translator whose output is meant to be
user-maintainable C; indeed the C code will normally be deleted as soon as the Imp77 program
is compiled (though during development that is not being done).  If you need a high-level,
maintainable translation of old Imp code to C, there is a different project, "imptoc", which
is not complete and which is not being worked on until i77 is fully released.

The i77 environment is expected to be Linux, using gcc.  The suite may work on other systems
(such as Windows) as long as they support gcc, but no support for those environments is envisaged
at this point.  Note that true gcc is a hard requirement - using clang in gcc-compatible mode (as
done on BSD unix or Mac OS) is not an option, because clang does not support gcc's style of nested
procedure definitions, and never will.

i77 programs currently require perms.c to be compiled and linked with them, and the Imp77
%include file "perms.inc" must be installed, which is now done by the INSTALL.sh script supplied
with the download.


PORTABILITY
===========

Imp77 code compiles on both 32 bit and 64 bit Intel, and 32 bit and 64 bit ARM, however the
fact that Imp77 stores addresses in standard integer variables means that Imp77 with its
current definition cannot work in 64 bit mode for all language constructs.  Some programs will
work but many may not.  However both the Intel and ARM platforms allow for compilations to be
restricted to using 32 bit code, and you can force a 64-bit platform to generate 32 bit code
by using the "-m32" option to the i77 command.

(Just FYI: on Intel this is done by passing the -m32 flag to gcc, on ARM it is implemented
by installing a second gcc cross-compiler)

i77 will remind you of the information below if you try to compile in 32 bit mode on a 64 bit
ARM system that doesn't have the correct gcc support already installed:

the following commands will install 32-bit compatible binaries and runtime on a Pi 4 or Pi 5...

sudo apt install gcc-arm-linux-gnueabihf
sudo dpkg --add-architecture armhf
sudo apt-get update
sudo apt-get install libc6:armhf


At the moment I am not interested in Windows (other than WSL) or BSD Unix or Mac OS testing.
I already know they're going to be problematical.  This is a Linux suite for now.  I *am* interested
in any problems that show up specifically under 32 bit ARM but not 32 bit Intel.



DIAGNOSTICS
===========

i77 attempts to provide the same level of runtime checking that historical Imp compilers offered,
however those features have to be implemented somewhat differently to how a true compiler that
outputs binary files directly would handle diagnostics.  Some checks are handled by explicit
tests within the generated C code, others rely on linux mechanisms such as 'valgrind' and 'gdb'
and the various "sanitize" options to the gcc compiler which enable specific checks.

Sometimes the linux valgrind checks are over-enthusiastic, and definitely confusing; so the Imp77
programs sometimes have to be run without those checks enabled (using the --no-check option).  Valgrind
also adds a significant runtime overhead compared to the low overhead that traditional Imp compilers
typically imposed.  That said, all checks are on by default.  Tuning the checks for efficiency and
clarity are tasks scheduled for handling after the generated code itself is fully implemented and reliable.

The code to invoke valgrind and several of the gcc compile-time options which depend on the
version of gcc that you have available is somewhat hairy.  It works on my system, I can't
promise it will work elsewhere.  Unassigned variable testing in particular uses a mechanism
that only exists in V12 and later of gcc and may need serious tweaking to avoid false positives
with earlier versions of GCC.  I'm in the process of making those tests conditional on a
command-line option.

valgrind support was prototyped in the original i2c command and is not fully implemented under
the replacement i77 command so during these early days you may not even come across it

This alpha-tester release is via my personal web site.  It will not be moved to github until
it is significantly better developed and tested.  Please don't share outside our small group -
there's no fate worse for a compiler than getting a reputation for being buggy in its early
days of release (as we saw at Edinburgh a couple of times).  So I don't want potential users
other than the four of us to use i77 at this stage, as I know (and have documented above) that
it definitely has bugs.  This is alpha testing, not beta testing.


When you report problems, don't take offence if you get one of these canned replies :-)


* that's one of the known problems I mentioned, fixing it is scheduled, it's just a case of me getting around to writing that code.

* Thank you indeed for the code to fix that problem or implement that library routine, I'll put it in right away!

* Oh shit. I hadn't spotted that one, thanks!

* That is going to be a bastard to fix, but I'll put it in the pending queue for once everything else is handled.  (My job queue is ordered by how likely it is that a problem will come up in a real program.  Currently procedure parameters are the first priority and refactoring to fix some translation issues is a close second.)

* Yeah, never going to change that, I'll just document it.

* That's actually correct behaviour.


(OK, I probably will respond with more that just those, but I do expect a significant number of things that you'll bump up against to be problems I'm already aware of.)



Contact me by email with comments, fixes, and advice: gtoal@gtoal.com


Graham
