Welcome to Allegro CL v11.0

 New Features and Fixes

Allegro CL v11.0 Documentation

3.2.1 New features

1. Support for Large Language Models: see llm-api.html.

2. New, native port to Apple Silicon. The table above, of supported operating systems, includes a new platform: Apple Silicon. At the time of the 11.0 release, there are variants of this platform called "M1" and "M2", but we expect more in the coming years. We believe all these variants will be compatible with the 11.0 release.

3. We have a second 64-bit ARM port that is targeted at Amazon Linux and a new glibc version. Amazon Linux runs mainly in the cloud on AWS, but you can download and run it yourself, with appropriate hardware. The version of the ARM hardware that Amazon supports is newer than our previous ARM target. This new target is ARMv8.1 (and later).

4. Apple software releases are notarized. You can read about notarization, but the basic idea is an assurance that our software has been checked for malware and it is now much easier to run after installation. Both the AMD/x86-64/Intel and Apple Silicon versions are notarized.

5. Allegro CL Express on macOS now uses the CG/JS version of Common Graphics and the IDE. Express users thus need not worry about XQuartz or GTK. Installation of the Express version onto MacOS is described in the excl-vars.html file. [Available in 10.1 with a patch]

6. New extensible lock-free hash-table implementation. Hash-tables have been reimplemented to be lock-free (even in SMP lisps), user-extensible, and more efficient. See the new :implementation option to make-hash-table and def-hash-table-implementation.

7. Prebuilt fast-but-unsafe clrhash implementation. Along with the reimplementation of hash-tables described above as being extensible, a specific prebuilt implementation called :unsafe-clrhash is provided, for use when a hash-table will never be used by more than one thread and a fast clrhash is required.

8. Significant changes to the runtime analyzer. There are significant changes to the runtime analyzer. See Changes to the runtime analyzer for more information. [Available in 10.1 with a patch]

9. New protection against trashing system constants. Storing a non nil value into nil's car or cdr or its symbol-value, or storing a non t value into t's symbol-value are bad for a running Lisp and can cause the Lisp to die or spin. The interpreter always performs this protection. However, protecting the Lisp against such bad stores in code compiled with high speed and low safety is very hard, without slowing down all stores. Starting in 11.0 the Lisp protects itself after the fact by periodically checking the values in nil and t and fixing these up if they are wrong. Also, error is called when the value has been trashed, allowing the user to isolate the approximate time of the bad store. See check-system-constants for more information.

10. New situations for atomic update operations on slot-value calls. Prior to 11.0, Atomic updates were allowed on slot-value, but not if there are any specializations which use slot-value-using-class methods. Starting with the 11.0 release, specializations can now be made on the new slot-atomic-update-using-class generic function.

11. New :libname keyword argument to ff:def-foreign-call. This argument to ff:def-foreign-call can specify the library where a foreign routine is defined, thus avoiding confusion when another library with the same or similar entry point is loaded.

12. MariaDB databases work with Allegro CL MySQL functionality. See mysql.html. Basically, a MariaDB database can be used with any function that accepts a MySQL database as an argument. [Available in 10.1 with a patch]

13. Support of package-local nicknames. A patch released in August, 2019, adds support for package-local nicknames. These are nicknames for packages defined as part of another package definition. These names may shadow other package names or nicknames. Package-local nicknames are described in Package-local Nicknames in packages.html. This change is for version 10.1 and later only. [Available in 10.1 with a patch]

14. The buffer argument to make-buffer-input-stream and with-input-from-buffer can be an aligned pointer. An aligned pointer is a fixnum which is interpreted as a machine integer, as described in Aligned Pointers and the :aligned type in ftype.html. The buffer argument to make-buffer-input-stream and with-input-from-buffer is usually a vector but can also be an aligned pointer so data can be read from C space. [Available in 10.1 with a patch]

15. New fixed index feature for class slots. A new feature added to Allegro CL 10.1 is the ability to specify the index which the value of a slot will have in an instance's slot values vector. When an instance of a class is created, it is provided a vector of slot values. Generally, which slot value is at which index in that vector cannot be easily determined and subsequent changes to classes may result in the index changing. A new slot option, excl:fixed-index (note: a symbol in the excl package, not in the keyword package) can be used to specify the index of a slot in the vector. That index is then fixed regardless of subsequect class changes. (Conflicting indices cause errors when an attempt is made to create an instance.) Knowing the index allows significant optimization of slot accesses. See Optimizing slot-value calls with fixed indices in implementation.html for more information. There are embellisher metaclasses (see next release note) which assist in optimizing slot accesses described in the referenced section and the embellisher metaclass section Metaclasses for embellishing class definitions in implementation.html. [Available in 10.1 with a patch]

16. New embellisher metaclass allows adding code to the class definition. Because defclass forms are macroexpanded when seen but macroexpanded code is usually executed later when the class is actually defined, it is difficult to add code to execute along with the macroexpanded defclass code. A new feature in Allegro CL defines the defclass-embellisher-class defclass macroexpansion. Two such metaclasses are defined (fixed-index-class and fixed-index-filling-class) and another is shown in an example. Users can write their own as well. See also Metaclasses for embellishing class definitions in implementation.html. [Available in 10.1 with a patch]

17. New pgid keyword argument to run-shell-command (UNIX only, 10.1 only). run-shell-command has a new keyword argument pgid. It allows control of the process group of the process spawned by the run-shell-command call, which allows better control over signal distribution. A Control-C typed to interrupt the Lisp will send a SIGINT to every process in that Lisp's process group. Using the pgid argument to specify that the spawned process will be in a separate process id group, Lisp can be interrupted while the spawned process continues to run. See the run-shell-command documentation for more information. [Available in 10.1 with a patch]

18. New function wait-for-io-available. wait-for-io-available is similar to wait-for-input-available but allows an application to wait for output possible on one set of streams as well as for input availble on another set. [Available in 10.1 with a patch]

19. New quiet keyword argument to build-lisp-image. build-lisp-image (and generate-executable and generate-application, which call build-lisp-image) can generate a great deal of output which is only of interest if there is a problem creating the desired image. The new quiet keyword argument, accepted by all three functions, suppresses that output unless an error is signaled. See building-images.html for details of the quiet argument. [Available in 10.1 with a patch]

20. AllegroServe documentation moved to github. The AllegroServe documentation is now in github rather than being part of the regular Allegro CL documentation set. [Available in 10.1 with a patch]

21. New macro with-at-most-one-form. The macro with-at-most-one-form will signal an error if its body contains more than one form and optionally, depending on the value of the if-null-body keyword argument, if the body contains no forms. The purpose is to ensure conditionalized code where only one form is supposed to be executed, will error if two or more forms or, optionally, no forms would execute. [Available in 10.1 with a patch]

22. New process pool facility allows reusing processes multiple times. The new process pool facility, described in the Process pools section in the multiprocessing.html document, allows setting up a pool of processes, each of which runs a task and then remains available for a further task. Using a pool saves the overhead of creating and destroying process objects. This is useful when you expect to have many short to medium tasks to be run in their own processes. [Available in 10.1 with a patch]

23. New :utf-8s strict external format. When this external format is in effect, non-UTF8 characters will be filtered out, replaced by a designated error character. Optionally, a warning or error may be signaled when an improper character or character sequence or associated improper octets are encountered. See Strict external formats which do not allow improper characters in iacl.html for more information. New functionality associated with the :utf-8s external format is described there. [Available in 10.1 with a patch]

3.2.2 Performance enhancements

1. sort reimplemented to perform better on specialized arrays. sort has long used the very fast merge sort algorithm when sorting simple vectors but used the quick sort algorithm on specialized vectors (like those with element type fixnum or single-float). While quick sort is generally fast, certain value arrangements (such as already sorted but in reverse order or all values the same) could be very slow. sort has now been modified to use merge sort when sorting a vector in all cases. A merge sort requires a temporary scratch vector the same size as the vector being sorted. A new keyword argument to sort, strategy, allows specifying whether this scratch vector will be allocated on the stack, in the Lisp heap, or will be a vector supplied by the caller to sort. A new variable, *simple-vector-sort-strategy*, provides the default value for this argument. See cl:sort implementation in implementation.html for more information. [Available in 10.1 with a patch]

2. The resource mechanism is more efficient on SMP lisps. Note that excl::defresource is not exported or documented, but is used internally by quite a few mechanisms. Allocation of a resource object was failing when there was a collision between two threads trying to allocate the same resource. It was acting as if the resource pool was exhausted, even if there were enough objects in the pool to satisfy both requests. When this happened there would be unnecessary allocation, and in some extreme cases, a noticeable increase in scavenging overhead. This bug has been fixed.

3. Performance improvement for many boolean operations. They now use conditional moves instead of branching on many architectures.

3.2.3 Compiler enhancements

1. New compile-file strategy A new compile-file strategy is implemented by the 11.0 release where fasl files are written to indirectly and by section, to ensure that the old fasl file is not disturbed until the last possible moment, and also to reduce the memory pressure from large heap loads during the compilation.

2. Position information enhancements. Enhancements have been made to position information, both during compiling and during execution of code. See position information. Also, operators are available for enhancing the likelihood of correct position information; see with-saved-file-positions and restore-file-positions.

3. Source context enhancements. Version 10.1 had a concept of a source context, which provided various information about the location and circumstance of a condition when it was created (especially during compilation) but with the 11.0 release a source context can stand on its own and can be invoked within any macro using source-context, which allows filename, position information, the form being processed, and compiler-backtrace if applicable to be accessed programmatically.

4. compiler backtraces are more available and readable. Compiler backtraces are now part of the source-context when any conditions are created during compilation, and they now can carry position information when optimization levels include high debug.

5. New macro with-current-source-form. A new macro with-current-source-form allows position information to be injected into source by a macro which would otherwise be opaque to the compiler. See also the example.

6. New compiler switch: comp:compile-macroexpansions-for-safety-switch. comp:compile-macroexpansions-for-safety-switch is examined at macroexpand time when the compiler is in the image, and working on compiler macros as well as regular macros, this switch affects how defstruct accessors are expanded. If the switch returns true, an accessor that checks for the correct struct type is generated.

7. New compiler switch: comp:reorder-folded-constants-switch. comp:reorder-folded-constants-switch if true allows agressive reordering of expressions so that constants can be merged at various levels of computation. This may result in interpreted code and compiled code getting different numerical results (see the examples on the description page). Results from version 10.1 (where this switch did not exist) may also differ from current results when this switch is true, which it is when speed is 3 and safety is 0 or 1

8. add-typep-transformer has a new re-expand keyword argument. add-typep-transformer allows the compiler to transform the form (typep x type) into (funcall predicate x). The new argument, when it is true and when the predicate is a symbol naming a function, causes the transformed form to be (predicate x), potentially allowing compiler macros for predicate (if such exist) to further inline code. This feature is for 10.1 and later only. [Available in 10.1 with a patch]

3.2.4 Debugging enhancements

1. Fwrapping and tracing now work on foreign-callable functions. See defun-foreign-callable, fwrappers-and-advice.html and The tracer.

2. New ability to conditionalize assert forms. With the new assert enable facility, described in Conditionalizing assert calls in miscellaneous.html, you can now enable or disable calls to assert at runtime. Thus code does not have to be recompiled to disable calls to assert and assert forms can be enabled in applications if a problem is detected. The :generic-checks assert-enable is enabled by default and checks calls to defstruct accessors when compiled.

3. New printer control variables control printing of long strings. Printing of very long strings (containing hundreds or thousands of characters) when it is not the actual contents that are of interest can make it difficult to see the desired content and can also cause problems if buffers or screens overflow. Several new printer control variables have been added to deal with this issue. *print-long-string-length* is analogous to *print-length*. The additional variables tpl:*print-long-string-length*, tpl:*zoom-print-long-string-length* *step-print-long-string-length*, and *trace-print-long-string-length*, provide control in other contexts. See Printing long strings top-level.html and also Allegro CL print variables can follow the CL print variable value in implementation.html for more information.

4. New :intermdiate :zoom backtrace mode. In addition to :brief, :moderate, and :verbose modes, backtraces produced by :zoom can be displayed in :intermediate mode, which is similar to :moderate with the addition of pc offset information (which is also displayed by :verbose mode). See :brief, :moderate, :intermediate, and :verbose modes of :zoom in debugging.html.

5. New debug-control field for processes. Process now have a debug-control field. It is displayed as the second column in the information displayed by the :processes top-level command. When true it causes dead locals to be preserved (rather than garbage collected) on the stack so that debugging is made easier. See the :debug-control top-level command for more information.

6. New functions for finding gc paths and new objects. A gc path is the path from the Allegro CL root object to a specific live object. New operators allow finding gc paths and tracking newly-created objects. See the section Functions for finding gc paths and new objects in gc.html for more information. [Available in 10.1 with a patch]

7. memtrace facility modified, now works identically in a file and at the top-level. A new macro memtrace-do has also been added. See Memlog: A Facility for Minimally Intrusive Monitoring of Complex Application Behavior in smp.html for more information. [Available in 10.1 with a patch]