hissp.reader module

The Lissp language reader and associated helper functions.

The reader is organized as a lexer and parser. The parser is extensible with Lissp tags. The lexer is not extensible, and doesn’t do much more than pull tokens from a relatively simple regex and track its position for error messages.

hissp.reader.GENSYM_BYTES = 5

The number of bytes gensym hashes have.

The default 5 bytes (40 bits) should be more than sufficient space to eliminate collisions with typical usage: dozens of gensyms in the same scope would have a less than a one-in-a-billion chance of collision, even assuming they all have the same suffix. (Even 3 bytes gets that number down to around one in ten thousand.)

For unusual applications (if more than dozens of gensyms are expected in a shared scope, or one in a billion is still too high), hash length can be increased, up to a maximum of 32 bytes.

Even 8 bytes is enough space for a hundred thousand gensyms in the same scope with similar collision probability, or dozens with a one-in-quadrillion chance, which is probably lower than the risk of a hardware failure. It’s unlikely you’ll ever need more than 16 bytes, which has more space than uuid.uuid4.

Each hash character encodes 5 bits (Base32), so a multiple of 5 is recommended, although 3, 8, or 13 bytes are also fairly efficient for their size:

bytes	bits	chars	example
3	24	5	Qzthink__G
5	40	8	Qzthinking__G
8	64	13	Qzinvestigation__G
10	80	16	Qzincomprehensible__G
13	104	21	Qzelectroencephalograph__G
15	120	24	Qzmagneticresonanceimaging__G
16	128	26	Qzpositronemissiontomography__G

exception hissp.reader.SoftSyntaxError

Bases: SyntaxError

A syntax error that could be corrected with more lines of input.

When the REPL encounters this when attempting to evaluate a form, it will ask for more lines, rather than aborting with an error.

class hissp.reader.Lexer(code: str, file: str = '<?>')

Bases: Iterator

The tokenizer for the Lissp language.

Most of the actual tokenizing is done by the regex. The Lexer adds some position tracking to that to help with error messages.

position(pos: int) → tuple[str, int, int, str]

Compute the filename, lineno, offset and text for a SyntaxError, using the current character offset in code.

class hissp.reader.Comment(token: str)

Bases: object

Parsed object class for a comment token (line comment block).

The reader normally discards these, but they can be tag arguments.

contents() → str

Gets the comment text inside the comment token.

Strips any leading indent, the ; character(s), and up to one following space for each line in the comment block.

class hissp.reader.Kwarg(k: str, v: Any)

Bases: NamedTuple

Contains a read-time keyword argument for a tag.

Normally made with a kwarg token, but can be constructed directly.

k: str

Alias for field number 0

v: Any

Alias for field number 1

class hissp.reader.Lissp(qualname: str = '__main__', env: dict[str, Any] | None = None, evaluate: bool = False, filename: str = '<?>')

Bases: object

The Lissp Reader

Wraps around a Hissp compiler instance and creates a Lissp parser.

template_count()

property env: dict[str, Any]

The wrapped Compiler’s env.

compile(code: str) → str

Read Lissp code and pass it on to the Hissp compiler.

reads(code: str) → Parser

Read Hissp forms from code string.

parse(tokens: Lexer) → Parser

Read Hissp forms from a Lexer instance.

class hissp.reader.Parser(lissp: Lissp, tokens: Lexer)

Bases: Iterator

The parser for the Lissp language.

Parses Lissp tokens into Hissp syntax trees.

The special tags are handled here. They are

'	quote
` (backtick)	template quote (starts a template)
_#	discard tag
.#	inject tag (evaluate at read time)

Plus the three built-in template helpers, which are only valid inside a template.

,	unquote
,@	splice
$#	gensym tag

And finally, the stararg token special tags *= and **=.

Special tags are reserved by the reader and cannot be reassigned.

position(index: int | None = None) → tuple[str, int, int, str]

Get the filename, lineno, offset and text for a SyntaxError, from the Lexer given to parse.

gensym_context()

Start a new gensym context for the current template.

unquote_context()

Start a new unquote context for the current template.

qualify(symbol: str, invocation=False) → str

Qualify symbol based on current context.

static escape(atom: str) → str

Process the backslashes in a token.

static bare(v: str)

Preprocesses a bare token. Handles escapes and munging.

hissp.reader.is_hissp_string(form: object) → TypeGuard[str | tuple[Literal['quote'], str]]

Determines if form would directly represent a string in Hissp. (A Hissp string.)

Allows readerless mode-style strings: ('quote','foo',) and any string literal fragment: '"foo"' (including the "('foo')" form produced by the Lissp reader).

Macros often produce strings in one of these forms, via quote or repr on a string object.

hissp.reader.is_lissp_unicode(form: object) → TypeGuard[str]

Determines if form could have been read from a Lissp Unicode token.

It’s not enough to check if the form has a string type. Several token types such as a control token, symbol token, or fragment token, read in as a str atom. Macros may need to distinguish these cases.

hissp.reader.is_string_literal(form: object) → TypeGuard[str]

Determines if ast.literal_eval on form produces a string. (A string literal fragment.)

hissp.reader.is_qualifiable(symbol: str) → bool

Determines if symbol can be qualified with a module.

Can’t be quote, __import__, any Python reserved word, a prefix auto-gensym, fully qualified, method syntax, or a module handle; and must be a valid identifier or attribute identifier.

hissp.reader.transpile(package: str | None, *modules: str) → None

Transpiles the named Python modules from Lissp.

A .lissp file of the same name must be present in the module’s location. The Python modules are overwritten. Missing modules are created. If the package is “” or None, transpile writes non- packaged modules to the current working directory instead.

hissp.reader.transpile_packaged(resource: str, package: str) → None

Locates & transpiles a packaged .lissp resource file to .py.

hissp.reader.transpile_file(path: Path | str, package: str | None = None) → None

Transpiles a single .lissp file to .py in the same location.

Code in .lissp files is executed upon compilation. This is necessary because macro definitions can alter the compilation of subsequent top-level forms. A packaged Lissp file must know its package at compile time to handle templates and macros correctly.

After the .py file is written, __file__ will be set to it, if it doesn’t exist already.