hissp.compiler module#

The Hissp data-structure language compiler and associated helper functions.

hissp.compiler.ENV: ContextVar[dict[str, Any]] = <ContextVar name='ENV'>#

Expansion environment.

Sometimes a macro needs the current environment when expanding, instead of its defining environment. Rather than pass in an implicit argument to all macros, it’s available here. readerless and macroexpand use this automatically.

hissp.compiler.MAX_PROTOCOL = 5#

Compiler pickle protocol limit.

When there is no known literal syntax for an atom, the compiler emits a pickle.loads expression as a fallback. This is the highest pickle protocol it’s allowed to use. The compiler may use Protocol 0 instead when it has a shorter repr, due to the inefficient escapes required for non-printing bytes.

A lower number may be necessary if the compiled output is expected to run on an earlier Python version than the compiler.

hissp.compiler.macro_context(env: dict[str, Any])[source]#

Sets ENV during macroexpansions.

Does nothing if env is already the current context.

exception hissp.compiler.CompileError[source]#

Bases: SyntaxError

Catch-all exception for compilation failures.

exception hissp.compiler.PostCompileWarning[source]#

Bases: Warning

Form compiled to Python, but its execution failed.

Only possible when compiling in evaluate mode and not in __main__. Would be a “Hissp Abort!” instead when in __main__, but other modules can be allowed to continue compiling for debugging purposes.

Continuing execution after a failure can be dangerous if non-main modules have side effects besides definitions. Warnings can be upgraded to errors if this is a concern. See warnings for how.

class hissp.compiler.Compiler(qualname: str = '__main__', env: dict[str, Any] | None = None, evaluate: bool = True)[source]#

Bases: object

The Hissp recursive-descent compiler.

Translates the Hissp data-structure language into a functional subset of Python.

static new_env(name: str, doc: str | None = None) → dict[str, Any][source]#

“Imports” the named module, creating it if necessary.

Dynamically created modules have a None __spec__. After creating the types.ModuleType using name and doc, it initializes an empty __annotations__, a __package__ based on name (assumes module is not itself a package), and a __builtins__.

Returns the module’s __dict__.

compile(forms: Iterable) → str[source]#: Compile multiple forms, and execute them if evaluate mode enabled.

compile_form(form) → str[source]#: Compile Hissp form to the equivalent Python code in a string. tuple and str have special evaluation rules, otherwise it’s an atom that represents itself.

tuple_(form: tuple) → str[source]#: Compile call, macro, or special forms.

special(form: tuple) → str[source]#

Try to compile as a special form, else invocation().

The two special forms are quote and lambda.

A quote form evaluates to its argument, treated as literal data, not evaluated. Notice the difference in the readerless compiled output:

>>> print(readerless(('print',42,)))  # function call
print(
  (42))
>>> print(readerless(('quote',('print',42,),)))  # tuple
('print',
 (42),)

lambda_(form: tuple) → str[source]#

Compile the anonymous function special form.

(lambda (<parameters>): <body>)

The parameters tuple is divided into (<singles> : <pairs>)

Parameter types are the same as Python’s. For example,

>>> print(readerless(
... ('lambda', ('a',':/','b'
...            ,':', 'e',1, 'f',2
...            ,':*','args', 'h',4, 'i',':?', 'j',1
...            ,':**','kwargs',)
...  ,42,)
... ))
(
 lambda a,
        /,
        b,
        e=(1),
        f=(2),
        *args,
        h=(4),
        i,
        j=(1),
        **kwargs:
    (42))

The special control words :* and :** designate the remainder of the positional and keyword parameters, respectively.

Note this body evaluates expressions in sequence, for side effects:

>>> print(readerless(
... ('lambda', (':',':*','args',':**','kwargs',)
...  ,('print','args',)
...  ,('print','kwargs',),)
... ))
(lambda *args, **kwargs:
   (print(
      args),
    print(
      kwargs))  [-1]
)

You can omit the right of a pair with :? (except the final **kwargs). Also note that the body can be empty.

>>> print(readerless(
... ('lambda', (':','a',1, ':/',':?', ':*',':?', 'b',':?', 'c',2,),),
... ))
(
 lambda a=(1),
        /,
        *,
        b,
        c=(2):
    ())

The : may be omitted if there are no paired parameters.

>>> print(readerless(('lambda', ('a','b','c',':',),),))
(lambda a, b, c: ())
>>> print(readerless(('lambda', ('a','b','c',),),))
(lambda a, b, c: ())
>>> readerless(('lambda', (':',),),)
'(lambda : ())'
>>> readerless(('lambda', (),),)
'(lambda : ())'

The : is required if there are any pair parameters, even if there are no single parameters:

>>> readerless(('lambda', (':',':**','kwargs',),),)
'(lambda **kwargs: ())'

parameters(parameters: Iterable) → str[source]#: Process params to compile lambda_.

body(body: Iterable) → str[source]#: Compile body of lambda_.

invocation(form: tuple) → str[source]#: Try to compile as macro, else normal call.

expand_macro(form: tuple) → str | Sentinel[source]#: Macroexpand and start over with compile_form, if macro.

classmethod get_macro(symbol: object, env: dict[str, Any])[source]#

Returns the macro function for symbol given the env.

Returns None if symbol isn’t a macro identifier.

call(form: Iterable) → str[source]#

Compile call form.

Any tuple that is not quoted, (), or a special form or macro is a run-time call form. It has three parts:

(<callable> <singles> : <pairs>).

Each argument pairs with a keyword or control word target. The :? target passes positionally (implied for singles).

For example:

>>> print(readerless(
... ('print',1,2,3
...         ,':','sep',('quote',":",), 'end',('quote',"\n\n",),)
... ))
print(
  (1),
  (2),
  (3),
  sep=':',
  end='\n\n')

Either <singles> or <pairs> may be empty:

>>> readerless(('foo',':',),)
'foo()'
>>> print(readerless(('foo','bar',':',),))
foo(
  bar)
>>> print(readerless(('foo',':','bar','baz',),))
foo(
  bar=baz)

The : is optional if the <pairs> part is empty:

>>> readerless(('foo',),)
'foo()'
>>> print(readerless(('foo','bar',),),)
foo(
  bar)

Use the :* and :** targets for position and keyword unpacking, respectively:

>>> print(readerless(
... ('print',':',':*',[1,2], 'a',3, ':*',[4], ':**',{'sep':':','end':'\n\n'},),
... ))
print(
  *[1, 2],
  a=(3),
  *[4],
  **{'sep': ':', 'end': '\n\n'})

Method calls are similar to function calls:

(.<method name> <self> <args> : <kwargs>).

A method on the first (self) argument is assumed if the function name starts with a dot:

>>> readerless(('.conjugate', 1j,),)
'(1j).conjugate()'
>>> eval(_)
-1j
>>> readerless(('.decode', b'\xfffoo', ':', 'errors',('quote','ignore',),),)
"b'\\xfffoo'.decode(\n  errors='ignore')"
>>> eval(_)
'foo'

fragment(code: str) → str[source]#: Compile a fragment atom. This preprocessing step converts a fully-qualified identifier or module handle into an import. No further compilation is necessary. The contents are assumed to be Python code already.

static qualified_identifier(qualname: str, code: str) → str[source]#: Compile fully-qualified identifier into import and attribute.

static module_identifier(code: str) → str[source]#: Compile a module handle to an import.

atomic(form) → str[source]#

Compile forms that evaluate to themselves.

Returns a literal if possible, otherwise falls back to pickle:

>>> readerless(-4.2j)
'((-0-4.2j))'
>>> print(readerless(float('nan')))
# nan
__import__('pickle').loads(b'Fnan\n.')
>>> readerless([{'foo':2},(),1j,2.0,{3}])
"[{'foo': 2}, (), 1j, 2.0, {3}]"
>>> spam = []
>>> spam.append(spam)  # ref cycle can't be a literal
>>> print(readerless(spam))
# [[...]]
__import__('pickle').loads(b'(lp0\ng0\na.')
>>> spam = [[]] * 3  # duplicated refs
>>> print(readerless(spam))
# [[], [], []]
__import__('pickle').loads(b'(l(lp0\nag0\nag0\na.')

pickle(form: object) → str[source]#: Compile to pickle.loads. The final fallback for atomic.

static linenos(code: str) → str[source]#: Adds line numbers to code for error messages.

eval(code: str, form_number: int) → tuple[str] | tuple[str, str][source]#: Execute compiled code, but only if evaluate mode is enabled.

hissp.compiler.readerless(form: object, env: dict[str, Any] | None = None) → str[source]#

Compile a Hissp form to Python without evaluating it.

Returns the compiled Python in a string.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

hissp.compiler.evaluate(form: object, env: dict[str, Any] | None = None)[source]#

Convenience function to evaluate a Hissp form.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

>>> evaluate(('operator..mul',6,7))
42

hissp.compiler.execute(*forms: object, env: dict[str, Any] | None = None) → str[source]#

Convenience function to compile and execute Hissp forms.

Returns the compiled Python in a string.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

>>> print(execute(
...     ('hissp.._macro_.define','FACTOR',7,),
...     ('hissp.._macro_.define','result',('operator..mul','FACTOR',6,),),
... ))
# hissp.._macro_.define
__import__('builtins').globals().update(
  FACTOR=(7))

# hissp.._macro_.define
__import__('builtins').globals().update(
  result=__import__('operator').mul(
           FACTOR,
           (6)))
>>> result
42

hissp.compiler.is_str(form: object) → TypeGuard[str][source]#: Determines if form is a str atom. (Not a str subtype.)

hissp.compiler.is_node(form: object) → TypeGuard[tuple][source]#: Determines if form is a nonempty tuple (not an atom).

hissp.compiler.is_symbol(form: object) → TypeGuard[str][source]#: Determines if form is a symbol.

hissp.compiler.is_import(form: object) → TypeGuard[str][source]#: Determines if form is a module handle or has full qualification.

hissp.compiler.is_control(form: object) → TypeGuard[str][source]#: Determines if form is a control word.

hissp.compiler.macroexpand1(form, env: dict[str, Any] | None = None)[source]#

Macroexpand outermost form once.

If form is not a macro form, returns it unaltered.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

hissp.compiler.macroexpand(form, env: dict[str, typing.Any] | None = None, *, preprocess=<function <lambda>>)[source]#

Repeatedly macroexpand outermost form until not a macro form.

If form is not a macro form, returns it unaltered.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

preprocess (which defaults to identity function) is called on the form before each expansion step.

hissp.compiler.macroexpand_all(form, env: dict[str, typing.Any] | None = None, *, preprocess=<function <lambda>>, postprocess=<function <lambda>>)[source]#

Recursively macroexpand everything possible from the outside-in.

Pipes outer form through preprocess, macroexpand(), and postprocess, then recurs into subforms of the resulting expansion, if applicable.

Pre/postprocess are called with macro_context so, e.g., macroexpand1 may be called by preprocess to handle intermediate expansions.

If expansion is not a macro form, returns it. As in the compiler, lambda parameter names are not considered expandable subforms, but default expressions are.

Unless an alternative env is specified, uses the current ENV (available in a macro_context) when available, otherwise uses the calling frame’s globals.

hissp.compiler.parse_params(params) → tuple[tuple, dict[str, Any]][source]#: Parses a lambda form’s params into a tuple of singles and a dict of pairs.