(This is a draft)

Functional lazy evaluated python

_{Tags:
python
funny}

One day while trying to sleep after a day of writing lisp I got out of bed and on my phone I programmed something looking roughly like this:

Σ = lambda f,*a: f(*a)

define = lambda f: globals().__setitem__(f.__code__.co_varnames[0],_k:=lambda *n:f(_k, *n)) or True

Pair = type("Pair",(),{"__init__":lambda s,a,b:setattr(s,"t",(a,b)),"__getitem__":lambda s,i:s.t[i],"__repr__":lambda s:f"({s[0]} {s[1]})".replace(" ("," ").replace("))",")")})

That is just boilerplate + sugar, I know it looks ugly, but imperative languages usually do.

What's interesting about that however is that the first line is all you need to get something similar to lisp Sexpr-likes; Σ(foo, 1, 2, 3) instead of the regular foo(1,2,3).

The third line is of no interest, just a class that pretty prints lisp lists, it's completely useless but we want lisp pairs as separate classes for code hygiene, as things get messy easily using bare tuples.

The second one is more interesting, it takes a function, and stores in globals() a lambda that calls the function, with itself as first argument + any other argument passed to it, named with the name of the first argument of the function. In other words a define function, and that is really all we need to start with the most basic iteration of a lisp, look at this;

Σ(define, lambda eq, a,b : a==b)
Σ(define, lambda cons, a,b : Pair(a,b))
Σ(define, lambda car, p : p[0])
Σ(define, lambda cdr, p : p[1])

Σ(define, lambda add, *num : Σ(sum, num) )

Cool isn't it?

Now we can call Σ(eq, 10, 20) and it'll evaluate to False, it supports variadic args and anything a regular python lambda supports.

For the next bit we need to use a little hack, python sadly isn't lazy evaluated (well, it kinda is), so we need to create an if that takes both branches as functions that take no arguments, aka thunks.

Σ(define, lambda IF, cond, true, false :
	Σ([false,true][bool(cond)]))

This does make writing ifs a little more cumbersome but it's worth it so far, we can see it in action by defining list functions

Σ(define, lambda list, *ar :
	Σ(cons, Σ(car, ar),
		Σ(IF, Σ(eq, 1, Σ(len, ar)),
			lambda: (),
			lambda: Σ(list, *ar[1:]))))

Σ(define, lambda map, fn, ls :
	Σ(IF, Σ(eq, ls, ()),
		lambda: (),
		lambda: Σ(cons,
			Σ(fn,Σ(car,ls)),
			Σ(map,fn,Σ(cdr,ls)))))

You can keep iterating over that and build arbitrary python programs in lisp style with that, but not arbitrary lisp programs sadly. That is because, as you saw with the if, python is eager to evaluate the values of everything it sees, and while you can save stuff in thunks, that just means writing more and more boilerplate LoC for no reason, that also makes writing macros very hard because you need to mess with the internals of the thunk (a function object) instead of just handling the code as data.

What we need is to get rid of python's early evaluation entirely, we want to define our own evaluator and call it only when we want, but we can't really do much terms of new data types in our current system because that would make this implementing lisp in python rather than python being already a lisp, to do what we want we need a special type of data/code definition scheme, and it has to be builtin.

Lazy Evaluation

Earlier I said python isn't lazy evaluated and you need to manually put lambdas if you want to defer evaluation, that isn't quite true, there are many ways of doing lazy evaluation in python, the official way of doing it is with generators but that is boring, a little known way I like quite a bit is using the type keyword;

type a = print("hello")
type b = print("goodbye")

[a,b][1].evaluate_value()

That will only print "goodbye", because a maps to print("hello") not to the result given by print("hello"), which only after evaluating the function python knows it is None

why do I like this method you ask? Just look

type car[_] = ...
type cdr[_] = peanut
type list[_] = quit()

print(car[cdr[cdr[list[0,1,2]]]])

That prints car[cdr[cdr[list[0, 1, 2]]]], had I not used types this would've stopped long before reaching the print, but typing in python is lazy, it's not evaluating anything, this allows us to compose and modify code as data just like we wanted, because type objects are just a collection of fancy tuples with more types inside.

And besides defining the types themselves there is no need for new syntax clutter, only repurposing foo[] to work similar to foo().

Now we want to actually evaluate what they have inside, and not only evaluate but pass arguments to it, you might be tempted to do something like this:

type A[foo,bar] = print(foo, bar)

A[10,20].__value__ #__value__ is the same as .evaluate_value()

And this will print, as expected: foo bar... Not quite what we want, as a matter of fact, the "arguments" you define inside the type are not actually mapped to the ones you passed to A, because they're not arguments to begin with, they're just a template.

Exercise for the reader...

type A[B=int] = list[B]
for 1..4 do; echo `hello!` end

Where is the syntax error?
if you said in line 2, then python3.14 (latest release as I'm writing this) is going to disagree with you.
Python says that the = in type A[B=int] is invalid syntax despite being valid and the canonical way of passing default type vars, furthermore it will then suggest you use == or :=, both make 0 sense in this scenario and they are in fact invalid syntax.

What we want is a custom evaluator, one that is going to take the arguments inside the square brackets and apply any arbitrary transformation on them.

β = lambda T:T.__value__(*T.__args__)if(hasattr(T,"__value__")and hasattr(T,"__args__"))else T

That is going to take the value assigned to the type, which we're going to define as a function, and it will give it the arguments inside the square brackets, saved in .__args__, this is kinda similar to the evaluator Σ we defined earlier for Sexprs, except in this case the function only receives a single object; either a Type or a literal, the type already contains everything it needs inside, and the literals don't transformation, so β acts as the identity β(literal) = literal.

Originally I was using tuples to represent pairs here, but python has a normally nice but in this case annoying syntax of interpreting tuples inside of a type argument as the values inside the tuples expanded, so that list[(int,str)] is equivalent to list[int,str], that allows us to have things like this lambda *v: list[tuple(map(type, v))] to programatically create types for things whose actual type we don't yet know. What is that useful for I don't know, but no linter should EVER execute code stored in a type, I'll just blindly trust none does.

As such we need now a Pair class and an exhaust evaluator, since, as you can see, our β only advances a singular step each time, so evaluating nested expressions will just yield the second unevaluated expression, for the pair we could use lists but I want something immutable and tuples are not great to pass as type arguments, so a custom class it is.

We also want to update our β so that it steps each element of the Cons, otherwise lists which are just chains of pairs will get stuck forever at the fist step of evaluation.

Our program would look now like this:

class  Cons(metaclass=type("",(type,),{"__str__":lambda _:"NULL"})):
	__slots__ = __match_args__ = "a","b"
	def __init__(self, a, b): self.a,self.b = a,b
	def __eq__(self, o): return isinstance(o,Cons) and self.a==o.a and self.b==o.b
	def __hash__(self): return hash((self.a, self.b))
	def __repr__(self):#we'll use the class itself as NULL
		return f"<{self.a} {"/"if self.b is Cons else self.b}>"
	def __iter__(self):yield from(self.a, self.b)

β = lambda T: Cons(β(T.a),β(T.b))if isinstance(T,Cons) else\
	T.__value__(*T.__args__)if(hasattr(T,"__value__")and hasattr(T,"__args__"))else T
λ = lambda v: (lambda b:λ(b)if b!=v else v)(β(v))

That Cons looks comparatively big but most of it is just pretty printing it's content when you do print(Cons(foo, Cons(bar, Cons))), like I said before, you can even use normal lists if you want but they will look ugly and be unperformant, the __eq__ dunder however is required otherwise λ will recur forever.

With that done, we can just define a function as

type FOO[_] = lambda arg1, arg2, argn: Value

But having a define function instead of a statement is more convenient in the long run, since we won't have to worry about python statement rules, so we'll define a new define anyways, this time we will make a LAMBDA too, and a SET just for convenience while we're at it.

They will be almost the same as the previous define function in the Σ version except it will create types instead of python's lambdas.

def LAMBDA(lamb):
	type L[l] = lamb
	return L
def DEFINE(lamb):
	type F[l] = lambda*arg: lamb(F,*arg)
	globals()[lamb.__code__.co_varnames[0]]=F
def SET(lamb):
	globals()[lamb.__code__.co_varnames[0]]=lamb(lamb)

As you can see DEFINE is merely a lambda that has been given a name, and it receives itself as first argument so it can recur over it's own name, it is functionally identical to SET(lambda name: LAMBDA(lambda args: result)).

You don't actually need to call and edit globals() at all if you don't want to, if you pass the lambda to a Y-combinator you can recur over an anonymous function just like you can do over a globally defined one, but we have no real reason to concern ourselves with such level of functional purity considering we're using python and we only have a limited amount of recursion before the system shits the bed, you still can if you want tho.

And that is all we need, a program is defined as anything inside a λ( ), we'll just iterate over a list of expressions to get a complete "useful" program:

# Builtins, for now only CAR, CDR and PRINT
type CAR[_]= lambda cons:β(β(cons).a)
type CDR[_]= lambda cons:β(β(cons).b)
type PRINT[*_] = lambda *s:print(*map(λ,s))


for i in(
# Our program starts here
DEFINE(lambda IF, cond, true, false: true if λ(cond)else false),
DEFINE(lambda FROM, start, step=1: Cons(start, FROM[start+step, step])),
DEFINE(lambda TAKE, n,of:(lambda en:β(IF[en<=0,Cons,Cons(λ(CAR[of]),TAKE[en-1,CDR[of]])]))(λ(n))),

PRINT[TAKE[10, FROM[4]]]
):λ(i)

That will print <4 <5 <6 <7 <8 <9 <10 <11 <12 <13 />>>>>>>>>> exactly what we wanted

Let's go over the code step by step.

DEFINE(lambda IF, cond, true, false: true if λ(cond)else false),

A fully lazy IF, no need for any thunk boilerplate, it will receive a condition expression and 2 code expressionS, then it will exhaust the condition till it starts returning literals, and only then it will return the unevaluated expression that the condition dictates, this means I actually lied above, our type FOO[_] = lambda: BAR is not actually equivalent to a lisp function, but rather it is a lisp macro, a function as a mater of fact is just a macro where you evaluate the expression you handled, like the PRINT, CAR and CDR in the stdlib.

Most of the time you want to use the regular ternary inside rather than the IF macro, because it is too lazy for most usecases, but you might need it if you want to defer the evaluation of the condition till the very last moment.

DEFINE(lambda FROM, start, step=1: Cons(start, FROM[start+step, step])),

This is a macro too, it will give us an infinite stream of numbers starting from start, it does so by returning a pair with (Value NextStep) where Value is a literal and NextStep is an unevaluated expression that goes FROM one step further.

Since β() evaluating a Cons will evaluate both sides sequentially, something like FROM[1] would produce (1 _) on the first step, the second time it is evaluated it will produce (1 (2 _)), the third time (1 (2 (3 _))) and so on. Doing an exhaust evaluation λ(FROM[1]) will just recur infinitely, and thus PRINT[FROM[1]] would never finish since it internally exhausts it's arguments, despite that our program does not halt, because we can handle unevaluated pieces of code as regular data.

DEFINE(lambda TAKE, n,of:(lambda en:β(IF[en<=0,Cons,Cons(λ(CAR[of]),TAKE[en-1,CDR[of]])]))(λ(n))),

This one is far more interesting, it takes a an expression that generates a number and a chain, then it we use Currying to evaluate n as a number only once, if the number, now evaluated, is 0 or less we return an empty pair, otherwise we return the concatenation of the first value of our of chain, with the expression that TAKEs n-1 elements from the rest of the chain.

We are evaluating here because take doesn't necessarily needs to take a constant number every time, imagine if we DEFINE(lambda GET: int(input("Enter a number: "))) and instead of taking 10 we just call TAKE[GET[()], Cons(1,Cons(2,Cons(3,Cons)))], if we don't evaluate it we will soon reach a TypeError, but if we don't curry it and just evaluate twice on both places were we are using it the program would've asked twice.

We didn't evaluate FROM's arguments because those are part of what the caller requests, but here n is our own internal counter variable, in actuality we will evaluate it only once and all the other times we know it'll be the literal that results from subtracting 1 to it.

It is a function since we're β evaluating it, but the thing we're evaluating is an IF that returns an expression anyways, notice that β(IF[...]) is the same as doing what IF does internally; a regular python ternary.

And that is pretty much it, you can now write any program you wish, but I recommend doing something like sys.setrecursionlimit(6000000) before running any because python's default of 1000 runs out way too fast

Still, there is a lot of room for improvement, I'll define some standard functions with the type keyword from now on because they're easier to debug.

The first thing that catches my eye is that IF is going to exhaust the condition, which is not good because it wouldn't be out of the question passing a stream to it, λ should be using as sparingly as possible, with that in mind we will define a TRUTH function, rewrite the IF macro and make some derived combinators

def isPair(p): return p is Cons or isinstance(p,Cons)

type TRUTH[*_] = (lambda val:
	(lambda v:(v is not Cons)if isPair(v)else(bool(v)if v==val else β(TRUTH[v])))(β(val))
)

type IF[*_] = lambda cond,true,false:true if β(TRUTH[cond])else false
type COND[*_] = (lambda *pairs:
	(lambda cond,val:val if β(TRUTH[cond])else β(COND[pairs[1:]]))(*pairs[0])) if pairs else None
type NOT[*_] = lambda val:not β(TRUTH[val])

Wish is wasn't reserved so I could put that isPair inside Cons.is

Do keep in mind TRUTH evaluates once the code passed to it, so you have to be careful with what you pass to it.

Anyways, with that done we can now continue with the list functions: