From 5dd0e0052af74961c6e688122f6fb53273023010 Mon Sep 17 00:00:00 2001 From: Taylan Kammer Date: Sat, 3 Jan 2026 05:00:13 +0100 Subject: Add "full-stack" note. --- notes/260102-full-stack.md | 250 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 250 insertions(+) create mode 100644 notes/260102-full-stack.md (limited to 'notes/260102-full-stack.md') diff --git a/notes/260102-full-stack.md b/notes/260102-full-stack.md new file mode 100644 index 0000000..3870bf5 --- /dev/null +++ b/notes/260102-full-stack.md @@ -0,0 +1,250 @@ +# A full-stack programming language + +_2026 January_ + +As I become more and more ambitious with my dreams about what I want +Zisp to become, it becomes less and less likely that I'll actually +create and finish the language, ever. But I just can't help it. + +The notion of a "full-stack programming language" was first widely +introduced, from what I know, by the Red language project. To quote +their website: + +> Red’s ambitious goal is to build the world’s first full-stack +> language, a language you can use from system programming tasks, up +> to high-level scripting through DSL. You've probably heard of the +> term "Full-Stack Developer". But what is a full-stack Language, +> exactly? +> +> Other languages talk about having "one tool to rule them all". Red +> has that mindset too, pushed to the limit - it's a single executable +> that takes in your source files on any platform, and produces a +> packaged binary for any platform, from any other. The tool doesn’t +> depend on anything besides what came with your OS...shipping as a +> single executable that about a megabyte. [sic] +> +> But that technical feat alone isn't enough to define Red's notion of +> a "Full-Stack Language". It's about the ability to bend and redefine +> the system to meet any need, while still working with literate code, +> and getting top-flight performance. So what's being put in your +> hands is more like a "language construction set" than simply "a +> language". Whether you’re writing a device driver, a platform-native +> GUI application, or a shared library... Red lets you use a common +> syntax to code at the right level of abstraction for the task. + +Source: [Red: About](https://www.red-lang.org/p/about.html) + +This is exactly what I dream of, although keeping the entire thing to +about a megabyte in size seems unrealistic. (I'm thinking of using +libgccjit for native compilation, which already has an installed size +of tens of megabytes, though statically linking a reduced portion may +bring it down a bunch.) + +In fact, I may have independently come up with the idea of a language +implementation being almost more of a "programming language toolbox." +Or maybe I subconsciously stole the idea from Red because I read about +their "language construction set" idea years ago; who really knows. +In any case, it's a very exciting idea. Such a toolbox would allow +you to create code-bases using specialized dialects of the language, +while still using the base machinery provided by the language. This +stands in contrast to a language being "opinionated" in how you should +write your code and having its own mind on how to actually do things +at run-time, which tends to be the case with higher level languages. + +It would mean that the language is not for the faint of heart. Though +I want it to be possible to write high-level Zisp code without having +to think of any of the more complex mechanisms, offering an experience +comparable to writing Python or JavaScript -- or rather, Scheme -- the +real power of the language would only show itself to those who are +familiar with low-level concepts and typical implementation machinery +of languages. A seasoned developer could take control of the behavior +of the garbage collector to fine-tune it for the allocation patterns +of their application, or even write modules that entirely avoid GC, +opting for less automatic memory management strategies such as pools +or totally manual alloc/free. They could have Zisp produce highly +optimized machine code (while remaining cross-platform) by providing +all the static type declarations necessary to eliminate run-time +overhead, and use a low-level record type system that allows defining +the exact in-memory layout of data records so as to make the best use +of CPU cache lines and whatnot. A Zisp code-base could be as simple +as a beginner-level Python code-base, or as complex as an advanced C +code-base. + +In some sense, Emacs has a similar philosophy, but applied to making a +text editor. At its core, it's a lisp machine, containing a bunch of +primitives that are conductive to creating a text editor; and then the +text editor is implemented on top of that and has a flexibility you +won't find in many other editors. Zisp would be something akin to a +set of tools useful to creating a programming language, and then kind +of a "default" language constructed with those tools. It's like the +language is telling you: "You can either give me a very simple high +level description of what you want to happen, and I'll make it happen +somehow; or you can tell me every little detail of how you want me to +do it." + +Red is described as a homoiconic language, like Lisps. I'm not sure +why they didn't just go with some kind of s-expression syntax. They +don't use a syntax that's any more familiar to the average programmer; +it seems like they have their own unique thing. If I went through the +documentation of Red, I'm sure I would find many other reasons why I +want to create my own thing instead of simply joining their project. +Could be a great source of inspiration though; I'll have to take a +closer look one day. + +To finish off this note, I want to provide an example of how a little +Zisp snippet could look when written in the high-level "don't care" +style, and then transformed into a lower-level style to take control +of more details of its run-time behavior. + +This is an imaginary script that walks through a directory and creates +HTML files from Markdown files. + +Note that this is essentially pseudo-code. The Zisp parser that +exists as of the time I'm writing this should be able to parse the +snippets, but everything beyond that is fantasy and not necessarily +representative of what actual Zisp code will eventually read like. + +```scheme +(import (zisp base)) ;import Zisp base language & stdlib + +(link (zisp regex)) ;dynamically link regex library +(link (de tkammer markdown)) ;dynamically link a Markdown library + +(define (md2html from to) + (print "Converting {from} to {to} ...") + (with ((in (file.reader from)) + (out (file.writer to))) + ;; Get title from first line of Markdown + (define title (regex.replace "^# " "" (in.first-line))) + (define head-template (file.read "head.html")) + (define head (head-template.replace "__TITLE__" title)) + (out.write head) + (out.write "") + (out.write (format "

{title}

")) + (markdown.stream in out) + (out.write ""))) + +(define (main) + (md2html "index.md" "index.html") + (loop (mdfile (glob "notes/*.md")) + (define name (mdfile.basename)) + (md2html mdfile (format "notes/{name}.html")))) +``` + +A simple script like this may not realistically require optimization, +but we will do it for the sake of the example. Let's identify sources +of overhead and unnecessary duplicate operations: + +1. The regex library is dynamically linked, so the regex `"^# "` will + need to be re-compiled on every loop. + +2. The file `head.html` is read on every iteration. + +3. Strings like `title` and `head` are allocated at every iteration + and would eventually need to be collected by the GC. + +You may think there's a lot more than that, such as a bunch of type +checks at run-time, since there are no explicit type declarations. +However, given the static type information available for the standard +library, and some basic type inference, this program should already be +mostly free of type checking overhead. For example, `file.reader` is +known to return a reader, so `in.first-line` is known to return a +string, and so on. (In fact, assuming that the dynamically linked +libraries also provide sufficient type information at compile time, +this program could be entirely statically type-checked, if I'm not +mistaken.) + +Some of the optimizations we will perform are not tied to any Zisp +specialty; any sane language would, for example, allow you to read +`head.html` once and save the string for re-use. Nevertheless, the +transformation of the program will provide some examples of what I +want Zisp to be capable of. + +Here's the transformed code: + +```scheme +(import (zisp base)) ;import Zisp base language & stdlib +(import (zisp regex)) ;import regex library (compile-time) + +(link (de tkammer markdown)) ;dynamically link a Markdown library + +(define title-regex (regex.compile "^# ")) +(define head-template (file.read "head.html")) + +(define title-limit 80) + +(define title:buffer) +(define head:buffer) + +(define (md2html from:string to:string) + (print "Converting {from} to {to} ...") + (with ((in (file.reader from)) + (out (file.writer to))) + ;; Get title from first line of Markdown + (title.reset) + (unless (title.read-line in) + ;; read-line returns false if the buffer fills up before the + ;; line ends, so skip the rest of the first line + (ignore (in.first-line))) + (title-regex.replace "" title) + (head.reset) + (head.read-string head-template) + (head.replace "__TITLE__" title) + (out.write head) + (out.write "") + (out.write (format "

{title}

")) + (markdown.stream in out) + (out.write ""))) + +(define (main) + (set! title (buffer.make title-limit)) + (set! head (buffer.make (+ head-template.length title-limit))) + (md2html "index.md" "index.html") + (loop (mdfile (glob "notes/*.md")) + (define name (mdfile.basename)) + ;; format still allocates but I'm too lazy to change that too + (md2html mdfile (format "notes/{name}.html")))) +``` + +To be honest, there's nothing too crazy going on here. The strategy +of using statically sized buffers that are allocated once globally +could be implemented even in standard Scheme, using the bytevectors +library. Indeed, Scheme is already quite good at enabling you to +write fairly efficient code, as far as dynamically typed, garbage +collected languages are concerned. + +Nevertheless, we see for instance that the globals `title` and `head` +are defined with static type information, ensuring that no run-time +checks need to be added to their usage later. + +(The addition of type information to the arguments of `md2html` is +rather stylistic, since they could be inferred via bidirectional type +inference anyway.) + +One surprising difference from Scheme would be that the values of the +top level definitions such as `title-regex` would actually be created +at compile time, and serialized into the resulting binary. See the +note ["Compilation is execution"](250210-compile.html) on that. + +As I'm writing this, I notice that escape analysis is crucial for the +automatic elimination of some heap allocations. In this case, that +would apply to variables like `mdfile` and `name`, for example. + +All in all, I guess I've ended up demonstrating nothing more here than +the possibility of adding some static types to a program written in a +Scheme-like language to eliminate some dynamic type checks it would +otherwise need to have. Big deal. I really hoped that this little +snippet would be conductive to demonstrating a bunch of other Zisp +features enabling "low-level programming" (imaginary Zisp features, +that is) but I guess I should have come up with a better example. + +In fairness to me, it's easy to underestimate just how efficient a +Scheme-like language (or indeed a Scheme implementation) could already +be, if the compiler implemented all kinds of optimizations like type +inference and escape analysis across function boundaries, and if the +programmer writes allocation-avoiding code. + +It would be interesting to rewrite the above snippet in a style that +uses memory pools instead of avoiding dynamic allocation. That could +lift the static limits while still eliminating GC use. This shall be +left as a later exercise for myself. -- cgit v1.2.3