Interim OS: Reclaiming the Computer through Minimalism and Genericity

Lukas F. Hartmann (lukas@mnt.mn). Berlin, June 10th, 2015

1. Introduction

"Am Heimcomputer sitz' ich hier / Und programmier' die Zukunft mir" – Kraftwerk: Heimcomputer (1981)

Computers, networks and the software running them today are shrouded in mysteries and corporate secrecy. As miniaturization progresses in the name of mobility and energy-efficiency, an increasing amount of complex functionality is crammed into ever smaller System-on-Chip dies.

The so-called "Home computers" of the 1980s contained comparably larger and simpler circuit boards with blocky, easily discernible DIP (Dual in-line package) components and circuits that could be visually understood by the human eye. The central processing unit (CPU) was easily identified by its size and exposed placement. The separate memory chips were neatly arranged like terraced houses. The computers worked in pedestrian single-digit-Mhz speeds and memory was measured in kilobytes. They shipped with handbooks that taught a novice reader how to program the machine, and a circuit diagram of the whole machine – useful for repairs – was easily available.

In the 1980s home computer era, operating systems where typically stored in read-only memory (ROM) chips. As in modern proprietary operating systems, the source code was not directly available, but this was not strictly necessary, as they were written in assembly language and not "compiled" from a higher level language. Commented "dis-assemblies", machine code listings, were available in printed book form [Schineis1984] for popular computers like the Commodore 64 and its "KERNAL" OS and BASIC language interpreter.

Today, we have Linux, probably the most successful open source Unix-like operating system and the BSD family of OSes, but these systems and most of the platforms they run on (PCs, ARM-based telephones) are so complex and contain so many obscure components that no single book can describe their operating principles in full detail, and trying to understand and master them is a task that takes many years of study.

With "Interim", I try to describe a computer and operating system that takes advantage of modern-day hardware technology while ideally being fully comprehensible in a couple of days. My strategy is to use minimalism and generic, reusable patterns wherever possible while learning from historical, ultimately unsuccessful but valuable attempts like Lisp machines or the operating system Plan 9 from Bell Labs [Pike]. The Interim system is supposed to be a pointer in the right direction, not a perfect blueprint, and a documentation of my own experimental attempts. Others may build upon these ideas.

2. Where Complexity Hides

2.1. Drivers and Protocols

A majority of the millions of lines in the Linux kernel [Linux2015] are parts of so-called device drivers. Linux' main job is to abstract away the details of thousands of different pieces of hardware modules that were at some point available on the market and somehow turn them into a running whole, an operable programming interface that takes input signals and produces visual, audio or network output. The developers of the Linux kernel try to make the OS work on a large variety of computers with totally incompatible architectures. This is a noble goal, and Linux is stable and usable, but the sheer amount of source code produced means that no single person can understand what is going on in the whole of the system. It is hard to understand which of the many available "kernel" modules are necessary to construct a working system from them. Documentation is fragmented.

For business reasons, contemporary hardware vendors are having a hard time adhering to compatible standards. A driver for a NVidia graphics card does not work with an AMD or an Intel graphics card. Executables compiled for an ARM processor do not run on an Intel processor. Every printer needs a different driver, even if they all output raster and vector graphics. Only in some parts of the ecosystem, for example for USB keyboards and mice (so called "HID" devices), have small standardized corners been established.

In modern monolithic operating system kernels, drivers run "in the kernel", which means that they have unchecked read/write access to any system resource. Technical errors in these drivers can lead to system instability ("crashes") and open up attack surface for security exploits (see SEC Consult 2015 [SECConsult2015]). If a driver overwrites critical kernel data structures, the whole system will freeze (in the best case) and the user will lose their unsaved work.

To activate and communicate with hardware, the OS and its drivers have to implement so-called "protocols", sets of rules that govern which signals or data packets are valid at which point in the communication with the device. Every family of devices uses their own set of protocols. More complex protocols are layered on top of these. A popular example is the OSI model for networking, which defines how Internet communication is built from a stack of layered protocols [ITU1994].

2.2. Graphics and Compression

Moving computer-generated graphics are often used to demonstrate a new system's power. Since the dawn of multimedia and 3D games like DOOM, engineers have struggled to make computers display moving graphics that live up to the resolution, gamut and speed of human visual perception. For audio, this was much simpler; high definition stereo audio playback and recording is a problem that was considered solved in the 1990s. At CD quality (44100 Hz, 65536 discrete steps of volume), audio quality is "good enough" for most people, its bandwidth easily manageable even when uncompressed. Playback of MP3-compressed recordings requires almost no CPU resources on modern computers and can be implemented in a few hundred lines of C code.

But graphics are still problematic. When Apple released the first "Retina Display" device, the iPhone 4 with a pixel density of 326 PPI (pixels per inch), Steve Jobs claimed that it would match the resolution of human visual perception at a typical viewing distance. Retinal scientist Bryan Jones confirmed this claim by calculating average human visual perception density at 287 PPI [Jones2010]. A 30 inch desktop display that matches this density would require around 8000 × 4000 pixels resolution (298 PPI). To work with non-aliased text and imagery on computer screens, 2D resolutions between Full-HD (1920 × 1080) and 4K are acceptable, though. Still, the available mainstream CPUs are, on their own, not capable of painting scenes at 50 Full-HD frames per second.

This is where specialized chips called GPUs (graphics processing units) come into play. One basic operation of a GPU is to paint a triangle into a bitmap. The way this triangle is colored is specified by a so-called "shader". The GPU consists of many small, simpler CPUs that can work in parallel and that can execute "shader code" (a specialized list of GPU instructions) at high speed. The software application breaks down the desired visual scene into many triangles and submits them to the GPU for rendering. The GPU is a computer and operating system on its own: it accepts the triangle painting job and distributes the work automatically among its tens or hundreds of little shader cores. The main CPU is then immediately free to do other work. Moreover, the GPU has the privilege of circumventing the CPU altogether and can just write to main memory without intervention, a principle called DMA (direct memory access).

While offloading the painting work to a dedicated processor is a simple concept in theory, contemporary implementations are complicated, un(der)documented and potentially insecure (see [Hartman2008]). In the popular low-cost Raspberry Pi computer, for example, the idea of privileging the GPU, originally an add-on device, has been taken to the extreme: the undocumented Broadcom VideoCore GPU is the main processor, having full control of a secondary ARM CPU that runs the user's operating system and software [BeyondLogic2014]. The architecture and processes running in the main core of a "device that enables people of all ages to explore computing" are kept secret, which runs counter to the goals of computer science education.

While audio and still image compression can be seen as well-solved problems, video compression and streaming are – again due to their high bandwith nature – a major factor that fuels software and hardware complexity and obfuscation. The specification for the H.264 video compression standard, for example, is 790 pages long [ITU2014].

2.3. Complex, Closed CPUs

Putting together a working computer system gets very complicated quickly. Mainstream processors in the Intel 8086 tradition carry in themselves the tree rings of over 30 years of legacy architectures (see [Intel2015]). When a modern 64-bit Intel CPU is powered on, it starts in a historical 16-bit mode to be compatible with ancient BIOS ROM code before it is eventually switched by an operating system through a ritual of instructions to 32-bit "real mode" and finally to 64-bit "long mode". At every boot, computing history is repeated in time-lapse. In the 1990s, during the Intel-AMD-wars, new technologies were piled on top of each processor generation in quick sucession to generate new marketing buzzwords: MMX, 3dnow, SSE, SSE2, SSE3, SSSE3, AVX, AES, VT-x and so on. Most of this is kept alive to ensure binary compatibility with software of the past.

The Apple iPhone and smartphones in general led to a surge in demand for a much simpler RISC architecture designed by ARM, whose lineage stems from British 1990s personal computer manufacturer Acorn (ARM originally meant Acorn Risc Machine) [ARM2015]. The simplicity of RISC allows for simpler system construction, and RISC CPUs generally require less power than rivalling CISC machines.

While the R for RISC in ARM hints at a future of simpler systems, ARM is not a perfect solution for an open system, because the architecture is heavily patented and licensed out to only a few, high-paying chip manufacturers. Efforts to create open source ARM-compatible cores have been thwarted by ARM's legal measures [Clarke2001]. This opposes the goal of having a fully open system.

Serious efforts to produce an open RISC CPU are underway in the form of lowRISC [Mullins2015], which builds upon Berkeley's RISC-V architecture, that, according to its developers, is "a new instruction set architecture (ISA) that was originally designed to support computer architecture research and education, which we now hope will become a standard open architecture for industry implementations" [RISCV2015].

2.4. Post-PC Operating Systems

In the "Post PC" era (see [Fried2010]), commercial operating systems investments shifted towards mobile platforms like Apple iOS (a derivative of BSD), Google Android (a derivative of Linux) and mobile versions of Microsoft Windows. A focus on touch screen input and simple finger gestures of these platforms indicates their optimization for media consumption as opposed to content creation by users (with the exception of short text messages and photo posts in social media). Applications are not meant to be created on these platforms themselves; for this and other "professional" media creation tasks, Apple sells a separate, incompatible line of computers, the Macintosh and OS X (incompatible in the sense that you cannot run iOS application bundles on a Mac and vice versa). The latter is a fork (derivative) of open source BSD components with proprietary, closed-source device drivers and user interface stacks mixed in. According to the End User License Agreement, it is a violation of this agreement to run OS X or iOS on non-Apple computers, even if they are technically compatible [Apple2011].

When a developer creates an iOS application on a Mac, this application cannot be (legally) run on any other platform, even if there is no technical reason for this. There could be a standard format or API for platform-independent applications, but to protect their income streams, Apple and others have chosen the path of "vendor lock-in", so developers have to invest in separate developments for each different platform.

Open systems like Linux or FreeBSD [FreeBSD2015], on the other hand, have a notoriously bad reputation for being hard to configure and learn (see [Kegel2006]). This stems from a lack of minimalism and genericity which make up user-friendly design: the fewer different concepts and patterns a user has to master, the better; the more areas a user can apply the same learned strategies to, the more empowered they are.

3. Design of the Operating System

3.1. Language Based System

To start describing "Interim", a new, simpler operating system, we start at the core of communication between a user and a computer: a language. Giving a nod to Lisp machines and the boot-to-BASIC home computers of the late 1980s, Interim shall be a "Language based system" (see [Schneider2000]). This means in practice that a compiler and interactive environment for the "native" language of the OS is included in its core, eliminating the difference between a "command line shell" and programming languages for building "applications". In a language-based system, these are one and the same. When you switch on a Commodore 64, you can immediately interact with it in its native BASIC language and write programs in this language; a similar principle applied to graphical Lisp workstations [Symbolics1984] which used Lisp at every level of the system, or Smalltalk environments, which are made up of uniform objects that pass messages to each other, and where every aspect of the system can be inspected and manipulated at runtime (see [Kay1993]).

A language based system has these advantages:

  1. Text and symbols are the means by which the bare computational resources of the machine can be made available for meaningful operation by a human.
  2. By designing the language in a "safe" way, i.e. incorporating structural rules that provide for crash-resistant behaviour, the task of memory protection (protecting one application's data or code from being overwritten by another's) gets simpler. In practice, safety is achieved by making available only the symbols that a function (read: "application") may access and providing no means to directly access the lower levels of the machine. For example, no access to memory by reading and writing arbitrary addresses ("pointers") is possible. All access to machine resources is abstracted and mediated through named (human readable) symbols provided by the language-based kernel.

Traditional Unixes provide unsafe C APIs (Application Programmer Interfaces) and boot into an "init" program which in turn can launch more programs or interactive "shells". A shell, like "bash", the "Bourne Again Shell", provides a language based interface to the system, but does this in a flawed way: By typing a program's name, the user can launch arbitrary binary executable files which in turn contain raw machine instructions. By exploiting subtle errors in the operating system implementation, these binaries can escalate their "privileges" and damage the user's system and data (see [PrivEsc2015]).

Moreover, this traditional approach of command shells and binary executables introduces additional complexity in many layers. A plethora of incompatible (in terms of language) shells is available to choose from; access to system resources is not mediated through visibility of symbols (an exception to this is the Plan 9 operating system, which introduced "name spaces"); "shell languages" often have irregular syntaxes and comparably slow performance. In many cases, shell languages are not suitable for writing applications (except for small tools), so other, non-shell languages have to be used instead.

Looking back at computing history, there are multiple options for choosing a system language. At least three well-known historical projects have successfully implemented language based systems/computers: Smalltalk environments, Lisp machines, and the Oberon System [Wirth1988]. The Oberon System is a fascinating, perfectly valid candidate for a language based OS, but I chose Lisp because of its simpler core. I did not choose Smalltalk because I think objects and messages are additional abstractions on top of symbols und functions; as the Common Lisp Object System (CLOS) demonstrates, Lisp can be easily grown into an object oriented system if one so desires (see [HyperSpec2005]).

3.2. Lisp: "Everything is a symbol"

The central ideas of Lisp ("list processor") originate from John McCarthy's 1960 seminal paper [McCarthy1960] wherein he specified on a single page a set of functions that would implement a Lisp interpreter in Lisp itself, known as the "meta-circular evaluator". To quote McCarthy, "The Lisp programming system is a system for using the IBM 704 computer to compute with symbolic information in the form of S-expressions"; S-expression means "symbolic expression" [McCarthy1960]. The goal of Lisp was thus to allow the user to operate the computer through symbols instead of pure numbers or on-off-switches.

A symbol is just a name that you can assign to any object in the system, and if you give the system this name later, it will recall the object (it's "meaning") for you. A symbol points to an atom. An atom can be a number, a text, a program listing, an image, a formula – or indeed anything else. Programs can be built through the combination of symbols.

Alan Kay famously called [Kay2004] the source code on Page 13 of the "Lisp 1.5 Programmer's Manual" [McCarthy1962] the "Maxwell's equations of software", comparing McCarthy's "evalquote" function to the foundational equations of classical electrodynamics. Lisp is elegant and simple because it defines only one data structure: The "cons" cell (here: pair). This design allows the construction of arbitrarily nested trees of symbols. Not only data, but also computer programs themselves are conveniently manifested as such trees – the "abstract syntax tree" (AST). Thus Lisp programs are written in the same list structure that they can manipulate. This principle of equality of program structure and data is called "homoiconicity".

As the Interim operating system's language core, a Lisp dialect is beneficial because:

  • Lisp systems are relatively simple to define and implement.
  • Lisp has a very regular, homoiconic syntax.
  • Lisp is easy to learn.

In fact, complete Lisp-based computers like the CONS and CADR were successfully built in the 1970s by members of MIT's Artificial Intelligence Laboratory (see [Knight1977]).

3.3. Interim: "Everything is an expression"

As the linguistic core of Interim OS I propose a minimal Lisp-like programming language called "Interim". It is closely related to the Scheme language ([R6RS]) but has less features and the following differentiating design principles:

  • Everything is an expression.
  • All behaviour is defined (languages like C and C++ contain undefined behavior [Lattner2011]).
  • The whole system consists only of symbols, atoms and links between atoms (pairs).
  • At every point in the program, there is one visible environment, which is a mapping of symbols to atoms.
  • Atoms are immutable, but symbols, which point to them, can be replaced.
  • The language can be easily compiled to stack-machine or register-machine code.

Interim is on the one hand a suitable compilation target for higher-level traditional languages like Python or JavaScript and represents an alternative to asm.js (see [AsmJS2014]) or LLVM's IL (see [LLVM2015]); but at the same time, Interim code can be hand-written and read by humans with relative ease. It is thus middle ground between the low-level but cryptic world of assembler and the comfort of modern high-level languages like Python ([Python2015]), Haskell (see [Jones2007]) or APL ([Iverson1962]).

As a prototype, I implemented a usable just-in-time (JIT) compiler for Interim in C99 that targets the ARMv7 and Intel x64 instruction sets, and a run-time environment that handles timing/scheduling, memory allocation/garbage collection and exposes system resources symbolically in a virtual file system (for source code see [Hartmann2015]). The garbage collector is a simple mark-and-sweep collector that descends the roots of all global and stack-defined symbols to discover "live" symbols, discarding all non-reachable symbols and freeing their memory. A process can be spawned by appending a function to the global "processes" list. An interrupt-based scheduler divides available CPU time between those functions.

Interim's syntax is defined as follows.

An atom is one of the following:

  • an integer: 1234
  • an ASCII symbol: hello
  • a UTF-8 string: "こんにちは"
  • a hexadecimal byte string: [deadbeef01020304]
  • a pair: (123 . 456)
  • a "lambda", which is an applicable function
  • nothing

A pair is a link between two atoms; it has a left-hand side (traditionally called "car", "contents of address register") and a right-hand side (traditionally called "cdr", "contents of data register").

A list atom is a pair whose left side points to an atom of any type and whose right side points to the next list atom or an "empty list" atom (a pair of two "nothings"). Lists are entered and displayed simply by separating atoms with whitespace and wrapping the result in parentheses: (1 2 3 4).

Interim programs are simply Interim lists. The first item of a progam list must be the symbol of the function to apply to the following parameters; the remaining items of the list are the parameters:

(+ 1 2)    ; evaluates to 3.
(- 5 4)    ; evaluates to 1.
(def a 23) ; evaluates to 23. as a side-effect, the symbol "a" is defined as the number 23.
(def double (fn x (+ x x))) ; defines the function "double" that takes one parameter called "x"
(double a)                  ; now evaluates to 46.

3.4. Interim core functions

A number of built-in symbols are available at system startup. These are the building blocks of the environment that the user can interact with and shape. From these, more complex functions and more useful user interfaces can be built.

Symbol definition and functions

(def <symbol> <atom>)
Definition. Binds an atom to a symbol.
(fn <arg1 argn …> <body>)
Function. Defines a function with parameter symbols ("arguments") which can be used as placeholders anywhere in the function body.
(<symbol> <arg1 argn…>)
Application. Apply arguments to the function bound to the symbol.

Arithmetic

(+), (-), (*), (/), (%)
Add, subtract, multiply, divide, modulus. These operate on integers. A non-nothing non-integer is interpreted as 1. Nothing is interpreted as 0. This allows logic to be constructed from arithmetic.
(gt <a> <b>), (lt <a> <b>), (eq <a> <b>)
Compare two values (a and b). Gt returns 1 if a is greater than b and 0 if not. Lt does the opposite. Eq returns 1 if a and b are equal (they represent the same number or object) and 0 if not.

Flow control

(if <condition> <then> <else>)
Conditional branch. If condition evaluates to non-zero, <then> is evaluated, or else <else> is evaluted. The value of the evaluated branch is returned.
(while <condition> <then>)
Conditional loop. As long as condition evaluates to non-zero, <then> is evaluated over and over again. The value of the last evaluation is returned.
(do <expr1> <exprn> …)
Sequencing. Expressions are evaluated in the given order; The value of the last evaluation is returned.

Pairs & Lists

(cons <a> <b>)
Returns a pair of a and b: (a.b).
(car <pair>)
Returns the left side of a pair.
(cdr <pair>)
Returns the right side of a pair.
(quote <s-expression>)
Returns the given S-expression unmodified without evaluating it.
(list <a> <b> …)
Constructs a list from the given parameters.
(map <fn> <list>)
Applies the function fn to each of the list's items and returns a new list containing each application result in the same order.

Homoiconicity & Reflection

(read <string>)
Deserialization. Parses a string (text) into atoms.
(write <op>)
Serialization. Returns a string representation of the given atom.
(symbols)
Reflection. Returns a list of all visible symbols.

Strings & Buffers

Strings (text), byte strings (binary data) and files have a uniform interface in Interim OS. All system resources (see Section 3.6) in Interim OS are exposed as file systems which can be addressed using (open <path>). The opened "file" behaves exactly like a byte string; thus, a string created at run-time is a nameless file in memory.

(alloc 65536)
Returns a 64-kilobyte string of zeroes.
(size <str>)
Returns the length of a string in bytes.
(slice <str> <offset> <length>)
Returns a substring of length <length> that starts at <offset> bytes in the given string.
(put <str> <offset> <byte>)
Replaces the byte at position <offset> in the given string with a copy of <byte>.
(get <str> <offset>)
Returns the byte at position <offset> in the given string.
(send <sym> <str2>)
Sends (appends) a string to the given symbol, which can be a file, a device or a string (uniform access).
(receive <sym> <length>)
Receives (consumes) up to <length> bytes from the string or file represented by <sym>.

3.5. Emacs: "Everything is text editing"

To interact in a textual dialog with the system, it needs a facility for text editing. We can learn a great deal from the design of the "Emacs" text editor in this regard. The earliest working version of Emacs was written in 1976 by Guy Steele and Richard Stallman (Stallman is best known for founding the free software advocating GNU Project in 1983). The central design idea of Emacs is to have a relatively small fixed core of text rendering and manipulation capabilities, text buffers and windows. Everything else can be changed and extended interactively by writing Emacs Lisp programs in Emacs itself. Emacs (now available in its 24th version) has thus earned a reputation for being flexible and extensible. Email clients, web browsers and support for all kinds of programming languages have been implemented in Emacs Lisp. [Harvey2015]

This extraordinarily broad spectrum of use cases offered by a single application is possible because most online communication and programming itself is fundamentally text-based and differs only in the syntax, semantics and protocols that govern how the text can be manipulated and exchanged, be it in the long form of emails or web articles, the short form of chat messages or the sprawling file trees of software source code. The central benefit for the user is that all text buffers operate in the same, customizable way (genericity). The same keyboard shortcuts work the same in every text buffer, while the fonts and colors can be globally tuned to one's tastes and preferences.

Critics may point out that Emacs is a prime example of "software bloat" because its swiss-army-knife nature contradicts the Unix philosophy of "do one thing well". In "Emacs is Dead", Tomohiro Matsuyama states that "The reason why [the] Emacs platform is good is that it cooperates with [the] OS, not because it is good by itself" [Matsuyama2010]. I argue that something like Emacs would not need to exist (as a monolithic application) if the OS itself supplied the building blocks for integration that Emacs or "integrated development environments" (IDEs) can offer. In contrast, today we have a situation where multiple incompatible large application "suites" (e.g. the Adobe Creative Suite or Microsoft Office) run in parallel on top of the OS. If users are lucky, they can use the copy & paste mechanism to transfer data (mostly plain text) from one program to another.

I argue that an operating system can be designed as a database of well-formed functions that each do one thing well. The functions can then be composed to form ad-hoc "applications". A requirement for this is that unlike in proprietary/closed/binary designs, all source code must be open and all functions reusable in new contexts.

Interim OS includes functional building blocks for text display and editing, and an example development environment written in the Interim language:

  • A list of "buffers". A buffer is an array of bytes, preferably "buffered" in working memory, but a buffer can also be a window into a file or a stream of data from the Internet.
  • A list of rectangular "views". A view is a visual representation of a buffer on a graphical display. It has a 4 coordinates that define it's rectangular position on the screen and a 5th "depth" coordinate that defines its pseudo 3D ordering among other overlapping views.

A view displays one buffer and has one "mode". The mode defines how the view interprets and renders the buffer (as plain text, as formatted text, as an image, as a movie…) and can offer editing capabilities to the user, such as inserting new or deleting existing characters in a text or plotting colored pixels with a virtual pen. Modes can be switched at any time by the user, so the same buffer can be interpreted in different ways: you could listen to an image, look at the structure of a text graphically or paint into an audio sample.

3.6. Plan 9 from Bell Labs: "Everything is a file system"

There are a few proven models for making digital systems interact well with each other:

  1. The Unix model of having many small programs (functions) that do one thing well and that provide standard input and output streams, so they can be plugged together to form more complex pipelines.
  2. Uniform access to system resources as "files" that can be read from or written to, one character at a time.
  3. RESTful web services that allow the transfer of "state" (data) between clients and servers over the Hypertext Transfer Protocol, HTTP. [Fielding2010]

The Plan 9 operating system, developed as the intentional successor to Unix System 8 at Bell Labs from 1992-2002, combined all these models in its "everything is a (networked) file system" design. The computer, its resources, connected networks and world wide resources should all be available in a tree-structured global name space. Access control and security can then be established by deciding which parts of this tree to expose under which symbolic names to which running processes.

Interim embraces this idea and takes it one step further by offering the same unified read/write interface to buffers, strings and files. This allows the same operations (functions), for example image manipulation or text translation, to be used in interactive GUIs and in automated server applications. However, it is non-trivial or almost impossible to use, i.e., an Adobe Photoshop image filter in a web-based image processing service.

As hardware resources such as displays are files, it becomes trivial to share a screen over a network and perform image filtering or construct chat or collaboration services around it.

3.7. Multimedia

As established in section 3.6, all system resources are exposed to Interim as files; the same holds for the display frame-buffer. Drawing a magenta pixel at display position 200:150 is the same as putting three bytes (red, green, blue) starting at the corresponding position of the display file:

(def magenta #ff00ff)
(def screen-width 1920)
(def display (open "/dev/display"))
(put display (+ (* screen-width 200) 150) magenta)

A graphics accelerator can be viewed as a processing node that has direct access to the display frame-buffer and can output specialized calculations (geometry drawing) very quickly, freeing the main CPU of the computer for other work.

As an example of how graphics acceleration can be implemented, to send a complex drawing job to an auxiliary graphics accelerator, a higher-level description of the geometry could be sent to the accelerator's command file:

(def accel (open "/dev/display/accelerator"))
(send accel "(triangle #ff00ff 50 50 100 50 100 100)") ; draw a magenta triangle with the three points 50:50, 100:50 and 100:100.

4. Design of a Physical Computer

An ideal physical computer for Interim OS has one or more RISC general purpose processors (CPUs) and one or more (depending on the number of required display outputs) FPGA-based graphics processors (GPUs) (e.g. [Nadeau2010]), as well as one microcontroller-based peripheral controller. If electronically feasible, the peripheral controller can be of the same design as the main CPU, so the same programming tools can be re-used.

A minimal version of an Interim computer with parts that can be hand-soldered can be constructed, for example, around a Zilog eZ80 CPU. Fast and simple Microcontrollers like Teensy 3.1 (based on Freescale ARM Cortex-M4 MPU) can be used as a "poor man's GPU" to generate VGA text or graphics output. The prototype RS-16 computer I built and several "homebrew" computer projects by hobbyists successfully demonstrate this (see [Hartmann2014]). The VGA signal is simple and requires only a few resistors to generate a variety of color shades. A computer built from components like these can serve as an educational tool or for simpler automation tasks.

A language-based system is very portable and automatically makes all its software portable. To demonstrate this, I developed a version of Interim (then called "Bomberjacket OS") that boots on a standard Intel x86 PC as well as a version that runs on the ARM-based Raspberry Pi low-cost computer. Both can run the same unmodified text editing software, which is JIT-compiled upon loading. I also wrote a very minimal TCP/IP stack and successfully connected to Internet Relay Chat (IRC) and fetched simple websites on Interim OS (see Illustration1, Illustration2 and Illustration3).

For the Raspberry Pi version, I ported a USB driver, a network driver, an SD card reader and a low-level driver for the Pi's VideoCore IV GPU. This allowed the Interim OS to boot into a responsive graphical environment on the Raspberry Pi, connect to the internet and render 3D-accelerated graphics. I encountered countless glitches, crashes and performance problems during this effort; these I attribute to the over-complex nature of the used Broadcom SoC and the severe lack of proper documentation. This is why I advocate simple and open hardware design as a requirement for a stable platform. The source code for all Interim OS experiments is available on GitHub [Hartmann2015].

While computer chips cannot yet be manufactured by hobbyists, it is possible to hand-craft input peripherals like mice or keyboards. During the research for Interim, I constructed a mechanical keyboard from individual Cherry MX switches and hand-soldered a scanning matrix on the back of the keys (Illustration5). High-quality enclosures can be designed in a vector graphics program and constructed from laser cut acrylics (see Illustration4). I successfully tested this by building a physical keyboard for Interim in roughly two days.

5. Conclusion

Computers, free and open operating systems, programming language and hardware design have come a long way. Now it is time to integrate lessons from over 50 years of trial and error into minimalistic, understandable and elegant computers. Interim is, as the name implies, an experiment of combining the homoiconicity of Lisp (manipulation of symbols), the genericity of Plan 9 (everything is a file system) with the minimalism and hackability of home computers. New languages like Rust [Rust2015], Unikernel systems like Mirage OS [Mirage2015] and hardware designs like lowRISC or the Novena laptop [Novena2015] show that it is absolutely possible to build simpler, more open and safer systems than we take for granted today.

Thanks to Florian Hadler, Sebastian I. Hartmann, Michael Haupt and vrs (on Freenode) for constructive criticism of the paper, to Colm Kelly for corrections, to Nils D. Moskopp for extensive feedback during practical OS development, and to Greta Melnik and GE Germany for assistance during hardware prototyping.

6. Illustrations

illustrations_small/3254_overview_front_bw.jpg

Illustration 1: Interim OS running on Raspberry Pi 2 inside custom built keyboard. In the top left corner is a mouse driven pixel graphics program implemented in the Interim language.

illustrations_small/3264_cmdline_bw.jpg

Illustration 2: Interim OS interactive environment; IRC client connecting to Freenode chat network in the background.

illustrations_small/3250_interim_html_bw.jpg

Illustration 3: Interim OS fetching and displaying its own website via HTTP.

illustrations_small/lasercut_bw.jpg

Illustration 4: Lasercut acrylic keyboard top plate for Interim computer.

illustrations_small/keyboard-matrix_bw.jpg

Illustration 5: Manual wiring of mechanical keyboard for Interim computer.

7. References

[Apple2011]Apple Inc. 2011. "SOFTWARE LICENSE AGREEMENT FOR MAC OS X." http://www.apple.com/legal/sla/docs/macosx107.pdf
[ARM2015]ARM Ltd. 2015. "ARM Company Milestones." http://www.arm.com/about/company-profile/milestones.php
[AsmJS2014]Herman, David et al. 2014. "asm.js Working Draft — 18 August 2014." http://asmjs.org/spec/latest/
[BeyondLogic2014]Beyond Logic. 2014. "Understanding the Raspberry Pi Boot Process." http://wiki.beyondlogic.org/index.php?title=Understanding_RaspberryPi_Boot_Process
[Clarke2001]Clarke, Peter. 2001. "Student's ARM7 clone disappears from Web." EETimes. http://www.eetimes.com/document.asp?doc_id=1144157
[Fielding2010]Fielding, Roy Thomas. 2000. "On Representational State Transfer (REST)." https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm
[FreeBSD2015]The FreeBSD Project. 2015. "The FreeBSD Project." https://www.freebsd.org
[Fried2010]Fried, Ina. 2010. "Steve Jobs at D8: Post-PC era is nigh." CNet. http://www.cnet.com/news/steve-jobs-at-d8-post-pc-era-is-nigh
[Ghuloum2006]Ghuloum, Abdulaziz. 2006. "An Incremental Approach to Compiler Construction." University of Chicago. http://scheme2006.cs.uchicago.edu/11-ghuloum.pdf
[Hartman2008]Kroah-Hartman, Greg. 2008. "Kernel Driver Statement." http://www.linuxfoundation.org/collaborate/workgroups/technical-advisory-board-tab/kerneldriverstatement
[Hartmann2014]Hartmann, Lukas F. 2014. "Riot Systems RS-16." http://riotsys.com
[Hartmann2015](1, 2) Hartmann, Lukas F. 2015. "Interim OS Source Code." https://github.com/mntmn/interim
[Harvey2015]Harvey, Greg. 2015. "GNU Emacs." http://www.gnu.org/software/emacs
[HyperSpec2005]LispWorks Ltd. 2005. "Common Lisp HyperSpec: Object Creation and Initialization." http://www.lispworks.com/documentation/HyperSpec/Body/07_a.htm
[Intel2015]Intel Corporation. 2015. "Intel® 64 and IA-32 Architectures Software Developer’s Manual." http://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-1-manual.html
[ITU2014]ITU-T. 2014. "Advanced video coding for generic audiovisual services." https://www.itu.int/rec/T-REC-H.264-201402-I/en
[ITU1994]ITU-T. 1994. "X.200 Open Systems Interconnection – Model And Notation."
[Iverson1962]Iverson, Ken. 1962. "A Programming Language." John Wiley and Sons, Inc.
[Jones2007]Jones, Simon Peyton et al. "A History of Haskell: Being Lazy With Class." http://research.microsoft.com/en-us/um/people/simonpj/papers/history-of-haskell/history.pdf
[Jones2010]Jones, Bryan. 2010. "Apple Retina Display." http://prometheus.med.utah.edu/~bwjones/2010/06/apple-retina-display
[Kay1993]Kay, Alan. 1993. "The Early History Of Smalltalk." In: HOPL-II The second ACM SIGPLAN conference on History of programming languages. Association for Computing Machinery. Copy at http://worrydream.com/EarlyHistoryOfSmalltalk
[Kay2004]Kay, Alan. 2004. "A Conversation with Alan Kay." In: ACM Queue Vol. 2 No. 9. Association for Computing Machinery. https://queue.acm.org/detail.cfm?id=1039523
[Kegel2006]Kegel, Dan. 2006. "Towards Linux Desktop Comfort or, Why Doesn't Johnny Run Linux?" http://www.kegel.com/linux/comfort/#usability
[Knight1977]Knight, Thomas et al. 1977 "Memo No. 444. LISP Machine Progress Report." Massachusetts Institute of Technology, Artifical Intelligence Laboratory
[Kreminski2015]Kreminski, Max. 2015. "Unix is not an acceptable Unix." http://mkremins.github.io/blog/unix-not-acceptable-unix
[Lattner2011]Lattner, Chris. 2011. "What Every C Programmer Should Know About Undefined Behavior." http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html
[Linux2015]Linux Kernel Organization, Inc. 2015. "The Linux Kernel Archives." https://www.kernel.org
[LLVM2015]LLVM Project. 2015. "LLVM Language Reference Manual." http://llvm.org/docs/LangRef.html
[Mullins2015]Mullins, Robert et al. 2015. "lowRISC." http://www.lowrisc.org
[Matsuyama2010]Tomohiro, Matsuyama. 2010. "Emacs is Dead (English Translation)." http://tkf.github.io/2013/06/04/Emacs-is-dead.html
[McCarthy1960](1, 2) McCarthy, John. 1960. "Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I." Massachusetts Institute of Technology.
[McCarthy1962]McCarthy, John et al. 1962 "LISP 1.5 Programmer's Manual." The M.I.T. Press, Massachusetts Institute of Technology.
[Mirage2015]"A programming framework for building type-safe, modular systems." http://openmirage.org
[Nadeau2010]Nadeau, Eric and Whorton, S. 2010. "FPGA-Based Graphics Acceleration. A Major Qualifying Project Report." Worcester Polytechnic Institute. https://www.wpi.edu/Pubs/E-project/Available/E-project-122210-113417/unrestricted/full_report.pdf
[Novena2015]"Novena." https://www.crowdsupply.com/sutajio-kosagi/novena
[Pike]Pike, Robert et al. "Plan 9 from Bell Labs." http://doc.cat-v.org/plan_9
[PrivEsc2015]"Wikipedia: Examples of vertical privilege escalation." 2015. http://en.wikipedia.org/wiki/Privilege_escalation#Examples_of_vertical_privilege_escalation
[Python2015]Python Software Foundation. 2015. "Python." https://www.python.org
[R6RS]"The Revised6 Report on the Algorithmic Language Scheme." http://www.r6rs.org
[RISCV2015]"The RISC-V Instruction Set Architecture." http://riscv.org
[Rust2015]"Rust Programming Language." http://www.rust-lang.org
[Schineis1984]Schineis, Rudolf and Braun, O.-M. 1984. "C 64: ROM-RAM-I/O Assemblerlisting; BASIC u. Operatingsystem; ausführl. dokumentiert mit Crossreferenzlisten." Bayreuth: Hard + Soft R.S. Microcomputer.
[Schneider2000]Schneider, Fred, Morrissett G. and Harper R. 2000. "A Language-Based Approach to Security." In Informatics: 10 Years Back, 10 Years Ahead. Harvard University. http://www.cs.cmu.edu/~rwh/papers/langsec/dagstuhl.pdf
[SECConsult2015]SEC Consult. 2015. "KCodes NetUSB: How a Small Taiwanese Software Company Can Impact the Security of Millions of Devices Worldwide." http://blog.sec-consult.com/2015/05/kcodes-netusb-how-small-taiwanese.html
[Symbolics1984]Symbolics. 1984. "Genera Concepts." http://www.ugcs.caltech.edu/~weel/lispm/genera-concepts.pdf
[Wirth1988]Wirth, Niklaus. 1988. "The Programming Language Oberon." In Software - Practice and Experience vol. 18, issue 7. John Wiley & Sons Ltd. 671–690.