Luce Base
LuceSourceLuciaOS
Pages
On this page

12. Memory

12.1 Allocation is written, never hidden#

Only one operation allocates from an allocator, new, with memory.allocate asking the current allocator for raw bytes; and one call takes storage from the running function's own frame, memory.frame (§12.7). No other operation, expression, or built-in type allocates. There are no built-in collections; a list, map, or string builder is a library type that allocates with new like any other code. The compiler never inserts an allocation.

12.2 new and free#

let node = try new Node(value = 1, next = none)      # Node*!: one Node, initialised
let zeroed = try new Node                             # Node*!: the zero value; Node must be zeroable
let leaf = try new Expr.number(value = 2.0)           # Expr*!: an enum case
let items = try new u8[4096]                          # u8[]!: 4096 zeroed bytes
let pool = try new Node[64] ---                       # Node[]!: 64 nodes, left unwritten
let raw = try memory.allocate(size, alignment)        # u8[]!: unwritten bytes
let scratch = try new u8[size] in arena               # from a named allocator
free(node)
free(items)
free(scratch) in arena
  • new followed by a construction expression, a struct's initialiser call or an enum case, allocates storage for that value, constructs the value in it, and yields T*. new T with no arguments allocates the zero value and requires T to be zeroable (§6.1). new T[count] allocates count zeroed elements and yields T[]; count is a usize expression. To allocate a single fixed array rather than a span, parenthesise the type: new (u8[4]) yields u8[4]*.
  • new T[count] --- allocates count elements of any type and leaves them unwritten, as --- leaves a var (§6.2); reading an element before writing it is undefined (§12.6), and a diagnostic build fills the elements with the byte 0xAA (§19.4). It is the form for storage the program fills first, a buffer a read fills or a pool of structs that hold bare pointers, and the one a non-zeroable T takes.
  • memory.allocate(size, alignment) yields u8[]! of unwritten bytes from the current allocator, the raw form for code that lays memory out itself, and a diagnostic build fills them with 0xAA, as it does the bytes memory.grow adds; from another allocator, allocator.allocate(size, alignment) answers u8[]? (§12.4), as that allocator left them.
  • Counted new T[count] and new T[count] --- reject byte sizes above 2^62 as memory.exhausted before calling the allocator. Element counts are checked before multiplication; zero-sized elements consume no bytes.
  • Every new is fallible, because allocation can fail: the type is T*! or T[]!, and a fallible initialiser adds its own failures to the same result. try, catch, or else handles it as for any fallible call. Failure is memory.exhausted.
  • free(x) returns storage obtained from new or memory.allocate to its allocator. x is a T*, a T[], or a u8[]. free is a statement.
  • new and free take the current allocator (§12.3) unless followed by in expression, which names the allocator: new Node(...) in arena, new u8[n] --- in parent, free(p) in arena. The expression is a var whose type implements Allocator, or an Allocator view.

Why --- and not a second word. Zero is the default everywhere in Base (§6.1), and --- is already how a program says it will write the storage itself (§6.2); an allocation that skips the zeroing says it the same way, in the same conspicuous spelling, so one keyword covers every allocation and the unwritten ones can be searched for.

  • Storage must be freed with the allocator that provided it; §12.3 states how a structure keeps that pairing.

12.3 The current allocator and with#

import memory

var arena = try Arena.over(memory.allocator, 1 << 20)
defer arena.destroy()
with arena:
    let tokens = try new Token[capacity]     # allocated in the arena
    ...                                      # released all at once by arena.destroy()
  • memory.allocator is a thread-local var holding an Allocator view, the current allocator. new and free without in use it. memory.heap is the process's initial allocator: the C allocator when the C library is linked, the Luce runtime's heap inside a full Luce program (§18.9), and none in a --freestanding build.
  • with allocator: sets memory.allocator to allocator for the suite and restores the previous value on every exit, including return and error propagation. allocator is a var whose type implements Allocator, whose address is taken, or an Allocator view. The suite is a scope, so a defer free(x) inside it runs before the previous allocator is restored.
  • The startup shim sets memory.allocator to memory.heap before main, and thread.spawn sets it to memory.heap in the new thread (§15.3). In a --freestanding build the program assigns memory.allocator before its first new; a new or alloc with none set traps memory.unset.
  • A structure that allocates on behalf of its caller records the allocator it was built with and frees with free(...) in self.allocator, so that a caller's with block cannot change which allocator releases the structure's memory. Every standard library container does this. The linter reports a free without in whose argument did not come from a new or alloc without in in the same function.

Why a current allocator, and why in beside it. Zig passes an Allocator to every function that allocates; Odin keeps one in an implicit context passed as a hidden argument. The hidden argument changes the calling convention and would break the plain C ABI that every Base function has; the explicit parameter makes the most common operation in systems code the most verbose one. A thread-local reference keeps the convention plain and keeps new Node(...) short for the common case. The in form exists because a structure's memory must be released by the allocator that provided it, whatever the caller has current at the time, and a rule that depends on ambient state at every free is a rule that programs will get wrong; with in, the pairing is written where the structure is defined and the linter can check the rest.

12.4 Implementing an allocator#

pub interface Allocator:
    func allocate(size: usize, alignment: usize) -> u8[]?
    func resize(block: u8[], size: usize) -> bool
    func release(block: u8[])

A type that implements Allocator can be named by in and made current with with. Its implementations change their receiver, since an allocator has state, so it is named and made current as a var, through a mutable pointer, or through a view. allocate answers uninitialised memory of at least size bytes at the given alignment, or none. Zero alignment means one; for a nonempty block an allocator must satisfy the requested alignment or refuse it. FixedBuffer and Arena support arbitrary positive byte alignments; the C heap supports powers of two and the page allocator supports divisors of the page size. resize grows or shrinks a block in place and answers whether it could. release returns a block; an allocator that needs the block's alignment or size class recovers it from its own bookkeeping, as free in C does, so free never has to carry it.

The standard library provides PageAllocator (host pages), CAllocator (malloc and free), FixedBuffer (over a caller's u8[]), and Arena (bump allocation over a parent allocator, released all at once by reset or destroy; §24.3 shows its source). CAllocator and PageAllocator may be used from several threads at once; FixedBuffer and Arena may not. In a diagnostic build each of them quarantines released blocks, fills them with a pattern, and records allocation sites.

12.5 Ownership conventions#

There is no ownership syntax. A pointer parameter is borrowed for the call unless the documentation says the callee takes it; a pointer result is owned by the caller unless the documentation says it is borrowed. Use defer and errdefer to pair a free with its new. The linter reports a new whose result has no free, defer free, or errdefer free in scope and is not returned or stored.

12.6 The safety statement#

These two lists are exhaustive for the language. A library may add checks; nothing removes one.

Defined and checked in every build: integer overflow in +, -, *, //, % (trap); shift by the operand width or more (trap); division by zero and minimum_signed // -1 (trap); indexing and slicing of arrays, spans, and str (trap); unwrapping every optional (trap through else trap, or a compile error without it); dereference of a bare pointer (cannot be null by type, with the one boundary check of §17.1); reading an uninitialised local (compile error, except after ---); non-exhaustive match (compile error); converting an integer to an integer-backed enum with T(n) (trap); checked conversions T(x) (trap); float-to-integer casts (saturate); reads and writes through volatile (never elided or merged); every atomic operation; aliasing of compatible objects (no type-based aliasing rule, except where noalias is written); signed right shift (arithmetic); << past the width (discards).

Undefined, exactly as in C, and the programmer's responsibility: use after free and double free; freeing storage with an allocator other than the one that provided it; dereferencing a dangling pointer, including one the escape rule of §6.6 could not see; pointer arithmetic that leaves the object, including p[i] out of range; misaligned access after a (T*) cast; reading storage declared with ---, or obtained from alloc or memory.frame, before writing it; reading a union member with an invariant after another member was written; modifying a let or const object through a cast; (str)bytes on bytes that are not UTF-8; a noalias parameter that aliases; a data race on memory that is not @T; a longjmp through a Base frame; an asm block that violates its declared operands or options; a C callee that retains a lent pointer past the call; a C caller that violates a contract stated in an extern declaration.

Why two lists. Two exhaustive lists can be checked against the C standard's own catalogue of undefined behaviour, and every future rule has to add itself to one of them.

12.7 Storage in the frame: memory.frame#

import memory

func sum_of_squares(values: const f64[]) -> f64:
    # up to 64 elements, sized to the call: no allocator, freed when the function returns
    let squares = memory.frame[f64](values.length, 64)
    for i in 0..<values.length:
        squares[i] = values[i] * values[i]
    ...
  • memory.frame[T](count, most) yields count uninitialised elements of T as a T[] in the frame of the function that calls it, as C's alloca does: the storage lasts until that function returns, and nothing frees it. count is a usize expression; most is a constant expression, the most elements the call may take, and most * memory.size_of(T) is at most 4096 bytes. A count above most traps.
  • The span is the address of a local (§6.6): it may be passed down and not returned or stored beyond the call.
  • Each call takes more of the frame, even in a loop; the stack pointer moves down by the bytes taken, rounded to sixteen, and the new bottom is touched, so the guard page that catches stack exhaustion (§11.5) is never stepped over.
  • A function that calls memory.frame is never expanded into its callers, so the storage belongs to the frame the source names. Reading an element before writing it is undefined (§12.6), as for alloc; a diagnostic build fills the elements with the byte 0xAA (§19.4).

Why bounded. A frame whose size depends on the input is what an interpreter, a recursive parser, or a small temporary needs: the fixed array it would otherwise declare reserves the most for every call, and a recursion pays it at every level. Bounding each call by a constant keeps a frame's growth visible in the source, lets the backend touch each new page in order, and keeps the reason C's unbounded alloca and VLAs were left out (§20): a size the program never stated cannot overrun the stack unseen.