The Goose Standard Library — Design
This document records the original standard library design: its
scope, conventions, and the builtins and language rules it required
(§8). It preserves the reasoning at the time. Later changes include
const types, namespaces, generic aliases, serialization,
graphics, physics, user interfaces, directories and atomic replacement
in os, and the base64, csv,
json and regex modules; some rules and
limitations below have therefore been superseded. See
../stdlib.md for the current API and
../goose_spec.md for current language rules. References of
the form §N refer to that specification.
0. Summary
The original standard library comprised five Goose source files
(std, math, dictionary,
vec, os) with about 120 functions in total. It
introduced three families of named types:
dictionary<K, V> (the one facility the language has
no building block for), rng (one word of PRNG state), and
the vec2/3/4<T> math vectors that §6.1 already
promises. Everything else is a function over the user's own arrays,
structs and pools.
Everything is written in Goose except:
- the builtins of the
print/str/formatfamily,default<T>(),abort(msg),exit(code), andcopy(x)from the passing rule of §8.7; and - the
extern fndeclaration form (spec §7.10), through which themathandosmodules reach libm and a thin C runtime layer without a builtin per function.
The Goose library code uses no heap allocator: every temporary is a local resizable or a limited array, every result is constructed in its destination (§4.3), and every higher-order function compiles to a loop (§7.6).
1. Principles
Functions over types. Goose's strength is the array
family: eight kinds of array, pools, relative references, variable-mode
ADTs, all named and laid out by the user. The library should operate on
these user-defined containers rather than require wrappers of its own.
Concretely: a library function takes a slice (T[:]) or a
reference to whatever array the caller has, never a library-defined
wrapper, and returns either a scalar, a reference into the input, or a
fresh array built at the caller's destination. New nominal types are
introduced only where the language has no representation to build on (a
hash table) or the spec already asks for them (vectors).
Small. One well-chosen function per job, chosen so
that the obvious call covers the common cases; no
_if/_by_key/_reverse/_indices
families. The test applied to every candidate: is the idiomatic
hand-written loop more than three lines, or is it easy to get
subtly wrong (sorting, hashing, UTF-8, float printing, binary search)?
If neither, it is not in the library. §9 lists what was left out and the
one-line loop that replaces it.
Goose first, builtins by exception. A builtin is
justified only when it provides an operation the language cannot
express: custom typechecking (print of any type,
default<T>()), a diverging call the checker must know
about (abort), or a way out to the OS. Speed alone is not a
reason yet — the library's Goose code is expected to compile to the same
loops a C programmer would write — but where measurement later shows a
library function would gain from custom codegen (a word-at-a-time string
hash, memchr for byte search), that function moves into the
builtin table without changing its signature. The extern fn
mechanism (§8.4) keeps that door open without growing the
typechecker.
Make costs explicit. No hidden allocation, no hidden copies. The conventions in §2 are chosen so that the cheap call is the natural one to write and the expensive one cannot be written by accident.
Lobster's shape, Goose's cost model. From Lobster's
std.lobster we take the function-heavy surface, the
it blocks,
find/filter/fold/map,
insert_ordered-style pragmatism; we do not take the
allocate-a-new-vector-per-call semantics (results are built in place) or
the do-everything builtins (tokenize,
concat_string), which become small composable pieces. From
C++ and Rust we take the coverage list — sort/stable
sort/binary search/heap/dedup/retain/hash map/string search — and reject
the iterator protocols, trait bounds and wrapper types that carry it
there; Goose has T? references, multiple returns and
return from instead of Option,
Result and exceptions.
2. Conventions
Call-site spellings follow the passing rule of spec §4.1, whose
design §8.7 records: no & at a reference-typed
destination.
2.1 How containers are passed
| The function… | Parameter | Call sites | Works on |
|---|---|---|---|
reads, or mutates elements in place without changing the length
(sort, reverse, fill,
find, sum, …) |
xs: T[:] |
f(arr), f(arr[1..]),
f(slice) |
every array kind and slices |
changes the length (push_n, insert_at,
remove_at, retain, heap_push,
…) |
xs: A& — A a generic bound to the
whole array type |
f(arr) |
every array kind that has the operations used |
takes a library type (dictionary,
rng) |
d: dictionary<K, V>& |
d.insert(k, v) |
— |
Rationale, and what was ruled out:
- Arrays coerce to slice parameters copy-free (§3.10), so
sort(arr)costs nothing and reads naturally; writes through the slice are legal when the array is writable (§9.5), so in-place algorithms need no second form. Aletarray passed tosortis a compile error carrying the call chain, which is the intended behaviour. - A generic reference parameter
xs: A&bindsAto the whole array type and instantiates per kind: onepush_n<A, T>(xs: A&, v: T, n: i64)body serves a[>..<], a[>..], a[..k]field. It cannot accept a slice, so it is reserved for length-changing operations, where a slice would be meaningless anyway. - An untyped parameter (
fn f(xs)) binds the argument's exact type, which includes a reference type forf(&arr). By §8.7's rule a non-fixed lvalue argument reaches an untyped parameter by reference and a fixed one by value, sofn total(xs)serves every kind; the library still prefersT[:]for read-only functions because it names the element type (find -> T?,contains(xs, v: T)) and accepts sub-ranges. - Grow-shrink arrays
[>..<]reach the first row like any other array now that they can be sliced (§8.7); a shrink while such a slice is in scope is the checker's error, not the library's concern. - Nothing non-fixed is ever taken by value: the language enforces it
(spec §4.1), so a library function that wants its own copy says
copy(x).
2.2 Results
- A fresh array (
filter,map,split,concat,str) is returned asT[>..]: a resizable local built by NRVO straight into the caller's destination (§7.3), solet ys = xs.filter() { it > 0 },var ys = xs.filter() { … }(and keep pushing),out.append(xs.map() { … })andwords.push(str("w", i))all cost the elements and nothing more.T[>..]rather thanT[]because it has no length-field limit and the caller most often wants to keep growing or sorting it; the "fresh stack" it needs is a stack index for the value's lifetime, not a runtime cost. AT[]destination converts. - "Maybe an element" (
find,get,last) is aT?— an optional reference into the input, rooted where the input is rooted, null for "none". The caller narrows withif/guard(§3.8) and can write through it. There is noOption<T>value type and the library does not invent one. - A position (
find_index,index_of,min_by,lower_bound) is ani64, with-1for "none" where that can happen. - Two-outcome scalars (
parse_int,binary_search,read_line) use the spec's convention of a trailingboolin a multiple return:let v, ok = parse_int(s);. There is no error type; programming errorsabort. - Text output goes into a builder passed in
(
format(out, …),read_file(path, out)): the result appends transparently to whatever the caller is assembling. The one exception isstr(…), whose whole point is to be the fresh string.
2.3 Function-value parameters
- Function values are trailing blocks (§7.6):
xs.find() { it.key == k },xs.sort() { a, b => a.age < b.age }, or a named function. - A comparator is always a strict "a goes before b" predicate
lt(a, b). Sorting, heap and extremum functions come in two arities: with a block, and without one meaninga < b(sosort(xs)works for numbers andsort(xs) { a, b => compare(a, b) < 0 }for strings). Overload resolution separates the two by whether a function value is supplied, which was verified. - A key function returns a value
(
min_by(xs) { it.score }). - A block parameter binds a reference when the
library passes it
&x, so blocks can mutate:d.each() { k, v => v += 1; }works becauseeachcallsF(s.key, &s.val). Functions document which arguments they pass by reference. - Inside a block,
returnreturns from the enclosing named function (§7.6), so early exits are available to callers ofeach_*functions for free.
2.4 Trait-like overload sets
Goose's overloading-by-type is its trait mechanism. The library defines a few names that user code is expected to extend by adding overloads for its own types:
| Set | Provided for | Users add | Consumed by |
|---|---|---|---|
hash(x) -> u64 |
every scalar type, bool, u8[:] |
fn hash(k: mykey) -> u64 |
dictionary |
compare(a, b) -> i64 |
u8[:] |
fn compare(a: ver, b: ver) -> i64 |
sorting strings |
format(out: u8[>..]&, x) |
every type, structurally, by the builtin (§8.1) | fn format(out: u8[>..]&, p: point) |
print/str/format, which use a
user overload for its type wherever that type occurs |
Two resolution facts (§7.1) shape how these are written:
- A concrete overload beats a generic one (
hash(k: mykey)wins overhash<T>(x: T)), so user overloads take precedence automatically. - Every integer width needs its own overload: a
u8argument widens equally well intoi64andu64, and two coercion-tier matches are an ambiguity error. Sohashis ten one-line functions, not one generic.
2.5 UFCS
x.f(args) is f(x, args) (§7.1). By the
passing rule (§8.7) x.f() binds x by reference
exactly when f's first parameter is a reference type, for
the same reason a free call f(x) does, so
d.insert(k, v), q.heap_push(v) and
r.rand_int(6) all bind their receivers as their signatures
specify. An earlier idea, a per-function marker for by-reference
receivers, was withdrawn because it would have taught callers to drop
& everywhere, which brings back the accidental copy the
moment someone forgets to declare the marker, and because it made the
call form depend on knowing how each function was declared.
A reference-returning call is not an assignable location
(get_or_insert(d, k, 0) += 1 is rejected); binding it needs
the annotated form let c: i64& = d.get_or_insert(k, 0);
(§3.8). The dictionary API sidesteps this with the
block-taking update (§5.2).
2.6 Naming
- Library types are lowercase, like the builtin types:
dictionary<K, V>,rng,vec3<T>,float3. Heavy types get long names. each_*: iteration higher-order functions (each_rev,each_split,each_utf8,each_chunk).*_by: the key-function variant of a value function (min/min_by).*_at: index-addressed array mutation (insert_at,remove_at), keeping the bare verbs (insert,remove) fordictionary. Two generic functions of the same name and arity whose first parameters areA&anddictionary<K, V>&would be ambiguous (both bind at the generic tier), so the array and dictionary vocabularies are kept disjoint by construction.format_*: append a textual form to au8[>..]builder with formatting control;parse_*: the inverse;to_*: in-place case conversion;write_*: I/O, and only I/O.- No
is_prefix (any,all,contains,starts_with). - There is no namespacing in v1 (§11.1), so every library name is
global. Names were chosen to be the ones a user would also pick, on the
theory that a user who defines their own
findwith a concrete signature gets theirs (exact match beats generic), and a user who defines a generic one gets a clear duplicate or ambiguity error rather than silent shadowing.
2.7 Strings
There is no string type (§3.7); the library's string functions are
functions on u8 arrays with these roles:
| Role | Type | Notes |
|---|---|---|
| input | u8[:] |
literals, any u8 array and any slice coerce |
| output being built | u8[>..]& |
the "builder";
push/append/format are the
primitives |
| owned, stored | u8[], u8[varint],
u8[..k] |
fields and elements; a u8[>..] result lands in
them |
All functions are byte-oriented and ASCII by default
(to_lower, trim);
each_utf8/push_utf8 are the only
encoding-aware pieces. Comparison is == (structural, §4.5)
and compare. A character literal is an integer constant
(§2): format(out, 'x') appends 120, and a byte
goes in with out.push('x') or
format(out, "x").
3. Module map
| File | Imports | Contents | Leans on (§8) |
|---|---|---|---|
stdlib/std.goose |
— | scalars, bits, hashing, rng; array HOFs, searching,
sorting, heaps, resizing; strings, formatting control, parsing,
UTF-8 |
default<T>() for sum;
abort |
stdlib/math.goose |
— | libm as extern fn, PI/TAU,
radians/degrees,
is_nan/is_inf |
extern fn |
stdlib/dictionary.goose |
std |
dictionary<K, V> and its functions |
default<T>() |
stdlib/vec.goose |
std, math |
vec2/3/4<T>, float*,
double*, int*, vector functions |
— |
stdlib/os.goose |
std |
files, stdio, arguments, environment, time, sleep, entropy | extern fn, runtime C in
src/runtime/runtime_os.h |
import std; is explicit; nothing is auto-imported. The
compiler resolves import std; by looking, in order, in the
importing file's package root (spec §11.1: the main file's directory for
the program's own files, the library's directory for a library module),
in --stdlib <dir> or GOOSE_STDLIB if
given, and then by walking up from its own executable's directory
looking for stdlib/std.goose (three levels), so
build/Debug/goose.exe, build/goose,
bin/goose.exe and an installed
<prefix>/bin/goose beside
<prefix>/stdlib all find it without requiring a
single installation path. Unused library functions cost parse time only:
functions are typechecked in call-graph order from main, so
an unreferenced generic never instantiates.
4. std
Signatures are written in documentation form. Where a function's
return type is the element type of an A& parameter, the
source omits the annotation (it is inferred from the body, §7.1) and the
documentation writes -> T with "T = the element type".
xs: T[:] accepts every sliceable array (§2.1).
4.1 Scalars and bits
fn min<T>(a: T, b: T) -> T
fn max<T>(a: T, b: T) -> T
fn clamp<T>(x: T, lo: T, hi: T) -> T
fn abs<T>(x: T) -> T // integers; plus abs(f32) and abs(f64) overloads
fn rem<T>(a: T, b: T) -> T // integers; pairs with `/`
fn div_euclid<T>(a: T, b: T) -> T // integers; pairs with `%`
fn sign<T>(x: T) -> T // -1, 0, 1 at x's type; f32/f64 overloads
fn lerp(a: f64, b: f64, t: f64) -> f64 // and an f32 overload
fn next_pow2(x: i64) -> i64 // smallest power of two >= max(x, 1)
fn popcount(x: u64) -> i64
fn clz(x: u64) -> i64 // 64 for x == 0
fn ctz(x: u64) -> i64 // 64 for x == 0
fn swap<T>(a: T&, b: T&)rem and div_euclid supply the missing
partners of truncating / and Euclidean %
without changing either operator. They compute at the operands' integer
type, using the truncating quotient and its remainder, whose product and
subtraction fit whenever the quotient does. Forming
a - a % b instead can underflow. rem handles
divisor -1 before division, since its zero result fits even when the
quotient does not. The test for a negative value with all bits set
accepts unsigned types too and requires integer operands.
div_euclid uses / for its initial quotient,
preserving its overflow checks.
abs and sign have explicit float overloads.
popcount/clz/ctz use the
runtime's extern intrinsics (§8.4).
4.2 Hashing
fn hash(x: i8) -> u64 // … one overload per integer type, i8 … u64
fn hash(x: bool) -> u64
fn hash(x: f32) -> u64
fn hash(x: f64) -> u64
fn hash(s: u8[:]) -> u64 // 8 bytes per multiply; any u8 array coerces
fn hash_combine(seed: u64, h: u64) -> u64 // for composite keysThe integer hashes are a multiply-xorshift mix (Fibonacci hashing),
so the low bits are well distributed and dictionary can
mask rather than divide. The byte hash reads 8 bytes at a time: a
multiply and a rotate per word, the last word being the final 8 bytes
(or, below 8 bytes, the bytes gathered into one word) with the length as
the seed, then two multiply-xorshift rounds, so that every input bit
reaches the low 32 bits that dictionary masks and keeps as
slot tags. Keys that differ only in their last characters
(item_17, item_18) would otherwise share their
low bits. Each word's round waits for the previous one's multiply, so
above 24 bytes the words before the last 24 bytes or fewer go to two
lanes, the even and the odd words of each 16 bytes, whose rounds a
processor overlaps; the second lane is folded into the first, rotated,
before the remaining words. That form is a function of its own
(hash_lanes), called from an early return, which the
optimizer keeps out of line, so that the short keys' code stays small
enough for the C compiler to inline into a dictionary's lookups. A user
key type gets
fn hash(k: key) -> u64 { hash_combine(hash(k.a), hash(k.b)) }.
Fixed-capacity inline strings (u8[..16]) hash through the
slice overload, which is what makes them usable as
dictionary keys.
4.3 Random numbers
struct rng { s: u64 } // splitmix64 state; the seed is the state
fn rand_u64(r: rng&) -> u64
fn rand_int(r: rng&, n: i64) -> i64 // uniform in [0, n)
fn rand_flt(r: rng&) -> f64 // uniform in [0, 1)
fn shuffle<T>(xs: T[:], r: rng&) // Fisher-Yatesvar r = rng { 12345 }; then r.rand_int(6).
splitmix64 is one word of state, 64-bit arithmetic only, and passes the
usual batteries; the benchmarks' xorshift stays where it is. Seeding
from the OS is os's random_seed().
4.4 Arrays: reading and searching
fn each_rev<T, F>(xs: T[:]) // F(x), last to first
fn each_chunk<T, F>(xs: T[:], n: i64) // F(chunk: T[:], i); the last chunk may be short
fn find<T, F>(xs: T[:]) -> T? // first x with F(x), as a reference; null if none
fn find_index<T, F>(xs: T[:]) -> i64 // …or its index, -1 if none
fn position<T>(xs: T[:], v: T) -> i64 // first i with xs[i] == v, -1 if none
fn contains<T>(xs: T[:], v: T) -> bool
fn any<T, F>(xs: T[:]) -> bool
fn all<T, F>(xs: T[:]) -> bool
fn count<T, F>(xs: T[:]) -> i64 // number of x with F(x)
fn min<T>(xs: T[:]) -> T // asserts xs.len > 0; F(a, b) forms too
fn max<T>(xs: T[:]) -> T
fn min_by<T, F>(xs: T[:]) -> i64 // index of the smallest F(x); -1 if empty
fn max_by<T, F>(xs: T[:]) -> i64
fn last<T>(xs: T[:]) -> T& // asserts xs.len > 0
fn lower_bound<T, F>(xs: T[:]) -> i64 // first i with !F(xs[i]); F must be true on a prefix
fn binary_search<T>(xs: T[:], v: T) -> i64, bool // sorted by <; position (or insertion point), found
fn find<T>(xs: T[:], sub: T[:]) -> i64 // first occurrence of a subsequence, -1 if none
fn rfind<T>(xs: T[:], sub: T[:]) -> i64
fn starts_with<T>(xs: T[:], p: T[:]) -> bool
fn ends_with<T>(xs: T[:], p: T[:]) -> booleach_chunk exists because row slices are the documented
way to get bounds checks out of 2-D kernels
(bench/goose/blur_rows.goose);
for y in h { each_chunk(img, w) … } is that idiom with the
arithmetic done once.
find/rfind/starts_with/ends_with
are generic over the element type, so they are the string functions too
(find(line, "://")), and they coexist with the predicate
find because a function-value argument and a slice argument
never match the same overload.
4.5 Arrays: transforming
fn map<T, F>(xs: T[:]) -> U[>..] // U = F's result type (§8.5)
fn filter<T, F>(xs: T[:]) -> T[>..]
fn fold<T, A, F>(xs: T[:], acc: A) -> A // acc = F(acc, x) for each x
fn sum<T>(xs: T[:]) -> T // fold from default<T>()
fn concat<T>(a: T[:], b: T[:]) -> T[>..]map's body is
var out = []; for x in xs { out.push(F(x)); } return out;,
the empty-array declaration of §8.5. sum needs
default<T>();
fold(xs, 0.0) { a, x => a + x } is the form that needs
nothing.
4.6 Arrays: in place
fn fill<T>(xs: T[:], v: T)
fn copy_into<T>(dst: T[:], src: T[:]) // asserts equal lengths; memmove semantics
fn reverse<T>(xs: T[:])
fn sort<T>(xs: T[:]) // by <
fn sort<T, F>(xs: T[:]) // by F(a, b); unstable, in place, no allocation
fn stable_sort<T>(xs: T[:])
fn stable_sort<T, F>(xs: T[:]) // natural merge sort; one temporary of xs.len elements
fn to_lower(s: u8[:]) // ASCII, in place
fn to_upper(s: u8[:])sort is a pattern-defeating quicksort after pdqsort and
Rust's ipnsort. A first pass stops at the end of the run the input
starts with, and the sort is done if that is all of it. Otherwise each
round takes a median pivot (of three elements, or of three medians of
three from 128 on), partitions with a branchless Lomuto loop (every
element is swapped, and the boundary advances by the comparison's
outcome), pushes the larger side on an i64[..144] stack of
(lo, hi, budget) ranges and goes on with the smaller, so it recurses
nowhere, allocates nothing, and holds at most log₂(2⁴⁸) ranges.
Everything left of a range goes before or with everything in it, so when
the pivot does not go after the element just before the range, the
elements equal to it are moved to the front in one pass and dropped, as
pdqsort does: many equal keys cost linear passes. Each range carries a
budget of 2·log₂(n) rounds, after which heapsort finishes it, bounding
the worst case at O(n log n); test/stdlib/stdlib_sort.goose
builds an input against the pivot choice with McIlroy's adversary to
check that. Ten elements or fewer are insertion sorted: below that, the
branchy insertion sort measured faster than more partitioning. One shape
is slower than with the Hoare partition sort had before:
input in order but for a few misplaced elements. The branchless
partition swaps every element, which rotates each side of a sorted range
by one, so nothing below the first run check sees order to exploit, and
such input costs about two thirds of random input, where the old
quicksort's well-predicted scans took half the new time (200k elements:
1.6 ms against 0.8 ms; random input 2.5 ms against 8.3 ms).
stable_sort is a natural merge sort. One pass splits the
input into runs in order, reversing strictly descending ones and
extending those shorter than 16 by insertion sort; an input that is one
run is done there. The runs' ends go in a i64[>..], and
passes merge neighboring pairs between xs and a
T[>..] temporary on a fresh data stack (a stack index,
not an allocation), alternating direction so that no pass copies back. A
merge copies runs already in order. Its step either branches on the
comparison or selects the element and the side with conditional moves;
it branches while the previous merge switched between its runs on fewer
than one step in four, since a branch on random input mispredicts about
every other step, while on structured input a predicted branch lets the
comparisons overlap, which matters most for a comparator that loads what
it compares (BWT's rank pairs: 3.7x between the two forms on its
unsorted rounds, against 1.5x the other way on random integers).
Both leave a permutation of the input whatever the comparator returns: every index stays in range however the comparisons come out, so a comparator that is no strict weak ordering only leaves an unspecified order.
4.7 Arrays: changing the length
fn push_n<A, T>(xs: A&, v: T, n: i64) // n pushes; grow-only arrays' answer to resize
fn insert_at<A, T>(xs: A&, i: i64, v: T) // shifts [i..) up; needs push
fn remove_at<A>(xs: A&, i: i64) -> T // shifts down; needs pop; returns the element
fn swap_remove<A>(xs: A&, i: i64) -> T // O(1), reorders
fn retain<A, F>(xs: A&) // keeps x with F(x), stable; needs resize
fn dedup<A>(xs: A&) // drops adjacent duplicates (sort first for all)
fn heap_push<A, T>(xs: A&, v: T) // min-heap by <; F(a, b) forms of all three exist
fn heap_pop<A>(xs: A&) -> T
fn make_heap<A>(xs: A&) // O(n) heapifyEach instantiates only for array kinds that have the operations it
uses (a grow-only array pops through the reference wherever nothing in
the caller points into it, spec §5.1). A [>..<] is
the expected heap and queue container; a limited [..k]
inside a struct works too.
4.8 Strings
fn format_int(out: u8[>..]&, v: i64, base: i64, width: i64, pad: u8) // base 2..36, right-aligned
fn format_uint(out: u8[>..]&, v: u64, base: i64, width: i64, pad: u8)
fn format_flt(out: u8[>..]&, v: f64, decimals: i64) // fixed decimals
fn parse_int(s: u8[:]) -> i64, bool // optional sign, base 10, whole string
fn parse_int(s: u8[:], base: i64) -> i64, bool
fn parse_flt(s: u8[:]) -> f64, bool // sign, fraction, exponent
fn compare(a: u8[:], b: u8[:]) -> i64 // bytewise lexicographic: -1, 0, 1
fn trim(s: u8[:]) -> u8[:] // ASCII whitespace; a sub-slice, no copy
fn trim_start(s: u8[:]) -> u8[:]
fn trim_end(s: u8[:]) -> u8[:]
fn split(s: u8[:], sep: u8) -> u8[:][>..] // slices into s; empty parts kept
fn split(s: u8[:], sep: u8[:]) -> u8[:][>..] // a separator of several bytes
fn each_split<F>(s: u8[:], sep: u8) // F(part: u8[:]); no array built
fn each_split<F>(s: u8[:], sep: u8[:])
fn words(s: u8[:]) -> u8[:][>..] // the runs between whitespace; none empty
fn each_word<F>(s: u8[:]) // F(word: u8[:])
fn join<T>(out: u8[>..]&, parts: T[:], sep: u8[:]) // T any u8 array/slice type
fn format_replaced(out: u8[>..]&, s: u8[:], old: u8[:], with: u8[:]) // `from` is a keyword
fn each_utf8<F>(s: u8[:]) // F(cp: i64); malformed bytes yield 0xFFFD
fn push_utf8(out: u8[>..]&, cp: i64) // the UTF-8 bytes of a code pointPlain rendering of scalars and strings is the builtin
format (§8.1); the functions here add formatting control
and parsing. split returning slices into its input (no
copies, elements rooted at the caller's text) is the Goose replacement
for Lobster's tokenize; the each_split form
does not even build the array. words is the whitespace
tokenizer a command line wants, which split at
' ' is not: it drops the empty parts repeated spaces and
tabs would leave.
5. dictionary
5.1 Representation
struct dictionary_slot<K, V> { key: K, val: V, tag: u32 }
struct dictionary<K, V> { count: i64 = 0, slots: dictionary_slot<K, V>[>..<] = [] }- Open addressing, linear probing, power-of-two capacity,
hash(key) & mask.KandVmust be fixed-size (the slot array is indexed randomly, §3.3); strings are keyed asu8[:]slices into text the caller owns, or as inlineu8[..k]. Deletion is backward-shift (the techniquebench/goose/lru.gooseuses), so there are no tombstones and no periodic cleaning. - A
dictionaryis resizable-class (its tail is a[>..<]), so it lives where resizables live: a local, a global, a by-reference parameter, the tail of a struct. It is spec A.3, generalized. - Each slot keeps a tag: 0 when it is unused, else the low 31 bits of
the key's hash with bit 31 set. A probe compares keys only where the
tags agree (a slice key's length and bytes, an inline key's bytes), and
for a table of up to 2^31 slots the tag holds the home slot, so growth
and removal never hash a key again; a larger table hashes the keys it
moves. The tag costs 3 bytes per slot over a
usedbyte (12 bytes for au32 → i32table where 9 would do): n-gram counting withu32keys, which compare as cheaply as tags, measured between 3% slower and even, while string-keyed tables (word counts, an LRU cache, an interpreter's variables, a template engine's) gained 18-57% from the dictionary changes alone. - Growth doubles at ⅔ load, in place: the slot array tops its stack,
so
resizeextends it with unused slots, and the entries are re-placed by one walk over the old slots in cyclic order, starting just after an unused one. Every entry then lands within a run of slots the walk has already passed (its new home is its old home or that plus a multiple of the old size), so no entry still to be moved lies on a moved entry's probe path. No second table is built and nothing is copied back. The slot array is grow-shrink so that it can be resized through a reference (§5.2). A maximum load of ½ measured 16% faster on a small LRU table and 9% on n-gram counting, but 8% slower on growth-heavy word counting and up to 1.8 times slower on 150K-entry tables, which at twice the size leave the cache; ¾ was within noise of ⅔ but for the word count, 8% slower. insert,updateandget_or_insertcheck for growth first and then walk the probe sequence once, to the key or to the unused slot where it goes.- Fresh slots are filled with
default<K>()/default<V>()(§8.2), so a reference value is an optional one (Cell?, orin pool). An update stores a whole slot, since=into a referenceVwould write its pointee (spec §3.8).
5.2 API
fn get<K, V>(d: dictionary<K, V>&, key: K) -> V? // reference to the value, or null
fn contains<K, V>(d: dictionary<K, V>&, key: K) -> bool
fn insert<K, V>(d: dictionary<K, V>&, key: K, val: V) -> bool // overwrites; true if the key was new
fn get_or_insert<K, V>(d: dictionary<K, V>&, key: K, v0: V) -> V&
fn update<K, V, F>(d: dictionary<K, V>&, key: K, v0: V) // F(val&): inserts v0 if absent, then calls F
fn remove<K, V>(d: dictionary<K, V>&, key: K) -> bool
fn clear<K, V>(d: dictionary<K, V>&)
fn reserve<K, V>(d: dictionary<K, V>&, n: i64) // capacity for n entries without rehash
fn each<K, V, F>(d: dictionary<K, V>&) // F(key, val&); order unspecifiedWord counting, the canonical use (prototyped end to end):
var counts = dictionary<u8[:], i32> {};
each_split(text, ' ') { counts.update(it, 0) { it += 1; } };
counts.each() { w, n => if n > 100 { print(w, " ", n); } };update exists because the natural spelling,
get_or_insert(d, k, 0) += 1, is not an assignable location
and the annotated binding needs the value type spelled out (§2.5). A
block gets the reference without either problem.
A reference returned by get or
get_or_insert, or an each binder, may not be
in scope at the next insert, update,
remove or clear: the slot array is
grow-shrink, so the checker reports it (spec §5.2) rather than letting a
rehash or a backward shift leave the reference on a different,
well-typed slot. Bind such references in their own block, or use
update.
5.3 Sets and other idioms
- A set is
dictionary<K, bool>:s.insert(k, true),s.contains(k). A dedicatedset<K>needs generic type aliases (§8.7) and a zero-size value type, neither of which exists yet; the byte per slot is not worth a second implementation. - Owned string keys: keep a
u8[>..]text pool next to thedictionary<u8[:], V>(two locals of one scope, or a global pool); keys are slices into the pool. A struct cannot hold both (one resizable per struct, §3.4), which is spec TODO 8. An interner built on this idiom is a candidate for a later module. - Insertion order:
dictionarydoes not keep it. Where order matters, store values in the user's own[>..]and keep indices in the dictionary, which is whatlru.goosedoes.
6. vec
struct vec2<T> { x: T, y: T }
struct vec3<T> { x: T, y: T, z: T }
struct vec4<T> { x: T, y: T, z: T, w: T }
type float2 = vec2<f32>; type float3 = vec3<f32>; type float4 = vec4<f32>;
type double2 = vec2<f64>; type double3 = vec3<f64>; type double4 = vec4<f64>;
type int2 = vec2<i32>; type int3 = vec3<i32>; type int4 = vec4<i32>;
fn dot<T>(a: vec2<T>, b: vec2<T>) -> T // and vec3, vec4
fn cross<T>(a: vec3<T>, b: vec3<T>) -> vec3<T>
fn length_sq<T>(v: vec3<T>) -> T // all three sizes
fn length(v: float3) -> f32 // f32 and f64 instances, all sizes
fn normalize(v: float3) -> float3
fn distance(a: float3, b: float3) -> f32
fn lerp<T>(a: vec3<T>, b: vec3<T>, t: T) -> vec3<T>
fn xy<T>(v: vec3<T>) -> vec2<T> // also xy(vec4), xyz(vec4)
let float3_0 = float3 { 0.0, 0.0, 0.0 }; // _0 and _1 for all nine aliases
let float3_1 = float3 { 1.0, 1.0, 1.0 };Elementwise + - * / are the language's (§6.1); the
prototype confirmed a type float3 = vec3<f32> alias
constructs positionally, adds memberwise and instantiates generic
dot for f32 and i64.
int* are i32 because vectors are data (§3.1)
and an i32 component widens into an i64 index
implicitly. Elementwise
min/max/abs are deliberately
absent: a min<T>(a: vec3<T>, b: vec3<T>)
overload is ambiguous with the scalar
min<T>(a: T, b: T) (both generic tier), and
min(a.x, b.x) per component is what a user writes anyway.
The lengthening constructors (vec3(v, z)) were dropped:
vec3 { v.x, v.y, z } is as short and avoids a function
named like a type.
7. math and
os (extern-backed)
math is libm, declared once per function and float
width:
extern fn sqrt(x: f64) -> f64; extern "sqrtf" fn sqrt(x: f32) -> f32;
// sin cos tan asin acos atan atan2 exp log log2 log10 pow floor ceil round trunc, likewise
let PI = 3.141592653589793; let TAU = 6.283185307179586;
fn radians(deg: f64) -> f64
fn degrees(rad: f64) -> f64
fn is_nan(x: f64) -> bool // x != x
fn is_inf(x: f64) -> boolfloor/ceil/round/trunc
return floats, as in C; as i64 converts. Float
% is already fmod (§6.1).
os is a thin, deliberately incomplete layer — enough to
write tools and benchmarks — over C functions in
src/runtime/runtime_os.h:
fn read_file(path: u8[:], out: u8[>..]&) -> bool // appends the whole file
fn write_file(path: u8[:], data: u8[:]) -> bool
fn append_file(path: u8[:], data: u8[:]) -> bool
fn file_exists(path: u8[:]) -> bool
fn delete_file(path: u8[:]) -> bool
fn read_line(out: u8[>..]&) -> bool // stdin; strips the newline; false at EOF
fn read_stdin(out: u8[>..]&) // everything until EOF
fn write_stdout(s: u8[:])
fn write_stderr(s: u8[:])
fn flush_stdout()
fn args() -> u8[:][>..] // slices of a global buffer, over extern arg_count()/arg(i, out)
fn env(name: u8[:], out: u8[>..]&) -> bool
fn time() -> f64 // seconds since the epoch
fn clock() -> f64 // monotonic, high resolution
fn time_ns() -> i64 fn clock_ns() -> i64
fn sleep(seconds: f64) fn sleep_ms(ms: i64)
fn random_seed() -> u64 // OS entropy, for rngDirectory listing, subprocesses, sockets and date formatting are not
in v1. exit(code) is a builtin rather than an extern
because the checker must know it diverges (§8.3).
8. Builtins and language rules the library needed
All in the spec now; ordered by how much of the library depends on them.
8.1 The print /
str / format family
One rendering engine behind three builtins, each taking any number of arguments of any type, rendered in sequence with no separator:
print(a, b, …) // stdout, newline at the end
str(a, b, …) -> u8[>..] // a fresh string, constructed at the destination
format(out, a, b, …) // append to any growable u8 array; out.format(a, b, …) via UFCSstr is the answer to "quickly concatenate a few things
into a new string", which Goose does in place and should encourage:
words.push(str( "item", i)) writes straight into the
element slot. Rendering per type:
- integers, floats (shortest round-tripping form),
bool. - a
u8array or slice: raw bytes at the top level (concatenation is the common case), quoted and escaped when nested inside an aggregate. - other arrays and slices:
[1, 2, 3]. - structs and ADT variants: the type name and the positional literal
form,
float3 { 1, 2, 3 },Circle { 1.5 }, no field labels. The name keeps nested values and variants readable, and the output stays valid Goose. - references: their pointee; optionals:
nullor the pointee; relative references likewise. - a type with a user overload
fn format(out: u8[>..]&, v: T)renders through it instead, wherever it occurs — the builtin is the structural fallback of theformatoverload set, and user overloads are found by the same resolution as any call.
Code generation specializes rendering for each type, using a traversal like the one for equality. This follows Lobster's approach to printing values for debugging (spec §3.7).
8.2 default<T>()
The value a fixed-size T has before anything is written
to it, with declared defaults honoured: numbers 0, false,
empty arrays (length 0, capacity as declared), null optionals, an empty
slice, variant 0 with its own default payload, and structs field by
field — a field with a declared default (§3.2) gets it, every other
field gets its type's default. So a struct that declares
count: i64 = 1 as an invariant keeps it when the library
fills a slot with default. Needed by sum (an
accumulator seed for generic T) and by
dictionary (fill values for fresh slots); useful anywhere
resize(n, v) wants a neutral v. It is a
builtin because a generic function has no way to write a literal of an
unknown type (§4.1), and because the field defaults are the compiler's
to know.
8.3
abort(msg: u8[:]) and exit(code: i64)
Diverging calls: code after them is unreachable, so a branch ending
in one gives no value (§6.4). abort prints
goose runtime error: <msg> and exits nonzero, like
the checked aborts of §9.3; exit flushes and exits with the
code. The library uses abort for contract violations that
are not plain asserts (an unknown base in
format_int).
8.4 extern fn (spec
§7.10)
extern fn sqrt(x: f64) -> f64;
extern "sqrtf" fn sqrt(x: f32) -> f32;
extern "gs_os_read_file" fn read_file(path: u8[:], out: u8[>..]&) -> bool;
extern fn gl_vertex3(v: float3&);- A declaration without a body; the optional string is the C symbol (default: the Goose name). It joins overload resolution like any function and is called directly from the generated C, with a prototype emitted unless the runtime defines the symbol.
- Parameter and return types: the integer and float scalars and
bool; any flat, fixed-size struct or fixed array, passed by value as its generated packed C type (gs_float3);T&to such aT, passed as a pointer (const gs_float3 *— what an OpenGL-style API wants, since a packedfloat3is three contiguous floats);T[:]of such aT, passed as pointer and count; andu8[>..]&, passed as the resizable's header plus its stack, so C code can append through a runtime helper (gs_bld_append(bld, ptr, n)) — the one parameter shape that lets the wholeoslayer be externs instead of builtins. Returns are a scalar, a flat fixed struct, or nothing. - What does not cross:
varint, references inside structs, variable and resizable values by value, slices of variable elements. The checker rejects them with the reason. - Writability is checked at the call site as for
push(§9.5). Extern functions are usable from thread programs when the C is re-entrant, which the runtime's are. - Initial implementation, with room for later extensions: the compiler
always emits its own prototype from the Goose declaration; the C for the
library's externs lives in
src/runtime/runtime_os.h, part of the runtime like the other runtime files; user externs get their C in via--include <header>, emitted after the generated type declarations so a header can implement shims against thegs_<name>typedefs. An existing C API whose declaration differs from the emitted prototype (alignment,APIENTRY, pointer types) gets a one-line shim in that header.
8.5
map and the empty-array declaration (spec §4.2)
Nothing about generics is needed: a generic body is checked only at
an instantiation where every type is concrete, including what
F(x) returns (spec §7.7). What map needs is
its result local, var out = [];, whose [] has
no element type yet: an empty-array local without an annotation
gets a pending array type that the first
push/append/assignment of a concrete element
completes; using it in a way that needs the element type before then
(out[0]) is an error, out.len is fine, and a
body that never completes it is an error at its end. Codegen sees only
completed types. So map is
fn map<T, F>(xs: T[:]) { var out = []; for x in xs { out.push(F(x)); } return out; }with the return type inferred as usual (§7.1), and the same rule
makes var names = []; idiomatic user code.
8.6 Where
import std; looks (spec §11.1)
The search path of §3, a --stdlib flag and a
GOOSE_STDLIB variable; the compiler finds the source tree's
stdlib/ from its own location, so the test runner needs no
flag.
8.7 Checker and language rules
Passing by size class (spec §4.1). The rule replaced
"explicit & on both ends": it is keyed on the size
class of the value being connected to a destination — a call argument, a
let/var initializer, an assignment, a field
initializer, a push, a for binding, a match
binder, a queue put:
- A fixed value connects by value. A reference-typed
destination (
x: T&,T?, an annotatedlet r: T&) binds the lvalue's address implicitly;&xstays legal and means the same, so forwarding a reference variable is unchanged. - A non-fixed value is never copied implicitly. A
reference-typed destination binds it; an untyped or generic destination
binds it by reference too, that being the only free option; a
value-typed destination accepts an rvalue — a call result, a literal, a
slice or other conversion, a local being returned — or an explicit
copy(x), a builtin that constructs the copy at the destination (§4.3 semantics; an explicit marker that is easy to search for). &at call sites is thereby redundant rather than required, and a redundant&— one written at a source whose reference-typed destination would have bound it anyway — is a warning, so that code in the language keeps one style rather than a scatter of leftover&.
The rule applies to individual constructs as follows:
let y = x;for a non-fixedxis an error: writelet y = &x;(a reference, the inferred type) orlet y = copy(x);. For fixedxit copies.push(v)is a construction context (§4.2): a non-fixed lvaluevneedscopy(v); a slicev[..], a literal or a call result does not.append(src)takes its source as a slice and copies elements as its defined operation, sog.append(h)stays as it is.g = hbetween resizables (§4.4's clear-then-copy) becomesg = copy(h);g = f()stays. This is where amove(x)(spec TODO 3) would first be wanted: an assignment of a dying local.return xof a non-fixed local moves it (NRVO, no copy); returning a non-fixed field or parameter copies and takescopy. The one implicit copy of §7.3 — two different locals returned on different paths — stays implicit, because the programmer cannot know which of the two will be the copied one.for x in xsandCircle c =>copy fixed elements and payloads; non-fixed ones already require the&xbinder form (§6.5, §8.1) — the rule makes that a consequence rather than a special case.- An untyped or
<T>parameter given a non-fixed lvalue bindsTto the reference type; given a fixed lvalue, to the value type. A type variable normally adopts the caller's type exactly; this is the one pragmatic exception, and it is what letsfn total(xs)serve a fixed array (copied), a[>..]and a[>..<](by reference) with one body. A body that writes throughxsis an error at the fixed instantiation (an immutable copy) rather than a silently discarded write. - Overloads
f(x: T)andf(x: T&)for the same fixedTbecome ambiguous for an lvalue argument; the pairf<T>(xs: T[:])/f<A>(xs: A&)does not (a[>..<]reaches only the second, an array reaches the second at the generic tier, a slice only the first), so the library's two container conventions coexist. - An explicit
&xargument whose destination would have boundxwithout it draws the redundancy warning. Forwarding a reference variable (f(r)withr: T&) involves no&and is silent, as is&xwhere the destination is inferred (let r = &x;), since there the&is what selects the reference type.
For the library: no binder parameters, no & in any
example, UFCS works for every mutator, generic HOFs could even be
untyped, and the library's own copies become visible where they are
intended (stable_sort's temporary is a construction).
Pros, beyond the UFCS fix that motivated it: the rule matches the
cost model (registers for fixed values, in-place construction or a
reference for everything else); non-fixed copies become
impossible to write by accident, which is stronger than the
explicit-& rule, under which f(arr) to a
by-value non-fixed parameter compiled and silently copied; one rule
covers calls, let, =, fields,
push and for alike; copy gives
move a natural sibling; and §4.1's "everything is a value"
simplifies to "fixed values are values; non-fixed values live in one
place and are copied only on request".
Cons, and what limits them:
- Mutation of a fixed value is no longer visible at the call site:
normalize(v)may writev. Accepted: the UFCS discussion had already concluded that mutable arguments would go unmarked at the call site, and it is rarer than mutating arrays. Writability provenance (§9.5) confines it tovarlocals — aletargument yields a non-writable reference, and a callee that writes through it is a compile error with the call chain — so the reader's rule is "avarpassed to a function may change", which is the C++ situation restricted to values the programmer already declared mutable. - "Fixed" is not "small". A limited array
u8[..4096]or a fixedf32[1024]is fixed-class and copies silently. Accepted, with no mitigation for now: resizables will be far more common in this language than large fixed arrays, and the compiler may pass an unwritten by-value argument by pointer when the specialization's roots prove nothing writes the source during the call. copywill appear in places that feel like plain data flow —items.push(copy(item)),let name = copy(rec.name)— so readers can identify operations that copy data, and if it turns out to be everywhere a*spelling can be considered. The existing tests and benchmarks change in a handful of places (let copy: u8[][] = src;,g = h,d.slots = ns, pushes of variable-size locals); every&they already write stays valid, modulo the redundancy warning.&does not leave the language, only the places where the destination's type already says "reference": inferredlet r = &x;keeps it, because there it chooses the type. A reference-typed field initializer and.=bind an lvalue implicitly too, uniformly: the struct declaration is the source of truth exactly as a parameter list is.
Slices of [>..<] (spec §5.2).
Creating a slice or reference into a [>..<] is legal.
A shrinking operation on that array (pop,
resize, clear, whole assignment) while a
variable in scope may refer into it is a compile error at the shrink,
naming the variable and where it was bound, so the programmer can fix
either end: end the slice's block before the shrink, or move the shrink.
Scope stands in for liveness, which is the same test a grow-only local's
shrink already used (§5.1) and needs no last-use analysis; inside a loop
it is exactly right, since a binding made earlier in the body is in
scope at the shrink. Such a slice may be passed down and returned but
not stored into a container (a field, an element, a global), which is
what keeps every holder a variable the scan can see. The shrinks a scope
scan cannot see — through a reference parameter, or of a global — are
summarized per function specialization (which globals, which parameters'
pointees, transitively through callees), and a call is checked against
the summary the same way, so the error for a callee's shrink lands at
the call. The memory argument stands: a live [>..<]
owns its stack region exclusively (§1.3(2)), so the rule catches logic
errors, not memory ones. The same pass treats the whole assignment of a
grow-only array as the shrink it is (§5.1).
References to resizable tails (spec C.2). Before the
frame object, &bag.items was rejected because a
resizable-tailed struct was one frame header { base, len }
whose base is the struct's start on the data stack
(static prefix fields, then tail elements) and whose len is
the tail's count; a T[>..]& is a pointer to a header
whose base must be the element region, and there is no such
header for the tail alone. Offsetting the struct's header (an earlier
option) fixes it at the cost of every struct being addressed relative to
its tail. The review's instinct — give the tail its own independent
header — leads to this layout: a resizable-tailed struct is represented
as a frame object holding its fixed prefix fields exactly as a
fixed local would, followed by the tail's own gs_rhdr; only
the tail's elements live on the data stack. Then
&bag.items is a pointer to that header field — an
ordinary gs_rref, indistinguishable from a reference to a
standalone resizable, so nothing downstream needs to know it came from a
struct; &bag is a pointer to the frame object;
bag.n through either is a plain field access;
d.slots via d: dictionary& is a normal
header; and the implicit slice coercion count(bag.items)
follows. Copying such a value copies the frame object and the elements
(the RzShape prefix walk of the bytes-on-stack shape
becomes a struct copy plus EmitCopyElems); by-value
parameters and returns pass the frame object where an array passes its
header (C.3 passes "the metadata appropriate to the type"). Nesting
composes: a struct whose tail is itself a resizable-tailed struct nests
frame objects. The two awkward cases keep the bytes-on-stack shape
behind a base pointer in the frame object: a variable-size
prefix (a u8[] field before the tail, which
RzShape already refuses to copy), and a resizable-class
ADT, whose payload shape is per variant. Spec C.2 changes from
"outermost values have a frame header" to "a resizable-class value's
fixed part lives in the owning frame, and every resizable tail has its
own header there". In the compiler, StructInst::frameobj
marks the shape, codegen's IsFrameObj routes locals,
parameters, returns, globals, copies, literals and references through
the frame object, and the checker's "nested resizable" error is left
only for the variable-size-prefix and ADT shapes.
Untyped parameters bind exact types (spec §3.8). An
untyped parameter binds a reference argument as a reference, exactly as
an explicit <T> parameter does, so
f(&gs) into fn f(xs) mutates
gs.
return null is root-neutral (spec
§9.2). fn find<T, F>(xs: T[:]) -> T?
written as
for &x in xs { if F(x) { return x; } } return null;
typechecks: a null return names no root, so it adds nothing to the roots
the other returns give.
Generic type aliases
(type set<K> = dictionary<K, bool>;) do not
parse. Non-generic aliases of generic instances
(type float3 = vec3<f32>;) work and are all
vec needs, so this is a convenience only.
Constants in thread programs. let
globals of flat fixed type are static data (§11.1); a
thread_fn should be allowed to read them (PI,
float3_0), since they are not mutable shared memory. Still
open: the checker bans every global from a thread program.
Identity of references (spec TODO 0b) is not needed
by the library. index_of (spec §3.3), the subtraction the
checker validates through roots, is what lru.goose relinks
with.
9. Deliberately not included
These operations can be expressed using existing functions and short loops:
zip/map2/each2:for i in min(a.len, b.len) { … a[i] … b[i] … }. Without tuples there is nothing forzipto return.reduce(fold without a seed):fold(xs[1..], xs[0]) { … }.reversedcopies,enumerate:each_rev, andfor x, i in xs.product,average: two-line folds.first/top:xs[0], andlastexists only becausexs[xs.len - 1]is the one everybody gets wrong by one.flatten,partition,filter_indices,find_or_push,insert_ordered: each isfilter/lower_bound/insert_atplus a line.keys/valuesof a dictionary:eachplus a push.- String
concatof many pieces,repeat,pad,capitalize,escape:str(a, b, c),push_n,format_int's width, and a loop. lines:each_split(s, '\n')plustrim. (wordsis in §4.8:each_split(s, ' ')keeps the empty parts repeated spaces leave, which a command line wants dropped.)- Multiple-resizable containers (a deque, an owning string dictionary): the language has one resizable per struct (§3.4); idioms in §5.3.
- Binary serialization: Goose data is already its own serialized form
(
bench/design.md);read_fileinto au8[>..]and spec TODO 16's whole-region copy are the path, and typedread_u32_le-style accessors can wait for a format that needs them. - Date/time formatting, directories, subprocesses, networking:
osgrows when a program needs them, one extern at a time. - Thread helpers:
thread_fns take no function values and no generics (§11.2), so aparallel_forcannot be written in the library. Await_all(ids: i64[:])loop is one line. Revisit with spec TODO 9.
Goose