Goose

Independently owned dynamic data: an editor's open documents

Deprecated language features: reusable and reusable[]. Use slot_pool<T> for slots and slice_pool<T> for contiguous runs. Builtin behavior and historical examples below remain relevant to existing code; removal is planned separately, after the remaining migration gaps.

This note is a research example. The requirement is a runtime-sized set of independently editable buffers, with memory reclaimed when each document closes. The difficulty is independent growth and release in arbitrary order, not the representation of a list of strings. Stdlib slice_pool<T> expresses this with root-owned storage. This note shows that representation and what it costs, which earlier workarounds remain useful, and two possible future extensions that cover what slice pools leave out: bounded independent dynamic stacks and opaque API resources. Neither extension is an implemented language feature.

The ordinary owning representation

In C++ the core could be:

struct Document { std::vector<std::byte> text; };
std::unordered_map<DocumentId, std::unique_ptr<Document>> open;

The corresponding Rust shape is HashMap<DocumentId, Document> with Document { text: Vec<u8> }; a Box<Document> is optional if the document object itself needs a stable address. These examples do not promise that an interior pointer survives a text-buffer reallocation. Edits can use offsets or temporary borrows. Stable interior references are a separate requirement.

For example: open A (1 MB), open B (20 MB), append 5 MB to A, close A while B stays open, open C (2 MB), grow B, close C. There is no useful stack ordering: both allocation and release interleave. A document may later own undo history or tokens too, but one buffer per document already exposes the issue.

Documents as runs in a slice pool

A document is a descriptor in a stdlib slot_pool, and its text is one run of bytes in a stdlib slice_pool<u8>. The open documents are a list of links into the descriptor pool (§3.9), which is also what lets the operations take a Document&:

import slice_pool;
import slot_pool;

struct Document { id: u64, text: u8[:] }   // the document's current run of bytes

var docs = slot_pool<Document> {};
var bytes = slice_pool<u8> {};
var open: Document&<u32 in docs.items>?[>..] = [];   // in tab order; pruned elsewhere

fn open_doc(id: u64, size: i64) -> Document& {
    let d .= docs.alloc_ref(Document { id: id, text: bytes.alloc_slice(size) });
    open.push(d);
    return d;
}

fn grow(d: Document&, extra: i64) {
    d.text = bytes.realloc_slice(d.text, d.text.len + extra);
}

fn close(d: Document&) {
    bytes.free_slice(d.text);
    docs.free(docs.items.index_of(d));
}

The compact in docs.items links name the descriptor pool's array component. The descriptor and byte pools both use ordinary library APIs.

alloc_slice places a run in the first free span (in index order) that can hold it, or else at the end of the array. realloc_slice grows a run in place where the run ends the array or free bytes follow it. Otherwise it moves the run to where a new run of the new size would go, which can be the run's own old place. free_slice returns a run to the pool and merges it with the free spans beside it. Every byte a run gains is zero.

In the example, A and B are placed end to end. When 5 MB is appended to A, B directly follows A and there is no free 6 MB run, so A's megabyte is copied to the end of the array. Closing A leaves two free runs: A's first place, and the 6 MB at the end. C then takes the front of the 6 MB run. B is now followed by C, so growing B copies B's 20 MB to the first free run that can hold the new size. That run is B's own place merged with the free megabyte before it, if the growth fits there; otherwise it is at the end of the array, starting in the 4 MB left free after C. Closing C merges its bytes with whichever free bytes border them.

Views keep the pool's reuse semantics (spec 9.4). A slice or byte reference taken before a move or a close keeps its original range, and reads whatever bytes a later allocation puts there; it never reaches outside the array. Editing code therefore reads the document's text again after anything that can grow or close it, and keeps cursors as offsets rather than references. A link in the tab list outlives its document's close in the same way: it still reads a valid Document, which may by then be a different document.

index_of turns a Document& back into the slot that free takes. It recognises a reference that a caller passed in as one of docs' slots only because a type names that pool (spec 3.9). A pool that no T&<w in pool> type names has to pass slot numbers around instead; docs/implementation.md §10 records what would lift that restriction.

The checker cannot always tell which array a slice read out of docs points into. Any other global that can hold a u8, such as a log buffer, is as good a candidate as bytes (spec 9.5). realloc_slice and free_slice do not need the checker to know. When a slice passed to them is not shown to belong to bytes, it is checked when the call runs, and the call aborts unless the slice lies inside bytes on an element boundary. That check is a few compares, in addition to the work the call does anyway. A slice type naming its pool (u8[: in bytes], like T&<u32 in pool>) would make that knowledge static, if a program needs to avoid the runtime check or detect an invalid pool assignment earlier.

What the slice pool costs

The other Goose representations

The placement rules are unchanged. An array still cannot contain resizable elements (spec 3.4), so a growable array of growable documents remains inexpressible. Cycle functions still cannot keep nonfixed locals across a call into their cycle (spec 7.8). Slice pools are the in-language substitute, and they supersede several earlier workarounds:

The following approaches remain useful:

Approach Where it still fits
Inline limited arrays, u8[..MAX], as fixed-size slots in a slot_pool A real and small bound. Nothing is ever copied or fragmented, and each document is one fixed slot. Every slot pays the full capacity, and exceeding it cannot grow that buffer.
Reusable fixed-size chunks linked by indices or relative references, possibly as a rope or piece table Very large documents, or frequent inserts in the middle, which a contiguous run copies on every move. Chunks never move, so references into them stay current, growth copies at most a chunk, and fragmentation is bounded by the chunk size. It needs chunk addressing, cross-chunk iteration and a stale-slot policy, which a library could hide.
A worker owning each document Document actors wanted anyway. Reclamation happens at worker exit, at the cost of a message protocol, an OS thread per document and copying whatever crosses a queue.
Foreign owning storage Memory the program does not own. Option C below gives it a typed representation.

Future option A: bounded independent dynamic stacks

A slice pool has several limits: element addresses can change during growth, growth can copy data, and closing a document returns its memory to the OS only through trim, for u8 pools, a page at a time and without shrinking the array. Documents also share fragmentation and are limited to one reservation's total storage.

This option extends the dynamic stacks idea in spec 11.3. It proposes a runtime-sized collection in which each element owns an independently growing stack: a resizable array of resizable arrays. Today's placement rules do not permit this, so it requires a new owning container. Its descriptor table could grow or reuse slots while element storage stays in separately reserved address regions.

For a supported 64-bit target, define a virtual-address reservation budget rather than assuming that a 64-bit pointer provides 2^64 usable bytes. A runtime profile could guarantee a budget by reserving it before the collection accepts work; failure to obtain that reservation is reported at initialization. With usable capacity R per stack, guard space G, descriptor space H and reserved budget B, the configured total stack count must satisfy N * (R + G) + H <= B. This is a bound on all stacks using that budget, not a separate allowance for every container. Additional stacks for undo history or tokens count too.

An editor could choose a maximum of 256 open documents, each with a 1 GiB text ceiling. That requires about 256 GiB of reserved virtual address space plus guards and metadata, not 256 GiB of resident RAM. Such a configuration is usable only on a target where that reservation can be guaranteed. It is a reasonable trade for a bounded number of substantial buffers; many small buffers belong in a slice pool instead.

The two can be combined: a stack from this option can back a slice pool of its own, for example one per class of document, combining stable large buffers with pooled small ones. The remaining questions include per-stack versus pooled size budgets, the owning descriptor's transfer rules, and how much per-document lifetime precision the checker can provide without annotation-heavy APIs. Fixed size and count limits are part of this option's contract, not an accidental failure mode to hide. This option is worth that checker work only where measurements of slice pools show that copies, retained memory or a reservation limit matter.

Future option C: opaque API resources

Introduce a fixed-size resource<U> value, internally a u64, for arbitrary API pointers and handles such as GPU buffers, native byte buffers and file mappings. U is a binding-defined tag identifying the resource kind. The binding creates and interprets the opaque representation, whether it holds a native pointer, a native handle or an entry in a binding-owned table.

A resource can be stored wherever a fixed-size value is allowed: locals, globals, aggregate fields, array elements, arguments and return values. Its operations in Goose are limited to copying/storing it and passing it to APIs accepting the compatible resource type. Code cannot dereference it, perform arithmetic or comparisons on it, or convert it to or from a general integer. Copying the value copies the opaque handle; it does not itself duplicate the native allocation. The tag prevents mixing resource kinds, but native resource lifetimes and API validity still need binding-defined rules.

For example, a GPU binding could return a resource<GpuBuffer> and accept it in upload, draw and release operations. A byte-buffer binding could provide allocate, resize, release and bulk read/write or copy operations through a resource<ByteBuffer>. With slice pools in the language, this is no longer the way to get independently growing Goose buffers; it is the way to hold memory and objects that belong to someone else, accessed through functions and copies. Slice pools and option A remain preferable when the program wants direct array indexing, slices and ordinary Goose element references.

Existing FFI already passes scalar handles and Goose buffers to C, including builders that a C API can append copied bytes into. The proposed tagged resource type adds an opaque representation to that interface; it is not an implemented way to obtain a native pointer as a Goose reference.

A file-mapping binding is another example: a map operation could return a resource<FileMapping>, with API functions to query it, copy a range into a Goose buffer and unmap it. Ordinary growable Goose arrays use the runtime's own reserved address regions; the current runtime does not map a file into such an array. Holding a mapping resource therefore would not automatically make the mapped bytes a Goose array or slice. Direct mapped views would need a separate design connecting their bounds and lifetime to the resource.

Cleanup remains unresolved. A binding might optionally supply cleanup to run when a local resource leaves scope; whether or how that extends to resources inside aggregates is also open. Several further questions are unresolved: which copies would trigger cleanup, what happens to aliases after an explicit release, how repeated cleanup or double-free is prevented, and what aggregate copying means. The proposal does not yet promise RAII, destructors or automatic ownership transfer. These questions must be settled before a cleanup hook could be given reliable language semantics.

Extending slice pools

There are also smaller steps than either option, each addressing one of the costs above:

Evaluating what remains

Run the open/edit/close sequence against slice pools first, now that they exist. Measure live bytes, the array's high-water and committed bytes, copied bytes and latency separately. Include repeated fragmenting growth and release, a document that outgrows the reservation, and old views read after a move, a close and reuse. Those measurements establish where option A's stable, releasable storage justifies the additional lifetime analysis. Evaluating option A also covers the 257th document, a document exceeding its size ceiling, and references held across a document close. For option C, evaluate API buffer transfers and file-mapping access while keeping the unresolved cleanup policy explicit.