Writing native libraries for Objeck

← All docsdocs/native_interface.md on GitHub

Objeck reaches C and C++ through shared libraries it loads at runtime. Every binding in the standard library works this way — SDL2 and OpenGL, ODBC, OpenCV, ONNX, mbedTLS, LAME, the diagnostics engine behind the language server. If you want Objeck to call something the VM does not already know about, this is the mechanism, and it is the only one.

This document is the whole contract: what the VM guarantees, what your library must guarantee back, what the collector does with the memory you allocate, and what a call costs.

The shape of a call

A native library is two halves that meet at a name.

The Objeck half loads the library and calls into it by function name, passing an array of boxed values:

use System.API;

class Beep {
	@proxy : static : DllProxy;
	@fn_beep_play : static : String;

	function : Play(hz : Int, ms : Int) ~ Bool {
		if(@proxy = Nil) {
			@proxy := DllProxy->New("libobjk_beep");
		};

		array_args := Base->New[3];
		array_args[0] := IntRef->New();     # the result comes back here
		array_args[1] := IntRef->New(hz);
		array_args[2] := IntRef->New(ms);

		if(@fn_beep_play = Nil) { @fn_beep_play := "beep_play"; };
		@proxy->CallFunction(@fn_beep_play, array_args);

		return array_args[0]->As(IntRef)->Get() <> 0;
	}
}

The C++ half exports a function of that name taking a VMContext&:

#include "../../vm/lib_api.h"

extern "C" {
#ifdef _WIN32
  __declspec(dllexport)
#endif
  void load_lib(VMContext& context) { }

#ifdef _WIN32
  __declspec(dllexport)
#endif
  void unload_lib() { }

#ifdef _WIN32
  __declspec(dllexport)
#endif
  void beep_play(VMContext& context) {
    const size_t hz = APITools_GetIntValue(context, 1);
    const size_t ms = APITools_GetIntValue(context, 2);

    const bool ok = platform_beep(hz, ms);
    APITools_SetIntValue(context, 0, ok ? 1 : 0);
  }
}

There is no header generation, no IDL, no signature checking. The name is the only thing that ties the halves together, and the argument array is the only thing that crosses. If the two sides disagree about what index 2 holds, nothing tells you — you get a wrong number or a crash.

That bluntness is the trade. It also means the whole interface fits in one header, core/vm/lib_api.h, which is worth reading in full at least once.

What happens between them

  1. DllProxy->New(name) runs EXT_LIB_LOAD, which resolves name against the library directory, appends the platform's extension, loads it, and calls the library's load_lib.
  2. CallFunction(name, args) runs EXT_LIB_FUNC_CALL. The VM resolves name to a symbol, builds a VMContext around the argument array, and calls it.
  3. Your function reads and writes the argument array in place. It does not return a value; there is no return channel other than the array.
  4. Unload() runs EXT_LIB_UNLOAD, which calls unload_lib and unloads.

The library directory is $OBJECK_LIB_PATH/native/, falling back to ../lib/native/ relative to the working directory. The name you pass has no extension — the VM appends .dll, .dylib or .so — and by convention starts with libobjk_, which is what the deploy scripts name the built artifacts.

The C++ side

The three exports

Every library must export load_lib(VMContext&) and unload_lib(). The VM looks them up by name immediately after loading and exits the process if either is missing. Most libraries leave both empty; use them for one-time global setup and teardown that cannot be done lazily.

Then export one function per entry point, each void f(VMContext&).

All of it goes inside extern "C", and on Windows each export needs __declspec(dllexport). POSIX builds export by default.

Reading arguments

context.data_array is the Objeck Base[] you were passed. Never index it directly; the header's accessors know the layout.

size_t         n   = APITools_GetArgumentCount(context);
bool           b   = APITools_GetBoolValue(context, 1);
size_t         i   = APITools_GetIntValue(context, 1);
double         f   = APITools_GetFloatValue(context, 1);
wchar_t        c   = APITools_GetCharValue(context, 1);
unsigned char  y   = APITools_GetByteValue(context, 1);
const wchar_t* s   = APITools_GetStringValue(context, 1);
size_t*        obj = APITools_GetObjectValue(context, 1);

Every one of these is bounds-checked against the argument count and returns a zero value (or nullptr) when the index is out of range or the slot is empty. That is a guard against a mismatched call, not a licence to skip checking: APITools_GetStringValue returns nullptr for a Nil string, and dereferencing it is your bug, not the VM's.

Arrays arrive as a holder — an IntArrayRef, FloatArrayRef, ByteArrayRef and so on — which is an object wrapping the array. So there are two steps:

size_t* holder = APITools_GetArray(context, 1);      // the holder
size_t* array  = (size_t*)holder[0];                 // the array itself
size_t  count  = APITools_GetArraySize(array);
double* values = (double*)APITools_GetArray(array);  // past the 3-word header

An Objeck array is three header words (size, dimension count, then one length per dimension) followed by the elements, which is what ARRAY_HEADER_OFFSET is for. APITools_GetArray(array) returns the element pointer; cast it to the element type. The typed element accessors (APITools_GetFloatArrayElement and friends) do the same work one element at a time with bounds checks.

String[] is the exception with a shortcut: APITools_GetStringsValues(context, i) returns a std::vector<std::wstring> directly.

Returning values

You return by writing into the argument array. By convention index 0 is the result slot, and the Objeck side has already put a holder there for you.

APITools_SetIntValue(context, 0, 42);
APITools_SetFloatValue(context, 0, 3.5);
APITools_SetStringValue(context, 0, L"done");
APITools_SetObjectValue(context, 0, some_object);

The primitive setters write through the holder the caller supplied — they mutate the IntRef that is already in the slot. SetStringValue and SetObjectValue allocate something new and replace the slot's contents.

To hand back a new array, allocate one and store it in the holder:

size_t* holder = APITools_GetArray(context, 0);
size_t* result = APITools_MakeByteArray(context, bytes.size());
memcpy(result + ARRAY_HEADER_OFFSET, bytes.data(), bytes.size());
holder[0] = (size_t)result;

APITools_MakeIntArray, MakeFloatArray, MakeFloatMatrix, MakeCharArray and MakeByteArray cover the cases. APITools_CreateStringObject builds a System.String; APITools_CreateObject builds an instance of any class by qualified name.

Calling back into Objeck

APITools_CallMethod(context, instance, L"MyClass:Handler:o.System.String,");
APITools_CallMethod(context, instance, cls_id, mthd_id);

The qualified-name form wants the full mangled signature, which is why the by-id form exists for anything called repeatedly.

Push the method's parameters on context.op_stack (advancing *context.stack_pos) before the call; its return value, if any, is left there for you to pop after.

A call back runs on the calling thread's own call stack and operand stack, so it is bounded like any other call: a recursion through your library and back -- native code calling an Objeck method that calls the native code again -- stops at the thread's call-depth limit (--call-depth) or when its native stack runs low, with call stack bounds have been exceeded. A try block inside the method you called recovers from an error the usual way. When the only handler is outside your native call, the callback returns to you with zero in place of its result (and any further call back on that thread does the same at once): return as soon as you can, and the error is raised at your native call, where that handler takes it. With no handler at all the program reports the error and ends. A thread the VM did not start has no call stack to share; a call back from one runs as it always did, unchecked. programs/regression/native_callback/ holds a test library that exercises all of this (run_vm_flag_tests.py section 18).

Be aware that no shipped binding currently uses this, so it is the least exercised part of the interface. It also has a consequence for the memory rules below: calling back into Objeck runs a nested interpreter, whose allocations are ordinary Objeck allocations and can therefore trigger a collection.

Memory and the collector

Everything a native library allocates lives in the old generation. That is the whole rule, and it is worth understanding why, because it is what makes the plain stores in lib_api.h safe.

Objeck's collector is generational. Objects are normally allocated in a nursery and promoted — which moves them — when they survive a collection. Moving is safe because the collector fixes up every reference afterwards, and it finds those references two ways: by tracing from roots, and, for references stored into old-generation objects, through a write barrier that records the object as dirty when the store happens. A minor collection marks an old object without recursing into it, so an old-to-young reference that the barrier never recorded is invisible to both the mark phase and the fixup phase.

Now look at how a library returns an object:

void APITools_SetObjectValue(VMContext& context, int index, size_t* obj) {
  ...
  data_array[index] = (size_t)obj;
  APITools_WriteBarrier(context, context.data_array);
}

The barrier call is there now; it was not always. lib_api.h is compiled into your library, not into the VM, so it has no direct access to the collector — it reaches the barrier through a function pointer the VM puts on VMContext (write_barrier, added for #864). And the argument array is always an old-generation object, because AllocateArray never uses the nursery.

So the VM allocates on your behalf in the old generation instead. Old objects are never moved, so nothing needs fixing up, and a major collection recurses into old objects, so your value stays reachable through the argument array. The barrier is not needed because the hazard is not created.

This is enforced VM-side (MemoryManager::AllocateObjectNative), which means it covers every library already compiled, including yours, without a rebuild. You do not have to do anything. But you do have to know the rule, because it has one consequence you can feel: objects a native library allocates are reclaimed only by a major collection. A library that allocates a String per call in a hot loop grows the old generation. If you are returning many small values, return them through primitive holders, which allocate nothing at all.

programs/regression/native_gc_barrier_test.obs is the test that holds this down. It fails — with a dangling reference, in practice an "Invalid object cast" abort — against a VM that allocates native objects in the nursery.

Why both, and not just one

The allocator rule came first and the barrier hook came second. Both are here, and they cover different halves.

The allocator rule covers every library already compiled. lib_api.h compiles into each library, so a header-only fix reaches only libraries that are rebuilt. Allocating native objects old is VM-side, so it protects binaries nobody will ever rebuild. That was the right first move and it is still load bearing.

The barrier hook covers what the allocator cannot: a store of an object the library did not allocate. For a while the guard there was a written rule — do not store a received object into anything — because a barrier the author must remember to call is a bug that can be reintroduced by omission, and allocating old makes the bug unrepresentable rather than merely absent.

The trouble is that the rule only makes the bug unrepresentable for values the library allocates. A received object is not one of those, so for that case the written rule was the entire enforcement — and it was being broken. core/lib/onnx stored a caller's label String into a natively-allocated result object at three sites, unbarriered, for as long as those entry points have existed (#864). It never bit, because callers happen to keep the labels array alive, but the rule was not holding anything down. A prohibition nothing checks is not a guard.

So: the store is now expressible and correct rather than forbidden, and tools/cicd/check_native_write_barrier.py checks the shape in CI, which is what the written rule never could.

One thing to keep in view. Allocating native objects old costs almost nothing today — measured at +11% old-generation bytes with no additional collections — but only because arrays are already old-generation, so all that moved was a small object header. If array nursery allocation is ever enabled, that trade flips and native returns become a real old-generation cost. The hook being in place is what makes revisiting that a change of one allocator argument rather than an ABI change.

Arrays are born old, objects are born young

Worth stating separately, because it decides which stores are safe:

No collection can run inside your function as long as your function only allocates — the API's allocators pass collect=false, which neither parks at a safepoint nor drives a collection, so nothing you allocate moves while you are still holding it.

The exception is calling back into Objeck. APITools_CallMethod runs a nested interpreter, and allocation inside it is ordinary Objeck allocation with collect=true. So a collection can run in the middle of your function there, and a raw size_t* you were holding across that call may have moved. Re-fetch anything you need after a callback rather than holding it across one.

The consequence for you: everything you allocate is old, and the argument array is old, so every store you make between them is old-to-old and needs no barrier.

The store that does need one is of an object you did not allocate:

// a label that arrived as an argument, into a result object you built
result_obj[1] = labels_objs[class_id];
APITools_WriteBarrier(context, result_obj);   // <- required

Three kinds of value are "not yours": an object the caller gave you, anything you read out of one, and anything a callback returned. Any of them may be young, and every container you could put one in — an array, or an object you allocated — is old. APITools_SetObjectValue barriers on its own, so returning a value through the argument array is always safe; a direct store into an object is where you have to call it yourself.

Pass the container you stored into, not the value, and pass it as the allocator returned it — for an array that is the base pointer, where [0] is the element count. Calling the barrier when it was not needed is harmless: its fast path returns immediately when the container is young or already recorded. APITools_WriteBarrier is also a no-op against a VM too old to supply the pointer, so a library built against this header still loads on one.

This is not a hypothetical restriction — the VM's own traps had exactly this bug. A trap that returned String[] allocated the array (old), filled it with freshly created Strings (young), and ran no barrier; the next minor collection promoted the elements out from under an array the collector does not look inside. Directory->List followed by enough allocation to fill the nursery turned two thirds of a 393-entry listing into unreadable memory, and reading one segfaulted the VM. programs/regression/trap_array_barrier_test.obs guards it now.

What you still own

The collector manages what it allocated. It knows nothing about anything else.

Errors

There is no error channel. A native function returns void, and the VM has no notion of a failed call.

Do not let an exception escape. The VM calls you through a raw function pointer, across a module boundary, from a frame that knows nothing about unwinding. Catch everything at the boundary.

Do not call exit(), however tempting. Report the failure through the argument array and let Objeck decide.

The convention that has held up is a status in the result slot plus a last-error string the caller can ask for:

void thing_do(VMContext& context) {
  try {
    ...
    APITools_SetIntValue(context, 0, 1);
  }
  catch(const std::exception& e) {
    last_error = BytesToUnicode(e.what());
    APITools_SetIntValue(context, 0, 0);
  }
}

The VM itself is less forgiving than this advice: a missing load_lib, a missing symbol, or a library that fails to load all print a message and exit the process. A typo in a function name is a hard stop at the first call, not an exception you can catch.

Threading

Objeck threads call native functions concurrently, and nothing serialises them. A library with mutable global state needs its own locking. Several of the bindings here are single-threaded in practice only because their callers happen to be.

Which VM you are running on

VMContext only grows, at the end. A library compiled against an older lib_api.h reads the fields it knows at the offsets it knows, so it keeps working on a newer VM without a rebuild. The other direction is the hazard: a library built against a newer header, run by an older VM, is handed a shorter struct, and the fields past its end are whatever lies beyond it on that VM's stack.

Two markers make that detectable, and each field added since is guarded by one:

fieldpresent whenhow to check
write_barrierset by every VM since #864; an older VM's struct ends before it, and no marker says soAPITools_WriteBarrier no-ops on null only
blocking_abi, begin_blocking, end_blockingblocking_abi == APITOOLS_BLOCKING_ABI ("OBJKPARK")APITools_Parked checks it
abi_version, struct_sizeAPITools_GetAbiVersion(context) >= 1it checks blocking_abi, then the "OBJK" tag in the high half of abi_version

abi_version is APITOOLS_ABI_TAG | APITOOLS_ABI_VERSION and struct_size is sizeof(VMContext), both as the VM was built; every construction site in core/vm/interpreter.cpp sets them. A field appended after struct_size is present exactly when it ends within struct_size:

if(APITools_HasField(context, offsetof(VMContext, new_field) + sizeof(context.new_field))) {
  // safe to read context.new_field
}

APITools_GetAbiVersion returns 0 for any VM that predates the field, which says nothing about the fields before it: check those by their own markers. Whoever appends a field bumps APITOOLS_ABI_VERSION in the same change and adds a line for it to the comment beside the define. run_vm_flag_tests.py section 18 builds a library that reads both fields back and expects version 1 and a struct_size that covers the whole header.

The helpers in lib_api.h are all inline, so a library may include the header from more than one source file without duplicate-symbol errors.

What a call costs

Measured on Windows x64, per call, against a native function that does nothing:

ns
a call whose name is a string literal~1,655
the same call, name held in a field~130
a fully hoisted call with a reused argument buffer~125

The number that surprises people is the first one. An Objeck string literal is materialised into a fresh String every time it is evaluated — not once at load, not interned. A name written at the call site is therefore an allocation and a copy on every single call, and at ~1,525ns it is 92% of the cost of the call, spent in code that reads like a constant.

Two rules follow, and together they took a 500-box OpenGL frame from 15.02ms to 3.35ms.

Rule 1: hold the name in a field

# Before -- allocates "sdl_gl_mesh_draw" on every draw.
Proxy->GetDllProxy()->CallFunction("sdl_gl_mesh_draw", array_args);

# After.
@fn_sdl_gl_mesh_draw : static : String;
...
if(@fn_sdl_gl_mesh_draw = Nil) { @fn_sdl_gl_mesh_draw := "sdl_gl_mesh_draw"; };
Proxy->GetDllProxy()->CallFunction(@fn_sdl_gl_mesh_draw, array_args);

Static, because the names are class-wide constants and a function : cannot read an instance field. The lazy fill races harmlessly between threads: both write an equivalent String, one wins, the loser's allocation is discarded, and no one can observe a half-built value because the store is a pointer store.

tools/cicd/hoist_native_names.py applies this across a file, and --check reports any call site that still writes a literal:

python tools/cicd/hoist_native_names.py --check core/compiler/lib_src/*.obs

Worth knowing: core/lib/sdl/code_gen generates sdl2.obs, and its emitter writes the literal form. Regenerating it undoes the hoisting for all 353 call sites, and --check is how you find out.

Rule 2: reuse the argument buffer

The usual shape allocates a Base[] and a holder per element on every call:

array_args := Base->New[3];
array_args[0] := IntRef->New(mesh);
array_args[1] := IntRef->New(mode);
array_args[2] := IntRef->New(count);

Holders are mutable, so on a hot path you can build the buffer once and refill it:

if(@draw_args = Nil) {
	@draw_mesh := IntRef->New(@mesh);
	@draw_mode := IntRef->New(mode);
	@draw_count := IntRef->New(count);
	@draw_args := Base->New[3];
	@draw_args[0] := @draw_mesh;
	@draw_args[1] := @draw_mode;
	@draw_args[2] := @draw_count;
}
else {
	@draw_mesh->Set(@mesh);
	@draw_mode->Set(mode);
	@draw_count->Set(count);
};
Proxy->GetDllProxy()->CallFunction(@draw_name, @draw_args);

This is worth 6-8% of a draw-heavy frame, and it introduces exactly one new way to be wrong: a buffer that is not refilled sends the previous call's values. That failure is invisible to most tests — draw the same thing twice and a stale buffer produces the correct picture — so a fixture has to pass two different values in succession and check that the second one landed.

There is a second hazard, and it is the reason to be careful about what you park in a reused buffer. A buffer holding a caller's array or object keeps it reachable for as long as the buffer lives, which is now forever. Release those slots after the call:

Proxy->GetDllProxy()->CallFunction(@m4_name, @m4_args);
@m4_args[1] := Nil;              # do not pin the caller's matrix
@m4_matrix->Set(@m4_idle);

Reuse buffers that carry primitives. Be deliberate about buffers that carry references.

What is left, and what is not worth chasing

After both rules a call is ~125ns, nearly all of it spent before your C++ function is entered: building the argument array and dispatching the call.

It is tempting to blame the JIT for the rest. CallFunction contains EXT_LIB_FUNC_CALL, which is in neither JIT whitelist, so that method is always interpreted and every native call from JIT-compiled code crosses into the interpreter. That is true, and it costs nothing. Measured by running the same loop from a JIT-compiled caller and from an interpreted one — a method containing a native call cannot be compiled, which is how you get one of each — the compiled caller is faster by 155-270ns per call. The crossing is not a toll.

Nor is there much to win by making the opcode JIT-able. What that would remove is the interpreted method invocation, and that is ~20ns: about 16% of a minimal call and under 2% of a call doing any real work. It would also mean restructuring how the opcode reads its operands, since it takes them from frame->mem and a JIT callback has no frame — the same frame dependency that HasFrameDependentTrap exists to reject. A 20ns ceiling is not worth that.

The practical consequence: batch at the boundary. One call that draws 500 instances beats 500 calls that draw one, by far more than any tuning of the call itself — and unlike the call overhead, batching has no ceiling.

Building and shipping a library

Naming matters more than it looks. The Objeck side passes an extensionless name to DllProxy->New, the VM appends the platform extension, and the deploy scripts are what actually produce a file with that name — the build output is usually named something else and renamed on the way into lib/native/.

A new library needs to be added to each deploy script, to the CI build lists, and — if it has an Objeck-side .obl — to the library lists in update_version.sh, update_version_arm.sh, code_doc64.in, and the CI "Rebuild libraries" step. tools/cicd/check_doc_lists.py guards the doc lists.

One trap worth stating plainly, because it has bitten this repo: a library that CI compiles is not necessarily a library that ships. The obu updater was built and tested by CI for two releases while being packaged in no archive at all. Check the archive, not the build log.

Traps

Collected from things that have actually gone wrong here.

Is this a good design?

It is a good design for what it is: a thin, uniform, dependency-free bridge that any C or C++ code can be reached through, with one header and no tooling. The VM stays out of the way, and there is nothing to regenerate when a signature changes.

The costs are real and worth naming. The name-based dispatch has no type checking whatsoever, so a mismatch between the two halves is a runtime corruption rather than a compile error. Errors have no channel and every library invents its own. Everything crosses as size_t*, and correctness rests on the caller and callee agreeing about a layout that nothing verifies. And per-call overhead is high enough that the interface rewards batching much more than it rewards micro-optimisation.

None of that argues for replacing it. It argues for the two habits this document keeps coming back to: batch at the boundary, and write the fixture that can tell the difference — because a native call that returns the previous call's answer, or a stale buffer that draws the right picture for the wrong reason, will pass any test that only asks whether something happened.