AI Foundation — Design (BERT is the first consumer)

← All docsdocs/AI_FOUNDATION_DESIGN.md on GitHub

Objeck is meant to be a language for AI research. Today it can run models (API.Onnx), call hosted models (API.OpenAI, API.Google, API.Ollama), and train small networks (System.ML). It cannot train or fine-tune a transformer, and every model needs hand downloading and, often, Python.

This document designs a foundation that closes that gap: tensors on CPU and GPU, autograd, nn modules, optimizers, data loading, safetensors and a Hugging Face hub cache, tokenizers, task pipelines, and ONNX as both a runtime and an export target. BERT, "like with Python's transformers", is the first thing built on it.

This is a design only. No library code changes with it. Target: the release after v2026.10.2.

Every number below marked measured was measured for this document on 2026-10-29 (method in the appendix). Everything else is labelled as an estimate or as something a phase must check.

Summary

1. Goals, rules and non-goals

Goals

  1. Use pretrained models by name, as in Python's pipeline("text-classification", model=...): no hand downloads, no Python.
  2. Fine-tune and train transformers in Objeck, on CPU and GPU, with the results matching PyTorch.
  3. One set of layers for all future AI work. Decoder LLMs, vision, audio, embeddings and RAG, and reinforcement learning all build on the same tensor, nn and hub layers, not on one-off wrappers.
  4. Parity is the test oracle. Every task and every training step is checked against PyTorch, transformers or tokenizers on fixed inputs, with stated tolerances.

Rules

Non-goals

2. What exists, and what happens to it

Line counts are from the master checkout at c816f3f9ae.

Library (source)What it isDecision
System.ML ml_core.obs (2,500 lines): Matrix2D, NeuralNetwork, DenseNetwork, RandomPure-Objeck MLPs over Float[,]. DenseNetwork has PyTorch-matching Adam and deterministic seedingKeep unchanged. It needs no native library, which is right for teaching and small tabular work. A later, optional DenseNetwork backend on the tensor core gets the same API (Phase F)
System.ML trees, SVM, Bayes, clustering, outliers (ml_tree.obs, ml_svm.obs, …)Classical MLKeep. They are outside this design. They interoperate through Float[,], which the tensor core converts to and from (§6.3)
System.ML ml_embed.obs: EmbeddingClassifier (ScoreModel)A trainable head over frozen embeddingsKeep. It stays the frozen-encoder baseline. Phase B's fine-tuner also implements ScoreModel, so Metrics and CrossValidation accept both
API.Models models.obs: Tensor, TensorSpec, EndPointThe model I/O carrier: name, shape, flat Float[] / Int[]. Remote KServe/Triton endpointsKeep, and keep the name. It is the interchange type for named model inputs and outputs. The new compute tensor is a different class in a different bundle (System.AI.Tensor.Tensor), with converters both ways. The two have different jobs (§6.1)
API.Onnx onnx.obs (3,136 lines): Session, RunResult, YOLO, ResNet, DeepLab, OpenPose, face, Phi-3ONNX Runtime inferenceKeep. It becomes the ONNX runtime layer of the foundation. Phase A's pipelines run on it, and Phase E's ONNX export targets it
System.NLP nlp.obs: WordPieceTokenizer, TokenEncoding, ApiSequenceTokenizer, TF-IDF, …Pure-Objeck NLPKeep and extend. The tokenizer.json loader, BPE and Unigram are added here (Phase A3–A5). TokenEncoding gains offsets, word ids and sequence ids, as new fields and methods; existing constructors keep working
System.NLP.Onnx nlp_onnx.obs: BertEmbedderSentence embeddings over ONNXKeep, and generalise behind the same API. The text pipelines join it. A runner interface lets each pipeline run on ONNX or, later, on the torch backend without changing the caller's code (§7.4)
Query.Structured.ML / .Vision (query_ml.obs, query_vision.obs)Query sources over ML results and YOLOKeep. They gain an embedding source for RAG once the pipelines exist (§4)
System.AI ai_rl.obs, ai_search.obs, …Tabular Q-learning, SARSA, MDPs, searchKeep. Deep RL (a DQN or a policy gradient) builds on the tensor core as new classes (§4)
API.OpenAI, API.Google (Gemini), API.OllamaHosted and local model clientsKeep. Agents keep using them. The foundation adds local embeddings and rerankers beside them

The compatibility rule. No existing class changes its public signatures. Where a class gains a second backend, that is a new constructor or a new configuration key. Each phase's PR runs the existing ML, NLP and ONNX regression tests unchanged.

3. Dependency status (primary sources)

3.1 Proposed dependencies

DependencyRoleStatus evidenceLicense
libtorch (PyTorch C++)Tensor core, autograd, kernels, optimizers, devices (Phase B)GitHub release v2.15.0 on 2026-10-28. Earlier: v2.14.1 on 2026-09-30, v2.14.0 on 2026-09-02, then 2.13.0 (2026-07-08) and 2.12.0 (2026-05-13) on PyPI. That is a release about every 1–2 months. libtorch-*-2.15.0 archives were published on download.pytorch.org on 2026-10-28 (HTTP Last-Modified). The C++ frontend docs (docs/cpp/source/frontend.md) carry no deprecation notice. The repository adds a Stable ABI C++ API (docs/cpp/source/api/stable/index.md, "guaranteed to remain stable across PyTorch versions")BSD-style (LICENSE: "Redistribution and use in source and binary forms…"). GitHub reports NOASSERTION because the file concatenates several notices. Compatible with redistribution
ONNX Runtime (already vendored, inference only)ONNX runtime layer (Phase A, E)v1.31.1 on 2026-10-26 and v1.30.1 on 2026-10-29 (GitHub releases). Inference is actively releasedMIT
safetensors (file format; no code dependency)Weight I/O, read and written by Objeck (Phase B)The format is defined by huggingface/safetensors: v0.8.0 on 2026-06-09, v0.9.0-rc.1 on 2026-09-23, PyPI 0.8.0 on 2026-09-24. Objeck implements the format itself (an 8-byte little-endian header length, a JSON header, then raw bytes), so no library is linkedApache-2.0 (format reference)
Hugging Face tokenizers (reference only, in fixture generators)The parity oracle for the tokenizersv0.23.3 on 2026-10-09 (GitHub) and 2026-10-19 (PyPI); v1.0.0-rc.3 on 2026-10-19. Not linked. Objeck reimplements the algorithmsApache-2.0
transformers / torch (reference only, in fixture generators)The parity oracle for the tasks and for trainingtorch 2.15.0 on 2026-10-28; transformers 5.22.0 installed for the measurements. Never on a user's pathBSD / Apache-2.0
ONNX (the format)The export target (Phase E). Objeck writes the protobuf itselfonnx/onnx v1.23.2 on 2026-10-06. Not linkedApache-2.0

3.2 Rejected

3.3 Gaps the rule leaves, stated plainly

GapWhyWhat users get
GPU training on Windows arm64libtorch publishes only a CPU build for Windows arm64 (libtorch-win-arm64-shared-with-deps-2.15.0+cpu.zip). No CUDA, ROCm or XPU build exists for itCPU training on Windows arm64. GPU inference still works through ONNX Runtime (QNN on Snapdragon)
GPU training on AMD GPUs under WindowsROCm libtorch is Linux-only (rocm7.14, rocm10.0 builds are manylinux_2_28_x86_64). DirectML is excludedCPU on Windows. AMD GPU training on Linux through ROCm
Linux arm64 has no libtorch zipdownload.pytorch.org has no libtorch-*aarch64*.zip (404). The CPU wheel for manylinux_2_28_aarch64 carries the same libraries (torch/lib/libtorch_cpu.so and its dependencies, measured)The release pipeline takes the libraries from the wheel. That is a supported artifact, used as an archive, without running Python
DeBERTa-v3-base weights as safetensorsmicrosoft/deberta-v3-base ships only pytorch_model.bin (a Python pickle). Objeck must never unpickle: loading a pickle can run arbitrary codePhase B's DeBERTa support loads only safetensors repos; many fine-tuned DeBERTa repos have them. ONNX inference of DeBERTa (Phase A) is unaffected
Objeck's own Windows ONNX GPU path uses DirectMLThe DirectML README says it is in maintenance mode. Objeck ships the DirectML 1.22.1 ONNX Runtime build on Windows x64 and arm64Outside this design, but flagged: an existing dependency violates the new rule. A separate design should move Windows GPU inference to a maintained execution provider; ONNX Runtime publishes plugin EPs such as plugin-ep-webgpu/v0.5.0 (2026-10-29). See the follow-ups (§13)

4. What future AI work needs from the foundation

A short survey, to make sure the layers are the right ones. The foundation's job is to make each of these a library on top, not a new binding.

AreaWhat it needsLayers it usesBuilt on today's pieces
Decoder LLMs (GPT-style, Llama-style)Causal attention; RoPE; a KV cache updated in place; sampling (top-k, top-p, temperature); BPE tokenizers; int8/int4 weights; LoRA adapters (a frozen base, small trainable matrices); bf16/fp16tensor core (in-place IndexCopy, Narrow, dtypes), nn (Embedding, RMSNorm, Linear with an optional LoRA), autograd (parameters partly frozen), hub (sharded safetensors with model.safetensors.index.json), tokenizers (BPE)Phi3Session in onnx.obs keeps working for ONNX-exported decoders; the foundation adds native Objeck decoders that can also be trained
Vision (ViT, CNNs, detection)Conv2d, pooling, BatchNorm, patch embedding, image tensors from OpenCV mats with no extra copy where possibletensor core (FromImage from API.OpenCV pixel buffers), nn (Conv2d, BatchNorm2d), data (image datasets and augmentation)opencv.obs for I/O; query_vision.obs keeps YOLO-over-ONNX; a ViT classifier is §8.3
Audio (Whisper-style encoders, keyword spotting)STFT / mel features, 1-D convolutions, encoder–decoder attentiontensor core (FFT via libtorch torch::fft), nn (Conv1d), data (WAV loading)The existing audio I/O (lame.obs, SDL mixer) is playback-oriented; a WAV reader goes in the data layer
Embeddings and RAGSentence embeddings, rerankers, vector search over Query.Structuredpipelines (SentenceEmbedder, CrossEncoder), tensor core (cosine top-k), query_ml.obs (an embedding column source)BertEmbedder today; Phase A adds sentence-transformers pooling configs and rerankers
Reinforcement learningSmall MLP/CNN policies, autograd through custom losses, replay buffers, many small steps per secondtensor core, autograd, optimizers, data (replay buffer)ai_rl.obs keeps the tabular algorithms; DQN and PPO go beside it
AgentsTool calling against hosted LLMs, plus local embeddings, rerankers and small local modelspipelines; hubopenai.obs, gemini.obs, ollama.obs stay the LLM clients. The foundation supplies the local parts

What this asks of the core, beyond BERT:

Every one of these is in libtorch. The binding must not paint them out: the native shim's tensor handle carries any dtype and device from day one, even though BERT needs only f32 and i64 (§6).

5. The layers

Each layer is its own library, so a program links only what it uses. This follows the rule that made nlp_onnx a separate library: the compiler resolves every class a linked library names, so an inference-only program must never need libtorch.

 programs ─────────────────────────────────────────────────────────────────────────────
   pipelines        System.NLP.Onnx   (nlp_onnx.obl)      TextClassifier, TokenClassifier, QuestionAnswerer,
                                                          FillMask, SentenceEmbedder, CrossEncoder, BertEmbedder
                    System.AI.Torch.Runner (ai_torch_runner.obl)   runs those pipelines on torch-defined models
   model zoo        System.AI.Transformers (ai_transformers.obl)   Bert, DistilBert, Roberta, XlmRoberta, DebertaV2,
                                                          Gpt (decoder), Vit; task heads; FineTuner
   training         System.AI.Optim   (ai_optim.obl)       AdamW, SGD, LR schedules, gradient clipping, Trainer
                    System.AI.Data    (ai_data.obl)        Dataset, DataLoader, samplers, collate, text/image/audio readers
   modules          System.AI.NN      (ai_nn.obl)          Module, Parameter, Linear, Embedding, LayerNorm, RMSNorm,
                                                          Dropout, MultiHeadAttention, Conv1d/2d, functional ops
   core             System.AI.Tensor  (ai_tensor.obl  + libobjk_torch)  Tensor, DType, Device, autograd, Scope, Generator
   I/O              System.AI.Hub     (ai_hub.obl)         HubClient, HubCache, SafeTensors, ModelConfig (config.json)
                    System.NLP        (nlp.obl)            WordPiece, BPE, Unigram, TokenizerJson, TokenEncoding
   runtime / export API.Onnx          (onnx.obl)           Session (existing); OnnxExporter (Phase E)
   existing         System.ML, API.Models, Query.Structured.*, System.AI (RL, search), API.OpenAI/Google/Ollama

Dependency direction.

Bundle names. The new bundles extend the existing System.AI family, which today holds the classic AI library: search, games, optimisation and RL. Decision D7 below.

6. Language and VM fit

6.1 One Tensor class, dtype as a value

The compute tensor is System.AI.Tensor.Tensor: a handle to a libtorch tensor, plus a reference to any Objeck array it views.

t := Tensor->From(api_tensor);
api := t->ToModelTensor("logits");

6.2 No operator overloading: methods, chained

Objeck has no operator overloading (docs/ROSETTA_HARDENING.md: "Objeck has no operator overloading. Arithmetic types expose named methods."). The tensor API is therefore method-based and fluent:

scores := q->MatMul(k->Transpose(-2, -1))->Mul(scale)->Add(mask);   # Python: q @ k.transpose(-2, -1) * scale + mask
probs := scores->Softmax(-1);

6.3 Memory: native tensors, the GC, and zero-copy

loss := Tensor->Scope(\() => {
  logits := model->Forward(batch);
  return logits->CrossEntropy(batch->GetLabels());     # survives the scope
});

6.4 Autograd lives in libtorch; closures serve the training loop

The FEATURES guide's lambda forms (\() => body, and the annotated \(T) ~ R : (x) => body) are enough for all of these.

6.5 The JIT and native-call overhead

7. Phase A — use models: hub, tokenizers, pipelines (over ONNX)

7.1 The Hub has the models (measured)

Exports that already exist cover the common BERT family. Counted from the Hub API (/api/models, 2026-10-29):

Pipeline tagRepos tagged onnx (Hub-wide)Xenova/*onnx-community/*
text-classification1,8772490
token-classification1,6481657
question-answering218624
fill-mask1824035
feature-extraction1,12710847
sentence-similarity1,019026
zero-shot-classification167178
text-ranking (rerankers)14721

Specific models (/api/models/{repo}?blobs=true and each repo's config.json):

ModelReady ONNX exportFilestokenizer.jsonLabels in config.json
bert-base-uncasedXenova/bert-base-uncased; also google-bert/bert-base-uncased (model.onnx at the root, 532 MB)onnx/model.onnx 438 MB, model_quantized.onnx 111 MB, plus fp16, int8, uint8, q4, q4f16, bnb4yes— (fill-mask)
DistilBERTXenova/distilbert-base-uncased; the original repo has none8 variantsyes—
MiniLM (all-MiniLM-L6-v2)sentence-transformers/all-MiniLM-L6-v2 ships its own onnx/ (fp32 90 MB; qint8_arm64, qint8_avx512, … 23 MB; O1–O4)9 filesyes—
RoBERTaXenova/roberta-base, Xenova/distilroberta-basemodel 499 / 329 MB, fp16, quantizedyes (BPE)—
DeBERTa-v3none for microsoft/deberta-v3-base (it has no tokenizer.json either, only spm.model). Fine-tuned DeBERTa-v3 repos do have exports: Xenova/nli-deberta-v3-small, cross-encoder/nli-deberta-v3-base, mixedbread-ai/mxbai-rerank-xsmall-v1, protectai/deberta-v3-base-prompt-injection-v28 variants (Xenova)yes (Unigram)3 (NLI)
XLM-RXenova/xlm-roberta-base (model 1,114 MB)fp32, fp16, quantizedyes (Unigram, 17 MB)—
Classifier: SST-2Xenova/distilbert-base-uncased-finetuned-sst-2-english; the original distilbert/… has onnx/model.onnx but no tokenizer.json8 variantsyes2: NEGATIVE, POSITIVE
Classifiers: othersXenova/twitter-roberta-base-sentiment-latest (3), Xenova/bert-base-multilingual-uncased-sentiment (5), Xenova/toxic-bert (6), SamLowe/roberta-base-go_emotions-onnx (28)yesyes
NERXenova/bert-base-NER; dslim/bert-base-NER has onnx/model.onnx but no tokenizer.json; Xenova/bert-base-multilingual-cased-ner-hrl8 variantsyes9 (O, B-PER, I-PER, …)
QAXenova/distilbert-base-cased-distilled-squad, Xenova/distilbert-base-uncased-distilled-squad, onnx-community/roberta-base-squad2-ONNX8 variantsyes—
Rerankerscross-encoder/ms-marco-MiniLM-L6-v2 (own onnx/), Xenova/bge-reranker-base (XLM-R), mixedbread-ai/mxbai-rerank-xsmall-v1 (DeBERTa)yes1

The layout to support:

Variant names:

7.2 Model signatures (measured from real exports)

Read with the onnx package from the downloaded files:

ExportInputs (all INT64 [batch, sequence])Outputs (FP32)
DistilBERT SST-2input_ids, attention_masklogits [batch, 2]
bert-base-NERinput_ids, attention_mask, token_type_idslogits [batch, sequence, 9]
DistilBERT SQuADinput_ids, attention_maskstart_logits, end_logits [batch, sequence]
distilroberta-base (fill-mask)input_ids, attention_masklogits [batch, sequence, 50265]
bert-base-uncased (fill-mask)input_ids, attention_mask, token_type_idslogits [batch, sequence, 30522]
ms-marco cross-encoderinput_ids, attention_mask, token_type_idslogits [batch, 1]
mxbai-rerank (DeBERTa)input_ids, attention_masklogits [batch, 1]
nli-deberta-v3-smallinput_ids, attention_masklogits [batch, 3]
all-MiniLM-L6-v2input_ids, attention_mask, token_type_idslast_hidden_state [batch, sequence, 384]

All axes are dynamic. Only BERT-type exports declare token_type_ids. DistilBERT, RoBERTa and DeBERTa-v3 exports do not. The pipelines therefore do what BertEmbedder already does:

7.3 Hub client and cache (System.AI.Hub, ai_hub.obl)

What the Hub answers (measured with raw HEAD requests, no redirects followed):

What Objeck's HTTPS client does with that today (core/compiler/lib_src/net_secure.obs, net_common.obs):

Required changes, all in Phase A2:

The cache:

$OBJECK_HUB_CACHE   (default: ~/.objeck/hub; Windows: %USERPROFILE%\.objeck\hub)
  models--Xenova--distilbert-base-uncased-finetuned-sst-2-english/
    refs/main                       → 0b6928efcb76139cae2c6881d49cda67fe119f42
    snapshots/0b6928ef…/config.json
    snapshots/0b6928ef…/tokenizer.json
    snapshots/0b6928ef…/onnx/model_quantized.onnx
    .incomplete/…                   (resumable partial downloads)
    manifest.json                   (per file: size, sha256 or git-sha1, URL, time)
use System.AI.Hub;

hub := HubClient->New();                                  # cache, token and offline mode from the environment
dir := hub->Snapshot("Xenova/distilbert-base-uncased-finetuned-sst-2-english",
                     ["config.json", "tokenizer.json", "tokenizer_config.json", "onnx/model.onnx"]);
if(dir = Nil) { hub->GetError()->ErrorLine(); return; };

7.4 Tokenizers (System.NLP, nlp.obl)

What real tokenizer.json files contain (measured, from the downloaded files):

Modelmodelnormalizerpre_tokenizerpost_processordecoder
BERT / DistilBERT / MiniLM / NER / QAWordPiece (##, max 100 chars), 28,996–30,522 tokensBertNormalizer (lowercase true or false, strip_accents null)BertPreTokenizerTemplateProcessingWordPiece
RoBERTa / DistilRoBERTaBPE, 50,265 tokens, 50,000 mergesnoneByteLevel (add_prefix_space false, use_regex true)RobertaProcessingByteLevel
DeBERTa-v3 (nli-small, mxbai)Unigram, 128,000 tokensSequence[Strip, Precompiled, Replace]Metaspace (▁, prepend_scheme "always")TemplateProcessingMetaspace
XLM-RUnigram, 250,002 tokensPrecompiledSequence[WhitespaceSplit, Metaspace]TemplateProcessingMetaspace

Some files also carry truncation and padding: all-MiniLM-L6-v2 pads to a fixed 128, and mxbai truncates at 500. Those are defaults the loader honours and the caller can override.

Design. TokenizerJson->Load(path) builds a pipeline object from the four stages. Each stage is a small class behind an interface, so new component types are added without touching the others:

TokenEncoding gains offsets (character start and end per token), word ids, sequence ids (0, 1, or none for specials), and overflow windows. NER and QA need the offsets to return character spans. New fields and methods are additive.

Loading cost (measured): Data.JSON.JsonParser on the real files, under the interpreter, with the machine otherwise idle:

FileSizeParse time
DistilBERT tokenizer.json0.71 MB0.62 s
DistilRoBERTa2.0 MB0.28 s
XLM-R15.9 MB2.7 s
DeBERTa-v3 (nli-small)8.5 MB15.8 s

Parity tests:

7.5 Task pipelines (System.NLP.Onnx, nlp_onnx.obl)

Shared base. TransformerTask handles everything every task needs:

use System.NLP.Onnx;

classifier := TextClassifier->New("Xenova/distilbert-base-uncased-finetuned-sst-2-english");  # a hub id or a folder
results := classifier->Classify(["I loved it", "Not worth the ticket"]);     # Prediction[]: label, score
results[0]->GetLabel()->PrintLine();                                          # POSITIVE

ner := TokenClassifier->New("Xenova/bert-base-NER", TokenClassifier->Aggregation->SIMPLE);
each(e : ner->Extract("Ada Lovelace worked with Charles Babbage in London.")) {
  "{$e->GetGroup()} '{$e->GetWord()}' [{$e->GetStart()}, {$e->GetEnd()}) {$e->GetScore()}"->PrintLine();
};

qa := QuestionAnswerer->New("Xenova/distilbert-base-cased-distilled-squad");
answer := qa->Answer("Where did she work?", long_context);     # Answer: text, score, start, end

mask := FillMask->New("Xenova/bert-base-uncased");
fills := mask->Fill("Paris is the [MASK] of France.", 5);      # MaskFill[]: token, text, score, sequence

embedder := SentenceEmbedder->New("sentence-transformers/all-MiniLM-L6-v2");   # reads modules.json / 1_Pooling
sims := embedder->Similarity(["a cat sits"], ["a kitten is sitting", "stock prices fell"]);

reranker := CrossEncoder->New("cross-encoder/ms-marco-MiniLM-L6-v2");
ranked := reranker->Rank("what is BERT?", passages);         # scored pairs, best first

classify := Pipeline->Create("text-classification", "Xenova/toxic-bert");   # Python pipeline() spelling

How each task reads the model, matching transformers behaviour:

Long text:

Runner interface. Each task calls an EncoderRunner:

interface EncoderRunner {
  method : virtual : public : Inputs() ~ String[];
  method : virtual : public : Run(inputs : Vector<API.Models.Tensor>, outputs : String[]) ~ Vector<API.Models.Tensor>;
}

OnnxRunner is the only implementation in Phase A. Phase B adds TorchRunner in its own library, so a model defined and fine-tuned in Objeck runs through the same TextClassifier (§8.1).

7.6 Parity tests for the pipelines

Two tiers, following programs/regression/data/bert/.

Tier 1: gating, offline, small checked-in fixtures.

| Quantity | Against Python ONNX Runtime | Against PyTorch | |---|---|---| | Logits | ≤ 1e-5 | ≤ 1e-4 | | Probabilities | ≤ 1e-6 | — | | Labels, entity spans, answer spans, top-k orders | exact | — | | Mean-pooled embeddings | ≤ 1e-5 | — |

BertEmbedder already matches Python to within 6e-7 on MiniLM.

Tier 2: optional, network.

8. Phase B — the training foundation (libtorch)

8.1 Why libtorch, compared fairly

ONNX Runtime's training API is excluded (§3.2). The remaining credible, maintained routes:

libtorch 2.15candle (Rust, Hugging Face)burn (Rust)
Maintenance (primary evidence)releases 2026-09-02, 09-30, 10-28pushed 2026-10-29; tags only, latest 0.11.0; pre-1.0v0.23.0-pre.1 on 2026-10-29; pre-1.0
Platforms (prebuilt)Windows x64 and arm64 (CPU zips), Linux x64 (zip), Linux arm64 (from the wheel), macOS arm64 (zip)built from source by us on every platformbuilt from source by us
GPUCUDA 13.2 / 13.4 (Windows x64 and Linux x64 as zips; Linux arm64 as wheels), ROCm (Linux x64), Intel XPU (Windows and Linux x64), MPS (macOS: to be confirmed for the C++ zip in B0)CUDA, MetalCUDA, ROCm, Metal, wgpu (Vulkan/DX12/Metal)
Is it the PyTorch engine?yes: the same kernels and autograd, so parity is naturalnono (it has a LibTorch backend through tch)
Runtime sizelarge: §8.4small (no measurement here)small (no measurement here)
Python needednonono
Binding effortC++ shim, linked against libtorch headers, in the same pattern as libobjk_onnxa Rust cdylib exposing a C ABI, which adds a Rust toolchain to every platform buildthe same as candle

Recommendation: libtorch.

8.2 The native shim (libobjk_torch)

8.3 Architectures in Objeck, weights from safetensors

System.AI.NN provides the following building blocks, each with Forward and named parameters in Hugging Face's naming:

Because the parameter names match, module->LoadState(SafeTensors->Read(path)) maps one-to-one onto a Hugging Face checkpoint.

BERT, as it is written in System.AI.Transformers (abridged):

class BertLayer from Module {
  @attention : MultiHeadAttention;    # attention.self.{query,key,value}, attention.output.dense
  @attn_norm : LayerNorm;             # attention.output.LayerNorm
  @intermediate : Linear;             # intermediate.dense
  @output : Linear;                   # output.dense
  @out_norm : LayerNorm;              # output.LayerNorm
  @dropout : Dropout;

  method : public : Forward(x : Tensor, mask : Tensor) ~ Tensor {
    a := @attn_norm->Forward(x->Add(@dropout->Forward(@attention->Forward(x, mask))));
    h := @intermediate->Forward(a)->Gelu();
    return @out_norm->Forward(a->Add(@dropout->Forward(@output->Forward(h))));
  }
}

class BertForSequenceClassification from Module {
  @bert : BertModel;                  # embeddings + encoder.layer.N + pooler
  @classifier : Linear;

  function : FromPretrained(dir : String) ~ BertForSequenceClassification {
    config := ModelConfig->Read(dir + "/config.json");            # hidden_size, num_attention_heads, id2label, …
    model := BertForSequenceClassification->New(config);
    model->LoadState(SafeTensors->Read(dir + "/model.safetensors"));   # strict: missing or unexpected names fail
    return model;
  }

  method : public : Forward(ids : Tensor, mask : Tensor, types : Tensor) ~ Tensor {
    return @classifier->Forward(@bert->Forward(ids, mask, types)->GetPooled());
  }
}

The other two worked examples, abridged, show the same layers serving other models:

# A small GPT-style decoder: the same Embedding, Linear and attention, with a causal mask and a KV cache.
class GptBlock from Module {
  method : public : Forward(x : Tensor, cache : KvCache) ~ Tensor {
    h := x->Add(@attn->Forward(@ln1->Forward(x), true, cache));    # causal = true; the cache is updated in place
    return h->Add(@mlp->Forward(@ln2->Forward(h)));
  }
}
next := Sampler->TopP(gpt->Forward(last_token, cache)->Select(1, -1), 0.9, 0.8, rng);

# A ViT: Conv2d patch embedding + the same encoder layers + a CLS head; images from OpenCV with no Objeck-side loop.
patches := @patch_embed->Forward(Tensor->FromImage(mat)->Div(255.0))->Flatten(2)->Transpose(1, 2);
logits := @head->Forward(@encoder->Forward(@cls->Concat(patches, 1)->Add(@pos))->Select(1, 0));

Weights and Python-freedom (measured):

Saving.

8.4 Size and packaging (measured)

Runtime libraries in the official 2.15.0 archives, uncompressed. They were read from each zip's central directory over HTTP ranges, so nothing was downloaded in full:

PlatformArchive (compressed)Runtime needed, uncompressedLargest file
Windows x64 CPU204 MB≈ 313 MB (the archive has each DLL twice, in lib/ and bin/; torch_python.dll is not needed)torch_cpu.dll 309.5 MB
Windows arm64 CPU109 MB≈ 156 MBtorch_cpu.dll 116.2 MB, armpl_lp64.dll 37.8 MB
Linux x64 CPU129 MB≈ 448 MBlibtorch_cpu.so 439.9 MB
Linux arm64 CPU (wheel)171 MB wheel≈ 316 MB (without libtorch_python.so)libtorch_cpu.so 265.7 MB, OpenBLAS 24.6 MB, Arm Compute 19 MB
macOS arm64114 MB≈ 424 MBlibtorch_cpu.dylib 421.8 MB
CUDA 13.2 (Windows / Linux x64)4,167 / 1,965 MB——
ROCm (Linux x64)about 1,965 MB——

Today's Objeck release assets (v2026.10.1) are 20–48 MB per platform, so libtorch cannot go into the default download.

Recommendation, decision D5:

8.5 Training time (measured)

PyTorch 2.15.0 CPU, from Python. The Python layer is a thin wrapper here; the kernels and autograd are libtorch's, so a C++ shim should land close to these numbers. B0 checks that.

ModelParamsTrain step, batch 32per sequencebatch 16 per sequenceInference per sequenceTrain ÷ inference
MiniLM-L6-H38422.7 M1,970 ms62 ms63 ms10 ms6.1×
MiniLM-L12-H38433.4 M3,964 ms124 ms139 ms16 ms7.6×
DistilBERT-base65.8 M4,967 ms155 ms130 ms23 ms6.7×
BERT-base109.5 M7,763 ms243 ms274 ms49 ms4.9×

These are slower than #1449's estimates. That design assumed 3–4× inference and predicted 21–28 ms per sequence for MiniLM-L6 and 150–200 ms for bert-base. The measured ratios are 5–8×, because small models are dominated by the optimizer and kernel overhead.

For the lab's grid (5,000 training rows, 3 epochs, about 15 runs at length 128, the experiment count from #1449 §5), on this CPU:

ModelPer runWhole grid
MiniLM-L6about 16 minabout 4 h
DistilBERT33–39 min8–10 h
bert-baseabout 61 minabout 15 h

That is why GPU support matters. The lab itself used Colab's GPU, which was not measured here. The example defaults to MiniLM-L6 or DistilBERT on CPU, prints its estimate before starting, and takes --max-train.

8.6 Training API

use System.AI.Tensor;
use System.AI.Transformers;
use System.AI.Optim;
use System.AI.Data;

model := DistilBertForSequenceClassification->FromPretrained(hub->Snapshot("distilbert/distilbert-base-cased"), 2);
model->To(Device->Best());                                    # CUDA / MPS / XPU when present, else CPU

tokenizer := TokenizerJson->Load(dir + "/tokenizer.json");
train := TextDataset->New(texts, labels, tokenizer, 128);     # pads to MAX_SEQ_LENGTH
loader := DataLoader->New(train, 16, true, Generator->New(42));   # batch, shuffle, seed

optimizer := AdamW->New(model->Parameters(), 2e-5, 0.01);     # lr, weight decay; no decay on bias / LayerNorm
schedule := LinearSchedule->New(optimizer, warmup, total);

trainer := Trainer->New(model, optimizer, schedule);
trainer->SetClassWeights([1.0, 99.0]);                        # weighted cross-entropy for the lab's POSRATIO runs
trainer->OnStep(\(StepInfo) ~ Nil : (s) => { s->GetLoss()->PrintLine(); });
trainer->Fit(loader, 3);                                      # epochs

model->Save("malware_distilbert");                            # config.json + model.safetensors + tokenizer.json
classifier := TextClassifier->New(TorchRunner->New(model), tokenizer);   # the Phase A API, the torch backend

FineTuner is a convenience over these calls, and implements ScoreModel.

9. ONNX as the export target (Phase E)

Every op an Objeck model runs goes through the tensor API, so the API can record a forward pass and write the ONNX graph:

A model fine-tuned in Objeck can then run in API.Onnx, on any execution provider ONNX Runtime has, including the NPUs and GPUs libtorch does not reach (§3.3). That narrows the GPU gaps for inference.

The test: export, then run in Objeck's ONNX session, in Python ONNX Runtime and against the torch-backend logits, within 1e-5.

10. The lab, end to end

The user's Lab 1 Part 4 fine-tunes distilbert-base-cased as a binary malware classifier over API-call sequences on Colab: MAX_SEQ_LENGTH 128, AdamW, linear warm-up. part4_bert_malware.py evaluates in PyTorch and does not save the model (no save_pretrained call).

10.1 First target: run the Colab model from Objeck (Phase A1)

  1. In the notebook (the user's code, Python, once):
    • give the model real labels with model.config.id2label = {0: "benign", 1: "malware"};
    • save it with model.save_pretrained(out) and tokenizer.save_pretrained(out);
    • export it with optimum-cli export onnx --model out --task text-classification out/onnx.
  2. What the export will look like, going by the measured DistilBERT classifier signature (§7.2):
    • inputs input_ids and attention_mask, INT64 [batch, sequence];
    • output logits FP32 [batch, 2];
    • a cased WordPiece tokenizer (lowercase false).
  3. In Objeck:
classifier := TextClassifier->New("malware_distilbert");
preds := classifier->Classify(test_sequences);

Then compute precision, recall and F1 with System.ML's existing metrics.

  1. The check:
    • Objeck's malware probability matches Python's softmax for the lab's fixed test set within 1e-5 (ONNX against ONNX) and 1e-4 (against PyTorch);
    • the predicted classes are identical, so the metrics are identical.

10.2 Second target: fine-tune it in Objeck (Phase B2)

11. Phases

Sizes are engineering days for one person, including tests and docs. They are estimates, except where an earlier phase measures them.

Phase A: use models (no new native dependency)

PhaseContentSize
A1 (first PR)TextClassifier over a local folder: config.json labels and problem_type; WordPiece from tokenizer.json's WordPiece section or from vocab.txt with tokenizer_config.json; session-declared inputs; batching. A checked-in tiny DistilBERT-shaped classifier, with a BERT-shaped variant that has token_type_ids; parity against Python. Works for the lab's model3–4
A2Streaming GetToFile with resume; the cross-origin Authorization strip (security); an incremental SHA-256; HubClient and cache, offline mode, pinning, tokens; TextClassifier->New(hub_id)6–8
A3The TokenizerJson pipeline (component interfaces, TemplateProcessing, decoders); offsets, word ids and windows on TokenEncoding4–5
A4ByteLevel BPE (RoBERTa), with byte-exact parity4–5
A5Unigram, Precompiled, Metaspace (DeBERTa-v3, XLM-R), with byte-exact parity; profile and fix the 15.8 s JSON load6–9
A6TokenClassifier (aggregations, striding) and QuestionAnswerer (striding, impossible answers)6–8
A7FillMask, SentenceEmbedder (sentence-transformers configs), CrossEncoder, zero-shot, Pipeline->Create4–6
A8The Tier 2 network parity job; docs/AI.md and examples; objk_apis.json3
Total36–48

Phase B: the training foundation

PhaseContentSize
B0 (spike, no Objeck code)A C++ program against libtorch 2.15 CPU on Windows x64, then Linux x64 and macOS arm64. It loads bert-base and DistilBERT safetensors, writes their forward pass with libtorch ops (mirroring §8.3), runs 20 AdamW steps and compares losses with PyTorch (≤ 1e-4). It also measures: step time against §8.5; coexistence with onnxruntime.dll in one process; the per-call overhead through an Objeck native call; whether the Stable ABI covers autograd and optimizers; and whether the macOS zip has MPS3–4
B1libobjk_torch + System.AI.Tensor: handles, dtypes, devices, views, math, autograd, Scope, zero-copy views of Float[]/Int[], live-handle accounting; builds for 5 platforms10–14
B2System.AI.NN, System.AI.Optim, System.AI.Data; SafeTensors read and write; ModelConfig; BERT and DistilBERT with classification heads; Trainer; the lab --finetune; training parity (§10.2)10–14
B3TorchRunner: Phase A pipelines on torch models; Save in Hugging Face layout3–4
B4RoBERTa, XLM-R, DeBERTa-v2 (disentangled attention is the hardest one); token-classification and QA heads6–9
B5Packaging: the optional AI runtime archive per platform, the release workflow, OBJECK_LIBTORCH for GPU builds, version checks5–7
Total37–52

Later phases, each its own design when it starts

PhaseContent
CGPT-style decoder with a KV cache, sampling and BPE; LoRA; int8 weights
DViT and CNN blocks (Conv2d, BatchNorm), image datasets over OpenCV, FromImage
EONNX export by tracing (§9)
FDeep RL (DQN, PPO) beside ai_rl.obs; an optional DenseNetwork backend on the core
GAudio front-end (STFT/mel, WAV dataset)

Ordering.

12. Decisions for the user

#DecisionRecommendation
D1PR #1449 (ORT training design)Close it as superseded by this document. Its lab mapping and class-weight loss design are carried over here (§8.6, §10.2). Its ORT-specific parts violate the no-deprecated rule. Also delete its probe/ort-training-api branch
D2Fine-tuning enginelibtorch, with candle as the documented next option if B0 finds a blocker (§8.1)
D3Hub cache locationObjeck's own ~/.objeck/hub (OBJECK_HUB_CACHE), in the same layout with real files; Python's cache read, never written
D4Default ONNX variantfp32 model.onnx for parity; quantized as an explicit option; fp16/q4/bnb4 refused until tested
D5libtorch deliveryAn optional CPU AI-runtime archive per platform from the release workflow. GPU builds are user-installed with OBJECK_LIBTORCH
D6Binding granularityA fine-grained tensor API, with architectures in Objeck (Python-free, as the goal says). B0 measures call overhead, and fused ops are added only where measured
D7Bundle namesSystem.AI.Tensor, .NN, .Optim, .Data, .Hub, .Transformers, extending the existing System.AI family. Text pipelines stay in System.NLP.Onnx, so existing programs and docs are unchanged
D8Shim ABIRegular C++ API, pinned per release. Revisit the Stable ABI after B0
L1Operator overloading (a language change)Later, its own design, once real model code exists. Not a prerequisite
L2Finalizers or a using block for native resources (a VM change)Its own design, after B1. Scope covers the need until then
—The lab notebook adding save_pretrained + export (§10.1)Yes. It is the user's notebook, so the user's call

13. First PR, and follow-ups

First PR: A1, TextClassifier over a local folder.

Follow-ups outside this design:

Appendix: how the numbers were measured

All on 2026-10-29, on the Windows build box (AMD Ryzen AI 9 HX 370, 12 cores, 31 GB).