Tables are everywhere in science and industry, but general-purpose learning from tables remains a stubborn problem.
A table can mix continuous measurements, categories, missing values, identifiers, treatments, outcomes, timestamps, and human-defined roles. Two columns with the same storage type can mean completely different things. Two tables with different schemas can describe closely related mechanisms. The usual machine-learning workflow handles this by building a new pipeline for each dataset: clean the data, define features, select a model, tune it, and start over when the schema changes.
Tabular foundation models ask a harder question: can a model learn reusable structure across many tables and tasks, then adapt to a new table with little or no task-specific training?
Recent systems such as TabPFN and LimiX show that this is a serious research direction. They also make the open questions sharper. What should the model treat as context? What training distribution produces useful transfer rather than benchmark familiarity? How should it represent values, feature roles, missingness, and relationships between rows? Which gains come from pretraining, and which come from architecture, tuning, or evaluation choices?
We are starting TabU-lab to work on these questions in the open.
What TabU-lab is
TabU-lab is a WeHub open research lab for building tabular foundation models from scratch. It is both a model-building effort and a public record of how the models are built.
The repository will contain the code, configurations, experiment preregistrations, run receipts, curves, failures, and consolidated reports produced along the way. We do not want to publish only the checkpoint that survived. We want the path to that checkpoint to remain inspectable.
Our immediate goal is modest: establish a small-scale training loop that is reproducible, falsifiable, and strong enough to tell us when an idea is not working. Scaling comes later.
The lab begins with an empty ledger. We have not accepted a public model result or benchmark claim. The first result will appear only after its hypothesis, comparison, budget, and pass/kill criterion have been committed.
Our experiment contract
Each formal experiment will move through the same basic sequence:
- Preregister. Write the hypothesis, baseline, compute budget, target metric, and pass/kill criterion before execution.
- Implement. Commit the exact code and configuration needed to run the comparison.
- Run. Record the command, git revision, configuration hash, dataset version, split, seed, environment, host, and compute used.
- Publish the receipt. Preserve raw metrics, curves, logs, artifacts, failures, and limits on interpretation.
- Declare a verdict. Mark the experiment
pass,kill, orrevise. Consolidated reports may claim only what the receipts support.
Failures will remain in the ledger. A failed run with a clean protocol can save other researchers time. A successful run without provenance cannot support much.
The first gates
We will begin with a narrow vertical slice rather than a broad benchmark campaign: one data source, one model family, one objective, a small set of datasets, and multiple seeds.
- Gate 0: sanity. The implementation must overfit a tiny dataset of roughly 100 examples. If it cannot, larger runs are premature.
- Gate 1: reproducibility. One command must produce a checkpoint and a complete receipt from a clean environment.
- Gate 2: pretraining attribution. A pretrained model must beat the same architecture with random initialization under the same finetuning budget. Otherwise an apparent pretraining gain may come from the architecture or protocol.
- Gate 3: baseline comparison. Under a fixed protocol and matched budget, compare against strong tabular baselines such as XGBoost, CatBoost, and FT-Transformer. Report multiple-seed mean and variation rather than a single favorable run.
Passing these gates would not prove that we have built a general tabular foundation model. It would show that the training system works, the evidence chain closes, and the first claimed transfer effect survives basic controls.
What we want the model to learn
TabU-lab sits inside the broader TabU research program. TabU studies whether Unit should be treated as a primitive rather than as another name for a sample token. That perspective raises questions that standard tabular prediction does not automatically answer:
- What is the learning unit represented by a row, group of rows, or task context?
- Which parts of a table describe observed values, and which describe roles or mechanisms?
- How should missingness differ from an unknown state or a structural null?
- What should stay invariant when the schema, task, or observed context changes?
These are research questions, not current capabilities. A strong benchmark score would not by itself validate Unit semantics, causal identification, or the broader TabU theory. Every theoretical claim will need its own formal assumptions and falsification tests.
Our starting mathematical discipline is simple: new architectural components should correspond to an explicit object in the model's account of the unit, mechanism, or noise. We want to avoid adding machinery that improves a benchmark while leaving us unable to say what the model is learning.
What we are and are not announcing
We intend to publish experiment preregistrations, model and data-processing code, versioned configurations, dataset and split provenance, commands, seeds, environments, compute budgets, raw metrics, curves, logs, checkpoints when permitted, verdicts, and reports tied back to receipts.
This is not a release of a finished model. We are not claiming state-of-the-art performance. We are not claiming that pretraining already improves tabular prediction. We are not claiming causal reasoning, universal schema understanding, or broad task generalization. Those claims require evidence that does not yet exist in the public ledger.
We are announcing that the work has started, and that the standard of evidence is being set before the first public training result.
The most useful contribution is not a large promise. It is a hypothesis with a credible way to fail, followed by a receipt that lets the next person continue.