Chinmay Rawat

Overview

Linear probing is the workhorse of interpretability: train a simple classifier on a model's internal activations and see whether some property can be read out. The permissiveness is the point — and also the trap. A probe that predicts a label has not proven the model uses that label; it may be latching onto anything correlated with it.

This paper aims that critique at a specific, tempting result. ReAct traces alternate labelled Thought / Action / Observation segments, and a probe can decode which phase a token belongs to almost perfectly. That invites the reading that the model maintains an internal "I am thinking now / acting now" state. But phase is confounded by construction: thoughts contain prose and arithmetic, actions contain calls, observations contain structured results. Marker text, position in the template, and content type all move together.

An earlier one-week pilot on GPT-2 Small — the ReAct Interp repository — had already found phase trivially decodable at every layer and flagged exactly this confound; this paper is the controlled follow-up that pilot called for.

The response is a decorrelation ladder — a procedurally generated dataset that crosses every phase with mathematics, code, and prose, shuffles order and repetition, and must clear frozen lexical, position, and content-balance gates before any activation is examined. Thresholds and headline layers were committed to version control before confirmatory runs. Accuracy is then normalised against a matched natural-trace denominator into a retained-signal score, and markers are stripped rung by rung.

The result inverts the initial story, and the paper reports the collapse rather than the headline.

Overview

An LLM can describe an audience member fluently long before it can forecast one. That gap is the subject here. Given a listener's history and a candidate show, an agent will happily produce an individualised, confident judgement about whether that person keeps listening — and the fluency of the output tells you nothing about its accuracy.

The paper refuses to grade that output on its own terms. It sets three gates that must be cleared in order. Temporal validity: the predictor only sees evidence from before the cutoff, and the outcome stays hidden. Predictive validity: the agent must beat a cheap, prospective, non-LLM alternative on the same cohort and target — not merely another prompt. Construct validity: the logged event being predicted must actually measure the behaviour it is named after.

Each gate is tested and each result is reported as found. A richer representation does help some models. A calibrated statistical prior nonetheless remains the better probability estimator. Adding more historical context makes things worse, not better. And the audit then turns upstream to the label itself, examining what the platform's event log actually records.

Two ideas do the load-bearing work. Ranking (does the model order listeners correctly?) and calibration (are its stated probabilities honest?) are separated throughout, because a model can win one while losing the other. And listener-grouped cross-fitting keeps the same person out of both train and test, so repeated tasks cannot masquerade as independent evidence.

Overview

Health insurers write coverage policies — the rules that decide which claims get paid — and then find out what those rules cost once they are live and denials start landing on real patients. The premise here is that this order can be reversed.

The simulator replays a payer's historical claims against a proposed edit to the policy text, so the question "what would this rule have done?" gets a concrete answer drawn from claims that already happened rather than an actuarial estimate. A cost engine turns that replay into forecast movements in the denial rate and in per-member-per-month spend, the standard unit insurers budget in.

The public demo repository shows the shape of the pipeline from the outside. It walks three stages: structured extraction from source policy PDFs, vision-model extraction of the fields on prior-authorisation forms, and a comparison stage that puts approve/deny decisions side by side with a gap-analysis table. It ships with cached outputs precisely so the interface can be demonstrated without paying for the live extraction pass — a small, practical decision that says something about how the thing is actually operated.

The second public repository is the deployment surface: a containerised dashboard plus a numbered pipeline that populates analytics tables from claims data, wired to a build-and-deploy flow. The last stage of that pipeline turns a completed run into a written memo, so the output lands as something an executive reads rather than a table they have to interpret.

Overview

Migration is directional and relational: when conditions change in one municipality, the people who leave go somewhere specific, and where they go depends on what that destination looks like and who already went there. So the work models migration not as a per-place score but as directed flows between origin–destination pairs — dyads. Across Brazil's 5,570 municipalities there are more than 300,000 pairs with non-zero flows, and that framing captures not just how many people leave but the direction they move.

Before modelling, the team looked at who is actually leaving, grouping municipalities into quintiles by temperature, income, and electricity access. The patterns are consistent: the hottest quintile loses population roughly three times faster than the coolest; the poorest fifth declines about 2.2% while the richest gains about 2.0%; and areas without electricity show the highest out-migration. Local Moran's I shows strong spatial clustering — deforestation runs along the Amazon arc, pasture through the cerrado, and out-migration overlaps both — which is the argument for a model that respects geography. The poster is careful to call these correlations, not causal mechanisms.

Two models do two jobs. A graph neural network (GraphSAGE with attention, two layers, 64-dimensional hidden state) represents municipalities as nodes and flows as edges, letting message passing carry signal between neighbours — the piece that captures spatial and network structure. Alongside it, XGBoost with SHAP does the explaining, identifying which of 135 features move predictions most, under a 5-fold spatial cross-validation whose folds are large contiguous regions so geography can't leak between train and test. The GNN predicts; the tree explains. The features themselves are assembled from four sources that were never designed to be joined — Copernicus NDVI (a satellite deforestation proxy), the IBGE 2010 census, ERA5 climate, and IBGE SIDRA agriculture — linked by municipality code across 2005–2025.

Results are preliminary and stated as such: on held-out data the GNN reaches R² 0.453 on log-flows and 0.657 on the original scale up to the 95th percentile, with error growing on the largest flows, and it is still training. The intended payoff is a ranked list of at-risk Amazon towns to direct climate-adaptation funding, aligned to programs like PPCDAm, Bolsa Verde and REDD+. Next steps move from 2010 to 2022 census data and from correlation toward causal inference.

Overview

This is the small, honest study that comes before the careful one. The question is narrow: when a language model reads a ReAct-formatted trace as plain text, does its residual stream separate which phase of the trace each token belongs to?

The repository is structured as three escalating tiers. Tier 1 caches activations at every layer and projects them down to two dimensions to look for visible clustering. Tier 2 fits a linear probe per layer, splitting train and test by whole trace rather than by token so that neighbouring tokens cannot leak the answer across the split. Tier 3 loads a pretrained sparse autoencoder and ranks its features by how much more they fire on one phase than overall, writing out links so each candidate feature can be inspected individually.

What makes it worth showing is the README's own reading of its results. It notes that the accuracy curve is nearly flat across depth, and argues that flatness is a warning sign rather than a finding — a signal available everywhere is probably a signal that was cheap to obtain. It then lists exactly what would be needed to do better: formatting controls that hold content fixed while varying markers, and causal interventions rather than correlational probes.

That list became the specification for the preregistered Gemma-2 study that followed — the paper “When ReAct Phase Probes Look Abstract.” This pilot is that project's precursor: the quick, exploratory pass whose weaknesses it was built to fix. The scope-boundaries section states the limits up front rather than in a footnote.

Overview

Classic gravity and radiation models predict how many people move between two places from population and distance. Their modern deep-learning successors — Deep Gravity, Deep Radiation — fit that relationship with neural networks but keep two blind spots: they ignore spatial correlation between neighbouring regions, and they treat climate as just another input, leaving any interaction between climate and a region's economic circumstances implicit and uninterpretable.

This co-authored paper proposes the Spatially-Gated Deep Graph Network (SGDG) to close both gaps on Mexican internal migration. The spatial gap is handled by feeding each municipality not only its own features but the spatially-lagged features of its neighbours (an SLX design, a single hop over the 15 nearest municipalities), so a prediction is informed by its surroundings. The climate gap is handled by a deliberate architectural choice: the climate variables bypass the encoder and enter through a coefficient conditioned on the municipality's income percentile — a logistic gate, in the spirit of feature-wise linear modulation — so the model can express, and then expose, how a climate shock's effect on migration depends on how poor a place is.

The dataset is its own contribution: a complete directed panel of migration flows harmonised from ten institutional sources across Mexico's municipalities, with flows modelled as overdispersed counts under a negative-binomial likelihood. Splits are stratified by GDP per capita and made at the municipality level to avoid leakage, with negative sampling for the many zero-flow pairs.

The result is competitive where it counts. Against Deep Gravity and Deep Radiation, SGDG wins on the common-part-of-commuters metric and on the log-scale error and R² that reflect the small flows making up most of the data — the gravity model still fits the largest flows better on raw RMSE. The interpretable gate reads out a substantive finding: climate weighs more heavily on out-migration from poorer source municipalities than richer ones. This is the Mexico counterpart to the Brazilian-Amazon migration work elsewhere in this town — the same research group, a different country, and a gated architecture.

Overview

The pull requests here share a recognisable shape: someone hit a specific, reproducible defect while using a tool for real work, then went and fixed it upstream instead of routing around it.

Two of the merged fixes are in TransformerLens, the standard library for mechanistic interpretability research — the same library used in the Gemma-2 phase-probing work. One concerns a model adapter that ignored an architectural flag, causing a hook point to go missing for an entire family of checkpoints; it was filed as a bug report first and fixed in a follow-up. The other stops the tokenizer wrapper from prepending a beginning-of-sequence token when the underlying tokenizer has none. Both are the kind of quiet correctness bug that silently corrupts downstream experiments rather than throwing an error.

The marimo contribution follows the same two-step pattern of report-then-fix, replacing a bare Python KeyError with an actionable message. Three OpenClaw fixes are merged — one for prompt caching breaking on specific model versions (a failure that costs money and latency without announcing itself), plus two infrastructure fixes disclosed as AI-assisted. A merged pylint change refines its too-many-ancestors check.

The open pull requests span PyTorch's FX graph visualiser and checkpoint loading, LiteLLM's auth error codes, Prefect's client connection retries, vLLM's multimodal fallback, and Hermes' agent configuration. Several PRs are unmerged or closed, which is ordinary for drive-by contributions to large projects and is recorded here as such rather than glossed.

Overview

A penetration test is a loop, not a checklist. You look at what a site exposes, form a hypothesis about where it is weak, try something, read what came back, and let that reshape the next attempt. Automated scanners historically struggle here because they run a fixed battery of probes and cannot follow a chain that only becomes visible two steps in.

Phantom's design puts an LLM in that loop. A headless browser crawler first maps what the target actually exposes — pages, endpoints, forms, technologies — because you cannot reason about an attack surface you have not enumerated. The agent then runs a ReAct cycle over that map: reason about what to try, act, observe the response, and revise. The vulnerability classes it targets are the common web ones, and several of them are precisely the kind that need multi-step chains rather than single probes.

Findings come back scored on the standard industry severity scale with reproduction steps, which is the difference between a list of alarms and a report someone can act on. The interface streams the agent's intermediate reasoning live as it works rather than presenting only a final verdict — the right choice for a tool whose trustworthiness depends on a human being able to audit how it reached a conclusion.

Phantom was built at HackIllinois 2026 by a team led here, ships as a working live demo, and turns a cold target URL into a ranked, CVSS-scored report in under a minute — measured, not aspirational. It is meant for authorised testing of targets the operator owns or has permission to test.

Overview

Translation and localisation are different problems, and the gap between them is where adapted media usually fails. A literal translation preserves the words and loses the story: a joke that depends on a pun stops being funny, a character name carries the wrong associations, a setting that signalled something specific to the original audience signals nothing to the new one.

The stated design goal here is the harder one. The service takes a full script and produces a version for a different language in which names, settings, and references have been adapted to the target culture rather than carried across intact. That framing — adapted, not translated — is the whole specification, and it is what makes the problem well suited to a language model in the first place: the decision about whether a reference should be replaced, and with what, requires knowing what it was doing in the original.

Two things about the scope are worth stating plainly. It operates on full scripts rather than isolated lines, which means consistency has to hold across a long document — a name changed in one scene has to stay changed, and a running reference has to keep working. And it is described as shipped and in use on the platform, which places it in a different category from a prototype: it has real users with editorial standards, on real content.

Overview

Deciding what to publish is a forecasting problem with an expensive error term. Launch something that does not find an audience and you have spent production budget and a slot in the catalogue; the cost of a wrong call is concrete.

The approach combines two model families that fail differently. A language model fine-tuned with a lightweight adaptation technique can read the content itself — the text, the premise, the things that make a story work or not — and pick up signal that never appears in a structured table. Gradient-boosted trees and neural networks read the tabular side: the metadata and behavioural aggregates that describe how comparable content performed. Ensembling them is a hedge, because the cases where a text model is confidently wrong are not usually the cases where a tabular model is.

The engineering constraint shapes the design as much as the modelling does. Serving this over a large daily stream means the models cannot live in a notebook; packaging them as containerised microservices is what makes them callable by the rest of the platform and independently deployable.

Note what the two accuracy-shaped numbers in the record actually measure. One is a change in launch outcomes; the other is the classifier's accuracy at predicting them. They are different quantities and the record does not connect them causally.

Overview

College admissions advice is a retrieval problem wearing a counselling problem's clothes. The information a student needs is specific, scattered, and changes yearly — deadlines, requirements, what a given programme actually looks for — and it is exactly the kind of thing a general language model will answer confidently and wrongly.

Retrieval-augmented generation is the standard response to that failure mode. Rather than relying on what the model absorbed during training, the system looks up relevant source material at question time and answers from what it retrieved. The answer stays anchored to real documents, and updating the knowledge means updating the documents rather than retraining anything. Pairing that with lightweight fine-tuning addresses the other half: retrieval controls what the model knows, fine-tuning shapes how it responds.

The latency target in the record is a product decision more than a technical one. A conversational tool that takes long enough for the user to notice stops feeling like a conversation, and for a retrieval system that budget is genuinely constraining, because the lookup happens before generation starts. Hitting it means the retrieval step cannot be an afterthought.

The record for this project is short: a distribution figure, a partner count, and an outcome. The work is dated to a period before all the other projects here.

Overview

When a coding system is decomposed into several cooperating agents, one common safety design is to put a monitor on the channel between them: another model that reads what the sub-agents say to each other and flags anything it should not see passing through.

The question this work poses is whether that monitor actually holds. A channel that carries arbitrary natural language is a channel with enormous capacity for encoding things it was not meant to carry — meaning can be hidden in phrasing, in ordering, in choices that look innocuous individually. A monitor that reads for obvious leakage may pass all of it. This is an adversarial evaluation problem: the honest way to find out is to actively train and stress-test agents that try to get data past the monitor, rather than to assume good behaviour and check for accidents.

Two pieces have to be built to ask the question at all. An agent harness — the scaffolding that runs the multi-agent system, defines what each agent can do, and carries messages between them. And a monitor evaluation, which is the harder half: you have to measure what the monitor catches and what it misses, which means constructing cases where you know the ground truth about whether information was actually leaked.

A connected thread, developed as a research proposal with Prof. Hao Peng, asks a more basic question about systems like this: when an agent writes down its reasoning before it acts, is that reasoning what actually drives the action — or fluent narration alongside a decision the model already made? It matters because oversight of agents, the monitor above included, works by reading what agents say. The direction measures this at the moment an agent commits to an action, and treats the harness — the scaffolding around the model — as something to evaluate by what it does to the decision, not only by task success. In the two-agent case the question sharpens to the handoff: when one agent passes work to another, does a flaw in what crosses between them propagate into an unsafe action?

The work is early and the proposal is a private draft; it is described here only at a high level.

Overview

Coding agents are unusually sensitive to the environment they run in. Faced with a new issue in an unfamiliar repository, most of what an agent needs — the conventions the codebase already follows, the way a similar problem was solved before — is sitting in the repository itself, but a single pass tends to skate over it. The premise of this work is that letting an agent interact with that environment over many steps, rather than answering in one shot, can surface knowledge about the task that a single attempt misses.

Two engineering ideas carry it. The first is partitioning: rather than one agent reasoning over a whole repository at once, a task is split so that sub-agents each reason over a smaller, focused slice, which makes it easier for each to reach a correct, well-grounded conclusion. The second is orchestration: those sub-agents’ findings then have to be recombined and scaffolded back together into a single coherent solution.

The design constraint is cost. Exploration is only worth doing if the user does not pay for it in latency, so the intent is to run it in the background — in downtime, ahead of when it is needed — rather than on the critical path of a request. The project is early and deliberately provisional: approaches are expected to change as it is tested.