trace-to-training-data – skill Dữ liệu & Khoa học cho agent AI

Trace To Training Data

This skill assumes eval-harness-first

already graded the traces being

converted here — goldens, graders,

and runs/<run-id>/results.json

all exist before conversion

starts. This is the flywheel edge

that skill names in its own flow:

"the same labeled traces become

the training set." Conversion

happens here; grading already

happened upstream.

Input: graded traces —

eval/goldens.jsonl plus

runs/<run-id>/results.json, each

row carrying a task_id, a

verdict from the grader, and a

reward when the task supports a

scalar score (judge score,

execution partial-credit, or an

RLVR verifier):

{"task_id": "t-042", "trace_id": "t-042-a3",
 "messages": [{"role": "user", "content": "..."}],
 "verdict": "pass", "reward": 0.91,
 "grader": "exact_match"}

Output format: rows shaped

exactly like dataset-curation's

Format Selection table — SFT

messages rows or DPO

prompt/chosen/rejected

pairs — so this skill's output is

that skill's input with no

reshaping step in between.

The Principle

The eval harness already did the

labeling work: every trace in

results.json carries a verdict,

and often a reward, before this

skill ever touches it. Converting

a graded trace into a training

row is mechanical — pick a shape

from dataset-curation's table,

map fields, write JSONL.

**Curation is the work that

remains** — which traces clear a

quality bar, which pairs are

informative, and which rows must

never enter the training set at

all.

Treat any conversion step that

requires re-judging a trace as a

sign the harness is missing a

grader, not a gap this skill

should paper over. A trace with

no verdict or reward isn't

convertible yet — route it back

to eval-harness-first first,

don't hand-label it here to

unblock conversion.

SFT From Traces

• **Keep the top-reward fraction

of successful trajectories**,

not every passing one. Rank

passing traces by reward and

take a fraction (the

Agent-lightning pattern) rather

than every trace that merely

cleared the pass bar — a trace

that barely passed is a weaker

SFT signal than one that scored

well above threshold.

• **Expert-corrected failures

become gold SFT examples

directly** (the Langfuse

pattern) — when a human edits a

failing trace's output into a

correct one, that correction

needs no reward threshold; a

human already validated it.

Route corrections straight into

the SFT set.

• **Step-level masking beats

whole-trajectory discard for

multi-step traces.** When only

some steps in a multi-step

trajectory are bad, mask the

loss on the bad steps and keep

the good ones, rather than

discarding the whole trajectory.

SRFT reports 32.2% vs. 30.9% on

SWE-bench for step-level critic

masking over trajectory discard

— a real, if modest, gap from

the finer-grained cut.

Preference Pairs From Traces

• **Build pairs from

passing-vs-failing trajectories

on the SAME task**, never from

unrelated best- and

worst-scoring traces pulled

across different tasks —

cross-task pairs teach the

model to prefer one task over

another, not one response over

another.

• **Select the rejected member at

μ−2σ of the reward distribution

for that task, never the

absolute minimum.**

preference-optimization's

Pair Construction section owns

the full selection formula;

this skill supplies the graded

trajectories it consumes.

• **Judge-scored delta selection

cuts pair volume without

cutting signal.** Score each

candidate pair by

chosen-minus-rejected judge

delta and keep only the

highest-delta subset — the top

5k of a 16.5k candidate pool

matched the full pool's

downstream result. Build the

full candidate set first, then

filter by delta; don't cap

generation at 5k up front.

Hygiene

• **Scan for secrets and PII before any row ships,

and redact what's found.** Traces sourced from

production logs can carry credentials, API keys,

tokens, or customer data — run a secret/PII scan

over every SFT and DPO row and redact matches;

conversion fails closed (the row is dropped, not

shipped with the raw content) if sensitive fields

remain after redaction. Never commit secrets.

• **Eval goldens must never leak

into training data.** Hold

every eval/goldens.jsonl ID

out of every converted SFT and

DPO set — a trace that also

appears as a golden trains on

the exact item the checkpoint

gets graded against later,

silently inflating every

subsequent eval run.

• **Dedup against the training

set**, not just within the

newly converted rows —

exact-match or

embedding-similarity, matching

dataset-curation's dedup

method field, run against

whatever training data already

exists before this batch merges

in.

• **Provenance goes into the

dataset card.** Every converted

row must trace back to its

source run_id and trace_id

— dataset-curation's

Provenance field checks for

exactly this link back to

trace-to-training-data

output; a row with no traceable

source isn't ready to merge.

Related Skills

• eval-harness-first — produces

the graded traces this skill

converts; a trace with no

verdict or reward isn't

convertible yet, route it back

there before conversion.

• dataset-curation — owns the

target formats and the dataset

card this skill's provenance

data feeds; converted rows must

match its Format Selection

table field names exactly, not

an approximation of them.

• preference-optimization —

consumes the DPO pairs this

skill builds and owns the full

μ−2σ rejection-selection

formula referenced above.

Worked JSONL-to-JSONL conversions

— graded trace to SFT row, trace

pair to DPO pair, correction to

SFT row, the rejection-sampling

loop, and the goldens-holdout

check — live in

references/conversion-recipes.md.