R&D DOCS 02 STUDIES
RESEARCH.
SCRIPT NATURALNESS STUDY
At Pagard AC, an attempt to turn my lead's "doesn't sound natural
spoken" rejections into a testable standard. I evaluated the judges
first; when they failed, I built the standard from the national
corpus. How it fed the rulebook and automated checks is on
Work ↗.
01 / EVAL SET · JUDGESPagard AC
Before trusting a judge, I evaluated the judge
Pagard AC · Sep 2026 – present · Can a model judge my lead's "this doesn't sound natural spoken" rejections?
EVAL SET · 22 ORIGINAL → EDITED PAIRS
- Extracted the 22 edits I made in the 8th script revision as pairs
- Edit types labeled from document markup: strikethrough = delete · red = replace · green = add
- e.g. 손익비가 나빠지다 → 낮아지다 · 알아보기 어렵다 → 복잡하다
- Metric: how often the judge picks the side I picked; chance = 50%
POSITION BIAS CHECK
Asked each pair twice with A/B swapped → up to 98% of answers went to one slot. A judge that picks by position, not content, is out.
Fallback: count real usage → in 173 company scripts + 53 subtitle files, every pair with examples favored the edit — but 5 of 10 pairs had zero examples → 02
JUDGE AGREEMENT · AXIS 40–90%
Morphological LM (Kiwi) 50%
11 / 22
HyperCLOVA X SEED 0.5B 55–59%
12–13 / 22
5 large LLMs × 2 conditions = 10 combos 46–58%
HCX-005 · HCX-DASH-002 · HCX-007 · Solar Pro 3 · 4
Chance 50% Pass bar 75%
Verdict: all 12 judges fell short of the bar → build the standard from data, not a model
02 / CORPUS · GOLD · RULESPagard AC
I filed an official request to get the data
Modu Corpus → dictionary → 99 hand-built gold lines → rules → validation · error analysis
Official request
Modu Corpus raw, 8.2 GB
Morphology (MP)
Spoken + written, 336K words
Dictionary build
31K word forms · 17K endings
Gold data
99 subtitle lines split by hand
Rules · validation
Recall 0.77 · precision 0.76
Splitting by speech units instead of character count removed breaks like '알려|주는' and '객관적인|경고'.
Split guide (from gold)
- Always break after punctuation
- May break after adnominal clauses, -아/어/여 connectives, and subjects
- Never split auxiliary verbs, bound nouns, compound nouns, quotations, after '-의', or verbs that can't stand alone
- Ideal line 11 chars, long-line cap 15, tuned to the gold set (avg 12.1 vs gold 12.0)
Error analysis · limits
- Logged 4 misses by cause — e.g. ambiguous one-syllable endings, a lookup where '적' swallowed '적용'
- Recall and precision read together — never one agreement score
- Limits written down: gold covers one script, first 56% — "the sample is thin"
Cleaning · comparison
- Company script pool: fixed same-name overwrites → 181 → 173 scripts (467K chars); differing versions kept apart
- Compared 46 subtitle files from 12 trading channels on 9 axes (ending ratios, rhythm, question rate…) to pick the reference closest to the target voice
BLUE LAB ACTOR LORA PIPELINE
Company work at The Blue Studio’s BLUE LAB (March 2026): an
actor-identity LoRA pipeline. The documents, scripts, and numbers
are published with the company’s permission; actor names,
source-photo filenames, and dataset paths are removed (the actor
dataset stays private). Every choice was structured first as a
written document, then implemented in code (written with Claude
Code), then verified with a benchmark. Public copies are in
English. Click any title to open the full document.
00
Pipeline Manual ↗ OPEN Top-level execution map · 12 KB
Single-page index that maps the nine pipeline stages to specific scripts, lists key results (model, optimal checkpoint per character, ArcFace deltas, optimal CFG/Steps/Shift), and shows the file tree so anyone joining the project can orient in one read. Anchor doc.
CONTAINS - ▸ 9-stage flow chart from data prep to overfit watch
- ▸ File-to-stage mapping table
- ▸ Key results: male actor AVG 0.732 at 2200 steps · female actor AVG 0.692 at 1500 steps
- ▸ Final hyperparameters: CFG 3.0, Steps 70, max_shift 0.5
01
Dataset Bible ↗ OPEN Constructive principles for realistic-person LoRA datasets · 47 KB
The longest document in the project. Synthesizes consensus from FPHam (500+ LoRAs trained), djdante's verified Klein 9B run, CivitAI threads, and Reddit/SD community findings. Distinguishes what to keep constant (identity) versus what to vary (angle, distance, expression, lighting, wardrobe), and explains why caption-only diversity cannot replace image-level diversity.
CONTAINS - ▸ Image count guide: 10-30 optimal, 50+ degrades
- ▸ 'Identity constant, everything else varied' rule
- ▸ Caption mechanics: tagged features become conditional, untagged become identity
- ▸ Common pitfalls (statistical bias from overly-uniform backgrounds)
02
Training Strategy (djdante-based) ↗ OPEN Verified reference setup adapted for the two actors · 27 KB
Built on djdante's published Klein 9B training config (29 chest-up images, simple captions, ~4000 steps). Documents the trigger word strategy, caption philosophy, hyperparameter choices, and the deliberate departures we made for the two-actor project on a rented RTX 5090.
CONTAINS - ▸ Trigger word convention: trigger + descriptive caption
- ▸ 29-image guideline tested for chest-up identity capture
- ▸ AdamW · bf16 · qfloat8 quantize · EMA decay 0.99
- ▸ Reasoned departures from djdante where our use case differed
03
ArcFace Benchmark Results ↗ OPEN Full 3000-step experiment log · 19 KB
Per-checkpoint ArcFace identity similarity, measured every 100 steps for both actors. Picks the optimum at 2200 steps for the male actor (AVG 0.732) and 1500 for the female (AVG 0.692), beating the commercial baseline (Higgsfield Soul 2.0) at 0.5090. Documents the underfit / optimal / overfit transition shape.
CONTAINS - ▸ Male actor optimum: Step 2200, AVG 0.732
- ▸ Female actor optimum: Step 1500, AVG 0.692
- ▸ LoRA 0.6050 vs commercial baseline (Higgsfield Soul 2.0) 0.5090 (+18.9%)
- ▸ Curve shape evidence for 'pick balance, not max' rule
04
Pipeline Execution Report ↗ OPEN End-to-end run report (female actor v3) · 14 KB
Single-run write-up with timestamps, decisions, deviations from plan, and final selected checkpoint. Functions as both a record for future runs and a reproducibility note for anyone wanting to repeat the experiment.
CONTAINS - ▸ Timestamped run log
- ▸ Decisions and their evidence
- ▸ Final selected checkpoint and why
- ▸ Re-run instructions
05
LoRA Fine-tuning Settings Comparison ↗ OPEN Side-by-side configuration matrix · 22 KB
Cross-tab of training configurations across runs. Surfaces the tradeoffs between learning rate, optimizer, EMA, batch size, gradient accumulation, and quantization choice. Used to justify the final picked config over alternatives.
CONTAINS - ▸ Configurations matrix (lr, optimizer, batch, EMA)
- ▸ Tradeoff annotations per setting
- ▸ Justification for picked config
06
Dataset Guide (Web Document) ↗ OPEN Operational checklist for actor onboarding · 26 KB
Companion document distilling the dataset bible into a checklist for actor onboarding sessions: pose list, lighting requirements, wardrobe variety, do/don't examples. The on-the-ground operations layer above the bible.
CONTAINS - ▸ Pose checklist for shoot day
- ▸ Lighting / wardrobe requirements
- ▸ Do / don't examples