RLHF at scale across 11 Indic languages

Preference data becomes harder to produce as language coverage expands. The challenge is not only translation. Each language has regional variation, multiple registers, script choices, borrowed vocabulary, and different conventions for politeness and explanation.

For an 11-language RLHF program, the operating question is therefore not “Can we translate the English task?” It is “Can we create equally reliable preference signal in every language?”

Define the language matrix first

We begin by mapping the languages, scripts, target regions, domains, and expected forms of code-switching. This prevents a broad label such as Hindi or Tamil from hiding the actual traffic the model will receive.

The matrix also guides sampling. Formal writing, conversational support, safety-sensitive prompts, and mixed-language interactions should not be blended into one undifferentiated pool. Each slice needs enough examples to reveal systematic errors.

Recruit and calibrate native raters

Self-reported fluency is only a starting point. Raters complete language-specific qualification tasks and are reviewed by calibrated native speakers. When a task involves medicine, law, finance, or another specialist domain, language ability and subject knowledge are evaluated separately.

Calibration uses shared examples that expose subtle trade-offs: an answer may be factually correct but unnatural, safe but unhelpfully vague, or fluent in the wrong regional register. Raters discuss disagreements, and the rubric is revised before production volume increases.

Localise the rubric, not just the text

An English rubric often contains assumptions that do not transfer cleanly. Tone, refusal style, honorifics, and the expected amount of context vary by language and audience. We keep the evaluation dimensions consistent while adding local examples that show how each dimension should be applied.

Code-switching is treated as a normal conversational pattern. Reviewers distinguish useful switching from unnecessary mixing, and they flag cases where the model changes script or language in a way that disrupts comprehension.

Construct preference pairs carefully

Useful pairs should force a meaningful judgment. If one response is obviously broken, the comparison provides little information about finer model behaviour. We balance clear quality differences with closer pairs that test grounding, instruction following, cultural fit, safety, and completeness.

Raters can select a tie or escalate an ambiguous example. Forcing a preference when both answers are equally valid creates noise that later appears as unstable reward-model behaviour.

Monitor agreement by slice

Overall agreement is not enough. We track disagreement by language, region, task type, rubric dimension, and rater cohort. A low-performing slice triggers targeted recalibration or adjudication rather than a generic reminder to the entire pool.

Gold tasks and blinded overlap help detect drift, but they are not the whole quality system. Reviewer notes, escalation reasons, completion times, and repeated error patterns provide context for why agreement changed.

Deliver signal with provenance

Every preference record should carry the prompt version, response pair, rubric version, language and dialect tags, reviewer cohort, quality status, and adjudication history. That provenance allows model teams to filter, reweight, and reproduce training sets as their objectives change.

Scaling Indic RLHF is ultimately an operations problem built around linguistic judgment. Native review, localised rubrics, explicit code-switch handling, slice-level monitoring, and traceable data turn multilingual coverage into dependable training signal.