Indic RLHF: lessons from 11 languages
How dialect calibration, code-switch tagging, and rater pools changed our reward-model behavior across Indic languages.
Preference data for Indic languages cannot be produced by translating an English workflow. Each language brings its own dialects, scripts, registers, and patterns of code-switching. Those differences affect both the examples shown to raters and the way preferences should be interpreted.
Recruit for lived language use
Self-reported fluency is not enough for nuanced comparison work. Raters should be calibrated by native speakers and matched to the regions and registers represented in the data. A formal-language speaker may not be the right reviewer for colloquial or mixed-language conversations.
Treat code-switching as data
Users often move naturally between languages within a sentence or conversation. Labelling this behaviour as noise removes an important part of the task. We tag language spans, script changes, borrowed terms, and the conversational reason for switching when that context affects quality.
Localise the rubric
Helpfulness, politeness, and safety are expressed differently across cultures and domains. Rubrics need examples that show the expected register, forms of address, refusal style, and level of explanation in each language. Translation alone can preserve words while losing the intended behaviour.
Measure agreement by slice
An overall agreement score can hide problems concentrated in one dialect, task type, or safety category. We review agreement by language, region, prompt domain, and error type. Low-agreement slices trigger guideline changes and fresh calibration before more data is collected.
Separate model failure from rubric failure
When raters disagree, the model output is not always the cause. The prompt may be ambiguous, the rubric may combine conflicting goals, or both answers may be equally valid in different dialects. Adjudication should record why a preference was difficult instead of forcing false certainty.
Multilingual alignment improves when the workflow respects language as context, not just text. Native review, explicit code-switch handling, localised rubrics, and slice-level analysis produce a stronger signal for both training and evaluation.