Answering in the language of the question
A useful multilingual answer needs more than a translated surface. It needs to respond to the question, preserve the relevant information, and use the language expected by the person asking. This research explores that problem for maternal, sexual, and reproductive health questions in Akan/Twi, Luganda, Kiswahili, Amharic, and English subsets.
The work is organized as Colab experiments. The focus is understanding how pretrained models and adaptation behave across language subsets, with a repeatable path from a baseline to generated answers and error analysis.
Baselines before adaptation
The repository compares Gemma and Aya model baselines and includes a Gemma LoRA fine-tuning experiment. The baseline notebooks establish the data-loading, prompting, inference, and prediction-export workflow. Prompts include a language cue and ask for a concise answer in the same language as the question.
The adaptation notebook builds training examples from question-and-answer pairs, attaches LoRA adapters, and saves the adapter separately from the base model. This keeps the experiment artifacts manageable while making the adaptation step explicit. A sampled training run provides an initial way to investigate behavior before committing to a broader experiment.
Generation is checkpointed throughout. Long notebook sessions can be interrupted, so completed answers are saved incrementally instead of depending on a single uninterrupted run. Prediction export checks identifiers and follows the expected submission order.
Looking beyond one aggregate score
Local evaluation uses ROUGE-1 and ROUGE-L to compare generated answers with reference answers. Results are grouped by language subset, and low-scoring examples are collected for inspection. The challenge also defines an LLM-based assessment; local lexical scores represent only part of that evaluation.
This distinction shapes how I read the results. Overlap can reveal omitted terms or substantial differences from a reference, but it cannot establish that a health answer is correct. Per-language inspection is also necessary because an aggregate result can conceal a weak subset.
Research outcome and scope
The project provides a set of comparable baseline and adaptation notebooks, checkpointed prediction files, and a workflow for examining errors by subset. The fine-tuning experiment currently uses sampled validation; broader validation remains part of the research. The work is an experimental question-answering study, with no deployed clinical service.
The notebooks and experiment notes document the models, prompt setup, and evaluation workflow.