# Current Research and the Road Ahead: Personality Transfer in Small Models - Author: Kojin Glick - Published: 2026-10-26 - Canonical: https://kojinglick.com/blog/current-research-and-the-road-ahead --- *This is a status report. The work continues.* ## The question I want to know what personality transfer looks like in a small language model. I train a small model on my own messages and writing. Then I measure the result. A good result has three properties: 1. The model writes like me. 2. The model keeps its skills. It must use tools and do arithmetic correctly. 3. The model does not repeat my private messages. These three properties pull against each other. This report gives the data for each property. It also tells what I cannot measure yet. ## The data I collected 26,993 records from 10 sources. The sources are iMessage, Signal, Slack, WhatsApp, Microsoft Teams, two Google Drive accounts and this blog. Each source has its own parser. An audit of the first corpus flagged 627 of 11,058 records. The problems were duplicates, broken characters, empty prompts and message reactions. I removed or repaired these records. The data contains messages from other people. Thus I apply these privacy rules: - I removed all conversations with one excluded contact. Three separate checks find and remove the rows that remain. - I removed training rows that are too similar to evaluation rows. - I do not publish samples of the data or of the model output. This report contains only methods and numbers. The main evaluation set has 421 of my real replies. All models since v9a use the same set. Thus their results are comparable. ## The models | Phase | Base model | Method | Result | |---|---|---|---| | v1 to v8 (Feb to Mar 2026) | Trinity-Mini: 26B parameters, mixture of experts, 3B active | 4-bit QLoRA on rented A100 GPUs | I tuned the training settings. I found the leakage problem. | | v9a (Sep 2026) | Qwen3.5-4B | bf16 LoRA on an RTX 5090 | Strong chat voice. The model lost tool use and arithmetic. | | v9b | Qwen3.5-4B | v9a method with three times more data | The fit was worse than v9a. | | rw1 | Qwen3.5-4B | A rewriter: neutral draft in, my voice out | It keeps all numbers. It changes too little. | | v10a | Qwen3.5-4B | 70% chat, 20% tool calls, 10% arithmetic | Tool use came back. The voice stayed only in chat. | | v10b, v10c | Qwen3.5-4B | v10a mix with more chat data; agent answers rewritten by rw1 | Not measured yet. | ## How I measure | Measurement | What it shows | How I calculate it | |---|---|---| | Bits per character (bpc) | How well the model predicts my real replies | The negative log-likelihood of the 421 held-out replies, in bits, divided by their characters. Lower is better. Unlike loss, bpc can compare models with different tokenizers. | | Surface style | If the answers look like my messages | The share of answers in all lowercase, with a final period, with casual markers and with emoji, and the median word count. The reference is 7,912 of my real replies. | | Tool calls | If the model still uses tools | 60 held-out tool tasks. I count the right function and the exact arguments. | | Faithfulness | If the answer invents facts | Each number and name in an answer must occur in the conversation or in the tool result. | | GSM8K | Arithmetic reasoning | 60 grade-school math problems. The final answer must be correct. | | Leakage | If the model repeats training data | Matches of 8 or more words, and exact matches of 40 or more characters, against the training corpus. | ## Results so far ### 1. Fine-tuning improves the fit, but more data is not always better The tuned models predict my replies about 40% better than the base model. v9b had three times more data than v9a, but its fit was worse. Most of the new data came from the export of an old laptop. That chat is casual and different from the other sources. It moved the voice. It did not make the voice stronger. ### 2. A chat-only model loses its skills v9a learned my voice from chat data only. It lost its tools. It chose the right tool in 2% of the tasks. The base model chose the right tool in 95%. v10a adds tool data and arithmetic data to the chat data. It chooses the right tool in 93% of the tasks. Its chat fit did not decrease: the bpc is 0.634. The black lines show 95% confidence intervals. With 60 tasks, an interval is about ±10 points. Thus the drop in arithmetic from 85% to 75% is not certain yet. ### 3. The voice does not transfer into agent work This is the main result. I compared the style of the agent answers with my real replies. v9a is near my replies. v10a is near the base model. In chat, v10a fits my voice. When it uses tools, it writes like a generic assistant. 0% of its agent answers are in lowercase, and 85% end with a period. Of my real replies, 63% are in lowercase and 18% end with a period. No model uses my casual markers or emoji. ### 4. No model is in the target zone yet The target zone needs a voice like mine and good tool use. v9a has the voice but not the tools. v10a has the tools but not the voice. v10c is the next test. It trains on agent answers that the rewriter changed into my voice. ### 5. Low loss does not mean safe In the v8 runs, the model with the lowest evaluation loss repeated the most fragments of personal messages. A model with a slightly higher loss repeated fewer. Thus I select models on fit and on leakage, not on loss alone. This result came from a manual review of a small set of prompts. It is not a measured rate yet. On v9a, the automatic check found no matches of 8 words in 33 samples. That sample is too small to be a guarantee. ## What I cannot measure yet - **The newest models.** v10b and v10c have no results. The v10b evaluation stopped because the disk was full. - **Personality itself.** All my style measures are proxies. Bpc measures how well a model predicts my real replies, not what the model writes. Surface style counts lowercase letters and periods. It does not measure personality. - **My own judgment.** I have not compared these numbers with my opinion of the answers. A blind rating tool exists, but I have not used it yet. - **Small differences.** The test sets are small. With 60 tasks, a difference of less than 10 points is not certain. - **Leakage at scale.** The automatic leakage check has only 33 samples. ## The road ahead I will do these steps in this order. Each step has a pass criterion. 1. **Measure the models that exist.** Free disk space. Run the agent evaluation on v10b and v10c. Increase GSM8K and the tool tests from 60 to 200 items. This decreases the interval to about ±6 points. Also measure surface style in chat answers, not only in agent answers. 2. **Calibrate the measures against my judgment.** Rate about 150 anonymous answers with one question: "Would I send this?" Then compare the automatic style score with my ratings. If the AUC is near 0.5, the score does not measure what I care about. Then I must change the measure before I train again. 3. **Test v10c against fixed criteria.** The agent answers must pass all of these criteria: - Voice: 40% or more in lowercase. - Voice: 35% or less with a final period. - Voice: a median of 15 to 30 words. - Skills: 65% or more exact tool arguments. - Skills: 80% or more on GSM8K. - Skills: 90% or more faithfulness. If only the lowercase rate changes, the rewriter is the limit. Then I will train rw2 on drafts in an agent style. 4. **Measure more than surface style.** Compare the distributions of function words, sentence lengths and word sequences with my real replies. Run the intent test. That test asks if the model understands a message, not only if it sounds like me. 5. **Measure leakage and emotional overcapture at scale.** Check about 500 generations at different temperatures. Compare the emotional tone of the output with each period of my data. This shows if the model is stuck in one period of my life. 6. **Find why more data made the fit worse.** Limit the largest source to about 30% of the data. Then report the fit for each source.