I Kept a Diary for Seven Years. An LLM Finally Read It.

I have been maintaining an online diary since 2019 — documenting my life, targets, frustrations, and everything in between. In 2023, I switched from Evernote to GitHub. I exported all my notes to markdown and started using git commits to track new entries in my life.

I have been trying to use LLMs on those markdown files ever since. Privacy for my personal notes has been a major constraint. I could never really use ChatGPT. So, I had to experiment a lot.

I started with building a RAG application over my notes. To ensure privacy, I used GPU instances on AWS. I asked questions like “What are some negative patterns about this person?” and “What are some good habits of this person?” This would return chunks from different time periods, with each chunk carrying its date information. For higher-level questions like “How has this person evolved over time?”, I would ask them over the answers of the earlier questions. But the analysis was never on point. Retrieval would surface disconnected chunks of my life and any synthesis on top of them was always shallow.

The next evolution was reasoning models, so I tried Deepseek-R1. I concatenated my most important notes and passed them to Deepseek-R1 on my GPU instance on AWS. At that time, AI started being useful for self-analysis. Deepseek-R1 could not hold the entire diary in its context but it reasoned well over the parts it could see. I always read the reasoning traces more carefully than the final answers because they contained more honest signals. They showed how a third person would interpret my problems whereas the final answers often felt like an LLM trying to placate me.

I also tried finetuning LLMs on my diary. I wanted to see if my data makes an LLM more empathetic or improves their reasoning. Privacy was still a restraining factor for me. To create training data, I was using Ollama on my Mac Mini and using distilled Llama 3.1 models to create instruction-based training data. Since I was asking Llama to come up with a question whose answer would come from my diary entry, the training data ended up with bias from Llama rather than insights from my notes. I realized I wasn’t training on myself, rather I was training on Llama’s interpretation of me.

I still hadn’t gotten the answer I was looking for, “How have I changed over the years?”

I would need a powerful LLM for that and a harness that could orchestrate the flow analyzing 200+ markdown notes spread across years. Claude Code was an obvious candidate for the harness. But it didn’t really work with smaller models like Qwen and Gemma. Claude Code always stopped midway with these models even before hitting their context limits. The solution was to use Claude Code with Anthropic API instead of Claude.ai subscription. Anthropic API guarantees deletion of data after 30 days unless the data is flagged against usage policy, which is still a grey area because users are never notified if a conversation gets flagged at storage time.

So, I had a choice. Trust the general goodwill of people at Anthropic or wait another 2-3 years for small language models to become capable enough to get my answers.

I decided to go ahead with Claude Code with Anthropic API. And, it was a good decision. First off, I wouldn’t be writing this post if not for the answers I got from Claude. To give you a glimpse, this is what Claude told me:

1. You’ve been working on the same problem for 7 years without realizing it. Your interest in RAG, citation graphs, knowledge retrieval, and agentic memory isn’t scattered curiosity — it’s one question asked in different forms: how do systems find and use knowledge?
2. You rediscover the same lessons roughly every two years. The same principles — focus on fewer things, protect your energy, finish before starting something new — appear across years as if encountered fresh each time. Insight without an enforcement mechanism doesn’t stick.
3. Your peak periods were documented but never reconstructed. You wrote down in detail what clarity felt like when you had it. But you never reverse-engineered the conditions that produced it. You treated those states as weather rather than something buildable.
4. The vocabulary for your inner life got sharper even when the behavior didn’t change. Early entries describe events. Later entries have a taxonomy — specific frameworks for why a mental state is happening. That’s real growth, even when it doesn’t look like it from the inside.
5. The most important sentence in 7 years of writing was an admission of defeat. Accepting you wouldn’t become what you had imagined changed the texture of everything written after it — less fantasy, more presence.

To make sure Claude wasn’t telling me what I wanted to hear, I asked it to cite the specific entries behind each insight. Here’s one example — Insight #2, the claim that I rediscover the same lessons roughly every two years:

Lesson: “Only 2 things at a time“- don’t pursue more than two goals in a given period.

1. 2019 (190603): Naval quote — “PICK YOUR ONE OVERWHELMING DESIRE, let all others go so you can be calm.”
2. 2021 (210920): “ONLY TWO THINGS at a TIME. Do only two things at a time. If you are taking more things up then use 80-20 rule.”
3. 2024 (241021): “I have too many things on plate… Everything is important for me everytime.”
4. 2025 (250707): “I constantly failed in my tasks because I added more sub-tasks.”

I will continue to be my own dataset.