Researchers Launch Talkie, a 13B Historical Open-Weight LLM
Researchers announced Talkie, an open-weight historical large language model that they said was trained and fine-tuned as a 13 billion-parameter model on a newly curated dataset containing only pre-1930 data. The team described it as a way to interact with a model shaped by language from the past.
One early experiment with the model asked whether a system trained only on older text could learn to code.
From the sources (5 posts)
@iruletheworldmoalec is a genius and this is incredible, take a look
@soumithchintalaTest of Time LLM from some legends...pretty fun!
@danshipperCodex is doing AI research for me on Talkie using its in-app browser research question: can an LLM trained only on 1930s text learn to code?
@sleepinyourhatRT @DavidDuvenaud: Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset o…
@karpathyRT @status_effects: New work with @AlecRad and @DavidDuvenaud: Have you ever dreamed of talking to someone from the past? Introducing talk…