23 Jan 2025 #ai #ainews #agi Google TITANS and Sakana Transformer². AI gets memory and can keep learning after training. #ai #ainews #agi

Thanks to our sponsor Abacus AI. Try their ChatLLM platform here: http://chatllm.abacus.ai/?token=aisea…

Titans: Learning to Memorize at Test Time https://arxiv.org/abs/2501.00663 Sakana Transformer² https://sakana.ai/transformer-squared/

0:00 Intro 0:34 Google Titans architecture 1:44 Transformers model and limitations 5:14 Giving AI memory 6:35 AI training vs test time 8:57 Titans long term memory 12:59 Titans architecture 19:50 Pros and cons 24:00 Performance of Titans models vs Transformers 27:31 Sakana Transformer2 30:50 Model design and rationale 32:24 How Transformer2 works 36:00 Transformer2 performance 38:12 Conclusion

Newsletter: https://aisearch.substack.com/ Find AI tools & jobs: https://ai-search.io/ Support: https://ko-fi.com/aisearch

Here’s my equipment, in case you’re wondering: Dell Precision 5690: https://www.dell.com/en-us/dt/ai-tech… GPU: Nvidia RTX 5000 Ada https://nvda.ws/3zfqGqS Mouse/Keyboard: ALOGIC Echelon https://bit.ly/alogic-echelon Mic: Shure SM7B https://amzn.to/3DErjt1 Audio interface: Scarlett Solo https://amzn.to/3qELMeu