Sarah Catanzaro (@sarahcat21) started following @yaroslavvb on Oct 16, 2024

The point of "bitter lesson" is that simple things often work at scale. You can "muscle your way" past bad algorithms by scaling up compute. But if you fix your compute, the only way to scale up is through algorithms. Epoch AI found that in the last 12 years of LLMs, our

In the last 5 years, we mostly figured out how compress the knowledge of 100B people who ever lived into some matmuls, "System 1 thinking". What's next? 1) Collect more knowledge. But that takes robots which we don't have. 2) Come up with arch for "System 2 thinking". But that's

RT @Azaliamirh: AlphaChip is now used to design layouts for chips across Alphabet and externally, and has been extended to various stages o…

RT @annadgoldie: Here are some external perspectives on the work! 🙏 “AlphaChip’s groundbreaking AI approach revolutionizes a key phase of…

Inspired by a coworker's lolcat explorations and @goodside 's standup explorations...presenting a lolcat standup routine of Seinfeld. The ailuromorphic (yes, had to look that up!) characters are great, but coming up with "cat flap" took the cake for me! https://t.co/y1kscELewb

RT @Coltography: The simplest video inputs can turn out so wild 👽 #aivideo #stablediffusion #stablewarpfusion #ai https://t.co/FgrGLMYXix

The countries with the best food in the world https://t.co/vxh80SyWuB

desperation makes you feel alive

Is this how this linkedin thing works? https://t.co/LRhQ9ZrLWV

The state of academia.

And for a brief moment, they stopped arguing about model reasoning and existential risk and just marveled over reusable rockets 😌

After watching Camilla for the last few months, I’m convinced that LLMs just need to learn to put things in their mouth. Apparently, oral exploration is the key to self improvement and general intelligence.

The deep dives are done, and the results are in. How did the table formats compare concerning their change query support? I've updated the change query blog post with a summary of my findings. https://t.co/ldzMGKFwjo

Fizzbee, the formal verification project I've been using recently, now has simulation mode! I can now run it over much larger state spaces. I ran it on my Apache Iceberg spec and found one bug in the spec. I ran it on my Apache Paimon spec and found 6 spec bugs over 2 days! So

Database deployment models are shifting at a nutso pace. Was so interesting to hear how companies are adapting to customer preferences that shift faster than Gen Z deciding between Sabrina Carpenter and Olivia Rodrigo…

And the dirty secret is how much $$$ folks are spending on annotations of dubious quality.

Pretraining isn’t dead. We are just spending even more $$$ on post-training.

As more teams build genAI and other data intensive applications, they will need to collect more data. As they collect more data, maintaining quality will become increasingly challenging. But it doesn't need to. Datafold can help companies adapt.

if you use text embeddings, feel free to try cde-small-v1 on HuggingFace: https://t.co/t7wRKFh1jW as noted, it's slightly more involved to use, since there's an extra step of embedding context tokens beforehand but the model works really, really well https://t.co/VEQg6rJLZC

Finally, if you read our paper, let me know what you think 😎 (planning on making an update next week with a smaller figure and a few more details...) https://t.co/ehDgC9KcAU

Remember when companies used to build before they launched? ::sighs::

For years, data teams have been opining about ad hoc requests from colleagues for things they don’t really need…while making so many ad hoc requests for startups/from vendors for things they don’t really need

A man’s home is his castle

:( Guts

I'll vote for whichever candidate first dresses up as goku and fights this Majin Buu