td

thoughts on ai, philosophy, books & entrepreneurship

AI is beautiful and some notes

ai change revolution
There is a huge disparity between people who see what is happening and those who don't.

I see two good choices to pick from:

- Build AI start-ups (learn by doing).
- Become a researcher with focus on implementing AI into a discipline (e.g. Biology + AI).

World is changing so quickly.

It's insanity. It's beautiful.
____

Notes from the a great podcast episode (Ilya Sutskever):

Ilya frames everything around this pendulum between research and scaling. 2012-2020 was about ideas. Transformers, attention mechanisms, new architectures. Then 2020-2025 became about making those ideas bigger. More compute, more data, more parameters. Now the pendulum swings back. We need new ideas again.

In the process of generating new ideas, we should ask: "How far back do we rethink?" Everything is connected. Decisions from years ago still constrain what's possible now.

First principles are the essence of new opportunities, specifically how we train models. 

If humans can do something, that's proof machines can too. We just need to understand the mechanism. One thing humans have is meta-awareness. We know how good we are at things because society calibrates us. 

A small thought I got here is that: Maybe training is overly linear (even tho we think it's not). AI still can't learn the "unexpected twists", the way life gives them.

In the Silicon Valley it says: "Ideas are cheap, execution is everything." But if ideas are so cheap, why is no one generating them? Right now there are more companies than ideas. Everyone executes on the same few concepts.

Couple of important principals: 

Humans as blueprint not metaphor; diversity between agents; self-play; analogies across domains. 

Philosophy might matter more than we think. It's been wrestling with consciousness and agency for millennia.

And as AI gets more capable, paranoia rises, which forces security to be taken even more serious.