TALKS
S
S
U
U
B
B
S
S
C
C
R
R
I
I
B
B
E
E
T
T
O
O
N
N
E
E
W
W
S
S
L
L
E
E
T
T
T
T
E
E
R
R
TALKS
S
S
U
U
B
B
S
S
C
C
R
R
I
I
B
B
E
E
T
T
O
O
N
N
E
E
W
W
S
S
L
L
E
E
T
T
T
T
E
E
R
R
A
A
L
L
L
L
T
T
A
A
L
L
K
K
S
S
Keynote
The End of the Internet As We Know It
Description Coming Soon
Speakers
Raffi Krikorian
, CTO, Mozilla
View transcript
00:00
So, I think it's possible that some folks have not read the op-ed yet, even though it's
00:08
on New York Times, which is obviously a paradise for geeks, but I know it also showed up on
00:13
Hacker News and other places, so I'm sure they'll be able to find that after the chat.
00:18
But just give us a quick rundown of the main idea of the post, because you mentioned that
00:23
the detente is over and that there was this informal equilibrium that we had that kept
00:29
the Internet safe enough to use, because software engineering was hard and security
00:34
was hard.
00:35
And so the opaqueness of those two things sort of masked or hid a lot of vulnerabilities,
00:44
and that period appears to be over.
00:46
So talk to us a little bit about that concept.
00:48
Yeah.
00:49
I mean, what I was mapping out in the Times piece is sort of like maybe the background
00:53
story for a second.
00:54
So, Anthropic releases Mythos, Firefox team works with Mythos pretty closely in order
01:01
to excavate all these zero-day vulnerabilities inside the code base, some of which have been
01:06
there for decades.
01:07
This is happening to open-source software across the board, the FFmpeg people have the
01:10
exact same problem.
01:13
And what I was sort of pointing at is that the Internet has survived to this point for
01:17
a lot of reasons, one of them being this detente that we've all lived into.
01:22
We live in this weird world of temporal arbitrage, like it's really hard, or was really hard
01:28
to find vulnerabilities.
01:30
It's also really hard to write software.
01:32
But right now, we live in a world where both of those have just sort of disappeared, and
01:35
the equation is getting shuffled very quickly underneath all of us.
01:39
And now it's becoming really easy to write software, and it's really easy to find vulnerabilities.
01:43
And so the issue that we have is that the defenders, the people who keep the Internet
01:48
running to build all the infrastructure we depend on, are fundamentally under-resourced.
01:54
Like FFmpeg is like three guys.
01:57
We saw this with Heartbleed.
01:58
Heartbleed was like two people in OpenSSL.
02:01
Curl is one guy, even though my son's PlayStation has Curl on it, kind of thing.
02:06
And so Mythos is finding all these bugs across the board, and kudos to Anthropic that they
02:11
sort of are trying to, or tried to give it to a few set of people first to give them
02:16
a chance.
02:17
But we also know that the delta between these frontier models and what the rest of the world
02:23
can do is only a couple of months these days.
02:26
So we're rapidly getting to the point that all these vulnerabilities are going to be
02:30
exploited.
02:31
And it takes a long time for open source to get ruled out.
02:35
We saw this with Heartbleed.
02:37
What Heartbleed happened, Stripe within a day fixed all their code.
02:40
I guarantee you right now there are servers on the Internet that haven't patched it.
02:44
And so we're living this world where we're like, we might all be fundamentally fucked
02:48
in some way because this temporal arbitrage game is going to be screwing us over.
02:53
And then that doesn't even take into account all this next generation of vibe coders.
02:57
My dentist is probably writing a website for him to tackle appointments.
03:02
That's kind of screwed.
03:03
So we're living this very tumultuous time.
03:07
And so what's the problem, or what are your thoughts on the response from people who say,
03:13
well, the AI is going to fix the AI and basically just trust the tech.
03:18
And we've dealt with this type of Internet insecurity before.
03:23
And the community has rose up to the challenge.
03:26
Is this time different?
03:27
I mean, I have a lot of optimism and faith with all the people in this room, all the
03:31
people and all the Internet developers, all the people who maintain open source software.
03:35
I actually have a lot of faith in these folks.
03:38
But I don't buy the AI is going to fix the AI only because it's just a matter of, again,
03:43
just like temporal arbitrage of just like, yes, AI is going to fix the AI.
03:47
But that's going to happen a little bit later than when we discover all these things.
03:50
So we're still living in this open transaction where it's sort of like we're all vulnerable
03:54
or there are potential vulnerabilities here.
03:57
And so I don't know what exactly the right answer is, but AI will fix AI eventually.
04:03
The question is, what happens in that time period that we all have to grapple with?
04:07
So probably 60, 70, 80 percent of the people in this room have actually shipped something
04:15
using AI tools in the last 60 days.
04:20
How do you think builders now should be thinking about this and what should we be doing differently
04:25
in the moment?
04:26
Yeah, that's a great, I mean, so we have this, we're at Mozilla are facing the exact same
04:32
question of like, what does this transition look like?
04:34
So, you know, one end of the spectrum, we have the Firefox team, which has taken the
04:39
stance that AI, stance sounds too hard, has a position that AI is a tool, but it's humans
04:45
that are committing to the code base.
04:47
There's a Mozilla AI end of the spectrum.
04:50
So Mozilla is now a bunch of a portfolio of different companies.
04:52
So the Mozilla AI company has gone completely the other way.
04:56
It's like they have entire code bases that a human barely looks at in the grand scheme
05:00
of things.
05:01
So we're also seeing that entire spectrum across our world and we release everything
05:04
open source.
05:05
So like, we're also trying to grapple with the fact that we're getting, you know, slop
05:09
generated pull requests all the time and things like that.
05:12
I think we need to, we just need to acknowledge that we're racing toward a world where these
05:18
things are not a tool.
05:20
These things are a co-author, like this is a thing that actually is going to sit and
05:24
be a productive member of the team.
05:26
I know that's an uncomfortable position for some of us to sort of acknowledge, but like,
05:30
go with me for that thought experiment for a second.
05:32
So if you go with that thought experiment, then like, you start to think about the fact
05:36
that did you really look at that 400 line diff, that cursor or open code or whatever
05:42
DOP?
05:43
No, what you did is you probably did a skim of it kind of like you would have with another
05:47
human that's on your team.
05:49
And if that's the case, then we're not actually looking at the code.
05:52
What you're looking at is like, are the tests good?
05:54
Is the evaluation good?
05:55
So like, I think we need to have like, as an industry, like a little bit of a shift
05:59
to just how we think about this.
06:00
I'd like, the merge should not probably be on bytes, but it should be on behavior.
06:05
So like, we need to rethink about what Git really means in this world, because like Git
06:10
was operating on bytes and that doesn't matter anymore, because like the bytes could wildly
06:13
change the next time I run this thing.
06:15
So we need to think about like, can we merge based on evals instead of diffs or instead
06:20
of compilation?
06:21
And then if you think about that, like, we need to figure out like all the next steps
06:24
along the path that go there.
06:26
So, you know, in this time period where we're thinking about that kind of transition, we
06:30
should do the same thing that we probably did with junior engineers.
06:33
Like, there's a portion of the code base that's verboten.
06:35
Like no one touches that except humans that really know what they're doing until we have
06:40
an understanding of what's going on.
06:41
And we should treat models the same way we treat software interns.
06:45
So just like, you should all be demanding, like, what's the providence of the model?
06:49
Like when was the last version it was revved on, like we should be recording all that stuff
06:54
as well.
06:56
Because the next time someone else tries to do that behavior is going to do something
06:58
fundamentally different.
06:59
So I think if we sort of like, just do the thought experiment of leaning in a little
07:04
bit of like, these things could be co-authors, treat them like you would treat a software
07:08
intern like that would cause us to think about different tooling, different behaviors, etc.
07:12
And I think that's, that's what I've been encouraging teams to sort of think about.
07:16
So that's why I leave it all to you.
07:18
And as a CTO of a large engineering org, because Mozilla has hundreds of software engineers,
07:26
one of the most important things that you do is set the culture of the team.
07:29
And so are there other specific ways that you're sort of shaping the next version of
07:35
the Mozilla engineering culture along these lines, like considering all the AI tools that
07:39
engineers have at their disposal?
07:40
I mean, right now, I'm kind of a little bit in the like, a little bit of chaos is not
07:45
a bad thing kind of mode.
07:47
So like the fact that like the Firefox team, the Mozilla data collective team, the dot
07:51
AI team are all doing slightly different things is actually fine by me.
07:55
The thing that matters most is like, you know, once a month, and we should probably do this
07:58
more often, like all the.
07:59
your leaders at Mozilla do come together for our,
08:02
like we call it the technical council,
08:04
where we all just like share what's worked,
08:06
what hasn't worked, et cetera.
08:07
Just like we tried this on our team
08:08
that didn't work properly.
08:10
So I think like while we're all in this learning mode
08:12
and maybe what we should be doing is like,
08:14
since we're Mozilla,
08:15
maybe we should be publishing notes from that meeting.
08:17
But like, we should all just be sharing
08:19
on just like what that best practices looks like.
08:21
I mean, this is the one great thing
08:23
about the internet right now.
08:24
It's just like, you know, we all wonder
08:26
what's gonna be the future of open source
08:27
given all this stuff that's happening.
08:29
But the fact that we're all actually still sharing,
08:32
we're all still telling each other
08:34
all the things that are going on.
08:35
Like, you know, I clearly have emotional connections
08:37
to Twitter, not X.
08:39
But the fact that like all these engineers
08:40
are still posting on X of like,
08:42
this is the crazy thing I tried yesterday,
08:44
is kind of great.
08:45
And so I think we all just need to lean into that
08:47
a little bit.
08:48
So that's what we're doing at least inside Mozilla.
08:51
And you argued that security should be the default,
08:55
not a premium feature in the post.
08:58
But every developer tool has a business model.
09:01
So I wanted to like ease into the economics of open source
09:06
because I loved your comment about how we all share
09:09
and that's obviously the spirit of AI Council.
09:11
It's a notion of open source,
09:14
even if there's not bits
09:16
that this particular conference produces.
09:18
We traffic in open source type thinking
09:22
and knowledge sharing.
09:24
How does this affect like the security vulnerabilities
09:29
and also like getting to the business model behind it?
09:31
Like what should we be thinking about?
09:32
But maybe let's start a philosophy
09:34
and end that business model for a second.
09:35
So like right now,
09:37
the fact that like if you want to get SSO
09:40
on your SaaS service,
09:42
that you have to pay a markup is kind of bullshit.
09:45
Like we shouldn't be paying,
09:47
like we shouldn't be charging customers more
09:49
to have better security.
09:50
Like we need to figure out how to get rid of that.
09:53
Like this is gonna tie into the business model.
09:54
We have to figure out how to make that work.
09:56
But like philosophically we're kind of jerks
09:58
if we're doing that.
09:59
So like we shouldn't be doing that anymore.
10:01
So we need to figure out how to get rid of like the SSO tax.
10:03
We need to figure out how to get rid of all that.
10:05
But there are also better architectures
10:07
that we should be thinking about moving forward
10:10
on the kinds of tools we build.
10:11
Like there's a different question of retrofitting a,
10:13
you know, 50 year old Fortran soft system.
10:16
But like moving forward,
10:17
like we should all be thinking about things like Rust.
10:19
We should all be like actively moving in the directions
10:21
of like secure by design software.
10:24
So like that can help a lot of the situation.
10:26
So that handles all the stuff that's new.
10:29
And so we need to figure out that middle zone
10:31
of all this stuff that like is worth porting.
10:33
And in fact, like, you know, you can make an argument
10:35
that some of the best things that could come out
10:37
of like Opus 4, blah, blah, blah,
10:39
is that like we could port all this legacy software
10:42
into something that makes a lot more sense
10:44
for us to maintain easier to make use of today.
10:46
So we should think about that also in the middle.
10:48
Now the business model is the hardest part about this.
10:52
And so like we, I think as a industry need to think about
10:56
like how do we actually make our thinking of open source
10:59
to be less similar?
11:00
Like we would think about critical infrastructure
11:03
of just like, you know, I made the point in the piece
11:06
that like I thought about these guys
11:07
as like essential workers in some way.
11:09
And like the way we thought about it during the pandemic
11:11
of like, we need to give these nurses,
11:14
we need to give all these people as much support as we can.
11:17
And we need to be thinking about that right now
11:19
for that like one developer on curl
11:21
for the two developers on OpenSSL, like that kind of thing.
11:24
So like, you know, there's a few things we could do there.
11:26
Right?
11:27
Like Anthropic said they're going to do $4 million
11:29
for open source people using Mythos,
11:32
which is both a lot of money
11:33
and kind of like pennies at the same time.
11:35
But imagine a world where like
11:37
all the big frontier companies did $4 million.
11:39
Now we're talking 20, 20 something million dollars
11:42
all of a sudden, that's not nothing.
11:43
Like that's someplace to start.
11:44
So like we need to figure out how like all our employers,
11:48
all the people we work with
11:50
are donating something back to this ecosystem
11:53
that we all actually depend on.
11:54
So like, it could be money.
11:56
It could be like one engineer's time for a week.
11:58
Like there are all these different ways
12:00
that we could sort of like pull this off.
12:02
But I do think we all need to come together.
12:03
I mean, like, you know, in some ways
12:05
there is a business model around open source, right?
12:08
Like, you know, for all practical purposes,
12:09
Linux runs the world and then there are a few iPhones.
12:12
And so like, we like, but every company has figured out
12:15
how to like donate a little bit back to Linux, right?
12:18
Like Google has done it.
12:19
IBM did a whole bunch back in the day.
12:22
So like, we all can figure out how to like
12:24
give a little bit back to these pieces of software
12:26
that we actually depend on in a concerted way.
12:29
That actually could make a difference.
12:31
And we might not need to then truly figure out
12:33
the business model if we can just even do that.
12:35
But I acknowledge I kind of live in a like a,
12:38
in a pipe dream and like a world where unicorns exist.
12:40
But like, I would love that world to be the world.
12:42
Well, I feel like this has been the vision
12:44
of lots of open source engineers
12:47
in this community over the years.
12:48
I mean, Wes McKinney has done a lot of work
12:50
on working on alternate business models
12:53
and revenue models around open source.
12:56
Hannes from DuckDB, you know,
12:58
they have a consulting practice that does decent revenue
13:02
around DuckDB's implementation.
13:04
Do you think there's, is there a stone unturned here?
13:08
Is there something that we haven't done yet
13:10
that should be tried?
13:11
Or do we all just need to sort of, you know,
13:14
unite and pressure our employers to dedicate some,
13:18
allow us to dedicate some of our time to open source?
13:20
I mean, that's not really an XOR, right?
13:22
Like, it's like an or, like we should do it all.
13:24
Like, I do think that there are business models
13:27
that are yet unturned.
13:28
Like one of the beauties of the fact
13:30
that we just restructured Mozilla
13:32
into this portfolio of companies
13:34
is not because we have different technical domains,
13:37
which is true.
13:37
Like we want to spend more time in AI.
13:40
We want to spend more time in data,
13:41
but also so we can experiment
13:43
with different business models simultaneously.
13:45
So like, I encourage that for everyone.
13:46
It's just like innovation is not just in the software stack.
13:49
It's going to be in the business stack too.
13:51
But like, I don't have that answer.
13:53
We just need to constantly explore.
13:55
And when one of us stumbles upon it
13:57
and the rest of us will like share it, open source it,
13:59
then the rest of us can figure it out from there too.
14:01
Well, and how, like Mozilla is one of,
14:03
just not to leave the example right under our nose,
14:07
you know, left untouched.
14:08
Mozilla is one of the most successful
14:10
open source software companies in the world,
14:12
not just from a distribution standpoint,
14:14
but from a revenue standpoint.
14:17
What lessons do we have to learn
14:18
from how Mozilla did things
14:20
that might like point us to the future?
14:22
Well, I mean, for us,
14:23
I mean, maybe I'll start with the criticism
14:26
and then we can back into it.
14:27
Like, you know, a lot of our revenue right now
14:29
comes from the Google search deal.
14:31
A lot of my job is to figure out how to diversify that,
14:34
which is why we started all these other companies
14:36
with all these different business models.
14:39
So we can sort of balance that portfolio a bit.
14:40
But I think the lesson to be learned here
14:42
is that like there are partnership opportunities
14:45
that do matter.
14:46
Like people do care about openness.
14:48
Like when we talk about the philosophy of like,
14:51
why should things be open?
14:53
It's because we're actually benefiting humans in some ways.
14:56
Like we want agency, we want transparency,
14:58
we want all those things.
15:00
Like no company is gonna say to anyone's face,
15:02
I don't want a person to have agency, right?
15:04
So like there are good values
15:06
that can be like tagged along
15:08
to make the business model work through partnerships.
15:11
But again, I think that like,
15:13
we're working through this right now too.
15:14
Like we're in this exploration stage of just like,
15:17
what is that next model really look like?
15:19
So I just encourage all of us to be thinking about it.
15:22
Backstage, we were talking about sort of the next layer
15:26
of internet control or optimization
15:30
or just the touch point.
15:33
And I guess in the days of the browser wars,
15:37
because you have the Firefox experience,
15:39
we thought it was a war for the browser.
15:43
Now there's this war for the premier,
15:46
the preeminent AI model.
15:48
What's the right analogy from the past
15:51
that you use to think about like this point in time
15:54
that we're at now?
15:54
I don't think these are distinct points in time.
15:58
I think like, I mean,
15:59
I think these are all just like actually one big continuously evolving push and pull struggle
16:07
that we're dealing with. Like back in the browser days, it was a question which, you
16:11
know, I'm older than most of the people in this room, but like back in the browser days,
16:15
it was a question of just like, not just who controls the application or software, but
16:21
it's a question of like, who gets to build. And so like, that's the beauty of the lamp
16:24
stack back then. There's a beauty of something like Firefox. It's like anyone could build,
16:30
like you could do whatever you want to the internet. And like Firefox was created effectively
16:34
as a way to ensure that we maintain that like sovereignty control of like, I can put up
16:40
a website if I really wanted to, anyone could find my website kind of thing. You know, we
16:45
then lost the battle when it came to social media. Like in social media, the world didn't
16:50
go toward openness. And like, look, I'm not going to, I'm not going to do the causal inference
16:56
of like what happened if it went to openness, but I will say the world has gone to shit
16:59
because of social media. And so like, maybe there was alternative options that we just
17:04
never explored. And now, frankly, we'll have a hard time getting off the ground because
17:08
there's too much capture in that space. And you're right to say that the models are the
17:12
next battlefield. But again, I think these are all the same thing. It's a question of
17:16
just like, what does a human get to choose to see? What does a human get to choose to
17:21
build? How does a human get to choose to make their own actions? Like one of the things
17:25
I'm worried about right now, and there was just a paper last month by Princeton and UW
17:30
that confirmed this, is that like, you know, I think we've all underestimated what the
17:35
power of 10 blue links on a Google page did. Like not everyone, my mom definitely has no
17:40
idea what page rank is, but like she had a good sense of like what Google was doing
17:46
when it showed her information and she got to choose it. But now if you go to ChatGP
17:49
and ask it, it just gives you one answer. And like, I actually think that's incredibly
17:54
dangerous. And so like, we need to figure out like what's the right paradigm and user
17:58
interface because in that one, in that one case, like you've outsourced decision making.
18:03
And so I feel like the goal of technology here or the goal of technology on the right
18:08
side, or like the light side, might be to figure out how to bring that decision making
18:12
power back to a human being. So like that paper from Princeton and UW, they were literally
18:18
taught, they did this whole survey across all the platforms, large language models,
18:21
and asked it purchasing questions and stuff like that. And like for a non-trivial amount
18:26
of time, it was biasing toward the platform sponsored results. And so like that doesn't
18:31
seem great. And so like we've been, this has been the same battle for all this time.
18:36
And so I think that like when we're done with this AI model battle, which we probably
18:41
never will be, frankly, we're going to have to move to the next one because like technology
18:45
is just going to keep on moving. It's always going to be a question of just like, how do
18:48
we retain our humanness and how do we make sure technology is on our side in that process?
18:54
Yeah. As an interesting example, it happens to be a security example. I was chatting with
18:58
Diana from our security track, who's speaking on the security track today, and she was explaining
19:03
how the models are actually trained to encourage developers to do insecure things. And it's
19:11
insane to think that there's that level of control. Like we don't know what sorts of
19:18
reinforcement learning any of these models go through that we interact with. And it could
19:23
be teaching a whole new young generation of engineers sometimes to do the wrong thing,
19:29
believe it or not, just because of the taste of the person training the model.
19:32
Yeah. So like going back to that previous comment, like we need to treat them like interns
19:36
and like we need to rethink, you know, I have an experiment on GitHub, if anyone's curious,
19:40
it's called Morph. Just look up Morph in my name. But like a whole thing I'm trying to
19:44
understand is like, what if we reformed Git, not as a set of like bytes that got committed,
19:51
but a set of prompts and reasons and evaluations behind them? And then you can track the provenance
19:55
of the model because then it'll be like, well, this prompt on this model was kind of shit.
20:00
That way I can rerun it at a later date and get a better outcome from it. I think we need
20:04
to better understand exactly like what these things are doing with us. Remember that there
20:10
are other incentives, like not everyone, I mean, I'm not breaking any news here, so I hope no one's
20:14
getting depressed, but not everyone's altruistic. So like trying to figure out exactly how that all
20:18
plays out is going to be super important. So obviously you're a big fan of openness in general,
20:24
in life, in your philosophy, in code, in tech. Let's talk about what the world of open models
20:31
would mean. Like how does that manifest? And where do you think this heads from here? Like
20:38
are OpenAI and Anthropic going to control, and Gemini going to control the main endpoints for
20:46
all applications? Or is there a scenario here in spite of some of the economic weakness around
20:51
open source, is there a scenario where open source can win the day in the model wars?
20:57
I think open source can be a viable alternative. So right now open source is, like if you're a
21:03
developer right now and you have a weekend project you want to hack on, you're clearly going to hit
21:07
either OpenAI's API or Anthropic's, like either directly or via open router. Like it's just the
21:12
easiest thing to go do. Like I think we need to get to a world where like it's a viable choice.
21:17
I'm just like, no, I'm going to hit this other endpoint instead, or I'm going to run this model
21:21
on my laptop instead. And I think we're in a developer experience problem, frankly. I think
21:25
that like it's just too hard and like there are too many choices. Like when someone needs to do
21:30
like the, you know, the MacBook of open models of just like here is a really good set of default
21:37
choices that's not shitty like a Chromebook, but like more exciting like a MacBook or something
21:42
like that. Like I think like we need that thing to exist, like a rounded corner. Like we need good
21:47
rounded corners when it comes to, when it comes to open source AI. And I think we can get to a
21:52
world where we have credible alternatives. And like, I don't think it's a world necessarily
21:56
where Anthropic and OpenAI cease to exist. I think it's a world where we like coexist in a similar
22:02
way that like, again, Linux coexists with other operating systems on a planet. Like, you know,
22:07
Linux on the desktop might actually finally be useful 20 years later, but like for a long time,
22:11
like it was just like, it was another set of, another set of choices that we can go make.
22:15
So I want to live in a world where like we all can then like be fine tuning models. I want to live,
22:21
my end game is that I don't want seven AGI's in the world. I want 7 billion AGI's in the world
22:25
because we all have a variant of what we want running on our devices near us with access to
22:31
all my personal information in a way that would probably never want to give all that stuff to a
22:36
third, to a third party. So I think this is possible. Now we have to fix a lot of things
22:41
to make that work. Developer experience is one of them. We have to fix the compute scarcity that
22:46
we, like, I want to live in a world of compute abundance, not compute scarcity. So we need to
22:49
figure out how to break free of just purely the NVIDIA CUDA version of the world. And like, what
22:54
does it mean if we can work on heterogeneous hardware? What does it mean if we can do
22:57
distributed training or whatever, whatever it really looks like? We need to figure out how to
23:02
get actually like province based, like access to data sets that we can give to certain models,
23:07
but not others. So there's a lot of moving pieces. But again, like I actually have faith
23:13
in all of us. Like I think that the thing we need to all agree on is that we don't want a world of
23:17
just seven AGI's that are not clearly on our side. And if we can all agree there, then we can start
23:23
building toward it. And so that's the thing that I'm trying to do like every day is remind people,
23:27
this is the end game that we want. It seems like most of the open weight models, because of the
23:32
cost of training, and the cost of compute around these models, the meaningful open weight
23:39
models are coming from larger companies. And again, they're sponsored projects.
23:44
Now, in the old days of open source, or, you know, in the past 10 or 15 years,
23:49
it seemed like there was a economic incentive for companies to release open source as a recruiting
23:56
game. And so I used to do this, there's a
23:59
understood, you know, best-kept secret, worst-kept secret in Silicon Valley, that
24:04
one of the reasons Google and Facebook and, you know, Facebook has React and I
24:09
think of all the other meaningful open-source projects that have
24:12
significant adoption, sponsored or supported by large companies, a lot of it
24:16
was done because they wanted to attract more software engineers to their
24:19
platform, into their company, and there was this recruiting subtext. Like, is that
24:25
going to continue and is that enough of a motivation for companies continuing to
24:28
release these very expensive, open-weight models, or does there need
24:31
to be another economic incentive that comes around? I mean, if we look
24:36
at the way the LAMP stack was put together, so, like, Linux, Apache, MySQL, PHP
24:41
kind of thing, like, Meta did a lot of work in the PHP layer and then sort of
24:46
gave it to everyone in the process. Like, maybe not all of it, but they gave a
24:49
good chunk of it to everyone in the process. And, like, in that way, it is
24:53
partially a recruiting thing, it's partially a marketing thing, it's
24:56
partially like a, we want to be one with the developers and the hackers kind of
25:00
thing. So, I think there is something there still of, like, I want to make sure
25:04
that, like, all the people in this room, all the people in the world, like, see us
25:08
as, like, a friendly place. Because, again, I think that, like, when you actually
25:11
think about it for a second, you want to work at a place that's making positive
25:15
impact in the world. Like, they're all selling us that they're making positive
25:18
impact in the world. So, like, I think we just need to, like, critically examine
25:21
that. And so, like, if we can get through that critical examination barrier of
25:25
just, like, again, like, the models are making choices based on financial
25:29
incentives for the companies. Or, like, people are being made, decisions are
25:34
being made that's not on the side of the user, it's on the side of the platform.
25:37
Once we can start seeing through all that, then engineers can also make
25:41
rational choices on, like, exactly where they want to spend their time. So, I
25:45
think once we get through that cycle, then I think we get to a world where, like,
25:50
engineering just becomes a little more free, it sort of goes into all these
25:53
different places in the world. Now, a better question, I think, to ask about
25:56
the open models is, like, it's the best open models right now are coming from
26:01
China, are coming from other places that don't necessarily hold the same values
26:06
that we're holding right now. I was reading an interesting paper last year, my
26:09
favorite paper of last year was one around kidney donations. So, like, they
26:13
actually asked a whole slew of the large platform models, like, giving these
26:16
different kidney donations is a very complicated thing because you have to
26:19
take into account the ethics, the worthiness of the person, all these things.
26:23
It's not just a matter of being in a queue. And so, they asked all these large
26:27
language models to do the role of the kidney donation board. Like, would you
26:31
give this kidney to this person or that person? All the Western models did one
26:34
thing, all the Chinese models basically did a different thing. And so, like, so
26:38
those are the kind of questions that we really need to be asking around open
26:41
models and if we want to live in this world of fully open models. And so, like,
26:44
the answer to that is probably open data along with it. So, like, the bar is
26:48
actually slightly higher than just open weights. Like, we actually need to
26:52
understand evaluation metrics. We need to understand the data that goes into
26:55
it. We need to understand what post-training looked like in order to
26:58
actually make these type of decisions. So, I think if we can get to a world
27:01
where we, like, actually have transparency to every single part of the
27:05
chain. Now, the reason why a lot of people maybe are not worried about this
27:09
is because it all feels so abstract. One of the things you and I were talking
27:13
about is, like, you know, the next frontier is robots in some way. So, like,
27:17
these things are going to be taking physical action in the world. It's not
27:20
just like they deposited $100 out of my bank account, which is annoying. But,
27:24
like, this thing could, like, physically hurt me or could physically hurt
27:28
something else. Like, I clearly need to understand if it's going to live in my
27:32
house exactly what went into this thing because, like, it could harm me in some
27:36
way. So, I just encourage everyone to think about, like, where we are right now
27:41
is just part of this longer arc. And so, we got to get it right now because
27:45
otherwise, like, we're just going to have a really hard time later.
27:48
Yeah. If you think about another one of the philosophical benefits of open
27:52
source that was touted back in the day, it's to many eyeballs, all bugs are
27:59
shallow. And so, there was this notion that if you could see the bits and if
28:04
you could see how it works, you could trust the thing. And how far we've come
28:08
with these black box models. So, I think just, like, you know, in a childlike way
28:12
to, like, rechannel some of those values of original open source and to think
28:17
about how we might replay or recreate some of those benefits in the AI systems
28:21
that we're building, it might push us forward down the right path.
28:26
And I also contend that we don't need everything to be open. Like, I think what
28:30
we need is, like, you know, I'm going to pick on again Firefox, but maybe I'll
28:34
also pick on Signal for a second. We need a credible thing that is open out
28:39
there in the world that then forces the industry to reconsider. Like, RCS is
28:44
getting end-to-end encryption effectively because Signal exists and is
28:48
basically forcing them, not explicitly, but because of Signal's existence, they
28:52
reset the bar of what's needed for messaging. Because Firefox is open, Chrome
28:57
is open, right? So, I think, like, if we have a good, credible set of open
29:03
models, open stacks, applications using them, it'll force the industry in order
29:08
to do the right thing because there is a credible alternative out there. That's
29:11
actually what I'm shooting for. Like, I don't think it's going to be reasonable
29:15
to say the entire industry needs to change. We have to read bits all the way
29:19
down to the bits. My mom is never going to look at a line of source code. But the
29:22
fact that there is a credible alternative out there protects my mom. And so, like,
29:26
that's how we should be all thinking about it.
29:30
If folks have questions for Rafi, we'll send around a couple of mics. We'll have
29:34
a couple of minutes at the end for questions. So, I wanted to ask you, 2035,
29:40
2036, 10 years from now, more software engineers in the world or less?
29:45
That's a great question. I mean, I think that, like, you know, one of the things I
29:48
used to – so, I used to work in the philanthropic sector for a while before
29:52
coming to Mozilla. And one of the things I would say there, because, like, we were
29:56
going through this transition as a sector of just, like, how do you get more
29:59
technologists into the sector? And it was, like, a valid question of, like, is the
30:03
goal to get more technologists into the sector or is the goal to make everyone in
30:07
the sector a technologist in some way? So, I think if you look at it through that
30:11
lens, I think – I mean, like, without sounding like a socialist crazy person,
30:15
we're all coders, like, at this point. So, there's actually more software
30:18
engineers going in the world because, like, we now have the power to solve all
30:22
our problems. Like, I want to be in, like, 2035 – 2035 might be actually too far
30:26
away. I want to be in a world that instead of watching, like, a home
30:29
improvement TV show of, like, people fixing their kitchen, I want to see
30:32
someone, like, I've, like, coded a fix in my blah, blah, blah business kind of
30:35
thing. We can just watch that instead of HDTV all the time. So, I think that we
30:40
actually get to more, more builders in 2035 than there are today. And we're
30:45
going to redefine what software engineering means. Like, my son – so, I
30:50
have two sons. My 10-year-old son is the creative one in the family. Like, he
30:54
vibe codes video games these days. Like, he's 10. He was watching me tinkering
30:58
around with, at the time, it was Cursor. And he's, like, well, can I try? And he
31:02
just, like, vibe coded himself a D&D kind of, like, game. And now he just
31:06
does this, like, every weekend. He's just, like, can I borrow your laptop?
31:08
And he just, like, made – so, Cute and Stinky Productions. So, if you just
31:12
Google it, it's Cute and Stinky Dot Productions. Like, it's all the games
31:15
that he's vibe coded. He just creates a portfolio of them. But now he's been
31:19
asking me – so, like, he's a builder. A 10-year-old person who literally
31:22
knows nothing about coding is a builder. And he's been asking me recently. He
31:26
wants to learn how to code. He's, like, I kind of want to get under – I want
31:29
to understand how this actually works. So, I've been having this conversation
31:33
with my wife, who's a CS professor at Stanford, of, like, well, what do I
31:37
teach him? Her answer was Python. And I was, like, I think I'm going to teach
31:40
him Lisp. Because, like, it's, like, generally useless. But, like, it's, like,
31:44
kind of teaching him Latin of just, like, he'll learn some fundamental
31:48
things that sort of extrapolate to how computer scientists think. And so,
31:52
like, we have to rethink what it means to be an engineer. But to your
31:55
question, 2035, more builders in the world.
31:57
Definitely more builders. And I got
31:59
That is my secret underlying question, is the builder an engineer?
32:05
I think we all see the power of what experience, how experience drives taste, and how taste
32:10
drives really high-quality prompting of the models.
32:14
We can probably all see that in our life right now, and I guess I'm curious if that high
32:19
place will hold, and if an engineer is different than a builder because there's taste or experience
32:26
involved or a more atomic level of how the system operates.
32:31
I'm not sure where I stand on this, but I'm curious if you have thoughts.
32:34
I do worry about this reduction of experience points in the world.
32:40
I think that I'm a better vibe coder.
32:43
I used to be a good engineer, I think I'm a pretty okay engineer these days, but I'm
32:47
a better vibe coder than a lot of people because I think I understand architecture, and I have
32:51
a lot of battle scars, a lot of battle scars.
32:55
I understand when a model does something, I'm like, that'll never fucking work kind
33:00
of thing.
33:02
I do worry about this erosion of experience that's going to happen through all these things
33:08
playing out.
33:09
Now, you can make an argument that experience gets coded in the models.
33:12
If the models are better, maybe experience is not needed, but I still think I don't see
33:17
a world yet, and 10 years, maybe I'm wrong and I'm totally fine with that, but I don't
33:22
see a world yet where I have a visceral experience of this is going to work, this is not going
33:29
to work, or this is what I want, this is not what I want, this is what beauty is, this
33:33
is what elegance is.
33:34
I still think that's going to be important in everything that we go do, whether it be
33:40
music, whether it be sports games, I think there's still going to be that visceral, there's
33:46
a humanness thing there.
33:48
Someone asked me the other day, how do we know if these things are creative?
33:54
The thing I gave them was, there's a model recently, which was only trained in 1935 or
33:59
1925 or something like that, which is a fun tinkering.
34:03
My variant of that is, if you trained a model on all the music up until the 1940s, 1950s,
34:11
would it create punk rock?
34:13
Probably not.
34:15
Music got more complex through the 1950s, and punk rock is like a guy screaming into
34:20
a microphone.
34:22
I don't think a model extrapolates that.
34:25
We still have to figure out what that really means, whether that's encodable and how that
34:29
works.
34:30
Right now, there clearly is still something there for us to prompt our way to, that the
34:35
machines are not going to do themselves.
34:37
Well, Raf, you've always been such an inspiration to me, and watching how you've started your
34:42
career and where you've gone over the years, I'm curious, if you weren't working at Mozilla,
34:47
what would you be doing?
34:48
Education, 100%.
34:53
I've been very fortunate in my career that I've managed to pivot a lot of times.
34:58
I went from social media to robots and cars to politics to philanthropy, and now I'm back
35:06
in the fight for the internet.
35:08
I think the next thing, if not maybe the thing I should be working on right now, is what
35:12
do we teach our kids?
35:15
I think there is a notion with these systems right now, it's something that's been true
35:20
on the internet for a while, but now it's become really apparent in these systems that
35:23
we're optimizing for frictionless experiences, but that's counter to learning.
35:30
The way you learn is productive struggle.
35:33
You learn because you failed, you learn because you got back up again, it's those battle scars
35:37
I talked about.
35:39
We have to figure out how we still convey that to our children in a way that they can
35:44
then still make use of all this technology.
35:47
By no means am I saying, don't give a kid AI.
35:50
I think we should absolutely be giving kids these tools, but we haven't taught them how
35:55
to have a productive struggle while using these tools.
35:59
That's the next battle for all of us, I think.
36:02
We are all worrying about our careers, but we also need to be worrying about our kids.
36:06
Yeah, well said.