Thread by @ethereum on Thread Reader App – Thread Reader App

Thread by @ethereum on Thread Reader App – Thread Reader App

0/ As AI systems become more powerful, formal verification – which is the discipline of using mathematics to prove computer programs correct – shows more and more promise.

A guest thread by @big_tech_sux, lead developer of @vyperlang Image

1/ First of all, what is formal verification? Formal verification proves properties about your program through all execution paths through the program.

For example, `f(x) = x/2` can never return an output which is larger than the input. You can see this, and formal verification can prove this, by reasoning algebraically about the function.Image

2/ Sounds amazing, right? It is!

And formal verification can catch 1 in 1 quadrillion edge cases which would never come up during testing – and that edge case is the case that can lead to catastrophic failure. Image

3/ The only catch is that it’s *hard* – you used to need a team of experts to a) translate your code to a formal model and b) actually do the proofs.

Now that LLMs are rapidly approaching AGI (Claude Fable casually proved the Jacobian conjecture false after being an open problem in mathematics for 80 years), a lot of this work becomes (much!) easier.

x.com/__alpoge__/sta…

4/ That being said, formal verification is still pretty hard, even with AI assistance. Doing the proofs is expensive. And formalization (translating code to a formal model) can lead to inaccuracies which affect the correctness of the proofs.

This translation step in particular can either be shallow or deep, depending on how thorough the translation is, and formalizing may reflect more or less of what you want to know about the system (in the literature, this is called “minimizing the verification gap”).Image

5/ Why is formal verification becoming more important, now?

Besides being more feasible than before, it’s actually crucial to cybersecurity – advances in AI make it easier to produce exploits. For instance, an unreleased model at OpenAI recently discovered 0-day exploits to break through multiple layers of security.

Formal verification shifts the asymmetry towards defenders. While attackers only need to find one sequence of inputs which lets them break (into) a system, defenders can use formal verification to prove robustness against *all* inputs.

For mission-critical software like smart contracts where billions of user funds are at stake, formal verification is becoming a prerequisite, not an option.Image

6/ Beyond just cybersecurity, formal verification also ties together math and software.

Math and physics tell you the rules of what is possible; formal verification lets you build things that adhere to those rules.

For example you could specify “3d print me a structure which satisfies these constraints.” The AI would build it, and FV would ensure that it adheres to spec.

By having cheap, automated verification, you can try new things faster (“doing it right the first time”), ultimately accelerating human progress.

7/ This is why two projects I am affiliated with, @vyperlang and @Sequent_Inc, are betting hard on formal verification.

Vyper is using formal verification to verify language and compiler correctness down to bytecode, and Sequent is building infrastructure for AI and humans to harness formal verification.

I strongly think that formal verification is crucial for a future where AI and humanity coexist to make a better place for us all.

• • •


Missing some Tweet in this thread? You can try to
force a refresh

 

Source link

Share:

Atbildēt

3 latest news
News Archives
On Key

Related Posts

curated design references for AI agents

curated design references for AI agents

Reaching 1,146 sites, 3,205 captured sections, 2,811 full-page captures, 568 fonts and 507 creators. search_websites Full-text search across site names, descriptions, and design tags. browse_websites

No, local models will not win

No, local models will not win

Every time a new open-weight AI model is released, people say that local models are the future. Why spend billions of dollars building out datacenters

Want taste? Eat your slop.

Want taste? Eat your slop.

There are a lot of people now making software for the first time. In many ways it’s wonderful. We have to admit however, that the