\n<\/p>\n
Let\u2019s dig deeper into this new capability and how it\u2019s helping Bard improve its responses.<\/p>\n
Large language models (LLMs) are like prediction engines \u2014 when given a prompt, they generate a response by predicting what words are likely to come next. As a result, they\u2019ve been extremely capable on language and creative tasks, but weaker in areas like reasoning and math. In order to help solve more complex problems with advanced reasoning and logic capabilities, relying solely on LLM output isn\u2019t enough.<\/p>\n
Our new method allows Bard to generate and execute code to boost its reasoning and math abilities. This approach takes inspiration from a well-studied dichotomy in human intelligence, notably covered in Daniel Kahneman\u2019s book \u201cThinking, Fast and Slow\u201d \u2014 the separation of \u201cSystem 1\u201d and \u201cSystem 2\u201d thinking.<\/p>\n
In this analogy, LLMs can be thought of as operating purely under System 1 \u2014 producing text quickly but without deep thought. This leads to some incredible capabilities, but can fall short in some surprising ways. (Imagine trying to solve a math problem using System 1 alone: You can\u2019t stop and do the arithmetic, you just have to spit out the first answer that comes to mind.) Traditional computation closely aligns with System 2 thinking: It\u2019s formulaic and inflexible, but the right sequence of steps can produce impressive results, such as solutions to long division.<\/p>\n
With this latest update, we\u2019ve combined the capabilities of both LLMs (System 1) and traditional code (System 2) to help improve accuracy in Bard\u2019s responses. Through implicit code execution, Bard identifies prompts that might benefit from logical code, writes it \u201cunder the hood,\u201d executes it and uses the result to generate a more accurate response. So far, we’ve seen this method improve the accuracy of Bard\u2019s responses to computation-based word and math problems in our internal challenge datasets by approximately 30%.<\/p>\n
Even with these improvements, Bard won\u2019t always get it right \u2014 for example, Bard might not generate code to help the prompt response, the code it generates might be wrong or Bard may not include the executed code in its response. With all that said, this improved ability to respond with structured, logic-driven capabilities is an important step toward making Bard even more helpful. Stay tuned for more.<\/p>\n<\/div>\n