acknowledged the mistake<\/a> and temporarily paused image generation of people in Gemini while we work on an improved version.<\/p>\nWhat happened<\/h2>\n
The Gemini conversational app is a specific product that is separate from Search, our underlying AI models, and our other products. Its image generation feature was built on top of an AI model called Imagen 2.<\/p>\n
When we built this feature in Gemini, we tuned it to ensure it doesn\u2019t fall into some of the traps we\u2019ve seen in the past with image generation technology \u2014 such as creating violent or sexually explicit images, or depictions of real people. And because our users come from all over the world, we want it to work well for everyone. If you ask for a picture of football players, or someone walking a dog, you may want to receive a range of people. You probably don\u2019t just want to only receive images of people of just one type of ethnicity (or any other characteristic). <\/p>\n
However, if you prompt Gemini for images of a specific type of person \u2014 such as \u201ca Black teacher in a classroom,\u201d or \u201ca white veterinarian with a dog\u201d \u2014 or people in particular cultural or historic contexts, you should absolutely get a response that accurately reflects what you ask for. <\/p>\n
So what went wrong? In short, two things. First, our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not<\/i> show a range. And second, over time, the model became way more cautious than we intended and refused to answer certain prompts entirely \u2014 wrongly interpreting some very anodyne prompts as sensitive. <\/p>\n
These two things led the model to overcompensate in some cases, and be over-conservative in others, leading to images that were embarrassing and wrong. <\/p>\n
Next steps and lessons learned<\/h2>\n
This wasn\u2019t what we intended. We did not want Gemini to refuse to create images of any particular group. And we did not want it to create inaccurate historical \u2014 or any other \u2014 images. So we turned the image generation of people off and will work to improve it significantly before turning it back on. This process will include extensive testing.<\/p>\n
One thing to bear in mind: Gemini is built as a creativity and productivity tool, and it may not always be reliable, especially when it comes to generating images or text about current events, evolving news or hot-button topics. It will make mistakes. As we\u2019ve said from the beginning, hallucinations are a known challenge with all LLMs \u2014 there are instances where the AI just gets things wrong. This is something that we\u2019re constantly working on improving. <\/p>\n
Gemini tries to give factual responses to prompts \u2014 and our double-check feature helps evaluate whether there\u2019s content across the web to substantiate Gemini\u2019s responses \u2014 but we recommend relying on Google Search, where separate systems surface fresh, high-quality information on these kinds of topics from sources across the web. <\/p>\n
I can\u2019t promise that Gemini won\u2019t occasionally generate embarrassing, inaccurate or offensive results \u2014 but I can promise that we will continue to take action whenever we identify an issue. AI is an emerging technology which is helpful in so many ways, with huge potential, and we\u2019re doing our best to roll it out safely and responsibly.<\/p>\n<\/div>\n