Google dropped a new AI model that supposedly has upgraded “thinking skills” to tackle tough questions more thoughtfully. Developers dub these made-up elements from AI content generators as “hallucinations.” Google’s highly anticipated AI model, Gemini, is claimed to outshine OpenAI’s GPT-4 large language model (LLM) and can go head-to-head with “expert level” humans in an intelligence test.
CEO Sundar Pichai introduced the model at the company’s I/O conference in May. It will be available in three versions: Nano, Pro, and Ultra, with Ultra being the most powerful. Gemini went through trials assessing its problem-solving and knowledge across 57 subject areas, covering everything from math to humanities. Sundar Pichai, the boss, declared it marked the start of a “new era” for AI. Google took a careful stance when introducing its AI chatbot, Bard, earlier this year, labeling it as “an experiment.”
Bard goofed up in its own demo, giving the wrong answer to a space-related question. Still, Google is hyping up its new model, calling it their “most capable” one yet and claiming it can beat human experts in various intelligence tests. Gemini has the ability to identify and create text, images, and audio, but it’s not a standalone product. Instead, it’s a foundational model, slated to be incorporated into Google’s existing tools, including search and Bard.
Also Read: How ChatGPT could be abused by people with nefarious intentions
Gemini is giving ChatGPT a run for its money
Gemini seemed to establish a “fresh benchmark,” showcasing its capacity to learn not just from text but also from other sources like images, as noted by Gartner analyst Chirag Dekate.
Google has had a bit of a challenge grabbing the spotlight and user base compared to OpenAI’s popular chatbot ChatGPT. However, they argue that the most robust version of Gemini surpasses OpenAI’s GPT-4 – the engine behind ChatGPT – on 30 out of 32 widely-used academic benchmarks.
But there’s a beefed-up edition of the OpenAI software set to launch next year. CEO Sam Altman suggests that the upcoming products will make the current ones seem like “a quaint relative.” We’ll have to wait and see if the recent chaos at OpenAI, where Mr. Altman got the boot and then brought back within a few days, will have any effect on the upcoming launch.
Elon Musk’s xAI also in the scheme of things
The company is also up against new competition from Elon Musk’s xAI, aiming to raise up to $1 billion for research and development. Meanwhile, the Chinese firm Baidu is charging ahead with its own AI products. However, with technology advancing rapidly, concerns about its potential to cause harm are also on the rise.
Governments worldwide are working on creating rules or even laws to manage the potential future risks of AI. In November, folks talked about the matter at a summit in the UK. The attendees signed a declaration advocating for the safe development of AI. The King emphasized the need to tackle potential dangers with a sense of “urgency, unity, and collective strength.”
Bard already has Gemini AI support
Google has already added a “fine-tuned version of Gemini Pro” to its Bard chatbot, as mentioned in an official blog post. While Ultra is currently undergoing thorough “trust and safety checks” and “fine-tuning,” it’s expected to be integrated into Bard “early next year.”
Apart from a few straightforward video demos the company released today, that’s pretty much the extent of our knowledge about Google’s newest AI. Details like the number of parameters it has or the training data used are unknown, making it impossible to make a fair comparison with other AI models.
Not to forget the obvious absence of a widely accepted, industry-standard benchmark to gauge the intelligence of any AI model. In essence, what we’ve got so far is some marketing talk — and now, the company has a lot to demonstrate. Gemini is touted as “natively multimodal,” indicating its ability to “blend different types of information, including text, code, audio, image, and video,” as mentioned in the blog post.
Google flaunted Gemini’s abilities in a bunch of videos. In one, the program accurately identifies a drawing of a blue duck. Another video features the AI reading a student’s responses to math questions and breaking down why the student was either right or wrong.
Also Read: ChatGPT can actually make your work life a lot easier
Can Gemini really beat OpenAI’s GPT-4?
During the announcement, the company declared that its Gemini Ultra model achieved a 90 percent score on an MMLU (massive multitask language understanding) test. This test “uses a combination of 57 subjects such as math, physics, history, law, medicine and ethics for testing both world knowledge and problem-solving abilities.”
This makes it the “first model to outperform human experts” on the test. Human experts hit only 89.8 percent. GPT-4, as per Google, managed only 86.4 percent on the test. However, the intermediate Gemini Pro model, integrated into Google’s Bard chatbot, could only outperform GPT-3.5 in the same test.
Simply put, based on the data Google shared, the competition is still very close. Eli Collins, Google DeepMind’s VP of Product, mentioned that Gemini underwent the “most thorough safety evaluations” compared to any AI models the company has developed so far. This might be why Google reportedly had to delay its launch.
They also mentioned they won’t disclose the parameter count, adding another layer of difficulty to making comparisons. Apart from Gemini, Google teased its “experimental” Search Generative Experience earlier this year, aiming to improve the company’s search engine results. However, based on what we’ve seen, the tool falls short of expectations. In a nutshell, it’s still uncertain how Gemini stacks up against its tough competitors.
And that’s not likely to change anytime soon. For now, Bard users can only utilize text prompts. The release of image and audio interaction is expected “in the coming months,” according to Collins. Even the supposed GPT-4 outperformer, Gemini Ultra, isn’t ready for the big stage according to Google. So, we’ll have to hold off on drawing any definite conclusions until it’s fully ready. Only time will tell if Google’s Gemini AI will lead the race in this space.
