OpenAI’s ChatGPT vs. Google’s Bard AI: A Comparative Analysis

This blog post was written by Tonye Harry as part of the Deepchecks Community Blog. If you would like to contribute your own blog post, feel free to reach out to us via blog@deepchecks.com. We typically pay a symbolic fee for content that's accepted by our reviewers.

Introduction

Fig. 1 ChatGPT vs. Bard. Source

Fig. 1 ChatGPT vs. Bard. Source

In November 2022, Open AI’s ChatGPT sparked the commercialization of large language models (LLMs) and other technology copies, and researchers followed suit with variations of their own. Google, who has been leading the way in LLMs, released Bard AI as an answer to ChatGPT, and ever since, both have been in a race to outperform each other.

Advanced language models have revolutionized the field of natural language processing, enabling remarkable progress in tasks like conversational agents, information retrieval, and document analysis. Among these models, ChatGPT and Google Bard AI have emerged as prominent examples. Understanding their capabilities, strengths, and limitations is crucial for researchers, developers, and practitioners who seek to leverage these models effectively.

In this comparative analysis, we delve into the depths of ChatGPT and Google BARD to unveil the power they possess while examining their inherent limitations.

Comparative analysis of ChatGPT and Bard AI

Language models like ChatGPT and Bard AI, with their respective architectures, are designed for conversational tasks, simulating human-like interactions and generating responses in a conversational context. They showcase exceptional understanding and maintain coherent conversations over multiple turns.

They excel in applications such as chatbots, virtual assistants, and content generation, providing users with an engaging and interactive experience.

Together, these language models contribute to the advancement of natural language processing (NLP) and dialogue systems, enabling more sophisticated and interactive conversations between humans and AI.

To compare both tools, the following factors are taken into consideration to observe their nuanced distinctions and similarities:

  • Training data: Uncovering the types of data utilized to train their models.
  • Model Architecture: Assessing the underlying structure that powers the capabilities of these tools.
  • Contextual Understanding: Unraveling the tool’s ability to comprehend and interpret context.
  • Applications and use cases: Exploring the practical scenarios where these tools can be applied effectively.
  • Development Community and Ecosystem: Examining the collaborative community and supporting environment surrounding the tool’s development and expansion.
  • Openness and Accessibility: Evaluating the tool’s availability and ease of user access.

Training Data

Both ChatGPT and Bard AI leverage substantially vast training data to enhance their capabilities. This diverse training data enables both models to acquire a broad understanding of language and engage in meaningful conversations.

ChatGPT

GPT-3.5 model is trained on 300B tokens gotten from WebText2 or OpenWebText2 (22%), which is a massive dataset between 66GB and 195GB of uncompressed text that covers all Reddit submissions from 2005 up until April 2020, although it is trained on 17GB of text data in total. It is also trained on the following sources:

  • 60% 2016 – 2019 Common Crawl (websites, metadata, and text extracts)
  • 16% Books (science, social science, and art literature)
  • 3% Wikipedia

It is trained in a semi-supervised (Reinforcement Learning from Human Feedback (RLHF)) manner with human-generated dialogues and conversations. Most of its source data end in 2021.Bing AI uses ChatGPT as a component, but it does not have any access to the internet.

When a question is posed in Bing AI, ChatGPT generates a conversation starter to query Bing’s search index for relevant results. The retrieved information is then presented in a conversational format, with ChatGPT generating responses to subsequent inquiries.

While ChatGPT is a key component, Bing AI leverages additional elements, including its own expansive language model and knowledge graph, to deliver a more comprehensive and informative user experience beyond the capabilities of ChatGPT alone.

Bard AI

The underlying neural network is trained on a dataset called Infiniset which is a massive dataset of text and code. It is a blend of Internet content deliberately chosen to enhance the model’s ability to engage in dialogue. Infiniset is composed of the following sources:

  • 50% Dialogs data from public forums
  • 12.5% C4 data
  • 12.5% Code documents from sites related to programming like Q&A sites, tutorials, etc
  • 12.5% Wikipedia (English)
  • 6.25% English web documents
  • 6.25% Non-English web documents

Infiniset is a valuable resource for training large language models. It provides a wide range of content that can help models learn about different topics and to develop a better understanding of language. It was trained on 2.9 billion documents, 1.12 billion dialogs, and 13.39 billion dialog utterances. Bard maintains up-to-date information by continuously accessing and integrating data from the internet.

Bard AI is also trained in a semi-supervised manner (RLHF)

In general, ChatGPT and Bard AI are trained on large datasets and have access to extensive sources of information. Both models provide valuable resources for understanding language and delivering comprehensive user experiences, albeit with different training approaches and access to information.

Model Architecture

Generally, both LLMs share similar building processes and characteristics, which include:

  • Transformer Architecture: The core of both models is the transformer neural network architecture, which has revolutionized natural language processing tasks. Transformers consist of self-attention mechanisms that allow the model to weigh the importance of different words in a sentence and capture long-range dependencies efficiently. This architecture enables both models to understand and generate coherent text based on the context provided.
  • Pre-training and Fine-tuning: They follow a two-step process: pre-training and fine-tuning.During pre-training, the model is exposed to a massive amount of text data from the internet to learn the statistical patterns and structures of language. It predicts the next word in a sentence, utilizing a technique called unsupervised learning.
  • Language Representations: They both form language representations by creating word embeddings, which map each word to a high-dimensional vector representation. These embeddings capture semantic and syntactic information, allowing the model to understand relationships between words and generate meaningful responses.
  • Layer Stacking: Their architectures consist of multiple layers of transformers, allowing them to model complex interactions between words at different levels of abstraction. Each layer comprises self-attention mechanisms and feed-forward neural networks. The deep architecture with stacked layers enables the model to learn hierarchical representations and capture nuanced patterns in text.
  • Contextual Embeddings: They generate contextual word embeddings, which capture the meaning of a word based on its surrounding context. This contextual understanding helps the model generate responses that are coherent and contextually relevant.
  • Inference and Generation: Once the model is trained, it can be used for various tasks. Given an input prompt, They utilize their understanding of the context to generate text that follows the given prompt. It considers the preceding text to generate the most probable next word or sequence of words, resulting in coherent and contextually appropriate responses.

While these models share a similar development process, they possess distinct elements in their model architecture that set them apart from each other:

ChatGPT

The GPT (Generative Pretrained Transformer) model architecture has had upgrades over time, starting from GPT-1 in June 2018 to GPT-4 March 2023.

ModelRelease DateArchitectureNumber of modelsParametersMax. Sequence LengthNotable Features and Advancements
GPT-1Jun 2018Transformer decoder followed by linear-softmax activationOne model117 million1024Introduced the GPT architecture
GPT-2Feb 2019Built on GPT-1 with modified normalizationOne model1.5 billion2048Larger size, improved performance
GPT-3May 2020Built on GPT-2 with modification to allow larger scalingOne model175 billion4096Unprecedented size and performance
GPT-3.5January 2022
(*ChatGPT was released in November 2022)
Built on GPT-3 with Reinforced Learning and Human Feedback (RLHF)3 models (code-davinci-002, text-davinci-002 and ChatGPT)1.3 billion, 6 billion, and 175 billionBetween 4096 to 8001Improved performance from the fine-tuning GPT-3
GPT-4March 2023Builds on RLHF and multimodal (hybrid neural network architecture)Two models
GPT-4-8K
GPT-4-32K
Between 1 to 170 Trillion parameters (estimate)8,192
32,768
With its multimodal capabilities, the model can process images as input and interpret them in a manner similar to a textual prompt

Table 1: This shows the different GPT model iterations, release dates, information about their architecture, and notable features and advancements to each model.

different interactions and resulting models

Fig 2. This shows the different interactions and resulting models from the GPT-3 series. Source

 simple multimodal model

Fig 3. This demonstrated the idea behind a simple multimodal model that predicts house prices from input images, text, and numerical values. This gives an insight into how GPT-4 works with different types of inputs. Source

Bard AI

LaMDA, or Language Model for Dialogue Applications, represents a collection of conversational LLMs. Initially known as Meena upon its introduction in 2020, the model has undergone subsequent refinements and now serves as the foundation for Bard AI. Here are some details about the current model.

ModelRelease DateArchitectureNumber of modelsParametersMax. Sequence LengthNotable Features and Advancements
LaMDA (Bard AI)May 2022Decoder-only transformer with gated-GELU activationOne model2 billion
8 billion
137 billion
2560
4096
8192
Great for dialogues and content creation. Also has an ad-hoc information retrieval feature for providing relevant pages based on user prompts.

It is important to note that the additional models utilized for GPT3.5 are aimed at augmenting language generation capabilities. In the case of LaMDA, these extra models serve to assess the impact of scaling on some evaluation metrics provided by Google researchers.

Both models are sophisticated and highly trained and built on the transformer neural network architecture, incorporating attention mechanisms. As dynamic and evolving models, they are subject to regular updates based on user interaction and needs.

Deepchecks For LLM VALIDATION

OpenAI’s ChatGPT vs. Google’s Bard AI: A Comparative Analysis

  • Reduce Risk
  • Simplify Compliance
  • Gain Visibility
  • Version Comparison
TRY LLM VALIDATION

Contextual Understanding

Contextual understanding refers to the ability of a system or model to comprehend and interpret information within its given context. It involves understanding the nuances, relationships, and dependencies present in a conversation or text and using that understanding to generate meaningful responses or take appropriate actions.

It goes beyond simply recognizing individual words or phrases and involves grasping the broader meaning and implications based on the surrounding context. It enables language models to generate coherent and contextually relevant responses, leading to more effective communication and interaction with users.

To compare the levels of contextual understanding, both tools are given certain questions to test their contextual understanding under the following themes:

  • Abstract Reasoning
  • Ethical Dilemma
  • Contextual Inference
  • Complex Scientific Inquiry
  • Analyzing Literary Themes

Abstract Reasoning

Engaging in logical and conceptual thinking to solve problems or understand concepts without relying on concrete examples or specific context.

Question: “If all humans disappeared from Earth, how would it impact the ecosystem in the long term?”

ChatGPT:

  • It gives a balanced outlook on the topic of human extinction, clearly stating the positives and the negatives, eventually balancing the answer with the homeostatic behavior of nature.
  • The answer shows a clear but surface understanding of the different effects, which can be broadened with more research into the topic. Extra prompts can further your understanding, or you can use a search engine for research.

Chat GPT

Chat GPT

Bard AI:

  • This answer favors one viewpoint and doesn’t show the nuance of the complex situation. This might not be directly its fault, but these answers might be based on the data it’s fed. Especially that of Climate change because it is still a hotly debated topic.
  • It mentions “without us,” forgetting that it is not sentient in this particular context. This might be due to the model’s incline for conversational behavior mimicking a typical human dialog.

Brad AI

Ethical Dilemma

A situation where one must make a decision between two or more morally conflicting choices, often involving ethical principles and considerations.

Question: “Should autonomous vehicles prioritize the safety of the passenger or pedestrians in the event of an unavoidable accident?”

ChatGPT:

  • The answer provided clearly understands the context and comes up with a more academically detached response trying to explore different perspectives.
  • It is conscious not to provide examples or personal thoughts that might be seen as suggesting a solution to the ethical issue.

Chat GPT

Bard AI:

  • While the tool understands the complexity of the question and offers answers, its responses may not be as structured as those of ChatGPT.
  • With its human-like conversational responses, the AI tool provides examples and suggestions for addressing ethical issues involving humans, as a normal human-to-human conversation would do. The question is, should AI tools give us suggestions for what to do in the context of ethics? Or should it help us think about our options?

Brad AI

Contextual Inference

Drawing conclusions or making inferences based on the surrounding context or information available, filling in missing details, or connecting the dots.

Question: “Investigate and analyze the philosophical implications of Gödel’s incompleteness theorems in relation to the foundations of mathematics and the limits of formal systems, considering their impact on logic, computability theory, and the nature of truth”

ChatGPT:

  • It demonstrates a strong ability to comprehend the question and effectively make contextual inferences, particularly in filling in the missing information. This response is a valuable starting point, especially for users seeking to understand Gödel’s incompleteness theorems.
  • It draws conclusions from analyzing the two theorems without mentioning them.

Chat GPT

Bard AI:

  • Bard encounters challenges when it comes to philosophical questions like these due to their intricate nature and nuanced considerations. However, with a few prompts, it demonstrates an ability to grasp the essence of the question being posed. On the other hand, it handles other types of inquiries with relative ease, such as:
    • What is the meaning of life?
    • What is the nature of free will?
    • What is the nature of justice?
  • It understood the question and, while offering some insights, excelled in guiding the user to form their own connections and interpretations, allowing the philosophical aspect to be explored individually.

Brad AI

Brad AI

Complex Scientific Inquiry

Investigating and exploring intricate scientific phenomena or concepts through research, experimentation, analysis, and critical thinking.

Question: “Can you explain the principles behind quantum entanglement and its implications for quantum computing?”

ChatGPT:

While it provides a satisfactory explanation, it falls short in terms of meeting the requirements of a scientific inquiry, particularly in providing citations and incorporating up-to-date data on the subject.

Chat GPT

Chat GPT

Bard AI:

Bard employs citations to explain the concept at hand and exhibits indications of critical thinking in its response. Being a scientific inquiry, it is important for the information provided to be up to date and supported by references, indicating that it is not based solely on the AI’s own knowledge.

Brad AI
Brad AI

Analyzing Literary Themes

Examining and interpreting the underlying ideas, messages, or motifs in a literary work to gain insight into the author’s intentions and explore deeper meanings within the text.

Question: “In George Orwell’s ‘1984,’ what is the significance of the phrase ‘Big Brother is watching you’?”

ChatGPT:

In a highly structured manner, it adeptly explores the underlying ideas, offering reflections that provide valuable insights for the user to comprehend the significance of the phrase.

Chat GPT

Bard AI:

Although it lacks a strict structure, it demonstrates a commendable ability to grasp the context and effectively explain the meaning of the sentence. Also, it offers a valuable link to an important resource, allowing the user to delve deeper into their inquiry.

Brad AI

Limitations and Bias

One common challenge faced by language models is the potential for bias and incorrect or inappropriate responses. Both ChatGPT and Bard AI have been trained on data from the internet, which can contain biased or objectionable content. Both OpenAI and Google have made efforts to mitigate these issues, but they may still arise to some extent.

In general, ChatGPT typically leans towards engaging in structured conversations, aiming for thoroughness and displaying the ability to detach from answers in order to maintain objectivity. Although, having access to recent information through a search engine would have provided an added advantage.

On the other hand, Bard AI adopts a warm conversational style, which may occasionally result in opinionated responses and potential challenges in grasping nuanced requirements. However, the advantage of Bard AI lies in its access to a search engine, enabling it to offer more current information. Bard AI can be considered as a conduit for leveraging the Google search engine, which serves as a strategic approach to boost user searches on the platform.

Both models exhibit remarkable contextual understanding and programming. They can be employed based on the specific task at hand, leveraging their respective strengths and characteristics.

Applications and Use Cases

ChatGPT, based on the GPT-3.5 architecture, offers exceptional contextual understanding and the capability to maintain coherent conversations over multiple turns. This makes it well-suited for applications such as customer support, where it can provide personalized and helpful responses to user inquiries. For instance, ChatGPT can be integrated into a website’s chatbot to assist users with product information, troubleshooting, or general inquiries.

On the other hand, Google Bard AI aims to provide human-like conversational abilities. It may offer similar features and use cases as ChatGPT, enabling applications such as virtual assistants. Google Bard AI can assist users with providing recommendations and answering general knowledge questions. For example, a virtual assistant powered by Bard AI can help users manage their calendars, make restaurant reservations, or retrieve information from the web.

AI features Fig 4. Image showing a use case of Bard AI on Google’s search engine. Source

They enable more interactive and engaging conversations between humans and AI, providing valuable assistance and generating human-like responses. Their potential applications span various domains, bringing benefits to customer service, virtual assistance, content creation, language translation, and more.

Development Community and Ecosystem

ChatGPT has gained popularity among developers and researchers due to its availability, and it was the first to make LLMs accessible to the public. This helped it gain a massive active community around it that continues promoting its usage, creates plugins, and provides feedback.

OpenAI has encouraged developers to build applications and tools using ChatGPT, creating diverse applications and integrations. Since its inception in November 2022, ChatGPT has over 100 million users with an estimated 1.8 billion visitors per month.

Open AI

Fig 5. OpenAI’s developer community message board. Source

Bard AI is publicly available through APIs, although the APIs are currently in beta and are only available to a limited number of users. Enterprise customers can sign up for Vertex AI LLM program to test it, while independent developers can join their MakerSuite and PalmAPI waitlist.

It is worth noting that when you have access to this API, it doesn’t automatically have access to the internet like the tool itself. It answers your queries from its stored database.

Google Cloud’s community

Fig 6. Google Cloud’s community forum message trail. Source

Openness and Accessibility

OpenAI has made efforts to make ChatGPT accessible to the public, providing access to the model through various APIs. This might be due to OpenAI’s effort to collect user feedback, as it is still in the research phase. Although it is accessible to everyone with a mobile device and an internet connection, they have a premium version of ChatGPT (ChatGPT Plus), which utilizes GPT-4 architecture.

Initially, OpenAI was established as a nonprofit organization with an open-source approach, reflected in its name having “Open.” The primary intention was to provide an alternative to Google and foster a more balanced landscape. However, it has undergone a transformation, shifting to a closed-source model and operating as a profit-driven entity under significant influence from Microsoft.

Google’s Bard AI is publicly available as an experimental product. They are still in the experimental phase and are only available to a limited number of users. Google has not yet announced any plans to make Bard AI or the LaMDA model available to the general public.

Generally, both tools are computationally expensive in relation to the demand on it, so it makes some sense to be closed source or limit usage, but here are a few effects it might have on the industry:

  • As a closed-source company, developments and advancements may become more exclusive, limiting the accessibility of their technologies to a wider range of stakeholders. This could potentially reduce the availability of open-source AI solutions and create a more concentrated market dominated by a few major players, i.e., Microsoft (OpenAI) and Google.
  • The open-source approach encourages collaboration and innovation by allowing developers and researchers to freely access and contribute to AI technologies. With closed-source models, there might be a decrease in the level of collaboration and knowledge sharing, which could potentially slow down the pace of innovation and limit the collective progress of the industry.
  • OpenAI’s alignment with a profit-driven agenda under the influence of Microsoft raises questions regarding the prioritization of ethical considerations and societal impact. It is important for industry stakeholders and policymakers to monitor and address any potential conflicts of interest, ensuring that AI technologies developed by profit-driven companies are guided by ethical principles and uphold the interests of various stakeholders, including users and the broader society.
  • The transition of OpenAI may impact the accessibility of AI technologies for smaller organizations or individuals who heavily rely on open-source solutions. The availability of advanced AI models and tools developed by OpenAI might become limited or require licensing, potentially creating barriers to entry for those without significant resources or partnerships with commercial entities.

Amidst these concerns for the industry, a leaked Google memo acknowledges that Open Source AI has a competitive edge in the AI arms race compared to their own closed-source models and those of OpenAI. This has sparked discussions, with some advocating for increased open-source initiatives while others express fears about the potential chaos if proper regulations are not put in place. Overall, the rapid advancement of LLMs in recent months has been remarkable and shows no signs of slowing down.

Conclusion

The comparative analysis of OpenAI’s ChatGPT and Google’s Bard AI highlights the strengths and unique aspects of each tool. ChatGPT, based on the GPT-3.5 architecture, demonstrates exceptional contextual understanding and is well-suited for applications like chatbots, virtual assistants, and content generation. On the other hand, Google’s Bard AI offers human-like conversational abilities and can be utilized in similar use cases.

While both tools have their merits, it is essential to consider the specific requirements and objectives of each use case. Organizations and developers may opt for ChatGPT if they prioritize contextual understanding and coherent conversations, while Bard AI may be chosen for its conversational abilities and integration with other Google services.

Moreover, it is worth noting that the landscape of AI tools and technologies is continuously evolving. New advancements, updates, and releases can introduce further choices and considerations for users. It is crucial to stay informed about the latest developments and evaluate tools based on their specific capabilities, performance, and alignment with project requirements.

Ultimately, the choice between ChatGPT and Bard AI depends on factors such as contextual understanding needs, conversational capabilities, integration requirements, and the overall goals of the project or application. Conducting thorough evaluations and considering the unique strengths and limitations of each tool will assist in making an informed decision for leveraging these AI models effectively.

Resources

Deepchecks For LLM VALIDATION

OpenAI’s ChatGPT vs. Google’s Bard AI: A Comparative Analysis

  • Reduce Risk
  • Simplify Compliance
  • Gain Visibility
  • Version Comparison
TRY LLM VALIDATION

Recent Blog Posts

LLM Evaluation With Deepchecks & Vertex AI
LLM Evaluation With Deepchecks & Vertex AI
The Role of Root Mean Square in Data Accuracy
The Role of Root Mean Square in Data Accuracy
5 LLMs Podcasts to Listen to Right Now
5 LLMs Podcasts to Listen to Right Now