Google Gemini vs Azure OpenAI GPT: Pricing Considerations

For most of the early 21st century, Google was just about unanimously considered the king of AI. However, with OpenAI’s release of ChatGPT to the public at the end of 2022 and the whirlwind of AI innovation that followed, Google has taken the backseat. Due to the impressive 1M context window and competitive pricing, Gemini could be their chance to take the lead.

Google Gemini

Google’s generative AI offerings have been hard to follow. In December 2023, not even one year after Bard (Google’s former and short-lived flagship LLM) was announced, Gemini was announced as its “largest and most capable model” and since then, there have been rapid and significant developments.

February 2024, in particular, was a busy month for updates and rebranding efforts. The biggest of which was the rebranding of Bard to Gemini. Gemini builds upon and surpasses Bard in terms of functionalities and scalability. February also saw the consolidation of Duet AI for Workspace, the AI tools used to integrate with Google productivity tools (i.e. Gmail, Docs, and Sheets), and Duet AI for Developers into the Gemini framework.

Gemini is accessible via Google AI Studio or Google Cloud Vertex AI.

Google Gemini Models

Table of supported Google Gemini models (as of 2/29/2024).

Azure OpenAI GPT

OpenAI almost needs no introduction. Since the release of ChatGPT to the public in November 2022, OpenAI GPT models have seen widespread adoption across various sectors. Its impact on the AI landscape has been substantial, catalyzing advances in research and development and fostering increased awareness, with many people referring to AI chatbots and ChatGPT synonymously.

Azure OpenAI is a partnership between Azure and OpenAI that enables Azure users to use OpenAI (including OpenAI GPT models) via an API, Python SDK, or their web-based interface, while authenticated with their Azure cloud credentials. Azure OpenAI distinguishes itself from OpenAI by offering co-developed APIs, enhanced security, and private networking. Throughout this article, “GPT models” refers exclusively to Azure OpenAI GPT models for the sake of brevity.

Azure OpenAI GPT Models

Table of supported Azure OpenAI GPT models (as of 2/29/2024).

Google Gemini vs Azure OpenAI GPT Models Functionality

Gemini has been met with skepticism from users, due in part to Bard falling short of expectations. Gemini 1.0 has received mixed reviews from users, with some saying it falls short of GPT-4 and some saying they prefer it over GPT-4. However, the highly anticipated Gemini 1.5 and its massive context window are receiving glowing accolades from early adopters, with predictions that it is poised to surpass GPT-4. Aside from comparing benchmarks of the two, we can compare them at a service and model level.

Google AI vs Azure AI Service Comparison

As far as service offerings, Google and Azure provide varying levels of support.

• Documentation/Community: Based on anecdotal assessments, the documentation for both services is abysmal. Users of both are complaining about missing information and instructions that are hard to follow. Some users are even complaining that Google’s documentation is incorrect. This is likely because both services and the models within the services are so new and constantly changing.
• Accessibility: Both services can be accessed through APIs and cloud-based studios. Additionally, Azure OpenAI offers an SDK for developers. Both studios provide user-friendly interfaces, which can be particularly helpful for users with limited technical experience.

Google AI Studio

Azure AI Studio

• Fine-Tuning Models: Google does not currently provide support for fine-tuning models. With Azure, you can fine-tune GPT-3.5 Turbo.

• Data Use: Neither Google nor Azure uses customer data to train their AI models.

Gemini vs GPT Model Comparison

As far as the actual models go, there are several quantitative factors to consider.

• Max Tokens: The max token amount varies per model, but both Gemini and GPT models offer similar options, ranging from 8k to 128k tokens. However, Gemini 1.5 takes the overwhelming lead with an unprecedented 1M token context window (though still in preview), providing users with an exceptional capacity for processing large amounts of data. To put it into perspective, 1M tokens correspond to an immense amount of data, equating to approximately “1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code, or over 700,000 words.”
• Supported Regions: Availability may be model and feature-specific. Check Google and Azure to see if your region is supported.
• Supported Languages: Gemini can be used in over 40 different languages, all of which are clearly listed here. It is less clear which languages are supported for the GPT models. OpenAI has stated you can use the GPT models with multiple languages, however, they are optimized for English.
• Training Data Date: The training data date is model-specific. Gemini 1.0 Pro is trained up to February 2023. There is no information on other Gemini versions. GPT-3.5 Turbo and GPT-4 were trained up to September 2021 while the GPT-4 Turbo versions were trained until April 2023.

Google Gemini Models Pricing

Much of Gemini’s pricing is still up in the air. However, Gemini 1.0 Pro pricing is available. Their pricing is a bit different from other popular LLMs because they charge per character instead of token.

Pay-As-You-Go

Charges vary for different model types and input types.

Google Gemini model pricing table (as of 2/29/2024)

Azure OpenAI GPT Models Pricing

Charges for GPT models are fairly simple. It is a pay-as-you-go, with no commitment. There are additional customization charges. Price varies per region and is shown for the US East region.

Pay-As-You-Go

Charges vary for different model types and context.

Azure OpenAI GPT model pricing table (as of 2/29/2024)

Model Customization

Model customization charges are based on training time and hosting time.

Azure OpenAI GPT model customization pricing table (as of 2/29/2024)

Pricing Comparison Gemini vs GPT Models

Based on the available information and pricing, we can compare Gemini 1.0 Pro pricing to GPT-3.5 Turbo and GPT-4 (check back for updates as new pricing is released). It is less of a direct comparison than our previous Bedrock vs OpenAI example since Google charges per character whereas Azure charges per token.

For our estimates, we will operate under the assumption that one token is about four characters (in English). To roughly calculate what Google’s price to tokens is for easier comparison, simply multiply the price by four. The input price per 1k tokens is $0.0005 and the output price per 1k tokens is $0.0015. Again, this is a rough comparison and the actual cost may vary.

GPT-3.5 Turbo is $0.003 per 1k input token and $0.004 per 1k output token. GPT-4 is $0.06 per 1k input token and $0.12 per 1k output token. So, Gemini 1.0 Pro is much cheaper than both GPT-3.5 Turbo and GPT-4. Also, because the costs are per characters instead of tokens it will be easier to estimate Gemini cost.

Pricing Scenario Gemini 1.0 Pro vs GPT-4

To see a more exact pricing analysis, consider the following real-world scenario. A research company wants to analyze large datasets to summarize findings and identify trends and patterns. During one project they scanned several documents.

To estimate how many characters/tokens a research paper might have, we can use this sample that has 8200 words and is approximately 28 pages. This tokenizer calculates the text at 10,664 tokens and 54,112 characters (note—tokenization varies per model). In this case, it’s actually about five times more characters than tokens.

A response is around 500 tokens and 2,500 characters. If the company needs to scan roughly 10,000 similarly sized documents, the price is as follows:

Cost per input character = $0.000125 / 1000 characters = $1.25e-7
Cost per document = $1.25e-7 x 54,112 characters = $0.006764
Cost per 10,000 documents = $0.006764 x 10,000 documents = $67.64

Cost per output character = $0.000375 / 1000 characters = $3.75e-7
Cost per document = $3.75e-7 x 2,500 characters = $0.0009375
Cost per 10,000 documents = $0.0009375 x 10,000 documents = $9.375

Total Cost = $67.64 + $9.375 = $77.02

Cost per input character = $0.06 / 1000 tokens = $0.00006
Cost per document = $0.00006 x 10,664 tokens = $0.63984
Cost per 10,000 documents = $0.63984 x 10,000 documents = $6,398.4

Cost per output character = $0.12 / 1000 tokens = $0.00012
Cost per document = $0.00012 x 500 tokens = $ 0.06
Cost per 10,000 documents = $0.06 x 10,000 documents = $600

Total Cost = $6,398.4 + $600 = $6,998.4

As you can see, Gemini 1.0 Pro is substantially less expensive than GPT-4.

Conclusion

GPT-4 is still widely considered to be the most powerful AI model available on the market. However, based on the extraordinarily competitive pricing of Gemini 1.0 Pro as well as the 1M context window that is to come with Gemini 1.5, Google has positioned itself as a serious competitor in the realm of AI.