Originally published October 31, 2023.
Amazon has been questioned repeatedly by analysts and its own customers about its AI efforts. A quick listen to its Q3 earnings call tells you they’re listening. Much of Amazon’s efforts around Generative AI have culminated in two important releases: Bedrock and Titan.
But shipping AI and winning real-world customers are two very different things. How does Bedrock stack up against OpenAI? In two scenarios we found Amazon Bedrock models provided savings of 30% and a staggering 556% over corresponding Azure OpenAI models (read scenarios).
Editor’s Note: Amazon and Azure are continuously unveiling changes and releases to their AI services. To ensure you receive the most up-to-date and accurate information, we will be updating sections of this blog regularly. As we make these updates, we will mark the revision date on a section-by-section basis.
Types of Generative AI Models Available in the Cloud
Generative AI encompasses several categories of models available via OpenAI and Bedrock:
- Image Generation: As the name suggests, image generation models, such as Stable Diffusion and DALL-E, are designed to create images based on textual descriptions or other image inputs.
- Large Language Model (LLM): These models are extensively trained on vast text data and understand/produce textual content. LLMs power ChatGPT, the most popular consumer-facing Generative AI model.
- Base Model: A Foundation Model that serves as a starting point for further fine-tuning.
- Embeddings: Embeddings models are not focused on generating text, but on representing text or data in a numerical way that captures semantic relationships, e.g. “king” + “woman” = “queen”.
- Text Generation: Specializes in producing human-like text.
- Transcription: These models convert audio into text.
Amazon Bedrock
Bedrock is a fully managed, serverless service, that provides users access to FMs from several third-party providers and from Amazon through a single API. After you select a FM to use you can privately customize it and connect your propriety data sources and knowledge bases.
Bedrock Supported Models
| Model Family | Provider | Type | Functionalities | Max Request |
|---|---|---|---|---|
| Claude (1.3, 2.0, and 2.1) | Anthropic | LLM | Editing, outlining searching, Q&A, summarizing, and writing text. Automated workflows, coding, complex reasoning, content generation, comparison and analysis of documents, dialogue, detailed instruction, and trend forecasting. | 100-200k tokens |
| Claude Instant | Anthropic | LLM | Casual dialogue, comprehension, document comprehension, summarization, and text analysis. | 100k tokens |
| Command | Cohere | LLM | Chat, Q&A, summarization, and text generation. | 4k tokens |
| Command Light | Cohere | LLM | Chat, Q&A, summarization, and text generation. | 4k tokens |
| Embed (English, Multilingual) | Cohere | Embeddings | Classification, clustering, RAG, and semantic search. | 1k dimensions |
| Jurassic-2 Mid | AI21 Labs | LLM | Advanced information extraction, draft generation, ideation, Q&A, and summarization. | 8k tokens |
| Jurassic-2 Ultra | AI21 Labs | LLM | Advanced information extraction, draft generation, intricate ideation, Q&A, and summarization. | 8k tokens |
| Llama 2 (13B and 70B) | Meta | LLM | Dialogue systems, language modeling, language translation, sentiment analysis, and text classification, and text generation. | 4k tokens |
| Stable Diffusion XL (0.8 and 1.0) | Stability AI | Image Generation | Generate, edit, or update images from text. | 77 tokens |
| Titan Embeddings | Amazon | Embeddings | Clustering, semantic similarity, and text retrieval. | 8k tokens |
| Titan Image Generator | Amazon | Image Generation | Text to image generation, editing, and variations. | 77 tokens |
| Titan Multimodal Embeddings | Amazon | Embeddings | Personalization, recommendations, and searching by image and/or text. | 8k tokens |
| Titan Text Lite | Amazon | LLM | Chain of thought, chat, code generation, data formatting, extraction, paraphrasing, Q&A, rewrite, summarization, table creation, and text generation. | 4k tokens |
| Titan Text Express | Amazon | LLM | Chain of thought, chat, code generation, data formatting, extraction, paraphrasing, Q&A, rewrite, retrieval augmented generation, summarization, table creation, and text generation. | 8k tokens |
Table of supported Bedrock models (updated 1/9/24). Titan Image Generator is in preview. There may be several versions in each model family, check the models page for specifics and the most up-to-date information.
Azure OpenAI
Azure OpenAI is a partnership between Azure and OpenAI that enables Azure users to use OpenAI via an API, Python SDK, or their web-based interface while authenticated with their Azure cloud credentials. Azure OpenAI distinguishes itself from OpenAI by offering co-developed APIs, enhanced security, and private networking. Throughout this article, the term “OpenAI” refers exclusively to Azure OpenAI for the sake of brevity.
OpenAI Supported Models
| Model Family | Type | Functionalities | Max Request |
|---|---|---|---|
| Ada | Embeddings | Anomaly detection, classification tasks, clustering, recommendations, and searching. | 8k tokens |
| DALL-E (2 and 3) | Image Generation | Generate, edit, or update images from text. | 1k-4k characters |
| GPT Base (Babbage-002 and Davinci-002) | Base | Generate and understand text or code. Not trained with instruction following. | 16k tokens |
| GPT-3.5 Turbo | LLM | Advanced complex reasoning and chat, code understanding and generation, and traditional completions tasks. | 4k and 16k tokens |
| GPT-4 | LLM | Advanced complex reasoning and chat, advanced problem solving, code understanding and generation, and traditional completions tasks. | 8k and 32k tokens |
| GPT-4 Turbo | LLM | GPT-4 capabilities, instruction following, and parallel function calling. | 128k tokens |
| GPT-4 Turbo With Vision | LLM | GPT-4 Turbo capabilities, images analysis and Q&A. | 128k tokens |
| Whisper | Transcription | Convert audio to text. | 25 MB audio size |
Table of supported Azure OpenAI models (updated 1/9/24). GPT-4 Turbo, GPT-4 Turbo with Vision, Dalle, and Whisper are in preview. There may be several versions in each model family, check the models and the most up-to-date information.
Amazon Bedrock vs Azure OpenAI Functionality
OpenAI certainly has a lot of name recognition. Due to this people have a conception that it is leaps and bounds ahead of other Generative AI services. However, as Randall Hunt, VP of Cloud Strategy and Innovation at Caylent, relayed on in Yan Cui’s Real-World Serverless podcast, “There wasn’t anything crazy great about the way OpenAI did anything, it just happened to be one of the first times we could see the power of these LLMs through an interface.” Still, GPT-4 is generally recognized as the leader in terms of pure quality.
Service Comparison
Let’s compare some functionality on the service and model level to see how they fare. As this is the cloud, when comparing Bedrock to OpenAI we need to consider things like supported regions and security.
- Documentation/Community: Documentation and community support are challenging to quantify precisely, but based on anecdotal assessments, it’s fair to say that the documentation for both services is satisfactory at best. There is a lot of missing information and instructions that are all over the place. This is likely because both services and the models within the services are so new and constantly changing.
- No-Code Playgrounds: While both services are accessible via APIs and SDK as well, a no-code playground can be a helpful interface to utilize some of the models.
- Provisioned Throughput: Bedrock offers a Provisioned Throughput payment plan for some model types that is advantageous for large workloads.
Model Comparison
(Updated 1/9/24)
Similarly, there are several factors to take into consideration when comparing models within their respective categories, such as max tokens, supported languages, and training data date. Later, we’ll go in-depth on pricing and performance.
- Max Tokens: Max tokens range for model categories and types.
- Embeddings Models: Bedrock and OpenAI both have models with 8k tokens.
- Image Generation Models: Characters to tokens vary, however, 1000 characters correspond to roughly 250 tokens. So DALL-E has more tokens than Stable Diffusion and Titan Image Generator.
- LLMs: Both Bedrock and OpenAI provide models with 4k and 8k token options. OpenAI extends the range with 16k and 32k tokens. Bedrock and OpenAI offer models with 100k and 128k tokens, respectively, (corresponding to an impressive 300 pages). However, the Claude model takes the trophy with an impressive maximum capacity of 200k tokens. That corresponds to 500 pages of information.
- Supported Regions: Bedrock is available in Asia Pacific (Singapore), Asia Pacific (Tokyo), AWS GovCloud (US-West), Europe (Frankfurt), US East (N. Virginia), and US West (Oregon). OpenAI regions vary per model as follows:
| Region | Supported Models |
|---|---|
| Australia East | Ada, GPT-3.5 Turbo, GPT-4, GPT-4 With Vision |
| Canada East | Ada, GPT-3.5 Turbo, GPT-4 |
| East US | Ada, DALL-E, GPT-3.5 Turbo, GPT-4 |
| East US 2 | Ada, GPT-3.5 Turbo, GPT-4 |
| France Central | Ada, GPT-3.5 Turbo, GPT-4 |
| Japan East | Ada, GPT-3.5 Turbo, GPT-4 |
| North Central US | Ada, Base Models, GPT-3.5 Turbo, Whisper |
| Norway East | Ada, GPT-4 |
| South Central US | Ada, GPT-3.5 Turbo, GPT-4 |
| South India | GPT-3.5 Turbo, GPT-4 |
| Sweden Central | Ada, Base Models, DALL-E, GPT-3.5 Turbo, GPT-4.5 |
| Switzerland North | Ada, GPT-3.5 Turbo, GPT-4 |
| UK South | Ada, GPT-3.5 Turbo, GPT-4 |
| West Europe | Ada, GPT-3.5 Turbo, Whisper |
| West US | Ada, GPT-3.5 Turbo, GPT-4 |
Azure OpenAI supported models by region (as of 1/9/24). Note that model availability may vary based on the region and service, and there may be certain limitations in some regions.
- Supported Languages: Bedrock language capacity is model-specific. Command, Embed - English, Llama 2, Stable Diffusion, Titan Image Generator, Titan Multimodal Embeddings, and Titan Text Lite support only English. Jurassic supports 7 languages, Claude supports 12+, Titan Embeddings supports 25+, while Embed - Multilingual and Titan Text Express support over 100. See the model pages for the specific languages. OpenAI has less information available on which languages are supported, however, this response claims it is available for use in a variety of languages.
- Training Data Date: OpenAIs models Ada, GPT-3.5 Turbo, GPT-4, and the Base Models are trained until Sept 2021. GPT-4 Turbo is trained until Apr 2023. Bedrock’s training data date was a little harder to find, we had to go to the retrospective provider’s sites to find that the only publicly available dates are Claude’s (December 2022) and Jurassic’s (current up to mid-2022).
Bedrock Pricing
Charges for Bedrock are applied for model inference and customization. There are two plans available for model inference, On-Demand and Provisioned Throughput. Model customization and Provisioned Throughput are not be available for all models. Price varies per region.
On-Demand
The non-committal, pay-by-usage option. Charges vary depending on the model type. Text generation models incur charges per input token processed and output token generated. Embeddings models charge per input token processed. Image generation models charge per image generated.
| Model | Price per 1000 Input Tokens | Price per 1000 Output Tokens |
|---|---|---|
| Claude | $0.008 | $0.024 |
| Claude Instant | $0.0008 | $0.0024 |
| Command | $0.0015 | $0.002 |
| Command Light | $0.0003 | $0.0006 |
| Embed – English | $0.0001 | N/A |
| Embed – Multilingual | $0.0001 | N/A |
| Jurassic-2 Mid | $0.0125 | $0.0125 |
| Jurassic-2 Ultra | $0.0188 | $0.0188 |
| Llama 2 Chat (13B) | $0.00075 | $0.001 |
| Llama 2 Chat (70B) | $0.00195 | $0.00256 |
| Titan Text Lite | $0.0003 | $0.0004 |
| Titan Text Express | $0.0008 | $0.0016 |
| Titan Embeddings | $0.0001 | N/A |
| Titan Multimodal Embeddings | $0.0008 ($0.00006 per input image) | N/A |
Bedrock On Demand LLMs pricing table. (Updated 1/9/24).
| Model | Image resolution | Cost Per Image - Standard Quality (<51 steps) | Cost Per Image - Premium Quality (>51 steps) |
|---|---|---|---|
| SDXL 0.8 (Stable Diffusion) | 512X512 or smaller | $0.018 | $0.036 |
| Larger than 512X512 | $0.036 | $0.072 | |
| SDXL 1.0 (Stable Diffusion) | 1024X1024 or smaller | $0.04 | $0.08 |
| Titan Image Generator (Standard) | 512X512 | $0.008 | $0.01 |
| 1024X1024 | $0.01 | $0.012 | |
| Titan Image Generator (Custom Models) | 512X512 | $0.018 | $0.02 |
| 1024X1024 | $0.02 | $0.022 |
Bedrock On Demand Image Generation Models pricing table. (Updated 1/9/24).
Provisioned Throughput
You have the option to buy model units (specific throughput measured by the maximum number of input/output tokens processed per minute) for a specific model (including custom models). Pricing is charged hourly and you can choose a one-month or six-month term. This pricing model is best suited for “large consistent inference workloads that need guaranteed throughput.”
| Model | Price per Hour per Model Unit With No Commitment (Max One Custom Model Unit Inference) | Price per Hour per Model Unit With a One Month Commitment (Includes Inference) | Price per Hour per Model Unit With a Six Month Commitment (Includes Inference) |
|---|---|---|---|
| Claude | N/A | $63.00 | $35.00 |
| Claude Instant | N/A | $39.60 | $22.00 |
| Command | N/A | $39.60 | $23.77 |
| Command Light | N/A | $6.85 | $4.11 |
| Llama 2 Pre-Trained and Chat (13B) | N/A | $21.18 | $13.08 |
| Llama 2 Pre-Trained (70B) | N/A | $21.18 | $13.08 |
| SDXL1.0 (Stable Diffusion) | N/A | $49.86 | $46.18 |
| Titan Embeddings | N/A | $6.40 | $5.10 |
| Titan Image Generator (Standard) | N/A | $16.20 | $13.00 |
| Titan Image Generator (Custom Models) | $23.40 | $21.00 | $16.85 |
| Titan Multimodal Embeddings | $9.38 | $8.45 | $6.75 |
| Titan Text Lite | $7.10 | $6.40 | $5.10 |
| Titan Text Express | $20.50 | $18.40 | $14.80 |
Bedrock Provisioned Throughput Models pricing table. (Updated 1/9/24).
Model Customization
You’re charged for model customization based on the number of processed tokens and model storage. Keep in mind, inference on more than one model unit is only available for Provisioned Throughput.
| Model | Price to Train 1000 Tokens | Price for Storage per Custom Model per Month | Price to Infer from a Custom Model for One Model Unit per Hour with No Commitment |
|---|---|---|---|
| Command | $0.004 | $1.95 | $49.50 |
| Command Light | $0.001 | $1.95 | $8.56 |
| Llama 2 Pre-trained (13B) | $0.00149 | $1.95 | $23.50 |
| Llama 2 Pre-trained (70B) | $0.00799 | $1.95 | $23.50 |
| Titan Image Generator | $0.005 | $1.95 | $23.40 |
| Titan Multimodal Embeddings | $0.0002 | $1.95 | $9.38 |
| Titan Text Lite | $0.0004 | $1.95 | $7.10 |
| Titan Text Express | $0.008 | $1.95 | $20.50 |
Bedrock Model Customization pricing table. (Updated 1/9/24).
Azure OpenAI Pricing
Charges for OpenAI are fairly simple. It is a pay-as-you-go, with no commitment. There are additional customization charges. Price varies per region.
Pay-As-You-Go
Charges vary for different model types and if applicable, context. Text generation models charge per prompt tokens and completion tokens. Embeddings models and base models charge per usage tokens. Image generation models charge per 100 images generated.
| Model | Context | Price per 1000 Input Tokens | Price per 1000 Output Tokens |
|---|---|---|---|
| Ada | N/A | $0.0001 | N/A |
| Babbage-002 (GPT Base) | N/A | $0.0004 | $0.0004 |
| Davinci-002 (GPT Base) | N/A | $0.002 | $0.002 |
| GPT-3.5 Turbo | 4k | $0.0015 | $0.002 |
| GPT-3.5 Turbo | 16k | $0.003 | $0.004 |
| GPT-4 | 8k | $0.03 | $0.06 |
| GPT-4 | 32k | $0.06 | $0.12 |
OpenAI Pay-As-You-Go LLMs pricing table. (Updated 1/9/24).
| Models | Quality | Resolution | Price (per 100 Images) |
|---|---|---|---|
| Dall-E-2 | Standard | 1024X1024 | $2 |
| Dall-E-3 | Standard | 1024X1792, 1792X1024 | $8 |
OpenAI Pay-As-You-Go Image Generation Model (Dall-E) pricing table. (Updated 1/9/24).
| Models | Per Hour |
|---|---|
| Whisper | $0.36 |
OpenAI Pay-As-You-Go Speech Generation Model (Whisper) pricing table. (Updated 1/9/24).
Model Customization
Model customization charges are based on training time and hosting time with slightly different pricing per region.
| Model | Price for Training per Compute Hour | Price for Hosting per Hour |
|---|---|---|
| Babbage-002 (GPT Base) | $34 | $1.70 |
| Davinci-002 (GPT Base) | $68 | $3 |
| GPT-3.5 Turbo | $102 | $7 |
OpenAI Model Customization pricing table. (Updated 1/9/24).
Pricing Comparison: Bedrock vs OpenAI
(Updated 1/9/24)
On a model-by-model comparison, Bedrock is cheaper than OpenAI. However, cost does not tell the full story and the scenarios below are based purely on an analysis of pricing.
Standard Context Window: Command, Llama 2, Titan Text vs GPT-3.5 Turbo 4k
For a lower capacity model where we want to perform tasks such as chat, summarization on an article-length passage, Q&A, etc, we can consider one of the models with a 4k token max. There is one model from OpenAI that fits the criteria—GPT-3.5 Turbo 4k, and multiple of Bedrock—Command, Llama 2, Titan Text Lite, and Titan Text Express. The pricing for Command and GPT-3.5 Turbo 4k are the same, at $0.0015 per 1000 input tokens and $0.002 per 1000 output tokens.
Titan Text Lite, which can perform many of the same capabilities is much cheaper at $0.0003 per 1000 input tokens and $0.0004 per 1000 output tokens. Llama 2 is $0.00075 per 1000 input tokens and $0.001 per 1000 output tokens. Another option is Titan Text Express, the difference between the Lite version is that it has retrieval augmented generation ability and a maximum of 8k tokens. The price is $0.0008 per 1000 input tokens and $0.0016 per 1000 output tokens, cheaper than GPT-3.5 Turbo 4k.
Chatbot Scenario: Titan Text Express vs GPT-3.5 Turbo 4k
Consider a scenario where you want to develop a simple customer service chatbot. The chatbot will need to be able to handle customer inquiries, provide assistance, and answer questions on a range of topics related to your products and services. The model will need to handle short sentences as well as more detailed discussions.
A standard question could be about 15 tokens and the answer could be 85. If your chatbot is answering 250,000 similar tokened questions a month the estimated price would be:
15 tokens X 250,000 questions = 3,750,000 input tokens
85 tokens X 250,000 answers = 21,250,000 output tokens
Titan Text Express: 3,750,000 input tokens / 1000 X $0.0008 + 21,250,000 output tokens / 1000 X $0.0016 = $37
GPT-3.5 Turbo 4k: 3,750,000 input tokens / 1000 X $0.0015 + 21,250,000 output tokens / 1000 X $0.002 = $48
GPT-3.5 Turbo 4k is 30% more than Titan Text Express, making Bedrock the cheaper option for a lower-capacity model.
Long Context Window: Long Context Window: Jurassic-2 vs GPT-4 8k
For more advanced tasks such as advanced information extraction, draft generation, and summarization on larger passages, let’s compare some of the models with an 8k token max. Between the Jurassic-2 models the Ultra model stands out because of intricate ideation, and therefore compares well with GPT-4 8k. Jurassic-2 Ultra is much cheaper at $0.0188 per 1000 input tokens and $0.0188 per 1000 output tokens when compared to GPT-4 8k’s $0.03 per 1000 input tokens and $0.06 per 1000 output tokens.
Long Context Window: Claude Instant vs GPT-3.5 Turbo 16k
For even larger tasks consider Claude Instant (100k token max) and GPT-3.5 Turbo 16k (16k token max). The capabilities and pricing are relatively similar. However, the choice is much more case-dependent since with Claude Instant you are charged $0.0008 per 1000 input tokens and $0.0024 per 1000 output tokens, compared to $0.003 per 1000 input tokens and $0.004 per 1000 output tokens for GPT-3.5 Turbo 16k. So, because of the lower pricing, Claude Instant would be a great choice with a higher token input amount and lower output token amount.
Extra Long Context Window: Claude 2.1 vs GPT-4 32k
For high-capacity models with very advanced tasks, such as content generation and complex reasoning, consider Claude 2.1 vs GPT-4 32k. Claude 2.1 has an impressive maximum of 100k tokens, while GPT-4 32k provides 32k tokens. Claude 2.1 is a great choice since it is way cheaper at $0.008 per 1000 input tokens and $0.024 per 1000 output tokens. GPT-4 32k is $0.06 per 1000 input tokens and $0.12 per 1000 output tokens.
Text Summarization Scenario: Claude 2.1 vs GPT-4 32k
You work at a content creation agency and need to summarize lengthy articles and reports for clients. You want to process articles at around 25,000 tokens and summarize them to about 5,000 tokens. If you process 300 articles a month consider the estimated prices:
25,000 tokens X 300 articles = 7,500,000 input tokens
5,000 tokens X 300 responses = 1,500,000 output tokens
**Claude 2.1 **: 7,500,000 input tokens / 1000 X $0.008 + 1,500,000 output tokens / 1000 X $0.024 = $96
GPT-4 32k: 7,500,000 input tokens / 1000 X $0.06 + 1,500,000 output tokens / 1000 X $0.12 = $630
Claude 2.1 is remarkably cost-effective, positioning Bedrock as the more economical choice, with 556% cost savings in this situation.
In terms of functionality, this article does a great in-depth comparison of the the functionality of the two models and concludes that GPT-4 32k performs slightly better. Some takeaways are a similar performance for code generation and conversion, a better performance for GPT-4 32k’s dataset analysis and math skills, and Claude’s 2.1 distinct ability to summarize text over 32k tokens.
Conclusion
There are a number of dimensions to consider when comparing Bedrock and OpenAI, such as region availability, tokens, model quality, and price. Based on the variety of models, lower price, and large token max from the Claude Model, we think Bedrock is increasingly competitive for applications where the absolute best performance is not required.
