Provisioned Throughput Could Cut Azure OpenAI Costs

• The Azure OpenAI service, which can be used to build applications driven by generative AI, offers a Provisioned Throughput Unit purchase model.
• The purchase model provides guaranteed capacity for an upfront monthly customer commitment per provisioned OpenAI service resource.
• The purchase model could help organizations cut costs compared to using the standard purchase model, but estimating the capacity needed for new projects may be difficult.

The Azure OpenAI service Provisioned Throughput Unit (PTU) purchase model is similar to reservations available with other Azure services, although it requires a monthly (rather than a one- or three-year) customer commitment, and it is applied on a per-resource basis rather than pooled across them. The PTU purchase model could make the Azure OpenAI service more cost effective, although it is most viable for customers with a good understanding of their applications’ capacity demands, so it might not be viable for new projects.

AI-related services can be difficult to meter because their consumption of compute and storage resources varies greatly across usage scenarios. Microsoft is likely offering the PTU purchase model for the OpenAI service to make its own costs more predictable as it promotes the service and anticipates rapid adoption. Further tuning of purchasing models for the service will probably occur as the company makes adjustments for customer demand.