SageMaker Inference Supports OpenAI-Compatible APIs
Amazon SageMaker Inference now supports OpenAI-compatible APIs, allowing direct integration with tools like OpenAI SDK and LangChain by simply changing the endpoint URL. This feature simplifies connecting to SageMaker endpoints, enabling users to leverage existing code and authentication with custom models and VPCs. It offers flexibility in instance choice, data privacy, model execution, and autoscaling, with authentication managed via AWS credentials. The capability is available today in multiple AWS regions.
- →OpenAI-compatible API support for SageMaker inference endpoints
- →Enhanced flexibility and control
- →Simplified integration and authentication
- →Global Availability
Features (2) ›
- OpenAI-compatible API support for SageMaker inference endpoints
SageMaker Inference now supports OpenAI-compatible APIs, enabling direct integration with tools like the OpenAI SDK, LangChain, and Strands Agents. Users can connect to SageMaker endpoints simply by updating the endpoint URL, without requiring custom integration code or SDK wrappers.
- Enhanced flexibility and control
Users can now choose their own GPU instances, maintain data within their VPC, run any open-source or fine-tuned model, and utilize auto-scaling policies tailored to specific workloads.
Enhancements (1) ›
- Simplified integration and authentication
This update eliminates the need to adopt different API formats or change authentication approaches. Existing SDK calls, streaming logic, and framework integrations continue to function seamlessly with automatic AWS credential management and token refresh.
Notes (1) ›
- Global Availability
This capability is now available in numerous AWS regions across North America, Asia Pacific, Europe, and South America.
https://aws.amazon.com/about-aws/whats-new/2026/05/amazon-sagemaker-ai-openai-apis/
Related releases
- SageMaker Unified Studio Enhances Git Version Control Across Project Tools AWS What's New ·
- AWS Glue Data Quality Adds Distribution Statistics for Data Profiling AWS What's New ·
- SageMaker Inference Adds NVIDIA G7 Instances for Faster LLM Deployment AWS What's New ·
- Amazon SageMaker AI inference adds G6 instances in AWS GovCloud AWS What's New ·
- Amazon SageMaker G7e instances expand to Seoul, London, and Tokyo AWS What's New ·
- SageMaker Studio Integrates with Amazon OpenSearch for Unified Data Analysis AWS What's New ·