Batch Processing API
Batch processing is a powerful tool for handling large volumes of requests efficiently and cost-effectively. Instead of processing requests one at a time with immediate responses, you can submit multiple requests together for asynchronous processing.
Why Use Batch processing?
- Cost Efficiency: All usage in batch requests is charged at 50% of the standard API prices.
- High Throughput: Optimized for analytical or offline workloads with large data volumes.
- Asynchronous: Submit your batch and retrieve results whenever they are ready (typically within 24 hours).
Common Use Cases
- Processing large datasets for analysis.
- Running large-scale evaluations or benchmarks.
- Generating content for bulk tasks where immediate response is not required.
API Methods
| Method | Endpoint | Description |
|---|---|---|
| Create New Batch | POST /v1/batches |
Submit a new batch of requests for processing. |
| Get Batch Status | GET /v1/batches/{batch_id} |
Check the status and retrieve results of a batch. |
| Cancel Batch | POST /v1/batches/{batch_id}/cancel |
Stop a pending or in-progress batch. |
Request Limits
Batch requests are subject to specific limits depending on your account tier. For more details, see the Pricing and Fee Structure.