OpenAI Batch API for Cost-Efficient Processing

The OpenAI Batch API lets you submit large volumes of requests at a lower cost by processing them asynchronously. Instead of sending each request in real time, you bundle all requests into a JSONL file, upload it, and retrieve results after processing completes - typically within 24 hours. For ShopMax India, this is ideal for overnight tasks like generating product descriptions for thousands of SKUs.

Each line in the JSONL file is a self-contained request object with a custom_id, an HTTP method, a url endpoint, and a body containing the usual chat completions payload. You upload the file using the Files API, create a batch job referencing the file ID, poll for completion, and then download the output file.

The following example generates product descriptions for three ShopMax India products using the Batch API. Each product is a separate request in the JSONL file, processed together at 50 percent cost savings.

It gives the following output,

File uploaded: file-abc123xyz
Batch job created: batch_6789def - Status: validating

Poll the batch status with client.batches.retrieve(batch_job.id) until status equals completed. Then download results with client.files.content(batch_job.output_file_id) and parse each JSONL line using the custom_id to match results back to the original product. ShopMax India can schedule these batch jobs nightly, reducing API costs by up to 50 percent compared to real-time calls.