Skip to content
EN
Get API key
Browse documentation

API reference

Batch and bulk

Choose Batch for independent queries with different inputs. Choose Bulk for a file of video IDs that all need the same operation.

On this page

Choose by the work you need to do

The choice depends on the shape of your input, not a fixed number of items. Both modes run in the background. For one query, use realtime or async directly.

Your taskChoose
Search several topics for a daily briefingBatch: Each query has its own search terms, location or language.
Combine Google results and YouTube metadata in a research jobBatch: Mix sources and collect a separate result for each query.
Refresh metadata or collect transcripts for a video catalogBulk: One video-ID file, one source and the same options for every video.
Deliver a list of videos to a bucket in the same formatBulk: Use youtube_download with shared media settings and destination storage.
Use different output settings for each videoBatch: Per-query parameters can differ; Bulk applies one set to the whole file.

Batch: independent queries

Send a JSON object containing queries. Each item has its own source, input and params. Use Batch for Google searches, YouTube searches, mixed sources, or videos that need different options.

Batch input · two different tasks
{
  "queries": [
    {
      "source": "google_search",
      "query": "open source speech recognition",
      "params": {
        "num": 5
      }
    },
    {
      "source": "youtube_metadata",
      "video_id": "dQw4w9WgXcQ",
      "params": {}
    }
  ]
}

Bulk: one operation over a file

Upload a UTF-8 file with one YouTube video ID per line. Select one supported source and one set of options for the entire file. Bulk scans the file, submits work within account capacity and packages results into downloadable parts.

  1. Choose metadata, transcript, captions, license classification or download. Bulk takes video IDs; use Batch for keyword searches.
  2. Use a separate job when the source or options change. For example, metadata and transcripts for the same file are two Bulk jobs.
  3. For downloads, provide destination storage and owner access with youtube_download permission. Result rows describe delivered media; media files are stored in your bucket.
video-ids.txt
dQw4w9WgXcQ
jNQXAC9IVRw

Plan how you will collect results

ResponsibilityBatchBulk
Save after submissionEvery queries[].id, linked to its original input. There is no single Batch job ID.One Bulk job ID and the unchanged input file.
Track completionEach job finishes independently; one failure does not describe the whole batch.One job exposes progress counts. completed means results are packaged, not that every video succeeded.
Collect outputFetch each completed job's result and retain its input-to-ID mapping.Fetch the manifest and every compressed NDJSON part. Join by index and video_id, not row order.
Resume after interruptionRead the saved job IDs before considering new submissions.Check the saved Bulk ID. Resolve pauseReason and resume that job if paused.

Check capacity and cost before starting

CheckWhat to account for
BatchYour plan's maxBatchSize limits queries per submission. Split larger sets into smaller groups and respect pending-job capacity.
BulkUp to 2 GiB per ID file. Work is submitted gradually within shared account capacity; remove duplicates before upload.
CreditsGrouping is not one flat charge. Queries use per-source pricing; downloads are metered by delivered data size.
AccessThe key must allow every requested source. Check the active account's limits; a trial is not a full-capacity account.