Browse documentation
API reference
Create and control file-based YouTube jobs, then retrieve their manifests and result parts.
On this page
API playground
Choose a source and copy a complete Python or Bash workflow, a single request, or a sample response. The playground previews code; it does not submit a job. Complete workflows handle upload, status checks and retrieval of every result part.
Build a Bulk workflow
Local UTF-8 file, one YouTube ID per line, up to 2 GiB. Input requirements.
Leave blank to try en-orig, then en, and return one track. Or enter one code or up to 10 distinct codes separated by commas.
Result delivery
Leave this off to retrieve results through the FlashData API.
Bulk completion webhook
Manifest-completion notification and verificationhttps://api.flashdata.dev/v2/bulk-jobsEndpoints
All endpoints use https://api.flashdata.dev/v2. Create a job → upload a video-ID file → start → monitor → retrieve the manifest and every result part.
| Method | Path | Action |
|---|---|---|
| POST | /v2/bulk-jobs | Create a Bulk job |
| POST | /v2/bulk-jobs/{bulkId}/start | Upload and start |
| GET | /v2/bulk-jobs/{bulkId} | Get job status |
| GET | /v2/bulk-jobs | List jobs |
| POST | /v2/bulk-jobs/{bulkId}/pause | Pause a job |
| PATCH | /v2/bulk-jobs/{bulkId} | Update job options |
| POST | /v2/bulk-jobs/{bulkId}/resume | Resume a job |
| DELETE | /v2/bulk-jobs/{bulkId} | Cancel a job |
| GET | /v2/bulk-jobs/{bulkId}/results | Get the result manifest |
| GET | /v2/bulk-jobs/{bulkId}/results/{partNumber} | Download result parts |
Authentication and input
Set FLASHDATA_API_KEY to a key with access to the chosen source. Reuse the same key and account for creation, controls and results. Processing uses the account's balance and normal per-source pricing. YouTube Download requires owner access and options.storage.
Create a working folder named bulk-api/ and run these examples from it. The playground’s complete Python workflow uses the Python 3.10+ standard library; the individual Python endpoint examples below require Requests; cURL examples use Bash, cURL 7.76+, jq and gzip. Choose one language for each action so you do not create or start duplicate work.
Input is a UTF-8 file with one 11-character YouTube ID per line, up to 2 GiB. Use IDs, not URLs. Remove duplicate IDs before uploading; repeated IDs are processed separately.
| Source | Shared options |
|---|---|
| youtube_metadata | include_formats, include_chapters |
| youtube_transcript | language, auto |
| youtube_captions | language, auto |
| youtube_trainability | No options |
| youtube_download | Media options and storage destination |
python -m pip install requestsCreate request fields
| Field | Type | Usage |
|---|---|---|
| source | string | Required. One supported source for every input ID. |
| options | object | Shared source options. Data sources may set result_storage; YouTube Download requires storage. |
| callback_url | string | Optional public HTTPS completion callback. Sent after manifest publication. |
Create a Bulk job
Create a job, upload a UTF-8 text file with one 11-character YouTube video ID per line to upload.href, then call start. Creating the job alone does not begin processing.
This example writes video-ids.txt and saves the 201 receipt to bulk-created.json. Keep that receipt private and reuse job.id for every later request. Creating a job leaves it in awaiting_upload; the upload URL expires after 15 minutes.
POST https://api.flashdata.dev/v2/bulk-jobs
X-API-Key: YOUR_API_KEYSave bulk_create.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
os.umask(0o077)
Path("video-ids.txt").write_bytes(b"jNQXAC9IVRw\ndQw4w9WgXcQ\n")
response = requests.post(
"https://api.flashdata.dev/v2/bulk-jobs",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
json={"source": "youtube_metadata", "options": {"include_formats": False}},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
receipt = response.json()
Path("bulk-created.json").write_text(json.dumps(receipt, indent=2), encoding="utf-8")
print(json.dumps(receipt["job"], indent=2))python bulk_create.pyumask 077
printf '%s\n' 'jNQXAC9IVRw' 'dQw4w9WgXcQ' > video-ids.txt
curl --fail-with-body --silent --show-error \
'https://api.flashdata.dev/v2/bulk-jobs' \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--header 'Content-Type: application/json' \
--data '{"source":"youtube_metadata","options":{"include_formats":false}}' \
--output bulk-created.json &&
jq '.job | {id, source, status}' bulk-created.jsonResponses and request schema
| HTTP status | Meaning |
|---|---|
| 201 | Bulk job created with a 15-minute direct upload URL |
| 401 | Authentication required or invalid |
| 422 | Request validation failed |
| 503 | A required dependency or catalog is temporarily unavailable |
{
"$ref": "#/components/schemas/CreateBulkJobRequest"
}{
"$ref": "#/components/schemas/BulkJobCreated"
}Upload and start
Start processing the uploaded UTF-8 video ID file, up to 2 GiB (2147483648 bytes). Keep the file unchanged after starting.
First PUT video-ids.txt to the receipt's upload.href with upload.contentType, without X-API-Key. The example calls start only after upload succeeds. Keep the uploaded file unchanged afterward.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/start
X-API-Key: YOUR_API_KEYSave bulk_start.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
receipt = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))
upload = receipt["upload"]
if upload["method"] != "PUT" or not upload["href"].startswith("https://"):
raise ValueError("Expected an HTTPS PUT upload URL")
# Stream the file to storage without the FlashData key.
with open("video-ids.txt", "rb") as file:
response = requests.put(
upload["href"], data=file,
headers={"Content-Type": upload["contentType"]},
timeout=300, allow_redirects=False,
)
response.raise_for_status()
if not 200 <= response.status_code < 300:
raise RuntimeError("Upload did not complete; do not start the job yet")
bulk_id = receipt["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/start",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json()["job"], indent=2))python bulk_start.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
UPLOAD_URL=$(jq -er '.upload.href' bulk-created.json)
CONTENT_TYPE=$(jq -er '.upload.contentType' bulk-created.json)
curl --fail-with-body --silent --show-error --proto '=https' \
--header "Content-Type: $CONTENT_TYPE" \
--upload-file video-ids.txt "$UPLOAD_URL" &&
curl --fail-with-body --silent --show-error --request POST \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/start" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}"Responses
| HTTP status | Meaning |
|---|---|
| 200 | Bulk job started or its current idempotent state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
| 409 | Resource state or idempotency conflict |
| 422 | Request validation failed |
| 503 | A required dependency or catalog is temporarily unavailable |
{
"$ref": "#/components/schemas/BulkJobResponse"
}Get job status
Read job.status, input, counts, pauseReason and error. Recheck the same ID with a delay; completed means the manifest is published, not that every video succeeded.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEYSave bulk_status.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
Path("bulk-job.json").write_text(json.dumps(response.json(), indent=2), encoding="utf-8")
print(json.dumps(response.json()["job"], indent=2))python bulk_status.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output bulk-job.json &&
jq '.job | {id, status, counts, pauseReason, error, expiresAt}' bulk-job.jsonResponses
| HTTP status | Meaning |
|---|---|
| 200 | Bulk job state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
{
"$ref": "#/components/schemas/BulkJobResponse"
}List jobs
The response contains data and pagination. Pass pagination.nextCursor as the next request's cursor query parameter until it is null. Use limit from 1 to 50; the default is 20.
| Parameter | Location | Requirement |
|---|---|---|
| cursor | query | Optional · {"type":"string","maxLength":512} |
| limit | query | Optional · {"type":"integer","minimum":1,"maximum":50,"default":20} |
GET https://api.flashdata.dev/v2/bulk-jobs
X-API-Key: YOUR_API_KEYSave list_bulk_jobs.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
response = requests.get(
"https://api.flashdata.dev/v2/bulk-jobs?limit=20",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python list_bulk_jobs.pycurl --fail-with-body --silent --show-error --max-time 30 \
--request GET \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs?limit=20"Responses
| HTTP status | Meaning |
|---|---|
| 200 | Bulk job page |
| 401 | Authentication required or invalid |
{
"$ref": "#/components/schemas/BulkJobPage"
}Pause a job
Pause new submissions and result packaging. Existing jobs continue with their original settings and charges. Resume the job when ready.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/pause
X-API-Key: YOUR_API_KEYSave pause_bulk_job.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/pause",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python pause_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request POST \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/pause"Responses
| HTTP status | Meaning |
|---|---|
| 200 | Paused bulk job or its current idempotent state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
{
"$ref": "#/components/schemas/BulkJobResponse"
}Update job options
Update options while awaiting upload, or while paused before all input has been read. Existing jobs keep their settings. If pausing is still in progress, wait and retry a 409 response. After all input has been read, only delivery credentials can be updated. Set callback_url to null to remove it.
This example removes the completion callback. To change source options, send an options object instead. Pause first and wait for the pause to take effect; a 409 means the update cannot be applied yet. Existing child jobs keep their original options.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
PATCH https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEYSave update_bulk_job.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.patch(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
json={"callback_url": None},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python update_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request PATCH \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--header 'Content-Type: application/json' --data '{"callback_url":null}' \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID"Responses and request schema
| HTTP status | Meaning |
|---|---|
| 200 | Updated bulk job state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
| 409 | Resource state or idempotency conflict |
| 422 | Request validation failed |
{
"$ref": "#/components/schemas/UpdateBulkJobRequest"
}{
"$ref": "#/components/schemas/BulkJobResponse"
}Resume a job
Fix pauseReason first, such as adding credits or updating storage credentials, then resume the same Bulk ID.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/resume
X-API-Key: YOUR_API_KEYSave resume_bulk_job.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/resume",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python resume_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request POST \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/resume"Responses
| HTTP status | Meaning |
|---|---|
| 200 | Current bulk job state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
{
"$ref": "#/components/schemas/BulkJobResponse"
}Cancel a job
Stop new submissions. Existing jobs can still finish and incur their normal charges. A cancelled job does not publish a final manifest.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
DELETE https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEYSave cancel_bulk_job.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.delete(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python cancel_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request DELETE \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID"Responses
| HTTP status | Meaning |
|---|---|
| 200 | Current bulk job state |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
{
"$ref": "#/components/schemas/BulkJobResponse"
}Get the result manifest
Follow the redirect to download the JSON manifest and use its links to retrieve result files. Fetch signed storage URLs without your API key. Results remain available for 24 hours after the entire Bulk job completes.
The authenticated API request returns 302 with a Location header. Fetch that HTTPS storage URL in a separate request without X-API-Key. The example saves manifest.json. A 409 means results are not ready; 410 means they expired.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/results
X-API-Key: YOUR_API_KEYSave bulk_manifest.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/results",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
if response.status_code != 302:
raise RuntimeError(f"Expected a manifest redirect, got {response.status_code}")
url = response.headers["Location"]
if not url.startswith("https://"):
raise ValueError("Expected an HTTPS storage URL")
manifest = requests.get(url, timeout=120) # No FlashData API key.
manifest.raise_for_status()
Path("manifest.json").write_text(json.dumps(manifest.json(), indent=2), encoding="utf-8")
print(json.dumps(manifest.json(), indent=2))python bulk_manifest.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
# Only send the API key to FlashData; fetch storage URLs separately.
fetch_bulk_result() {
local storage_url
storage_url=$(curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/results$1" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output /dev/null --write-out '%{redirect_url}') || return
curl --fail-with-body --silent --show-error --proto '=https' \
"${storage_url:?Expected a signed result URL}" --output "$2"
}
fetch_bulk_result '' manifest.json &&
jq '{status, counts, parts, expires_at}' manifest.jsonResponses
| HTTP status | Meaning |
|---|---|
| 200 | JSON manifest downloaded after following the redirect |
| 302 | Redirect to a short-lived private R2 URL; API credentials must not be forwarded · Location |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
| 409 | Resource state or idempotency conflict |
| 410 | The retained result has expired |
{
"type": "object",
"additionalProperties": true
}Download result parts
Use each parts[].number from manifest.json. Each endpoint returns a 302 to a gzip NDJSON file; download without the API key, decompress, then parse one JSON object per line. Join by the zero-based index. The example retrieves all parts and keeps both .ndjson.gz and .ndjson files.
| Parameter | Location | Requirement |
|---|---|---|
| bulkId | path | Required · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
| partNumber | path | Required · {"type":"integer","minimum":0} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/results/{partNumber}
X-API-Key: YOUR_API_KEYSave bulk_parts.py in bulk-api/ beside bulk-created.json. Copy the complete script, then run the command below it.
import json
import os
from pathlib import Path
import requests
import gzip
import shutil
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
manifest = json.loads(Path("manifest.json").read_text(encoding="utf-8"))
for part in manifest["parts"]:
number = part["number"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/results/{number}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
if response.status_code != 302:
raise RuntimeError(f"Expected a part redirect, got {response.status_code}")
url = response.headers["Location"]
if not url.startswith("https://"):
raise ValueError("Expected an HTTPS storage URL")
# Save the original gzip bytes; do not decompress twice.
with requests.get(url, stream=True, timeout=120) as download:
download.raise_for_status()
with open(f"part-{number}.ndjson.gz", "wb") as file:
download.raw.decode_content = False
shutil.copyfileobj(download.raw, file)
with gzip.open(f"part-{number}.ndjson.gz", "rt", encoding="utf-8") as rows:
with open(f"part-{number}.ndjson", "w", encoding="utf-8") as file:
for line in rows:
file.write(line)
print(json.dumps(json.loads(line)))python bulk_parts.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
# Only send the API key to FlashData; fetch storage URLs separately.
fetch_bulk_result() {
local storage_url
storage_url=$(curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/results$1" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output /dev/null --write-out '%{redirect_url}') || return
curl --fail-with-body --silent --show-error --proto '=https' \
"${storage_url:?Expected a signed result URL}" --output "$2"
}
fetch_bulk_result '' manifest.json &&
jq '{status, counts, parts, expires_at}' manifest.json
# Run in the same Bash session as fetch_bulk_result.
jq -r '.parts[].number' manifest.json | while IFS= read -r PART; do
fetch_bulk_result "/$PART" "part-$PART.ndjson.gz" || exit
gzip -dc "part-$PART.ndjson.gz" > "part-$PART.ndjson" || exit
jq -c '{index, video_id, status, results, error}' "part-$PART.ndjson"
doneResponses
| HTTP status | Meaning |
|---|---|
| 200 | Gzip-compressed NDJSON downloaded after following the redirect |
| 302 | Redirect to a short-lived private R2 URL; API credentials must not be forwarded · Location |
| 401 | Authentication required or invalid |
| 404 | Resource not found or not owned by the caller |
| 409 | Resource state or idempotency conflict |
| 410 | The retained result has expired |
{
"type": "string"
}Job states and retention
Results remain available for 24 hours after the entire Bulk job completes. Save them before expiresAt. Media in your destination follows your bucket's retention policy. A completed job can include failed videos; counts.invalid records rejected input lines.
| State | Next action |
|---|---|
| awaiting_upload | Upload the file and call start. |
| running | Poll the same ID with backoff; input.complete shows whether scanning finished. |
| paused | Read pauseReason, fix it, then resume. |
| finalizing | Wait for result packaging and manifest publication. |
| completed | Download the manifest and every part. Inspect each row's status and error. |
| failed / cancelled | Inspect error and saved state. No final manifest is published. |
Storage delivery
Use the playground to configure a provider and preview the exact request fields. Customer storage credentials belong in the chosen storage object; X-API-Key authenticates only FlashData API requests.
| Source | Create request field | Delivery |
|---|---|---|
| Metadata, transcript, captions, license check | options.result_storage | Optional. Copies NDJSON result parts and a manifest to your bucket. |
| youtube_download | options.storage | Required. Delivers media to your bucket; result_storage and browser_delivery are not accepted. |
Find the delivered files
The data manifest is written last and lists object keys, sizes and SHA-256 checksums. It differs from the FlashData manifest, which lists API href values. YYYYMMDD in data paths uses the UTC Bulk creation date; in media paths it uses each video's UTC upload date. Bulk OSS data delivery requires an aliyuncs.com endpoint.
Private media file_url values need your storage SDK or a signed URL from that provider. A FlashData key does not grant bucket access. Optional metadata companion files are stored beside the media. Delivery retries appear in delivery.retryCount.
<prefix>/<source>/<bulkId>/results/part-NNNNNNNN.jsonl.gz
<prefix>/<source>/<bulkId>/manifest.jsonReceive a completion webhook
Place a public HTTPS callback_url outside options. Bulk sends completed only after publishing the manifest; it sends no per-video events and no completion event for failed or cancelled jobs. Keep status polling as a fallback.
- Persist and deduplicate delivery_id, then return 2xx promptly. Delivery retries up to eight times with backoff.
- Events are unsigned. Match id and source to your saved receipt; verify completed through the authenticated management API before queuing retrieval. Ignore URLs from an unverified event.
{
"source": "youtube_metadata",
"options": {
"include_formats": false,
"include_chapters": false
},
"callback_url": "https://your-app.example/webhooks/flashdata"
}{
"delivery_id": "bulk-completion:bulk_EXAMPLE123",
"id": "bulk_EXAMPLE123",
"source": "youtube_metadata",
"status": "completed",
"links": [
{
"rel": "self",
"href": "https://api.flashdata.dev/v2/bulk-jobs/bulk_EXAMPLE123",
"method": "GET"
},
{
"rel": "results",
"href": "https://api.flashdata.dev/v2/bulk-jobs/bulk_EXAMPLE123/results",
"method": "GET"
}
]
}Schema definitions
BulkJobCreated
{
"type": "object",
"additionalProperties": false,
"required": [
"job",
"upload"
],
"properties": {
"job": {
"$ref": "#/components/schemas/BulkJob"
},
"upload": {
"type": "object",
"additionalProperties": false,
"required": [
"href",
"method",
"expiresAt",
"contentType"
],
"properties": {
"href": {
"type": "string",
"format": "uri"
},
"method": {
"type": "string",
"const": "PUT"
},
"expiresAt": {
"type": "string",
"format": "date-time"
},
"contentType": {
"type": "string",
"const": "text/plain; charset=utf-8"
}
}
}
}
}BulkJob
{
"type": "object",
"additionalProperties": false,
"required": [
"id",
"source",
"status",
"input",
"counts",
"delivery",
"pauseReason",
"error",
"createdAt",
"startedAt",
"completedAt",
"expiresAt",
"links"
],
"properties": {
"id": {
"type": "string",
"pattern": "^bulk_[A-Za-z0-9_-]{8,}$"
},
"source": {
"type": "string",
"enum": [
"youtube_metadata",
"youtube_transcript",
"youtube_captions",
"youtube_trainability",
"youtube_download"
]
},
"status": {
"type": "string",
"enum": [
"awaiting_upload",
"running",
"paused",
"finalizing",
"completed",
"failed",
"cancelled"
]
},
"input": {
"type": "object",
"additionalProperties": false,
"required": [
"bytes",
"complete",
"scannedBytes",
"scannedLines"
],
"properties": {
"bytes": {
"type": "integer",
"minimum": 0
},
"complete": {
"type": "boolean"
},
"scannedBytes": {
"type": "integer",
"minimum": 0
},
"scannedLines": {
"type": "integer",
"minimum": 0
}
}
},
"counts": {
"type": "object",
"additionalProperties": false,
"required": [
"accepted",
"completed",
"failed",
"invalid",
"results"
],
"properties": {
"accepted": {
"type": "integer",
"minimum": 0
},
"completed": {
"type": "integer",
"minimum": 0
},
"failed": {
"type": "integer",
"minimum": 0
},
"invalid": {
"type": "integer",
"minimum": 0
},
"results": {
"type": "integer",
"minimum": 0
}
}
},
"pauseReason": {
"type": [
"string",
"null"
]
},
"delivery": {
"type": "object",
"additionalProperties": false,
"required": [
"customerStorage",
"storageType",
"uploadedParts",
"retryCount"
],
"properties": {
"customerStorage": {
"type": "boolean"
},
"storageType": {
"type": [
"string",
"null"
],
"enum": [
"s3",
"s3_compatible",
"oss",
"gcs",
"azure",
"cos",
"bos",
"tos",
null
]
},
"uploadedParts": {
"type": "integer",
"minimum": 0
},
"retryCount": {
"type": "integer",
"minimum": 0
}
}
},
"error": {
"oneOf": [
{
"type": "null"
},
{
"type": "object",
"additionalProperties": false,
"required": [
"code",
"detail"
],
"properties": {
"code": {
"type": "string"
},
"detail": {
"type": [
"string",
"null"
]
}
}
}
]
},
"createdAt": {
"type": "string",
"format": "date-time"
},
"startedAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"completedAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"expiresAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"links": {
"type": "array",
"items": {
"$ref": "#/components/schemas/JobLink"
}
}
}
}BulkJobPage
{
"type": "object",
"additionalProperties": false,
"required": [
"data",
"pagination"
],
"properties": {
"data": {
"type": "array",
"items": {
"$ref": "#/components/schemas/BulkJob"
}
},
"pagination": {
"$ref": "#/components/schemas/Pagination"
}
}
}BulkJobResponse
{
"type": "object",
"additionalProperties": false,
"required": [
"job"
],
"properties": {
"job": {
"$ref": "#/components/schemas/BulkJob"
}
}
}CreateBulkJobRequest
{
"type": "object",
"additionalProperties": false,
"required": [
"source"
],
"properties": {
"source": {
"type": "string",
"enum": [
"youtube_metadata",
"youtube_transcript",
"youtube_captions",
"youtube_trainability",
"youtube_download"
]
},
"options": {
"type": "object",
"additionalProperties": true,
"description": "Reused for every ID. Metadata accepts include_formats and include_chapters. Transcript and captions accept language (one code or an array of 1–10 distinct codes) and auto (applies to all requested languages). Omit language to prefer en-orig, fall back to en, and return one track. With auto=true, each requested code tries its -orig track first, then the plain code. Explicit -orig codes and auto=false stay exact. languageCode identifies the selected track; requestedLanguageCode is included when it differs from the requested code. A language array returns tracks and per-language errors for each video with at least one retrieved track. If no track is retrieved, the video job fails and its reserved credits are refunded. Bulk exports use one JSONL record per video. Trainability accepts no source options. These four data sources optionally accept result_storage. Supported storage_type values are s3, s3_compatible, oss, gcs, azure, cos, bos, and tos. storage_url is the bucket name (Azure container name); object_prefix is optional. S3, S3-compatible, OSS, COS, BOS, and TOS use access_key and secret_key. S3-compatible and OSS also require a public HTTPS endpoint_url. COS, BOS, and TOS require region and use fixed provider endpoints. COS bucket names include the APPID. BOS S3-compatible HTTPS delivery supports bj, su, and hkg. GCS requires service_account with project_id, client_email, and private_key. Azure requires account_name and a container SAS sas_token with write permission. Default object_prefix is YYYYMMDD, frozen to the UTC bulk creation date. Files are written to <prefix>/<source>/<bulkId>/results/part-NNNNNNNN.jsonl.gz; manifest.json in the same bulk directory is written last. Each JSONL row includes its input index and completed, failed, or invalid status. Internal R2 results remain available through the result links for 24 hours. Download accepts the normal YouTube download parameters, requires customer storage, and does not permit browser_delivery. Download storage supports the same eight provider types and credential shapes listed above under the storage field. Download storage may include object_prefix; YYYYMMDD tokens resolve to the UTC upload date, while literal prefixes are unchanged."
},
"callback_url": {
"type": "string",
"format": "uri",
"maxLength": 2048,
"description": "Optional public HTTPS endpoint notified once after manifest publication."
}
}
}JobLink
{
"type": "object",
"additionalProperties": false,
"required": [
"rel",
"href",
"method"
],
"properties": {
"rel": {
"type": "string",
"enum": [
"self",
"results",
"content",
"download"
]
},
"href": {
"type": "string",
"format": "uri"
},
"method": {
"type": "string",
"const": "GET"
},
"expires_at": {
"type": "string",
"format": "date-time"
},
"filename": {
"type": "string"
},
"content_type": {
"type": "string"
},
"size_bytes": {
"type": "integer",
"minimum": 1
}
}
}Pagination
{
"type": "object",
"additionalProperties": false,
"required": [
"hasMore",
"limit"
],
"properties": {
"nextCursor": {
"type": [
"string",
"null"
]
},
"previousCursor": {
"type": [
"string",
"null"
]
},
"hasMore": {
"type": "boolean"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100
}
}
}Problem
{
"type": "object",
"additionalProperties": false,
"required": [
"type",
"title",
"status",
"code",
"requestId"
],
"properties": {
"type": {
"type": "string",
"format": "uri-reference"
},
"title": {
"type": "string"
},
"status": {
"type": "integer",
"minimum": 400,
"maximum": 599
},
"code": {
"type": "string",
"pattern": "^[a-z0-9_]+$"
},
"detail": {
"type": "string"
},
"requestId": {
"type": "string",
"pattern": "^req_"
},
"fieldErrors": {
"type": "array",
"items": {
"$ref": "#/components/schemas/FieldError"
}
}
}
}FieldError
{
"type": "object",
"additionalProperties": false,
"required": [
"field",
"code",
"message"
],
"properties": {
"field": {
"type": "string"
},
"code": {
"type": "string"
},
"message": {
"type": "string"
}
}
}UpdateBulkJobRequest
{
"type": "object",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"options": {
"type": "object",
"additionalProperties": true,
"description": "Partial option update. Nested storage fields are merged so unchanged credentials do not need to be resent. Data result_storage fields are merged the same way. After all IDs have been scanned, a paused data bulk job accepts credential updates only: access_key and secret_key, GCS service_account, or Azure sas_token, depending on the storage type. The complete merged configuration is validated before it is stored."
},
"callback_url": {
"type": [
"string",
"null"
],
"format": "uri",
"maxLength": 2048,
"description": "Replacement completion callback, or null to remove the callback."
}
}
}