浏览文档
API 参考
创建和控制基于文件的 YouTube 任务,再获取清单与结果分片。
本页内容
API Playground
选择来源,复制完整 Python 或 Bash 工作流、单个请求或响应示例。Playground 仅预览代码,不会提交任务。完整工作流包含上传、状态检查和所有结果分片获取。
构建文件批处理流程
本地 UTF-8 文件,每行一个 YouTube ID,最大 2 GiB。 输入要求.
高级媒体选项
媒体目标位置
留空时使用媒体的 UTC 上传日期(YYYYMMDD)。
运行代码前,请在服务器上填写存储凭据,并确保其具有文件上传权限。
服务商凭证和存储要求文件批处理完成 Webhook
清单完成通知与验证https://api.flashdata.dev/v2/bulk-jobs端点
所有端点使用 https://api.flashdata.dev/v2。创建任务 → 上传视频 ID 文件 → 启动 → 检查进度 → 获取清单与所有结果分片。
| 方法 | 路径 | 操作 |
|---|---|---|
| POST | /v2/bulk-jobs | 创建 Bulk 任务 |
| POST | /v2/bulk-jobs/{bulkId}/start | 上传并启动 |
| GET | /v2/bulk-jobs/{bulkId} | 获取任务状态 |
| GET | /v2/bulk-jobs | 列出任务 |
| POST | /v2/bulk-jobs/{bulkId}/pause | 暂停任务 |
| PATCH | /v2/bulk-jobs/{bulkId} | 更新任务选项 |
| POST | /v2/bulk-jobs/{bulkId}/resume | 恢复任务 |
| DELETE | /v2/bulk-jobs/{bulkId} | 取消任务 |
| GET | /v2/bulk-jobs/{bulkId}/results | 获取结果清单 |
| GET | /v2/bulk-jobs/{bulkId}/results/{partNumber} | 下载结果分片 |
认证与输入
将 FLASHDATA_API_KEY 设为有权访问所选来源的密钥。创建、控制和结果请求使用同一密钥与账户。处理使用账户余额,按来源正常计费。YouTube 下载需要所有者访问与 options.storage。
创建 bulk-api/ 工作目录,在其中运行示例。Playground 的完整 Python 工作流使用 Python 3.10+ 标准库;下方各端点的 Python 示例需要 Requests;cURL 示例使用 Bash、cURL 7.76+、jq 和 gzip。每个操作选择一种语言,避免重复创建或启动任务。
输入为最大 2 GiB 的 UTF-8 文件,每行一个 11 字符 YouTube ID。使用 ID,不使用 URL。上传前去重,重复 ID 会分别处理。
| 来源 | 共用选项 |
|---|---|
| youtube_metadata | include_formats, include_chapters |
| youtube_transcript | language, auto |
| youtube_captions | language, auto |
| youtube_trainability | 无选项 |
| youtube_download | 媒体选项与存储目标位置 |
python -m pip install requests创建请求字段
| 字段 | 类型 | 用途 |
|---|---|---|
| source | string | 必填。为全部输入 ID 选择一种支持的来源。 |
| options | object | 共用来源选项。数据来源可设置 result_storage;YouTube 下载必须设置 storage。 |
| callback_url | string | 可选的公开 HTTPS 完成回调,在清单发布后发送。 |
创建 Bulk 任务
创建任务,将每行包含一个 11 位 YouTube 视频 ID 的 UTF-8 文本文件上传至 upload.href,再调用 start。仅创建任务不会开始处理。
此示例写入 video-ids.txt,并将 201 回执保存到 bulk-created.json。妥善保管回执,后续请求复用 job.id。创建后状态为 awaiting_upload;上传 URL 在 15 分钟后过期。
POST https://api.flashdata.dev/v2/bulk-jobs
X-API-Key: YOUR_API_KEY将 bulk_create.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
os.umask(0o077)
Path("video-ids.txt").write_bytes(b"jNQXAC9IVRw\ndQw4w9WgXcQ\n")
response = requests.post(
"https://api.flashdata.dev/v2/bulk-jobs",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
json={"source": "youtube_metadata", "options": {"include_formats": False}},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
receipt = response.json()
Path("bulk-created.json").write_text(json.dumps(receipt, indent=2), encoding="utf-8")
print(json.dumps(receipt["job"], indent=2))python bulk_create.pyumask 077
printf '%s\n' 'jNQXAC9IVRw' 'dQw4w9WgXcQ' > video-ids.txt
curl --fail-with-body --silent --show-error \
'https://api.flashdata.dev/v2/bulk-jobs' \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--header 'Content-Type: application/json' \
--data '{"source":"youtube_metadata","options":{"include_formats":false}}' \
--output bulk-created.json &&
jq '.job | {id, source, status}' bulk-created.json响应与请求结构
| HTTP 状态 | 含义 |
|---|---|
| 201 | 文件批处理任务已创建,并提供有效期为 15 分钟的直接上传 URL |
| 401 | 需要身份验证,或身份验证无效 |
| 422 | 请求校验失败 |
| 503 | 必需的依赖项或目录暂时不可用 |
{
"$ref": "#/components/schemas/CreateBulkJobRequest"
}{
"$ref": "#/components/schemas/BulkJobCreated"
}上传并启动
开始处理已上传的 UTF-8 视频 ID 文件,最大支持 2 GiB(2147483648 字节)。开始后请勿更改输入文件。
先使用 upload.contentType 将 video-ids.txt PUT 到回执中的 upload.href,不发送 X-API-Key。示例仅在上传成功后调用 start。启动后保持上传文件不变。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/start
X-API-Key: YOUR_API_KEY将 bulk_start.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
receipt = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))
upload = receipt["upload"]
if upload["method"] != "PUT" or not upload["href"].startswith("https://"):
raise ValueError("Expected an HTTPS PUT upload URL")
# Stream the file to storage without the FlashData key.
with open("video-ids.txt", "rb") as file:
response = requests.put(
upload["href"], data=file,
headers={"Content-Type": upload["contentType"]},
timeout=300, allow_redirects=False,
)
response.raise_for_status()
if not 200 <= response.status_code < 300:
raise RuntimeError("Upload did not complete; do not start the job yet")
bulk_id = receipt["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/start",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json()["job"], indent=2))python bulk_start.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
UPLOAD_URL=$(jq -er '.upload.href' bulk-created.json)
CONTENT_TYPE=$(jq -er '.upload.contentType' bulk-created.json)
curl --fail-with-body --silent --show-error --proto '=https' \
--header "Content-Type: $CONTENT_TYPE" \
--upload-file video-ids.txt "$UPLOAD_URL" &&
curl --fail-with-body --silent --show-error --request POST \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/start" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}"响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 文件批处理任务已启动,或返回其当前幂等状态 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
| 409 | 资源状态或幂等性冲突 |
| 422 | 请求校验失败 |
| 503 | 必需的依赖项或目录暂时不可用 |
{
"$ref": "#/components/schemas/BulkJobResponse"
}获取任务状态
读取 job.status、input、counts、pauseReason 与 error。间隔一段时间后检查同一 ID;completed 表示清单已发布,不表示每个视频都成功。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEY将 bulk_status.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
Path("bulk-job.json").write_text(json.dumps(response.json(), indent=2), encoding="utf-8")
print(json.dumps(response.json()["job"], indent=2))python bulk_status.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output bulk-job.json &&
jq '.job | {id, status, counts, pauseReason, error, expiresAt}' bulk-job.json响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 文件批处理任务状态 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
{
"$ref": "#/components/schemas/BulkJobResponse"
}列出任务
响应包含 data 与 pagination。将 pagination.nextCursor 作为下一次请求的 cursor 查询参数,直到它为 null。limit 范围为 1 至 50,默认 20。
| 参数 | 位置 | 要求 |
|---|---|---|
| cursor | query | 可选 · {"type":"string","maxLength":512} |
| limit | query | 可选 · {"type":"integer","minimum":1,"maximum":50,"default":20} |
GET https://api.flashdata.dev/v2/bulk-jobs
X-API-Key: YOUR_API_KEY将 list_bulk_jobs.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
response = requests.get(
"https://api.flashdata.dev/v2/bulk-jobs?limit=20",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python list_bulk_jobs.pycurl --fail-with-body --silent --show-error --max-time 30 \
--request GET \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs?limit=20"响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 文件批处理任务页面 |
| 401 | 需要身份验证,或身份验证无效 |
{
"$ref": "#/components/schemas/BulkJobPage"
}暂停任务
暂停提交新任务及结果打包。已提交的任务继续按原设置运行并计费。准备就绪后可恢复任务。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/pause
X-API-Key: YOUR_API_KEY将 pause_bulk_job.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/pause",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python pause_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request POST \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/pause"响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 已暂停的文件批处理任务或其当前幂等状态 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
{
"$ref": "#/components/schemas/BulkJobResponse"
}更新任务选项
可在等待上传时,或已暂停且尚未读取全部输入时更新 options。已提交的任务保持原设置。如暂停尚未完成并返回 409,请等待后重试。读取全部输入后仅允许更新交付凭据。将 callback_url 设为 null 可移除回调。
此示例移除完成回调。修改来源选项时改为发送 options 对象。先暂停并等待暂停生效;409 表示暂时不能应用更新。已有子任务保留原选项。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
PATCH https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEY将 update_bulk_job.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.patch(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
json={"callback_url": None},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python update_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request PATCH \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--header 'Content-Type: application/json' --data '{"callback_url":null}' \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID"响应与请求结构
| HTTP 状态 | 含义 |
|---|---|
| 200 | 文件批处理任务状态已更新 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
| 409 | 资源状态或幂等性冲突 |
| 422 | 请求校验失败 |
{
"$ref": "#/components/schemas/UpdateBulkJobRequest"
}{
"$ref": "#/components/schemas/BulkJobResponse"
}恢复任务
先解决 pauseReason,例如补充额度或更新存储凭据,然后恢复同一 Bulk ID。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
POST https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/resume
X-API-Key: YOUR_API_KEY将 resume_bulk_job.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.post(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/resume",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python resume_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request POST \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/resume"响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 当前文件批处理任务状态 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
{
"$ref": "#/components/schemas/BulkJobResponse"
}取消任务
停止提交新任务。已提交的任务仍可能完成并正常计费。取消的任务不会发布最终清单。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
DELETE https://api.flashdata.dev/v2/bulk-jobs/{bulkId}
X-API-Key: YOUR_API_KEY将 cancel_bulk_job.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.delete(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
print(json.dumps(response.json(), indent=2))python cancel_bulk_job.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
curl --fail-with-body --silent --show-error --max-time 30 \
--request DELETE \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID"响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 当前文件批处理任务状态 |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
{
"$ref": "#/components/schemas/BulkJobResponse"
}获取结果清单
跟随跳转下载 JSON 清单,再通过清单中的链接获取结果文件。访问签名存储 URL 时不要携带 API Key。结果在整个 Bulk 任务完成后保留 24 小时。
认证后的 API 请求返回 302 与 Location 头。另发请求读取该 HTTPS 存储 URL,不携带 X-API-Key。示例保存 manifest.json。409 表示结果未就绪,410 表示已过期。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/results
X-API-Key: YOUR_API_KEY将 bulk_manifest.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/results",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
if response.status_code != 302:
raise RuntimeError(f"Expected a manifest redirect, got {response.status_code}")
url = response.headers["Location"]
if not url.startswith("https://"):
raise ValueError("Expected an HTTPS storage URL")
manifest = requests.get(url, timeout=120) # No FlashData API key.
manifest.raise_for_status()
Path("manifest.json").write_text(json.dumps(manifest.json(), indent=2), encoding="utf-8")
print(json.dumps(manifest.json(), indent=2))python bulk_manifest.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
# Only send the API key to FlashData; fetch storage URLs separately.
fetch_bulk_result() {
local storage_url
storage_url=$(curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/results$1" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output /dev/null --write-out '%{redirect_url}') || return
curl --fail-with-body --silent --show-error --proto '=https' \
"${storage_url:?Expected a signed result URL}" --output "$2"
}
fetch_bulk_result '' manifest.json &&
jq '{status, counts, parts, expires_at}' manifest.json响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 跟随跳转后下载的 JSON 清单 |
| 302 | 重定向至短期有效的私有 R2 URL;不得转发 API 凭证 · Location |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
| 409 | 资源状态或幂等性冲突 |
| 410 | 保留的结果已过期 |
{
"type": "object",
"additionalProperties": true
}下载结果分片
使用 manifest.json 中各项 parts[].number。每个端点返回 302,指向 gzip NDJSON 文件;下载时不携带 API Key,解压后逐行解析 JSON。使用从零开始的 index 匹配输入。示例获取所有分片,并保留 .ndjson.gz 与 .ndjson 文件。
| 参数 | 位置 | 要求 |
|---|---|---|
| bulkId | path | 必填 · {"type":"string","pattern":"^bulk_[A-Za-z0-9_-]{8,}$"} |
| partNumber | path | 必填 · {"type":"integer","minimum":0} |
GET https://api.flashdata.dev/v2/bulk-jobs/{bulkId}/results/{partNumber}
X-API-Key: YOUR_API_KEY将 bulk_parts.py 保存到 bulk-api/,与 bulk-created.json 放在一起。复制完整脚本后,运行其下方命令。
import json
import os
from pathlib import Path
import requests
import gzip
import shutil
bulk_id = json.loads(Path("bulk-created.json").read_text(encoding="utf-8"))["job"]["id"]
manifest = json.loads(Path("manifest.json").read_text(encoding="utf-8"))
for part in manifest["parts"]:
number = part["number"]
response = requests.get(
f"https://api.flashdata.dev/v2/bulk-jobs/{bulk_id}/results/{number}",
headers={"X-API-Key": os.environ["FLASHDATA_API_KEY"]},
timeout=30, allow_redirects=False,
)
response.raise_for_status()
if response.status_code != 302:
raise RuntimeError(f"Expected a part redirect, got {response.status_code}")
url = response.headers["Location"]
if not url.startswith("https://"):
raise ValueError("Expected an HTTPS storage URL")
# Save the original gzip bytes; do not decompress twice.
with requests.get(url, stream=True, timeout=120) as download:
download.raise_for_status()
with open(f"part-{number}.ndjson.gz", "wb") as file:
download.raw.decode_content = False
shutil.copyfileobj(download.raw, file)
with gzip.open(f"part-{number}.ndjson.gz", "rt", encoding="utf-8") as rows:
with open(f"part-{number}.ndjson", "w", encoding="utf-8") as file:
for line in rows:
file.write(line)
print(json.dumps(json.loads(line)))python bulk_parts.pyBULK_ID=$(jq -er '.job.id' bulk-created.json)
# Only send the API key to FlashData; fetch storage URLs separately.
fetch_bulk_result() {
local storage_url
storage_url=$(curl --fail-with-body --silent --show-error \
"https://api.flashdata.dev/v2/bulk-jobs/$BULK_ID/results$1" \
--header "X-API-Key: ${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY}" \
--output /dev/null --write-out '%{redirect_url}') || return
curl --fail-with-body --silent --show-error --proto '=https' \
"${storage_url:?Expected a signed result URL}" --output "$2"
}
fetch_bulk_result '' manifest.json &&
jq '{status, counts, parts, expires_at}' manifest.json
# Run in the same Bash session as fetch_bulk_result.
jq -r '.parts[].number' manifest.json | while IFS= read -r PART; do
fetch_bulk_result "/$PART" "part-$PART.ndjson.gz" || exit
gzip -dc "part-$PART.ndjson.gz" > "part-$PART.ndjson" || exit
jq -c '{index, video_id, status, results, error}' "part-$PART.ndjson"
done响应
| HTTP 状态 | 含义 |
|---|---|
| 200 | 跟随跳转后下载的 gzip 压缩 NDJSON |
| 302 | 重定向至短期有效的私有 R2 URL;不得转发 API 凭证 · Location |
| 401 | 需要身份验证,或身份验证无效 |
| 404 | 未找到资源,或资源不属于调用者 |
| 409 | 资源状态或幂等性冲突 |
| 410 | 保留的结果已过期 |
{
"type": "string"
}任务状态与保留期限
结果在整个 Bulk 任务完成后保留 24 小时,请在 expiresAt 前保存。目标存储中的媒体遵循你的存储桶保留策略。已完成任务可能包含失败视频,counts.invalid 记录被拒绝的输入行。
| 状态 | 下一步 |
|---|---|
| awaiting_upload | 上传文件并调用 start。 |
| running | 使用退避轮询同一 ID;input.complete 表示扫描是否完成。 |
| paused | 读取并解决 pauseReason,然后恢复。 |
| finalizing | 等待结果打包与清单发布。 |
| completed | 下载清单与全部分片,检查每行的 status 与 error。 |
| failed / cancelled | 检查 error 与已保存状态,不会发布最终清单。 |
存储交付
在 Playground 中配置提供商,预览完整请求字段。客户存储凭据放在对应的存储对象中;X-API-Key 仅用于认证 FlashData API 请求。
| 来源 | 创建请求字段 | 交付内容 |
|---|---|---|
| 元数据、转录文本、字幕、许可检查 | options.result_storage | 可选。将 NDJSON 结果分片与清单复制到你的存储桶。 |
| youtube_download | options.storage | 必需。将媒体交付到你的存储桶,不接受 result_storage 或 browser_delivery。 |
查找交付文件
数据清单最后写入,列出对象 key、大小与 SHA-256,与列出 API href 的 FlashData 清单不同。数据路径中的 YYYYMMDD 使用 Bulk 的 UTC 创建日期;媒体路径使用各视频的 UTC 上传日期。Bulk OSS 数据交付需要 aliyuncs.com 端点。
私有媒体 file_url 需要你的存储 SDK 或该提供商签发的 URL。FlashData 密钥不授予存储桶访问权限。可选元数据伴随文件保存在媒体旁。交付重试显示在 delivery.retryCount。
<prefix>/<source>/<bulkId>/results/part-NNNNNNNN.jsonl.gz
<prefix>/<source>/<bulkId>/manifest.json接收完成 Webhook
将公网 HTTPS callback_url 放在 options 之外。Bulk 仅在清单发布后发送 completed,不发送逐视频事件,也不为 failed 或 cancelled 任务发送完成事件。保留状态轮询作为后备。
- 持久化并按 delivery_id 去重,然后及时返回 2xx。交付按退避策略最多尝试八次。
- 事件未签名。将 id 和 source 与已保存回执匹配,并通过认证的管理 API 验证 completed 后才排队获取结果。忽略未验证事件提供的 URL。
{
"source": "youtube_metadata",
"options": {
"include_formats": false,
"include_chapters": false
},
"callback_url": "https://your-app.example/webhooks/flashdata"
}{
"delivery_id": "bulk-completion:bulk_EXAMPLE123",
"id": "bulk_EXAMPLE123",
"source": "youtube_metadata",
"status": "completed",
"links": [
{
"rel": "self",
"href": "https://api.flashdata.dev/v2/bulk-jobs/bulk_EXAMPLE123",
"method": "GET"
},
{
"rel": "results",
"href": "https://api.flashdata.dev/v2/bulk-jobs/bulk_EXAMPLE123/results",
"method": "GET"
}
]
}结构定义
BulkJobCreated
{
"type": "object",
"additionalProperties": false,
"required": [
"job",
"upload"
],
"properties": {
"job": {
"$ref": "#/components/schemas/BulkJob"
},
"upload": {
"type": "object",
"additionalProperties": false,
"required": [
"href",
"method",
"expiresAt",
"contentType"
],
"properties": {
"href": {
"type": "string",
"format": "uri"
},
"method": {
"type": "string",
"const": "PUT"
},
"expiresAt": {
"type": "string",
"format": "date-time"
},
"contentType": {
"type": "string",
"const": "text/plain; charset=utf-8"
}
}
}
}
}BulkJob
{
"type": "object",
"additionalProperties": false,
"required": [
"id",
"source",
"status",
"input",
"counts",
"delivery",
"pauseReason",
"error",
"createdAt",
"startedAt",
"completedAt",
"expiresAt",
"links"
],
"properties": {
"id": {
"type": "string",
"pattern": "^bulk_[A-Za-z0-9_-]{8,}$"
},
"source": {
"type": "string",
"enum": [
"youtube_metadata",
"youtube_transcript",
"youtube_captions",
"youtube_trainability",
"youtube_download"
]
},
"status": {
"type": "string",
"enum": [
"awaiting_upload",
"running",
"paused",
"finalizing",
"completed",
"failed",
"cancelled"
]
},
"input": {
"type": "object",
"additionalProperties": false,
"required": [
"bytes",
"complete",
"scannedBytes",
"scannedLines"
],
"properties": {
"bytes": {
"type": "integer",
"minimum": 0
},
"complete": {
"type": "boolean"
},
"scannedBytes": {
"type": "integer",
"minimum": 0
},
"scannedLines": {
"type": "integer",
"minimum": 0
}
}
},
"counts": {
"type": "object",
"additionalProperties": false,
"required": [
"accepted",
"completed",
"failed",
"invalid",
"results"
],
"properties": {
"accepted": {
"type": "integer",
"minimum": 0
},
"completed": {
"type": "integer",
"minimum": 0
},
"failed": {
"type": "integer",
"minimum": 0
},
"invalid": {
"type": "integer",
"minimum": 0
},
"results": {
"type": "integer",
"minimum": 0
}
}
},
"pauseReason": {
"type": [
"string",
"null"
]
},
"delivery": {
"type": "object",
"additionalProperties": false,
"required": [
"customerStorage",
"storageType",
"uploadedParts",
"retryCount"
],
"properties": {
"customerStorage": {
"type": "boolean"
},
"storageType": {
"type": [
"string",
"null"
],
"enum": [
"s3",
"s3_compatible",
"oss",
"gcs",
"azure",
"cos",
"bos",
"tos",
null
]
},
"uploadedParts": {
"type": "integer",
"minimum": 0
},
"retryCount": {
"type": "integer",
"minimum": 0
}
}
},
"error": {
"oneOf": [
{
"type": "null"
},
{
"type": "object",
"additionalProperties": false,
"required": [
"code",
"detail"
],
"properties": {
"code": {
"type": "string"
},
"detail": {
"type": [
"string",
"null"
]
}
}
}
]
},
"createdAt": {
"type": "string",
"format": "date-time"
},
"startedAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"completedAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"expiresAt": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"links": {
"type": "array",
"items": {
"$ref": "#/components/schemas/JobLink"
}
}
}
}BulkJobPage
{
"type": "object",
"additionalProperties": false,
"required": [
"data",
"pagination"
],
"properties": {
"data": {
"type": "array",
"items": {
"$ref": "#/components/schemas/BulkJob"
}
},
"pagination": {
"$ref": "#/components/schemas/Pagination"
}
}
}BulkJobResponse
{
"type": "object",
"additionalProperties": false,
"required": [
"job"
],
"properties": {
"job": {
"$ref": "#/components/schemas/BulkJob"
}
}
}CreateBulkJobRequest
{
"type": "object",
"additionalProperties": false,
"required": [
"source"
],
"properties": {
"source": {
"type": "string",
"enum": [
"youtube_metadata",
"youtube_transcript",
"youtube_captions",
"youtube_trainability",
"youtube_download"
]
},
"options": {
"type": "object",
"additionalProperties": true,
"description": "Reused for every ID. Metadata accepts include_formats and include_chapters. Transcript and captions accept language (one code or an array of 1–10 distinct codes) and auto (applies to all requested languages). Omit language to prefer en-orig, fall back to en, and return one track. With auto=true, each requested code tries its -orig track first, then the plain code. Explicit -orig codes and auto=false stay exact. languageCode identifies the selected track; requestedLanguageCode is included when it differs from the requested code. A language array returns tracks and per-language errors for each video with at least one retrieved track. If no track is retrieved, the video job fails and its reserved credits are refunded. Bulk exports use one JSONL record per video. Trainability accepts no source options. These four data sources optionally accept result_storage. Supported storage_type values are s3, s3_compatible, oss, gcs, azure, cos, bos, and tos. storage_url is the bucket name (Azure container name); object_prefix is optional. S3, S3-compatible, OSS, COS, BOS, and TOS use access_key and secret_key. S3-compatible and OSS also require a public HTTPS endpoint_url. COS, BOS, and TOS require region and use fixed provider endpoints. COS bucket names include the APPID. BOS S3-compatible HTTPS delivery supports bj, su, and hkg. GCS requires service_account with project_id, client_email, and private_key. Azure requires account_name and a container SAS sas_token with write permission. Default object_prefix is YYYYMMDD, frozen to the UTC bulk creation date. Files are written to <prefix>/<source>/<bulkId>/results/part-NNNNNNNN.jsonl.gz; manifest.json in the same bulk directory is written last. Each JSONL row includes its input index and completed, failed, or invalid status. Internal R2 results remain available through the result links for 24 hours. Download accepts the normal YouTube download parameters, requires customer storage, and does not permit browser_delivery. Download storage supports the same eight provider types and credential shapes listed above under the storage field. Download storage may include object_prefix; YYYYMMDD tokens resolve to the UTC upload date, while literal prefixes are unchanged."
},
"callback_url": {
"type": "string",
"format": "uri",
"maxLength": 2048,
"description": "Optional public HTTPS endpoint notified once after manifest publication."
}
}
}JobLink
{
"type": "object",
"additionalProperties": false,
"required": [
"rel",
"href",
"method"
],
"properties": {
"rel": {
"type": "string",
"enum": [
"self",
"results",
"content",
"download"
]
},
"href": {
"type": "string",
"format": "uri"
},
"method": {
"type": "string",
"const": "GET"
},
"expires_at": {
"type": "string",
"format": "date-time"
},
"filename": {
"type": "string"
},
"content_type": {
"type": "string"
},
"size_bytes": {
"type": "integer",
"minimum": 1
}
}
}Pagination
{
"type": "object",
"additionalProperties": false,
"required": [
"hasMore",
"limit"
],
"properties": {
"nextCursor": {
"type": [
"string",
"null"
]
},
"previousCursor": {
"type": [
"string",
"null"
]
},
"hasMore": {
"type": "boolean"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100
}
}
}Problem
{
"type": "object",
"additionalProperties": false,
"required": [
"type",
"title",
"status",
"code",
"requestId"
],
"properties": {
"type": {
"type": "string",
"format": "uri-reference"
},
"title": {
"type": "string"
},
"status": {
"type": "integer",
"minimum": 400,
"maximum": 599
},
"code": {
"type": "string",
"pattern": "^[a-z0-9_]+$"
},
"detail": {
"type": "string"
},
"requestId": {
"type": "string",
"pattern": "^req_"
},
"fieldErrors": {
"type": "array",
"items": {
"$ref": "#/components/schemas/FieldError"
}
}
}
}FieldError
{
"type": "object",
"additionalProperties": false,
"required": [
"field",
"code",
"message"
],
"properties": {
"field": {
"type": "string"
},
"code": {
"type": "string"
},
"message": {
"type": "string"
}
}
}UpdateBulkJobRequest
{
"type": "object",
"additionalProperties": false,
"minProperties": 1,
"properties": {
"options": {
"type": "object",
"additionalProperties": true,
"description": "Partial option update. Nested storage fields are merged so unchanged credentials do not need to be resent. Data result_storage fields are merged the same way. After all IDs have been scanned, a paused data bulk job accepts credential updates only: access_key and secret_key, GCS service_account, or Azure sas_token, depending on the storage type. The complete merged configuration is validated before it is stored."
},
"callback_url": {
"type": [
"string",
"null"
],
"format": "uri",
"maxLength": 2048,
"description": "Replacement completion callback, or null to remove the callback."
}
}
}