跳转到正文
中文
获取 API 密钥
浏览文档

示例

下载视频到你的存储

使用 Python 或 Bash 将一个 MP4 交付到你的 Amazon S3 存储桶,再取回文件并验证。

本页内容

开始前准备

需要具有 youtube_download 权限的所有者访问及可用额度。临时智能体试用不包含下载。准备已有的 Amazon S3 存储桶,以及你有权下载的视频。

选择 Python 3.10+、requests 和 boto3,或 Bash、cURL、jq、OpenSSL 和 AWS CLI v2。Windows 的 Bash 版本请使用 Git Bash。

凭据用途
FLASHDATA_API_KEY提交任务并获取状态和交付记录。
FLASHDATA_STORAGE_ACCESS_KEY / FLASHDATA_STORAGE_SECRET_KEY发送给 FlashData 的 S3 上传身份,用于写入指定存储桶及前缀。
本地 AWS 配置档或角色在步骤 4 使用 boto3 或 AWS CLI 读取私有对象,需要 s3:GetObject 权限。

1. 选择视频和目标存储

在代码仓库外创建私有 download-video 文件夹,将 request.json 保存到其中,并在该文件夹打开终端。将 storage_url 替换为存储桶名称,region 替换为其 AWS 区域;视频使用 11 位 ID。

本例将带音频的视频以最高 720p 的 MP4 交付到 recipes/clip-001/ 下。保持存储桶私有。另一次下载请使用新前缀,避免其他任务覆盖此对象路径。

download-video/request.json
{
  "source": "youtube_download",
  "video_id": "jNQXAC9IVRw",
  "params": {
    "download_type": "audio_video",
    "video_quality": "<=720",
    "output_format": "mp4",
    "storage": {
      "storage_type": "s3",
      "storage_url": "your-media-bucket",
      "region": "us-east-1",
      "object_prefix": "recipes/clip-001"
    }
  }
}

2. 提交一个下载任务

选择一种版本,将脚本保存到 request.json 旁,再运行。脚本读取上方三个环境变量,或通过隐藏输入提示获取。智能体应从密钥存储提供这些变量。不要将密钥写入 request.json。

新建 job-001 文件夹时发送一次异步请求,保存不含上传凭据的请求、时间戳、HTTP 响应及任务 ID。复用该文件夹会停止提交,不会创建第二个任务。

安装到 Python 环境
python -m pip install requests boto3
download-video/submit.py
# Save as download-video/submit.py.
import getpass
import json
import os
import re
import sys
from datetime import datetime, timezone
from pathlib import Path

import requests

ROOT = Path(__file__).resolve().parent


def read_json(path):
    return json.loads(path.read_text(encoding="utf-8-sig"))


def secret(name):
    value = os.environ.get(name) or getpass.getpass(name + ": ")
    if not value.strip():
        raise ValueError("Missing " + name)
    return value


def main():
    os.umask(0o077)
    folder = sys.argv[1]
    if not re.fullmatch(r"[A-Za-z0-9_-]+", folder):
        raise ValueError("Use a simple output folder name, such as job-001.")
    directory = ROOT / folder
    if directory.exists():
        raise ValueError("Keep this attempt. Run collect.py with the same folder.")
    request = read_json(ROOT / "request.json")
    storage = request["params"]["storage"]
    if (request["source"] != "youtube_download"
            or not re.fullmatch(r"[A-Za-z0-9_-]{11}", request["video_id"])
            or request["params"]["download_type"] != "audio_video"
            or request["params"]["output_format"] != "mp4"
            or storage["storage_type"] != "s3"
            or set(storage) != {"storage_type", "storage_url", "region", "object_prefix"}
            or not re.fullmatch(r"[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]", storage["storage_url"])
            or not re.fullmatch(r"[a-z0-9-]+", storage["region"])
            or not re.fullmatch(r"[A-Za-z0-9_-]+(/[A-Za-z0-9_-]+)*", storage["object_prefix"])):
        raise ValueError("Check the video, MP4 options and Amazon S3 destination.")
    key = secret("FLASHDATA_API_KEY")
    access = secret("FLASHDATA_STORAGE_ACCESS_KEY")
    secret_key = secret("FLASHDATA_STORAGE_SECRET_KEY")
    directory.mkdir(mode=0o700)
    # Save the plan BEFORE sending. Keep upload credentials out of this file.
    (directory / "request.json").write_text(json.dumps(request, indent=2), encoding="utf-8")
    (directory / "submitted-at.txt").write_text(
        datetime.now(timezone.utc).isoformat(), encoding="utf-8")
    storage.update(access_key=access, secret_key=secret_key)
    response = requests.post(
        "https://data.flashdata.dev/v1/queries", json=request,
        headers={"X-API-Key": key, "Accept": "application/json"},
        timeout=(10, 120), allow_redirects=False)
    (directory / "submission.json").write_bytes(response.content)
    (directory / "submission-headers.txt").write_text(
        "\n".join(f"{k}: {v}" for k, v in response.headers.items()), encoding="utf-8")
    (directory / "submission-http.txt").write_text(str(response.status_code))
    if response.status_code not in (200, 201, 202):
        raise ValueError(f"HTTP {response.status_code}. Inspect the saved attempt.")
    receipt = response.json()
    if (receipt.get("source") != "youtube_download"
            or not re.fullmatch(r"[A-Za-z0-9_-]+", receipt.get("id", ""))):
        raise ValueError("Missing job ID. Check History before submitting again.")
    print("Submitted:", receipt["id"], "— keep", directory)


if __name__ == "__main__":
    try:
        main()
    except (IndexError, KeyError, OSError, ValueError, requests.RequestException) as error:
        print("Network request failed. Keep this attempt and inspect its saved files."
              if isinstance(error, requests.RequestException) else str(error), file=sys.stderr)
        sys.exit(1)
在 download-video/ 中运行
python submit.py job-001

预期回执

成功后输出 Submitted: job_… 并保存 submission.json。回执表示任务已被接受,不代表交付完成或最终扣费。

job-001/submission.json · 精简示例
{
  "id": "job_example",
  "source": "youtube_download",
  "status": "pending",
  "usage": {
    "credits": 0
  }
}

3. 等待交付并保存记录

将 collect.py 保存到 submit.py 旁,或将 collect.sh 保存到 submit.sh 旁。使用相同的 job-001 文件夹运行。它检查一次已保存任务;如果状态为 pending 或 processing,等待 10 秒后运行相同命令。

状态为 completed 时保存 delivery.json。此处的签名重定向用于获取 JSON 交付记录,不是 MP4;脚本不会向重定向地址转发 FlashData 密钥。已有 delivery.json 时直接复用,不再调用 API。

download-video/collect.py
# Save as download-video/collect.py, beside submit.py.
import os
import re
import sys
from urllib.parse import urlsplit

import requests
from submit import ROOT, read_json, secret


def main():
    os.umask(0o077)
    folder = sys.argv[1]
    if not re.fullmatch(r"[A-Za-z0-9_-]+", folder):
        raise ValueError("Use the saved output folder name.")
    directory = ROOT / folder
    if (directory / "delivery.json").exists():
        print("Already saved:", directory / "delivery.json")
        return
    receipt = read_json(directory / "submission.json")
    job_id = receipt["id"]
    if receipt["source"] != "youtube_download" or not re.fullmatch(r"[A-Za-z0-9_-]+", job_id):
        raise ValueError("Inspect submission.json before continuing.")
    headers = {"X-API-Key": secret("FLASHDATA_API_KEY"), "Accept": "application/json"}
    url = "https://api.flashdata.dev/v2/jobs/" + job_id

    def get(url, name, headers):
        response = requests.get(url, headers=headers, timeout=(10, 120), allow_redirects=False)
        (directory / name).write_bytes(response.content)
        (directory / (name + ".headers")).write_text(
            "\n".join(f"{k}: {v}" for k, v in response.headers.items()), encoding="utf-8")
        (directory / (name + ".http")).write_text(str(response.status_code))
        return response

    status = get(url, "job-status.json", headers)
    if status.status_code != 200:
        raise ValueError(f"HTTP {status.status_code}. Inspect job-status.json and its headers.")
    job = status.json()["job"]
    if job["id"] != job_id or job["source"] != "youtube_download":
        raise ValueError("The job does not match the saved receipt.")
    if job["status"] in ("pending", "processing"):
        print(job["status"] + ". Wait 10 seconds, then run this command again.")
        return
    if job["status"] != "completed":
        raise ValueError("Job " + job["status"] + ". Inspect job-status.json; do not resubmit.")
    result = get(url + "/results", "delivery.pending.json", headers)
    if result.status_code == 302:
        location = result.headers.get("Location", "")
        parsed = urlsplit(location)
        if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password:
            raise ValueError("Expected an HTTPS result URL.")
        # This URL retrieves the JSON delivery record, not the video.
        # Never forward the FlashData key to the signed storage URL.
        result = get(location, "delivery.pending.json", {})
    if result.status_code != 200:
        raise ValueError(f"HTTP {result.status_code}. Run collect.py again after resolving it.")
    delivery = result.json()
    if (delivery.get("source") != "youtube_download" or delivery.get("status") != "done"
            or not isinstance(delivery.get("results"), list) or len(delivery["results"]) != 1):
        raise ValueError("Inspect delivery.pending.json before using the result.")
    (directory / "delivery.pending.json").replace(directory / "delivery.json")
    print("Saved:", directory / "delivery.json")
    print("Usage:", delivery.get("usage"))


if __name__ == "__main__":
    try:
        main()
    except (IndexError, KeyError, OSError, ValueError, requests.RequestException) as error:
        print("Network request failed. Keep this attempt and inspect its saved files."
              if isinstance(error, requests.RequestException) else str(error), file=sys.stderr)
        sys.exit(1)
检查已保存任务
python collect.py job-001

delivery.json 包含什么

结果包含 file_url、以字节为单位的 file_size 和 md5。usage 报告已完成任务的费用。保留完整记录。文件 URL 标识你的 S3 对象,不会让私有存储桶变为公开可读。

job-001/delivery.json · 字段示意
{
  "source": "youtube_download",
  "status": "done",
  "results": [
    {
      "file_url": "https://your-media-bucket.s3.us-east-1.amazonaws.com/recipes/clip-001/jNQXAC9IVRw.mp4",
      "file_size": 1048576,
      "md5": "0123456789abcdef0123456789abcdef"
    }
  ]
}

4. 读取私有对象并验证

将验证脚本保存到其他脚本旁。使用已配置且具有该对象读取权限的本地 AWS 身份。脚本先核对交付是否对应计划中的 S3 存储桶、前缀和视频文件名,再将文件下载为 media.mp4.part。

只有字节数及 MD5 一致时,才生成 media.mp4 和 verified.json。再次验证时在本地检查已有 media.mp4。此步骤读取你的 S3 对象,不会再次提交 FlashData 下载。

download-video/verify.py
# Save as download-video/verify.py, beside submit.py.
import hashlib
import json
import os
import re
import sys
from contextlib import closing
from datetime import datetime, timezone
from urllib.parse import urlsplit

import boto3
from submit import ROOT, read_json


def main():
    os.umask(0o077)
    folder = sys.argv[1]
    if not re.fullmatch(r"[A-Za-z0-9_-]+", folder):
        raise ValueError("Use the saved output folder name.")
    directory = ROOT / folder
    request = read_json(directory / "request.json")
    receipt = read_json(directory / "submission.json")
    job = read_json(directory / "job-status.json")["job"]
    delivery = read_json(directory / "delivery.json")
    if (request["source"] != "youtube_download" or receipt["source"] != "youtube_download"
            or job["id"] != receipt["id"] or job["source"] != "youtube_download"
            or job["status"] != "completed" or delivery.get("source") != "youtube_download"
            or delivery.get("status") != "done" or len(delivery["results"]) != 1):
        raise ValueError("Check the saved request, receipt, status and delivery.")
    storage = request["params"]["storage"]
    bucket, region, prefix = (storage[k] for k in ("storage_url", "region", "object_prefix"))
    file = delivery["results"][0]
    parsed = urlsplit(file["file_url"])
    # This recipe uses an ASCII prefix and the MP4 filename generated for the video.
    object_key = prefix + "/" + request["video_id"] + ".mp4"
    if (storage["storage_type"] != "s3" or parsed.scheme != "https"
            or parsed.netloc != bucket + ".s3." + region + ".amazonaws.com"
            or parsed.path != "/" + object_key or parsed.query or parsed.fragment
            or type(file.get("file_size")) is not int or file["file_size"] <= 0
            or not re.fullmatch(r"[A-Fa-f0-9]{32}", file.get("md5", ""))):
        raise ValueError("Delivery URL, byte count or MD5 does not match this recipe's request.")
    local = directory / "media.mp4"
    pending = directory / "media.mp4.part"
    if not local.exists():
        # Boto3 uses your local AWS read identity, not the upload key sent to FlashData.
        with pending.open("xb") as output:
            s3 = boto3.client("s3", region_name=region)
            with closing(s3.get_object(Bucket=bucket, Key=object_key)["Body"]) as stream:
                while chunk := stream.read(1024 * 1024):
                    output.write(chunk)
        local = pending
    size, digest = 0, hashlib.md5(usedforsecurity=False)
    with local.open("rb") as stream:
        while chunk := stream.read(1024 * 1024):
            size += len(chunk)
            digest.update(chunk)
    if size != file["file_size"] or digest.hexdigest() != file["md5"].lower():
        raise ValueError("Size or MD5 mismatch. Keep the local file for inspection.")
    final = directory / "media.mp4"
    if local == pending:
        pending.rename(final)
    record = {
        "job_id": receipt["id"], "video_id": request["video_id"],
        "bucket": bucket, "object_key": object_key, "bytes": size, "md5": digest.hexdigest(),
        "verified_at": datetime.now(timezone.utc).isoformat(), "usage": delivery.get("usage"),
    }
    (directory / "verified.json").write_text(json.dumps(record, indent=2), encoding="utf-8")
    print("Verified:", final, "—", size, "bytes; MD5 matches.")


if __name__ == "__main__":
    try:
        main()
    except Exception as error:
        print(str(error), file=sys.stderr)
        sys.exit(1)
验证已保存交付
python verify.py job-001

成功结果

上方字节数仅为示意。此时存储桶中已有 MP4,本地有经验证的副本,verified.json 记录文件对应的任务 ID、视频 ID、对象键、校验和及返回的用量。

终端
Verified: .../job-001/media.mp4 — 1048576 bytes; MD5 matches.

恢复操作,避免重复下载

遇到的情况下一步
提交超时或未保存任务 ID保留 job-001,根据时间戳和视频检查请求历史。不要删除文件夹来强行重新提交。先恢复原任务,再判断是否需要新请求。
pending / processing等待 10 秒,再使用 job-001 运行 collect。它只读取已有任务。
任务失败检查 job-status.json 中的错误。修复视频或存储配置后,明确使用新文件夹及前缀启动新任务。
401 / 403 / 402 / 429检查密钥、权限或余额。遇到 429 时读取已保存响应头中的 Retry-After 并等待。脚本不会自动重试提交。
获取结果失败或签名 URL 已过期使用 job-001 重新运行 collect,再次获取已有结果。
S3 AccessDenied修复本地 AWS 读取身份或存储桶策略。FlashData 上传权限与本地读取权限彼此独立。
部分文件、路径、字节数或 MD5 不匹配保留交付记录及文件以供检查。解决原因后,如需重新读取 S3,将 media.mp4.part 移到其他位置,再运行 verify。不要重新运行 submit。