Skip to content
EN
Get API key
Browse documentation

Guides

FlashData vs yt-dlp

Compare features, map your existing settings and move a video dataset workflow to managed cloud delivery.

On this page

Choose by the work you want to own

  • Keep yt-dlp for local files, custom FFmpeg processing, plugins or an existing worker you want to operate.
  • Use FlashData for highly concurrent YouTube collections delivered to cloud storage, with built-in YouTube error handling and a console or API for tracking jobs.
  • Need a finished dataset? Use managed dataset delivery when you also want help with source selection and packaging.

Compare video and data features

Featureyt-dlpFlashData
Video qualitySelect and filter source formats, including 4K and 8K.Up to 8K; best, exact, minimum or maximum resolution settings.
Video and audioSeparate streams, FFmpeg merging and custom post-processing.Video + audio, video-only or audio-only jobs. Video: MP4, WebM, MKV. Audio: M4A, MP3, Opus.
Format selection-f selectors and -S sorting.yt_dlp_format accepts a format selector; it overrides quality, FPS and audio-language selection.
Clips--download-sections with FFmpeg; time ranges or chapters.start_at and end_at; one interval for every video in a Bulk job.
Metadata--write-info-json saves a local sidecar.upload_metadata=True delivers a JSON sidecar beside the media; metadata bytes are not charged as download volume.
Captions and transcriptsRetrieve supplied or automatic tracks; save VTT, SRT or other available formats.Separate YouTube Data requests return timestamps and text as JSON. Select language and caption origin explicitly.

Compare collection and delivery features

Featureyt-dlpFlashData
Batch input--batch-file accepts a list of URLs.Bulk Jobs accepts a UTF-8 text file of YouTube IDs, up to 2 GiB.
StorageLocal files; connect an uploader for cloud storage.Direct delivery to S3, S3-compatible storage, GCS, Azure, OSS, COS, BOS or TOS.
ProgressCLI progress, Python hooks and your own queue or dashboard.Saved job IDs, Bulk progress, completed/failed/invalid counts and console history.
Pause and resumePartial-file resume and archives; your process supervisor keeps workers alive.Pause new Bulk submissions and resume the saved job. Accepted work can finish while paused.
Delivery recordsFiles and info JSON; add your own upload receipts and checksums.Media location, file size and MD5 in result records, plus metadata-file details when requested.
Deduplication--download-archive tracks completed IDs for a download policy.Your application filters delivered IDs using its inventory and saved Bulk results.

Keep the delivery records. Bulk result parts are retrievable for 24 hours after completion. Save them in your dataset inventory; media retention follows your own storage policy.

Let FlashData handle YouTube errors at scale

Run downloads at near-unlimited concurrency. FlashData handles YouTube 403s, bot checks, session renewal and 429s internally, so your team can focus on the delivered files.

ProblemWith your yt-dlp workersWith FlashData
YouTube changes break extractionUpdate nightly and check Deno, EJS and your worker image.FlashData maintains the retrieval stack. Your application submits IDs and output settings.
403s or persistent bot checksReduce concurrency, test healthy exit IPs and keep one route through extraction and transfer.Automatic bot handling, proxy selection and recovery resolve these errors inside FlashData.
Browser sessions need renewalValid cookies can help small jobs; you maintain and renew those sessions.FlashData manages access and session handling internally. No browser cookies to supply, rotate or renew.
YouTube 429s as traffic increasesCool down the exit IP and coordinate request pacing across workers.FlashData handles YouTube throttling, request routing and recovery internally, including during large concurrent collections.
High concurrent download volumeSize worker pools, maintain healthy proxy capacity and coordinate request pacing.Scale concurrent downloads through FlashData's managed retrieval infrastructure, with bot handling and recovery built in.

Map your existing yt-dlp settings

Existing settingFlashData settingMigration detail
-f / --formatyt_dlp_formatUse a format selector, not a CLI command. It overrides video_quality, fps and audio_language; output format and clipping still apply.
--merge-output-format mp4download_type=audio_video, output_format=mp4Choose the required codec policy in the selector when your decoder needs particular codecs.
-x --audio-formatdownload_type=audio, output_formatChoose M4A, MP3 or Opus; audio_quality accepts best or a supported bitrate string.
--download-sectionsstart_at, end_atSplit collections into separate jobs when clip intervals differ.
--write-info-jsonupload_metadata=TrueConsume the returned metadata-file reference; adapt readers to its schema.
--write-subs / --write-auto-subsyoutube.captions(..., language=..., auto=...)A separate request. false selects uploader-provided captions; true selects automatic captions.
-o and an upload hookstorage.object_prefixChoose a bucket prefix. Keep custom naming and downstream processing in your application.
--batch-file and --download-archiveBulk IDs file and your delivery inventoryExpand URLs/playlists into IDs and remove completed items before upload. Archives are not imported automatically.

Move the remaining queue in five steps

  1. Reconcile completed work. Match your archive against actual media, uploads and required caption files. Keep a separate list of items needing only caption or upload repair.
  2. Prepare the remaining IDs. Expand playlists and channels, remove duplicates, and save one YouTube ID per line. Group videos by output and caption policy.
  3. Choose output and storage. In Bulk Jobs, select YouTube Download, attach the file, configure video + audio, MP4 and your quality or format selector, then enable metadata if needed.
  4. Start and save the Bulk ID. Use a new object prefix for this run. Follow Completed, Failed and Invalid counts; pause or resume this same job when needed.
  5. Collect and join the results. Save the manifest and result parts, then join caption results by video ID. Create a new retry file only for reviewed failures.
video-ids.txt
jNQXAC9IVRw
dQw4w9WgXcQ
UTF-8, one ID per line. No URLs, header row or youtube archive prefixes.

One output policy per Bulk job. Preserve your existing files and archive. The example below establishes the media, metadata and caption record for one item before you apply that policy to a collection.

Example: a video dataset item

Output: best available video + audio merged to MP4, source metadata, and available automatic English captions with timestamps. Both workflows use the same video ID and format selector.

ArtifactLocal yt-dlp workflowFlashData workflow
Video + audioMP4 under dataset-local/VIDEO_ID/.MP4 in your bucket; its location and properties appear in delivery.json.
MetadataA local .info.json sidecar.A JSON sidecar in the same bucket, referenced by the media result.
CaptionsAn available English automatic-caption track saved as VTT.A separate captions request; the example saves tracks and errors in local captions.json.
Dataset inventoryJoin files, archive and caption availability by ID.Local item.json joins the video ID, accepted job, media result and caption status.

Caption origin is explicit. This example requests automatic captions. For uploader-provided captions, use --write-subs instead of --write-auto-subs, and set auto=False in the API call.

1. Run the local yt-dlp equivalent

Download media, metadata and English captions
python -m yt_dlp --js-runtimes deno --impersonate="" --no-playlist \
  -f "bestvideo+bestaudio/best" --merge-output-format mp4 --remux-video mp4 \
  --write-info-json --write-auto-subs --sub-langs "en" --sub-format "vtt/best" \
  --sleep-subtitles 5 --concurrent-fragments 1 -P dataset-local -o "%(id)s/%(id)s.%(ext)s" \
  "https://www.youtube.com/watch?v=jNQXAC9IVRw"
Use the latest nightly with [default,curl-cffi], Deno, FFmpeg and ffprobe. MP4 keeps the selected source codecs.

Check captions separately. No matching track and a failed caption request require different follow-up. Keep the media already saved; do not replay a completed download to repair text.

2. Run the FlashData dataset example

Media and metadata go to your bucket. The Python example saves delivery records, caption JSON and the joined item record locally. Your dataset publisher can upload these JSON records alongside the media.

Install the Python SDK
python -m pip install "flashdata>=1.2.0,<2"
managed_dataset.py
import json
from pathlib import Path

from flashdata import FlashData


def save_json(path, value):
    path.write_text(json.dumps(value, indent=2), encoding="utf-8")


def collect_dataset_item(client, video_id, storage, directory="dataset"):
    folder = Path(directory) / video_id
    folder.mkdir(parents=True, exist_ok=True)
    receipt = folder / "download-job-id.txt"

    if not receipt.exists():
        # Reserve the checkpoint before submitting; never overwrite an old ID.
        with receipt.open("x", encoding="utf-8") as checkpoint:
            job = client.youtube.download(
                video_id,
                storage=storage,
                download_type="audio_video",
                output_format="mp4",
                yt_dlp_format="bestvideo+bestaudio/best",
                upload_metadata=True,
            )
            checkpoint.write(job["id"])

    job_id = receipt.read_text(encoding="utf-8").strip()
    if not job_id:
        raise RuntimeError("Submission outcome unknown; inspect job history first")

    # A rerun waits for the saved job instead of downloading the video again.
    client.jobs.wait(job_id, timeout=900)
    delivery = client.jobs.result(job_id)
    save_json(folder / "delivery.json", delivery)

    # Captions are a separate YouTube Data request, with their own usage charge.
    response = client.youtube.captions(video_id, language=["en"], auto=True)
    captions = response["results"][0]
    save_json(folder / "captions.json", captions)

    item = {
        "video_id": video_id,
        "download_job_id": job_id,
        "media": delivery["results"][0],
        "captions_file": "captions.json",
        "caption_status": "available" if captions["tracks"] else "needs_review",
    }
    save_json(folder / "item.json", item)
    return item


if __name__ == "__main__":
    storage = {
        "storage_type": "s3",
        "storage_url": "your-media-bucket",
        "object_prefix": "datasets/ytdlp-migration/run-01",
        "region": "us-east-1",
        "access_key": "your-storage-access-key",
        "secret_key": "your-storage-secret-key",
    }
    with FlashData(api_key="your-flashdata-api-key") as client:
        item = collect_dataset_item(client, "jNQXAC9IVRw", storage)
    print(json.dumps(item, indent=2))
Replace the placeholder API key and storage values, then run python managed_dataset.py. The request uses your account credits. Reruns reuse the saved download job and request captions again.

3. Verify the item before publishing it

CheckWhat to record
MediaRead the delivered object with your storage client. Check file size, MD5, duration, video/audio streams and decoder compatibility.
MetadataPreserve the source video ID and the metadata-file reference. Keep that relationship when renaming objects.
CaptionsCheck tracks and errors in captions.json. Store actual languageCode, isAutoGenerated and timestamped segments.
Missing or failed captionsneeds_review means no track was returned. language_unavailable needs a different caption policy or transcription; retrieval_failed can be retried separately.
Dataset indexJoin media and captions by video ID. Publish only items that meet your collection's required fields.

A caption failure does not erase the media result. The script saves delivery.json before requesting captions. If that request raises an error, rerun with the same directory to reuse the accepted download job.

Recover a stopped run without downloading again

  • Client stopped while waiting: rerun managed_dataset.py with the same directory; it reads the saved ID and resumes polling.
  • Download job failed: inspect its error and the original source before starting a replacement job.
  • Media succeeded, captions failed: keep delivery.json and the media. Repeat the caption stage; the saved media job is reused.
  • Submission outcome is unknown: an empty checkpoint deliberately blocks another submission. Find the original job in history and reconcile its ID first.
  • Scaling to Bulk: save the Bulk ID and result parts, keep successful media IDs, and submit a new file only for reviewed failures.

Compare the cost of the completed collection

OptionWhat you pay for
yt-dlp$0 software usage fee, plus your actual compute, temporary disk, network/proxies and operating time.
FlashData PAYG$0.10 per GB delivered ($100 per TB), with no monthly subscription. 100 GB uses $10 of credits.
FlashData subscriptionsLower published usage rates with monthly fees and included shared credits. Business publishes $0.04/GB with a $1,999 monthly fee.

PAYG volume pricing can be discussed. For recurring collections, compare your expected delivered GB with the available plan rates and included credits. Count usage already covered by a subscription only once.

Companion metadata JSON is excluded from download-volume charges. Caption requests use YouTube Data pricing; destination storage is separate.

Would a different local interface solve the problem?

  • Python integration: yt-dlp already exposes YoutubeDL; the Python guide shows direct calls without shell commands.
  • A desktop interface: a GUI can simplify manual work, while its downloader engine still needs updates.
  • A self-hosted HTTP wrapper: clients get an API, while your team continues to operate extraction, queues and delivery.

Take the next step