Browse documentation
Guides
Keep a result with its original input, source links and usage, then hand it to another process without another query.
On this page
Query once, use the result across tasks
Build a report, rerun your parser or give a later agent step the same evidence. This workflow saves one search as a portable archive; reading that archive needs neither a FlashData key nor the original CLI state.
archive/saved-search/
request.json original query and parameters
response.json unchanged API response
citations.ndjson titles, source URLs and snippets
manifest.json run ID, usage, file sizes and SHA-256Before you start
Use a new working folder, the FlashData CLI with google_search permission and available credits. Reuse an authenticated profile or FLASHDATA_API_KEY. Bash inspection uses jq; Windows uses PowerShell 7. The local archive scripts use Python 3.10+ and its standard library.
Continue when data.accessVerified is true. For CLI retries, keep the same profile, OS user and state directory. An agent should load its existing credential from its secret store.
flashdata auth status --json1. Keep the input and save one response
Save request.json below, then run one command. --input uses this exact request; keep the file unchanged afterward. --output saves the API body in search.json; --json prints CLI metadata. saved-search is this task's local run ID.
{
"source": "google_search",
"query": "battery recycling research",
"params": {
"num": 5
}
}flashdata query --input request.json \
--run-id saved-search --output search.json --jsonflashdata query --input request.json `
--run-id saved-search --output search.json --json2. Validate the saved response
Expect source: google_search, status: done and a results array. Print each organic result's title, link and snippet. Actual results vary; an empty successful collection is different from a failed request or a missing field.
jq -e '.source == "google_search" and .status == "done"
and (.results | type == "array")
and all(.results[]; (.organic | type == "array"))' search.json > /dev/null &&
jq -c '.results[] | .organic[] | {title, link, snippet}' search.json$result = Get-Content -Raw -LiteralPath search.json | ConvertFrom-Json
if ($result.source -ne 'google_search' -or $result.status -ne 'done' -or
-not ($result.results -is [array])) {
throw 'Inspect search.json before treating the task as complete.'
}
foreach ($page in $result.results) {
if (-not ($page.organic -is [array])) { throw 'Missing organic results array.' }
$page.organic | Select-Object title, link, snippet
}3. Archive the input, response and citations
Save archive.py below, then run it with the run ID from step 1. It preserves the raw files, extracts source links and writes manifest.json last with file checksums and usage. Expect complete: true; the citation count depends on the search results. No network requests are made.
Save archive.py
Create archive.py beside request.json and search.json in your working folder. Copy the complete code below into it and save it.
The example archives Google Search results. Existing archive directories are kept intact. Keep request.json, search.json and the matching run together until archiving completes.
# archive.py — Python 3.10+, standard library only; no network requests.
import argparse
import hashlib
import json
import re
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlsplit
parser = argparse.ArgumentParser()
parser.add_argument('--run-id', required=True)
args = parser.parse_args()
if not re.fullmatch(r'[A-Za-z0-9_-]+', args.run_id):
raise ValueError('Use the CLI run ID from this task.')
request_bytes = Path('request.json').read_bytes()
result_bytes = Path('search.json').read_bytes()
request = json.loads(request_bytes.decode('utf-8-sig'))
result = json.loads(result_bytes.decode('utf-8-sig'))
if (request.get('source') != 'google_search' or result.get('source') != request['source']
or result.get('status') not in ['done', 'completed']
or not isinstance(result.get('results'), list)):
raise ValueError('Inspect the request and result before archiving.')
citations = []
for page in result['results']:
if not isinstance(page, dict) or not isinstance(page.get('organic'), list):
raise ValueError('Expected an organic results array, including when it is empty.')
for item in page['organic']:
if not isinstance(item, dict) or not isinstance(item.get('link'), str):
raise ValueError('A result is missing its source URL.')
url = urlsplit(item['link'])
if url.scheme not in ['http', 'https'] or not url.netloc:
raise ValueError('Inspect an unexpected source URL.')
citations.append({key: item.get(key, '') for key in ['title', 'link', 'snippet']})
files = {'request.json': request_bytes, 'response.json': result_bytes,
'citations.ndjson': ''.join(json.dumps(row, ensure_ascii=False) + '\n'
for row in citations).encode('utf-8')}
manifest = {'version': 1, 'run_id': args.run_id, 'source': request['source'],
'archived_at': datetime.now(timezone.utc).isoformat(),
'usage': result.get('usage'), 'citation_count': len(citations),
'files': {name: {'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest()}
for name, data in files.items()}}
folder = Path('archive') / args.run_id
folder.mkdir(parents=True) # Existing archives are preserved. Choose a new run ID for new work.
for name, data in files.items():
with (folder / name).open('xb') as output:
output.write(data)
# Write the manifest last. Without it, the archive is incomplete.
with (folder / 'manifest.json').open('x', encoding='utf-8') as output:
json.dump(manifest, output, ensure_ascii=False, indent=2)
print(json.dumps({'archive': str(folder), 'citations': len(citations), 'complete': True}))Create the archive
Run from the folder containing archive.py, request.json and search.json.
python3 archive.py --run-id saved-searchpython archive.py --run-id saved-search4. Reuse the archive without credentials
Save reuse.py below. Copy the complete archive/saved-search folder to the process that needs it, then run this reader. It verifies every file before printing the original query, usage and citations. It works after API-key revocation or hosted-result expiry because it reads only your saved copy.
Save reuse.py
Create reuse.py in the working folder containing archive/. On another machine, copy both reuse.py and the complete archive/saved-search directory, keeping this layout.
# reuse.py — run without a FlashData key or CLI.
import hashlib
import json
import sys
from pathlib import Path
folder = Path(sys.argv[1])
manifest = json.loads((folder / 'manifest.json').read_text(encoding='utf-8'))
expected = {'request.json', 'response.json', 'citations.ndjson'}
if manifest.get('version') != 1 or set(manifest.get('files', {})) != expected:
raise ValueError('Unknown or incomplete archive manifest.')
files = {}
for name in sorted(expected):
data = (folder / name).read_bytes()
record = manifest['files'][name]
if len(data) != record['bytes'] or hashlib.sha256(data).hexdigest() != record['sha256']:
raise ValueError('Archive changed or was not fully copied: ' + name)
files[name] = data
request = json.loads(files['request.json'].decode('utf-8-sig'))
citations = [json.loads(line) for line in files['citations.ndjson'].decode('utf-8').splitlines()]
if len(citations) != manifest['citation_count']:
raise ValueError('Citation count does not match the manifest.')
print(json.dumps({'run_id': manifest['run_id'], 'query': request['query'],
'archived_at': manifest['archived_at'], 'usage': manifest['usage'],
'citations': citations}, indent=2))Read the saved archive
Save reuse.py, then run from its folder. The final argument is the path to the copied archive.
python3 reuse.py archive/saved-searchpython reuse.py archive/saved-searchExport from the CLI journal instead
If you still have the original CLI profile and state directory, export the saved run to a new filename. This makes no new query, but it verifies live access first. An existing result file or archive can be read locally without that check.
flashdata runs show saved-search --result --output search-copy.json --jsonDecide when to reuse or refresh
archived_at records when you saved the files, not when a source page changed. Your application decides how recent the evidence must be; copying a file does not refresh its contents.
| Task | Action |
|---|---|
| Rerun a parser, regenerate a report or continue an agent step | Reuse the saved response. Keep source URLs, query parameters and usage with it. |
| Produce a new daily briefing or answer a changed query | Create a new task/run ID, working folder and archive. Keep older snapshots as history. |
| Count usage across derived files | Count the original query once. Copying a response or making another report does not create another charge. |
Know what expires
| Result | What to keep | Retention |
|---|---|---|
| Realtime query | The response file and original input | Your saved copy follows your storage policy |
| Async / Batch job | Job ID, submission receipt and result JSON | 24 hours after completion |
| Bulk | Manifest and all NDJSON parts | 24 hours after the whole job completes; check expiresAt |
| YouTube Download | Delivery JSON and media in your destination | Delivery JSON: job retention. Media: your bucket lifecycle |
| MCP chunk cache | The fully assembled result | 10 minutes; only the four newest large results per API key are retained |
Keep a small sidecar instead
Keep the original response unchanged. Save a small sidecar with the input, source, save time and usage so a later process can understand what produced the file. Keep source URLs and video timestamps when transforming results.
(
set -euC
jq --arg run_id 'saved-search' --slurpfile request request.json \
'{run_id: $run_id, source, input: $request[0],
saved_at: (now | todateiso8601), result_file: "search.json", usage}' \
search.json > search-record.json
cat search-record.json
)Save directly with HTTP
Use this instead of the CLI query in step 1. Save the same request.json first; this runs one new query using Bash, cURL 7.76+ and jq. Reuse FLASHDATA_API_KEY and choose a folder without search.json. After validation, the same archive scripts work with this result; choose your own task ID for --run-id.
A nonzero cURL exit means the result is not ready for use. Inspect the saved response before retrying; a connection timeout can happen after a query was accepted. Use the validation command in step 2.
# Bash: paste the key into a hidden prompt, not into this command.
read -r -s -p "FlashData API key: " FLASHDATA_API_KEY
printf '\n'
export FLASHDATA_API_KEY(
set -euC
: "${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY first}"
test -f request.json
curl --fail-with-body --silent --show-error --max-time 120 \
--request POST 'https://data.flashdata.dev/v1/queries/realtime' \
--header "X-API-Key: $FLASHDATA_API_KEY" \
--header 'Content-Type: application/json' --header 'Accept: application/json' \
--data-binary @request.json > search.json
jq . search.json
)Save an existing background job
Use submission.json from the async guide and the same account credential. These commands retrieve existing work; they do not submit another query. The CLI waits for completion and handles the result redirect. Save the result within 24 hours of completion.
(
set -euC
JOB_ID=$(jq -er '.remoteId // .id // .submission.id' submission.json)
flashdata jobs wait "$JOB_ID" --json
flashdata jobs result "$JOB_ID" --output job-result.json --json
jq '{source, status, results, usage}' job-result.json
)The status check stops if the job is not completed. If it is still queued or running, wait and run this read-only snippet again; inspect a failed job instead. A 302 is downloaded without forwarding your API key. The temporary response is validated before job-result.json is created.
(
set -euC
: "${FLASHDATA_API_KEY:?Set FLASHDATA_API_KEY first}"
test ! -e job-result.json || { echo 'Choose a new output filename.' >&2; exit 1; }
JOB_ID=$(jq -er '.remoteId // .id // .submission.id' submission.json)
printf '%s' "$JOB_ID" | grep -Eq '^[A-Za-z0-9_-]+$'
SOURCE=$(jq -er '(.submission // .).source' submission.json)
API="https://api.flashdata.dev/v2/jobs/$JOB_ID"
curl --fail-with-body --silent --show-error \
--max-time 30 --header "X-API-Key: $FLASHDATA_API_KEY" "$API" >| job-status.json
jq -e --arg id "$JOB_ID" --arg source "$SOURCE" \
'.job.id == $id and .job.source == $source and .job.status == "completed"' job-status.json >/dev/null
STATUS=$(curl --silent --show-error \
--header "X-API-Key: $FLASHDATA_API_KEY" \
--max-time 120 --dump-header result-headers.txt --output job-result.pending \
--write-out '%{http_code}' "$API/results")
case "$STATUS" in
200) ;;
302)
LOCATION=$(awk 'tolower($1) == "location:" { sub(/\r$/, "", $2); print $2 }' result-headers.txt)
# No FlashData API key on the signed storage request.
curl --fail-with-body --silent --show-error --compressed --max-time 120 \
--proto '=https' --output job-result.pending "$LOCATION" ;;
*) echo "HTTP $STATUS: inspect job-result.pending and result-headers.txt." >&2; exit 1 ;;
esac
jq -e --arg source "$SOURCE" '.source == $source and (.status == "done" or .status == "completed")
and (.results | type == "array")' job-result.pending >/dev/null
cat job-result.pending > job-result.json
jq '{source, status, results, usage}' job-result.json
)Save every Bulk result part
Replace BULK_ID with the ID in your submission receipt. After the job completes, the CLI follows the manifest and saves every part as one NDJSON file. Keep the manifest too when you need counts and part metadata.
The file can contain both successful and failed rows. Reconcile input indexes and counts before marking the whole task complete. Do not rerun the entire input file to recover one failed row.
flashdata bulk results BULK_ID --output bulk-results.ndjson --timeout 30m --jsonKeep the delivery record and media
delivery.json contains a delivery record, not the video or audio bytes. Save its URL, size and checksum alongside the task. The media lives in your destination storage and follows that storage's lifecycle.
For a private destination, use that provider's credentials or signed URL to inspect the object. A missing temporary result JSON does not mean your stored media was deleted.
jq '.results[] | {file_url, file_size, md5, downloaded_bytes, output_format}' delivery.jsonAssemble MCP chunks before saving
When a tool returns result_id, chunk and next_offset, read the remaining chunks with get_result using the same API key. Use the exact next_offset returned by the tool; the values below are placeholders.
- Append each chunk in offset order. Follow next_offset until it is null.
- Parse the joined text once as JSON, then write the full result to your own file or storage.
- Save before expires_at. New large responses can evict older results before the 10-minute limit; the cache keeps four per API key.
{
"name": "get_result",
"arguments": {
"result_id": "RESULT_ID_FROM_TOOL",
"offset": 16000
}
}Fix saving and retrieval problems
| Problem | Fix |
|---|---|
| Output file already exists | Inspect it first. For a completed CLI run, use runs show with a new output filename instead of submitting another query. |
| CLI cannot find the run | Use the original OS user, profile and state directory. If you kept a remote job ID, retrieve that job directly. |
| 401 / 403 on reuse | Run flashdata auth status --json and restore valid access. A local file already saved can still be read without API access. |
| 409 / 410 | 409: wait for the existing job. 410: use your saved copy; the temporary result has expired. |
| Disk error after a query | Keep the run ID or job ID. Fix storage and recover existing work before considering a new submission. |
| Archive has no manifest or a checksum fails | Restore or copy the original files again. Keep an incomplete archive for inspection; rebuilding the archive does not require a new query. |
| MCP · result_unavailable | Check result ID, offset, API key and expiry. Use a saved copy if the cache has expired; do not loop by repeating a paid query. |