Consumer Data Access

Consumers call search APIs with API keys. Access scope is controlled by data products and entitlements, while canonical data remains owned by the data center. Consumers do not directly access databases, object storage, search indexes, or vector stores.

The current validation endpoint is http://20.169.21.11. Every request requires:

Authorization: Bearer whale_consumer_xxx

Contents

Integration Flow

Production integrations should start with a small time window and explicit platforms, then expand after result quality is verified.

Search Modes

External consumers should use POST /v1/contents/search as the unified entrypoint and select retrieval behavior with mode. The lower-level /v1/search/keyword, /v1/search/vector, and /v1/search/hybrid endpoints remain available for compatibility and specialized debugging.

When mode is omitted, Whale uses keyword unless query_vector is present, in which case it uses hybrid.

The unified entrypoint returns document-level results by default. The same document_id appears only once per page. body is the best matched chunk snippet, not the full document body; pass include_chunks=true when chunk-level hit details are needed.

Filters

Common filter dimensions:

Example request:

{
  "mode": "hybrid",
  "query": "AI video generation",
  "dataset_ids": ["social_media_raw"],
  "source_platforms": ["youtube", "bilibili", "xiaohongshu"],
  "published_from": "2026-08-01T00:00:00Z",
  "published_to": "2026-09-01T00:00:00Z",
  "languages": ["zh", "en"],
  "top_k": 20
}

Document Details

Unified search results are document-level. To load a full document, take document_id from a search result and call:

• POST /v1/documents/batch-get: batch load document metadata, current versions, optional full content, latest snapshots, parts, and facts
• GET /v1/documents/{document_id}: document metadata and current version
• GET /v1/documents/{document_id}/content: current full text
• GET /v1/documents/{document_id}/content?version_id=ver_xxx: full text for a specific version
• GET /v1/documents/{document_id}/versions: body version history
• GET /v1/documents/{document_id}/snapshots: metric, author-state, and platform-state snapshot history
• GET /v1/documents/{document_id}/parts: transcripts, OCR, specs, chapters, and other multi-part content
• GET /v1/documents/{document_id}/facts: prices, companies, locations, skills, and other structured facts

Body, title, URL, transcripts, and specs create new versions when they change. Time-varying values such as likes, comments, plays, follower counts, prices, and inventory should be read from snapshots. Use parts for readable sub-content and facts for filterable or aggregatable structured fields.

External responses do not expose Blob/R2 raw, text, manifest, or part object keys. Full text and part content must be retrieved through authorized APIs.

For list pages, use batch-get for lightweight metadata and latest snapshots first. Enable include_content=true only on detail or export flows.

Versions, Parts, And Snapshots

Metrics Dimensions

External metrics are usually interpreted by these dimensions:

Authorization

A consumer does not own the data. It only owns access rights. Entitlements can restrict:

• dataset
• source platform
• language
• region
• published time range
• allowed fields

Daily quotas are tracked by UTC calendar day, consumer, and search operation, and are reserved atomically before a request enters the search backend. Accepted requests still count when a downstream search fails; requests rejected with 429 do not.

Real-time admission is controlled by entitlement requests_per_second, burst, and max_concurrency. Rejected requests return 429 rate_limit_exceeded with Retry-After: 1, do not reach downstream services, and do not count toward usage. This limit is enforced per Gateway process; multi-Gateway deployments must mirror the policy at the shared API Gateway layer.

Returned Fields

The default response includes document metadata, chunks, title, text summary, and score information. The service redacts each result against the entitlement that actually matched it; without body permission, neither body text nor highlights are returned. Request fields can only narrow the response and cannot expand access. Document detail APIs are also protected by entitlements.

Error Handling

See Quickstart for executable examples and Authentication for key and error semantics.