---
title: Search Access
description: Consumers retrieve authorized data with API keys.
---

# Consumer Data Access

Consumers call search APIs with API keys. Access scope is controlled by data products and entitlements, while canonical data remains owned by the data center. Consumers do not directly access databases, object storage, search indexes, or vector stores.

The current validation endpoint is `http://20.169.21.11`. Every request requires:

```http
Authorization: Bearer whale_consumer_xxx
```

## Contents

| Section | Description |
| --- | --- |
| Integration flow | From API key to first query |
| Search modes | When to use keyword, vector, or hybrid search |
| Filters | Platform, dataset, time, content type, language, and facts |
| Document details | Current version, full content, history, snapshots, parts, and facts |
| Metrics dimensions | Platform, dataset, content type, time windows, and processing state |
| Error handling | Common error semantics |

## Integration Flow

```mermaid
sequenceDiagram
  participant Client as Consumer Client
  participant API as Whale API
  participant Auth as Entitlement
  participant Search as Search Backend
  participant Store as Content Store

  Client->>API: POST /v1/contents/search
  API->>Auth: validate API key and scope
  API->>Search: retrieve authorized chunks
  Search-->>API: ranked hits
  API-->>Client: document_id + snippet + score
  Client->>API: POST /v1/documents/batch-get
  API->>Auth: validate document access
  API->>Store: load details and latest snapshot
  API-->>Client: document cards
```

Production integrations should start with a small time window and explicit platforms, then expand after result quality is verified.

## Search Modes

External consumers should use `POST /v1/contents/search` as the unified entrypoint and select retrieval behavior with `mode`. The lower-level `/v1/search/keyword`, `/v1/search/vector`, and `/v1/search/hybrid` endpoints remain available for compatibility and specialized debugging.

| API | Best for | Notes |
| --- | --- | --- |
| `POST /v1/contents/search` | Recommended external entrypoint | `mode` can be `keyword`, `vector`, or `hybrid` |
| `POST /v1/search/keyword` | Exact keywords, brands, people, URLs, topics | Explainable deterministic retrieval |
| `POST /v1/search/vector` | Semantic similarity, cross-language retrieval, inconsistent wording | Can accept only `query`; server generates query embedding |
| `POST /v1/search/hybrid` | Recommended default | Combines lexical retrieval, vector retrieval, quality, and recency |

When `mode` is omitted, Whale uses `keyword` unless `query_vector` is present, in which case it uses `hybrid`.

The unified entrypoint returns document-level results by default. The same `document_id` appears only once per page. `body` is the best matched chunk snippet, not the full document body; pass `include_chunks=true` when chunk-level hit details are needed.

## Filters

Common filter dimensions:

| Filter | Example | Meaning |
| --- | --- | --- |
| `dataset_id` | `social_media_raw` | Data product or data domain |
| `source_platform` | `youtube`, `weibo`, `xianyu` | Source platform |
| `content_type` | `video`, `article`, `product` | Content type |
| `published_from/to` | `2026-08-01T00:00:00Z` | Source publish-time window |
| `language` | `zh`, `en` | Content language |
| `region` | `CN`, `US` | Platform or content region |
| `facts` | `price <= 100`, `company=Example` | Structured fact filters |

Example request:

```json
{
  "mode": "hybrid",
  "query": "AI video generation",
  "dataset_ids": ["social_media_raw"],
  "source_platforms": ["youtube", "bilibili", "xiaohongshu"],
  "published_from": "2026-08-01T00:00:00Z",
  "published_to": "2026-09-01T00:00:00Z",
  "languages": ["zh", "en"],
  "top_k": 20
}
```

## Document Details

Unified search results are document-level. To load a full document, take `document_id` from a search result and call:

- `POST /v1/documents/batch-get`: batch load document metadata, current versions, optional full content, latest snapshots, parts, and facts
- `GET /v1/documents/{document_id}`: document metadata and current version
- `GET /v1/documents/{document_id}/content`: current full text
- `GET /v1/documents/{document_id}/content?version_id=ver_xxx`: full text for a specific version
- `GET /v1/documents/{document_id}/versions`: body version history
- `GET /v1/documents/{document_id}/snapshots`: metric, author-state, and platform-state snapshot history
- `GET /v1/documents/{document_id}/parts`: transcripts, OCR, specs, chapters, and other multi-part content
- `GET /v1/documents/{document_id}/facts`: prices, companies, locations, skills, and other structured facts

Body, title, URL, transcripts, and specs create new versions when they change. Time-varying values such as likes, comments, plays, follower counts, prices, and inventory should be read from snapshots. Use `parts` for readable sub-content and `facts` for filterable or aggregatable structured fields.

External responses do not expose Blob/R2 raw, text, manifest, or part object keys. Full text and part content must be retrieved through authorized APIs.

For list pages, use `batch-get` for lightweight metadata and latest snapshots first. Enable `include_content=true` only on detail or export flows.

## Versions, Parts, And Snapshots

| Data | When to read it | Example |
| --- | --- | --- |
| Current version | Latest title, body, and URL are enough | Latest article body, latest video description |
| Historical versions | Need to know whether content changed | Title edit, product description edit |
| `parts` | One document has multiple readable pieces | YouTube transcript, audio transcription, Xiaohongshu OCR, product specs |
| `facts` | Need structured filtering or aggregation | Price, company, location, salary, skills |
| `snapshots` | Need time-varying state | Views, likes, price, inventory, author followers |

## Metrics Dimensions

External metrics are usually interpreted by these dimensions:

| Dimension | Meaning |
| --- | --- |
| Time | `published_at` for business publish time, `snapshot_at` for metric trends, `created_at` for system ingest time |
| Platform | `source_platform` identifies the source platform |
| Dataset | `dataset_id` identifies the data product or data domain |
| Content type | `content_type` identifies video, article, post, product, job, and similar types |
| Metric | Unified metric fields plus platform extension metrics |
| Processing state | Whether data is searchable, vectorized, or analyzed |

## Authorization

A consumer does not own the data. It only owns access rights. Entitlements can restrict:

- dataset
- source platform
- language
- region
- published time range
- allowed fields

Daily quotas are tracked by UTC calendar day, consumer, and search operation, and are reserved atomically before a request enters the search backend. Accepted requests still count when a downstream search fails; requests rejected with `429` do not.

Real-time admission is controlled by entitlement `requests_per_second`, `burst`, and `max_concurrency`. Rejected requests return `429 rate_limit_exceeded` with `Retry-After: 1`, do not reach downstream services, and do not count toward usage. This limit is enforced per Gateway process; multi-Gateway deployments must mirror the policy at the shared API Gateway layer.

## Returned Fields

The default response includes document metadata, chunks, title, text summary, and score information. The service redacts each result against the entitlement that actually matched it; without `body` permission, neither body text nor highlights are returned. Request `fields` can only narrow the response and cannot expand access. Document detail APIs are also protected by entitlements.

## Error Handling

| Error | Meaning | Suggested action |
| --- | --- | --- |
| `401 unauthorized` | Missing or invalid API key | Check the authorization header |
| `403 entitlement_denied` | API key cannot access this dataset, platform, or field | Narrow the query or request access |
| `400 invalid_query` | Request fields are invalid or mode is unknown | Fix parameters according to API Reference |
| `429 quota_exceeded` | Quota or rate limit reached | Reduce concurrency and wait for the seconds specified by `Retry-After` |
| `429 rate_limit_exceeded` | Real-time RPS, burst, or concurrency limit reached | Wait for `Retry-After` and reduce concurrency |
| `503 search_backend_unavailable` | OpenSearch, Milvus, embedding, or rerank is unavailable | Retry with exponential backoff |

See [Quickstart](/en/docs/quickstart) for executable examples and [Authentication](/en/docs/authentication) for key and error semantics.