Datasets API

Overview

The API follows an asynchronous request-response pattern:

1. Request — Submit a dataset request with filters and format preferences
2. Poll — Check request status until processing completes
3. Download — Stream the data in your preferred format

Browse all available datasets and their schemas on the Datasets page.

Quick Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// 1. Create a dataset request
const response = await fetch('https://rebrowser.net/api/datasets/requests/create', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YOUR_API_KEY',
  },
  body: JSON.stringify({
    datasetSlug: 'iheart',
    entitySlug: 'stations',
    viewSlug: 'all',     // optional server-side saved/config view; no long filter URL
    format: 'json',
    page: null,         // null = export all pages; omit or set number for single page
    allowPurchase: true,
  }),
})
const { requestId } = await response.json()

// 2. Poll for completion
let status
do {
  await new Promise(r => setTimeout(r, 5000))
  const statusRes = await fetch(
    `https://rebrowser.net/api/datasets/requests/${requestId}`,
    { headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }
  )
  status = await statusRes.json()
} while (['billing', 'pending', 'processing'].includes(status.status))

// 3. Download the data
const dataRes = await fetch(status.downloadUrl, {
  headers: { 'Authorization': 'Bearer YOUR_API_KEY' },
})
const rows = await dataRes.json()
console.log(rows)

API Endpoints

POST /api/datasets/requests/create

Create a new dataset export request. Returns either a preview (with cost estimate) or creates a purchase request.

When allowPurchase is not set or false:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
  "meta": {
    "datasetSlug": "iheart",
    "entitySlug": "stations",
    "format": "json",
    "totalMatches": 15420,
    "totalRows": 50,
    "page": 1,
    "pageSize": 50,
    "returnedRows": 50,
    "ownedRows": 1000,
    "freeRows": 5000,
    "unownedRows": 9420,
    "estimatedCost": 47.10,
    "selectionFingerprint": "…"
  },
  "message": "Set allowPurchase: true to purchase 9,420 missing rows for $47.10 and create the export request.",
  "rows": [
    {
      "stationId": "1234",
      "name": "Station Name",
      "premiumField": "**HIDDEN**"  // Premium fields show this value if plan doesn't include premium access
    }
  ]
}

When allowPurchase=true:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
  "success": true,
  "meta": {
    "datasetSlug": "iheart",
    "entitySlug": "stations",
    "format": "json",
    "totalRows": 15420,
    "ownedRows": 1000,
    "freeRows": 5000,
    "unownedRows": 9420,
    "estimatedCost": 47.10
  },
  "requestId": "507f1f77bcf86cd799439011",
  "datasetRequestId": "507f1f77bcf86cd799439011",
  "status": "pending",
  "message": "Dataset request created. Poll GET /api/datasets/requests/507f1f77bcf86cd799439011 for status."
}

GET /api/datasets/requests/:requestId

Check the status of a dataset request. Poll this endpoint until status is success or error.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
  "success": true,
  "requestId": "507f1f77bcf86cd799439011",
  "status": "success",
  "datasetSlug": "iheart",
  "entitySlug": "stations",
  "format": "json",
  "page": null,
  "estimatedRows": 15420,
  "estimatedCost": 47.10,
  "selectionFingerprint": "…",
  "manifestCount": 15420,
  "chargedRows": 9420,
  "createdAt": "2026-02-05T10:30:00Z",
  "updatedAt": "2026-02-05T10:32:15Z",
  "completedAt": "2026-02-05T10:32:15Z",
  "rowCount": 15420,
  "expiresAt": "2026-02-12T10:32:15Z",
  "downloadUrl": "/api/datasets/requests/.../download?format=json",
  "downloadUrls": {
    "json": "/api/datasets/requests/.../download?format=json",
    "jsonl": "/api/datasets/requests/.../download?format=jsonl",
    "csv": "/api/datasets/requests/.../download?format=csv",
    "parquet": "/api/datasets/requests/.../download?format=parquet"
  }
}

GET /api/datasets/requests/:requestId/download

Stream the completed dataset. Returns the data file directly.

Data Formats

Premium Fields

Some datasets include premium fields that require a premium plan subscription. When accessing data:

• Without premium access: Premium fields will contain the value **HIDDEN** in all responses
• With premium access: Premium fields will contain the actual data values
• Filtering by premium fields: Requests with filters on premium fields will be rejected if your plan doesn't include premium access
• Free dataset rows: Premium fields are fully visible in free dataset rows regardless of plan tier

Check each dataset's field list in the UI to identify which fields require premium access.

JSON

Returns data as a JSON array of objects. Each row contains the full record with all fields.

1
2
3
4
[
  {"stationId":"1234","name":"KIIS-FM","market":"Los Angeles","genres":["Pop","Top 40"]},
  {"stationId":"5678","name":"KROQ-FM","market":"Los Angeles","genres":["Alternative","Rock"]}
]

Best for: General programmatic access, maximum flexibility

JSONL (JSON Lines)

Returns data as newline-delimited JSON. Each line is a standalone JSON object — no wrapping array, no commas between records.

1
2
{"stationId":"1234","name":"KIIS-FM","market":"Los Angeles","genres":["Pop","Top 40"]}
{"stationId":"5678","name":"KROQ-FM","market":"Los Angeles","genres":["Alternative","Rock"]}

Best for: Streaming pipelines, line-by-line processing, large datasets

Plan requirement: Business plan and above

CSV

Standard comma-separated values with header row.

Best for: Excel, Google Sheets, pandas, general data analysis

Parquet

Apache Parquet columnar format with zstd compression.

Best for: Big data pipelines, Spark, Athena, data lakes, efficient storage

Plan requirement: Business plan and above

Billing & Pricing

Row-Based Pricing

Cost Calculation

1
2
3
4
5
6
7
billableRows = Math.ceil(unownedRows / 50) * 50
totalCost = (billableRows / 1000) * ratePer1kRows

// Examples:
// 1 row → 50 billable rows
// 51 rows → 100 billable rows
// 235 rows → 250 billable rows

Row Ownership States

Important Limits

• Row retention: Purchased rows expire after 30 days. Re-downloading after expiration requires a new purchase.
• Request expiration: Completed download requests expire after 7 days.
• Selection manifest: A paid request freezes its exact ordered identities for 14 days to cover processing, retry, and reconciliation. Field values are read when the export runs; identity membership does not change.
• Free access: Some datasets offer free access for older data (configurable per dataset).

Filtering & Sorting

View Settings

Use the viewSettings object in the request body to specify filters, sort order, and page size:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
const response = await fetch('https://rebrowser.net/api/datasets/requests/create', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YOUR_API_KEY',
  },
  body: JSON.stringify({
    datasetSlug: 'iheart',
    entitySlug: 'stations',
    format: 'json',
    page: null,  // null = all pages, 1/2/3... = specific page
    viewSettings: {
      filters: {
        conjunction: 'and',
        rules: [
          { type: 'rule', field: 'market', operator: 'is', value: 'Los Angeles' },
          { type: 'rule', field: 'genres', operator: 'contains', value: 'Pop' },
        ],
      },
      sort: {
        field: 'cume',
        dir: 'DESC',
      },
      pageSize: 100,
    },
    allowPurchase: true,
  }),
})

Filter Operators

Explorer's field list is the source of truth for filterable fields and operators. Unknown fields, unsupported operators, incomplete values, and invalid groups are rejected; they are never silently ignored.

Text fields: is, isNot, isAnyOf, isNoneOf, contains, notContains, startsWith, endsWith, isEmpty, isNotEmpty

Number fields: eq, neq, gt, gte, lt, lte

DateTime fields: is, before, after

Boolean fields: isTrue, isFalse

Complexity limits: 64 KiB JSON body, 100 filter rules, 1,000 total set values, three nested group levels, and 1,000 characters per value. A saved view can be referenced by viewSlug, so large filters do not need to be placed in a URL.

Error Handling

HTTP Status Codes

Error Response Format

1
2
3
4
5
{
  "status": "error",
  "code": "UNKNOWN_FIELD",
  "error": "Unknown filter field"
}

A recoverable paid-request interruption returns 503 with status: "recovering" and the existing requestId. Do not resubmit the selection; poll that request until it reaches pending, processing, success, or error.

Common Errors

Best Practices

• Preview First: Use allowPurchase: false to validate filters, pagination, identity counts, and the selection fingerprint before purchase. Preview data can change until a paid request freezes its manifest.
• Use Preview Mode: Call without allowPurchase to see cost estimates and preview rows before committing to a purchase.
• Check Premium Fields: Review which fields are marked as premium in the dataset UI. Premium fields will show **HIDDEN** values unless your plan includes premium access.
• Export All vs. Single Page: Pass page: null to export all matching rows. If page is omitted, only page 1 is returned — this is a safety default to prevent accidental large purchases.
• Paginate Large Requests: For large datasets, start with a single page to test your filters and control costs before exporting all pages.
• Handle Polling Gracefully: Implement exponential backoff when polling for status (start at 5s, increase to 30s max).
• Choose the Right Format: Use JSON or CSV for general use. Use JSONL for streaming large datasets. Use Parquet for data warehouse integrations.
• Monitor Row Expiration: Track when purchased rows expire (30 days) if you need ongoing access.
• Download Before Expiration: Download completed requests within 7 days before they expire.