# Select: Provider Network Data

Ideon's Provider-Network dataset across all major markets can be delivered in bulk file format. The data is delivered via SFTP, in a single compressed JSON Lines (jsonl) file containing all applicable providers and data on their address-level network participation. The bulk IdeonSelect files are refreshed daily and optimized for scalable ingestion, containing comprehensive, standardized provider-network data.

## Directory and Refresh Cadence 

Bulk files are delivered via AWS S3. The file directory is: `s3://bulk-data.vericred.com/networks/{username}`; the username will be shared along with access credentials.

The bulk file for each user is a complete dataset of all providers participating in one or more of the user's licensed networks. Bulk files are generated on a daily basis. The generated file is a complete replacement of the prior file; changes-only files are not available.

Filenames and extensions in S3 will vary depending on the compression methodology and other configurations applied to the customer's delivery. Supported compression formats include:

* `gzip` (e.g., `providers.jsonl.gz`)
* `tar + gzip` (e.g., `providers.tar.gz`, which extracts to `providers.jsonl`)
* `bzip2` (e.g., `providers.jsonl.bz2`)
* `gzip`, partitioned by provider type (e.g., separate compressed files per provider type)

Because exact filenames vary by configuration, customers should always rely on the `manifest.json` file (described below) as the source of truth for which files to download.

## Manifest Files

A `manifest.json` file in the root directory (`s3://bulk-data.vericred.com/networks/{username}/manifest.json`) is the source of truth for the latest files available for download. The manifest is used for **all customers**, regardless of whether they have opted into partitioned files, and is updated each time new files are generated. Customers should always read `manifest.json` first and download the file URLs listed within it, rather than assuming a fixed filename.

The manifest contains a `file_urls` array that lists:

* The **metadata file**, which provides context on the current run.
* The **providers file(s)**, containing the provider-network data. The file extension will match the customer's configured compression methodology.

An example `manifest.json` file is shown below:

```json
{
    "bulk_run_id": 1234,
    "run_date": "2026-01-01",
    "file_urls": [
        "s3://bulk-data.vericred.com/networks/{username}/1234/metadata.json",
        "s3://bulk-data.vericred.com/networks/{username}/1234/providers.jsonl.gz"
    ]
}
```

When new files are generated, the manifest file will be updated with the new file URLs. Previous files and dated folders will be automatically removed.

## Data Schema

Each line within the `providers.jsonl` file contains a single provider object. A provider object encapsulates all available and licensed data for an individual or organization healthcare provider. Provider objects are unique by `npi`. Within each object, provider metadata, specialty, address, and network participation information is included.

A complete data dictionary can be found in the [Provider Data Dictionary section](/quote-and-select/flat-file-data-dictionaries/select-provider-network-data/provider-data-dictionary.md).

## File Partitioning

Optionally, Ideon can enable bulk file partitioning to deliver a set of smaller files that are easier to download and process in parallel, while maintaining the same underlying data. To enable partitioned files, please contact your Customer Success Manager.

Bulk files are partitioned based on a configurable maximum file size, reflecting the uncompressed size of each JSON Lines file. The file size is approximate; each file will always include complete provider records, meaning files may be slightly larger or smaller than the configured amount.

When partitioning is enabled, the `manifest.json` file will list multiple providers files in the `file_urls` array rather than a single providers file. An example partitioned `manifest.json` is shown below:

```json
{
    "file_urls": [
        "s3://bulk-data.vericred.com/networks/{username}/{run_id}/metadata.json",
        "s3://bulk-data.vericred.com/networks/{username}/{run_id}/providers_1.jsonl.gz",
        "s3://bulk-data.vericred.com/networks/{username}/{run_id}/providers_2.jsonl.gz"
    ]
}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ideonapi.gitbook.io/quote-and-select/flat-file-data-dictionaries/select-provider-network-data.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.