Select: Provider Network Data

IdeonSelect provider-network datasets, delivered in bulk file format

Ideon's Provider-Network dataset across all major markets can be delivered in bulk file format. The data is delivered via SFTP, in a single compressed JSON Lines (jsonl) file containing all applicable providers and data on their address-level network participation. The bulk IdeonSelect files are refreshed daily and optimized for scalable ingestion, containing comprehensive, standardized provider-network data.

Directory and Refresh Cadence

Bulk files are delivered via AWS S3. The file available for download will be named providers.tar.gz, a tar file compressed using gzip to facilitate efficient transfer. Upon extraction, it will yield a file named providers.jsonl, a JSON Lines formatted file in which each line represents a discrete, valid JSON object. The file directory is: s3://bulk-data.vericred.com/networks/{username}; the username will be shared along with access credentials.

The bulk file for each user is a complete dataset of all providers participating in one or more of the user's licensed networks. Bulk files are generated on a daily basis. The generated file is a complete replacement of the prior file; changes-only files are not available.

Data Schema

Each line within the providers.jsonl file contains a single provider object. A provider object encapsulates all available and licensed data for an individual or organization healthcare provider. Provider objects are unique by npi. Within each object, provider metadata, specialty, address, and network participation information is included.

A complete data dictionary can be found in the Provider Data Dictionary section.

File Partitioning

Optionally, Ideon can enable bulk file partitioning to deliver a manifest-driven set of smaller files that are easier to download and process in parallel, while maintaining the same underlying data. To enable partitioned files, please contact your Customer Success Manager.

Bulk files are partitioned based on a configurable maximum file size, reflecting the uncompressed size of each JSON Lines file. The file size is approximate; each file will always include complete provider records, meaning files may be slightly larger or smaller than the configured amount.

If enabled, a new manifest.json file will be available in the root directory in s3://bulk-data.vericred.com/networks/{username}. This manifest will list the individual data files that are available for download, which will be organized in a dated folder. When new files are generated, the manifest file will be updated and the new files will be added to the corresponding date's directory. Additionally, the previous files and dated folders will be automatically removed. An example manifest.json file is shown below:

{
    "files": [
        "s3://bulk-data.vericred.com/networks/{username}/{run_id}/providers_1.jsonl.gz",
        "s3://bulk-data.vericred.com/networks/{username}/{run_id}/providers_2.jsonl.gz"
    ]
}