Bulk

Bulk download + Parquet ingest for PubChem BioAssay activity data.

BIOACTIVITIES_PATH = 'Extras/bioactivities.tsv.gz'

Path (under the bioassay root) to the whole-database bioactivity dump.

PUBCHEM_BIOASSAY_FTP = 'https://ftp.ncbi.nlm.nih.gov/pubchem/Bioassay'

HTTPS mirror of the PubChem BioAssay FTP tree.

PubChemBulk(base_url=PUBCHEM_BIOASSAY_FTP, **kwargs)

Bases: BaseAPI

Client for the PubChem BioAssay bulk activity dump. Download once the assay info for all AIDs into a single Parquet file, then load individual AIDs on demand.

Initialize the bulk client.

PARAMETER	DESCRIPTION
base_url	the HTTPS for the PubChem BioAssay FTP root. TYPE: str DEFAULT: PUBCHEM_BIOASSAY_FTP

download(file_path, path=BIOACTIVITIES_PATH, force=False, verify_md5=False)

Stream the bulk file to disk, skipping the download if already current.

PARAMETER	DESCRIPTION
file_path	Local path for the .tsv.gz file. TYPE: str
path	Remote path under the bioassay root. TYPE: str DEFAULT: BIOACTIVITIES_PATH
force	Re-download even if the local file appears current. TYPE: bool DEFAULT: False
verify_md5	If True, fetch checksum.md5 and verify the file's MD5 after download (slower, but catches truncation). TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
str	The path to the downloaded file.

RAISES	DESCRIPTION
TransientAPIError	If the download stream fails.

load_assay(aid, parquet_path)

Build an AssayInfo for one AID from the Parquet file. Note that the AssayInfo returned does not have the CIDs properties like SMILES etc.

PARAMETER	DESCRIPTION
aid	PubChem Assay ID. TYPE: int
parquet_path	Path to the Parquet created from the bioactivities tsv.gz file. TYPE: str

RETURNS	DESCRIPTION
AssayInfo	An AssayInfo. records is empty if the AID is absent; per-row PMIDs
AssayInfo	ride along on each record, and AssayInfo.pubmed_id is derived.

remote_signature(path=BIOACTIVITIES_PATH)

Return a change signature for the remote file via a HEAD request. so that we can skip downloading if the file is unchanged.

PARAMETER	DESCRIPTION
path	File path under the bioassay root. TYPE: str DEFAULT: BIOACTIVITIES_PATH

RETURNS	DESCRIPTION
str	A signature string that changes if the remote file changes

to_parquet(tsv_gz_path, parquet_path)

Convert the downloaded bioactivities tsv file to a single Parquet file.

PARAMETER	DESCRIPTION
tsv_gz_path	Path to the downloaded .tsv.gz. TYPE: str
parquet_path	Path to write the Parquet file to. Will be overwritten if it exists. TYPE: str

RETURNS	DESCRIPTION
str	The parquet_path written.