Skip to content

Cluster wrapper

Implementation for a wrapper for MMseqs2 clustering

AID_2_CIDS_FILE = '../aid_2_cids.csv'

AID_2_TARGET_FILE = '../aid_2_target.csv'

BATCH_SIZE = 400

FASTA_FILE = '../protein_sequences.fasta'

FINAL_FULL_PARQUET = 'final_ligand_cluster.parquet'

GENE_ID2PDB_ID_FILE = 'gene_id2pdb_id.csv'

HDF5_FILE = '../PubChem_data_edited.hdf5'

LIGAND_CLUST_FILE = 'ligand_cluster.parquet'

PROTEIN_CLUSTER_FILE = 'protein_cluster_4_10.parquet'

PubChem_protein_ids = fasta_parser(FASTA_FILE)

RAW_ID_MAPPING_FILE = 'id_mapping.csv'

VALID_PDBS_FILE = 'valid_gene_id2pdb_id.csv'

aid_2_cid_df = pd.DataFrame(aid_2_cid.items(), columns=['AID', 'CIDs'])

aid_2cids_df = pd.read_csv(AID_2_CIDS_FILE)

aid_2target_df = pd.read_csv(AID_2_TARGET_FILE)

aids_2_target_df = pd.DataFrame(aids_2_target.items(), columns=['AID', 'Gene_id'])

batch = [(protein_id.strip()) for protein_id in (unique_protein_ids[i:(i + BATCH_SIZE)])]

batch_results = map_genid_to_pdb(batch)

data_no = []

duplicates = final_data[final_data.duplicated(subset=['Compound ID'])]

filtered_data = pd.read_csv('./filtered_version_6_9_1_2025.csv')

final_data = final_parq.convert_to_pandas()

final_parq = ParquetParser(FINAL_FULL_PARQUET, ligand_cluster_schema)

flattened_result = {'Gene ID': result['Gene ID'], 'UniprotID': result['UniprotID'], 'Organism': result['Organism'], 'Protein Name': result['Protein Name'], 'Gene Name': result['Gene Name'], 'PDB IDs': ','.join(result['PDB IDs']), 'AlphaFold IDs': ','.join(result['AlphaFold IDs'])}

gene2pdb = mapped_geneids_df[['Gene ID', 'PDB IDs']]

lig_data = lig_parq.convert_to_pandas()

lig_parq = ParquetParser(LIGAND_CLUST_FILE, ligand_cluster_schema)

ligand_cluster_schema = pa.schema([('Protein Cluster number', pa.int64()), ('PDB/Gene ID', pa.string()), ('Compound ID', pa.string()), ('Smiles', pa.string()), ('Ligand Cluster number', pa.int64())])

mapped_geneids = []

mapped_geneids_df = pd.DataFrame(mapped_geneids)

member_compound = prot_data[prot_data['Represenatives'] == rep_id]['member_compound'].to_list()[0]

missing_represenatives = [rep_id for rep_id in represenatives_id if rep_id not in prot_data['Represenatives'].to_list()]

pdb_ids = final_data['PDB/Gene ID']

prot_data = prot_parq.convert_to_pandas()

prot_parq = ParquetParser('../protein_clustered_data.parquet', schema)

prot_with_compounds = ParquetParser('new_protein_cluster_4_10.parquet', schema)

represenatives_id = filtered_data['From']

schema = pa.schema([('Cluster number', pa.int64()), ('Represenatives', pa.string()), ('members', pa.string()), ('member_compound', pa.list_(pa.string()))])

sum = 0

unique_data = final_data.drop_duplicates(subset=['Compound ID'])

unique_parq = ParquetParser('unique_final_ligand_cluster.parquet', schema)

unique_protein_ids = list(set(PubChem_protein_ids))

valid_gene2pdb = pd.read_csv(VALID_PDBS_FILE)

add_compounds(original_file, data_source, overwrite=True)

Add the compounds to the parquet file.

original_file : ParquetParser ParquetParser object of the original file. data_source : tuple Tuple containing the dictionaries of the aids and cids. overwrite : bool Whether to overwrite the original file. Default is True. if false, the new file will be created and names as original_file + "_new".

add_smiles_cluster(old_parquet, similarity_cutoff, new_parquet)

Clustering smiles and Adding the smiles cluster to the parquet file.

old_parquet : ParquetParser ParquetParser object of the old file. similarity_cutoff : float The cutoff value for the tanimoto clustering similarity. new_parquet : ParquetParser ParquetParser object of the new file.

clean_cluster_files(cluster_filepath, valid_csvpath)

Clean the cluster file by removing the clusters with no raw no members and the clusters with invalid PDB ids.

cluster_filepath : str Path to the parsed mmseq cluster file. valid_csvpath : str Path to the csv mapping the gene ids to the valid PDB ids. Returns: None

create_fasta_file(hdf5_file, fasta_file)

Create a FASTA file from the HDF5 file.

hdf5_file : str Path to the HDF5 file. fasta_file : str Path to the new FASTA file.

fasta_parser(fasta, delimiter=None)

Parse FASTA files to get the protein ids

fasta : str |TextIO Path to the FASTA file. delimiter : str| None Delimiter to split the protein id from the FASTA header. Default is None. Returns: list List of protein ids.

hdf5_raw_file_parser(hdf5_file)

Parse the HDF5 file to get the protein ids and the ligand ids.

hdf5_file : str Path to the HDF5 file.

tuple Tuple of dictionaries containing the protein ids and the ligand ids.

make_ligand_cluster_file(old_parquet, new_file_name, aid_2_cids_df, aid_2_target_df, hdf5_file, initial_data=None)

Make a new Parquet file for the ligand clustering.

old_parquet : ParquetParser ParquetParser object of the old file. new_file_name : ParquetParser ParquetParser object of the new file. aid_2_cids_df : pd.DataFrame DataFrame containing the aids and cids. aid_2_target_df : pd.DataFrame DataFrame containing the aids and gene ids. hdf5_file : str Path to the HDF5 file. intial_data : list List of initial data to be added to the new file. Default is None.

make_protein_cluster_file(new_file, cluster_csv)

Make a parquet cluster file with the new lines.

new_file : ParquetParser Parquet object. if the file not found, it will be created. cluster_csv : str Path to the cluster file.

pdb_validations(csvfilenames)

Validate the PDB files for the proteins in the CSV file.

csvfilenames : str Path to the CSV file containing the protein ids.