qsprpred.data.sources.papyrus package
Submodules
qsprpred.data.sources.papyrus.papyrus_class module
Creating dataset from Papyrus database.
- class qsprpred.data.sources.papyrus.papyrus_class.Papyrus(data_dir: str = '/home/runner/.Papyrus', version: str = 'latest', descriptors: str | list[str] | None = None, stereo: bool = False, disk_margin: float = 0.01, plus_only: bool = True)[source]
Bases:
DataSourceCreate new instance of Papyrus dataset. See
papyrus_filterandPapyrus.downloadandPapyrus.getDatafor more details.- Variables:
DEFAULT_DIR (str) – default directory for Papyrus database and the extracted data
dataDir (str) – storage directory for Papyrus database and the extracted data
_papyrusDir (str) – directory where the Papyrus database is located, os.path.join(dataDir, “papyrus”)
version (list) – Papyrus database version
descriptors (list, str, None) – descriptors to download if not already present
stereo (bool) – use version with stereochemistry
nostereo (bool) – use version without stereochemistry
plusplus (bool) – use plusplus version
diskMargin (float) – the disk space margin to leave free
Create new instance of Papyrus dataset. See
papyrus_filterandPapyrus.downloadandPapyrus.getDatafor more details.- Parameters:
data_dir (str) – storage directory for Papyrus database and the extracted data
version (str) – Papyrus database version
descriptors (str, list, None) – descriptors to download if not already present (set to ‘all’ for all descriptors, otherwise a list of descriptor names, see https://github.com/OlivierBeq/Papyrus-scripts)
stereo (str) – include stereochemistry in the database
disk_margin (float) – the disk space margin to leave free
plus_only (bool) – use only plusplus version, only high quality data
- DEFAULT_DIR = '/home/runner/.Papyrus'
- download()[source]
Download Papyrus database with the required information.
Only newly requested data is downloaded. Remove the files if you want to reload the data completely.
- getData(name: str | None = None, acc_keys: list[str] | None = None, quality: str = 'high', activity_types: list[str] | str = 'all', output_dir: str | None = None, drop_duplicates: bool = False, chunk_size: int = 100000.0, use_existing: bool = True, **kwargs) MoleculeTable[source]
Get the data from the Papyrus database as a
DataSetTSVinstance.- Parameters:
acc_keys (list) – protein accession keys
quality (str) – desired minimum quality of the dataset
activity_types (list, str) – list of activity types to include in the dataset
output_dir (str) – path to the directory where the data set will be stored
name (str) – name of the dataset (the prefix of the generated .tsv file)
drop_duplicates (bool) – remove duplicates after filtering
chunk_size (int) – data is read in chunks of this size (see
papyrus_filter)use_existing (bool) – use existing if available
kwargs – additional keyword arguments passed to
MoleculeTable.fromTableFile
- Returns:
the filtered data set
- Return type:
- getDataSet(target_props: list[TargetSpec | dict], name: str | None = None, **kwargs) QSPRTable
Get the dataset from the source as a
QSPRDataSet. Essentially just creates aQSPRDataSetfrom theMoleculeTableobtained from thegetDatamethod.- Parameters:
target_props (list[TargetProperty | dict]) – The target properties to add.
name (str, optional) – The name of the dataset.
kwargs – Additional keyword arguments to pass to the
getDatamethod and theQSPRDataSetconstructor.
- Returns:
The dataset.
- Return type:
qsprpred.data.sources.papyrus.papyrus_filter module
Filter Papyrus data.
- qsprpred.data.sources.papyrus.papyrus_filter.papyrus_filter(version: str, acc_key: list[str], quality: str, out_dir: str, activity_types: list[str] | str = 'all', prefix: str | None = None, drop_duplicates: bool = True, chunk_size: int = 100000.0, use_existing: bool = True, stereo: bool = False, plusplus: bool = False, papyrus_dir: str | None = None)[source]
Filters the downloaded Papyrus dataset for quality and accession key (UniProt) and outputs a .tsv file of all compounds fulfilling these requirements.
- Parameters:
version (str) – Papyrus database version
acc_key (list) – list of UniProt accession keys
quality (str) – str with minimum quality of dataset to keep
out_dir (str) – path to the location of Papyrus data
prefix (str) – prefix for the output file
drop_duplicates (bool) – boolean to drop duplicates from the final dataset
chunk_size (int) – integer of chunks to process one at the time
use_existing (bool) – if
True, use existing data if availablestereo (bool) – if
True, read stereochemistry data (if available)plusplus (bool) – if
True, read high quality Papyrus++ data (if available)papyrus_dir – path to the location of Papyrus database
- Returns:
filtered dataset outfile (str): path to the output file
- Return type:
dataset (pd.DataFrame)
qsprpred.data.sources.papyrus.tests module
- class qsprpred.data.sources.papyrus.tests.PapyrusSourceTest(methodName='runTest')[source]
Bases:
DataSetsPathMixIn,QSPRTestCaseCreate an instance of the class that will use the named test method when executed. Raises a ValueError if the instance does not have a method with the specified name.
- classmethod addClassCleanup(function, /, *args, **kwargs)
Same as addCleanup, except the cleanup items are called even if setUpClass fails (unlike tearDownClass).
- addCleanup(function, /, *args, **kwargs)
Add a function, with arguments, to be called when the test is completed. Functions added are called on a LIFO basis and are called after tearDown on test failure or success.
Cleanup items are called even if setUp fails (unlike tearDown).
- addTypeEqualityFunc(typeobj, function)
Add a type specific assertEqual style function to compare a type.
This method is for use by TestCase subclasses that need to register their own type equality functions to provide nicer error messages.
- Parameters:
typeobj – The data type to call this function on when both values are of the same type in assertEqual().
function – The callable taking two arguments and an optional msg= argument that raises self.failureException with a useful error message when the two arguments are not equal.
- assertAlmostEqual(first, second, places=None, msg=None, delta=None)
Fail if the two objects are unequal as determined by their difference rounded to the given number of decimal places (default 7) and comparing to zero, or by comparing that the difference between the two objects is more than the given delta.
Note that decimal places (from zero) are usually not the same as significant digits (measured from the most significant digit).
If the two objects compare equal then they will automatically compare almost equal.
- assertCountEqual(first, second, msg=None)
Asserts that two iterables have the same elements, the same number of times, without regard to order.
- self.assertEqual(Counter(list(first)),
Counter(list(second)))
- Example:
[0, 1, 1] and [1, 0, 1] compare equal.
[0, 0, 1] and [0, 1] compare unequal.
- assertDictEqual(d1, d2, msg=None)
- assertEndsWith(s, suffix, msg=None)
- assertEqual(first, second, msg=None)
Fail if the two objects are unequal as determined by the ‘==’ operator.
- assertFalse(expr, msg=None)
Check that the expression is false.
- assertGreater(a, b, msg=None)
Just like self.assertTrue(a > b), but with a nicer default message.
- assertGreaterEqual(a, b, msg=None)
Just like self.assertTrue(a >= b), but with a nicer default message.
- assertHasAttr(obj, name, msg=None)
- assertIn(member, container, msg=None)
Just like self.assertTrue(a in b), but with a nicer default message.
- assertIs(expr1, expr2, msg=None)
Just like self.assertTrue(a is b), but with a nicer default message.
- assertIsInstance(obj, cls, msg=None)
Same as self.assertTrue(isinstance(obj, cls)), with a nicer default message.
- assertIsNone(obj, msg=None)
Same as self.assertTrue(obj is None), with a nicer default message.
- assertIsNot(expr1, expr2, msg=None)
Just like self.assertTrue(a is not b), but with a nicer default message.
- assertIsNotNone(obj, msg=None)
Included for symmetry with assertIsNone.
- assertIsSubclass(cls, superclass, msg=None)
- assertLess(a, b, msg=None)
Just like self.assertTrue(a < b), but with a nicer default message.
- assertLessEqual(a, b, msg=None)
Just like self.assertTrue(a <= b), but with a nicer default message.
- assertListEqual(list1, list2, msg=None)
A list-specific equality assertion.
- Parameters:
list1 – The first list to compare.
list2 – The second list to compare.
msg – Optional message to use on failure instead of a list of differences.
- assertLogs(logger=None, level=None)
Fail unless a log message of level level or higher is emitted on logger_name or its children. If omitted, level defaults to INFO and logger defaults to the root logger.
This method must be used as a context manager, and will yield a recording object with two attributes:
outputandrecords. At the end of the context manager, theoutputattribute will be a list of the matching formatted log messages and therecordsattribute will be a list of the corresponding LogRecord objects.Example:
with self.assertLogs('foo', level='INFO') as cm: logging.getLogger('foo').info('first message') logging.getLogger('foo.bar').error('second message') self.assertEqual(cm.output, ['INFO:foo:first message', 'ERROR:foo.bar:second message'])
- assertMultiLineEqual(first, second, msg=None)
Assert that two multi-line strings are equal.
- assertNoLogs(logger=None, level=None)
Fail unless no log messages of level level or higher are emitted on logger_name or its children.
This method must be used as a context manager.
- assertNotAlmostEqual(first, second, places=None, msg=None, delta=None)
Fail if the two objects are equal as determined by their difference rounded to the given number of decimal places (default 7) and comparing to zero, or by comparing that the difference between the two objects is less than the given delta.
Note that decimal places (from zero) are usually not the same as significant digits (measured from the most significant digit).
Objects that are equal automatically fail.
- assertNotEndsWith(s, suffix, msg=None)
- assertNotEqual(first, second, msg=None)
Fail if the two objects are equal as determined by the ‘!=’ operator.
- assertNotHasAttr(obj, name, msg=None)
- assertNotIn(member, container, msg=None)
Just like self.assertTrue(a not in b), but with a nicer default message.
- assertNotIsInstance(obj, cls, msg=None)
Included for symmetry with assertIsInstance.
- assertNotIsSubclass(cls, superclass, msg=None)
- assertNotRegex(text, unexpected_regex, msg=None)
Fail the test if the text matches the regular expression.
- assertNotStartsWith(s, prefix, msg=None)
- assertRaises(expected_exception, *args, **kwargs)
Fail unless an exception of class expected_exception is raised by the callable when invoked with specified positional and keyword arguments. If a different type of exception is raised, it will not be caught, and the test case will be deemed to have suffered an error, exactly as for an unexpected exception.
If called with the callable and arguments omitted, will return a context object used like this:
with self.assertRaises(SomeException): do_something()
An optional keyword argument ‘msg’ can be provided when assertRaises is used as a context object.
The context manager keeps a reference to the exception as the ‘exception’ attribute. This allows you to inspect the exception after the assertion:
with self.assertRaises(SomeException) as cm: do_something() the_exception = cm.exception self.assertEqual(the_exception.error_code, 3)
- assertRaisesRegex(expected_exception, expected_regex, *args, **kwargs)
Asserts that the message in a raised exception matches a regex.
- Parameters:
expected_exception – Exception class expected to be raised.
expected_regex – Regex (re.Pattern object or string) expected to be found in error message.
args – Function to be called and extra positional args.
kwargs – Extra kwargs.
msg – Optional message used in case of failure. Can only be used when assertRaisesRegex is used as a context manager.
- assertRegex(text, expected_regex, msg=None)
Fail the test unless the text matches the regular expression.
- assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
An equality assertion for ordered sequences (like lists and tuples).
For the purposes of this function, a valid ordered sequence type is one which can be indexed, has a length, and has an equality operator.
- Parameters:
seq1 – The first sequence to compare.
seq2 – The second sequence to compare.
seq_type – The expected datatype of the sequences, or None if no datatype should be enforced.
msg – Optional message to use on failure instead of a list of differences.
- assertSetEqual(set1, set2, msg=None)
A set-specific equality assertion.
- Parameters:
set1 – The first set to compare.
set2 – The second set to compare.
msg – Optional message to use on failure instead of a list of differences.
assertSetEqual uses ducktyping to support different types of sets, and is optimized for sets specifically (parameters must support a difference method).
- assertStartsWith(s, prefix, msg=None)
- assertTrue(expr, msg=None)
Check that the expression is true.
- assertTupleEqual(tuple1, tuple2, msg=None)
A tuple-specific equality assertion.
- Parameters:
tuple1 – The first tuple to compare.
tuple2 – The second tuple to compare.
msg – Optional message to use on failure instead of a list of differences.
- assertWarns(expected_warning, *args, **kwargs)
Fail unless a warning of class warnClass is triggered by the callable when invoked with specified positional and keyword arguments. If a different type of warning is triggered, it will not be handled: depending on the other warning filtering rules in effect, it might be silenced, printed out, or raised as an exception.
If called with the callable and arguments omitted, will return a context object used like this:
with self.assertWarns(SomeWarning): do_something()
An optional keyword argument ‘msg’ can be provided when assertWarns is used as a context object.
The context manager keeps a reference to the first matching warning as the ‘warning’ attribute; similarly, the ‘filename’ and ‘lineno’ attributes give you information about the line of Python code from which the warning was triggered. This allows you to inspect the warning after the assertion:
with self.assertWarns(SomeWarning) as cm: do_something() the_warning = cm.warning self.assertEqual(the_warning.some_attribute, 147)
- assertWarnsRegex(expected_warning, expected_regex, *args, **kwargs)
Asserts that the message in a triggered warning matches a regexp. Basic functioning is similar to assertWarns() with the addition that only warnings whose messages also match the regular expression are considered successful matches.
- Parameters:
expected_warning – Warning class expected to be triggered.
expected_regex – Regex (re.Pattern object or string) expected to be found in error message.
args – Function to be called and extra positional args.
kwargs – Extra kwargs.
msg – Optional message used in case of failure. Can only be used when assertWarnsRegex is used as a context manager.
- clearGenerated()
Remove the directories that are used for testing.
- countTestCases()
- createLargeMultitaskDataSet(name='QSPRDataset_multi_test', target_props=[{'name': 'HBD', 'task': TargetTasks.MULTICLASS, 'th': [-1, 1, 2, 100]}, {'name': 'CL', 'task': TargetTasks.REGRESSION}], random_state=42)
Create a large dataset for testing purposes.
- Parameters:
- Returns:
a
QSPRDataSetobject- Return type:
- createLargeTestDataSet(name='QSPRDataset_test_large', target_props=[{'name': 'CL', 'task': TargetTasks.REGRESSION}], random_state=42, n_jobs=1, chunk_size=None, drop_empty_target_props=True)
Create a large dataset for testing purposes.
- Parameters:
- Returns:
a
QSPRDataSetobject- Return type:
- createSmallTestDataSet(name='QSPRDataset_test_small', target_props=[{'name': 'CL', 'task': TargetTasks.REGRESSION}], random_state=42, drop_empty_target_props=True)
Create a small dataset for testing purposes.
- Parameters:
- Returns:
a
QSPRDataSetobject- Return type:
- createTestDataSetFromFrame(df, name='QSPRDataset_test', target_props=[{'name': 'CL', 'task': TargetTasks.REGRESSION}], random_state=None, n_jobs=1, chunk_size=None, drop_empty_target_props=True)
Create a dataset for testing purposes from the given data frame.
- Parameters:
df (pd.DataFrame) – data frame containing the dataset
name (str) – name of the dataset
target_props (List of dicts or TargetProperty) – list of target properties
random_state (int) – random state to use for splitting and shuffling
prep (dict) – dictionary containing preparation settings
n_jobs (int) – number of jobs to use for parallel processing
chunk_size (int) – size of chunks to use per job in parallel processing
- Returns:
a
QSPRDataSetobject- Return type:
- debug()
Run the test without collecting errors in a TestResult
- defaultTestResult()
- classmethod doClassCleanups()
Execute all class cleanup functions. Normally called for you after tearDownClass.
- doCleanups()
Execute all cleanup functions. Normally called for you after tearDown.
- classmethod enterClassContext(cm)
Same as enterContext, but class-wide.
- enterContext(cm)
Enters the supplied context manager.
If successful, also adds its __exit__ method as a cleanup function and returns the result of the __enter__ method.
- fail(msg=None)
Fail immediately, with the given message.
- failureException
alias of
AssertionError
- classmethod getAllDescriptorSets()
Return a list of (ideally) all available descriptor sets. For now they need to be added manually to the list below.
TODO: would be nice to create the list automatically by implementing a descriptor set registry that would hold all installed descriptor sets.
- getBigDF()
Get a large data frame for testing purposes.
- Returns:
a
pandas.DataFramecontaining the dataset- Return type:
pd.DataFrame
- classmethod getDataPrepGrid()
Return a list of many possible combinations of descriptor calculators, splits, feature standardizers, feature filters and data filters. Again, this is not exhaustive, but should cover a lot of cases.
- Returns:
a generator that yields tuples of all possible combinations as stated above, each tuple is defined as: (descriptor_calculator, split, feature_standardizer, feature_filters, data_filters)
- Return type:
grid
- classmethod getDefaultCalculatorCombo()
Makes a list of default descriptor calculators that can be used in tests.
It creates a calculator with only morgan fingerprints and rdkit descriptors, but also one with them both to test behaviour with multiple descriptor sets. Override this method if you want to test with other descriptor sets and calculator combinations.
- static getDefaultPrep(add_imputer=None)
Return a dictionary with default preparation settings.
- classmethod getPrepCombos()
Return a list of all possible preparation combinations as generated by
getDataPrepGridas well as their names. The generated list can be used to parameterize tests with the given named combinations.
- getSmallDF()
Get a small data frame for testing purposes.
- Returns:
a
pandas.DataFramecontaining the dataset- Return type:
pd.DataFrame
- getStorage(df, name, n_jobs=1, chunk_size=None)
- id()
- longMessage = True
- maxDiff = 640
- run(result=None)
- classmethod setUpClass()
Hook method for setting up class fixture before running tests in the class.
- setUpPaths()
Create the directories that are used for testing.
- shortDescription()
Returns a one-line description of the test, or None if no description has been provided.
The default implementation of this method returns the first line of the specified test method’s docstring.
- skipTest(reason)
Skip this test.
- subTest(msg=<object object>, **params)
Return a context manager that will return the enclosed block of code in a subtest identified by the optional message and keyword parameters. A failure in the subtest marks the test case as failed but resumes execution at the end of the enclosed block, allowing further test code to be executed.
- tearDown()
Remove all files and directories that are used for testing.
- classmethod tearDownClass()
Hook method for deconstructing the class fixture after running all tests in the class.