drugex.parallel package

Submodules

drugex.parallel.collectors module

collectors

Created by: Martin Sicho On: 14.07.22, 10:58

class drugex.parallel.collectors.ListAppend

Bases: ListExtend

class drugex.parallel.collectors.ListExtend

Bases: ListCollector

getList()

drugex.parallel.evaluator module

evaluator

Created by: Martin Sicho On: 29.05.22, 18:12

class drugex.parallel.evaluator.ParallelSupplierEvaluator(supplier_class, n_proc=None, chunk_size=1000, chunks=None, args=None, kwargs=None)

Bases: ParallelProcessor

Class implementing parallel evaluation of MolSupplier instances on input data (see ParallelSupplierEvaluator.apply).

apply(data, collector, error=None, desc_string=None)

Apply the ParallelSupplierEvaluator.run() across a Pool of workers.

Parameters:

• data – input data to divide into chunks and process in parallel

• collector – a ResultCollector that receives results of parallel processes

• error – a callable to handle errors during evaluation of each parallel supplier

• desc_string – progress bar description string

Returns:

None

error(data)

Catch and log an error occurring in the parallel process.

Raises:

ParallelException –

Parameters:

data – error data

initSupplier(supplier_class, chunk)

Initialize a MolSupplier instance on the given chung of data. :param supplier_class: MolSupplier to initialize. :param chunk: Data chunk.

Returns:

initialized MolSupplier

run(chunk, error)

Initialize and start evaluation of the MolSupplier instance on the given chunk of data.

Parameters:

chunk – Current chunk of data.

Returns:

result of the MolSupplier.toList() method for the given chunk.

drugex.parallel.interfaces module

interfaces

Created by: Martin Sicho On: 29.05.22, 18:09

class drugex.parallel.interfaces.ListCollector

Bases: ResultCollector, ABC

abstract getList()

exception drugex.parallel.interfaces.ParallelException

Bases: Exception

class drugex.parallel.interfaces.ParallelProcessor(n_proc=None, chunk_size=1000, chunks=None)

Bases: ABC

Simple interface to define parameters for parallel processing of data.

abstract apply(data, collector)

Apply the processor to the given data.

Parameters:

• data – input data (format depends on the implementation)

• collector – a ResultCollector that is used to collect data produced from each process.

Returns:

None.

getChunkSize(data)

Determine the chunk size from data.

Parameters:

data – input data (needs to have len)

Returns:

int representing the size of one chunk sent to the parallel process

class drugex.parallel.interfaces.ResultCollector

Bases: ABC

drugex.parallel.tests module

tests

Created by: Martin Sicho On: 12.04.22, 15:07

class drugex.parallel.tests.TestFileParsers(methodName='runTest')

Bases: TestCase

static getTestFile(name)

test_parallel()

test_parallel_with_suppliers()

Module contents

__init__.py

Created by: Martin Sicho On: 29.05.22, 18:09