Functional pipe API

A left-to-right pipe for the whole QSAR workflow, in the spirit of R’s %>%. Chain stages with >> (or |); > is rejected, because Python parses a > b > c as the chained comparison (a > b) and (b > c) and would silently discard your data.

Note

Every example on this page is executed by the test suite. They all start from the shared demo dataset described in User guide: DEMO_SMILES (24 compounds in four substituent series), DEMO_Y (synthetic pIC50 values) and demo_mols.

Two domains, one notation

A pipeline moves through two value types. MoleculeSet carries molecules and labels; FeatureSet carries a matrix and labels. featurize() is the hinge between them.

molecules()          -> MoleculeSet     chemistry
  >> desalt()        -> MoleculeSet
  >> drop_invalid()  -> MoleculeSet
  >> featurize(...)  -> FeatureSet      <-- the transition
  >> scale()         -> FeatureSet      linear algebra
  >> fit(...)        -> fitted model    terminal

Steps check what they receive, so a mis-ordered chain names the offending stage rather than failing somewhere deeper:

>>> from qsarkit.functional import desalt, fingerprint, molecules
>>> molecules(["CCO"]) >> fingerprint(n_bits=16) >> desalt()
Traceback (most recent call last):
    ...
TypeError: Step 'desalt' works on molecules, but it received a FeatureSet...

Starting a pipeline

molecules() accepts RDKit molecules, SMILES, InChI, or any mixture:

>>> from rdkit import Chem
>>> from qsarkit.functional import molecules
>>> ms = molecules([
...     Chem.MolFromSmiles("c1ccccc1"),
...     "CCN",
...     "InChI=1S/C2H6O/c1-2-3/h3H,2H2,1H3",
... ])
>>> ms.smiles
['c1ccccc1', 'CCN', 'CCO']

Anything unparseable becomes None rather than raising, so it stays aligned with y until you decide what to do with it:

>>> from qsarkit.functional import drop_invalid
>>> ms = molecules(["CCO", "not-a-molecule", "CCN"], [1.0, 2.0, 3.0])
>>> [m is None for m in ms.mols]
[False, True, False]
>>> mols, y = ms >> drop_invalid()
>>> y.tolist()
[1.0, 3.0]

That index alignment is the whole point of the API. It is the bookkeeping hand-written curation scripts get subtly wrong: a molecule dropped without its label shifts every subsequent activity by one, and the model still trains.

Curating

>>> from qsarkit.functional import remove_duplicates, standardize
>>> ms = (
...     molecules(DEMO_SMILES, DEMO_Y)
...     >> standardize()
...     >> drop_invalid()
...     >> remove_duplicates(agg="mean")
... )
>>> len(ms)
24

Each set carries its own provenance, which is what a QMRF report needs under OECD Principle 2:

>>> ms.history
['molecules(n=24)', 'standardize()', 'drop_invalid()', "remove_duplicates(agg='mean')"]

Steps compose with each other, so a protocol is defined once and reused on train and test alike:

>>> curate = standardize() >> drop_invalid() >> remove_duplicates()
>>> train = molecules(DEMO_SMILES[:12], DEMO_Y[:12]) >> curate
>>> test = molecules(DEMO_SMILES[12:], DEMO_Y[12:]) >> curate
>>> len(train), len(test)
(12, 12)

Featurizing

featurize() takes any transformer with a transform(mols) method, so the pipe inherits the whole of qsarkit.representation without duplicating it:

>>> from qsarkit.functional import featurize
>>> from qsarkit.representation import MorganFingerprint
>>> fs = molecules(DEMO_SMILES, DEMO_Y) >> featurize(MorganFingerprint(n_bits=512))
>>> fs.shape
(24, 512)

fingerprint() and describe() are shorthands for the common cases:

>>> from qsarkit.functional import describe, fingerprint
>>> (molecules(DEMO_SMILES) >> fingerprint("maccs")).shape
(24, 167)
>>> fs = molecules(DEMO_SMILES) >> describe("lipinski")
>>> fs.feature_names[:3]
['MolWt', 'MolLogP', 'NumHDonors']

Modelling

>>> from qsarkit.functional import fit, scale, select_features, split
>>> train, test = (
...     molecules(DEMO_SMILES, DEMO_Y)
...     >> fingerprint(n_bits=512)
...     >> split("scaffold", test_size=0.25)
... )
>>> len(train) + len(test)
24

The default split is by scaffold, not at random. A random split of a QSAR dataset measures interpolation: public sets are dense with near-duplicate analogues, so random assignment scatters a congeneric series across both sides and scores the model on compounds whose close relatives it has memorized.

>>> model = train >> fit("rf", random_state=0)
>>> model.predict(test.X).shape
(6,)

fit() accepts any estimator, not just the built-in names:

>>> from sklearn.linear_model import Ridge
>>> model = train >> fit(Ridge(alpha=1.0))
>>> type(model).__name__
'Ridge'

cross_validate() and applicability_domain() end a pipe the same way:

>>> from qsarkit.functional import applicability_domain, cross_validate
>>> report = (
...     molecules(DEMO_SMILES, DEMO_Y)
...     >> fingerprint(n_bits=512)
...     >> cross_validate("rf", n_splits=3, random_state=0)
... )
>>> sorted(report)[:3]
['mae_cv', 'method', 'n_splits']
>>> domain = train >> applicability_domain("tanimoto", threshold=0.3)
>>> domain.predict(test.X).shape
(6,)

Warning

scale() and select_features() fit on whatever flows through them. Placing them before split() leaks the test set’s statistics — and, for selection, its labels — into the transform, and the held-out score comes out optimistic. Put them after the split, or inside a sklearn.pipeline.Pipeline given to cross_validate().

Drawing the pipeline

A pipeline is a graph, and drawing it is the quickest way to confirm the stages are in the order you meant:

>>> pipe = standardize() >> drop_invalid() >> fingerprint() >> scale() >> fit("rf")
>>> figure = pipe.plot()                    # Plotly, no extra dependency
>>> type(figure).__name__
'Figure'
>>> print(pipe.to_dot().splitlines()[0])    # Graphviz DOT source
digraph qsarkit_pipeline {

render() writes PNG, PDF or SVG, using Graphviz when it is installed and Plotly otherwise:

pipe.render("workflow.pdf")

Nodes are coloured by domain — molecules, features, terminal — so the point where the pipeline crosses from chemistry into a feature matrix is visible at a glance.

Writing your own steps

step() and feature_step() turn an ordinary function into a dual-mode pipe stage: called without data it defers, called with data it runs immediately.

>>> from qsarkit.functional import step
>>> @step
... def heaviest(mols, y=None, n=5):
...     order = sorted(range(len(mols)), key=lambda i: -mols[i].GetNumHeavyAtoms())
...     keep = order[:n]
...     return [mols[i] for i in keep], (None if y is None else y[keep])
>>> mols, y = molecules(DEMO_SMILES, DEMO_Y) >> heaviest(n=3)
>>> len(mols), len(y)
(3, 3)

Core types

class qsarkit.functional.MoleculeSet(mols, y=None, history=None)[source]

Bases: object

A set of molecules with optional labels, flowing through a pipe.

This is the value that moves left to right through a qsarkit.functional pipeline. It carries the molecules, the optional labels y kept index-aligned with them, and a provenance log recording what each step did.

Unpacks as (mols, y):

X, y = molecules(smiles, y) >> desalt() >> remove_duplicates()
Parameters:
Variables:

  • mols (list of Mol) – The molecules.

  • y (ndarray or None) – The labels.

  • history (list of str) – What each step did, in order.

Examples

>>> from qsarkit.functional import molecules
>>> ms = molecules(["CCO", "c1ccccc1"], [1.0, 2.0])
>>> len(ms)
2
>>> mols, y = ms
>>> len(mols), y.tolist()
(2, [1.0, 2.0])

References

mols: List[Any]
y: ndarray[tuple[Any, ...], dtype[Any]] | None
history: List[str]
iter_mols()[source]

Iterate the molecules (__iter__ is reserved for unpacking).

Return type:

Iterator[Any]

property smiles: List[str | None]

Canonical SMILES for each molecule (None where invalid).

to_frame()[source]

Render as a DataFrame with smiles and, if present, y.

Return type:

DataFrame

to_dot(rankdir='TB', include_input=True)[source]

Graphviz DOT source for this pipeline’s flowchart.

Parameters:

  • rankdir (str) – Layout direction.

  • include_input (bool) – Draw the input node.

Returns:

DOT source. See to_dot().

Return type:

str

plot(**kwargs)[source]

Plotly flowchart of this pipeline.

Parameters:

**kwargs (Any) – Passed to plot_pipeline().

Return type:

Any

render(path, **kwargs)[source]

Write this pipeline’s flowchart to a PNG, PDF or SVG file.

Parameters:
Returns:

The path written.

Return type:

str

replace(mols, y=None, note=None)[source]

Return a new set with different contents and an extended history.

Steps use this instead of mutating, so a pipeline never modifies the set handed to it.

Parameters:
Return type:

MoleculeSet

class qsarkit.functional.FeatureSet(X, y=None, mols=None, history=None, feature_names=None)[source]

Bases: object

A feature matrix with labels, flowing through a pipe.

What a MoleculeSet becomes once it has been featurized. It carries the matrix X, the labels y, the molecules the rows came from (so a downstream step can still reach the chemistry), and the same growing provenance log.

Unpacks as (X, y):

X, y = molecules(smiles, y) >> desalt() >> featurize(MorganFingerprint())
Parameters:
Variables:

  • X (ndarray) – The feature matrix.

  • y (ndarray or None) – The labels.

  • mols (list of Mol or None) – The molecules, still aligned with the rows.

  • history (list of str) – What each step did, in order.

  • feature_names (list of str or None) – Column names, where known.

Examples

>>> from qsarkit.functional import featurize, molecules
>>> from qsarkit.representation import MorganFingerprint
>>> fs = molecules(["CCO", "c1ccccc1"], [1.0, 2.0]) >> featurize(
...     MorganFingerprint(n_bits=64))
>>> fs.shape
(2, 64)
>>> X, y = fs
>>> X.shape, y.tolist()
((2, 64), [1.0, 2.0])
X: ndarray[tuple[Any, ...], dtype[Any]]
y: ndarray[tuple[Any, ...], dtype[Any]] | None
mols: List[Any] | None
history: List[str]
feature_names: List[str] | None
property shape: Tuple[int, ...]

Shape of the feature matrix.

to_frame()[source]

Render as a DataFrame, using feature_names where known.

Return type:

DataFrame

to_dot(rankdir='TB', include_input=True)[source]

Graphviz DOT source for this pipeline’s flowchart.

Parameters:

  • rankdir (str) – Layout direction.

  • include_input (bool) – Draw the input node.

Returns:

DOT source. See to_dot().

Return type:

str

plot(**kwargs)[source]

Plotly flowchart of this pipeline.

Parameters:

**kwargs (Any) – Passed to plot_pipeline().

Return type:

Any

render(path, **kwargs)[source]

Write this pipeline’s flowchart to a PNG, PDF or SVG file.

Parameters:
Returns:

The path written.

Return type:

str

replace(X, y=None, mols=None, note=None, feature_names=None)[source]

Return a new set with different contents and an extended history.

Parameters:
Return type:

FeatureSet

class qsarkit.functional.PipeStep(name, params=None)[source]

Bases: object

Base class for anything that can appear on the right of >>.

A pipe step holds a function plus the arguments it was configured with, and applies them when a value is piped in. Steps also compose with each other, so a pipeline can be built once and reused.

Three concrete kinds exist, distinguished by what they consume and produce:

Step

MoleculeSet -> MoleculeSet. Curation, filtering, anything that stays in the chemistry domain.

featurize and its shorthands

MoleculeSet -> FeatureSet. The transition into the modelling domain.

FeatureStep

FeatureSet -> FeatureSet. Scaling, selection, anything that reshapes the matrix.

A chain that mixes them is checked as it runs, and a mismatch names both the step and what it received.

Parameters:

  • name (str) – Display name, used in the provenance log.

  • params (Optional[Dict[str, Any]]) – Keyword arguments applied when the step runs.

name
params
to_dot(rankdir='TB', include_input=True)[source]

Graphviz DOT source for this pipeline’s flowchart.

Parameters:

  • rankdir (str) – Layout direction.

  • include_input (bool) – Draw the input node.

Returns:

DOT source. See to_dot().

Return type:

str

plot(**kwargs)[source]

Plotly flowchart of this pipeline.

Parameters:

**kwargs (Any) – Passed to plot_pipeline().

Return type:

Any

render(path, **kwargs)[source]

Write this pipeline’s flowchart to a PNG, PDF or SVG file.

Parameters:
Returns:

The path written.

Return type:

str

class qsarkit.functional.Step(func, name, params=None)[source]

Bases: PipeStep

One deferred molecule -> molecule operation in a pipe.

Holds a function plus the arguments it was configured with, and applies them when a MoleculeSet is piped in.

Parameters:

Examples

>>> from qsarkit.functional import desalt, drop_invalid, molecules
>>> curate = desalt() >> drop_invalid()      # reusable pipeline
>>> mols, y = molecules(["CCO.[Na+]"]) >> curate
>>> len(mols)
1
func
class qsarkit.functional.FeatureStep(func, name, params=None)[source]

Bases: PipeStep

One deferred features -> features operation in a pipe.

The FeatureSet counterpart of Step: scaling, feature selection, and anything else that reshapes the matrix while keeping y (and the originating molecules) aligned with it.

Parameters:

Examples

>>> from qsarkit.functional import featurize, molecules, scale
>>> from qsarkit.representation import PhysicochemicalDescriptors
>>> fs = (
...     molecules(["CCO", "c1ccccc1", "CCN"], [1.0, 2.0, 3.0])
...     >> featurize(PhysicochemicalDescriptors())
...     >> scale()
... )
>>> bool(abs(fs.X.mean()) < 1e-9)     # standardized to zero mean
True
func
qsarkit.functional.molecules(X, y=None, fmt='auto')[source]

Start a pipeline from RDKit molecules, SMILES or InChI.

The entry point of the functional API. Accepts rdkit.Chem.Mol objects, SMILES strings, InChI strings, or any mixture of the three. Strings that fail to parse become None rather than raising, so they stay aligned with y until you decide what to do with them – normally a drop_invalid() step, which removes the matching labels too.

Parameters:

  • X (Sequence[Any]) – Molecules, SMILES strings, or InChI strings.

  • y (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str], None]) – Labels, one per molecule.

  • fmt (Literal['auto', 'smiles', 'inchi', 'mol']) – How to read string entries. "auto" treats a string beginning with InChI= as InChI and anything else as SMILES, deciding per entry so mixed input works. Name the format explicitly when you would rather have malformed input fail than be silently reinterpreted.

Returns:

The set to pipe onward.

Return type:

MoleculeSet

Raises:

ValueError – If fmt is not one of the four accepted values, or if fmt="mol" and an entry is not an RDKit molecule.

Examples

From SMILES:

>>> from qsarkit.functional import desalt, drop_invalid, molecules
>>> mols, y = (
...     molecules(["CC(=O)Oc1ccccc1C(=O)[O-].[Na+]", "CCO"], [1.0, 2.0])
...     >> desalt()
...     >> drop_invalid()
... )
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'CC(=O)Oc1ccccc1C(=O)[O-]'

From InChI, recognised without being told:

>>> ms = molecules(["InChI=1S/C2H6O/c1-2-3/h3H,2H2,1H3"])
>>> Chem.MolToSmiles(ms.mols[0])
'CCO'

From RDKit molecules, or any mixture of the three:

>>> mixed = molecules([
...     Chem.MolFromSmiles("c1ccccc1"),
...     "CCN",
...     "InChI=1S/C2H6O/c1-2-3/h3H,2H2,1H3",
... ])
>>> [Chem.MolToSmiles(m) for m in mixed.mols]
['c1ccccc1', 'CCN', 'CCO']

Unparseable entries survive as None so nothing shifts out of alignment with y:

>>> ms = molecules(["CCO", "not-a-molecule"], [1.0, 2.0])
>>> [m is None for m in ms.mols]
[False, True]
>>> mols, y = ms >> drop_invalid()
>>> y.tolist()
[1.0]

References

qsarkit.functional.step(func)[source]

Turn an (X, y=None, **params) -> (X, y) function into a pipe step.

The decorated callable works two ways, which is what lets the same function serve both the pipe API and ordinary imperative code:

  • Called with no molecules — desalt(), balance(method="under") — it returns a deferred Step for use in a pipe.

  • Called with molecules — desalt(mols, y) — it runs immediately and returns (mols, y).

Parameters:

func (Callable[..., Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]]) – Implementation taking (mols, y, **params) and returning (mols, y).

Returns:

The dual-mode wrapper.

Return type:

Callable[..., Any]

Examples

>>> from qsarkit.functional import step
>>> @step
... def keep_first(mols, y=None, n=1):
...     return mols[:n], (None if y is None else y[:n])
>>> from qsarkit.functional import molecules
>>> mols, y = molecules(["CCO", "CCN", "CCC"]) >> keep_first(n=2)
>>> len(mols)
2
qsarkit.functional.feature_step(func)[source]

Turn an (X, y=None, mols=None, **params) -> (X, y, mols) function into a pipe step.

The FeatureSet counterpart of step(), and dual-mode in the same way:

  • Called with no data — scale(), select_features(k=10) — it returns a deferred FeatureStep for use in a pipe.

  • Called with a matrix — scale(X, y) — it runs immediately and returns (X, y, mols).

Parameters:

func (Callable[..., Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]]) – Implementation taking (X, y, mols, **params) and returning (X, y, mols).

Returns:

The dual-mode wrapper.

Return type:

Callable[..., Any]

Examples

>>> import numpy as np
>>> from qsarkit.functional import feature_step, featurize, molecules
>>> from qsarkit.representation import PhysicochemicalDescriptors
>>> @feature_step
... def first_columns(X, y=None, mols=None, n=2):
...     return X[:, :n], y, mols
>>> fs = (
...     molecules(["CCO", "c1ccccc1"], [1.0, 2.0])
...     >> featurize(PhysicochemicalDescriptors())
...     >> first_columns(n=3)
... )
>>> fs.shape
(2, 3)
qsarkit.functional.pipeline(*steps)[source]

Compose steps into one reusable pipeline.

Equivalent to chaining with >>, but easier to build programmatically from a list.

Parameters:

*steps (PipeStep) – Steps to run in order.

Returns:

A single step running all of them.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import desalt, drop_invalid, molecules, pipeline
>>> curate = pipeline(desalt(), drop_invalid())
>>> mols, y = molecules(["CCO.[Na+]", "not-a-molecule"]) >> curate
>>> len(mols)
1

Curation steps

qsarkit.functional.standardize(X, y=None, remove_salts=True, neutralize=True, normalize_tautomers=True, handle_stereochemistry='retain', normalize_hydrogens=True)[source]

Run the full standardization pipeline over the set.

Sanitizes, strips salts and solvates, neutralizes charges, canonicalizes tautomers, handles stereochemistry and normalizes hydrogens. Molecules that fail become None rather than raising, so one bad record cannot abort a pipeline; follow with drop_invalid() to remove them.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, standardize
>>> mols, _ = molecules(["CC(=O)[O-].[Na+]"]) >> standardize()
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'CC(=O)O'

References

qsarkit.functional.desalt(X, y=None)[source]

Keep only the largest organic fragment of each molecule.

Strips counter-ions, solvates and hydrates. Activity data is routinely reported for salt forms while the activity belongs to the parent, so this is usually the first curation step.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import desalt, molecules
>>> mols, _ = molecules(["CC(=O)[O-].[Na+]"]) >> desalt()
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'CC(=O)[O-]'

References

qsarkit.functional.neutralize(X, y=None)[source]

Neutralize charges where a neutral form exists.

Leaves permanent charges (quaternary ammonium, for instance) untouched.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, neutralize
>>> mols, _ = molecules(["CC(=O)[O-]"]) >> neutralize()
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'CC(=O)O'

References

qsarkit.functional.canonicalize_tautomers(X, y=None)[source]

Map each molecule to its canonical tautomer.

Without this, the same compound drawn in two tautomeric forms counts as two distinct structures, which silently defeats duplicate removal.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import canonicalize_tautomers, molecules
>>> mols, _ = molecules(["Oc1ccccn1"]) >> canonicalize_tautomers()
>>> mols[0] is not None
True

References

qsarkit.functional.deglycate(X, y=None, keep_originals=False)[source]

Remove sugar moieties, keeping the aglycone.

Natural-product datasets are full of glycosides whose activity belongs to the aglycone. With keep_originals=True the glycoside is kept alongside its aglycone (and its label duplicated), which is what you want when you are augmenting a training set rather than replacing entries.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import deglycate, molecules
>>> glucoside = "OC[C@H]1O[C@@H](Oc2ccccc2)[C@H](O)[C@@H](O)[C@@H]1O"
>>> mols, _ = molecules([glucoside]) >> deglycate()
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'c1ccccc1'
>>> mols, _ = molecules([glucoside]) >> deglycate(keep_originals=True)
>>> len(mols)
2

References

qsarkit.functional.remove_protecting_groups(X, y=None)[source]

Strip protecting groups, linkers, tags and click handles.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, remove_protecting_groups
>>> mols, _ = molecules(["CC(C)(C)OC(=O)NCc1ccccc1"]) >> remove_protecting_groups()
>>> from rdkit import Chem
>>> Chem.MolToSmiles(mols[0])
'NCc1ccccc1'

References

qsarkit.functional.drop_invalid(X, y=None)[source]

Drop None entries, and their labels with them.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import drop_invalid, molecules
>>> mols, y = molecules(["CCO", "!!bad!!"], [1.0, 2.0]) >> drop_invalid()
>>> len(mols), y.tolist()
(1, [1.0])
qsarkit.functional.remove_duplicates(X, y=None, on='inchikey', agg='mean', max_spread=None)[source]

Collapse duplicate structures, aggregating their labels.

Public activity data is full of the same compound measured several times. Dropping duplicates blindly throws away replicate information; averaging them without checking hides disagreement. max_spread lets you do both — average the consistent ones and discard the pairs that disagree by more than you are willing to accept.

Parameters:

  • X (List[Any])

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]])

  • on (Literal['inchikey', 'smiles', 'scaffold']) – What counts as “the same molecule”. InChIKey is the most robust; scaffold collapses whole Bemis-Murcko series and is a deliberately blunt instrument.

  • agg (Literal['mean', 'median', 'min', 'max', 'first']) – How to combine the labels of duplicates. Ignored when unlabelled.

  • max_spread (Optional[float]) – Discard duplicate groups whose labels span more than this. For log-scale activities, 1.0 (a ten-fold disagreement) is a common cutoff.

Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, remove_duplicates
>>> mols, y = (
...     molecules(["CCO", "CCO", "c1ccccc1"], [1.0, 3.0, 5.0])
...     >> remove_duplicates(agg="mean")
... )
>>> len(mols), sorted(y.tolist())
(2, [2.0, 5.0])

References

qsarkit.functional.balance(X, y=None, method='undersample', random_state=None, featurizer=None)[source]

Balance a classification set across its label values.

Parameters:

  • X (List[Any])

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]]) – Class labels. Required – balancing an unlabelled set is meaningless, so this raises rather than silently doing nothing.

  • method (Any) –

    "undersample" discards majority-class molecules and "oversample" duplicates minority-class ones, both by simple random choice.

    Alternatively, any imbalanced-learn sampler with fit_resample(X, y). It must be one that selects existing samples rather than synthesizing new ones – see the note below.

  • random_state (Optional[int]) – Seed, for reproducibility. Ignored when method is a sampler instance, which carries its own.

  • featurizer (Optional[Any]) – Transformer used to featurize the molecules for a sampler that needs a feature matrix (TomekLinks, EditedNearestNeighbours, NearMiss, …). Defaults to MorganFingerprint. Unused by the two built-in string methods, which work on indices alone.

Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Raises:

ValueError – If y is None, if method is an unrecognized string, or if a supplied sampler synthesizes samples instead of selecting them.

Notes

Why SMOTE cannot be used here. SMOTE and its relatives (ADASYN, BorderlineSMOTE) balance a dataset by interpolating new feature vectors between existing ones. In descriptor space that is a defensible trick; at the molecule stage it is not, because the interpolated vector corresponds to no molecule – there is nothing to put in the returned list. Such a sampler is therefore rejected with an explanation rather than silently producing rows whose structures are fabricated.

If you want SMOTE, apply it after featurization, where the objects being synthesized are honestly just vectors:

molecules(smiles, y) >> fingerprint() >> resample(SMOTE())

See resample().

Balance the training set only. Resampling the test set changes the class prior you are measuring against, so a balanced test score does not describe the deployment population. Put this step after split(), or apply it to the training half alone.

Examples

>>> from qsarkit.functional import balance, molecules
>>> mols, y = (
...     molecules(["CCO", "CCN", "CCC", "c1ccccc1"], [0, 0, 0, 1])
...     >> balance(random_state=0)
... )
>>> sorted(y.tolist())
[0, 1]

Oversampling keeps every majority-class molecule and repeats the minority ones:

>>> mols, y = (
...     molecules(["CCO", "CCN", "CCC", "c1ccccc1"], [0, 0, 0, 1])
...     >> balance("oversample", random_state=0)
... )
>>> sorted(y.tolist())
[0, 0, 0, 1, 1, 1]

An imbalanced-learn sampler that selects rather than synthesizes works directly:

>>> from imblearn.under_sampling import RandomUnderSampler
>>> mols, y = (
...     molecules(smiles, labels) >> balance(RandomUnderSampler())
... )

References

  • He, H. & Garcia, E. A. (2009). “Learning from Imbalanced Data.” IEEE Trans. Knowl. Data Eng., 21(9), 1263-1284. https://doi.org/10.1109/TKDE.2008.239

  • Chawla, N. V. et al. (2002). “SMOTE: Synthetic Minority Over-sampling Technique.” J. Artif. Intell. Res., 16, 321-357. https://doi.org/10.1613/jair.953

  • Lemaitre, G., Nogueira, F. & Aridas, C. K. (2017). “Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning.” J. Mach. Learn. Res., 18(17), 1-5. https://jmlr.org/papers/v18/16-365

qsarkit.functional.keep_if(X, y=None, predicate=None)[source]

Keep molecules satisfying a predicate.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import keep_if, molecules
>>> mols, _ = molecules(["CCO", "c1ccccc1"]) >> keep_if(
...     predicate=lambda m: m.GetNumAtoms() > 3
... )
>>> len(mols)
1
qsarkit.functional.drop_if(X, y=None, predicate=None)[source]

Drop molecules satisfying a predicate.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import drop_if, molecules
>>> mols, _ = molecules(["CCO", "c1ccccc1"]) >> drop_if(
...     predicate=lambda m: m.GetNumAtoms() > 3
... )
>>> len(mols)
1
qsarkit.functional.filter_by_property(X, y=None, mw=None, logp=None, heavy_atoms=None, rotatable_bonds=None)[source]

Keep molecules whose physicochemical properties fall in given ranges.

Each bound is an inclusive (low, high) tuple; None disables that filter.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import filter_by_property, molecules
>>> mols, _ = molecules(["CCO", "CCCCCCCCCCCCCCCCCC"]) >> filter_by_property(
...     mw=(0, 100)
... )
>>> len(mols)
1

References

qsarkit.functional.to_pactivity(X, y=None, unit='nM')[source]

Convert concentration labels to pActivity (-log10 molar).

QSAR models should be fitted on a log scale: potency spans orders of magnitude, and the activity-cliff and SALI thresholds throughout qsarkit are all expressed in log units.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, to_pactivity
>>> _, y = molecules(["CCO"], [1.0]) >> to_pactivity(unit="nM")
>>> round(float(y[0]), 2)
9.0
qsarkit.functional.sample(X, y=None, n=None, fraction=None, random_state=None)[source]

Take a random subset.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, sample
>>> mols, _ = molecules(["CCO", "CCN", "CCC"]) >> sample(n=2, random_state=0)
>>> len(mols)
2
qsarkit.functional.shuffle(X, y=None, random_state=None)[source]

Shuffle the set, keeping molecules and labels aligned.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from qsarkit.functional import molecules, shuffle
>>> mols, y = molecules(["CCO", "CCN"], [1.0, 2.0]) >> shuffle(random_state=0)
>>> len(mols)
2
qsarkit.functional.apply(X, y=None, func=None)[source]

Apply an arbitrary per-molecule function — the escape hatch.

Parameters:
Return type:

Tuple[List[Any], Optional[ndarray[tuple[Any, ...], dtype[Any]]]]

Examples

>>> from rdkit import Chem
>>> from qsarkit.functional import apply, molecules
>>> mols, _ = molecules(["CCO"]) >> apply(func=Chem.AddHs)
>>> mols[0].GetNumAtoms()
9

Representation steps

qsarkit.functional.featurize(transformer, keep_mols=True)[source]

Turn molecules into a feature matrix, crossing into the modelling half.

This is the hinge of the functional API. Everything before it works on molecules; everything after it works on a matrix.

Parameters:

  • transformer (Any) – Any transformer with transform(mols) – typically one from qsarkit.representation, but a scikit-learn Pipeline or FeatureUnion of them works too.

  • keep_mols (bool) – Keep the molecules alongside the matrix for later steps.

Returns:

A step producing a FeatureSet.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import featurize, molecules
>>> from qsarkit.representation import MorganFingerprint
>>> fs = molecules(["CCO", "c1ccccc1"], [1.0, 2.0]) >> featurize(
...     MorganFingerprint(n_bits=128))
>>> fs.shape
(2, 128)

Because it takes any transformer, combining representations needs no new syntax:

>>> from qsarkit.representation import FingerprintCombiner, MACCSKeysFingerprint
>>> combined = FingerprintCombiner([
...     ("morgan", MorganFingerprint(n_bits=128)),
...     ("maccs", MACCSKeysFingerprint()),
... ])
>>> (molecules(["CCO", "c1ccccc1"]) >> featurize(combined)).shape
(2, 295)

Invalid molecules are refused rather than silently producing junk rows, because a None here would break the alignment with y that the rest of the pipe maintains:

>>> molecules(["CCO", "not-a-molecule"]) >> featurize(MorganFingerprint())
Traceback (most recent call last):
    ...
ValueError: Cannot featurize: 1 of 2 entries are None...

References

qsarkit.functional.fingerprint(kind='morgan', keep_mols=True, **kwargs)[source]

Featurize with a named fingerprint, for the common case.

A shorthand for featurize(MorganFingerprint(...)) and friends, so a quick pipeline does not need a second import.

Parameters:

  • kind (str) – Which fingerprint to compute. "ecfp" is an alias for "morgan"; "fcfp" selects the feature-based variant.

  • keep_mols (bool) – Keep the molecules alongside the matrix, as for featurize().

  • **kwargs (Any) – Passed to the underlying transformer (n_bits, radius, …).

Returns:

A step producing a FeatureSet.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import fingerprint, molecules
>>> (molecules(["CCO", "c1ccccc1"]) >> fingerprint("morgan", n_bits=64)).shape
(2, 64)
>>> (molecules(["CCO"]) >> fingerprint("maccs")).shape
(1, 167)

References

qsarkit.functional.describe(kind='physicochemical', keep_mols=True, **kwargs)[source]

Featurize with a named descriptor block.

Parameters:

  • kind (str) – Which descriptor set to compute.

  • keep_mols (bool) – Keep the molecules alongside the matrix, as for featurize().

  • **kwargs (Any) – Passed to the underlying transformer.

Returns:

A step producing a FeatureSet.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import describe, molecules
>>> fs = molecules(["CCO", "c1ccccc1"]) >> describe("lipinski")
>>> fs.feature_names is not None
True

Descriptors are continuous and on wildly different scales (molecular weight in the hundreds, logP in single digits), so they almost always want a scale() step before a distance-based model:

>>> from qsarkit.functional import scale
>>> (molecules(["CCO", "c1ccccc1", "CCN"]) >> describe() >> scale()).shape
(3, 9)

References

Feature steps

qsarkit.functional.scale(X, y=None, mols=None, method='standard')[source]

Scale the feature matrix.

Parameters:

  • X (ndarray[tuple[Any, ...], dtype[Any]]) – Feature matrix.

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]]) – Labels, passed through untouched.

  • mols (Optional[List[Any]]) – Molecules, passed through untouched.

  • method (Literal['standard', 'minmax', 'robust', 'none']) – "standard" centres and scales to unit variance, "robust" uses the median and IQR (resistant to the outliers that descriptor blocks routinely contain), "minmax" maps onto [0, 1], and "none" is a no-op for parametrized pipelines.

Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Notes

Scaling inside a pipe like this fits on whatever data is flowing through it. That is correct for a single curated dataset, but if you are holding out a test set, scale after split() or inside a sklearn.pipeline.Pipeline given to cross_validate() – otherwise the test set’s statistics leak into the transform.

Examples

>>> from qsarkit.functional import describe, molecules, scale
>>> fs = molecules(["CCO", "c1ccccc1", "CCN"]) >> describe() >> scale("robust")
>>> fs.shape
(3, 9)

References

qsarkit.functional.impute(X, y=None, mols=None, strategy='median')[source]

Fill or remove non-finite values in the feature matrix.

Descriptor calculators emit NaN for undefined quantities (a 3D descriptor on a molecule with no conformer, a ratio with a zero denominator), and most estimators refuse to fit on them.

Parameters:

  • X (ndarray[tuple[Any, ...], dtype[Any]]) – Feature matrix.

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]]) – Labels; subset alongside X when strategy="drop".

  • mols (Optional[List[Any]]) – Molecules; subset alongside X when strategy="drop".

  • strategy (Literal['mean', 'median', 'most_frequent', 'zero', 'drop']) – How to handle them. "drop" removes offending rows (and the matching labels and molecules); the others fill column-wise.

Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Examples

>>> import numpy as np
>>> from qsarkit.functional import impute
>>> X = np.array([[1.0, np.nan], [3.0, 4.0], [5.0, 6.0]])
>>> filled, _, _ = impute(X, strategy="median")
>>> float(filled[0, 1])
5.0

Dropping instead keeps labels aligned with the surviving rows:

>>> kept, y, _ = impute(X, np.array([1.0, 2.0, 3.0]), strategy="drop")
>>> kept.shape, y.tolist()
((2, 2), [2.0, 3.0])

References

qsarkit.functional.drop_constant(X, y=None, mols=None, threshold=0.0)[source]

Remove features whose variance is at or below threshold.

A bit that is set in every molecule, or in none, cannot separate them. Fingerprint blocks are mostly this: a 2048-bit Morgan fingerprint over a few hundred compounds typically has fewer than 300 columns that vary at all.

Parameters:
Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Examples

>>> import numpy as np
>>> from qsarkit.functional import drop_constant
>>> X = np.array([[1.0, 5.0], [2.0, 5.0], [3.0, 5.0]])
>>> reduced, _, _ = drop_constant(X)
>>> reduced.shape
(3, 1)

References

qsarkit.functional.drop_correlated(X, y=None, mols=None, threshold=0.95, method='pearson')[source]

Remove one of every pair of features correlated above threshold.

Parameters:
Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Examples

>>> import numpy as np
>>> from qsarkit.functional import drop_correlated
>>> X = np.array([[1.0, 2.0, 9.0], [2.0, 4.0, 1.0], [3.0, 6.0, 5.0]])
>>> reduced, _, _ = drop_correlated(X, threshold=0.99)
>>> reduced.shape                # columns 0 and 1 are perfectly correlated
(3, 2)

References

qsarkit.functional.select_features(X, y=None, mols=None, method='mutual_info', k=20, task='regression', estimator=None, **kwargs)[source]

Select the k most informative features.

Parameters:

  • X (ndarray[tuple[Any, ...], dtype[Any]]) – Feature matrix.

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]]) – Labels. Required for every method except "variance".

  • mols (Optional[List[Any]]) – Molecules, passed through untouched.

  • method (Literal['mutual_info', 'rfe', 'boruta', 'variance']) – Selection strategy, from qsarkit.feature_selection.

  • k (int) – Number of features to keep. Ignored by "variance", and by "boruta", which determines the count itself.

  • task (Literal['regression', 'classification']) – Whether y is continuous or categorical. Chooses the underlying scoring function.

  • estimator (Optional[Any]) – Base estimator for "rfe" and "boruta".

  • **kwargs (Any) – Passed to the underlying selector.

Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Notes

Selecting features on the full dataset and then splitting is selection bias: the choice of columns has already seen the test labels, and the held-out score is optimistic. Put this step after split(), or inside a pipeline handed to cross_validate().

Examples

>>> import numpy as np
>>> from qsarkit.functional import select_features
>>> rng = np.random.default_rng(0)
>>> X = rng.normal(size=(30, 10))
>>> y = X[:, 0] * 2 + rng.normal(scale=0.1, size=30)
>>> reduced, _, _ = select_features(X, y, k=3)
>>> reduced.shape
(30, 3)

References

qsarkit.functional.resample(X, y=None, mols=None, sampler='undersample', random_state=None)[source]

Rebalance the classes in feature space.

The counterpart of balance(), applied after featurization. Because the rows here are just vectors, a sampler that synthesizes new ones – SMOTE, ADASYN, BorderlineSMOTE – is meaningful, which it is not at the molecule stage.

Parameters:

  • X (ndarray[tuple[Any, ...], dtype[Any]]) – Feature matrix.

  • y (Optional[ndarray[tuple[Any, ...], dtype[Any]]]) – Class labels. Required; resampling an unlabelled set is meaningless.

  • mols (Optional[List[Any]]) – The molecules the rows came from. Kept when the sampler selects existing rows, and dropped with a warning when it synthesizes new ones – a synthesized vector has no molecule, and returning a mismatched list would be worse than returning none.

  • sampler (Any) –

    A built-in random strategy, or any imbalanced-learn sampler with fit_resample(X, y).

  • random_state (Optional[int]) – Seed for the built-in strategies. Ignored for a sampler instance, which carries its own.

Returns:

(X, y, mols).

Return type:

Tuple[ndarray[tuple[Any, ...], dtype[Any]], Optional[ndarray[tuple[Any, ...], dtype[Any]]], Optional[List[Any]]]

Raises:

ValueError – If y is None, or sampler is neither a recognized string nor an object with fit_resample.

Notes

Resample the training set only. Rebalancing the test set changes the class prior you are measuring against, so a balanced test score does not describe the population the model will meet. Place this step after split(), applying it to the training half alone.

Synthetic oversampling is also not free: SMOTE interpolates between neighbours, and in a sparse binary fingerprint space the midpoint of two molecules is a vector no molecule would produce. It often helps with descriptors and often does not with fingerprints – measure it rather than assuming.

Examples

>>> import numpy as np
>>> from qsarkit.functional import fingerprint, molecules, resample
>>> smiles = ["CCO", "CCN", "CCC", "CCCl", "c1ccccc1", "c1ccncc1"]
>>> labels = np.array([0, 0, 0, 0, 1, 1])
>>> features = molecules(smiles, labels) >> fingerprint(n_bits=64)
>>> balanced = features >> resample(random_state=0)
>>> sorted(balanced.y.tolist())
[0, 0, 1, 1]

Oversampling instead keeps every majority row:

>>> balanced = features >> resample("oversample", random_state=0)
>>> sorted(balanced.y.tolist())
[0, 0, 0, 0, 1, 1, 1, 1]

An imbalanced-learn sampler is passed directly:

>>> from imblearn.over_sampling import SMOTE
>>> features >> resample(SMOTE(k_neighbors=1))

References

Terminal steps

qsarkit.functional.split(method='scaffold', test_size=0.2, random_state=None, splitter=None, **kwargs)[source]

Split into train and test sets, unpacking as train, test.

Parameters:

  • method (str) – One of "scaffold", "stratified_scaffold", "random", "butina", "sphere_exclusion", "maxmin", "kennard_stone", "perimeter" or "time". Ignored when splitter is given.

  • test_size (float) – Fraction held out.

  • random_state (Optional[int]) – Seed, where the splitter uses one.

  • splitter (Optional[Any]) – A splitter instance to use instead of building one from method – any of qsarkit.model_selection, or a scikit-learn splitter.

  • **kwargs (Any) – Passed to the splitter’s constructor.

Returns:

A step returning (train, test), each a FeatureSet.

Return type:

PipeStep

Notes

The default is a scaffold split, not a random one, because a random split of a QSAR dataset measures interpolation: public datasets are dense with near-duplicate analogues, so random assignment scatters a congeneric series across both sides and the model is scored on compounds whose close relatives it has memorized. A scaffold split keeps whole series together and reports what you actually want to know.

Examples

>>> from qsarkit.functional import fingerprint, molecules, split
>>> smiles = ["c1ccccc1C", "c1ccccc1CC", "c1ccncc1C", "CCO", "CCN"]
>>> train, test = (
...     molecules(smiles, [1.0, 2.0, 3.0, 4.0, 5.0])
...     >> fingerprint(n_bits=64)
...     >> split(test_size=0.4)
... )
>>> len(train) + len(test)
5

References

  • Bemis, G. W. & Murcko, M. A. (1996). “The Properties of Known Drugs. 1. Molecular Frameworks.” J. Med. Chem., 39(15), 2887-2893. https://doi.org/10.1021/jm9602928

  • Sheridan, R. P. (2013). “Time-Split Cross-Validation as a Method for Estimating the Goodness of Prospective Prediction.” J. Chem. Inf. Model., 53(4), 783-790. https://doi.org/10.1021/ci400084k

qsarkit.functional.fit(estimator='rf', task='auto', **kwargs)[source]

Fit a model on the features, ending the pipe with a fitted estimator.

Parameters:

  • estimator (Any) – A backend name for QSARRegressor / QSARClassifier ("rf", "svm", "gbm", "xgboost", "lightgbm", "knn", "pls", "gp", "mlp", …), or any estimator instance – including a plain scikit-learn one.

  • task (Literal['regression', 'classification', 'auto']) – Which facade to build when estimator is a name. "auto" infers it from y: few distinct integer labels means classification, anything else regression.

  • **kwargs (Any) – Passed to the facade’s constructor (random_state, model_params).

Returns:

A step returning the fitted estimator.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import fingerprint, fit, molecules
>>> smiles = ["CCO", "CCN", "CCC", "CCCl", "c1ccccc1", "c1ccncc1"]
>>> model = (
...     molecules(smiles, [1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
...     >> fingerprint(n_bits=64)
...     >> fit("rf", random_state=0)
... )
>>> model.predict(fingerprint(n_bits=64)(molecules(["CCO"])).X).shape
(1,)

Any estimator instance works, so the pipe is not limited to the facades:

>>> from sklearn.linear_model import Ridge
>>> model = (
...     molecules(smiles, [1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
...     >> fingerprint(n_bits=64)
...     >> fit(Ridge())
... )
>>> type(model).__name__
'Ridge'

References

qsarkit.functional.cross_validate(estimator='rf', task='auto', **kwargs)[source]

Cross-validate on the features, ending the pipe with a score report.

Parameters:

  • estimator (Any) – As for fit().

  • task (Literal['regression', 'classification', 'auto']) – As for fit().

  • **kwargs (Any) – Passed to CrossValidator (method, n_splits, random_state).

Returns:

A step returning the cross-validation report as a dict.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import cross_validate, fingerprint, molecules
>>> smiles = ["CCO", "CCN", "CCC", "CCCl", "c1ccccc1", "c1ccncc1"]
>>> report = (
...     molecules(smiles, [1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
...     >> fingerprint(n_bits=64)
...     >> cross_validate("rf", n_splits=3, random_state=0)
... )
>>> "q2" in report or "r2" in report
True

References

qsarkit.functional.applicability_domain(method='knn', **kwargs)[source]

Fit an applicability domain on the features, ending the pipe.

Parameters:

  • method (str) – One of "knn", "leverage", "range", "bounding_box", "pca", "convex_hull", "tanimoto", "kde", "isolation_forest" or "ensemble".

  • **kwargs (Any) – Passed to the domain’s constructor.

Returns:

A step returning the fitted domain.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import applicability_domain, fingerprint, molecules
>>> smiles = ["CCO", "CCN", "CCC", "CCCl", "c1ccccc1", "c1ccncc1"]
>>> domain = (
...     molecules(smiles)
...     >> fingerprint(n_bits=64)
...     >> applicability_domain("tanimoto", threshold=0.3)
... )
>>> domain.predict(fingerprint(n_bits=64)(molecules(["CCO"])).X).tolist()
[True]

References

qsarkit.functional.collect(as_frame=False)[source]

End a pipe explicitly, returning the set or a DataFrame of it.

Useful when a pipeline is built programmatically and you want the terminal stage to be a step like any other, and when you want the result as a table rather than as arrays.

Parameters:

as_frame (bool) – Return to_frame() instead of the set itself.

Returns:

A step returning the value flowing into it.

Return type:

PipeStep

Examples

>>> from qsarkit.functional import collect, desalt, molecules
>>> frame = molecules(["CCO", "CC(=O)[O-].[Na+]"]) >> desalt() >> collect(as_frame=True)
>>> list(frame.columns)
['smiles']
>>> len(frame)
2

Flowchart

class qsarkit.functional.PipelineNode(label, domain, detail=None)[source]

Bases: object

One stage of a pipeline, as it appears in the flowchart.

Parameters:

  • label (str) – Step name with its configured arguments.

  • domain (str) – What the step consumes and produces, which sets its colour and the label on its outgoing edge.

  • detail (Optional[str]) – Second line of the node, e.g. the estimator or transformer class.

Variables:
label
domain
detail
qsarkit.functional.pipeline_nodes(pipe, include_input=True)[source]

Flatten a pipeline into the nodes of its flowchart.

Parameters:

  • pipe (Any) – A step, a composed pipeline, or a set whose history is drawn.

  • include_input (bool) – Prepend an input node representing the incoming molecules.

Return type:

List[PipelineNode]

Examples

>>> from qsarkit.functional import desalt, fingerprint, pipeline_nodes
>>> [n.domain for n in pipeline_nodes(desalt() >> fingerprint())]
['input', 'molecules', 'transition']
qsarkit.functional.to_dot(pipe, name='qsarkit_pipeline', rankdir='TB', include_input=True)[source]

Render a pipeline as Graphviz DOT source.

Parameters:

  • pipe (Any) – The pipeline to draw.

  • name (str) – Graph name.

  • rankdir (str) – Layout direction: top-to-bottom or left-to-right.

  • include_input (bool) – Draw the input node.

Returns:

DOT source, renderable with dot -Tpng or by the graphviz Python package.

Return type:

str

Examples

>>> from qsarkit.functional import desalt, fingerprint, to_dot
>>> dot = to_dot(desalt() >> fingerprint())
>>> dot.splitlines()[0]
'digraph qsarkit_pipeline {'
>>> "desalt()" in dot
True

References

qsarkit.functional.plot_pipeline(pipe, title='Pipeline', include_input=True, orientation='vertical')[source]

Render a pipeline as a Plotly flowchart.

The dependency-free renderer: it needs only what qsarkit already requires, and returns a figure like every other plot in the package (never shown, never written to disk).

Parameters:

  • pipe (Any) – The pipeline to draw.

  • title (str) – Figure title.

  • include_input (bool) – Draw the input node.

  • orientation (str) – Direction of flow.

Return type:

Figure

Examples

>>> from qsarkit.functional import desalt, fingerprint, plot_pipeline
>>> figure = plot_pipeline(desalt() >> fingerprint())
>>> type(figure).__name__
'Figure'

Export needs kaleido (pip install qsarkit-learn[reporting]):

>>> figure.write_image("pipeline.png")

References

qsarkit.functional.render_pipeline(pipe, path, engine='auto', include_input=True, **kwargs)[source]

Write a pipeline flowchart to a PNG, PDF or SVG file.

Parameters:

  • pipe (Any) – The pipeline to draw.

  • path (str) – Output file. The extension chooses the format: .png, .pdf or .svg.

  • engine (str) – Renderer. "auto" uses Graphviz when the graphviz package and its dot binary are both present, and Plotly otherwise.

  • include_input (bool) – Draw the input node.

  • **kwargs (Any) – Passed to the chosen renderer (rankdir for Graphviz, title/orientation for Plotly).

Returns:

The path written.

Return type:

str

Raises:

  • ValueError – If the extension is not a supported format, or engine is not one of the three accepted values.

  • OptionalDependencyError – If the requested engine’s dependency is missing.

Examples

>>> from qsarkit.functional import desalt, fingerprint, render_pipeline
>>> render_pipeline(desalt() >> fingerprint(), "pipeline.pdf")
'pipeline.pdf'

Notes

The Plotly path needs kaleido for static export (pip install qsarkit-learn[reporting]); the Graphviz path needs the dot binary, which is a system package rather than a Python one.

References

References

  • Bache, S. M. & Wickham, H. (2014). “magrittr: A Forward-Pipe Operator for R.” https://CRAN.R-project.org/package=magrittr

  • Fourches, D., Muratov, E. & Tropsha, A. (2010). “Trust, But Verify: On the Importance of Chemical Structure Curation in Cheminformatics and QSAR Modeling Research.” J. Chem. Inf. Model., 50(7), 1189-1204. doi:10.1021/ci100176x

  • Heller, S. R. et al. (2015). “InChI - the Worldwide Chemical Structure Identifier Standard.” J. Cheminform., 7, 23. doi:10.1186/s13321-015-0068-4