Persistence

Saving a model so it still works next year, and so someone else can load it safely.

Why not pickle

pickle is the obvious choice and the wrong one for a model you intend to keep:

  • It embeds the exact class layout of every object, so a file written under one scikit-learn or NumPy release can fail to load — or load into a subtly different object — under the next.

  • Loading one executes arbitrary code, which makes a shared model file a security problem rather than a data file.

qsarkit writes a directory bundle instead:

model.qsar/
  manifest.json      what is in here, and what wrote it
  metadata.json      endpoint, task, feature names, provenance
  estimator.skops    the fitted estimator, in skops' inspectable format
  pipeline.skops     the preprocessing pipeline, when there is one

Everything except the estimator is plain JSON, readable without importing qsarkit at all. The estimator uses skops, the format scikit-learn recommends: it stores parameters as data rather than as a pickled object graph, and refuses to reconstruct types that were not explicitly trusted.

Saving and loading

>>> import os, tempfile
>>> from qsarkit.models import QSARRegressor
>>> from qsarkit.persistence import ModelMetadata, load_model, save_model
>>> X, y = demo_fingerprints(256), DEMO_Y
>>> model = QSARRegressor("rf", random_state=0).fit(X, y)
>>> path = save_model(
...     model,
...     os.path.join(tempfile.mkdtemp(), "demo"),
...     metadata=ModelMetadata(
...         name="benzoic acid pIC50",
...         endpoint="pIC50 (-log10 M)",
...         task="regression",
...         n_training_samples=len(y),
...     ),
... )
>>> path.endswith(".qsar")
True

The reloaded model predicts identically:

>>> import numpy as np
>>> bundle = load_model(path)
>>> bool(np.allclose(bundle.predict(X), model.predict(X)))
True
>>> bundle.metadata.endpoint
'pIC50 (-log10 M)'

Predicting from molecules

Save the transformer alongside the estimator and the bundle can go straight from structures to predictions — which also removes the commonest way to misuse a saved model, feeding it features from a different fingerprint:

>>> from qsarkit.representation import MorganFingerprint
>>> fingerprint = MorganFingerprint(radius=2, n_bits=256)
>>> path = save_model(
...     model,
...     os.path.join(tempfile.mkdtemp(), "with_pipeline"),
...     pipeline=fingerprint,
...     metadata=ModelMetadata(name="demo", endpoint="pIC50 (-log10 M)"),
... )
>>> bundle = load_model(path)
>>> bool(np.allclose(bundle.predict_mols(demo_mols), model.predict(X)))
True

Without a pipeline that is refused rather than guessed at:

>>> bare = load_model(save_model(model, os.path.join(tempfile.mkdtemp(), "bare")))
>>> bare.predict_mols(demo_mols)
Traceback (most recent call last):
    ...
ValueError: This bundle has no pipeline, so molecules cannot be featurized...

Guardrails

A feature-width mismatch is the failure that otherwise produces confident nonsense, so it is checked:

>>> bundle.predict(np.zeros((2, 64)))
Traceback (most recent call last):
    ...
ValueError: This model expects 256 features but was given 64...

And a model loaded under different package versions says so, because it is not guaranteed to reproduce its original predictions:

>>> import warnings
>>> bundle.metadata.environment["scikit-learn"] = "0.1"
>>> from qsarkit.persistence import save_model as _save
>>> stale = _save(bundle, os.path.join(tempfile.mkdtemp(), "stale"))
>>> with warnings.catch_warnings(record=True) as caught:
...     warnings.simplefilter("always")
...     _ = load_model(stale)
>>> "scikit-learn" in str(caught[0].message)
True

Inspecting before loading

The safe first step with a model from someone else. It reports what the bundle contains without reconstructing anything:

>>> from qsarkit.persistence import inspect_bundle
>>> report = inspect_bundle(path)
>>> report["manifest"]["estimator_class"]
'qsarkit.models._facades.QSARRegressor'
>>> report["untrusted"]
[]

untrusted lists the third-party types skops would need permission to build. qsarkit’s own classes are trusted automatically — that is no more dangerous than import qsarkit, since the class comes from the installed package and the file supplies only attribute values. Anything else you must recognize and name:

bundle = load_model(path, trusted=["mypackage.MyTransformer"])

Warning

allow_pickle_fallback=True exists for estimators skops cannot represent, and reintroduces exactly the version fragility and arbitrary-code-execution risk this module avoids. It warns when used. Prefer wrapping the object in a scikit-learn-compatible estimator.

Provenance

>>> from qsarkit.persistence import environment_summary
>>> summary = environment_summary()
>>> sorted(summary)[:3]
['numpy', 'platform', 'python']

Recording this is what OECD principle 2 — an unambiguous algorithm — asks for in practice: a year later, the metadata is how you establish what the model was and what its numbers meant.

API

Robust, pickle-free persistence for QSAR models and their pipelines.

pickle is the obvious way to save a model and the wrong one for a model you intend to keep: it embeds the exact class layout of every object, so a file written under one scikit-learn release can fail to load under the next, and loading one executes arbitrary code.

This package writes a directory bundle instead – JSON metadata beside a skops representation of the estimator, which stores parameters as data and refuses to reconstruct types that were not explicitly trusted.

Examples

>>> import numpy as np, os, tempfile
>>> from sklearn.linear_model import Ridge
>>> from qsarkit.persistence import ModelMetadata, load_model, save_model
>>> rng = np.random.default_rng(0)
>>> X, y = rng.normal(size=(40, 5)), rng.normal(size=40)
>>> path = save_model(
...     Ridge().fit(X, y),
...     os.path.join(tempfile.mkdtemp(), "demo"),
...     metadata=ModelMetadata(name="demo", endpoint="pIC50 (-log10 M)"),
... )
>>> bundle = load_model(path)
>>> bundle.metadata.endpoint
'pIC50 (-log10 M)'
>>> bool(np.allclose(bundle.predict(X), Ridge().fit(X, y).predict(X)))
True

References

class qsarkit.persistence.ModelBundle(estimator, pipeline=None, metadata=None)[source]

Bases: object

A fitted model, its preprocessing, and the provenance to interpret it.

The unit qsarkit saves and loads. Holding the three together is the point: an estimator without its representation is not a model, and a model without its endpoint and feature layout cannot be used safely a year later.

Parameters:

  • estimator (Any) – The fitted model. Any scikit-learn-compatible estimator.

  • pipeline (Optional[Any]) – The transformer (or sklearn.pipeline.Pipeline) that turns molecules into the feature matrix estimator expects. Supply it whenever you have one: it is what makes predict_mols() possible, and what stops a reloaded model being fed the wrong features.

  • metadata (Optional[ModelMetadata]) – Provenance. A default is created if omitted, but an empty endpoint is worth filling in.

Variables:

Examples

>>> import numpy as np
>>> from sklearn.linear_model import Ridge
>>> from qsarkit.persistence import ModelBundle, ModelMetadata
>>> rng = np.random.default_rng(0)
>>> X, y = rng.normal(size=(40, 5)), rng.normal(size=40)
>>> bundle = ModelBundle(
...     Ridge().fit(X, y),
...     metadata=ModelMetadata(name="demo", endpoint="pIC50 (-log10 M)"),
... )
>>> bundle.predict(X).shape
(40,)

Round-tripping through a directory preserves the predictions exactly:

>>> import tempfile, os
>>> path = os.path.join(tempfile.mkdtemp(), "demo.qsar")
>>> _ = bundle.save(path)
>>> reloaded = ModelBundle.load(path)
>>> bool(np.allclose(reloaded.predict(X), bundle.predict(X)))
True
>>> reloaded.metadata.endpoint
'pIC50 (-log10 M)'

References

predict(X)[source]

Predict from an already-featurized matrix.

Parameters:

X (Any)

Return type:

Any

Raises:

ValueError – If X has a different number of columns than the model was fitted on. Checked rather than passed through, because a width mismatch otherwise produces confident nonsense.

predict_proba(X)[source]

Class probabilities, for a classification bundle.

Parameters:

X (Any)

Return type:

Any

Raises:

AttributeError – If the underlying estimator has no predict_proba.

predict_mols(mols)[source]

Predict straight from molecules, using the stored pipeline.

Parameters:

mols (Sequence[Any]) – RDKit molecules, or SMILES if the stored pipeline begins with a SMILES parser.

Return type:

Any

Raises:

ValueError – If the bundle carries no pipeline, so there is no way to know which representation the estimator expects.

save(path, allow_pickle_fallback=False, overwrite=True)[source]

Write the bundle to a directory. See save_model().

Return type:

str

classmethod load(path, trusted=None, warn_on_environment_change=True)[source]

Read a bundle from a directory. See load_model().

Return type:

ModelBundle

class qsarkit.persistence.ModelMetadata(name='', endpoint='', task='regression', qsarkit_version='', format_version='1', created='', environment=<factory>, n_features=None, feature_names=None, n_training_samples=None, description='', extra=<factory>)[source]

Bases: object

What a saved model needs to carry to remain interpretable.

Variables:

  • name (str) – Human-readable model name.

  • endpoint (str) – What the model predicts, and in what units – "pIC50 (-log10 M)" rather than "activity".

  • task (str) – "regression" or "classification".

  • qsarkit_version (str) – Version that wrote the file.

  • format_version (str) – Version of the bundle layout.

  • created (str) – UTC ISO-8601 timestamp.

  • environment (dict) – Output of environment_summary() at save time.

  • n_features (int, optional) – Expected width of the feature matrix. Checked on load, because a width mismatch is the failure that otherwise produces confident nonsense.

  • feature_names (list of str, optional) – Column names, where the representation provides them.

  • n_training_samples (int, optional) – How many compounds the model was fitted on.

  • description (str) – Free text.

  • extra (dict) – Anything else worth recording – dataset DOI, assay, curation settings, validation scores.

Examples

>>> from qsarkit.persistence import ModelMetadata
>>> meta = ModelMetadata(name="demo", endpoint="pIC50", task="regression")
>>> meta.task
'regression'
>>> restored = ModelMetadata.from_dict(meta.to_dict())
>>> restored.name == meta.name
True
name: str
endpoint: str
task: str
qsarkit_version: str
format_version: str
created: str
environment: Dict[str, str]
n_features: int | None
feature_names: List[str] | None
n_training_samples: int | None
description: str
extra: Dict[str, Any]
to_dict()[source]

Render as a JSON-serializable dict.

Return type:

Dict[str, Any]

classmethod from_dict(data)[source]

Rebuild from to_dict() output, ignoring unknown keys.

Unknown keys are dropped rather than raising, so a bundle written by a newer qsarkit that added a field still loads here.

Parameters:

data (Dict[str, Any])

Return type:

ModelMetadata

environment_differences()[source]

Packages whose current version differs from the recorded one.

Returns:

Package name -> "saved X, now Y" for each mismatch.

Return type:

Dict[str, str]

Examples

>>> from qsarkit.persistence import ModelMetadata
>>> meta = ModelMetadata(name="demo")
>>> meta.environment_differences()          # same session, so none
{}
>>> meta.environment["numpy"] = "0.0.1"
>>> "numpy" in meta.environment_differences()
True
qsarkit.persistence.save_model(bundle, path, pipeline=None, metadata=None, allow_pickle_fallback=False, overwrite=True)[source]

Write a model to a qsarkit bundle directory.

Parameters:

  • bundle (Union[ModelBundle, Any]) – A prepared bundle, or a bare fitted estimator – in which case pipeline and metadata are used to build one.

  • path (Union[str, Path]) – Destination directory. .qsar is appended when the path has no suffix, purely as a convention.

  • pipeline (Optional[Any]) – Only used when bundle is a bare estimator.

  • metadata (Optional[ModelMetadata]) – Only used when bundle is a bare estimator.

  • allow_pickle_fallback (bool) – If skops cannot represent an object, fall back to joblib (which pickles). Off by default: the fallback reintroduces exactly the fragility and the arbitrary-code-execution risk this module exists to avoid, so it has to be asked for.

  • overwrite (bool) – Replace an existing bundle at path. With False, an existing directory raises.

Returns:

The directory written.

Return type:

str

Raises:

Examples

>>> import numpy as np, tempfile, os
>>> from sklearn.linear_model import Ridge
>>> from qsarkit.persistence import load_model, save_model
>>> rng = np.random.default_rng(0)
>>> X, y = rng.normal(size=(30, 4)), rng.normal(size=30)
>>> out = os.path.join(tempfile.mkdtemp(), "ridge")
>>> written = save_model(Ridge().fit(X, y), out)
>>> written.endswith(".qsar")
True
>>> sorted(p.name for p in __import__("pathlib").Path(written).iterdir())
['estimator.skops', 'manifest.json', 'metadata.json']

References

qsarkit.persistence.load_model(path, trusted=None, warn_on_environment_change=True)[source]

Read a qsarkit bundle written by save_model().

Parameters:

  • path (Union[str, Path]) – The bundle directory.

  • trusted (Optional[List[str]]) – Extra type names to allow skops to reconstruct, beyond what it trusts by default. Inspect a bundle first with inspect_bundle() and pass only what you recognize – this is the mechanism that makes loading a third-party model safe, so blanket-trusting everything defeats it.

  • warn_on_environment_change (bool) – Warn when the current package versions differ from those recorded at save time. A model is not guaranteed to reproduce its original predictions across versions, and silence would hide that.

Return type:

ModelBundle

Raises:

  • FileNotFoundError – If path is not a bundle directory.

  • ValueError – If the manifest is missing, unreadable, or written in a bundle format this version does not understand.

Examples

>>> import numpy as np, tempfile, os
>>> from sklearn.linear_model import Ridge
>>> from qsarkit.persistence import load_model, save_model
>>> rng = np.random.default_rng(0)
>>> X, y = rng.normal(size=(30, 4)), rng.normal(size=30)
>>> out = save_model(Ridge().fit(X, y), os.path.join(tempfile.mkdtemp(), "m"))
>>> bundle = load_model(out)
>>> bundle.metadata.task
'regression'
>>> bundle.predict(X).shape
(30,)
qsarkit.persistence.inspect_bundle(path)[source]

Describe a bundle without reconstructing any object from it.

The safe first step with a model from someone else: it reports the manifest, the metadata and the list of types skops would need to build, so you can decide what to trust before anything is executed.

Parameters:

path (Union[str, Path]) – The bundle directory.

Returns:

manifest, metadata, untrusted and auto_trusted. untrusted holds the type names that would block a load until they are passed to load_model(trusted=...); auto_trusted holds the ones qsarkit accepts on your behalf – its own classes, plus the scikit-learn internals its estimator menu produces (see _AUTO_TRUSTED_PREFIXES). An empty untrusted means the bundle loads as it stands, not that nothing in it is executable, so inspect auto_trusted too for a bundle you did not produce.

Return type:

Dict[str, Any]

Raises:

Examples

>>> import numpy as np, tempfile, os
>>> from sklearn.linear_model import Ridge
>>> from qsarkit.persistence import inspect_bundle, save_model
>>> rng = np.random.default_rng(0)
>>> X, y = rng.normal(size=(30, 4)), rng.normal(size=30)
>>> out = save_model(Ridge().fit(X, y), os.path.join(tempfile.mkdtemp(), "m"))
>>> report = inspect_bundle(out)
>>> report["manifest"]["estimator_class"]
'sklearn.linear_model._ridge.Ridge'
>>> report["untrusted"]
[]

References

qsarkit.persistence.environment_summary()[source]

Versions of the packages a saved model’s behaviour depends on.

Recorded at save time so a later load can warn about a mismatch rather than silently predicting something different.

Returns:

Package name -> version, plus python and platform.

Return type:

Dict[str, str]

Examples

>>> from qsarkit.persistence import environment_summary
>>> summary = environment_summary()
>>> "qsarkit" in summary and "python" in summary
True

References