Changelog

0.10.5 (unreleased)

Narrowed the package back to the QSAR workflow proper, and filled the gaps that narrowing exposed.

Scope

qsarkit is now a focused QSAR library rather than a general cheminformatics platform. Literature mining, database retrieval, docking, de-novo design, ADMET filtering, active learning, experiment design and benchmarking were removed: they are separate disciplines with separate failure modes, and covering them made the package broad rather than trustworthy. 29 subpackages became 21.

New modules

  • persistence — pickle-free model saving. A directory bundle holding JSON metadata beside a skops representation of the estimator, so a saved model survives a scikit-learn upgrade and can be inspected before it is loaded. Carries the provenance OECD principle 2 asks for.

Changed

  • YScrambling now scores out of fold by default (cv=5), with stratify="auto" choosing stratified folds when the target looks categorical. The previous default scored the apparent in-sample fit, which is uninformative for any flexible model: a random forest fits permuted labels almost as well as real ones on its own training data, so the test reported a passing p-value on an effect size of 0.001. Pass cv=None for the old behaviour, which remains the right choice for a low-capacity model on a small descriptor set.

    Scores reported by this class will change. The documented examples moved from \(R^2\) 0.95 against 0.84 on permuted labels to 0.69 against -0.26; the second pair is what the model does on compounds it has not seen.

New functionality

  • data_quality gained representation_conflicts, which measures the ceiling a feature matrix imposes before any model is fitted. Two molecules mapping to the same vector with different activities are an error no estimator can avoid, and the diagnostic reports the attainable accuracy, balanced accuracy or RMSE so a score can be read against what was reachable. It complements DuplicateDetector, which finds duplicate structures: two structures distinguishable by InChIKey can still collide under a hashed fingerprint.

  • sar gained draw_activity_cliff, draw_activity_cliffs and cliff_difference_atoms. A SAS map says how many cliffs a dataset holds; these say what changed, rendering the pair with the atoms outside their maximum common substructure highlighted. An empty highlight is itself the result — the pair differs in something a topological representation does not encode.

  • ActivityCliffDetector’s documentation now states what method="fingerprint" computes: Tanimoto over binary Morgan bits. A similarity of exactly 1.00 is therefore not necessarily a near-duplicate but a pair the representation cannot distinguish — homologues, because a binary bit cannot record that an environment repeats, and stereoisomers, because the fingerprint is topological. use_counts and use_chirality address the two cases respectively.

Fixed

  • QSARRegressor and QSARClassifier now expose their fitted backend’s attributes. The facades delegate fit and predict but hid everything a consumer introspects, so BorutaSelector and scikit-learn’s RFE rejected qsarkit’s own default estimator for lacking feature_importances_ or coef_, and EnsembleUncertainty missed estimators_ and silently refitted a bagging ensemble of forests instead of reusing the forest’s trees — slower, and markedly worse calibrated (ENCE 2.9 against 0.3 on the example in Uncertainty). Names beginning with an underscore are not forwarded, so cloning, copying and pickling are unaffected.

  • ChemicalSpaceAnalyzer.fit consumed entries from self.kwargs with pop(), mutating a constructor argument. A user’s perplexity was honoured by the first fit() and silently forgotten by the second, so two identical calls on one object returned different embeddings. The options are now copied before use.

  • KNNApplicabilityDomain with the default metric="euclidean" computed distances by materialising an (n_query, n_train, n_features) array. On 5000 training compounds described by 2048-bit fingerprints that is over 100 TB, so the domain was killed by the operating system on any realistic dataset while working fine on the two dozen molecules in the documentation. It now uses scipy.spatial.distance.cdist: the same distances in \(O(n \times m)\) memory, 19 s and 0.6 GB for that case.

  • SHAPExplainer could not explain qsarkit’s own estimators. shap.TreeExplainer rejects a QSARClassifier outright, and explainer_type="auto" classified the facade by its own class name and fell back to the kernel explainer, which is far slower and needs a background set. The facade is now unwrapped to its fitted backend — by type, since scikit-learn’s ensembles also carry an estimator_ attribute holding an unfitted base-estimator template.

  • SHAPExplainer now uses interventional perturbation when a background sample is supplied. It is the better-defined estimator and avoids the additivity-check failure that tree-path-dependent perturbation hits on wide fingerprint matrices.

  • QSARRegressor and QSARClassifier raised TypeError: got multiple values for keyword argument when model_params set a keyword the facade also sets, so QSARClassifier("rf", model_params={"n_estimators": 100}) — the package’s default estimator — failed outright. The caller’s value now wins.

New functionality

  • chemspace gained projection_trustworthiness and ChemicalSpaceAnalyzer.trustworthiness, which measure how much of a 2D projection’s local structure survived the embedding. A t-SNE or UMAP figure that destroyed the neighbourhood structure looks exactly like one that preserved it, so the score belongs with the figure. One neighbourhood size returns a float and an iterable returns an array in the order given, matching the convention the validators use; subsample makes the \(O(n^2)\) distance matrix tractable on a screening library.

  • Every validator in validation – CrossValidator, YScrambling, ExternalValidator and BootstrapValidator – now takes a scoring argument instead of being fixed to \(R^2\). Name a metric from available_metrics(), pass a (y_true, y_pred) callable, or wrap one with make_scorer when it needs probabilities or is a loss. Pass an iterable of metrics and every score in the result becomes an array in the order given, computed from a single pass over the folds.

    roc_auc, pr_auc and brier are given predict_proba’s positive-class column rather than a thresholded label. A loss declares greater_is_better=False, which is what keeps a y-randomization p-value from being reported backwards when it is computed on RMSE.

    YScrambling also gained cv and stratify, which score out of fold rather than in sample. This is necessary rather than cosmetic for a ranking metric: a random forest separates permuted labels in-sample as perfectly as real ones, so an in-sample ROC-AUC comparison reads 1.0 against 1.0 and detects nothing. The default stays in-sample, so existing results are unchanged.

  • validation gained YScrambling (y-randomization, required evidence under OECD principle 4), BootstrapValidator (out-of-bag score with a confidence interval) and ExternalValidator.

  • metrics gained probability calibration (calibration_curve, expected_calibration_error, maximum_calibration_error, calibration_report), residual diagnostics (qq_data, residual_normality) and decision-threshold selection (threshold_sweep, optimal_threshold, threshold_report).

  • explainability gained AttributionAtomMapper, which projects SHAP or LIME per-bit attributions back onto atoms through the fingerprint’s bit-provenance map, and draw_atom_weights, which renders them as an RDKit similarity map. from_shap takes a SHAPExplainer; from_lime takes whatever LIME produces — an explain_one frame, a LIME Explanation, a {bit: weight} mapping or pairs.

    The mapper asks the fingerprint which atoms set each bit rather than re-deriving the numbering from radius and n_bits, so the attribution describes the matrix the model was actually fitted on. The two disagree whenever another setting changes the hashing — use_chirality is the plain case — and the disagreement is invisible in the picture. A fingerprint that cannot report provenance is refused at construction: MACCS keys and path-based fingerprints have no bit-to-atom-environment correspondence, so there is nothing to project onto an atom.

  • representation gained MorganFingerprint.bit_info (inherited by FeatureMorganFingerprint), which reports the {bit: ((atom, radius), ...)} provenance for one molecule using the same generator transform uses, so the bit numbering necessarily matches. This is what the atom-contribution maps are built on.

  • QSARRegressor and QSARClassifier now accept any scikit-learn-compatible estimator — XGBoost, LightGBM, CatBoost or your own — as a class or an instance, with model_args, model_params, fit_params, predict_params and predict_proba_params.

  • functional.molecules accepts InChI as well as SMILES and RDKit molecules, auto-detected per entry.

  • functional.balance accepts an imbalanced-learn sampler. Samplers that synthesize feature vectors (SMOTE and relatives) are rejected at the molecule stage with an explanation, since no molecule corresponds to an interpolated vector.

  • The functional pipe reaches the whole workflow: featurize, fingerprint, describe, scale, impute, drop_constant, drop_correlated, select_features, split, fit, cross_validate, applicability_domain, collect, plus flowchart rendering to PNG, PDF and SVG.

  • QSARReport renders to plain text and PDF in addition to Markdown, HTML and JSON, with tables and plots carried into each.

  • New plots: plot_calibration_curve, plot_qq, plot_threshold_sweep, plot_precision_recall, plot_atom_contributions.

Fixes

  • Passing a feature matrix where molecules are needed now says so. DiversityAnalyzer, ScaffoldAnalyzer, ClusterAnalyzer, NearestNeighborAnalyzer, ChemicalSpaceCoverage and ActivityCliffDetector previously surfaced RDKit’s Boost ArgumentError quoting a C++ signature, from whichever frame happened to touch the array first — GetFingerprint or MurckoDecompose — which does not say what to pass instead. A pipeline’s result carries both the matrix and the molecules, and .X is the more obvious attribute than .mols, so the error now names the array’s shape and the attribute to use. Unparsed SMILES and None entries are reported the same way. ChemicalSpaceAnalyzer still accepts either, as documented.

  • StructureValidator now checks net formal charge. It previously flagged every zwitterion — glycine, ciprofloxacin, any betaine — as a record that had escaped neutralization.

  • check_activity_units no longer reports raw molar concentrations as logarithmic when they happen to span a narrow numeric range. A pActivity of 1e-9 would mean an IC50 near 1 M.

  • NearestNeighborAnalyzer.nearest_similarity(exclude_self=True) masks each query’s own row rather than every perfect match, so redundancy detects exact duplicates instead of hiding them.

  • DiversityAnalyzer and ScaffoldAnalyzer now agree on n_scaffolds: both exclude acyclic molecules, which have no Bemis-Murcko framework. Previously a library of straight chains looked scaffold-diverse.

  • CorrelationFilter drops zero-variance columns deliberately instead of as a side effect of NaN comparisons, and no longer emits a divide-by-zero warning per column on fingerprint input.

  • GlycanDescriptors and FragmentRemover no longer substitute defaults in __init__, which broke clone() and made get_params() report values the caller never passed.

  • QSARReport.to_markdown no longer requires the undeclared tabulate package for reports containing a table.

  • functional.molecules and the SMILES readers restore the caller’s RDKit logging state instead of switching it back on.

  • Per-molecule error handling no longer uses bare except Exception, which swallowed MemoryError. A documented RDKIT_MOLECULE_ERRORS tuple replaces it.

  • DuplicateDetector computes activity spread with vectorized NumPy.

Documentation

  • Every example in the docstrings, the guide, the API reference and the notebooks is executed by the test suite, so none can go stale silently.

  • Four example notebooks covering every public subpackage, committed with their output.

  • package.md describes the whole package organization in one file.

  • LaTeX formulas render in HTML and PDF; the Read the Docs build treats warnings as errors.

  • Added a LICENSE file, which pyproject.toml had referenced without shipping.

0.2.0

Expanded qsarkit from a QSAR modeling library into a broad cheminformatics platform. Much of this was removed again in 0.4.0; see above.

New modules

  • chemistry — standardization, glycan detection/removal/descriptors, fragment and protecting-group removal, scaffold extraction, molecular graphs.

  • sar — matched molecular pairs, activity cliffs, SALI, SARI, activity-landscape (SAS) maps, R-group tables, Free-Wilson analysis.

  • applicability — eleven applicability-domain definitions behind one interface, plus coverage and accuracy-vs-coverage analysis.

  • neighbors and cluster — Tanimoto/Jaccard similarity search and k-NN estimators; Taylor-Butina, sphere exclusion, MaxMin and hierarchical clustering with a scikit-learn API.

  • data_quality, representation, uncertainty, explainability, chemspace, reporting, functional.

Infrastructure

  • mypy --strict clean; ships a py.typed marker.

  • All plotting standardized on Plotly.

  • Heavy dependencies made lazy and grouped into pip extras.

  • Sphinx documentation with per-class scientific references.

0.1.0

Initial QSAR modeling library.