Neighbors

Tanimoto/Jaccard similarity search and k-NN estimators for fingerprints.

Warning

Euclidean distance is the wrong metric for sparse binary fingerprints. Two molecules that share no substructures are “close” in Euclidean terms because they agree on the thousands of bits that are jointly zero — bits that carry no chemical information. Every distance here is Tanimoto/Jaccard, verified against RDKit’s BulkTanimotoSimilarity.

Distances

>>> from qsarkit.neighbors import is_binary, jaccard_distance, tanimoto_similarity_matrix
>>> X = demo_fingerprints(256)
>>> round(float(jaccard_distance(X[0], X[1])), 3)
0.45
>>> is_binary(X)
True
>>> similarity = tanimoto_similarity_matrix(X)
>>> similarity.shape, round(float(similarity[0, 1]), 3)
((24, 24), 0.55)

k-NN estimators

>>> from qsarkit.neighbors import JaccardKNeighborsRegressor
>>> model = JaccardKNeighborsRegressor(n_neighbors=1).fit(X, DEMO_Y)
>>> round(float(model.predict(X[:1])[0]), 2)   # its own label, exactly
5.1

With more neighbours the prediction averages their labels, and because of the ties above the exact average depends on which tied molecule is drawn in. What the method guarantees is the bound, not the value:

>>> averaged = JaccardKNeighborsRegressor(n_neighbors=3).fit(X, DEMO_Y)
>>> prediction = float(averaged.predict(X[:1])[0])
>>> bool(DEMO_Y.min() <= prediction <= DEMO_Y.max())
True

weights="similarity" weights neighbours by Tanimoto similarity rather than inverse distance, which is the natural reading for fingerprints:

>>> weighted = JaccardKNeighborsRegressor(
...     n_neighbors=3, weights="similarity").fit(X, DEMO_Y)
>>> weighted.predict(X[:1]).shape
(1,)

A k-NN model on Tanimoto distance is worth fitting even when you intend to use something else: it is the direct expression of the similar property principle, so it is the baseline any more complex model has to beat to justify itself.

API

Fingerprint similarity search and k-NN estimators under Jaccard/Tanimoto distance.

qsarkit.neighbors.jaccard_distance(u, v)[source]

Jaccard (1 - Tanimoto) distance between two fingerprint vectors.

This is the callable to hand to scikit-learn estimators that accept metric=<callable>.

Parameters:
Returns:

Distance in [0, 1].

Return type:

float

Examples

>>> round(jaccard_distance([1, 1, 0, 0], [1, 0, 0, 0]), 4)
0.5
qsarkit.neighbors.jaccard_similarity(u, v)[source]

Tanimoto/Jaccard similarity between two fingerprint vectors.

Uses the binary expression when both vectors are binary and the generalized MinMax expression otherwise.

Parameters:
Returns:

Similarity in [0, 1]. Two all-zero vectors are defined to have similarity 1.0 (they are identical), following the convention used by RDKit’s BulkTanimotoSimilarity for empty fingerprints.

Return type:

float

Examples

>>> round(jaccard_similarity([1, 1, 0, 0], [1, 0, 0, 0]), 4)
0.5
qsarkit.neighbors.jaccard_distance_matrix(X, Y=None)[source]

Pairwise Jaccard distance matrix (1 - tanimoto_similarity_matrix).

Parameters:
Returns:

Pairwise distances in [0, 1].

Return type:

ndarray[tuple[Any, ...], dtype[double]]

Examples

>>> D = jaccard_distance_matrix(np.array([[1, 1, 0], [1, 0, 0]]))
>>> bool(np.allclose(np.diag(D), 0.0))
True
qsarkit.neighbors.tanimoto_similarity_matrix(X, Y=None)[source]

Vectorized pairwise Tanimoto/Jaccard similarity matrix.

Computes all pairs at once with matrix algebra rather than a Python loop, which is what makes fingerprint similarity searches over tens of thousands of molecules practical.

For binary inputs the intersection is X @ Y.T and the union is |x| + |y| - intersection. For count inputs the MinMax form is computed in chunks.

Parameters:
Returns:

Pairwise similarities in [0, 1].

Return type:

ndarray[tuple[Any, ...], dtype[double]]

Examples

>>> S = tanimoto_similarity_matrix(np.array([[1, 1, 0], [1, 0, 0]]))
>>> S.shape
(2, 2)
>>> bool(np.allclose(np.diag(S), 1.0))
True
qsarkit.neighbors.is_binary(X)[source]

Return True if every entry of X is 0 or 1.

Parameters:

X (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]]) – Array to inspect.

Returns:

Whether the array holds only 0/1 values.

Return type:

bool

Examples

>>> is_binary(np.array([[0, 1], [1, 0]]))
True
>>> is_binary(np.array([[0, 2]]))
False
class qsarkit.neighbors.JaccardNeighborSearch(n_neighbors=5, chunk_size=1024)[source]

Bases: BaseEstimator

Nearest-neighbour search over fingerprints under Jaccard/Tanimoto distance.

This is the similarity-search primitive behind chemical-similarity applicability domains, read-across, and virtual screening triage. It is deliberately exact (brute force over a vectorized similarity matrix) rather than approximate: chemical fingerprint spaces are high-dimensional and sparse, where tree-based indices degrade to linear scans anyway.

Parameters:

  • n_neighbors (int) – Default number of neighbours returned by kneighbors().

  • chunk_size (int) – Number of query rows processed per block, bounding peak memory at chunk_size x n_train floats.

Variables:

  • X (ndarray of shape (n_samples, n_features)) – The fitted reference fingerprints.

  • n_features_in (int) – Number of features seen during fit().

Examples

>>> import numpy as np
>>> X = np.array([[1, 1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 1]])
>>> search = JaccardNeighborSearch(n_neighbors=2).fit(X)
>>> dist, idx = search.kneighbors(np.array([[1, 1, 0, 0]]))
>>> int(idx[0, 0])
0
>>> float(dist[0, 0])
0.0

References

X_: ndarray[tuple[Any, ...], dtype[float64]]
n_features_in_: int
fit(X, y=None)[source]

Store the reference fingerprint matrix.

Parameters:
Returns:

The fitted estimator.

Return type:

JaccardNeighborSearch

kneighbors(X, n_neighbors=None, return_distance=True)[source]

Find the k most similar reference fingerprints for each query.

Parameters:
Return type:

Union[Tuple[ndarray[tuple[Any, ...], dtype[double]], ndarray[tuple[Any, ...], dtype[int_]]], ndarray[tuple[Any, ...], dtype[int_]]]

Returns:

  • distances (ndarray of shape (n_queries, n_neighbors)) – Jaccard distances, ascending.

  • indices (ndarray of shape (n_queries, n_neighbors)) – Indices into the fitted reference set.

radius_neighbors(X, radius=0.3)[source]

Find all reference fingerprints within radius Jaccard distance.

Parameters:
Return type:

Tuple[list[ndarray[tuple[Any, ...], dtype[double]]], list[ndarray[tuple[Any, ...], dtype[int_]]]]

Returns:

  • distances (list of ndarray) – Per-query distances to the in-radius neighbours, ascending.

  • indices (list of ndarray) – Per-query reference indices.

Classic similarity search: hits above a Tanimoto threshold.

Parameters:

  • X (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]]) – Query fingerprints.

  • threshold (float) – Minimum Tanimoto similarity for a hit. 0.7 on ECFP4 is the long-standing rule-of-thumb cutoff for “similar” molecules.

  • max_hits (Optional[int]) – Truncate each query’s hit list to this many best hits.

Returns:

Per-query hits sorted by descending similarity.

Return type:

list[list[Tuple[int, float]]]

References

class qsarkit.neighbors.JaccardKNeighborsClassifier(n_neighbors=5, weights='uniform', chunk_size=1024)[source]

Bases: ClassifierMixin, _BaseJaccardKNN

k-nearest-neighbours classifier under Jaccard/Tanimoto distance.

The canonical similarity-based QSAR classifier: a compound is predicted from the observed classes of its most structurally similar training neighbours. Using Tanimoto rather than Euclidean distance matters because fingerprints are sparse binary vectors, where Euclidean distance is dominated by the (huge) number of jointly-absent bits — exactly the bits that carry no chemical information.

Parameters:

  • n_neighbors (int) – Number of neighbours voting on each prediction.

  • weights (Literal['uniform', 'distance', 'similarity']) – Vote weighting. "similarity" weights each neighbour by its Tanimoto coefficient; "distance" by 1/d (with exact matches taking the row outright).

  • chunk_size (int) – Query block size for the similarity computation.

Variables:

  • classes (ndarray of shape (n_classes,)) – Sorted class labels seen during fit().

  • n_features_in (int) – Number of features seen during fit().

Examples

>>> import numpy as np
>>> X = np.array([[1, 1, 0, 0], [1, 1, 1, 0], [0, 0, 1, 1], [0, 0, 1, 0]])
>>> y = np.array([1, 1, 0, 0])
>>> clf = JaccardKNeighborsClassifier(n_neighbors=1).fit(X, y)
>>> int(clf.predict(np.array([[1, 1, 0, 0]]))[0])
1

References

classes_: ndarray[tuple[Any, ...], dtype[generic]]
fit(X, y)[source]

Store training fingerprints and labels.

Parameters:
Returns:

The fitted estimator.

Return type:

JaccardKNeighborsClassifier

predict_proba(X)[source]

Class probabilities as the weighted neighbour vote share.

Parameters:

X (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]])

Returns:

Rows sum to 1.

Return type:

ndarray[tuple[Any, ...], dtype[double]]

predict(X)[source]

Predict the majority (weighted) class of each query’s neighbours.

Parameters:

X (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]])

Returns:

Predicted labels drawn from classes_.

Return type:

ndarray[tuple[Any, ...], dtype[generic]]

set_score_request(*, sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object

class qsarkit.neighbors.JaccardKNeighborsRegressor(n_neighbors=5, weights='uniform', chunk_size=1024)[source]

Bases: RegressorMixin, _BaseJaccardKNN

k-nearest-neighbours regressor under Jaccard/Tanimoto distance.

Predicts a continuous endpoint (pIC50, logS, …) as the weighted mean of its structurally nearest training neighbours — the numerical form of read-across, and a strong similarity-only baseline that any fitted QSAR model should be expected to beat.

Parameters:

  • n_neighbors (int) – Number of neighbours averaged for each prediction.

  • weights (Literal['uniform', 'distance', 'similarity']) – Averaging weights, as in JaccardKNeighborsClassifier.

  • chunk_size (int) – Query block size for the similarity computation.

Variables:

n_features_in (int) – Number of features seen during fit().

Examples

>>> import numpy as np
>>> X = np.array([[1, 1, 0, 0], [1, 1, 1, 0], [0, 0, 1, 1]])
>>> y = np.array([7.0, 6.5, 4.0])
>>> reg = JaccardKNeighborsRegressor(n_neighbors=1).fit(X, y)
>>> float(reg.predict(np.array([[1, 1, 0, 0]]))[0])
7.0

References

fit(X, y)[source]

Store training fingerprints and endpoint values.

Parameters:
Returns:

The fitted estimator.

Return type:

JaccardKNeighborsRegressor

predict(X)[source]

Predict as the weighted mean of the neighbours’ target values.

Parameters:

X (Union[Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]])

Returns:

Predicted values.

Return type:

ndarray[tuple[Any, ...], dtype[double]]

set_score_request(*, sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object

References

  • Willett, P. (2006). “Similarity-Based Virtual Screening Using 2D Fingerprints.” Drug Discov. Today, 11(23-24), 1046-1053. doi:10.1016/j.drudis.2006.10.005

  • Bajusz, D., Racz, A. & Heberger, K. (2015). “Why Is Tanimoto Index an Appropriate Choice for Fingerprint-Based Similarity Calculations?” J. Cheminform., 7, 20. doi:10.1186/s13321-015-0069-3

  • Johnson, M. A. & Maggiora, G. M. (1990). “Concepts and Applications of Molecular Similarity.” Wiley.