scikit-learn
ELI5 supports many estimators, transformers and other components from the scikit-learn library.
Additional explain_weights
and explain_prediction
parameters
For all supported scikit-learn classifiers and regressors
eli5.explain_weights()
and eli5.explain_prediction()
accept
additional keyword arguments. Additional eli5.explain_weights()
parameters:
vec
is a vectorizer instance used to transform raw features to the input of the classifier or regressor (e.g. a fitted CountVectorizer instance); you can pass it instead offeature_names
.
Additional eli5.explain_prediction()
parameters:
vec
is a vectorizer instance used to transform raw features to the input of the classifier or regressor (e.g. a fitted CountVectorizer instance); you can pass it instead offeature_names
.vectorized
is a flag which tells eli5 ifdoc
should be passed throughvec
or not. By default it is False, meaning that ifvec
is not None,vec.transform([doc])
is passed to the estimator. Set it to True if you’re passingvec
(e.g. to get feature names and/or enable text highlighting), butdoc
is already vectorized.
Linear estimators
For linear estimators eli5 maps coefficients back to feature names directly. Supported estimators from sklearn.linear_model:
Linear SVMs from sklearn.svm
are also supported:
SVC (only with
kernel='linear'
, only for binary classification)SVR (only with
kernel='linear'
)NuSVC (only with
kernel='linear'
, only for binary classification)NuSVR (only with
kernel='linear'
)OneClassSVM (only with
kernel='linear'
)
For linear scikit-learn classifiers eli5.explain_weights()
supports
one more keyword argument, in addition to common argument and extra arguments
for all scikit-learn estimators:
coef_scale
is a 1D np.ndarray with a scaling coefficient for each feature; coef[i] = coef[i] * coef_scale[i] if coef_scale[i] is not nan. Use it if you want to scale coefficients before displaying them, to take input feature sign or scale in account.
Note
Top-level eli5.explain_weights()
and eli5.explain_prediction()
calls are dispatched to these functions for linear scikit-learn estimators:
Decision Trees, Ensembles
eli5 supports the following tree-based estimators from sklearn.tree
:
eli5.explain_weights()
computes feature importances and prepares
tree visualization; eli5.show_weights()
may visualizes a tree
either as text or as image (if graphviz is available).
For DecisionTreeClassifier and DecisionTreeRegressor
additional eli5.explain_weights()
keyword arguments
are forwarded to sklearn.tree.export_graphviz function when graphviz
is available; they can be used to customize tree image.
Note
For decision trees top-level eli5.explain_weights()
calls are
dispatched to eli5.sklearn.explain_weights.explain_decision_tree()
.
The following tree ensembles from sklearn.ensemble
are supported:
For ensembles eli5.explain_weights()
computes feature importances
and their std deviation.
Note
For ensembles top-level eli5.explain_weights()
calls are
dispatched to eli5.sklearn.explain_weights.explain_rf_feature_importance()
.
eli5.explain_prediction()
is less straightforward for ensembles and
trees; eli5 uses an approach based on ideas from
http://blog.datadive.net/interpreting-random-forests/ :
feature weights are calculated by following decision paths in trees
of an ensemble (or a single tree for DecisionTreeClassifier and
DecisionTreeRegressor). Each node of the tree has an output score, and
contribution of a feature on the decision path is how much the score changes
from parent to child.
There is a separate package for this explaination method (https://github.com/andosa/treeinterpreter); eli5 implementation is independent.
Note
For decision trees and ensembles eli5.explain_prediction()
calls are dispatched to
eli5.sklearn.explain_prediction.explain_prediction_tree_classifier()
and eli5.sklearn.explain_prediction.explain_prediction_tree_regressor()
.
Transformation pipelines
eli5.explain_weights()
can be applied to a scikit-learn Pipeline as
long as:
explain_weights
is supported for the final step of the Pipeline;eli5.transform_feature_names()
is supported for all preceding steps of the Pipeline. singledispatch can be used to registertransform_feature_names
for transformer classes not handled (yet) by ELI5 or to override the default implementation.
For instance, imagine a transformer which selects every second feature:
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils.validation import check_array
from eli5 import transform_feature_names
class OddTransformer(BaseEstimator, TransformerMixin):
def fit(self, X, y=None):
# we store n_features_ for the sake of transform_feature_names
# when in_names=None:
self.n_features_ = check_array(X).shape[1]
return self
def transform(self, X):
return check_array(X)[:, 1::2]
@transform_feature_names.register(OddTransformer)
def odd_feature_names(transformer, in_names=None):
if in_names is None:
from eli5.sklearn.utils import get_feature_names
# generate default feature names
in_names = get_feature_names(transformer, num_features=transformer.n_features_)
# return a list of strings derived from in_names
return in_names[1::2]
# Now we can:
# my_pipeline = make_pipeline(OddTransformer(), MyClassifier())
# my_pipeline.fit(X, y)
# explain_weights(my_pipeline)
# explain_weights(my_pipeline, feature_names=['a', 'b', ...])
Note that the in_names != None
case does not need to be handled as long as the
transformer will always be passed the set of feature names either from
explain_weights(my_pipeline, feature_names=...)
or from the previous step
in the Pipeline.
Currently the following transformers are supported out of the box:
any transformer which provides
.get_feature_names()
method;nested FeatureUnions and Pipelines;
SelectorMixin-based transformers: SelectPercentile, SelectKBest, GenericUnivariateSelect, VarianceThreshold, RFE, RFECV, SelectFromModel, RandomizedLogisticRegression;
scalers from sklearn.preprocessing: MinMaxScaler, StandardScaler, MaxAbsScaler, RobustScaler.
Reversing hashing trick
eli5 allows to recover feature names for HashingVectorizer and FeatureHasher
by computing hashes for the provided example data.
eli5.explain_prediction()
handles HashingVectorizer as vec
automatically; to handle HashingVectorizer and FeatureHasher for
eli5.explain_weights()
, use
InvertableHashingVectorizer
or
FeatureUnhasher
:
# vec is a HashingVectorizer instance
# clf is a classifier which works on HashingVectorizer output
# X_sample is a representative sample of input documents
import eli5
from eli5.sklearn import InvertableHashingVectorizer
ivec = InvertableHashingVectorizer(vec)
ivec.fit(X_sample)
# now ``ivec.get_feature_names()`` returns meaningful feature names,
# and ``ivec`` can be used as a vectorizer for eli5.explain_weights:
eli5.explain_weights(clf, vec=ivec)
HashingVectorizer is also supported inside a FeatureUnion:
eli5.explain_prediction()
handles this case automatically, and for
eli5.explain_weights()
you can use eli5.sklearn.unhashing.invert_hashing_and_fit()
(it works for plain HashingVectorizer too) - it tears FeatureUnion apart,
inverts and fits all hashing vectorizers and returns a new FeatureUnion:
from eli5.sklearn import invert_hashing_and_fit
ivec = invert_hashing_and_fit(vec, X_sample)
eli5.explain_weights(clf, vec=ivec)
Text highlighting
For text data eli5.explain_prediction()
can show the input document
with its parts (tokens, characters) highlighted according to their
contribution to the prediction result:

It works if the document is vectorized using
CountVectorizer, TfIdfVectorizer or HashingVectorizer, and a fitted
vectorizer instance is passed to eli5.explain_prediction()
in a vec
argument. Custom preprocessors are supported, but custom
analyzers or tokenizers are not: highligting works only with ‘word’, ‘char’
or ‘char_wb’ analyzers and a default tokenizer (non-default token_pattern
is supported).
Text highlighting also works if a document is vectorized using FeatureUnion with at least one of CountVectorizer, TfIdfVectorizer or HashingVectorizer in the transformer list; features of other transformers are displayed in a regular table.
See also: Debugging scikit-learn text classification pipeline tutorial.
OneVsRestClassifier
eli5.explain_weights()
and eli5.explain_prediction()
handle
OneVsRestClassifier by dispatching to the explanation function for
OvR base estimator, and then calling this function for the
OneVsRestClassifier instance. This works in many cases, but not for all.
Please report issues to https://github.com/eli5-org/eli5/issues.