eli5.keras

eli5 has Keras support - eli5.explain_prediction() explains predictions of image classifiers by using an impementation of Grad-CAM (Gradient-weighted Class Activation Mapping, https://arxiv.org/pdf/1610.02391.pdf). The function works with both Sequential model and functional Model.

eli5.keras.explain_prediction

explain_prediction_keras(estimator, doc, target_names=None, targets=None, layer=None)[source]

Explain the prediction of a Keras image classifier.

We make two explicit assumptions
  • The input is images.
  • The model’s task is classification, i.e. final output is class scores.

See eli5.explain_prediction() for more information about the estimator, doc, target_names, and targets parameters.

Parameters:
  • estimator (keras.models.Model) – Instance of a Keras neural network model, whose predictions are to be explained.
  • doc (numpy.ndarray) –

    An input image as a tensor to estimator, from which prediction will be done and explained.

    Currently only numpy arrays are supported.

    The tensor must be of suitable shape for the estimator.

    For example, some models require input images to be rank 4 in format (batch_size, dims, …, channels) (channels last) or (batch_size, channels, dims, …) (channels first), where dims is usually in order height, width and batch_size is 1 for a single image.

    Check estimator.input_shape to confirm the required dimensions of the input tensor.

    raises TypeError:
     if doc is not a numpy array.
    raises ValueError:
     if doc shape does not match.
  • target_names (list, optional) – Not Implemented. Names for classes in the final output layer.
  • targets (list[int], optional) –

    Prediction ID’s to focus on.

    Currently only the first prediction from the list is explained. The list must be length one.

    If None, the model is fed the input image and its top prediction is taken as the target automatically.

    raises ValueError:
     if targets is a list with more than one item.
    raises TypeError:
     if targets is not list or None.
  • layer (int or str or keras.layers.Layer, optional) –

    The activation layer in the model to perform Grad-CAM on: a valid keras layer name, layer index, or an instance of a Keras layer.

    If None, a suitable layer is attempted to be retrieved. The layer is searched for going backwards from the output layer, checking that the rank of the layer’s output equals to the rank of the input.

    raises TypeError:
     if layer is not None, str, int, or keras.layers.Layer instance.
    raises ValueError:
     if suitable layer can not be found.
    raises ValueError:
     if differentiation fails with respect to retrieved layer.
Returns:

expl (eli5.base.Explanation) –

An eli5.base.Explanation object with the following attributes:
  • image a Pillow image with mode RGBA.
  • targets a list of eli5.base.TargetExplanation objects for each target. Currently only 1 target is supported.
The eli5.base.TargetExplanation objects will have the following attributes:
  • heatmap a rank 2 numpy array with floats in interval [0, 1] with the localization map values.
  • target ID of target class.
  • score value for predicted class.

eli5.keras.gradcam

gradcam(weights, activations)[source]

Generate a localization map (heatmap) using Gradient-weighted Class Activation Mapping (Grad-CAM) (https://arxiv.org/pdf/1610.02391.pdf).

The values for the parameters can be obtained from eli5.keras.gradcam.gradcam_backend().

Parameters:
  • weights (numpy.ndarray) – Activation weights, vector with one weight per map, rank 1.
  • activations (numpy.ndarray) – Forward activation map values, vector of matrices, rank 3.
Returns:

lmap (numpy.ndarray) – A Grad-CAM localization map, rank 2, with values normalized in the interval [0, 1].

Notes

We currently make two assumptions in this implementation
  • We are dealing with images as our input to estimator.
  • We are doing a classification. estimator’s output is a class scores or probabilities vector.
Credits
gradcam_backend(estimator, doc, targets, activation_layer)[source]

Compute the terms and by-products required by the Grad-CAM formula.

Parameters:
  • estimator (keras.models.Model) – Differentiable network.
  • doc (numpy.ndarray) – Input to the network.
  • targets (list, optional) – Index into the network’s output, indicating the output node that will be used as the “loss” during differentiation.
  • activation_layer (keras.layers.Layer) – Keras layer instance to differentiate with respect to.

See eli5.keras.explain_prediction() for description of the estimator, doc, targets parameters.

Returns:(weights, activations, gradients, predicted_idx, predicted_val) ((numpy.ndarray, …, int, float)) – Values of variables.