XAI for DNNs

There are several explainability models for deep neural networks that have been proposed in recent years. Some of the most popular ones are:

Layer-wise Relevance Propagation (LRP): LRP is a model-agnostic approach that assigns relevance scores to each neuron in a deep neural network by recursively propagating relevance scores from the output layer to the input layer.
Integrated Gradients: Integrated Gradients is a method for attributing feature importance to each input feature of a deep neural network by computing the integral of the gradients of the output with respect to the input along a straight-line path from a baseline input to the actual input.
Grad-CAM: Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique for visualizing which parts of an input image were most important for a specific output class in a convolutional neural network. It does this by computing the gradients of the output class with respect to the feature maps of the last convolutional layer, and using these gradients to weight the feature maps.
SHAP (SHapley Additive exPlanations): SHAP is a model-agnostic method for explaining the output of any machine learning model, including deep neural networks. It uses game theory to assign feature importance scores to each input feature based on their contribution to the output.
DeepLIFT (Deep Learning Important FeaTures): DeepLIFT is a model-agnostic approach that computes the contribution of each input feature to the output by comparing the activation of each neuron in the network to a reference activation.

These are just a few examples of the many explainability models that have been proposed for deep neural networks. Each of these models has its own strengths and weaknesses, and the choice of model depends on the specific use case and the desired level of explainability.

Full List of Models

A comprehensive list of DNN XAI models was developed by the Ohio Federal Research Network in a paper titled Explainable Deep Learning: A Field Guide for the Uninitiated by Gabrielle Ras, Ning Xie, Marcel van Gerven and Derek Doran. This paper gives a very thorough overview of many options that are available and what data types they are applicable for found on pages 36 and 37.