NeurIPS 2022


This paper tackles post-hoc interpretability for audio processing networks. Our goal
is to interpret decisions of a trained network in terms of high-level audio objects
that are also listenable for the end-user. To this end, we propose a novel interpreter
design that incorporates non-negative matrix factorization (NMF). In particular,
a regularized interpreter module is trained to take hidden layer representations
of the targeted network as input and produce time activations of pre-learnt NMF
components as intermediate outputs. Our methodology allows us to generate
intuitive audio-based interpretations that explicitly enhance parts of the input signal
most relevant for a network’s decision. We demonstrate our method’s applicability
on popular benchmarks, including a real-world multi-label classification task.


J. Parekh, S. Parekh, P. Mozharovskyi, F. d’Alché-Buc and G. Richard (2022). Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF. 36th Conference on Neural Information Processing Systems (NeurIPS 2022).