Seminar: Explainable Machine Learning (WS 2021/2022)

Content

Current publications on machine learning / computer vision are covered in this seminar. In particular, topics from the area of explainable machine learning (in particular vision and language based deep learning, attention models, transformers) are in focus.

This is a Master's level course. Since these topics are very complex, prior participation in at least one of the following lectures is required:

• Deep Learning
• Probabilistic Machine Learning
• Statistical Machine Learning

Organisation

The schedule of the seminar is as follows:

• October 20th, 2-6pm
• October 27th, 2-6pm
• November 3rd, 2-6pm
• November 10th, 2-6pm
• November 17th, 2-6pm
• November 24th, 2-6pm
• December 1st, 2-6pm

All seminars will take place on zoom. All the accepted participants will receive the zoom link on their email that they used in ILIAS.

The course awards 3 LP Credits.

Requirements

A successful participation in the seminar includes:

• Active participation in the entire event: We have 70% attendance policy for this seminar. You need to attend at least 5 of the 7 sessions.
• Short presentation on October 27th or November 3rd (10 minutes talk, 5 min questions)
• Presentation on November 10th, November 17th, November 24th or December 1st (20 minutes talk, 10 minutes questions) on a selected topic

Topics to be covered

Interpretability in psychology and cognitive sciences:

• The structure and function of explanations, Tania Lombrozo, Trends in Cognitive Sciences, 2006
• Rational quantitative attribution of beliefs, desires and percepts in human mentalizing, Baker etal, Nature Human Behaviour 2017
• Explanation in Artificial Intelligence: Insights from the social sciences, Tim Miller, Artificial Intelligence, Elsevier, 2019

Machine Attention as Explanations in Computer Vision:

• Learning Deep Features for Discriminative Localization,Zhou etal, CVPR 2016
• Grad-cam: Visual explanations from deep networks via gradient-based localization, Selveraju etal, ICCV 2017
• Visualizing Deep Neural Network Decisions: Prediction difference Analysis, Zintgraf et al.ICLR 2017
• Attribute Prototype Network for Zero-Shot Learning, Xu etal, NeurIPS 2020
• Keep CALM and Improve Visual Feature Attribution, Kim etal, ICCV 2021

Communication-based learning for natural image data:

• Visual Dialog, Das etal, CVPR 2017
• Improving Generative Visual Dialog by Answering Diverse Questions, Murahari etal, EMNLP 2019
• Learning Decision Trees Recurrently Through Communication, Alaniz et.al, CVPR 2021
• Machine Theory of Mind, Rabinowitz et al., ICML 2018
• Modeling Conceptual Understanding in Image Reference Games, Corona etal, NeurIPS 2019

Generating Natural Language Explanations in Computer Vision:

• Generating Visual Explanations, Hendricks etal, ECCV 2016
• Grounding Visual Explanations, Hendricks etal, ECCV 2018
• Textual Explanations for Self-Driving Vehicles, Kim etal, ECCV 2018
• Multimodal Explanations: Justifying Decisions and Pointing to the Evidence, Park etal, CVPR 2018
• Faithful Multimodal Explanation for Visual Question Answering, Wu, Mooney at ACL 2019
• Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs; Marasovic et.al. EMNLP 2020
• e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks, Kayser etal., ICCV 2021

Compositional Learning:

• Attributes as operators: factorizing unseen attribute-object compositions, Tushar and Grauman, ECCV 2018
• Task-driven modular networks for zero-shot compositional learning, Purushwalkam etal, ICCV 2019
• A causal view of compositional zero-shot recognition, Atzman etal, NeurIPS 2020
• Learning Graph Embeddings for Compositional Zero-shot Learning, Naeem etal, CVPR 2021
• Open World Compositional Zero-Shot Learning, Mancini etal, CVPR 2021

Registration

The registration opens on October 1st via ILIAS.