About

LIMPID (Leveraging Interpretable Machines for Performance Improvement and Decision) is a threeyear interdisciplinary research program started in December 2020. It is conducted by Télécom Paris and IDEMIA, and funded in part by the French National Research Agency (ANR).

About

Our objective

Achieving trustworthy image recognition systems by design is
based on:

  • Methods to test for bias, and to apply bias mitigation measures to image recognition systems, including facial recognition,
  • Develop approaches to explainability by design for various image recognition use cases,
  • Confront bias-mitigation and explainability by design with regulatory requirements for fair and explainable image recognition, and identify gaps between regulatory requirements and proposed technical solutions.

Studying trustworthiness through two generic uses cases

LIMPID addresses the four main pillars of trustworthiness—reliability, robustness, fairness, explainability—
through two generic use cases:

Use case #1
In an image recognition context

We want to detect automatically people exhibiting certain types of behavior considered as incorrect. This consists of two binary classification tasks (infringement or not), performed using deep convolutional neural network (DCNN) trained on millions of labeled images (AI system). The challenges related to this class of use case are fourfold:

  • This classification is unbalanced, both in learning and in production: indeed, a small percentage of people exhibits the targeted behavior. Alarms must be raised for the minority class, and false alarms should be limited: alarms should be raised only if the AI system is confident enough. A false positive (identifying the person as faulty when she has a correct behavior) has not the same impact than a false negative.
  • On the field, when the system is used in production, we suppose there is a massive visibility issue. Reflection on surrounding objects, occlusions, variations of point of view towards the person to be observed, similar color on relevant objects on the scene and the overall weather conditions are among the challenges making the classification task more difficult. Here we tackle the reliability of our AI system, which must work properly with a range of inputs and in a range of situations.
  • As eventually a positive match should lead to the automatic generation of a legal document, people targeted by the AI system must be able to understand the reason of the decision (explainability). This is a mandatory for system deployment.
  • Legal and ethical requirements are extremely high.
Read moreless
Use case #2
In a face recognition context

Deep convolutional neural network are also used within LIMPID to identify people by comparing them automatically to a previously stored picture. The main challenge here is fairness: the algorithm should behave in the same way whatever the visible characteristic of the people are (age, sex, skin color, nationality…).

Face recognition is a projection problem and not a classification problem. DCNNs for face encoding learn how to project images of faces in a high dimension space The projection process should not put all the blurry images or non-faces in the same part of the space or it would lead to high impostor scores for images from this category.

The influence of blur on the position in a high dimension space and the loss functions to constrain this position having not yet been widely studied, LIMPID is working on increasing the robustness of the AI systems facing such considerations.

At last, explainability is also of prime importance. For instance, the system operator wants to verify the merits of the decision and the oversight bodies with the help of technical staff may want to be able to dive into the system decision in order to ensure fairness of the final decision if it is discussed.

From the perspective of trustworthiness, both use cases are highly relevant. In both cases:

  • All people must be treated equally, whatever their physical characteristics are.
  • Automatic decisions can harm people.
  • Security and safety are at stake.
Read moreless

Back and forth between
technical solutions
and legal considerations

Each of the use cases is supposed to eventually require a government order, or perhaps even a specific law. This can happen only if the quality of the algorithms can be demonstrated, including reliability, robustness, level of local interpretability, absence of discrimination, the rate of false positives, and level of human control and oversight. The LIMPID project helps formulate the legal/ethical requirements for reliability, fairness and local explainability, and then design or adapt tools to meet the identified requirements. Iteratively, each proposed technical solution is refined based on feedback from the legal specialists.

Based on international benchmarks and examples from use cases similar to ours, LIMPID develops and discusses with institutional stakeholders an initial set of legal and ethical requirements that one could reasonably expect to apply to an image recognition tool in our specific use cases. The requirements are refined as we experiment with different approaches to reliability and as a function of stakeholder input. At the end of the project, a final list of requirements will be proposed, and a gap analysis performed with the technical reliability solutions.

3 research themes of trustworthiness

LIMPID addresses the main pillars of trustworthiness in AI along three axes, intertwined and studied through the above use cases: reliability & robustness, fairness, explainability.

Explainability

Explainability

Providing a general learning framework—joint learning of a predictive model and an explanatory model—for explainability by design, applicable to different use cases
More information
Fairness

Fairness

Developing a sound validity framework for fairness definition, measurement and improvement, towards the insertion of fairness constraints into algorithms
More information
Reliability & Robustness

Reliability & Robustness

Devising new methods to model’s prediction reliability, including the ability to abstain from making a decision, and to match reliability tools with relevant legal/ethical requirements
More information

Team & Partners

LIMPID (Leveraging Interpretable Machines for Performance Improvement and Decision) is a three-year interdisciplinary research program started in December 2020. It is conducted by Télécom Paris and mainly funded in part by the French National Research Agency (ANR, Project 20-CE23-0028). IDEMIA is scientific coordinator of the project.