2022-05289 - PhD Position F/M Interpretable Deep Learning for image-based species identification
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
About the research centre or Inria department
The Inria Sophia Antipolis - Méditerranée center counts 34 research teams as well as 7 support departments. The center's staff (about 500 people including 320 Inria employees) is made up of scientists of different nationalities (250 foreigners of 50 nationalities), engineers, technicians and administrative staff. 1/3 of the staff are civil servants, the others are contractual agents. The majority of the center’s research teams are located in Sophia Antipolis and Nice in the Alpes-Maritimes. Four teams are based in Montpellier and two teams are hosted in Bologna in Italy and Athens. The Center is a founding member of Université Côte d'Azur and partner of the I-site MUSE supported by the University of Montpellier.
Context
Methodological context: XAI
With data-driven AI methods becoming prevalent in more and more aspects of our life, there is a growing effort within the AI community to ensure that these methods do not only perform well, but also communicate to humans some elements of the learned internal reasoning that leads to a particular output. This effort towards Explainable AI (XAI) [1] aims at ensuring that humans can understand the process within the AI model, ultimately allowing for a more human-centric AI. The vast majority of methods, those based on feature attribution, aim at pointing out which parts of the input are most responsible for the output, potentially resulting in an ambiguous interpretation that may end up being misleading [2, 3]. We want to explore how to achieve richer and more useful types of explanations, such as those using prototypical examples [4] or natural language [5], and explore the impact that these explanations have on the users in terms of trust and of how much the users are able to learn from the system. In addition, rich explanations, such as those in the form of visual characteristics in natural language, can be designed to provide an interpretable representation of each data sample that can be leveraged for few-shot and zero-shot learning, as long as the characteristics of new classes are known.
Application context: Species ID and biodiversity mapping
The methods proposed within this project will be devoted to the analysis of real world data for biodiversity ecosystems assessment and monitoring at different spatial scales. The first task is related to the automatic identification of plant species in user-generated photographs with the goal to pave the way towards better Species Distribution Models [6] and even enable us to monitor fine-grained trends in species distribution shifts caused by climate change or other anthropogenic pressures. The computer vision models for species identification used by platforms such as iNaturalist [7] and Pl@ntNet [8] have improved enormously in the last few years and can often be trusted when presented with high quality photographs of data-rich species. However, they struggle with images taken under suboptimal conditions (bad lighting, blurriness, partial news, etc.) or of species for which not many images are available. In such cases an incorrect automatic label may contribute more to adding noise to the platform than no label at all, since new users tend to overtrust [9] the computer vision model. New computer vision methods are needed that perform better on identifying uncommon species and that allow users to understand when they should be trusted and when not. We will develop new methods that leverage species descriptions based on morphological traits in order to achieve these goals.The second task concerns the analysis of remote sensing imagery to predict habitat (or landscape) characteristics. In a similar fashion to how morphological traits can be used to describe species, these landscape characteristics can be used to describe specific habitats. We will thus explore the development of description-based XAI methods that are useful at both scales.
Assignment
In order to improve the performance on data-poor species and to help users gauge which automatic predictions to trust, we aim at designing new computer vision methods that work more like an expert taxonomist or ecologist: by first looking for multiple morphological traits on the image and then finding which species are compatible with the observed traits in the case of species identification, or by looking for habitat characteristics in the case of habitat characterization. To fulfill this, the project will pursue two different methodological goals: (1) explore the use of natural language bottlenecks describing visible traits or other visual characteristics and (2) design computer vision models that discover discriminative visual characteristics in an unsupervised manner.
Main activities
Within the first goal, the first task will consist of exploiting a recently created dataset of morphological traits per species for several tens of thousands of species, obtained automatically by parsing textual species descriptions from the web. The noisy nature of this dataset will require the development of noise robust vision/text models that learn to detect traits on images. This will be followed by the development of an additional model that is able to infer species probabilities by looking at the detected traits.
The second goal will help us discover visual traits that, although important for the identification, have not been included in the original trait dataset. To do this, we will work towards the development of computer vision architectures that are able to discover trait-like visual characteristics, both in a weakly supervised setting, using only images and their species labels, and by leveraging the existing trait annotations.
Skills
Technical skills and level required :
- Good command of Python programming.
- Understanding of Deep Learning methods (incl. good basis of algebra, calculus, etc.)
- Experience with Pytorch, Tensorflow or similar.
Languages : Fluency in spoken and written English.
Relational skills : We are seeking a self-motivated PhD candidate, able to work autonomously while collaborating with the rest of the team.
Other values appreciated : We value passion for both method development and the advancement of biodiversity knowledge.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
- Supplementary social protection
Remuneration
Duration: 36 months
Gross Salary per month: 2051€ per month (year 1 & 2) and 2158€ per month (year 3)
General Information Theme/Domain : Vision, perception and multimedia interpretation Town/city : Montpellier Inria Center : CRI Sophia Antipolis - Méditerranée Starting date : 2022-12-01 Duration of contract : 3 years Deadline to apply : 2022-10-12 Contacts Inria Team : ZENITH PhD Supervisor : Marcos Gonzalez Diego / [email protected] About Inria Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, oBen at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact. Instruction to apply Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment. Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities. Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.