Project Title: Deception Detection Using a Multimodal Approach

Faculty Sponsor: Mihai Burzo

Department: Computer Science, Engineering, & Physics



Project Description:  The goal of this project is to use physiological, visual, thermal, and linguistic signals to detect lying. Both infrared (thermal) and visible cameras along with sensors that can detect heart rate, respiration rate, galvanic skin response, skin temperature  will be employed to identify relevant physiological features that can be used for lie detection. The project will explore a new generation of computational tools for joint modeling of physiological, visual, thermal, and linguistic signals of human deceptive behavior. To achieve this goal, we target the following three research objectives. First, we will build a novel multimodal dataset of deceit covering different domains by creating several deception scenarios. Second, we will explore rule-based classifiers for deception detection using physiological features (e.g., heart rate, respiration rate, galvanic skin response, skin temperature), visual, thermal, and linguistic features. Third, we will develop data-driven learning approaches for multimodal deception detection, taking advantage of the recent progress in early, late, and temporal fusion models.

This project is a UM Flint – UM Ann Arbor collaboration and is IRB-approved.

Student Tasks & Responsibilities:

Key Student Tasks & Responsibilities:

  • Distribute recruiting materials and help with finding participants for the study.
  • Collect multimodal data.
  • Help with data analysis and processing.
  • Attend weekly project meetings
  • Prepare articles, reports, and presentations • Manage and respond to project related email.
  • Prepare a final report of the project findings.

Additional (Optional) Tasks & Responsibilities:

  • Prepare materials for submission to granting agencies and foundations.
  • Request or acquire equipment or supplies necessary for the project.
  • Attend area seminars and other meetings as necessary.

Minimum Qualifications: Engineering students that have completed at least one-year towards a BSE in Mechanical Engineering (or Engineering) Degree