Affectiva Automotive AI

Affectiva’s In-Cabin Sensing redefines the occupant experience and improves road safety

The Automotive Challenge

Consumer expectations are evolving. With the ubiquity of mobile devices, information and entertainment are at our fingertips. Consumers expect comfortable and entertaining transportation experiences that are tuned to their needs in the moment. Safety will continue to be a priority. Euro NCAP requirements are accelerating the deployment of cameras and sensors to detect driver impairment and a child left behind. To enable these advanced safety features and to deliver personalized transportation experiences, car manufacturers need a deep understanding of what's happening in a vehicle.

The Affectiva Solution

Affectiva Automotive AI is an In-Cabin Sensing (ICS) platform that understands what is happening inside of a vehicle. Affectiva's patented software uses in-vehicle cameras to measure in real time, the state of the cabin, and that of the driver and occupants in it. From complex and nuanced emotions and cognitive states, such as drowsiness and distraction, to activity, object and child presence detection. OEMs, Tier 1s, ridesharing providers and fleet management companies can deliver optimized occupant experiences and advanced safety features using Affectiva's AI.

“Moving forward, transparency will play an important role in building trust in the AI systems that will slowly find their way into our cars—if we choose to trust it at all. El Kaliouby stresses that automakers and mobility services providers must be clear about in-cabin sensing technology and educate consumers on what the technology does, what data it collects, and how it stores and uses the data.”


Ben Dickson

“Another area of the in-vehicle experience where AI is pushing developments is in-cabin monitoring. Applications such as distraction detection are already in place in some segments, and could be critical for automakers should they wish to introduce advanced driver assistance systems which will still rely on driver attentiveness. But some companies see far greater potential than this, and AI is the key. Affectiva is one such player. Older than most start-ups, the company was founded in 2009 and has raised some US$60.3m in funding to date, with notable investors including Aptiv. Affectiva considers itself a pioneer in emotion AI, which can interpret user emotion via a number of means including vision processing of facial expressions, or audio processing of voice.”

Automotive world

Xavier Boucherat

“Advances in facial recognition technology mean machines can not only recognize different people, but also how they are feeling. This means the next generation of automobiles may contain features that scan drivers’ faces for fatigue or other signs of impairment. Companies, including Boston-based Affectiva, are already making software to help the auto industry integrate such technology.”


Jeff John Roberts

“As auto manufacturers race to build self-driving vehicles of the future, the company is betting that its technology could play a role in shaping this new world of transportation…In the future, if all goes as planned, emotion AI in cars could help calm us down, wake us up and get us mentally ready for the day’s commute or a big night out. Affectiva, which already has contracts with auto-makers and suppliers, hopes to have its technology embedded in cars on the road by 2021.”


The Forbes Insights Team

“…with the camera pointed at the safety driver, Renovo can then tell whether that person is tired or distracted, and deliver the right prompts or warnings to ensure attention remains on the road ahead. And that’s where Renovo and Affectiva’s collaboration perhaps could have prevented the fatal Uber collision last month.”


Andrew Hawkins

"The future of transportation is AI whether computers do the driving or not. We desperately need to change the way we approach the manufacture and operation of motor vehicles. Over one million people die every year in vehicle collisions. With Automotive AI, it’s possible the roads could become a little safer thanks to Affectiva and other companies working on changing things for the better."


Tristan Greene

    We work with leading OEMs and automotive suppliers:

    Porsche Porsche
    Renovo Auto Renovo Auto
    Autoliv Veoneer
    nvidia nvidia
    intel Wind River
    faurecia grey faurecia
    Nuance and Affectiva
    Driver monitoring for fleet affectiva greenroad
    aptiv-grey aptiv-color

    Affectiva's In-Cabin Sensing identifies key events related to humans

    Cabin State

    Insight into the state of the cabin helps personalize safety features (seatbelt, airbag), content (video, music and advertising), and the environment of the cabin (heating and lighting). It can also detect if a child or objects have been left behind. 

    • Detect number of people, who they are, and where they are sitting. 
    • Identify an occupied infant or child seat.
    • Identify personal objects that are left behind in a vehicle. 

    Occupant State

    Insight into the state of the occupants helps improve their comfort by adapting music, lighting, temperature. Understanding passenger engagement with content allows for personalized recommendations and for opportunities to monetize media data.

    • Detect when passengers are getting drowsy or are asleep
    • Understand occupant mood and reactions
    • Personalize and monetize passenger engagement with content

    Driver State

    Understand complex and nuanced states of driver impairment. Provide "quantified driver" services that provide insight into what driving modes, vehicles settings, routes, and times get you to your destination in the best emotional state. 

    • Detect increased levels of drowsiness in a driver
    • Detect driver distraction due to cellphone use 
    • Understand driver mood and reactions


    Affectiva's deep learning algorithms are trained and tested with our large, proprietary data set that is representative of human appearance variances and real-world use cases. Global in nature, our data is diverse in age, gender and ethnicity, helping us mitigate for bias.

    • Captured in real world conditions
    • Acquired with opt-in and consent
    • Automotive in-cabin data: 18k+ hours, 4k+ unique individuals
    • Foundational data: 9M+ faces, 87 countries, 5.1B facial frames, 6 years of video data

    Camera Position

    As the industry moves to holistic in-cabin sensing, interior cameras will be placed where they have a view of the entire cabin, including the driver and passengers in it. Overhead camera positions are proving ideal, and Affectiva has tuned its models for this location.

    • Affectiva's AI supports both RGB and NIR cameras
    • Enables OEMs to use one camera for detecting both state of the driver, state of the cabin, and state of the occupants in it
    • Overhead cameras are typically integrated in the rear view mirror or roof module

    Embedded Systems

    Affectiva's deep learning models are tuned to run on automotive Systems on Chip (SoC). We have optimized our footprint to run alongside other services, in real time, with automotive grade accuracy. Our models support TensorFlow Lite deep learning runtime.

    • Affectiva models are quantized. 
    • Embedded deep learning runtime supports ARM® or Intel CPU
    • Embedded deep learning runtime can be swapped with hardware specific runtime to make use of accelerators available in target System on Chip (SoC)

    Affectiva's Automotive AI is Getting Noticed

    Click below to see who else is as excited about it as we are:

    See All Press

    See Affectiva Automotive AI in Action

    Like what you see? This is just the beginning of a long road trip.

    Contact us to be a part of the AI-enabled vehicle revolution

    Related Content


    Balancing Challenges and Potential: the Future of Autonomous Vehicle Technology

    Affectiva's Co-Founder and CEO Dr. Rana el Kaliouby interviews Dr. Karl Iagnemma, President of Autonomous Mobility at Aptiv about how autonomous vehicle technology will transform the future of transportation and mobility.

    Read More


    How will user experience and HMI be affected by start-ups in 2020?

    Advances in connectivity and electronics, coupled with the increasing number of features built into cars, are introducing ever more advanced human-machine interfaces (HMI) and transforming the user experience (UX) for drivers. More radical transformation can be expected as further autonomy is introduced to the road, as the function of cockpits changes.

    Keep Reading


    Veoneer: How to Build Trust with Vehicles to Improve Road Safety

    How can we create trust in mobility? That is the question Ola Boström, the Vice President of Research wrestles with at Veoneer. We recently caught up with Ola after a component of Veoneer’s CES 2019 demo featured Affectiva’s Emotion AI technology, to learn a little bit more about his work within the space of automotive safety electronics.

    Read More


    What happens when cars get emotional?

    “Ambient computing” and other sensor-driven technologies are promising to make cars of the very near future much safer.

    Keep Reading


    Nuance: How to Design Effective Human-Vehicle Interactions from the Consumer Perspective

    What will the experience of future drivers or passengers will look like in the cars of tomorrow? This is the question that Adam Emfield, senior manager of user experience at Nuance Automotive, studies. 

    Keep Reading


    How Facial Recognition Will Change Driving

    Advances in facial recognition technology mean machines can not only recognize different people, but also how they are feeling. This means the next generation of automobiles may contain features that scan drivers’ faces for fatigue or other signs of impairment.

    Keep Reading


    Redefining the Occupant Experience with Human Perception AI


    Find out how Human Perception AI technology can improve the in-cabin experience in this eBook download. 

    Download Now


    Move aside, backseat driver! New tech at CES monitors you inside car

    Whether by generating alerts about drowsiness, unfastened seat belts or wallets left in the backseat, emerging technology aims not only to cut back on distracted driving and other undesirable behavior, but eventually help automakers and ride-hailing companies make money from data generated inside the vehicle.

    Read More


    Your Next Car Will Be Watching You More Than It's Watching the Road

    Thanks to advances in deep learning, it has become possible to develop technologies that can determine what is happening inside vehicles and make the ride safer and more pleasant—all while creating new privacy and security risks.

    Read More