Affectiva Automotive AI

Road safety continues to be top priority. Euro NCAP requirements are accelerating the deployment of driver monitoring systems that use cameras and sensors to detect driver fatigue and distraction, and a child left behind. At the same time, consumers expect comfortable and entertaining transportation experiences that are tuned to their needs in the moment. To enable these advanced safety features and to deliver personalized and more comfortable transportation experiences, car manufacturers need a deep understanding of what takes place in a vehicle.

Affectiva Automotive AI is the leading In-Cabin Sensing (ICS) solution that understands what is happening inside of a vehicle. Affectiva's patented deep learning-based software uses in-vehicle cameras to measure in real time, the state of the cabin, and that of the driver and occupants in it — from complex and nuanced emotions and cognitive states, such as drowsiness and distraction, to occupancy, activity, object and child seat detection. OEMs, Tier 1s, ridesharing providers and fleet management companies can deliver optimized occupant experiences and advanced safety features using Affectiva's AI.

Driver State

Understand complex and nuanced states of driver impairment such as levels of drowsiness, distraction and anger to ensure road safety. Gain insight into driver interactions with vehicle systems and other passengers to improve the driving experience.

Occupant State

Insight into the state of the occupants helps improve their comfort by adapting music, lighting, temperature. Understanding passenger engagement with content allows for personalized recommendations and for opportunities to monetize media data.

Cabin State

Insight into the state of the cabin helps personalize safety features (seatbelt, airbag), content (video, music and advertising), and the environment of the cabin (heating and lighting). It can also detect if a child or objects have been left behind.

9copy Drowsiness
Analyze eye closure, yawn, head pose,
and other facial features to determine
level of drowsiness.

18copy

Distraction
Analyze head position, head rotation,
eye gaze, and cell phone usage to
determine driver distraction.

23-COPY2

Child Seat
Detect the presence of infant seats to
avoid children left unattended in vehicle.

11copy

Occupancy
Understand where people are sitting
in the vehicle to inform safety and
hospitality features.

36copy

Body Key Points
Track occupant body joints to estimate
their body posture and check if they
are safely seated in the vehicle.

17copy

Object Left Behind
Detect objects like cell phones to ensure
that passengers don’t forget personal items.

14copy

Facial Expression Analysis
Map facial action units to emotions
and cognitive states to understand
occupant state, reactions and intent.

Mood & Emotion Detection
Use facial signals and body key points to
understand occupant reactions to vehicle
UI and automation features, and personalize
content recommendations.

1copy

Activity Detection
Use body key points and object detection to recognize occupant behaviors and activities, such as use of cellphone, for example.

Automotive Grade

Data

Affectiva's deep learning algorithms are trained and tested with our large, proprietary data set that is representative of human appearance variances and real-world use cases. Global in nature, our data is diverse in age, gender and ethnicity, helping us mitigate for bias.

Captured in real world conditions
Acquired with opt-in and consent
Automotive in-cabin data: 20k+ hours, 4k+ unique individuals
Foundational data: 9.7M+ faces, 90 countries, 5B+ facial frames, 6 years of video data

Camera Position

As the industry moves to holistic in-cabin sensing, interior cameras will be placed where they have a view of the entire cabin, including the driver and passengers in it. Overhead camera positions are proving ideal, and Affectiva has tuned its models for this location.

Affectiva's AI supports both RGB and NIR cameras
Enables OEMs to use one camera for detecting both state of the driver, state of the cabin, and state of the occupants in it
Overhead cameras are typically integrated in the rear view mirror or roof module

Embedded Systems

Affectiva's deep learning models are tuned to run on automotive Systems on Chip (SoC). We have optimized our footprint to run alongside other services, in real time, with automotive grade accuracy. Our models support TensorFlow Lite deep learning runtime.

Affectiva models are quantized
Embedded deep learning runtime supports ARM® or Intel CPU
Embedded deep learning runtime can be swapped with hardware specific runtime to make use of accelerators available in target System on Chip (SoC)

“Moving forward, transparency will play an important role in building trust in the AI systems that will slowly find their way into our cars—if we choose to trust it at all. El Kaliouby stresses that automakers and mobility services providers must be clear about in-cabin sensing technology and educate consumers on what the technology does, what data it collects, and how it stores and uses the data.”

Ben Dickson

“Another area of the in-vehicle experience where AI is pushing developments is in-cabin monitoring. Applications such as distraction detection are already in place in some segments, and could be critical for automakers should they wish to introduce advanced driver assistance systems which will still rely on driver attentiveness. But some companies see far greater potential than this, and AI is the key. Affectiva is one such player. Older than most start-ups, the company was founded in 2009 and has raised some US$60.3m in funding to date, with notable investors including Aptiv. Affectiva considers itself a pioneer in emotion AI, which can interpret user emotion via a number of means including vision processing of facial expressions, or audio processing of voice.”

Xavier Boucherat

“Advances in facial recognition technology mean machines can not only recognize different people, but also how they are feeling. This means the next generation of automobiles may contain features that scan drivers’ faces for fatigue or other signs of impairment. Companies, including Boston-based Affectiva, are already making software to help the auto industry integrate such technology.”

Jeff John Roberts

“As auto manufacturers race to build self-driving vehicles of the future, the company is betting that its technology could play a role in shaping this new world of transportation…In the future, if all goes as planned, emotion AI in cars could help calm us down, wake us up and get us mentally ready for the day’s commute or a big night out. Affectiva, which already has contracts with auto-makers and suppliers, hopes to have its technology embedded in cars on the road by 2021.”

The Forbes Insights Team

“…with the camera pointed at the safety driver, Renovo can then tell whether that person is tired or distracted, and deliver the right prompts or warnings to ensure attention remains on the road ahead. And that’s where Renovo and Affectiva’s collaboration perhaps could have prevented the fatal Uber collision last month.”

Andrew Hawkins

"The future of transportation is AI whether computers do the driving or not. We desperately need to change the way we approach the manufacture and operation of motor vehicles. Over one million people die every year in vehicle collisions. With Automotive AI, it’s possible the roads could become a little safer thanks to Affectiva and other companies working on changing things for the better."

Tristan Greene

Press

New tech monitors you inside car

Emerging technology aims not only to cut back on distracted driving and other undesirable behavior, but eventually help automakers and ride-hailing companies make money from data generated inside the vehicle.

BLOG

BMW: How ICS Helps Build the In-Vehicle Experience

Sean Batir, Sr. Machine Learning Engineer at BMW, speaks about his work with Affectiva on the emotional in-cabin experience, and how In-Cabin Sensing (ICS) re-imagines the future of driving.

Press

What happens when cars get emotional?

“Ambient computing” and other sensor-driven technologies are promising to make cars of the very near future much safer. Self-driving cars are nice—but the real revolution is automotive empathy.

Affectiva Automotive AI

Redefining the occupant experience and improving road safety
with In-Cabin Sensing

The Automotive Challenge

The Affectiva Solution

Affectiva identifies key events related to humans with In-Cabin Sensing

Driver State

Occupant State

Cabin State