AI light-field camera reads 3D facial expressions

Date:
January 21, 2022
Source:
The Korea Advanced Institute of Science and Technology (KAIST)
Summary:
Machine-learned, light-field camera reads facial expressions from
high- contrast illumination invariant 3D facial images.



FULL STORY ==========================================================================
A joint research team led by Professors Ki-Hun Jeong and Doheon Lee
from the KAIST Department of Bio and Brain Engineering reported the
development of a technique for facial expression detection by merging near-infrared light-field camera techniques with artificial intelligence
(AI) technology.


========================================================================== Unlike a conventional camera, the light-field camera contains micro-lens
arrays in front of the image sensor, which makes the camera small enough
to fit into a smart phone, while allowing it to acquire the spatial and directional information of the light with a single shot. The technique
has received attention as it can reconstruct images in a variety of
ways including multi- views, refocusing, and 3D image acquisition,
giving rise to many potential applications.

However, the optical crosstalk between shadows caused by external light
sources in the environment and the micro-lens has limited existing
light-field cameras from being able to provide accurate image contrast
and 3D reconstruction.

The joint research team applied a vertical-cavity surface-emitting laser (VCSEL) in the near-IR range to stabilize the accuracy of 3D image reconstruction that previously depended on environmental light. When
an external light source is shone on a face at 0-, 30-, and 60-degree
angles, the light field camera reduces 54% of image reconstruction
errors. Additionally, by inserting a light-absorbing layer for visible and near-IR wavelengths between the micro-lens arrays, the team could minimize optical crosstalk while increasing the image contrast by 2.1 times.

Through this technique, the team could overcome the limitations of
existing light-field cameras and was able to develop their NIR-based light-field camera (NIR-LFC), optimized for the 3D image reconstruction
of facial expressions.

Using the NIR-LFC, the team acquired high-quality 3D reconstruction
images of facial expressions expressing various emotions regardless of
the lighting conditions of the surrounding environment.

The facial expressions in the acquired 3D images were distinguished
through machine learning with an average of 85% accuracy -- a
statistically significant figure compared to when 2D images were
used. Furthermore, by calculating the interdependency of distance
information that varies with facial expression in 3D images, the
team could identify the information a light-field camera utilizes to distinguish human expressions.

Professor Ki-Hun Jeong said, "The sub-miniature light-field camera
developed by the research team has the potential to become the new
platform to quantitatively analyze the facial expressions and emotions
of humans." To highlight the significance of this research, he added,
"It could be applied in various fields including mobile healthcare,
field diagnosis, social cognition, and human-machine interactions."
This research was published inAdvanced Intelligent Systemsonline on
December 16, under the title, "Machine-Learned Light-field Camera that
Reads Facial Expression from High-Contrast and Illumination Invariant
3D Facial Images." This research was funded by the Ministry of Science
and ICT and the Ministry of Trade, Industry and Energy.

special promotion Explore the latest scientific research on sleep and
dreams in this free online course from New Scientist -- Sign_up_now_>>> ========================================================================== Story Source: Materials provided by The_Korea_Advanced_Institute_of_Science_and_Technology_ (KAIST). Note:
Content may be edited for style and length.


========================================================================== Related Multimedia:
* Facial_expression_readings ========================================================================== Journal Reference:
1. Sang-In Bae, Sangyeon Lee, Jae-Myeong Kwon, Hyun-Kyung Kim,
Kyung-Won
Jang, Doheon Lee, Ki-Hun Jeong. Machine‐Learned
Light‐Field Camera that Reads Facial Expression from
High‐Contrast and Illumination Invariant 3D Facial
Images. Advanced Intelligent Systems, 2021; 2100182 DOI:
10.1002/aisy.202100182 ==========================================================================

Link to news story: https://www.sciencedaily.com/releases/2022/01/220121094321.htm

--- up 6 weeks, 6 days, 7 hours, 13 minutes
* Origin: -=> Castle Rock BBS <=- Now Husky HPT Powered! (1337:3/111)