Self-Supervised Pretraining for Label-Efficient Facial Expression Recognition using Smart Glasses

Authors: Zhengtao Gao , Francesco Bombassei De Bona , Leonardo Alchieri , Ivana Kiprijanovska , Simon Stankoski , Bojan Sofronievski , Hristijan Gjoreski , Marc Langheinrich , Martin Gjoreski

View a PDF of the paper titled Self-Supervised Pretraining for Label-Efficient Facial Expression Recognition using Smart Glasses, by Zhengtao Gao and 8 other authors

View PDF

HTML (experimental)

Abstract: Facial Expression Recognition (FER) on optomyographic (OMG) smart glasses such as OCOsense enables privacy-preserving, camera-free affect sensing for health and well-being. Most FER systems are fully supervised, i.e., they rely either on hand-crafted expert features with classical machine learning, or on end-to-end deep models trained from labelled data. Both depend on large quantities of labelled expressions, which are costly to collect and limit deployment in real-world, personalized settings. We investigate Self-Supervised Learning (SSL) as a route to label-efficient FER on these glasses. We pretrain a Vision Transformer with a masked-autoencoder objective on unlabelled OCOsense recordings and finetune it for five-class FER under person-independent cross-validation. Across label budgets from 5% to 100%, pretraining consistently improves performance; with only 5% of the labels, the pretrained model outperforms the from-scratch model trained with all labels. Our results show that SSL pretraining is a practical way to reduce labelling cost for smart eyewear FER, and we discuss the implications for future personalization and continual adaptation.

Comments:

7 pages, 5 figures, 2 tables. Best Paper Award at the SEPH 2026 workshop, UbiComp Companion '26

Subjects:

Artificial Intelligence (cs.AI)

ACM classes:

I.2.6; I.5.4

Cite as:

arXiv:2611.05711 [cs.AI]

(or

arXiv:2611.05711v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2611.05711

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

Journal reference:

Companion of the 2026 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp Companion '26), October 11-15, 2026, Shanghai, China

Related DOI :

https://doi.org/10.1145/3798063.3839146

Focus to learn more

DOI(s) linking to related resources

Submission history

From: Francesco Bombassei De Bona [ view email ]

[v1]

Tue, 3 Nov 2026 22:37:17 UTC (4,539 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Self-Supervised Pretraining for Label-Efficient Facial Expression Recognition using Smart Glasses, by Zhengtao Gao and 8 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.AI

< prev

next >

new

recent

| 2026-11

Change to browse by:

cs

References & Citations

SciX

Google Scholar

Semantic Scholar

export BibTeX citation

Loading...

BibTeX formatted citation

loading...

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer ( What is the Explorer? )

Connected Papers Toggle

Connected Papers ( What is Connected Papers? )

Litmaps Toggle

Litmaps ( What is Litmaps? )

scite.ai Toggle

scite Smart Citations ( What are Smart Citations? )

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv ( What is alphaXiv? )

Links to Code Toggle

CatalyzeX Code Finder for Papers ( What is CatalyzeX? )

DagsHub Toggle

DagsHub ( What is DagsHub? )

GotitPub Toggle

Gotit.pub ( What is GotitPub? )

Huggingface Toggle

Hugging Face ( What is Huggingface? )

ScienceCast Toggle

ScienceCast ( What is ScienceCast? )

Demos

Demos

Replicate Toggle

Replicate ( What is Replicate? )

Spaces Toggle

Hugging Face Spaces ( What is Spaces? )

Spaces Toggle

TXYZ.AI ( What is TXYZ.AI? )

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower ( What are Influence Flowers? )

Core recommender toggle

CORE Recommender ( What is CORE? )

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Which authors of this paper are endorsers? |

Disable MathJax ( What is MathJax? )