Klasifikasi Ekspresi Emosi Bayi Berbasis Citra Wajah Menggunakan CNN untuk Aplikasi Mobile

Authors

  • Desty Asty Alamanda Program Studi Teknik Informatika, Fakultas Teknik, Universitas Krisnadwipayana, Jakarta, Indonesia
  • Elmi Devia Program Studi Teknik Informatika, Fakultas Teknik, Universitas Krisnadwipayana, Jakarta, Indonesia
  • Harry Dwiyana Program Studi Teknik Informatika, Fakultas Teknik, Universitas Krisnadwipayana, Jakarta, Indonesia

DOI:

https://doi.org/10.35316/jimi.v11i2.10751

Keywords:

Convolutional Neural Network (CNN), Baby Emotion Classification, MobileNetV2, BabyFACS, Flutter

Abstract

Infants aged 0–2 years rely primarily on facial expressions to convey affective states, yet parental interpretation remains subjective, ambiguous, and prone to misjudgment. This study develops a lightweight infant facial emotion classification model using Convolutional Neural Network (CNN) MobileNetV2 deployed into an offline mobile application, following the CRISP-DM framework. A balanced dataset of 600 images across three affective categories (happy, neutral, distress) based on the Infant Facial Affect (IFA) approach was trained using two-stage transfer learning with fine-tuning on the top 30 layers. Experimental evaluation on 60 independent test images demonstrated that data augmentation improved model performance, achieving an overall accuracy of 96.67% and macro-average precision, recall, and F1-score of 97.00%, outperforming the non-augmented baseline (95.00%). Spatial interpretability using Grad-CAM validated that network attention consistently localized on critical facial landmarks corresponding to BabyFACS Action Units (AU6/AU12 for happy and AU4/AU20 for distress). The model was successfully converted to TensorFlow Lite (9 MB) and integrated into a Flutter mobile application, achieving an average inference latency below 150 ms and 100% functional validity through Black Box testing. This system delivers real-time, objective emotion recognition alongside actionable parenting insights to support prompt and accurate infant caregiving.

Downloads

Download data is not yet available.

References

[1] M. A. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Harlow, UK: Pearson Education, 2022.

[2] E. Oster, “Baby FACS: Analyzing facial movement in infants,” in What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), P. Ekman and E. L. Rosenberg, Eds., 2nd ed. Oxford, UK: Oxford University Press, 2006, pp. 24–40.

[3] P. Ekman and W. V. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement. Palo Alto, CA: Consulting Psychologists Press, 1978.

[4] Y. Ahn, D. S. Messinger, and M. I. Stifter, “Automated measurement of infant smiling and positive affect through Duchenne markers,” IEEE Transactions on Affective Computing, vol. 14, no. 2, pp. 1120–1131, 2023.

[5] Ö. Önal Ertuğrul, J. F. Cohn, and L. A. Camras, “Cross-cultural validation of cry-face Action Units in young infants,” Emotion, vol. 22, no. 4, pp. 789–801, 2022.

[6] E. Trinhammer, M. S. Vinding, and L. V. Petersen, “Infant Facial Affect (IFA): A validated scheme for coding emotional expressions in infancy,” Infant Behavior and Development, vol. 68, p. 101732, 2026.

[7] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.

[8] R. R. Amaanullah, G. R. Pascifa, S. A. Nugraha, M. R. Zein, dan F. D. Adhinata, “Implementasi CNN untuk Deteksi Emosi melalui Wajah,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 6, no. 4, pp. 621–628, 2022.

[9] I. Ekawati, F. N. R. Putra, M. Sumadyo, dan R. N. Whidhiasih, “Deteksi Emosi Menggunakan Convolutional Neural Network Berdasarkan Ekspresi Wajah,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), vol. 13, no. 1, pp. 45–54, 2024.

[10] A. Faizah, S. Imron, A. Rewur, J. N. Makasunggal, dan P. H. Saputro, “Komparasi Algoritma Support Vector Machine (SVM) dan Convolutional Neural Network (CNN) untuk Klasifikasi Ekspresi Wajah,” Jurnal Tekno Kompak, vol. 19, no. 1, pp. 12–23, 2025.

[11] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4510–4520.

[12] R. S. Sihombing, T. Sitompul, dan V. Sitorus, “Pengenalan Ekspresi Wajah Menggunakan CNN MobileNetV2,” Jurnal Algoritma, Logika dan Komputasi, vol. 6, no. 2, pp. 189–198, 2023.

[13] F. A. Zahra, P. Setiaji, dan W. A. Triyanto, “Klasifikasi Ekspresi Emosi Wajah Bahagia dan Tidak Bahagia Menggunakan Arsitektur MobileNetV2 Berbasis Deep Learning,” Jurnal Sains Komputer & Informatika (J-SAKTI), vol. 9, no. 1, pp. 102–113, 2025.

[14] R. Wirth and J. Hipp, “CRISP-DM: Towards a standard process model for data mining,” in Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining, 2000, pp. 29–39.

[15] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618–626.

Published

10-11-2026

How to Cite

Alamanda, D. A., Devia, E., & Dwiyana, H. (2026). Klasifikasi Ekspresi Emosi Bayi Berbasis Citra Wajah Menggunakan CNN untuk Aplikasi Mobile. Jurnal Ilmiah Informatika, 11(2), 139–147. https://doi.org/10.35316/jimi.v11i2.10751