Dae Hyeon Kim RESEARCH 한국어

Multimodal time-series fusion

Fused signal-specific similarity networks to learn multimodal representations with limited labels.

GitHub profile
Role
Technical work-package lead
Contribution
Developed heterogeneous graph fusion and semi-supervised learning. Supported certification tests and intellectual-property deliverables.
Period & affiliation
2022.03 ~ 2023.12 IITP

01 / RESEARCH DETAILMSGCN: fusion within and across signals

Challenge
Different feature spaces and limited labels make simple feature concatenation insufficient for representing modality relationships.
Approach
First fused DE and PSD similarity within signals, then diffused cross-signal similarity and classified the resulting graph with a GCN.
Contribution
Developed graph fusion and semi-supervised models and supported certification and intellectual-property outputs.
Fusion of DE and PSD features from EEG, EDA and PPG
Fusion of DE and PSD features from EEG, EDA and PPG
View full-size figure
DEAP emotion classification accuracy (%) Accuracy: Valence 97.37, Arousal 97.38 0 25 50 75 100 97.37 Valence 97.38 Arousal Accuracy
DEAP emotion classification accuracy (%)
Evaluation protocol and project outputs
Evaluation / outputs Description
DEAP 32 participants, subject-dependent evaluation
Certified tests 2 tests
Intellectual property 1 patent application, 3 software registrations
Evaluation conditions and scopeDEAP, subject-wise transductive evaluation with 120 labels per class. Valence accuracy: 97.37 ± 1.75%; arousal: 97.38 ± 1.68%. The patent is an application, not a granted patent.