Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks

ICLR 2020 Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks Joonyoung Yi, Juhyuk Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang  Handling missing data is one of the most fundamental problems in machine learning. Among many approaches, ...

Self-supervised Label Augmentation via Input Transformations

ICML 2020 Self-supervised Label Augmentation via Input Transformations Hankook Lee, Sung Ju Hwang, Jinwoo Shin  Self-supervised learning, which learns by constructing artificial labels given only the input signals, has recently gained considerable attention for learning repr...

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition

ICLR 2020 Scalable and Order-robust Continual Learning with Additive Parameter Decomposition Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang  While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, there are issues that ...

Federated Continual Learning with Weighted Inter-client Transfer

ICML 2020 workshop in Lifelong Learning Federated Continual Learning with Weighted Inter-client Transfer Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, Sung Ju Hwang  There has been a surge of interest in continual learning and federated learning, both of which are importa...

Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation

EMNLP 2020 Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation Minki Kang, Moonsu Han, Sung Ju Hwang  We propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pr...

Neural Complexity Measures

NeurIPS 2020 Neural Complexity Measures Yoonho Lee, Juho Lee, Sung Ju Hwang, Eunho Yang, Seungjin Choi  While various complexity measures for diverse model classes have been proposed, specifying an appropriate measure capable of predicting and explaining generalization in deep ne...

MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures

NeurIPS 2020 MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures Jeongun Ryu, Jaewoong Shin, Hae Beom Lee, Sung Ju Hwang  Regularization and transfer learning are two popular techniques to enhance generalization on unseen data, which is a fun...

Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs

INTERSPEECH 2020 Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs Seong Min Kye, Youngmoon Jung, Hae Beom Lee, Sung Ju Hwang, Hoirin Kim  In practical settings, a speaker recognition system needs to identify a speaker given a short utter...

Meta Variance Transfer: Learning to Augment from The Others

ICML 2020 Meta Variance Transfer: Learning to Augment from The Others SeongJin Park, Seungju Han, Jiwon Baek, Insoo Kim, Juhwan Song, Hae Beom Lee, Jae-Joon Han, Sung Ju Hwang  Humans have the ability to robustly recognize objects with various factors of variations such as n...

Meta Dropout: Learning to Perturb Latent Features for Generalization

ICLR 2020 Meta Dropout: Learning to Perturb Latent Features for Generalization Hae Beom Lee, Taewook Nam, Eunho Yang, Sung Ju Hwang  A machine learning model that generalizes well should obtain low errors on unseen test examples. Thus, if we know how to optimally perturb tra...