Adaptive Proximal Gradient Methods for Structured Neural Networks

NeurIPS 2021 Adaptive Proximal Gradient Methods for Structured Neural Networks Jihun Yun, Aurelie C. Lozano, Eunho Yang  We consider the training of structured neural networks where the regularizer can be non-smooth and possibly non-convex. While popular machine learning librarie...

Accurate Learning of Graph Representations with Graph Multiset Pooling

ICLR 2021 Accurate Learning of Graph Representations with Graph Multiset Pooling Jinheon Baek, Minki Kang and Sung Ju Hwang  Message-passing graph neural networks have been widely used on modeling graph data, achieving impressive results on a number of graph classification and li...

Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks

ICLR 2020 Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks Joonyoung Yi, Juhyuk Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang  Handling missing data is one of the most fundamental problems in machine learning. Among many approaches, ...

Self-supervised Label Augmentation via Input Transformations

ICML 2020 Self-supervised Label Augmentation via Input Transformations Hankook Lee, Sung Ju Hwang, Jinwoo Shin  Self-supervised learning, which learns by constructing artificial labels given only the input signals, has recently gained considerable attention for learning repr...

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition

ICLR 2020 Scalable and Order-robust Continual Learning with Additive Parameter Decomposition Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang  While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, there are issues that ...

Federated Continual Learning with Weighted Inter-client Transfer

ICML 2020 workshop in Lifelong Learning Federated Continual Learning with Weighted Inter-client Transfer Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, Sung Ju Hwang  There has been a surge of interest in continual learning and federated learning, both of which are importa...

Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation

EMNLP 2020 Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation Minki Kang, Moonsu Han, Sung Ju Hwang  We propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pr...

Neural Complexity Measures

NeurIPS 2020 Neural Complexity Measures Yoonho Lee, Juho Lee, Sung Ju Hwang, Eunho Yang, Seungjin Choi  While various complexity measures for diverse model classes have been proposed, specifying an appropriate measure capable of predicting and explaining generalization in deep ne...

MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures

NeurIPS 2020 MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures Jeongun Ryu, Jaewoong Shin, Hae Beom Lee, Sung Ju Hwang  Regularization and transfer learning are two popular techniques to enhance generalization on unseen data, which is a fun...

Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs

INTERSPEECH 2020 Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs Seong Min Kye, Youngmoon Jung, Hae Beom Lee, Sung Ju Hwang, Hoirin Kim  In practical settings, a speaker recognition system needs to identify a speaker given a short utter...