College of Engineering, Design & Computing Events

Henok Ghebrechristos presents "Deep Curriculum Learning Optimization"

Henok is a computer science PhD student and is part of the Parallel Distributed Systems Lab.

Abstract

We describe a quantitative and practical framework to integrate Curriculum Learning (CL) optimization into deep learning training pipeline to improve feature learning in deep feed-forward networks. The framework has several unique characteristics: 1. dynamicity – it proposes a set of batch-level training strategies (syllabi or curricula) that are sensitive to data complexity 2. adaptivity – it dynamically estimates the effectiveness of a given strategy and performs objective comparison with alternative strategies making the method suitable both for practical and research purposes. 3. employs replace-retrain mechanism when a strategy is unfit to the task at hand. In addition to these traits, the framework combines CL with several variants of GD has been used to generate efficient batch-specific or data-set specific strategies. Comparative studies of various current and past state-of-the-art vision models such as VGG and ResNet on several benchmark datasets including CIFAR10 demonstrate the effectiveness of the proposed method. We present results that show training loss reduction by as much as a factor 9. Additionally, we present a set of practical curriculum strategies to improve the generalization performance of select networks on various datasets.

If you would like to give a seminar or have any questions, please contact hawkar.oagaz@ucdenver.edu  

You can find more information about the seminar schedule here 
https://sites.google.com/view/ucden-cse-seminars