College of Engineering, Design & Computing Events

Abdel Khaleq, Abeer
PhD candidate, Computer Science and Information Systems

Design and evaluation of QoS aware intelligent autoscaling services for microservices in cloud computing environments

Thesis directed by Associate Professor Ilkyeun Ra

ABSTRACT

Cloud applications are becoming more containerized in nature. Developing a cloud application imposes different challenges including scalability at the container level. What adds to the challenge is that cloud applications impose quality of service (QoS) requirements and have various resource demands requiring a customized scaling approach. Existing autoscaling technologies such as Kubernetes offer some customization to a set of threshold values for autoscaling. Identifying the right values for the different autoscaling parameters requires good knowledge of the system behavior and resource requests that will guarantee QoS in a changing dynamic environment.

My research aims to utilize advancements in machine learning and reinforcement learning to provide a mean for autoscaling in cloud applications while maintaining QoS constraints intelligently and autonomously. In my work, I will be utilizing reinforcement learning to identify the Kubernetes autoscaling threshold values to autoscale a microservice cloud application while maintaining QoS.  I will be demonstrating the key benefits of this approach on a Twitter analytics microservice cloud application using response time as a QoS constraint.

I begin by presenting the Twitter analytics microservices framework, the system consists of multiple microservices deployed on public cloud for disaster classification, disaster phase discovery, and disaster information extraction. I then describe our generic autoscaling algorithm that autoscales based on identifying the microservice resource demand. The algorithm adapts the Kubernetes autoscaling paradigm based on the application characteristics and resource requirements. Testing the algorithm on Google cloud shows that response time is improved by a factor of 20% compared to the default Kubernetes autoscaling. And finally, I will be presenting the reinforcement learning agents designed and developed to learn and identify the autoscaling threshold values based on the resource demand and QoS. The RL agents were trained and tested in both a simulated environment and real cloud environment. Testing the agents on multiple microservices shows an enhancement in the average response time compared to the default Kubernetes horizontal pod autoscaler. The agents provide an intelligent and autonomous way to autoscale the microservices with minimum user intervention while maintaining QoS in a cloud setting application.