Managing batch ML jobs is a central competency for Data Science (DS) teams in the ad tech space. According to PWC research, digital ad spend has increased by 23% to $50 Billion in the first half of 2018. To deal with this growth, DS teams need flexible tools. We present our k8s-workqueue system. A pluggable scheduling mechanism for ML Kubernetes workloads where tens of thousands of models are built every day on our platform. The focus on simplicity, led us to the design of this system that combines familiar features of traditional cron jobs and containers, with the power of the Kubernetes API.
