Automating Modeling Pipelines

At Intent Media we make real time decisions about how to treat users based on machine learning models. These models are trained daily on huge historical datasets, use real time data and return predictions in tens of milliseconds.  All of this is difficult, however the real challenge is that we have hundreds of models, each of which have their own configurations, defaults, accuracy thresholds, decision strategy etc. I will describe how we coordinate these models, how we automate the model publishing pipeline and how we monitor everything to ensure the best model is used every time.