das42

Enabling Q4 Growth for an Ad Server

Our client runs thousands of servers for their clients, serving up digital ads and gathering data on multiple aspects of each ad; what ads they serve, the campaign associated with it, what device it was seen on, clicks, etc. Every ad they serve up generates a significant amount of data, and across the internet ads are being served up constantly. All of that data needs to be seamlessly aggregated from the logs generated in each server. From that data, they generate reports that they present to their advertising agency clients.

Looking for a Solution

Their engineering team was looking for a modern robust platform where failures were immediately addressed with automatic retries, they'd have transparency into the data pipeline, they could work with a set of industry standard tools, and they could grow their business.

Data & Cloud Analytics Strategy

Running in Parallel

We worked with the engineering team to understand and untangle all of the technologies being employed. We needed to determine how to port over their existing data while new data was flowing in. The business logic of the new system had to remain the same. 

After a multi-hour white board session, the decision was made to build a parallel platform and jettison their entire system after a few months of verifying the quality of the results on the new platform. The parallel approach allowed us to test the new platform and perform QA testing in order to be certain that the new platform was producing the same results as the legacy system. Through this process we were able to remove a lot of complexity and provide them with a modern and simplified techstack—all while keeping their business running at full speed.

Data Platform Modernization & Data Migration

Migrating to a Modern Platform

The technology they had been employing had some drawbacks. First, they were using a variety of tools, some of them homegrown, running various types of open source MySQL scripts, which is more of a transactional database than an analytical database. Second, they had insufficient visibility into their data pipeline. At times, their engineering team had to get up in the middle of the night to fix something to make sure that their customers were well-served. 

Lastly, there was a variety of legacy technology being used that the engineers hadn’t built themselves. They were able to maintain the system and provide valuable information to their clients, but it was difficult to maintain and nearly impossible to scale. This is a situation we often see at companies with homegrown tools.

We set them up with Airflow and Snowflake. Airflow is an industry standard data pipeline able to gather data from their servers, centralize it, and bring it into Snowflake’s cloud data warehousing environment. We optimized Snowflake to give them the robust scalable platform they were looking for to house their legacy and newly generated data in a single, consistent place.

With a simplified platform designed for data analytics, our client now has excellent visibility into their pipeline and a hearty and easy to maintain platform that lets their engineers sleep at night. The streamlined platform also let them reduce the number of vendors they needed to maintain their system. Most importantly, the new platform allowed them to grow their operations.

Services We Provided

Icon-Data-Platform-Migration

Data Platform Modernization & Data Migration

Icon-Data-And-Cloud-Analytics-Strategy

Data & Cloud Analytics Strategy

In Conclusion

A Q4 Expansion

Retail businesses significantly increase their ad spend during the Holidays. During this time of year, Ad Servers often hit operational capacity serving their existing clients. The company’s CTO told us that due to the new platform, they were processing data, “flawlessly”. This transformation allowed them to take on more business in Q4, their busiest time of the year.

Ready to talk about your data needs?

Icon-Contact

Contact us to start building a data culture.