Machine Learning Results in 44% Improvement in Sales Forecast Accuracy

Overview
Our client engaged us for a machine learning project to apply cutting-edge techniques to achieve significantly more accurate business forecasts—and vastly decrease the time and effort required to generate them.
The project related to our client’s operations at multiple large distribution centers in the U.S. where the organization packages and ships products to its customers. To manage operations at these centers, our client generates a monthly product sales forecast. The data output is absolutely critical to business operations, because these figure drive everything from manufacturing to packaging and supply chain decisions.

Prior to the machine learning project, our client’s forecasting team used a combination of ERP tools and Excel—and significant manual effort—to set up the data for computational analysis. Then, the actual computer processing time to generate the forecast was over 18 hours—and often had to be re-run due to failures. Finally, yet more days of analysis would be required in Excel to prepare raw outputs for business use.

Given the organization’s outstanding historical data availability, its business situation was ideal for applying machine learning and predictive analytics.

Solution
Concurrency’s Data Analysis team, including the firm’s Data Scientist, worked with our client’s IT leaders and forecasting team, as well as the Microsoft data science team, to completely revolutionize the firm’s approach to monthly sales forecasting. The project involved preparing historical data for analysis, training computer models produce high-quality forecasts, and ultimately going live with a new approach to forecasting that cut computational time from 18 hours to 10 minutes—with more accurate results.

The following graphic illustrates steps involved in a machine learning project such as this one. The process begins with business understanding—and results in improved business understanding upon deployment. Intermediate steps include data understanding and preparation, modeling, and evaluation.

Clover-ML-graphic-1.png

A closer look at the evaluation phase, as carried out in this particular case, helps to elucidate each of the preceding points. Consider the following image, in which the vertical bars of lower height indicate more accurate forecasts. These figures were generated by comparing our client’s prior monthly forecasts with machine-learning-based forecasts applied to historical periods. That is, the new model was applied retrospectively, to evaluate what the generated forecast “would have been.” This type of analysis is only possible with high-quality historical data, which the organization had in abundance. The graphic below shows the machine learning (ML) over- or under-forecasts on a percentage basis of actual sales; these ML bars are shown in red. To the right of each red bar is the historical forecast’s variation from actual sales results. In all cases, the ML forecast was more accurate.

Clover-ML-graphic-2.png

We built the machine-learning-based modeling to be an automated process that does not require manual effort. (The specific technology involved includes Microsoft’s flexible machine learning platform, which gives organizations flexibility to run it either in SQL Server or as an Azure cloud service.)

Our results indicate a 44% increase in demand forecast accuracy, which will positively affect inventory positions and supply-chain directives. These efficiency gains suggest a monthly savings of $1-2MM per month in inventory and operational costs – without including our most recent initiative, which will substantially reduce current on hand inventory costs.

Furthermore, the machine learning forecasts run in only about 10 minutes, compared to over 18 hours for the prior approach.

This project yielded two major gains through operational intelligence. Importantly, the cost of selling a broad diversity of products was identified based on up-to-date information describing product sales and procurements, which resulted in a decision to begin reducing the number of unique products offered to customers by consolidating similar products into a single brand or identity. Furthermore, we enabled real-time extraction of inventory characteristics including the substantial cost of an inflated inventory position relative to an “optimal” inventory and procurement policy.

As forecasting is improved, our anticipates significant and continuing business benefits, including:
  • Less re-boxing
  • Lower backorder levels
  • Decrease in brand subs
  • Increase in customer satisfaction
  • Decrease in overtime for manufacturing and packaging
  • Decrease in production line costs to build more product
  • Decrease in supply chain costs
Furthermore, the organization can apply the lessons and approach of this first machine learning project in other aspects of the business to continue to improve operations. For example, a key business focus is increasing the velocity of order fulfillment, such as by optimizing the physical location of inventory within distribution centers—and among distribution centers—to maximize efficiency in order preparation and delivery.

Given many thousands of individual products and tens of millions of dollars quarterly in shipping costs, this organization recognizes important opportunities for further efficiency gains to be derived from powerful insights provided by machine learning.


 

 
Organization

Manufacturing & Distribution
 

Organization Type

Organization Profile