View Recording: From Forecast to Factory Floor: How ML Models Are Cutting Production Waste

Nick Miller Hello, welcome everybody. Thanks for joining us today. I’m Nick Miller with Concurrency, and joining me is Brian Hayden. He’s one of our solution architects. Over the next 45 minutes, we’re going to talk about something that comes up in almost all the manufacturing conversations we’ve been having lately. And that’s where does machine learning actually reduce production waste and how to figure out where to start. We’re not going to have a demo today. We’re not going to walk through some code and bore you to death. 0:0:31.135 –> 0:0:49.895 Nick Miller But what we will do is we’ll leave you with a practical framework for identifying the right use cases and an honest read in what’s changed in the last 12 months. We’ll also add a clear next step if any of this resonates with you. And then I guess I’ll hand it off to Brian. Anything you’d like to add before we start up? 0:0:50.655 –> 0:1:10.575 Brian Haydin Yeah, I guess just one little thing. I mean, the conversations that we’ve been having with our manufacturers this year are starting to shift in a really meaningful way and a lot different than they were two years ago. Some of it’s hype, you know, some of it’s real though. And I think what we want to cover today is like what people should actually be thinking about. 0:1:11.775 –> 0:1:14.95 Nick Miller Alright, yeah, it makes sense. Well, let’s get right into it. 0:1:15.935 –> 0:1:34.775 Nick Miller So when we talk about manufacturing waste, here’s the reframe I want to put on the table before we go any further. When we talk about the manufacturers, we’re talking about waste, it’s generally, it’s a symptom. It’s what do they see on the factory floor? Hey, wow, we’ve got all this overtime scheduled. Why? 0:1:23.775 –> 0:1:24.15 Brian Haydin Thanks. 0:1:25.175 –> 0:1:25.295 Brian Haydin Okay. 0:1:28.735 –> 0:1:28.895 Brian Haydin Kate. 0:1:34.855 –> 0:1:54.95 Nick Miller Why are we paying all this expediting freight? Why do we have too much of this inventory? We have all this inventory aging problems. You know, why do we have so much scrap? And those are real problems. They definitely hurt the P&L, but they’re not where the waste starts. It’s the symptom, right? So, but waste actually starts upstream. And so the reframe I’d like to look at is, 0:1:43.775 –> 0:1:43.935 Brian Haydin No. 0:1:54.375 –> 0:2:10.615 Nick Miller hey, it starts as a bad gas. So waste starts with a bad gas. So think about it this way. Over time, it didn’t start on the schedule. It started earlier when we underestimated the demand for labor during that shift. You know, expedited freight didn’t start. 0:2:14.572 –> 0:2:37.52 Nick Miller They got missed in the forecast. Or it could be, you know, lead times for raw materials were creeping up or were expected to creep up, but we didn’t see it. You know, excess inventory didn’t just start, it didn’t appear in the warehouse. It started because we have, you know, a suboptimal inventory level setting process, or we misread seasonality. 0:2:37.452 –> 0:2:41.452 Nick Miller Six ago, six months ago, and we bought too much of, you know, that item. 0:2:43.212 –> 0:3:2.972 Brian Haydin Yeah, you know, and I think the gap between those decisions and, you know, the consequences is really the part that surprises most people. We talked about an operate, I recently was talking to like an operations leader and, you know, he told us by the time that we see the problem, the decision that caused it is like 2 planning cycles behind. 0:3:3.292 –> 0:3:17.852 Brian Haydin You know, we can’t go back and we can’t fix it. All we can really do at this point is just learn from it. So that’s the reality. The waste that you can see today was created weeks or months ago by a decision that was made with more or less incomplete information. Right? 0:3:17.772 –> 0:3:37.612 Nick Miller Okay. Yeah, no, that makes sense. So if waste starts as a bad guess, then improving our guesses will be the way that we can improve the quality of our decisions before the waste occurs. And that’s really where the leverage is. And that’s where machine learning earns its place in the manufacturing operation. Not because it produces A forecast, 0:3:38.492 –> 0:3:57.292 Nick Miller but it reduces the cost of being wrong. And so that phrase is going to come up a few more times in the session, the cost of being wrong, and I want you to hold on to it. It’s the simplest way I know to explain why ML matters to your environment. Brian, did you want to add anything else to that? 0:3:58.12 –> 0:4:18.972 Brian Haydin Just that the math is getting pretty interesting really fast. You know, a 5% improvement in demand forecasting accuracy, you know, at most of the mid-size manufacturing companies here in Wisconsin, that shows up as like a seven-figure inventory reduction. So it’s not just model improvement, that’s like balance sheet improvement, right? 0:4:19.612 –> 0:4:41.292 Nick Miller Yeah, that makes a lot of sense, the forecast. And if you’re talking about the inventory and manufacturing operations and stability, that forecast is the leading part of that process. So that makes a lot of sense. And you can really, when you have those better forecasts, you can take better advantage of that data you have and really optimize that inventory or make better decisions ahead of time. So 0:4:41.692 –> 0:4:47.452 Nick Miller Let’s talk about what has actually changed in the last 12 months to make it more achievable than it was before. 0:4:48.572 –> 0:5:7.212 Brian Haydin Yeah, you know, I’ll go ahead and take this slide. So, you know, it really has been changing a lot this year. And I want to give, you know, the people that are listening to this a clean way to think about it. Five years ago, if a manufacturer wanted to do like a machine learning project, the product lived almost entirely on the left side of this picture. 0:5:7.612 –> 0:5:26.52 Brian Haydin You pulled in operational data into a warehouse, maybe you trained a forecast model, and then you put the output on a dashboard. Somebody would eventually look at that dashboard and decide if there was something that meant something that they should act on. And I guess that worked, but it also leaked a lot of decisions everywhere along the way. 0:5:26.572 –> 0:5:47.692 Brian Haydin The forecast was probably good, but the path from a forecast to action was something that was being missed. So in 2026, what’s changing is the right side of the picture. Microsoft has been building what they’re calling like the intelligence layer on top of the data platform and the ecosystem. And, you know, the name is going to get a little bit dense, you know, so I’m going to try to like… 0:5:47.772 –> 0:6:8.492 Brian Haydin give you the plain English version of it. But data, when we think about data, it lives in OneLake, in Fabric. That part really hasn’t fundamentally changed that much. But what’s new is this context layer. We’ve had some other webinars about this in the past, but it’s called Fabric IQ. And it expresses the business meaning of the data. 0:6:8.852 –> 0:6:33.92 Brian Haydin So what you actually call a customer, what counts as on time and in full, what defect is that, you know, is actually in your operation, that’s the semantic layer. And it used to live in spreadsheets and just the tribal knowledge of people that, you know, within the organization. But now we have a way to keep it in a shared governed model that the machines and the people both can understand at the same time. 0:6:33.772 –> 0:6:53.772 Brian Haydin And so on top of that reasoning layer, that’s where Microsoft Foundry and the agent framework is going to sit. This is where like ML models, Forecast, and the agents actually are doing the work. And then those agents can connect to action. They can start to do things like scheduling an update, you know, or update a schedule, trigger a work order, 0:6:54.92 –> 0:7:15.92 Brian Haydin maybe route some of the alerts, you know, things along that nature. And then the last step is used to require human translating dashboard really into the decisions. So increasingly, we’re seeing that the platform supports the connection directly with the right governance and the right approvals being in place so that people, you know, can feel like they’re empowered to do it safely. 0:7:15.652 –> 0:7:38.12 Brian Haydin And the reason that this matters for manufacturing isn’t really just these product names over here. The reason that it matters is that the distance between we have a forecast and something changed because of the forecast is shorter than it used to be. Forecast used to die in a dashboard, you know, when we just did, when that’s all that we had. And now when we can connect that to operational change, that’s the shift this year. 0:7:38.572 –> 0:7:57.452 Brian Haydin And I guess one more thing before we move on. None of this really removes the hard parts of ML. Nick, you’re the smart guy on this. You’re going to deal with, you know, you’re going to explain this, but that’s the plumbing tax. You know, I mean, there is this cost of getting a model into production, but that even has come down, you know, really significantly. 0:7:58.332 –> 0:8:9.932 Brian Haydin You know, and so I think that’s why we’re having different conversations, being able to do more, doing more with less and cooler projects in 2026 than we were able to do like in 2024. 0:8:11.52 –> 0:8:30.612 Nick Miller Yeah, I think that makes a lot of sense. You know, fuel or data rather is the fuel for these ML models. And the fact that we have more semantic meaning and a shared understanding of that, it makes that process easier. But yeah, the data aggregation, the data cleaning, the data understanding still is the hard part. So let’s get into that. 0:8:32.572 –> 0:8:33.52 Brian Haydin You bet. 0:8:32.892 –> 0:8:52.12 Nick Miller So for most manufacturing use cases, they typically fall into one or three buckets, and they’re not always neatly divided. There’s some gray area between them, or maybe to reframe capacity as something else or process waste as optimization. But in general, these are kind of three buckets that we see. 0:8:52.812 –> 0:9:12.332 Nick Miller And we’re going to walk through each one. So what I want to do is try to understand for your personal, for your personal business and your pain points you see, where does each one of those pain points, does it fit into one of these areas of the waste? Do you think it’d be solved by one of these three areas? 0:8:52.892 –> 0:8:52.972 Brian Haydin The. 0:9:13.932 –> 0:9:36.412 Nick Miller So the first bucket is forecast waste, and this is just when you guess wrong about what is coming. There’s different types of forecasts, but you know, in general, demands patterns shift, seasonality moves, a customer changes buying behavior, and your forecast doesn’t keep up. So everything downstream, and again, we talked earlier that forecasts really are at the front of this process, and then 0:9:33.12 –> 0:9:33.252 Brian Haydin Thank you. 0:9:36.572 –> 0:9:57.372 Nick Miller you know, from the forecast drives purchasing. It also forecasts error can drive inventory optimization and inventory level setting. But basically everything downstream, your production, purchasing, labor, inventory, all of that inherits the error. So the second one is capacity waste. So this is when your plan doesn’t match reality. 0:9:57.852 –> 0:10:16.452 Nick Miller Or maybe you have capacity, you’re just not taking advantage of that capacity. Maybe it was right, the forecast was perfect, but the schedule wasn’t turned into actual production constraints. You know, bottlenecks, idle labor, unnecessary square footage, you know, anything that is… 0:10:16.572 –> 0:10:35.372 Nick Miller reducing our throughput with the amount of capacity that we should, that we have is a capacity waste issue. And you can’t solve all capacity waste issues. I mean, if your warehouse is large and you just have the warehouse, maybe you don’t need to fill it up and that’s okay. But in cases where capacity is a problem or if you’re filling bottlenecks, 0:10:36.332 –> 0:10:54.812 Nick Miller You may not need new capacity. You may need to take better advantage of that capacity. Maybe the capacity is being wasted. The 3rd is process waste. And that’s when something on the line itself drifts. A machine, a coating thickness, a temperature profile, a torque value. But any of those, if you don’t catch it, 0:10:55.52 –> 0:11:14.12 Nick Miller then those often result in scrap, reduced throughput, rework, etc. You know, that’s process waste. And depending on your manufacturing processes, there are some processes that still have operator influence. So if you have any manufacturing processes where 0:11:14.132 –> 0:11:33.572 Nick Miller You have a seasoned operator, and it’s kind of hard to learn the exact process, but they know how to control this complex system. And if they aren’t the guy there, if that person or the gal there, if that person does come in, you have a new operator, and it’s just not as efficient, the process is not quite as good. That could also be considered process waste. 0:11:33.692 –> 0:11:48.12 Nick Miller but for a different reason. But regardless, those are the three things that matter. And the thing is that for each one of them, they need different data. They need different model patterns and different decision paths. We’re going to take them one at a time. So let’s look at Forecast Weiss first. 0:11:50.412 –> 0:12:10.172 Nick Miller So Forecast waste is where most manufacturers we work with should start because the data is usually available and the leverage is usually highest. In addition, it just is at the beginning of the process, right? But let’s walk through these packs. The first is a straight demand forecasting. You’re trying to predict what your customers will need, when, and in what quantity. 0:12:3.372 –> 0:12:3.692 Brian Haydin Yep. 0:12:10.572 –> 0:12:29.772 Nick Miller And increasingly, at what location? So yeah, you can have a company-wide Forecast, but is it good enough to put the material at the location where it’s needed, when it’s needed? But in this case, the 5% accuracy improvement at most mid-size manufacturers translates into significant inventory reduction. You have fewer working capital dollars tied up in safety stock. 0:12:30.732 –> 0:12:53.932 Nick Miller fewer SKUs that go obsolete, fewer expedited freight charges to chase a demand surprise. And the business case really does right itself when the model improves, excuse me, when the model improves accuracy by even a few points. The second pattern is scenario modeling. This is less about having a single forecast and more about understanding kind of the range of outcomes. 0:12:54.732 –> 0:13:15.612 Nick Miller What does our production schedule look like if demand comes in 10% above plan? What about 10% below? What’s the cost of being wrong in each direction? Manufacturing leaders make better capital inventory and labor decisions when they can see the range, not just the point estimate. So a model that outputs a confidence interval is a little bit more valuable operationally than a model that just outputs that number. 0:13:16.172 –> 0:13:37.932 Nick Miller And the third is production lifecycle and obsolescence. This is a quiet source of waste. So you’re carrying inventory of a SKU, it’s three months from being phased out, but nobody updated the forecast, or you ramped down too early and now you’re scrambling to support a customer through that transition. Email helps you see those curves and just safety stock and reorder points before the scrap or the shortage even happens. 0:13:39.292 –> 0:13:57.652 Nick Miller So one concrete example we can give, we have a global automaker. We won’t name them, but they’re public about their work. They deployed an AI-powered planning assistant in about two weeks on Microsoft’s platform. And then to that two weeks, they removed the data. They were able to put that data foundation in place. 0:13:58.92 –> 0:14:5.692 Nick Miller Five years ago, there would have been a six-month project. So, Brian, can you talk to us about the planning piece for that? 0:14:6.972 –> 0:14:26.252 Brian Haydin I mean, at first, like 2 weeks, right? I mean, like, how crazy is it that, like, in just a couple of weeks, yeah, you can do some of this stuff. So, look, you know, the reason I think it can move that fast is, you know, the within the Microsoft stack is things like the planning and Fabric IQ. 0:14:10.332 –> 0:14:10.652 Nick Miller Yeah. 0:14:17.292 –> 0:14:17.692 Nick Miller Yeah. 0:14:26.772 –> 0:14:46.172 Brian Haydin What used to live in a separate planning tool, things like your budgets and the forecasts, those are now integrated into like the same governed framework, you know, the same governed data framework and foundation is all the actual data. So that might sound like a small thing, but it really isn’t. 0:14:46.572 –> 0:15:7.612 Brian Haydin The reason that most planning processes are painful is the reconciliation process. I know we do a lot of work in like building agents to help with some of those like reconciliation things, but you’re constantly stitching together what was planned with what, what was what you had planned with what’s actually happening on the floor. And so, you know, when those two things live in different tools that don’t talk to each other, 0:15:7.892 –> 0:15:27.52 Brian Haydin That’s where the pain points are. So when they live in the same place, which is the Microsoft ecosystem, then that ML forecast in the plant can naturally reconcile themselves. And so the decisions that come out of them are more of a comparison versus like a reconciliation process. You’re not fighting with the data as much. 0:15:27.252 –> 0:15:27.372 Brian Haydin Yeah. 0:15:28.332 –> 0:15:47.52 Nick Miller Okay. And so really, I guess the highlight from this is that quote at the bottom. It’s we’re not trying to create the perfect Forecast, but what we want to do is reduce the cost of being wrong. And I guess that’s the framing we’d like everyone to take back to any of the groups where planning is part of those conversations. 0:15:45.692 –> 0:15:45.932 Brian Haydin Yeah. 0:15:47.612 –> 0:15:48.92 Nick Miller Ann. 0:15:49.52 –> 0:15:54.412 Nick Miller So now that we’ve highlighted or double clicked on forecasting, let’s look at capacity. 0:15:55.292 –> 0:16:16.172 Nick Miller So what do we mean by capacity waste? So capacity waste is what happens when your plan and the reality don’t line up. You know, the forecast might have been right, the schedule might have looked good on paper, but the day didn’t go like the plan said it would, and then it cost you money. There are three places where we see this most often, and the 1st is scheduling, labor scheduling. 0:15:55.532 –> 0:15:55.852 Brian Haydin Yeah. 0:16:11.692 –> 0:16:11.732 Brian Haydin I. 0:16:16.292 –> 0:16:36.412 Nick Miller manufacturers often overpay for overtime because we carry idle labor and because shift planning is done, you know, by rough averages instead of predictive signals. And if you can’t predict with reasonable actually how many people with which skill you need on, let’s say, Tuesday, third shift in week 14, 0:16:36.892 –> 0:16:55.292 Nick Miller you can’t schedule for it. So you’ll spend less on premium pay and you’ll have fewer empty seats. So that labor scheduling, it’s difficult, right? You have to have the right data there. And also, you know, just understanding that plan and how is it tied to those people and who’s available. 0:16:38.892 –> 0:16:39.292 Brian Haydin Right. 0:16:54.972 –> 0:17:14.492 Brian Haydin I want to interject that really quick because we were talking to this production company that last year that was really, it blew my mind the way that they do business. They have like, they, on an average day, they probably have like 100 people working in their production floor, but they only have like 20 full-time employees. 0:16:56.972 –> 0:16:57.612 Nick Miller Yeah, sure. 0:17:3.692 –> 0:17:4.172 Nick Miller Yeah. 0:17:14.812 –> 0:17:36.812 Brian Haydin The rest of it, they use temp workers. And the interesting thing about the waste with the temp workers is that the day before, they have to predict how many people that they need. And they’re going to get the temp workers. They call them up and say, we need 72 people. They get 72 people to go up. The minute they show up, they got to pay them for the day. Like, that’s the contract. That’s how the contract works. So if they miss that forecast, 0:17:15.92 –> 0:17:15.452 Nick Miller Yeah. 0:17:28.892 –> 0:17:29.292 Nick Miller Right. 0:17:33.52 –> 0:17:33.372 Nick Miller Yeah. 0:17:37.212 –> 0:17:52.252 Brian Haydin Like they send these people home and they just have to pay for the labor. It’s crazy how that how that works. And so that’s why it’s really important to, you know, to get this right. I mean, when we were working on this project, we were looking at four or five, $600,000 a year in savings. 0:17:53.532 –> 0:18:16.92 Nick Miller Right, absolutely. And I remember that there are a lot of different levers to pull. Part of it was that forecasting plan of what was needed. And that was that wasn’t often not correct. And then I think some other stuff, there was some basic things in there that didn’t even need ML. Like, if you have the people there, let’s go ahead and pull in these, pull those production jobs in a little bit and make use of those, the labor that we had. 0:18:0.372 –> 0:18:0.732 Brian Haydin Yeah. 0:18:16.492 –> 0:18:35.612 Nick Miller A lot of issues going on there, a lot of room for improvement. But hey, that’s what we’re here to help. You know, if you have lots of room for improvement, then that makes the projects even better. You get much better outcomes. Yeah, thanks for having that, Brian. So that second, so in addition to the scheduling of labor scheduling, the other thing is production sequencing. 0:18:26.172 –> 0:18:26.492 Brian Haydin Yeah. 0:18:35.932 –> 0:18:55.132 Nick Miller So that’s the order in which you run the jobs. And this matters a lot, because the order in which you run them determines how many changeovers you do, determines how much downtime you create between configuration changes or where bottlenecks emerge. And ML doesn’t replace your scheduler, but it can suggest sequences that minimize change over time. 0:18:52.652 –> 0:18:52.692 Brian Haydin I. 0:18:55.452 –> 0:19:17.852 Nick Miller given the constraints you’re working with. So, you know, we’ve seen 5 to 10% throughput improvement on already tuned lines just from better sequencing. And that’s really just revenue that you didn’t have to spend capital to get. So you basically increase your capacity without any sort of capital expenditure, which is what you want. That’s where you’re looking at. Extra revenue, no additional cost. 0:19:15.412 –> 0:19:15.532 Brian Haydin Yeah. 0:19:20.252 –> 0:19:37.892 Nick Miller That one’s a really fun one, actually, and it can be fairly complex, but it’s really nice when you get to squeeze more out of that capital you’ve already spent. The third one that we’re looking at is warehouse and material flow. And this is where you slot inventory. So where do you slide it? And that matters. If you haven’t put any thought in it, 0:19:38.12 –> 0:19:56.732 Nick Miller then you have things all over the place, all right? And you may have fast movers at the back of the warehouse. But what you really want is you want fast moving SKUs near the door, slow movers in the back, you know, seasonal items repositioned, and getting them ready for whenever they need the demand, when that seasonality demand hits, that you have them staged and ready. 0:19:57.292 –> 0:20:15.692 Nick Miller And the data for this is almost always in your warehouse management system. From inventory transactions also comes in as part of your forecasting information. And what Email adds there is the ability to predict velocity changes, so seasonality, product launches, customer behavior shifts. 0:20:16.92 –> 0:20:24.732 Nick Miller And we can adjust, and with that, we can adjust the layout before the inefficiency shows up as a picker’s walking miles that they didn’t need to walk. 0:20:25.852 –> 0:20:36.12 Nick Miller So one pattern worth pointing out is, you know, looking at the bottom of the slide, where it says Forecast, and that leads into Operations Agent, it leads into Schedule Change. 0:20:37.132 –> 0:20:57.492 Nick Miller So that error used to require a human in the middle, and that human would translate the model output into a decision and then to an action into another system. So it was a very manual process. And increasingly, operations agents, and we’ll get into what that means a little bit later, but increasingly, those folks can connect those directly. 0:20:58.732 –> 0:21:4.332 Nick Miller Brian, do you want to talk quickly about how that happens and what operations agent, you know, what they actually are? 0:21:1.932 –> 0:21:2.292 Brian Haydin Yeah. 0:21:4.732 –> 0:21:27.52 Brian Haydin Yeah, you bet. So, like an operation agent is really, it’s a piece of software that monitors the live data and then reasons about what’s happening using the business rules that we’ve kind of that we’ve set up and configured. And when appropriate, it’ll take the, you know, some sort of action, you know, that matches the conditions, that matches a policy, something that we’ve set up. 0:21:27.532 –> 0:21:46.692 Brian Haydin It’s not like, I wouldn’t think of this as like a wild, wild west, you know, autonomous, you know, set, you know, agent just making willy-nilly decisions. Typically what we’re doing is we’re doing bounded scenarios. We know what the, you know, what is supposed to happen given these certain conditions. You, we are defining what it allows. 0:21:41.132 –> 0:21:41.292 Nick Miller Okay. 0:21:46.812 –> 0:22:5.132 Brian Haydin What it what the agent’s allowed to do, and then and then we can define when the escalations happen, like when the approvals, you know, need to happen or when there’s certain thresholds, you know, in order for to perform these actions. You’ll get a log of each one of these decisions as part of the audit process, which I think is really important. 0:22:6.12 –> 0:22:26.732 Brian Haydin And right now, the Microsoft platform is starting to come out with a lot of these features. Some of them are preview, some of them are emerging as like GA things, but we’re seeing this pattern emerge and the tools to support it, right? So an ML model is going to produce a prediction. 0:22:27.52 –> 0:22:48.52 Brian Haydin You know, an agent is going to take those predictions, apply business rules to it, and then somehow there’s a decision that’s made whether a person is going to make it, you know, perform the actions, or if this is good enough that we’re going to allow the agent, you know, to perform those. And so that’s kind of the key point at the bottom, right? The combination here is more valuable 0:22:48.892 –> 0:22:52.492 Brian Haydin that this all works together than just each one of those little pieces on their own. 0:22:53.172 –> 0:23:15.852 Nick Miller Yeah, absolutely. And I think a key part of that is that when you have this new setup with that agent in there, you can go a lot faster. You don’t have to slow down your processes. And the way that this usually happens, and it can be scary, right? Oh, we’re going to give, you know, the AI overlords, you know, the ability to do what they want in my production floor. 0:23:4.252 –> 0:23:4.652 Brian Haydin Yeah. 0:23:16.12 –> 0:23:39.692 Nick Miller No, that’s not what we’re saying. But you have bounded, and there’s a process of getting comfortable with it. So maybe if you’re concerned, the way that we often do it is that you have the agent would recommend, hey, here’s the action I would take, and you have a human approving it. And then once your team gets comfortable with the fact that this agent is making pretty good decisions, then maybe you can automate some of the lower costs to the lower 0:23:40.92 –> 0:24:0.132 Nick Miller impact decisions so that you can focus on the higher value decisions, right? And maybe you never fully automate away some of the big picture things, right? So if we have, for example, you see you have a customer order that is maybe predicted and it’s a 50% probability, but it’s huge, maybe we don’t start building that. 0:24:0.332 –> 0:24:7.52 Nick Miller without human approval, right? So that’s the idea there. We can start slow, build into it, but the point is, is you make it faster and faster. 0:24:0.572 –> 0:24:0.932 Brian Haydin Yeah. 0:24:2.412 –> 0:24:2.732 Brian Haydin Right. 0:24:7.932 –> 0:24:14.252 Brian Haydin Yeah, absolutely. So, yeah, I’ll go ahead and take this slide. You know, the… 0:24:15.692 –> 0:24:34.572 Brian Haydin I would say like, where things kind of get interesting in manufacturing is process waste. And I think this is where the Microsoft stack, you know, can really, where we can lean in in other more creative ways. So process waste is like that, that like pretty much lives closest to the production line. 0:24:35.132 –> 0:24:56.172 Brian Haydin And, you know, this is where I think the story has changed in the last, you know, four or five years. Some of the patterns that I’ve seen is like using things like computer vision. We’ve been doing this for quite a while, but it’s gotten really, really good. One scenario might be like a camera looking at a product or a process. 0:24:56.492 –> 0:25:17.852 Brian Haydin And then, you know, just having the model do a classification exercise, like what is it that it’s seeing? But you can do that with a little bit more refinement and do defect detection. Maybe that’s on a coding line. Maybe that’s on like, I don’t know, there’s this manufacturer that had these like plastic, you know, mold injections to shrink wrap around a thing. 0:25:18.252 –> 0:25:39.292 Brian Haydin and was looking for like cracks on the surface. And so you get like, you know, surface, you know, inspections on some of that as well. And, you know, I think that like that computer vision story, it’s been around for a while, but like the models themselves, the amount of training that has to happen, you know, today versus what you used to have to do. 0:25:39.692 –> 0:25:58.812 Brian Haydin is significantly different. So take that scenario with the form, you know, the form stamping and the plastic and looking for a crack or some sort of thing. I mean, you’d have to have 10s or, you know, almost hundreds of thousands of images to train for some of those minute, like little defects. 0:25:59.132 –> 0:26:20.92 Brian Haydin And now you can get away with a lot smaller sample size. You know, often we can get results in the hundreds rather than thousands. Not perfection, but we can start seeing results, right? I mean, that’s kind of like to me is just like how many, I’ll even give you just a fun example, one that I played around with. 0:25:59.212 –> 0:25:59.452 Nick Miller Yeah. 0:26:10.12 –> 0:26:10.812 Nick Miller Yeah, problem. 0:26:21.92 –> 0:26:40.932 Brian Haydin when like ChatGPT 2 and 3 kind of came out, I started throwing like I was doing a bear hunting that year and I had all these trail cam images of bears. And I’d have like, I had this one picture that I kept trying to figure out how could I get this to work right? And it had a picture of a sow with like 3 little cubs and the cubs were like, 0:26:25.212 –> 0:26:25.612 Nick Miller Yeah. 0:26:41.452 –> 0:27:0.212 Brian Haydin like in different places, right? Like one’s got its head poking out behind the log and the other one’s kind of like here. And I’m like, I was giving it instructions. How many, how many bears do you see? And it’d be like, okay, I see two bears, you know, and I’d be like, no, there’s three sows or there’s one sow and three cubs. And it’d be like, 0:27:0.332 –> 0:27:19.772 Brian Haydin Oh yeah, you’re right. I see three, like, and I’m like, are you sure you’re right? Can you tell me where they are? So I kept really trying to give these things. And I mean, man, I pretty much gave up for it, gave up on it a couple years ago. It just wasn’t working. But I went back like a couple months ago and like, man, these models can pick that stuff up really, really well now. 0:27:13.532 –> 0:27:13.852 Nick Miller Yeah. 0:27:20.252 –> 0:27:20.692 Nick Miller Yeah. 0:27:20.252 –> 0:27:22.412 Brian Haydin Out of the box, it’s cool. 0:27:22.652 –> 0:27:40.812 Nick Miller Yeah, and not only are the models just better out of the box, but even when you have, now that we have these multimodal models, you can actually use them to help you create that training data set for the model that’s even better out of the box. So you could even, and depending on the specific process, 0:27:34.172 –> 0:27:34.332 Brian Haydin Ohh. 0:27:35.772 –> 0:27:36.52 Brian Haydin Yeah. 0:27:41.212 –> 0:28:1.52 Nick Miller and the urgency of the timeliness of the prediction, you can actually use these multimodal models for prediction first. And then as your process is getting these trained data, then you can use these really awesome out-of-the-box models to then, you know, train a traditional computer vision model and then retire some of that multimodal. 0:28:1.172 –> 0:28:4.812 Nick Miller a little bit more expensive model prediction. 0:28:2.612 –> 0:28:2.892 Brian Haydin Yeah. 0:28:4.52 –> 0:28:4.252 Brian Haydin Yeah. 0:28:5.452 –> 0:28:27.612 Brian Haydin Well, speaking of prediction, that was the next thing that’s on the slide here, right? So we’re talking about a different prediction, but like predictive maintenance. So this scenario, people been doing this a while. I know a lot of manufacturers are really investing in this, but it’s having like, you know, pretty decent sensor data coverage. Maybe that’s, you know, on some of the motors, the compressor, the hydraulic system. 0:28:28.12 –> 0:28:51.772 Brian Haydin And like over time, the model is going to be trained and understand what normal looks like. And then when it starts to see signature drift in, you know, the performance of the equipment, it can send out some alerts and have you look into like a failure mode. So the business case for this is pretty well established. And what we’re seeing in a lot of scenarios where it makes sense. 0:28:52.172 –> 0:29:11.612 Brian Haydin is that you can shave, you know, 20 to 30, you know, maybe even like, you know, 40% of downtime off, you know, and those aren’t aspirational numbers now. Those are like realistic targets that companies are using. And then like third, I would say like anomaly detection. 0:29:6.812 –> 0:29:7.12 Nick Miller No. 0:29:12.652 –> 0:29:34.492 Brian Haydin You know, and again, someone on the processing signals, but like temperature, pressure, torque, you know, things like that that you might be using in the, you know, inside of your manufacturing process. Anytime the series, like these time series signals drift off of its normal envelope, you can set up alerts. And it’s not just that the model is telling you that something’s wrong. 0:29:36.12 –> 0:29:54.252 Brian Haydin you know, it’s helping you figure out where the problem is and helping you guide, you know, to a solution so you don’t have like a catastrophically bad, you know, outcome. So I think these are kind of like the areas where, you know, some of the like more advanced ML like tooling within Azure. 0:29:54.572 –> 0:30:13.372 Brian Haydin has started to mature and kind of help out a little bit. So, but anyway, moving on, like, what does this mean? You know, where is this stuff going to run? Because, you know, some of these facilities, they might be like in the middle of nowhere. We’ve got, you know, some West Texas places that we’re dealing with stuff. 0:30:14.412 –> 0:30:36.332 Brian Haydin You know, the cloud, you know, can be pretty good for like training models, but like when I’m trying to run these like low latency decisions or like I have high velocity machines that are like pushing parts through a thing, the cloud’s not good enough, right? So it’s not really great for that 200 millisecond latency. 0:30:30.812 –> 0:30:30.972 Nick Miller Okay. 0:30:31.892 –> 0:30:32.252 Nick Miller Bye. 0:30:37.212 –> 0:30:58.92 Brian Haydin And the round trip, you know, heading up in Azure to, you know, a facility in West Texas is just, it’s kind of painful, right? So I also think that’s where like this is getting better. We can do some of these like models locally, you know, and use the Foundry tools, the Microsoft Foundry. 0:30:58.652 –> 0:31:20.972 Brian Haydin in a local sense, running on Azure Local, or even having models that we can deploy to some of these edge devices that can do the processing there too. So that lets you take the models, you know, basically from your like Foundry catalog where you’ve developed it and run them on the plant hardware, like the stuff that’s on the floor. And now you can start getting instead of to, you know, 200, 300 or 0:31:21.292 –> 0:31:45.612 Brian Haydin you know, 500 millisecond latency, you can get like sub 50s, you know, and not have a cloud dependency. So if something happens with a spotty internet connection, you’re not like, you know, having to shut your production floor down. So those are some of the cool things that I think, you know, we’re starting to see. I’ve been talking about local Foundry and local AI workloads for a couple of years. 0:31:25.612 –> 0:31:26.12 Nick Miller Yeah. 0:31:28.172 –> 0:31:28.332 Nick Miller Yeah. 0:31:46.132 –> 0:32:3.852 Brian Haydin And it was kind of aspirational to a certain degree, but I think 2026, like you’re really seeing maturity, not only in the platform tooling, like, you know, like Foundry Local, but the hardware itself is becoming a lot more capable to run some of these as well. So those are some of the practical changes, I would say. What do you think? 0:31:49.52 –> 0:31:49.532 Nick Miller Right. 0:31:57.612 –> 0:31:57.812 Nick Miller Yeah. 0:32:1.132 –> 0:32:1.332 Nick Miller Yeah. 0:32:4.172 –> 0:32:23.692 Nick Miller Yeah, I totally agree. I was listening to a podcast with Lex Friedman and Jensen Wong from Nvidia, and they were talking about they’ve got GPUs now on satellites. And the use case for that is, it’s obvious actually, these camera, the cameras on these satellites are just 0:32:11.212 –> 0:32:11.612 Brian Haydin Yeah. 0:32:23.772 –> 0:32:44.812 Nick Miller absolutely amazing to have incredible resolution, but you’re talking about, you know, 10s or hundreds of gigabytes per image. You can’t stream that down to a cloud service and get a prediction, right? So they’ve got GPUs on the satellite and what you can still get, and you can do this, you know, I think the local machine learning and the local 0:32:45.612 –> 0:33:4.732 Nick Miller Even some local small LLMs are valuable where you have the process and where the heavy data is and where you have the low latency, you can still get the prediction and you can extract the information you need and you extract that smaller signal that you need from the process and still have it remote. 0:32:58.92 –> 0:32:58.212 Brian Haydin Matt. 0:33:4.972 –> 0:33:25.532 Nick Miller And so anyways, I thought that was pretty cool that they can get the predictions or they can get the, they can extract the features of the things from these satellite images without transferring those really huge images back for prediction. So the edge, I think, is definitely the capability of the edge for models and also 0:33:20.492 –> 0:33:20.852 Brian Haydin Yeah. 0:33:25.812 –> 0:33:29.772 Nick Miller some of these small language models is really powerful. Absolutely. 0:33:29.532 –> 0:33:43.452 Brian Haydin Be nice if we could just send some of those GPUs to, you know, some of those exploratory satellites that are like out on the, you know, outer regions of the, yeah, it’d be kind of cool. Well, all right, so. 0:33:38.492 –> 0:33:40.12 Nick Miller Yeah, yeah. 0:33:42.492 –> 0:34:6.52 Nick Miller Yeah, I think the upgrade process is fairly difficult. So I think maybe it’s going forward, but still, it’s still, I think it’s a cool use case and it really does show that local prediction is very valuable in many circumstances, actually. But so we’ve walked through this, right? We’ve walked through forecasting waste, capacity waste, process waste, those 3 buckets. They’ve got several use cases we talked about. 0:34:6.412 –> 0:34:18.652 Nick Miller how we were grounding those and reducing the cost of being wrong. And so I think the question we all need to ask ourselves right now is, yeah, is this achievable for me? Is this achievable for my company? So let’s talk about that. 0:34:21.212 –> 0:34:25.532 Brian Haydin Uh, yeah, no, I was gonna say it, like, um… 0:34:21.532 –> 0:34:22.972 Nick Miller So, yeah, go ahead, Brad. 0:34:26.972 –> 0:34:43.532 Brian Haydin I was kind of looking forward to this slide because before we started this today, before we started the webinar today, you and I were bantering about just this topic, right? That what really is important. So I’ll let you take this slide or I’ll let you lead it off. 0:34:37.212 –> 0:34:37.532 Nick Miller Yeah. 0:34:44.412 –> 0:34:44.892 Nick Miller Okay. 0:34:46.772 –> 0:35:9.212 Nick Miller So when we talk about manufacturers and machine learning, the question we get most often is, you know, are we ready? Because if you haven’t done it, how do you know if you’re ready? And what most leaders mean by that, and hopefully what they mean is, is our data good enough? And that’s a very good question, but it’s not always the right question. 0:35:9.532 –> 0:35:30.892 Nick Miller I think the right question is, is our organization ready to run a model in production? Because those are two different things, right? You can have the data, but then you as a company, you don’t have all the other things to support running that model in production. So the model itself, you know, getting the data, training it, evaluating, deploying it, it’s maybe 20% of the work in the neural project. 0:35:17.52 –> 0:35:17.372 Brian Haydin Yep. 0:35:31.132 –> 0:35:38.132 Nick Miller and the other 80% lives in three other places. So Brian, do you want to walk us through those other, the everything else? 0:35:39.452 –> 0:35:58.252 Brian Haydin The 3 places. So, well, yeah, I mean, we talked a little bit about the semantic foundation and what does your data actually mean? And it sounds like a trivial question, but it really isn’t. Every manufacturing organization that I’ve worked with has at least three different definitions of what on-time in full means. 0:35:44.172 –> 0:35:44.572 Nick Miller Yeah. 0:35:58.812 –> 0:36:20.92 Brian Haydin Like, you know, or just even just the first part, what on-time delivery means. So it depends on who you ask. Like the shipping team is going to measure when the truck leaves, right? Or the customer team, you know, is saying, well, did we get there on the customer’s promise date? So, you know, it’s the same data, three different definitions. 0:36:3.452 –> 0:36:3.692 Nick Miller But. 0:36:20.812 –> 0:36:39.692 Brian Haydin And how that really affects us in this scenario is that that turns into three different forecast targets. So before a model can start to produce useful predictions, the organization really has to agree on what number is it going to predict. So that’s how this semantic foundation is really going to help out. It helps you 0:36:27.572 –> 0:36:28.92 Nick Miller Right. 0:36:40.12 –> 0:37:0.252 Brian Haydin get to a common understanding and a common definition so that what you’re designing for is the right target. And that’s where, you know, Fabric is going to play a big part of it. It helps you with that ontology and definition of what on time is really going to mean. And so the second phase, the second piece of this is the decision path. 0:37:0.652 –> 0:37:24.332 Brian Haydin Who’s actually going to act on the prediction? And not only that, but what is their authority? What’s the workflow definition going to look like? What happens if they disagree with the model? Because just like you’ve mentioned this already today, it’s prediction. It’s not always right. You still have to use good judgment. And what happens when the model is wrong? So these are organizational design questions, not 0:37:24.652 –> 0:37:44.812 Brian Haydin like technical data questions to your point earlier. They have to be answered before you take that model and put it in a live production sense. And then the third piece is really the operationalizing it. What happens after the model is, you know, in production. And this is the one that most teams don’t really think about until like the pain. 0:37:45.52 –> 0:38:2.12 Brian Haydin is enough that it starts to hurt. And I’m going to go a little bit deeper into this in the next slide, because that’s where I’ve been spending a lot of my time talking. And frankly, I think this is one of the least understood parts of the process that we need to dive into. What are your thoughts? 0:37:45.692 –> 0:37:46.172 Nick Miller Thanks. 0:38:3.292 –> 0:38:12.492 Nick Miller Yeah, I think that’s great. And I like this slide because at the beginning we talked about what has changed and what makes these projects more realistic. And I think we 0:38:13.692 –> 0:38:32.572 Nick Miller really talk about this 80% is what are the other parts of the problem that we can solve a lot more easily now? Not that it’s easy, but easier. So it solved a lot of that plumbing. You know, we’ve got data into model, we get into production, we have monitoring infrastructure. And so there are things that it hasn’t solved, right? It hasn’t solved that. 0:38:20.972 –> 0:38:21.452 Brian Haydin Yeah. 0:38:33.212 –> 0:38:48.892 Nick Miller organizational and operational work around that model. That part is still there. We have to make those decisions, come to the agreements, et cetera. And that’s part still on you or us together, you know, with the help from a partner. But let’s get to that last piece. 0:38:44.572 –> 0:38:44.772 Brian Haydin Okay. 0:38:47.852 –> 0:38:48.12 Brian Haydin Barr. 0:38:49.692 –> 0:39:7.292 Brian Haydin Yeah, so like operationalization, like we’re both struggling with this word, right? I’m shortening it up to ops. I’m just going to use ops, right? But look, I want to be a little bit direct about this part of the conversation. 0:39:8.492 –> 0:39:27.452 Brian Haydin Nobody else is going to bring up, you know, this type of topic, you know, you know, today, and this is going back to like a talk that I’ve been doing around quite a bit, Agent Ops, and how do we actually like think about the operationalization of, you know, some of these agents that we’re building. 0:39:28.12 –> 0:39:48.332 Brian Haydin Most manufacturers, they’re going to run their first machine learning model in production with like less observability than they have on their actual factory floor. Think about that. Like you have these agents that are doing things and you have no idea what’s happening and you have no idea to go back and like look at the decisions that it made. You have torque sensors, vibration monitors. 0:39:32.732 –> 0:39:33.12 Nick Miller Thanks. 0:39:49.132 –> 0:40:6.972 Brian Haydin Maybe some like pressure transducers on a CNC machine. And so these things are really expensive pieces of equipment, right? Some of these are hundreds of thousands of dollars, if not more. And you spend so much time on like maintenance and everything, you check everything, but you’re not doing that with your forecast models and your agents. 0:40:7.292 –> 0:40:27.412 Brian Haydin You’re not like, you know, checking the oil pressure, like, you know, on your agent. And we see this kind of stuff happening constantly. There was such a push to run, run, run with these agents. Nobody thought about the bigger picture of how do you maintain them? How do you look for problems in the plan? 0:40:27.852 –> 0:40:47.52 Brian Haydin So there’s kind of four things that I’ll talk about that, you know, that you need to start thinking about and planning for up front. The first is like this concept of drift. And when the world changes and your model doesn’t really keep up with those changes, doesn’t have the understanding of what 0:40:47.212 –> 0:41:6.172 Brian Haydin is going on. That’s what I’m talking about with Drift. So like a new product line launches and the model’s never really seen what those demand patterns are going to look like. They don’t know how to detect the signals. Or maybe a supplier changes the lead times, you know, or maybe they go out of business, right? And you got to switch. 0:41:6.572 –> 0:41:27.532 Brian Haydin So those are like, you know, some of the things that happen underneath the covers that can cause these models to drift in their predictability. And so it’s not that the model broke, it’s that it’s just got out of date and we weren’t doing like some sort of like MLOps or continuous improvement on it to keep it up to date. 0:41:28.252 –> 0:41:47.292 Brian Haydin The second topic is like scoring. So how do you know how the prediction is right? And there’s a bunch of different ways that you can look at this, but you have to think about what right means in the context of your particular scenario. You might have actuals coming in every day, and if you’re comparing them to the… 0:41:47.772 –> 0:42:6.822 Brian Haydin to like the model, is that the right thing to be doing? Most of ML systems that we have, you know, that we’ve done audit have weaker post-deployment scoring than most of the QA stuff that like developers are doing. So you gotta like think about like, am I doing similarity scoring? Am I doing actual 100%? 0:42:6.852 –> 0:42:25.932 Brian Haydin break fix, you know, different kind of scenarios. So know what your scoring is going to be and then define it and build that as part of your process so you can monitor when you see some of those drips. And the Microsoft platform has those capabilities kind of baked into it. 0:42:26.332 –> 0:42:45.292 Brian Haydin In the agent framework, we have eval, you know, harnesses being baked into it so that, you know, if you have the scoring baked into, if you know what your eval scoring should look like, you can get that automatically fed back out. The Microsoft Foundry has the control plane. 0:42:45.692 –> 0:43:5.572 Brian Haydin We can use things like Agent 365 to control, you know, to look at some of those things, then make actions or decisions about what the agent can’t do based on certain scoring scenarios. So this is where the mature, the platforms are starting to mature, where the agents need to as well. 0:43:6.212 –> 0:43:11.972 Brian Haydin Um, and then let’s talk a little bit about latency and cost, you know, decisions, so… 0:43:13.332 –> 0:43:33.732 Brian Haydin Manufacturers typically operate in really, really tight, you know, kind of financial margins. And so we need to be tight on how much money we’re investing in running these operations in the plant floor as well. So in like a cloud-based ML system, you need to measure the cycle time per prediction and cost per decision that you’re making. 0:43:34.92 –> 0:43:55.732 Brian Haydin How long does it take that model to respond? Is it keeping up with the pace of your machinery? How much does each call cost in the compute stuff that you’re actually spending on Azure? Can you predict that 100, you know, like ahead of time? So that, I mean, if you’re making something like a million predictions a month, like, and they’re a penny a piece or, you know. 0:43:50.852 –> 0:43:51.12 Nick Miller Yeah. 0:43:55.812 –> 0:44:15.892 Brian Haydin you know, it starts to add up really quick, right? So we want to help you make sure that you don’t have like a cost surprise at the end. Now we can shift, we talked about like doing local AI processing. We can shift some of that, but some of it is going to require more advanced models or different types of processing. Can’t shift everything into a local scenario. 0:44:16.932 –> 0:44:36.172 Brian Haydin But like, you got to start thinking about how this is, you know, how this is going to play out in the long run. And then we talked a little bit about the ML Ops as part of like drift. So the 4th kind of concept is retraining. So how often do you retrain your models? The answer might be daily, right, Nick? 0:44:36.252 –> 0:44:57.412 Brian Haydin The answer might be monthly. You know, it depends on the scenario. But people aren’t planning for that action to happen. They’re just saying, well, the forecast is wrong. We got to go retrain the model. And there’s better ways to do that. Like we don’t need to have a reactive policy anymore. We should, you know, look at like what are some of the drift signals that occur. 0:44:39.572 –> 0:44:40.612 Nick Miller Yeah, obviously. 0:44:57.892 –> 0:45:17.292 Brian Haydin so that when we detect some of these drift signals, that we can start to rebuild these models, you know, on an automated fashion. So the reason like I want to walk, you know, through some of these concepts is not really to scare anybody off, it’s really to give you the vocabulary that you should be thinking about. 0:45:17.372 –> 0:45:37.572 Brian Haydin before you walk into projects like this, so that when we’re having conversations or you’re having conversations with another partner, that you can start asking the right questions to make sure that you’re not just getting this off-the-shelf, half-baked solution that’s not going to provide results. You want something that’s going to be done intentionally and 0:45:38.132 –> 0:45:56.932 Brian Haydin that works for your business. So what I like to tell, you know, customers, and this is kind of what we were bantering about a little bit, is like the model, that’s kind of like the easy part. And, you know, but, you know, building those models, that’s a kind of like, you know, well understood sort of process. 0:45:57.532 –> 0:46:16.772 Brian Haydin In fact, like there’s not a lot of, I don’t know, maybe Nick, not even necessarily a lot of value in like pure custom model building as much anymore with the out-of-box capabilities, but it’s running it. That’s where the discipline and that’s where the cost is. Making the right decisions, understanding how you’re going to use the predictions. 0:46:17.52 –> 0:46:24.852 Brian Haydin and evaluate those scenarios that are coming in, that’s the stuff where I think concurrency can provide the most value. What do you, what would you add? 0:46:23.892 –> 0:46:24.292 Nick Miller Yeah. 0:46:25.812 –> 0:46:46.852 Nick Miller You know, I think you just described my job is totally easy to do and custom model training is not valid anymore. I might disagree with that. Maybe I didn’t hear you correctly. But I will say that, yes, it is easier, but I think the process that as far as building a model 0:46:39.172 –> 0:46:41.812 Brian Haydin It’s easier, it’s easier. 0:46:47.252 –> 0:47:9.12 Nick Miller The point that you’re making is still valid, that there is all this other stuff around building a model that needs to be considered, right? So yeah, the process of building a model, understanding that business process, the, you know, trying to represent that business process with data, you know, making all those decisions, that’s where you still kind of make your money as a data scientist, but the like model training piece, 0:47:9.92 –> 0:47:30.612 Nick Miller there’s so much automation in that now that it’s really more about the data understanding. And then again, outside of that, the bigger picture that you said for sure. And there’s so many things I could add to this, but I won’t. But one small example for drift is, you know, we have lead times for our suppliers. You know, if you’re in manufacturing, how many of you have 0:47:13.172 –> 0:47:13.492 Brian Haydin Yeah. 0:47:31.12 –> 0:47:50.572 Nick Miller procurement lead times and when does it get updated, right? Probably never. So maybe do we need to have a lead time prediction? Do we need to build a model around lead time, like stated versus actual? And that may be a thing. You know, in the past, I was working at a company that… 0:47:35.332 –> 0:47:35.732 Brian Haydin Yeah. 0:47:39.172 –> 0:47:39.412 Brian Haydin Ken. 0:47:51.172 –> 0:48:10.332 Nick Miller was basically a high-tech manufacturing company. I call it, we built Lego, we built Legos for R&D engineers, right? So whatever they wanted to do, we had all these different Lego pieces and they could build their own R&D test thing, right? And in that time, we had semiconductor constraints. 0:48:10.772 –> 0:48:30.212 Nick Miller and the lead times kept going up and up and up and up. And so if you aren’t incorporating those new lead times into your modeling and your forecasting, your inventory optimization, et cetera, you’re going to be in a bad place. You’re going to have a bad day, right? Sorry, a little soft part put there. But yeah, I think that 0:48:25.812 –> 0:48:26.132 Brian Haydin Yeah. 0:48:28.532 –> 0:48:28.772 Brian Haydin The. 0:48:30.372 –> 0:48:38.212 Nick Miller This is all relevant, right? So I just disagree with Brian a little bit. Whenever I can, I try to publicly disagree with him if possible. 0:48:38.612 –> 0:48:54.212 Brian Haydin And Pat does that too. She always says, when I use the word easy, it’s all relative to the person that you’re talking to, right? So let’s let, why don’t we land the plane and we promised something in the beginning. Yeah. 0:48:40.692 –> 0:48:40.892 Nick Miller Yeah. 0:48:42.932 –> 0:48:43.412 Nick Miller Yeah. 0:48:46.772 –> 0:48:47.892 Nick Miller Yeah, exactly. 0:48:49.172 –> 0:48:49.972 Nick Miller We can move on. 0:48:52.772 –> 0:48:53.332 Nick Miller Yes. 0:48:54.172 –> 0:49:14.772 Nick Miller A practical framework, right? So we described the high level framework. So now hopefully your brain has all those little compartments with which to put the information that we gave you. And then what we’d like to do is give you this ability to, you know, evaluate an opportunity or use case. So this is what I’d like you to take back to your team. 0:48:56.252 –> 0:48:56.612 Brian Haydin Yeah. 0:49:15.812 –> 0:49:34.892 Nick Miller on tomorrow, Monday, whatever day you meet back with your team. If you glean anything from this, if you feel like it makes sense, bring this back. On this card, there are 6 dimensions, and you just score each one from 1 to 5. And it tells you whether your first use case is actually a good first use case, or whether you’d be better off 0:49:35.172 –> 0:49:55.252 Nick Miller picking a different one. And what is exactly a one? What’s exactly A5? You know, that isn’t quite as important as having sort of a relatively score, a relative score that makes sense against different use cases. So when you talk about business value, you know, the $1,000,000 should be, if $1,000,000 is a three, 0:49:55.332 –> 0:50:14.212 Nick Miller Great. Just make sure that when you’re looking at all use cases, $1,000,000 is still a three, and $1,000,000 is in a five for one and three for the other. So it’s all relative. And I think part of this value is just going through the process of evaluating and having those discussions, because most people don’t even go through this level of detail when they 0:49:59.852 –> 0:50:0.172 Brian Haydin Yeah. 0:50:15.492 –> 0:50:33.372 Nick Miller Sometimes they come to us and say, hey, we’d like to do this thing. But there hasn’t been a lot of thought process or evaluation behind it, right? So let’s walk through each dimension. First is business value. Is the waste pool you’re trying to reduce big enough to matter? If you’re going to spend three months building a model that saves you $100,000 a year, it may not. 0:50:33.492 –> 0:50:52.692 Nick Miller be the right first project. And it may not. It could be, but maybe not, right? So decision leverage. Will someone actually change a decision because of this forecast? Or will it just sit in a dashboard? So if we have this awesome model and it doesn’t get integrated anywhere and no one’s going to change behavior based off of the model output. 0:50:53.12 –> 0:51:3.12 Nick Miller Yeah, decision leverage would be a one. If we can automate some of it and still high value or high impact decisions maybe are still human, maybe that’s a four or a five. 0:51:4.292 –> 0:51:23.732 Nick Miller And the third thing here is data availability. Do you have the historical signal needed to train a model? This is a little bit harder to quantify for folks who don’t do machine learning, but there’s a general thought that I have is that the more complex the process, the more kind of data points we need. 0:51:24.292 –> 0:51:43.252 Nick Miller And if it’s a very complex process with lots of data points, then the volume of data we need is going to be very large, right? If you’re not sure, we’re happy to talk to you about those things. So you don’t have to go through this all on your own. But in general, data availability. Do you have enough data? You know, quality data, too. You know, machine learning and AI, 0:51:43.572 –> 0:52:3.92 Nick Miller You can’t just, it’s not like pixie dust where you sprinkle a little on bad data and it becomes magical gold, unfortunately. So the fourth thing here is the outcome clarity. Can you measure whether the model helped out or not? Or did it hurt? Fourth, operational path. Can the prediction land in a workflow where the action gets taken? 0:52:3.572 –> 0:52:22.652 Nick Miller So for example, if we could do the prediction with the data, but we need, we can’t make the prediction locally or the tool isn’t available to happen locally, and we can’t get the prediction there until after the event has already passed, well, that’s no good, right? So that’s kind of the operational path. 0:52:23.92 –> 0:52:42.292 Nick Miller And then last is time to value. So can we prove or disprove this in 6 to 12 weeks? And when we say prove or disprove, we don’t mean deploy an entire solution that is fully baked out 100% with all the ins and outs, because you don’t really want to just jump right into it. 0:52:42.452 –> 0:53:3.492 Nick Miller You know, a lot of times you’ll want to make sure that this thing can work and have a part of that project or part of that process where we get the data, we do the analysis, we prove that it can work or that it can’t. Because sometimes there just isn’t signal in the data. Maybe the process is so complex or maybe we just don’t capture the right data signals. That’s a possibility. 0:53:3.772 –> 0:53:4.772 Nick Miller So, time to value. 0:53:5.652 –> 0:53:8.132 Nick Miller So, Brian, you want to score one together here? 0:53:17.92 –> 0:53:19.172 Nick Miller Yeah, you’re right. I don’t know if you have any time there. 0:53:19.212 –> 0:53:40.932 Brian Haydin We’re right up against the time. But I will tell you this, how about for those of you that are still on the phone call on the webinar, what I really would like to do is to score one with you guys. You know, so, you know, here’s, you know, here’s what I would probably, you know, recommend is for you and the organization. 0:53:41.652 –> 0:54:0.852 Brian Haydin Find one of these waste pools that you have in your organization that you want to uncover, and what does that actually cost you? We’re happy to share the deck with anybody that wants to reach out to us. I think Paige dropped a link into the chat. So fill that out, give us a little bit of feedback. 0:54:1.292 –> 0:54:24.92 Brian Haydin and let us know if you want us to, you know, score one of these things together with you. Pick a decision that you wish you could make earlier or a little bit more accurately, or pick one data set that you currently use that’s going to help you make that decision. If you bring those three things together, Nick and I can sit with you for half hour, an hour, and give you kind of an opportunity assessment. 0:54:24.172 –> 0:54:42.772 Brian Haydin Might have to, you know, go a little bit deeper to figure out what that would look like. But we’ll tell you if it’s a good use case or if you have what to take, you know, like we usually tell within the first half hour, hour, if this is something that’s worth, you know, diving a little bit deeper into. So any other last minute thoughts? I mean, any you want to add? 0:54:38.292 –> 0:54:38.492 Nick Miller Yeah. 0:54:43.92 –> 0:54:43.652 Brian Haydin Nick. 0:54:44.292 –> 0:55:2.972 Nick Miller No, I just think that if you were happy to jump in there with you, you know, going through that checklist, some of it’s judgment based and we’re happy to join that. And, you know, we have wonderful personalities, so we’d love to get to spend the time with you and have those conversations. So I hope this was valuable and look forward to, you know, connecting and 0:54:55.12 –> 0:54:55.172 Brian Haydin I… 0:55:3.92 –> 0:55:4.852 Nick Miller Hopefully, solving some of these waste issues. 0:55:5.252 –> 0:55:7.492 Brian Haydin Yeah, thanks everybody for joining us. Appreciate it.