View Recording: Show Me the Money: Measuring Real ROI on AI Agents

Derek Steckel (Panthers Conference Room) 0:20 Happy to be on one of these again. Derek Steckel. I lead a large portion of our enablement program here at concurrency, so training anyone from executives down to the department level down to individual level, how to utilize copilot, how to create agents, etc. So excited to be talking about money today. (Panthers Conference Room) 0:38 I love money. Need more money for AI? So I’m Jen Brown. Speaker 3 (Panthers Conference Room) 0:42 I’m a solutions architect here at concurrency, listening to customers and creating projects that do some really, really cool things. AI keeps changing. Agents are everywhere, and increasingly we’re focusing on how do we get the most bang for the buck. What things to start on 1st and you know it just it’s a great time to be in IT. (Panthers Conference Room) 1:08 Want to thank. Speaker 3 (Panthers Conference Room) 1:09 Our marketing team for putting this together. This is please follow us or link in to us on the start information. Here we put a lot of these events on and you’ll see them on our website on insights so. So that’ll be posted right after that, and so take a look. A lot of interesting topics. (Panthers Conference Room) 1:31 OK. Speaker 3 (Panthers Conference Room) 1:31 So we put together a model and so we are now Gartner and we’ve created our own model. But this model really isn’t for the industry, it’s for you. It’s about how to think about the ROI journey. And we’ve we’ve mapped out the steps Derek’s going to talk a lot about the vision and the analyzed the workflow. And I really like what you’ve got to share now. We’ll kind of take that through. But this model is what guides us. And I think you know, when you go talk to the finance part of the organization or really get consensus, it’s really important to do this work up front. And in my mind, I mean, it’s even something we can help do. Evaluations, I mean, it creates the design for the testing and the creation of the agent and certainly you know makes it look good when you when you can say this is what we said we were gonna do and you measure it and there you did it. Yeah, I. Derek Steckel (Panthers Conference Room) 2:28 Mean. I kind of think of it as a project plan. First of all, but also I mean if you wanna actually show ROI, you have to define what that starts with it. What is your starting point where those metrics that you’re looking at right now, those KPIs? So if you don’t have that baseline, then to your point, Jen, later on you’re gonna implement something and it’s like, OK. (Panthers Conference Room) 2:46 Did we change anything? Like what is this doing? What was? What was the result? Speaker 3 (Panthers Conference Room) 2:50 Of the money I I put into this. (Panthers Conference Room) 2:51 Yeah, exactly. Right. Derek Steckel (Panthers Conference Room) 2:54 So moving on. You know, I’m gonna hit on that VNA part of it to start. And what? I wanna start by talking about is. There’s a lot of tools, especially in the Microsoft ecosystem, so concurrency or Microsoft Shop. We talked a lot about copilot, copilot, studio and then foundry. Obviously there’s a lot of other AI tools, but I wanna start those today and when we talk about where we can get ROI from a lot of this conversation is going to be around copilot studio and foundry, more of the kind of right hand side of the slide. That you’re seeing right now. As a lot of folks may know that have joined previous webinars, I love talking about M365 copilot. I would say about half of my week is typically copilot. Microsoft 365 Copilot excuse me, and I’d be remiss if I didn’t hit on what cop. Can do, which I’ll do in a SEC. But I want to emphasize this right now that not everything needs to be an agent just because people have heard the term agent. That doesn’t mean that, oh, let’s move everything. Thing to be an agent. Let’s figure out how this process can be an agent or that process can be an agent. Sometimes M365 copilot just the chat that a lot of people have access to that can do a good enough job to get you to where you want to be. Also power automate flows. I will never forget a time when I was kicking off a project at concurrency and I. We were kind of figuring out, OK, how do we wanna go about solutioning this? How do we want to go ahead and make this a reality for our clients? And I share it in the room that during the kickoff, hey, we don’t think we’re actually gonna need to utilize AI for this. Like we think we can do more like a power automate automation flows in the background and a lot of people were like whoa, like, wait a minute, why do we not use AI for it? It’s like, well, there, there can be downsides, right? Like we can if something maybe more deterministic, we might want to use power automate for example. There can be certain costs that are associated with AI that aren’t associated compared to M365 copilot power automate. So just because we hear the term agent doesn’t necessarily mean that we should always say let’s move a process to be an agent. Not a problem once I talk about Microsoft 365 Copilot, which is a lot of ways I can consider it my baby then create it. But I do. A lot of training and adoption around it and I released it up to date on a lot of. What’s new with copilot? I love it. I think that everyone should utilize it and maximize their license if they have access to a tool like copilot or ChatGPT CLAUD, etc. But you can’t really. Quantify what that license cost gives you, at least in my opinion. So it’s a really powerful tool, but a lot of times your organization may pay $30 per user per month. Or maybe you have it through one of the E7 licensing for example. Well, that Microsoft puts out there. Yeah, but in my opinion, there’s no way where you can. If you’re acfo, you can go ahead and say, oh, I’m paying $30 per for Derek’s license per month. I can see that he is getting we’re getting $72 in ROI. For our company, there really isn’t a great way to do that. Speaker 3 (Panthers Conference Room) 6:13 Yeah, like, like you said, between automation and AI, there’s a difference. There’s a difference with people and their use of AI. Some people are more familiar with. Prompting and other things are trained. Do you see kind of a difference between productivity between people who are focusing on it versus? (Panthers Conference Room) 6:30 Yeah, absolutely. Derek Steckel (Panthers Conference Room) 6:31 I think one of the biggest ways I do see that is when people move beyond just using it for as another Google for example, like there’s an art to prompting right. And there’s this different use cases that if you’re HR, you’re going to be able to gravitate towards instead of someone that’s in finance, for example, or in sales. So showing them the art of the possible and getting them used to when they have to provide more details, more context on something, that’s what takes them to the next level. (Panthers Conference Room) 6:56 Yeah. Derek Steckel (Panthers Conference Room) 6:57 Going back here though, you know with that gap. The best example if you’re a CFO, if you’re someone on the call where you want to push me a little bit more on that, like there’s got to be some way, the best way that I have potentially seen to show some sort of ROI with copilot. Which I’ve tried with a few folks is if you go into the admin portal within M365 you can see the different users that have access to copilot. You can see things like how many prompts they generated over the past month. Week, quarter, et cetera. And from there you can try to say, OK. This will quantify an average prompt to maybe be 10 characters, 50 characters. Maybe do it by tokens. Regardless, when you go through that process. In my opinion, you can try to say just based off of what a token costs or some sort of metric that relates to a dollar value. I see that Derek submitted 400 prompts that we’re gonna quantify that as $120.00 that he’s used over the past month. Awesome. We paid $30.00 for this. He’s using $20 worth of tokens. That means that we’re making 4X on that, which sounds good in theory, but OK. What if I’m not using it in the right way? Maybe I’m saying the copilot. Try that again, it’s not working. Try that again. It’s not working. I’m not using it effectively, but I’m still generating tokens, right? So that’s still not an effective way to do that. Think about if you were to try to quantify a workflow by your organization that say you’re looking at how much copilot can ROI we’re getting from copilot. For people sending an e-mail. Well, some people don’t send emails utilizing copilot. Some people are using it in different ways. Some people may just want to have copilot draft at the start. Others might want copilot to ask him a few questions about the context. And then it’s gonna formulate that response. There’s no it’s not apples to apples. When you do this stuff. And that’s why, in my opinion, there isn’t a great way to quantify the ROI. For a license for Microsoft 365 Copilot, even though I will double and triple down that it is worth it, there’s a lot of soft ROI factors that do make people more. Speaker 3 (Panthers Conference Room) 9:12 Yeah, we know there’s AI there. You said something the other day that was interesting and that people should ask for AI. You should just give them a copilot license, but there should be sort of a small process of getting it ’cause if if it’s just free and you just have it, there’s not as much. (Panthers Conference Room) 9:27 Value there. Derek Steckel (Panthers Conference Room) 9:27 Yeah, it’s just like turning on licenses without having some sort of adoption cycle. To your point, Jim, one of the I talked to a leader at an organization a year or so ago and what their strategy was around copilot, which I found really fascinating, was they are going to give it to you by default. But what you can do is you can request a license and what that looks like is you go and present to like an AI committee or someone on that AI committee. Maybe sometimes it’s easy. Even your manager, depending on the size of your organization. In what you’ll do in that presentation is you’re going to showcase for 15 or 30 minutes, however long it is, what you’re going to do with that license, like how it’s going to make you more productive in almost every single time they’re going to give you the LIC. Whether you did a great job with the presentation or maybe there are some holes in it. But then they’re going to monitor and the portal usage over the next like 30 sixty 90 days to see. OK, is Derek actually utilizing a license like or is it just kind of turned on? He didn’t end up you doing anything? With it, so that gives you a good sense of OK, like this person is seems to be getting value out of it. They’re utilizing a lot. Maybe you can set up a check in call with them afterwards versus they’re not using it. Maybe we should have a conversation with them and or turn it off. Speaker 3 (Panthers Conference Room) 10:48 Yeah, little bit of training, little bit of you know, bump and and planning to to use it. Yes great value. Derek Steckel (Panthers Conference Room) 10:54 Yeah, definitely. So you already come here today to have me talk about in 365 copilot. You wanna know more about what? Where does ROI become measurable? And that’s when you utilize tools like Copilot, Studio, and Foundry. When you have these fairly complex agents that can do the work of. A process for example. I want to distinguish now between soft ROI and immeasurable ROI. When we talk about that M365 copilot, that’s where I think about soft ROI, right? It’s helping me do some research for a client. It’s helping me kind of go through as a search assistant, guiding me in knowledge. I QA and discovery helping me write an e-mail, draft a document, etc. But I still kind of going back and forth with it, still saying, hey, this looks good. This doesn’t look good. Keep trying to incorporate this. I’m using it as a thought partner. It’s effective, but still that’s soft ROI or the more you get into the measurable ROI, that’s where we can actually say, OK. We’re creating a file automatically. We are going through this process end to end, whether it’s a form, whether it’s a database update, whether we’re kind of communicating in e-mail format to a salesperson, for example. Hey we updated the database. The more work that the agent actually does, that’s where we kinda get into the ROI. Those hours save can be measured. The tasks eliminated can be measured. Different labor reallocation can be estimated. Process cycles can be compared, you know. Those are all options on the table. When you’re having an agent do something more autonomously, or a chunk of a task more autonomously, where it’s less so someone is, they’re helping someone do the work and it’s more so. It’s doing the work. With those checks and balances at the end. Speaker 3 (Panthers Conference Room) 12:44 So it’s like putting verbose on the agent. You know you can see what it’s doing, listing out and then kind of making value judgments based on that. Derek Steckel (Panthers Conference Room) 12:52 Definitely it’s a good way to think about it now. OK, Derek, that’s great. But how do we actually aggregate these different ideas that people may have? We’re a large organization. We have a team, an IT team, a marketing team. How do we decide what to build? I recommend having some sort of prioritized pipeline that you can go ahead and delve into, and the best way to do that in my opinion is have some sort of intake process where you have different folks from departments that are able to write up proposals or fill out. A form that then some sort of centralized AI committee can ahead in evaluate. So think about this. If you have an intake process, I wouldn’t recommend that intake process being challenging people at the organization as to how AI can better process. They might be seeing some things in the news they might see have some crazy ideas where they don’t necessarily know how it works at the company, or just how AI works in general. So what I recommend is having people talk about those manual painful tasks. What are those things that bog them down each day? They don’t need a solution. We just wanted that requirements gathering first. What’s bogging them down? What is taking up a lot of their manual valuable time where their brain power could be spent elsewhere at the organization? Making more meaningful change, driving some more creative decisions rather than just maybe entering things into an Excel sheet or doing some verbose manual processes. That’s what they can then route to that AI committee in a more aggregated form. To do that evaluation. Speaker 3 (Panthers Conference Room) 14:34 I mean, how much work is there? Just logging in to systems. That could be, you know, done more efficiently moving stuff around from place to place. (Panthers Conference Room) 14:38 Yeah. Speaker 3 (Panthers Conference Room) 14:42 Yeah, you know, that’s friction time. Derek Steckel (Panthers Conference Room) 14:44 Definitely. (Panthers Conference Room) 14:44 You’re like ah. Derek Steckel (Panthers Conference Room) 14:46 That’s. Yeah, that’s what you want to challenge folks to do is like, look for the friction. Don’t necessarily solution. Don’t tell us how. We could potentially figure it out. Maybe they do have a kind of pie in the sky idea and they wanna include that, but what that intake process should be more focused on is what are those manual tasks and where can we start from when we’re kind of solutioning here. Speaker 3 (Panthers Conference Room) 15:06 Yeah. And you know when it when it comes to solutioning, you extend it beyond just the tasks that have been referenced in, right? So there’s there becomes a whole process, absolutely. Derek Steckel (Panthers Conference Room) 15:18 So that’s where the BXT framework comes in. If you’re not as new to concurrency, maybe you’ve been on a handful of these before. You may have heard us talking about this before. Microsoft this is also an area that they like to recommend as well. But what it really is is it’s a structured lens to identify and prioritize those agent opportunities. We break it up into three areas, business experience and technology. Business you know what outcome are we trying to improve? What did someone submit through that intake form? That it’s a very manual task where we think that we could potentially take that on. Experience what friction exists for employees or customers? Is that something that’s more internally? That would be more of an employee satisfaction, save time internally. Or is there? Are there certain things on the customer side that we could be driving more value? Uh, driving more financials, etcetera. And then technology, what can realistically be automated? We’re gonna get into this a little bit more, but especially when it comes to technology. You may have to maybe consult with a third party software that you don’t necessarily have the capabilities in House. You have to go to a third party. Your data may be a nightmare where you have this really cool pie in the sky idea. You know, there’s a lot of ROI to it, but that your data. (Panthers Conference Room) 16:39 ‘S a mess. Derek Steckel (Panthers Conference Room) 16:40 And if you build AI AI solution on top of your bad data, it’s gonna be garbage and garbage out, yeah. You know, and just circling back to where we started, this is where a lot of the ROI in metrics can be defined. So let’s think about a logistics optimization use case that we have and we’ll go through the prioritization of what VST framework looks like in a second. But you know, can we define at this point how much our costs are for this process? Can we define how much of A labor force in hours this given process takes? How about the time it takes to get an order to a customer under this current process? If we can define that right now, that’s gonna help us assuming that this gets prioritized or we go through this framework that we can look at it down the road. Look at. The new KP is those new metrics and compare them to the beginning. You really wanna get those lines right now? Because if you don’t do that, then you’re gonna go through the build of this project and you’re gonna end up saying, OK, like was there ROI here? Did we do something good? If you don’t have any metrics to based off of, you’re not really gonna know. It’s gonna be a tougher process to figure out. Speaker 3 (Panthers Conference Room) 17:54 Yeah, and it’s useful for testing. (Panthers Conference Room) 17:54 Yep. Yes, absolutely, yeah. Derek Steckel (Panthers Conference Room) 17:58 So let’s look at this in action a little bit. So I have the BXT matrix here. We’re gonna look at each of these areas in a second. The X axis. So we talk when we talk about executional fit and what that means, some characteristics of that could be we have the pilot feedback. So maybe we did a small pilot here, or the person that submitted this proposal to the intake process. They have some data to start with, so we kinda have a sense of executional fit. Technical Feasibility’s another one too. Do we have something stood up? Like what? What would we have to build? What would have to change? Is it gonna be a more? Is it gonna be a tougher process to be able to figure out that execution of fed to actually have the right technology in place to be able to do this? Are we waiting on a road map from a client where? Hey, in six months, 12 months, this third party that we leverage, they’re gonna have that functionality there. Then we’d be able to do it. That’s something that you wanna take into account here. Data readiness. I harp on this a lot, but if your data is not in the right places if. Two different terms that seem to have the same definition. No one really knows the difference between the two, and you’re not gonna know which one to leverage. Those are all cases where that execution will fit. Probably would not be great. When we think about the Y axis, that’s more of the value potential. So when you talk about whether it’s an internal opportunity and or an external opportunity, a higher score here would mean there’s strong confidence that we can have high ROI. So going back to what we’re thinking through right now, can we talk about that baseline? Do we know how those metrics, those KPI’s exist right now and where we could take them to? Or is it something that, hey, things are kinda going well right now? There’s always room for improvement, but in terms of that increased value, that increased ROI, we feel like we might not be able to define it right now. And then when we rank those, so when we plot both those values for a given process for a given intake form, here we have 4 different sections that they could land the goldmine. Is that operationalized section? So when we have that high value we have that high executional fit. That would mean that we should probably build this now with all the signs are pointing to this being a really good AI use case. Top left is going to be backlog. Backlog, you know high value but low fit you know this. (Panthers Conference Room) 20:32 Could be a. Just not the right time, yeah. Derek Steckel (Panthers Conference Room) 20:33 Yeah, and it’d be the right time. Maybe you’re going back to that example of that third party. There’s something on their road map where in 12 months. Hey, this is going to make it a whole heck of a lot easier to implement this. (Panthers Conference Room) 20:44 OK. Derek Steckel (Panthers Conference Room) 20:44 Let’s put on the backlog. Let’s wait until it happens. Yeah. Or maybe it’s something on your side. Going back to the data, if we can just get this data cleaned up and in a good spot here. This is going to make this use case to move into more of operationalized section. Let’s put the AI piece of this on the backlog and let’s focus first on getting our data clean. Yeah, getting our ducks in a row before we then take this off the backlog and start prioritizing it. Speaker 3 (Panthers Conference Room) 21:10 Yeah. And what I really like about this is it gives. An objective, you know, screen for everybody to look at and get consensus around. Organizations can only take so much change. They can only do so many AI projects, so it really helps to kind of lay it out. Out so people can move together to do some, you know, good things. (Panthers Conference Room) 21:28 Absolutely. Derek Steckel (Panthers Conference Room) 21:30 Iterate bottom right hand corner, low value, high fit. So that would be more refined or deprioritized. So what I like to say here is this is the if there is time section, if we’re, if someone’s maybe at a low for example like some a quick win for example. They can go ahead kind of bang this out in a few weeks or a month or so. They have that time. Then let’s absolutely do that. But if the value potential isn’t there as much, maybe we can wait. Just wait for someone to kind of be more freed up. Maybe things? Kind of turn up where this does turn out to be something that there’s a higher value potential. We’re waiting on something this turns to be more of a business problem. Then that’s gonna move up into the operational last section and then archive bottom left hand corner. Something’s low value, low fit. We’re just gonna park it for now. (Panthers Conference Room) 22:18 Ignore it. Speaker 3 (Panthers Conference Room) 22:18 You, but not now. (Panthers Conference Room) 22:20 Not now. Derek Steckel (Panthers Conference Room) 22:20 Yeah, good. Good idea. Like we kinda evaluated it. We appreciate the thought that went behind this, but. There’s a lot better use cases that we want to move forward with as a business. We’re gonna park it for now. Now let’s see some examples of what this could potentially look like. I kind of hit on these characteristics, but sometimes it can be really good to kinda think through an actual use case and how it would define that here. So first one, we’re gonna go 1 by 4 AP invoice automation. You know, let’s say that we have the right team for this project. Our data’s cleaned up. Our financing was a really good job with the data. They keep it in a nice centralized location. It’s very clean. We have really good data definitions around it. We have metrics that we think we can improve here. So there’s friction in the process. We could collect a whole heck of a lot more money if we had some sort of process that automated a lot of these work flows for us and freed up some more time. Awesome. That’s gonna be more in that operationalized section. We’re gonna rank it. See value 4.5. 4.6, let’s go ahead. This is a really good project that we should be taking on here. Demand forecasting is the second one here, so we’re saying that it’s high value. So if we’re in the retail space here, we are constantly thinking about, OK. What’s gonna be coming in in the month of October versus April? What are we going to be hit hardest with, made with different trends that are out there? Just where people are, the industry is starting to shape. That seems like there be a really high ROI be able to turn around more products quicker, be able to collect more money because of it. Logistics wise, we’d be able to figure out, OK. When should we be moving this freight, this equipment? But let’s say we go through this and we’re like, wait, our data readiness needs to be pretty good here and our our data’s not good right now our data frankly. Is terrible. We need to clean this up. We’re gonna implement this demand forecasting agent and it’s gonna be giving folks all of the wrong information. It’s gonna be routing things in a misleading way. It’s going to cause us more harm because what it’s going to be looking at isn’t going to be right. It isn’t AI and agents are not a silver bullet. (Panthers Conference Room) 24:31 Yeah. Derek Steckel (Panthers Conference Room) 24:35 Whatever you put into AI is what you’re going to get out of AI. Using that information, it can’t clean up your data for you. Garbage in this garbage out. (Panthers Conference Room) 24:45 Yeah, you have. Speaker 3 (Panthers Conference Room) 24:46 To have a process before. (Panthers Conference Room) 24:47 You can automate it. Yes, exactly. Derek Steckel (Panthers Conference Room) 24:50 Meeting recap drafts, #3. So here that could be something that’s easy to build, right? I I could even go into agent builder within Microsoft 365 Copilot to create something like that. But Microsoft already has something that does that fairly well. I think a lot of us may know that if we transcribe a meeting. Or record it. Microsoft will give us that AI summary and something that’s somewhat new is that custom summary section. So if you don’t like having that AI summary with the meeting, those in the follow-ups, you can go ahead to custom summaries and say hey follow this. Note template for example. It’s gonna be able to put that, that meeting in that format. (Panthers Conference Room) 25:29 So then you. Derek Steckel (Panthers Conference Room) 25:30 Could go ahead and send it off now again, maybe there’s some enhancements there where you wanna create some sort of flow to be able to send that if people choose. Send off that meeting. Recap automatically to the folks that are on that invite list. (Panthers Conference Room) 25:44 Sure, sounds good. Derek Steckel (Panthers Conference Room) 25:46 But again, it’s something that someone could just go ahead, click copy, paste into an e-mail and send it off if they want to. It doesn’t really have as much of A value. (Panthers Conference Room) 25:56 For the organization. Speaker 3 (Panthers Conference Room) 25:57 We have lots of MD’s for personal use. Yeah, MD files are you know. Scripts to process transcripts. We do that all the time, but I can’t really see a reason to create a enterprise autonomous agent to do it, yeah. Derek Steckel (Panthers Conference Room) 26:11 Probably a lot better things. (Panthers Conference Room) 26:12 To focus on. Yeah. Derek Steckel (Panthers Conference Room) 26:14 4th use case here, custom in house large language model, you know, let’s say that we get a proposal in there that we heard that companies are doing this in Silicon Valley. But we don’t have any idea where to start. And we don’t know if this is the right thing to work on, how we would even do this. If it would even be of value to the organization, there’s no real friction here. There’s not no real ROI metrics, OK. Let’s park that for now. Maybe we’ll come back to it eventually, but for now, we’re not gonna do anything with this, all right? So JB, I talked a lot about BX team with that incorporated. Was that the VNA part of that right? Vision and analyze. Yeah. Can you talk to us more about launching this and where we go from here? Yeah. Speaker 3 (Panthers Conference Room) 26:58 Absolutely. So that was great. Thank you. So you know, we build this as show me the money and it’s about talking to the CFO. And so we’re building a business case and of course we’re doing the usual analysis, but I think there’s some interesting things to add into that because it’s AI and also underst. The CFO. Point of view not as necessarily a barrier to everything we do, but enlisting them so they can trust us and they can see the value like we do. Let’s go ahead and jump to the next slide. So I left this in here just to keep the deck complete. But we’ve already talked about vision, so we can move right through vision here and also analyze. We’ll go right to launch. And so for launch the business case, we come up with these equations saying, wow, if I could save X then that means this and Y. And when Acfo kind of looks at those kinds of things. They understand that there’s risk that there’s real process and reality happening, and so they tend to discount some of those things. They also tend to look at who you hang out with and who is, you know, agreeing with you and getting consensus. This on can we really achieve this? Is this something that sounds great? But can we really do it? I think Acfo also, you know, focuses on the hard costs like you mentioned before, not the soft costs. There’s still value. They still want to know what those soft costs are, but they’re focusing on the hard costs. (Panthers Conference Room) 28:18 Yeah, make this more tangible for me. Speaker 3 (Panthers Conference Room) 28:19 Make it. Make it more tangible and realistic. Yeah. Now do I trust? And you know, if you have the acumen and you follow this process and you think it through, it’s going to increase your value, it’s going to increase the trust and the next time it’s going to be easier. You know, you’ve established that communication and that trust. So on here. You know, confidence is really the bottom line. You know how good is your estimate and cost and value go into that. But there’s other things that go into that too. So CFO is the name. Or so I was alluding to this. But you know the CFO. (Panthers Conference Room) 28:55 Picture them. Speaker 3 (Panthers Conference Room) 28:55 They they’re evaluating the decision, not necessarily whether it’s a good idea or not. They don’t necessarily know if automating, you know, the the PO and invoice matching is is great value. But they’re looking for the conversations you’ve had with the business owner that that is a great value, right? They’re looking for consensus and then your business case should talk about, OK. What is going to be the impact of actual impact of that? And maybe your business case has a little bit of a range to it because we just don’t know. One thing about AI is it always requires a human in the loop. You know, and sometimes that’s easier than others. But you know that that human in the loop is going to constantly evaluate. Is the AI doing what we said it was going to do? There’s always exceptions, things that just can’t be processed, you know, so being able to separate those things and have the human in the loop be able to process those. So there’s definitely going to be savings, say in doing that PO to invoice matching. But if I’m a CFO, I’m like, ah, are you exaggerating? What’s the business owner think? Who’s giving me this information? How should I discount it? And then there’s some, you know, there’s some thinking around. Well, what are the odds? You know what’s my worst case and what’s my best case and where do I kind of think this is going to fit? And so providing that having a business case that brings not just the numbers but the consensus. And the process you went through is is very important to from a CFO point of view. Derek Steckel (Panthers Conference Room) 30:29 Yeah. I mean, if you’re acfo, you’re not just gonna say, oh, this JB said that this is gonna save me $100,000. (Panthers Conference Room) 30:36 Alright, perfect. Derek Steckel (Panthers Conference Room) 30:37 I’m not gonna Fact Check it at all. Like let me assign the check for you. (Panthers Conference Room) 30:40 Right, love JB. He’s a smart guy. Speaker 3 (Panthers Conference Room) 30:42 He’s really good, but man, I don’t. If my organization can even change. (Panthers Conference Room) 30:45 Yeah, you know, I’m. Speaker 3 (Panthers Conference Room) 30:46 I’m not sure I can absorb. (Panthers Conference Room) 30:47 All of. Hold it. Derek Steckel (Panthers Conference Room) 30:48 And that’s where especially AI governance committee can come into play too, where you’re not just taking the proposal as fact. Let’s take HR example to start. Let’s say someone from our HR department. They submit a proposal and we ideally have representation from HR on this committee. So the person that’s there, they can go ahead and say, OK, yeah, in vouch for this like we did. Double check these metrics. Here’s how XY and Z lays out makes sense where they can challenge and say I’m I’m a little wonky on that to be honest. Like maybe we should double click into that before we actually go ahead and move forward with this. Speaker 3 (Panthers Conference Room) 31:24 You know that. And we’ve talked about Coes and that. (Panthers Conference Room) 31:27 Outer outer limit. Speaker 3 (Panthers Conference Room) 31:28 You know, they’re they’re saying, well, should we even be doing this? Are people doing multiple things? Are you know kind of align that whereas your human and you know the business line that’s sponsoring the package, they’re pro AI? They’re like, we’re going to change the world. So that tension is really important, ’cause it’s it’s productive. It really gets to a good conc. (Panthers Conference Room) 31:46 Definitely. Speaker 3 (Panthers Conference Room) 31:48 All right, so OK, what question should you be answering or asking as you develop your business case? And some of these things are obvious. But you might forget to include them because they’re so obvious. But what’s happening today? What is the day in life? What’s the storyline? Can you communicate the pain to such, you know, a level that? It’s understandable not just by the owner who’s sponsoring this, but by a group of people. And typically with these workflows, there’s always multiple stakeholders involved. It’s typically not just, well, this is financed and so finance needs this. If there’s downstream effects or there’s other effects. So, you know, having that clear description of the problem that you’re trying to solve you mentioned. (Panthers Conference Room) 32:34 E-mail. Speaker 3 (Panthers Conference Room) 32:34 Is really important. Who’s affected and who benefits the most? One of the complaints, or one of the barriers to adoption, is if I free up somebody’s time. So they’re not doing that task anymore. What are they gonna do? (Panthers Conference Room) 32:49 Are we? Speaker 3 (Panthers Conference Room) 32:50 Are we eliminating people or, you know, or we’ve got lots of other things typical organizations have tons of things to be doing? But if I can identify who’s impacted and how and what I expect them to be impacted and I can do that. In an early timeline, in exception when we’re developing this business case. I can talk to the CFO or the business owner about how we’re going to repurpose those resources. People fill gaps naturally. People you know, it could. Be slack. But you know, if I free up somebody’s time, how are they gonna use that time? And how am I gonna capture it so that it’s not just falling into slack or it’s not just getting filling gaps for stuff that’s lesser of importance. And so I think that. A great organizational story. I think it’s people first story. Part of your business case should be these are the resources that are gonna be impacted. Could be licensing. It could be, you know, but it could be people. And how am I gonna repurpose those? What higher level? How can I map that change? Right in the beginning. You know, if we achieve these results, this is what we can do with these organizational resources. Derek Steckel (Panthers Conference Room) 33:54 Yeah. I mean think about it from the BXT framework to start like you’re going through that process and it’s like, OK, like we see these ROI savings that we’re gonna have, but. What else this might be the equivalent of .5 full time employees, for example. Can that person be reallocated somewhere else like we have this idea where we wanna launch a new product? We want to kinda work through a lot more of the financial use cases that we might have at our company, do a lot more there in terms of making folks lives easier with invoicing just with the more customer satisfaction. (Panthers Conference Room) 34:28 But we’ve always. Derek Steckel (Panthers Conference Room) 34:29 Put that on the backlog because we’re so busy right now. What are those things? You can tie that to ROI too. We can plug this person that’s gonna get 20 hours of their life back into something more meaningful where it’s going to improve the customer experience, it’s going to improve how we operate as a company, so. Speaker 3 (Panthers Conference Room) 34:46 Yeah. And you can think of yourself as sort of like a a factory. And we’re gonna throw AI at this one step in the factory, and it’s gonna improve speed and time and accuracy and everything stays the same. Well, with AI mean the big opportunity is to look at it holistically. You don’t need to necessarily do it the same way anymore to really free up value. And then what are you gonna do with that? So this this is called show me the money. So now we’re talking about quantifying the value. And so as you quantify the value, you develop metrics saying we’re going to save X, we’re going to do Y, it’s going to perform this way. And then then you have to kind of communicate well, how do we know that? It’s working. What are the measurements that we’re going to be committing to? Are the outcome of this, you know, ROI and I’ve mentioned this a couple times, but that actually can can create the test and evaluations for the agent. So it’s all linked. You know what’s begun well. Executed well, ends well. And then you know, we talked about risk and perception and confidence having this complete story of not just. Yeah, here’s the cost. Here’s the value. This is our return. But also, here’s how we’re gonna repurpose resources. Here’s how the world is today, and here’s how it’s gonna change. And here are the consensus that we have on the group because we’ve gone through this DXT process. (Panthers Conference Room) 36:08 All of that. Speaker 3 (Panthers Conference Room) 36:09 Creates confidence when you’re talking to the CFO that they can take this information now and make business decisions. You know it’s well, well positioned if if a big investment, of course not everything needs this kind of rigor, but maybe some of the best things need this kind of rigor for sure. Something that’s interesting about AI, and I’ve heard it spoken this way is, you know, AI is probabilistic. Imagine dropping a little peg down a peg wall, and sometimes it hits one way or hits another, and who knows why. And everything we do is to make it deterministic. We take that, you know, it’s what. (Panthers Conference Room) 36:48 LLM’s are trained. Speaker 3 (Panthers Conference Room) 36:49 On trillions of pieces of information now, maybe, probably more now. What’s what’s after a Truman? (Panthers Conference Room) 36:57 I can’t even. Speaker 3 (Panthers Conference Room) 36:57 Tell you you know, but so everything we do is to make it deterministic so that if someone goes in and asks a question of your data, they get the same results. And if someone else goes in, they get the same results. But when you think about a workflow, you know that that if you think about a simple workflow where you’re receiving emails and you’re making changes, there’s probably actually 2 Roys there. So the thing about AI is it it’s changing. The workflow it’s it’s value reengineering. Often your ROI isn’t going to be just one calculation, it’s going to be. Here are the hard savings based on you know, these calculations because we’re saving time because we’re able to control our inventory better. You know, there’s all sorts of hard costs that are there, but then there’s the soft cost. Customer SATIS satisfaction employee satisfaction. And so you might actually have probably 3 or 4 roys within your single model when you’re doing a workflow. And that’s a lot of work and it’s a lot of details, maybe thinking through all those things, but that’s you know, what’s begun well, ends well and so it creates your measurements and all those other things. So I think. You know, for every work workflow you’re gonna have, you know, it’s sort of a, call it a multi barrier problem. You’re gonna have multiple Roys to work on them. (Panthers Conference Room) 38:14 Yeah. Speaker 3 (Panthers Conference Room) 38:18 This is this is I think one of the most important things for me is, hey, you’re doing these things. We want to be successful because we want to do more. We think the opportunities are are endless with with. AI, you need to capture the game. You need to define the impact and what the gain is going to be, and you need to capture it and redirect it and so that is how you really achieve the ROI. I’m I’m of the opinion that cutting resources, sure, those are those are hard savings. But in every organization that I’m dealing with, people are, you know, retiring or it’s hard to find good people. And you know, I feel like identifying early in the process allows you to make the best of the resources you have and actually realize the gains. And guess what? You got all the metrics and evaluations. So you can actually prove it, and all of a sudden now you’re you’re you’re financially acumen. You know you’ve got improved visibility within the organization. You’re you’re doing a great job. And so this is where the business case leads to your leadership. Elevation your skills, but actually doing good work. Doing good projects, improving it and showing the evidence. (Panthers Conference Room) 39:37 Yeah. Awesome. I’ll take this one, JB. Derek Steckel (Panthers Conference Room) 39:40 So this is actually a project that I helped shepherd. And the workflow here you can read about it more on the concurrency website is there’s a process where folks would go ahead. E-mail a few shared inboxes. Those emails would then be. Things like can you update the database for X&Y, then kind of confirm with me back. Can you update the address for this client for example, and? These analysts would go ahead, they would. But make those updates and respond back and you need to clarify things like that, etcetera. Like very manual process. We went ahead and was were able to create an agent for a large portion of this time, not just the triage piece, but that can go ahead, understand the request, update a database, can respond back to the person, say Yep, that looks good. Good. That was confirmed. And we still have that human in the loop that’s making sure that things look good and things aren’t breaking, but they don’t have to go through the manual process anymore of saying, hey, this is something that is a pain in the butt. Right. I. To go ahead, understand what this is. Maybe wait for them to respond. Move on to the next request. Go ahead, just go to the database and respond back to them. And now with this organization is able to do so, they’re able to say, OK, where else in the finance? Space in the finance department. Can you all make even more of an impact? You know, talented employees, you wanna make sure that we’re putting that their time and effort in the best way possible and that’s what this agent was able to do in addition to more of those forward ROI metrics. Like we’re able to turn more of these requests around, we are able to just save on time where the AI that the tokens are generating the background. It doesn’t really cost all that much because we’re able to leverage. A smaller, more cost efficient GPT model. Under the hood we created a dashboard for this organization that has all all those soft and hard ROI metrics that we can potentially put on there that we have data that we’re tracking so they can see. Live in real time. How many requests this has done in a given day? How much of A given category this is turned around? Different cost savings associated with that process as well. Speaker 3 (Panthers Conference Room) 42:04 And it’s a great example of how AI can go further than automation. You know, so like the quality of the e-mail request coming in, it can check that and it could say, oh, this is missing send the request and it’s stateful, it waits and then when it comes back in it can handle it just like a human being. Yeah, would. Maintain all that complexity. Only faster, more efficient. Yeah. Derek Steckel (Panthers Conference Room) 42:24 Yeah, definitely. (Panthers Conference Room) 42:26 No, I think. Derek Steckel (Panthers Conference Room) 42:26 They double down on JB that you kind of hit on there was there’s still an escalation process through this like there are still some high value things that need that human touch. We’re like we wanna make sure that if a certain request. Is this high of a number for example, or if it falls under this specific type of request, we’re escalating that to the human in the loop? We don’t want the agent touching that, but for those more just commonplace tasks, yes, we can route it through the agent and the human. (Panthers Conference Room) 42:55 In the loop, can simply. Derek Steckel (Panthers Conference Room) 42:56 Make sure on a daily basis through dashboard that we created. Yep, that looks good. (Panthers Conference Room) 43:01 Like the process is working. Speaker 3 (Panthers Conference Room) 43:02 So I love that dashboard. Dashboard has all the statistics of what’s been processed, makes it very easy to see the value. Yeah. So you’ve talked about the, you know, there was there was prioritization of this. You talked about the A and we’ve got the dashboard. What about? The the repurposing of resources. That’s a lot of stuff we don’t necessarily see with our customers, but you know how they reallocated or how they’re doing things a little bit differently. Derek Steckel (Panthers Conference Room) 43:27 Yeah. I can’t necessarily speak to where they were put at in the organization, but I do know that that was something where they were. The organization was able to say you can go to X department like we have really good high value work for you there. Speaker 3 (Panthers Conference Room) 43:44 It’s those were some of the, like, highly creative. Yeah, critical. Derek Steckel (Panthers Conference Room) 43:47 Yeah, these guys, I mean, this sincerely were extremely knowledgeable, very technical. They’re able to kind of focus in on a newer area of the business, getting that kind of growth mindset and it really cool to see that they don’t necessarily have to go through this very. (Panthers Conference Room) 44:04 Manual process anymore. Derek Steckel (Panthers Conference Room) 44:05 They don’t have their job, isn’t just moving things over to a database, updating and responding to an e-mail they can actually use their skill set and develop new skills to have an even bigger impact of this organization. Speaker 3 (Panthers Conference Room) 44:18 Yeah. So now you’ve got momentum. And so now they’re doing other agents and things and looking for the next following the. Derek Steckel (Panthers Conference Room) 44:25 Process and they got the skills from this process too, where they’re more comfortable and more familiar with AI. (Panthers Conference Room) 44:25 Say that again. Speaker 3 (Panthers Conference Room) 44:29 Yes, and they have the capability to process emails. Yeah, they have the capability to update databases. They have these tools that have been built. Like you were mentioning that now, OK, how do we stand on that and? (Panthers Conference Room) 44:41 Move forward. Exactly, yeah. Or exactly, yeah. Speaker 3 (Panthers Conference Room) 44:45 So this is underscoring the the utilizer EU in value but utilize the capacity name the receiving word before claiming the benefit. So make sure you capture the results and I think this is for me this is. I guess I’ve mentioned it before, but this is the this is the height of the story is you are going to receive benefits from doing these things. It’s well planned out. Then how are you going to use those resources? OK. So you mentioned the dashboard, but when I think of evidence, the results, that’s what the CFO needs to know. Yeah. Did we actually achieve the savings? Do I trust Derek next time he comes to me with a proposal and his friends because it was consensus based. And so and because of the way some of the things I don’t, I don’t think it’s necessarily unique to us, but maybe you know the way we put use some of the tools around observability, traceability, the logging of the activities and also. Things we can show we can. (Panthers Conference Room) 45:46 Show them the money. Speaker 3 (Panthers Conference Room) 45:47 Yeah, we can show them what’s going on. (Panthers Conference Room) 45:48 Yeah, exactly. Derek Steckel (Panthers Conference Room) 45:49 And it isn’t just something where, OK. You’re turning Adrienne on. OK, great. I see the value right now. You wanna monitor that over time? Like OK, I can see it on a weekly or monthly basis. But what about fiscal year 2026? What did this do for our business? How can we look back on that and potentially even expand on it? (Panthers Conference Room) 46:03 Yeah. Derek Steckel (Panthers Conference Room) 46:06 Maybe there is some additional capabilities we’re like, wow. Wouldn’t it be great if it could do this or that OK. Let’s lean into it through this process. Let’s lean into what metrics could this potentially generate even more. Where is the ROI. There, what would this cost etcetera. Weigh the options and then potentially make. (Panthers Conference Room) 46:23 Those enhancement stuff. Speaker 3 (Panthers Conference Room) 46:24 These are very practical implementations and thought processes for for doing AI in a in a business, there’s so many messages out there about AI is gonna end the world in three years. (Panthers Conference Room) 46:24 Cat. 10% chance. Speaker 3 (Panthers Conference Room) 46:37 1010% chance or simply my my staff might spend $500 million in one month next month. All of those are non confidence messages that a CFO hears and so having this deliberate process that’s traceable with, you know, it’s super important. To realizing the gains for for AI because there’s definitely gains there. I’ve underscored this yet once again. But you know, measurement is can be difficult sometimes on the soft ROI requirements and I you know I don’t know if you’ve experienced that where you know how, how do we do a lot of enablement training. What are some of the success stories? I guess there from a from an ROI standpoint, even though we can’t really put. (Panthers Conference Room) 47:26 An ROI on it. Derek Steckel (Panthers Conference Room) 47:27 Yeah, I think the most extreme is, you know, all challenge organizations and say. All right, great. You enabled 100 people with a copilot license, and no one’s using it. So sure, you can go ahead and say that you have this pilot or you have half your organization on copilot licenses. But if they’re not using it for anything, you’re wasting that money. So part of that AI enablement piece is saying, hey, can we help work with these specific departments to bring you in there and use documents, use processes that they are currently doing? And. And show how that could be optimized and copilot how they could take a Excel document that they constantly have to analyze, put in a certain format, and use copilot within there. Or maybe sometimes even create an. Agent within agent builder in Microsoft 365. An agent within agent builder in Microsoft 365 Copilot. To put that in work through that process and be able to save them time there so they can move on to the next thing a lot quicker so they can have more on their plate. Speaker 3 (Panthers Conference Room) 48:27 And so we’re going to the next slide. (Panthers Conference Room) 48:27 Yeah. Speaker 3 (Panthers Conference Room) 48:28 So this is the value part and the plus here on the end again is you know improving your leadership capabilities and and acumen. I’m strongly believe that our customers understand how to how to generate ROI’s. AI is a little bit different in that. It’s so holistic. (Panthers Conference Room) 48:45 It’s not just that. Speaker 3 (Panthers Conference Room) 48:45 It’s not just an IT project, right? It’s more than that. It’s a business project. It’s value engineering and I think you know this value plus model here is really for your thinking as you’re going into that arena. To you know, do the right things first, and to actually capture the value and keep building and and moving on to the next great achievement. Derek Steckel (Panthers Conference Room) 49:09 We made it. We made it up the mountain to the altitude’s a little tough up here. I’m kind of right now man, but I’m glad we made it. Speaker 3 (Panthers Conference Room) 49:18 Thanks for joining our. Webinar today. If there’s any questions from the group, we can answer those. You know, if you would like to talk with concurrency, whether it’s an engagement or not, we’re always willing to talk. And we can talk about ROI. I’ve got tons of, you know, ideas around ROI and you know. We gladly share those with the group. Just reach out to us, yeah. (Panthers Conference Room) 49:45 Absolutely. Derek Steckel (Panthers Conference Room) 49:46 I appreciate your time today that QR code. Is a survey. We value all feedback, constructive, positive, et cetera. So for the folks that are on the call, if you could please take just 30 seconds or a minute to briefly fill up that out, we’d love to learn more about what you like, what you didn’t like and how it can improve. Great. Otherwise, thank you so much for your time today. We’re happy to stay around a little longer if you see any questions pop up in the chat. Otherwise, have a rest, your day. Speaker 3 (Panthers Conference Room) 50:13 Yeah. Thank you, Derek. Thank. (Panthers Conference Room) 50:14 You. Ah. Derek Steckel (Panthers Conference Room) 50:29 How do you recommend a small medium business company to track intake forms? AI ROI for approved use cases and AI inventory. Do you recommend if I take that first? (Panthers Conference Room) 50:40 Step go ahead. Yeah. Awesome. Derek Steckel (Panthers Conference Room) 50:41 So what I would small, medium size business you know. You are going to maybe have departments that are smaller in terms of a workforce, right? Your HR team may only be, let’s call it, three people. In this use case. So in this scenario, so how do you actually? Rank that or quantify that in terms of an AI committee. You know, I would still say that you want to have equal representation where possible, because if an hour use case comes in and you don’t have anyone that knows hour well on this committee, how are you gonna? Be able to quantify it. How are you gonna be able to say that actually doesn’t make a lot of sense there? (Panthers Conference Room) 51:23 Or OK. This sounds good. Derek Steckel (Panthers Conference Room) 51:24 Like, let’s go ahead and implement it, right. You want that representation there. So because of that, I still recommend that you have that AI committee to start. Next thing that you ask, you know those intake forms. (Panthers Conference Room) 51:38 It can be. Derek Steckel (Panthers Conference Room) 51:38 As simple as you set up form with 5 or 10 questions, every organizations going to look a little different as to. (Panthers Conference Room) 51:46 What you care about? Derek Steckel (Panthers Conference Room) 51:47 But if it’s as simple as you don’t want there to be a huge barrier, you just want to generate those ideas, I’d recommend setting up a Microsoft form. I’ve worked worked with some other organizations where they have an actual process where there is a template. That one has an idea that wants to go before the committee that you have to write out in a Word document something that is fairly detailed. You’re not necessarily solutioning, but like, here’s the problem statement. These are the requirements for set problem. Just so you can go ahead and have that be pitched more and they have all the details on that AI committee level. Yeah. You wanna take the rest of it? (Panthers Conference Room) 52:24 Yeah, I was gonna. No. Speaker 3 (Panthers Conference Room) 52:25 Say keep it simple because you don’t wanna create hurdles to people you know, thinking creatively and using AI. I think within the Microsoft space you can set quotas to protect from any kind of experimentation. So you want to create. Sort of a a safe place where they can experiment, because sometimes it does take a little hands on to get you know. A value prompt clear Microsoft has a established an AI center of excellence page. There’s some Nuggets there, even for small companies. So that might be something you wanna wanna take a look at, yeah. Derek Steckel (Panthers Conference Room) 53:01 Definitely. And I’ll just finish on, you know, 2 quick things. One, make sure that you are looking at tasks and not just posing. What do you think AI could help us with here? Tasks. Those manual tasks documenting those. Are really good to do. And the second thing you hit on AI? ROI for approved use cases. The easiest way to do is with data, right? If you have data where you can point to use demand forecasting for example. You can point to what that currently looks like. What is driving that? How you currently route through different processes? How you forecast right now in what your kind of goal is there, what you think is realistic based off of maybe different industry trends where the business has been moving, etc. You can kind of put that as. Or the expectation and then you quantify the same way after the project you have something implemented after a week or a month. Let’s take a sliver of the same time period with the A metric that we already have aggregated, whether that’s their fabric or another resource. And how can we say, hey, look, this is we were going for a 10% jump here we got 15%. (Panthers Conference Room) 54:12 Yeah, Carol has. Derek Steckel (Panthers Conference Room) 54:13 Please advise how to access the recording of a client who should listen to it. ASAP. Thanks for your suggestion. (Panthers Conference Room) 54:18 Well, thank you, Carol. Derek Steckel (Panthers Conference Room) 54:20 These are typically posted on our concurrency website, so probably with definitely within the next week. I would expect it to be probably sooner. You will see the session there and that kind of the past session that we’re done. (Panthers Conference Room) 54:29 Yeah. Insights. Derek Steckel (Panthers Conference Room) 54:34 Yes, under insights on our website and then you’ll see the recording transcript there, yeah. Speaker 3 (Panthers Conference Room) 54:41 Yeah, I think, you know, talking about using AI, you don’t want you. You wanna make sure that the hurdles are the right hurdle. So that people can, you know, ideate and experiment, yeah. (Panthers Conference Room) 54:57 Keep it simple. Derek Steckel (Panthers Conference Room) 55:00 All right. I think we will go ahead and leave it there. Thank you all again. Really appreciate your time. Have a nice rest of your day.