View Recording: The Auditor Is Coming for Your AI

Jim “JB” Brown Hi, welcome everybody to our webinar today with Concurrency. The auditor is coming for your AI, one of the more catchy titles that we’ve had. I’d like to give a shout out to the Concurrency marketing team. They do a lot of work to drive these presentations and they couldn’t be done without them. So thank you. 0:0:29.752 –> 0:0:36.72 Jim “JB” Brown I’m Jim Brown, Solutions Architect, and today I have my partner in crime, Corey. Corey, do you want to introduce yourself? 0:0:36.952 –> 0:0:45.752 Corey MacDonald Yeah, nice to meet you everybody. I’m an account executive on the team here with Concurrency. Look forward to going through the content today. 0:0:48.216 –> 0:1:8.976 Jim “JB” Brown When I think about this title, the auditor is coming for your AI, I sort of get this vision of Elmer Fudd with his cork gun, with the flight of the Valkyries playing in the background. It’s pretty ominous, but this presentation really isn’t about the ominous part of it. It’s about the whole part of it, which is, you know, you can build trust, you can prove control. 0:1:9.456 –> 0:1:13.56 Jim “JB” Brown and you can scale Copilot with confidence. That’s our subtitle for today. 0:1:14.16 –> 0:1:33.456 Jim “JB” Brown I’ve met some pretty interesting auditors in my day, but for the most part, they’re pretty straightforward. They have a deliberate playbook. We typically know what that playbook is going to be in advance, and we prepare for it in advance, and you get good outcomes that way. I think most companies are adequately prepared, if not really well prepared, for all of the 0:1:34.96 –> 0:1:42.656 Jim “JB” Brown compliance and controls and other things that they have to do. And so with AI, there definitely is a process. There’s tools, and we’re going to talk about those today. 0:1:48.296 –> 0:1:49.376 Jim “JB” Brown Corey, do you want to lead us off? 0:1:49.816 –> 0:2:9.216 Corey MacDonald Yeah, before we dive in, I want to spend 30 seconds on who Concurrency is. So we’re a Microsoft Solutions partner and ServiceNow consulting firm, and we help organizations get the most out of their technology investments. A lot of companies that we’re talking to are asking, how do we actually get value from AI and what is value? 0:2:9.696 –> 0:2:31.696 Corey MacDonald And today we’re talking about how do I trust AI so we can effectively scale it. That’s where we spend a lot of our time. And we do take a people-first AI approach, meaning we focus on helping people work smarter, make better decisions, create better customer experiences, get better access to data and analytics, and so forth. 0:2:32.816 –> 0:2:50.576 Jim “JB” Brown Yeah, and we’ve got a lot of experience working with implementing AI, and it does come down to the human in the loop. You know, that’s very, very important that AI could do a lot of things. AI owns the task, but people own the outcomes. And so it’s very important to 0:2:51.456 –> 0:2:55.536 Jim “JB” Brown make sure that we keep AI in line and keep the human in the loop. 0:2:57.376 –> 0:3:15.496 Jim “JB” Brown So act one of our discussion here is just recognition that AI is creating a lot of value. In the not so distant past, we were trying to understand if it was going to create value. We know that it does. And sort of the implication of that is 0:3:15.616 –> 0:3:36.96 Jim “JB” Brown auditing is that control in order to scale this enterprise wide, we need to trust AI. Nobody audits experiments, right? But you know, this statement here is sort of the foundation of our presentation, which is copilot value depends on data trust. You know, data trust depends on evidence and evidence depends on governance. 0:3:36.136 –> 0:3:54.656 Jim “JB” Brown that operates every day. And so there’s lots of things to do. Many of the tools are already implemented or they’re available to you. And then there’s always something new around the corner. Corey, when I think about some workflows that we’ve seen that really are valuable to our customers, 0:3:49.456 –> 0:3:49.936 Corey MacDonald Yeah. 0:3:55.696 –> 0:3:58.736 Jim “JB” Brown You know, they’re all over the board in different departments. 0:3:59.616 –> 0:4:19.216 Corey MacDonald Yeah, and I think the one we’re seeing right now that is creating the most value comes actually from the finance and accounting space. Historically, a very, I don’t want to say archaic, but they can be a little bit slower to some of the developments in technology. And the biggest is really matching invoice to order. 0:4:19.696 –> 0:4:37.816 Corey MacDonald I went to school over in Whitewater and I naturally now know a lot of accountants, go Warhawks. But that is a really, really big area I hear talking with some of my colleagues and peers that I went to school with. And beyond that, a knowledge capture agent, capturing critical business processes and 0:4:38.336 –> 0:4:57.416 Corey MacDonald putting that into an agent and creating standard operating procedures based on, you know, institutional knowledge that somebody who has 20, 25 years of experience brings to the table and they don’t want to lose that when they leave. And then lastly is I would say the third would be not really just one agent. 0:4:57.496 –> 0:5:9.136 Corey MacDonald but multiple working together to help speed up a historically non-efficient process. So handling e-mail boxes, doing data validation, and taking action, all working simultaneously together. 0:5:9.536 –> 0:5:25.936 Jim “JB” Brown Yeah, and some fun ones like predictive maintenance on machinery and, you know, speeding up processes by using multiple agents to validate each other and working together. It’s kind of a very cool developments that are happening. 0:5:27.536 –> 0:5:47.56 Jim “JB” Brown So act two is, well, why does trust matter? The Auditor, when they come in, they’re not trying to see how efficient you are with your AI, but they’re looking at the governance of AI. They’re looking for some pretty basic things. And when I think about an enterprise, you have lots of compliance and controls in your organization. 0:5:47.336 –> 0:6:9.536 Jim “JB” Brown and they’re all mapped, you know, to help you achieve them. These are the ones that we kind of talk about and start with when we’re talking about trust. Know your data. Make sure that every organization has data sprawl, has changes that occur and permissions out of whack and other things. There’s tools now that you can know your data, you can market so that 0:6:10.96 –> 0:6:30.416 Jim “JB” Brown You’re taking the appropriate actions and security for it. There’s controls. There’s so many tools. There’s not one tool that does everything, but there’s lots of tools that work in concert together. You can think of it as layers that, you know, if you understand those tools, you can make a lot of headway on reducing 0:6:30.736 –> 0:6:49.216 Jim “JB” Brown the surface of the problem and getting to a spot where you can be comfortable and trusting your copilot agent. And then don’t forget, you know, every production agent should have an owner who’s accountable, the human in the loop, that can validate its output and what it’s doing. And that’s something that is a 0:6:49.416 –> 0:7:0.416 Jim “JB” Brown you know, recurring process, not just a one and done release process, but it’s something that you monitor. So, you know, our theme here is that auditors don’t audit your AI, they audit your governance of AI. 0:7:3.136 –> 0:7:11.696 Jim “JB” Brown So the hidden risk, so Corey, you know, what is the hidden risk? It’s people, agents accessing data, it can find it anywhere. 0:7:13.216 –> 0:7:32.496 Corey MacDonald Yeah, yeah, it’s important to realize, and I think a lot of peers are realizing that, you know, Copilot doesn’t suddenly create oversharing. It doesn’t invent or permissions. It doesn’t create, you know, stale content or sensitive documents that were never classified. A lot of those conditions typically existed before. 0:7:32.896 –> 0:7:49.296 Corey MacDonald AI arrived. So what it does is it pulls it forward and it is a very, very powerful tool at finding information and data that is deep in your system and digging and finding things that we have done over so many years and it can reveal some of those weak spots. 0:7:50.416 –> 0:8:9.736 Jim “JB” Brown Yeah, so as you’re preparing to release AI into the wild, you should do the due diligence and bring visibility before velocity. So do these data checks. Data readiness is always one of the first phases. And it’s not just something you do once, it’s something you do as part of your, let’s call it data hygiene within your system, because 0:8:10.336 –> 0:8:18.656 Jim “JB” Brown Things change, people move stuff around, and you have to keep repeating the process to make sure what you thought was secure is secure. 0:8:20.896 –> 0:8:39.776 Jim “JB” Brown So what does good look like? So simply in non, I guess, non-Microsoft terms, is you need to know your data, you need to make sure that it’s in the right place, that it’s secured. So for access, need to know and check privileges. Corey, you brought this up earlier today. 0:8:40.96 –> 0:9:0.16 Jim “JB” Brown Is that HR person still in HR or did they get a promotion and they still have access to the keys to the kingdom? Happens. So running checks, important. Know the controls, all those tools, they’re there, they’re available. And you can develop a rhythm and a process that’s also auditable. 0:9:0.816 –> 0:9:5.856 Jim “JB” Brown that helps you keep everything in order. Know the owner. 0:9:7.456 –> 0:9:26.496 Jim “JB” Brown And then know the improvement loop. So the pace of change is incredible. We do engagements and we’ll release an agent successfully. And then six to nine months later, there’s another way to do it that maybe would be better. And the changes just keep rolling. And so 0:9:27.696 –> 0:9:33.56 Jim “JB” Brown Keeping the human in the loop and always looking for continuous improvement is definitely part of the agent game. 0:9:33.816 –> 0:9:55.296 Corey MacDonald Yeah, Jim, and I would even add that a lot of organizations understand that, hey, how do we build momentum? And I think that momentum really stands in confidence, where the end state is confidence that once that trust exists, organizations can scale it effectively and much more aggressively to maybe try and reach some of those ideas that might have seemed really… 0:9:55.376 –> 0:9:57.856 Corey MacDonald out of reach initially when they first started their journey. 0:10:2.176 –> 0:10:9.856 Jim “JB” Brown Okay, going to double, triple down on human in the loop. AI owns tasks, humans own outcomes. So… 0:10:12.176 –> 0:10:34.416 Jim “JB” Brown AI kind of goes through this phase where you do experimentation, then you do POCing and piloting. When you go production, it’s really important to keep that human in the loop who is accountable for what the AI is doing. And there’s tools now, registries, there’s observability and other tracking mechanisms and 0:10:34.576 –> 0:10:44.576 Jim “JB” Brown tools to just keep track of those things. And then it’s rinse, repeat, do it again. I like my little human in the loop here. And the question is, well, how do you be a good human in the loop? 0:10:46.96 –> 0:10:57.136 Jim “JB” Brown Now, AI is fast, it can scale. Human expertise is what can make that a productive workload for the, in a trusted workload for the organization. 0:10:59.56 –> 0:11:5.976 Jim “JB” Brown So, Corey, do you want to go through some of the viewpoints on Agentic AI? 0:11:6.576 –> 0:11:26.976 Corey MacDonald Yeah, I think one of the biggest mistakes organizations make is assuming AI governance belongs to one single team. Trusted AI is not a technology project. It’s really a shared operating model amongst several different departments and teams. So, you know, CIO typically will set priorities, allocate funding, determine 0:11:27.376 –> 0:11:44.896 Corey MacDonald Okay, what is our risk appetite? What is our tolerance? And then ultimately decide how aggressively do we want to pursue AI? And then we go over to security. Their role is not to stop AI, but to help the business use it safely, own the controls, monitor it effectively, and respond accordingly and very quickly. 0:11:46.456 –> 0:12:6.496 Corey MacDonald Then we go over to that compliance team where they focus on evidence, accountability, assurance. They can really help answer the question, can we prove we are managing AI responsibly? Data owners, we need to know which data matters most. So this can really live with all parties, but 0:12:6.816 –> 0:12:27.456 Corey MacDonald They make decisions about classification, access, exceptions, and that almost goes back to that conversation before where, hey, there was somebody previously that was in HR that they now transferred to a role in operations. We need to know now that their classification has changed. They now maybe should not have access and having some type of recurring 0:12:27.776 –> 0:12:46.976 Corey MacDonald event where we review every few months or now it’s part of a checklist for an employee transfer or promotion. Going to our IT admins, they implement and operate the technology, turning those governance decisions that were previously made as a lead up into repeatable process and controls, and then lastly, business leaders. 0:12:47.936 –> 0:13:5.936 Corey MacDonald You know, how do we identify high value workflows amongst every department? And how do we drive adoption, creating that momentum and now creating the confidence and how do we measure those outcomes? So no matter how advanced AI becomes, people really still own the outcomes and AI may perform the tasks, but 0:13:6.296 –> 0:13:14.16 Corey MacDonald Accountability stays with humans; it stays amongst the different departments, their teams, their their inputs to lead the lead the use case. 0:13:17.856 –> 0:13:18.256 Jim “JB” Brown So. 0:13:20.136 –> 0:13:33.296 Jim “JB” Brown This slide is a very busy slide, and it’s meant to be busy. You know, at this point, we’ve established that AI creates value, that trust matters. So the question becomes, why does AI governance feel so complicated? 0:13:34.456 –> 0:13:36.336 Jim “JB” Brown In some respects it is complicated. 0:13:37.216 –> 0:13:47.216 Jim “JB” Brown But the challenge is that we’re not governed by a single tool. We’re governing an entire ecosystem of data, identities, permissions, agents. 0:13:48.736 –> 0:14:9.456 Jim “JB” Brown And AI kind of sits at the middle of everything. Every AI response depends on the data it can access, the permissions it inherits, the controls that we put around it. And for leaders, this is a big challenge. For security teams, it’s a risk challenge. For IT, it’s an operational challenge. For compliance, it’s an evidence challenge. 0:14:10.176 –> 0:14:28.376 Jim “JB” Brown So the good news is that most of these components in this control governance tool set, they’re not really new. We’ve been managing identities, permissions, data, compliance for a long time. AI is bringing them together in new ways and making them more important maybe than they were yesterday. 0:14:28.496 –> 0:14:34.976 Jim “JB” Brown some of the deeper data classification, information management type capabilities. 0:14:36.336 –> 0:14:52.16 Jim “JB” Brown So, you know, governance ends up being this connective tissue, if you will, but it actually becomes an enabler to scale AI. If you can trust it and you can prove it and everyone is seeing the same benefits, that will help you scale AI. 0:14:53.136 –> 0:15:12.656 Jim “JB” Brown So, used to be the challenge was choosing the right AI model. We know which one is going to be the best one today, but now the govern, you know, the challenge really is governing everything. To make sense of that complexity, it helps to take a step back and look at a neutral framework from an industry body, if you will. So, I always like NIST. 0:15:13.336 –> 0:15:32.456 Jim “JB” Brown Everybody likes to look at NIST. They provide a lot of guidance. They have an artificial intelligence risk management framework. It overlaps with what Microsoft’s saying, Forrester’s saying, you name it. The common denominators are know your data, know your risks, know your controls. 0:15:33.216 –> 0:15:47.216 Jim “JB” Brown prove your accountability and improve continuously. So they have a four pillar framework where you govern, you map, you measure and manage. And that’s split up into many different, many different line items. 0:15:49.856 –> 0:16:8.656 Jim “JB” Brown So understanding that we’re doing a process, I put together a roadmap for you to just help you visualize what the journey is to cleaning your data, putting in the controls, flexing your governance muscles, 0:16:8.976 –> 0:16:28.176 Jim “JB” Brown so that you free up concern about trusting AI so that you can go and be successful. And it starts with a discovery and finding the data, looking, using several different tools. Purview would be one, but SharePoint Advanced Management would be another. So there’s tools for doing discovery. 0:16:28.816 –> 0:16:46.976 Jim “JB” Brown And then once you do discover it, you put it in its right place or you classify it. And Copilot’s great because it will respect permissions and it will respect your information classification. So it’s very helpful in controlling how AI will use your data within your organization. 0:16:49.216 –> 0:17:5.536 Jim “JB” Brown then you protect it. So you DLP tools will go look for data loss prevention, think social security numbers or maybe credit cards. I feel like I need to knock on wood for that one. And 0:17:6.816 –> 0:17:26.336 Jim “JB” Brown Then you can put in policies in place and automate some of these checks so that on a regular basis, the machine is helping you manage, discover, and do different components of these activities. Then you’re getting into enablement mode, where now we can safely deploy agents, we can keep the human in the loop, 0:17:26.656 –> 0:17:47.216 Jim “JB” Brown We can go through a release process. Sometimes that can be very sophisticated with regression testing built in, evals, all sorts of things that then you can do releases and execute production business processes. Again, measuring what’s happening, making sure that it’s doing what 0:17:47.376 –> 0:18:6.736 Jim “JB” Brown we thought it was going to do, and sometimes making corrections based on your own control or findings of it. And then finally, scale it, scale it enterprise-wide. You know, once you implement AI and you clean the data, there’s lots of things you can do. AI becomes part of your infrastructure and you can apply it 0:18:6.816 –> 0:18:27.856 Jim “JB” Brown to many different workflows. And you start to get some capability around doing AI and change and a little bit of organizational change, depending on what workflows, maybe a lot of organizational change, depending on what workflows. Human in the loop remains consistent. People and outcomes AI accelerates work. 0:18:31.296 –> 0:18:49.136 Jim “JB” Brown For our readiness maturity, one thing I’ve learned is that organizations don’t need another maturity score or Microsoft providing a score somewhere for you to look at, but you do need a practical way to figure out what to do next. Think of this model as a roadmap, not a report card. The 0:18:49.616 –> 0:19:12.256 Jim “JB” Brown goal isn’t to reach level 4 overnight, and it’s not linear either necessarily, but the goal is to keep improving and moving forward. And I would suggest you make your own maturity card or plan or journey map, whatever you want to call it, because every organization’s in a little bit different place, has a little bit different priorities, has a little bit different 0:19:12.296 –> 0:19:22.656 Jim “JB” Brown tool set, but you generally know, you know, what’s going to be important for you under the theme of getting control and implementing governance so that AI can be effective. 0:19:24.496 –> 0:19:43.136 Jim “JB” Brown So this is just a simple method to classify your maturity, but there’s awareness, there’s being reactive to solving problems, there’s being managed, and there’s being optimized. And so as you improve, as you learn, as you develop a center of excellence around AI, you can get quite good at it. 0:19:43.536 –> 0:19:45.776 Jim “JB” Brown and reap the benefits thereof. 0:19:46.336 –> 0:20:4.816 Corey MacDonald Yeah, Jim, and even to add, I would say talking with leaders, I would say most organizations are somewhere really between the levels of one and three today, but that’s normal. I think what’s important here is understanding what is our current state and we need to identify what’s the next logical step forward. 0:20:5.456 –> 0:20:15.616 Corey MacDonald instead of jumping from 2 to 4 or 1 to 3, it’s understanding where are we at now, okay, and how do we move forward from here safely, effectively with the proper tools in place. 0:20:16.176 –> 0:20:33.376 Jim “JB” Brown Yeah, and you went through all those stakeholders. That’s one thing that governance and auditors and compliance do. They drive a common language that everybody can rally around and understand what you’re trying to do, what the problem is you’re trying to solve. And so developing your own common language around it is super important. 0:20:35.776 –> 0:20:55.736 Jim “JB” Brown So Act 5, this is a little bit more Microsoft-y. You know, Microsoft enables trusted AI. You know, up to this point, we’ve established the roadmap. The next question is how do you actually operationalize it? What I want you to take away from this is trust requires an ecosystem. 0:20:56.256 –> 0:21:11.856 Jim “JB” Brown not just a product. So this isn’t really product-based, but this is a suite of products and processes that help you govern AI. There’s no button for it. It requires multiple disciplines that become part of what you do, part of how you operate. 0:21:13.16 –> 0:21:32.896 Jim “JB” Brown From data understanding, it’s all about visibility. We need to know what data exists, where it lives, and where risk may exist. I know I’ve said that three times. Maybe that is the biggest take away from all of this. When we start engagements with customers, we always start with the data. AI is very much so driven by data. 0:21:33.296 –> 0:21:51.936 Jim “JB” Brown And if you have garbage in, it’s garbage out. So very critical to get a handle on that. Data protection, once we understand the data, we can apply the appropriate protections. Labeling, classification, all built into the Microsoft space that Copilot will respect and help you keep data 0:21:52.96 –> 0:21:53.216 Jim “JB” Brown where it’s supposed to be. 0:21:54.576 –> 0:22:13.56 Jim “JB” Brown Identity AI can only be trusted if we know who has access to what. So Copilot tends to use the ID of whoever is using it or whatever ID it has. And so those identity, those permissions, those groups, all those privileges are fundamental to what data AI 0:22:13.216 –> 0:22:32.536 Jim “JB” Brown the AI agent would have access to. For governance, governance provides accountability. We’ve talked about that. Compliance provides evidence. And so an Auditor will come in and be looking for evidence. All of these tools can be logged, provide evidence. 0:22:32.816 –> 0:22:42.576 Jim “JB” Brown And also, if something were to occur, they can be used to help you solve problems. So the evidence is built into the tool usage. 0:22:44.336 –> 0:23:4.656 Jim “JB” Brown Security monitoring, the environment doesn’t stop changing, changes all the time. It might not be missing, somebody missing from a group. It just might be, you know, a process or a migration occurred or, you know, change happens. And so doing that process regularly and monitoring what’s going on is super important. It also helps you with monitoring the agent. 0:23:5.56 –> 0:23:24.96 Jim “JB” Brown And so there’s been a lot of developments in the last year around controlling agents. And so that’s a great improvement. So that you can see what it’s doing, you can see why it did it, a whole bunch of things there with monitoring agents directly. 0:23:25.856 –> 0:23:44.976 Jim “JB” Brown So all of this enables the thing we actually want, which is copilot agents, workflow automation, better business outcomes. Note that AI kind of sits in the middle of a lot of things. So Microsoft doesn’t provide a single AI governance tool. There’s several tools, lots of tools. 0:23:45.616 –> 0:23:56.896 Jim “JB” Brown But Microsoft does connect them into a, you can think of it as a control plane that enables AI. So let’s make this more concrete. So I’ll go to the next slide here. 0:23:58.736 –> 0:24:6.416 Jim “JB” Brown Corey, do you want to maybe provide an overview of this scenario? Maybe we can add some other ones in from customers. 0:24:7.136 –> 0:24:18.176 Corey MacDonald Yeah, sure. So I think this goes back to that even scenario we had talked about before where there’s an overshared HR folder and it’s discovered before. There’s really broad. 0:24:20.176 –> 0:24:29.616 Corey MacDonald Copilot adoption. So first we identify the issue and that comes in that discovery piece where okay, we find the sensitive content. Oh, go ahead. 0:24:27.336 –> 0:24:39.656 Jim “JB” Brown Corey, yeah, Corey, so that definition of overshared, that’s almost a little technical description in here, but you know, it means that people have access to things they shouldn’t. 0:24:31.336 –> 0:24:31.936 Corey MacDonald Mhm. 0:24:41.296 –> 0:24:46.256 Jim “JB” Brown And usually it’s because of some group they’re in or because… 0:24:41.456 –> 0:24:41.856 Corey MacDonald Yeah. 0:24:47.336 –> 0:24:49.216 Jim “JB” Brown Whatever, but OK, go on. 0:24:49.536 –> 0:25:8.176 Corey MacDonald Yeah, and I think that even falls under the review access piece, where, okay, this is overshared content, but we have to 1st discover what’s the issue, what sensitive content is there, and understand who currently has access to gain visibility for, I think we’re at a counter now, four times we’ve talked about that visibility and piece, and then classify it. 0:25:6.16 –> 0:25:6.336 Jim “JB” Brown Yeah. 0:25:8.856 –> 0:25:30.496 Corey MacDonald What kind of data are we dealing with? Is it confidential? Is it regulated? Is it business critical information? And not all data truthfully requires that same response. And then going back to your piece of reviewing the access, this is where we should start engaging with the data owners. We validate who should have access to this, who should not, who should be accountable, who should own this. And that piece becomes really critical. 0:25:30.976 –> 0:25:49.456 Corey MacDonald And then that allows us to protect it effectively using labels, DLP policies, access controls, retention policies, and there’s a variety of different tools we can use to help reduce that exposure. And now we have to be able to prove it. We document what happened, what steps did we take to get there, what was discovered. 0:25:49.576 –> 0:26:8.656 Corey MacDonald what decisions were made, what exceptions were approved, and we create almost a little evidence trail to be able to track and also start building, you know, SOPs amongst this adoption. And now we can enable it with confidence. Copilot isn’t blocked. It’s deployed on a stronger foundation now that we went through all these steps to get here. 0:26:9.56 –> 0:26:9.696 Corey MacDonald Ann. 0:26:10.656 –> 0:26:31.696 Corey MacDonald We never started with, okay, let’s turn off copilot. You have to understand what is the environment and how do we improve it from there? Because maybe you have a really strong environment to begin with, but where can we improve on it? And copilot readiness really isn’t necessarily a switch. It’s A remediation workflow. The goal isn’t to eliminate. 0:26:32.856 –> 0:26:41.936 Corey MacDonald All risk right off the bat is to understand what is the risk? Where did it come from? How can we reduce it in confidently? How can we move forward with rolling this out further? 0:26:43.16 –> 0:26:44.16 Jim “JB” Brown Make it manageable. 0:26:44.656 –> 0:26:45.616 Corey MacDonald Make it manageable. 0:26:47.776 –> 0:27:6.456 Jim “JB” Brown Okay, so a little bit more deep Microsoft tool sets. You know, specifically, some of these are new, but most of them have been around quite a while. Microsoft Entra, controlling that access and that ID for folks, and also agents as well. Agents are like, 0:27:6.656 –> 0:27:15.136 Jim “JB” Brown The employee, what was the phrase? Employees that you hire now to do jobs, you have digital employees as well as regular employees. 0:27:17.296 –> 0:27:34.176 Jim “JB” Brown Microsoft Purview and Data Security Posture Management. That’s a set of tools for compliance, for discovery of data, and more so. And so it’s a workhorse in terms of providing that visibility to your data. 0:27:35.696 –> 0:27:55.696 Jim “JB” Brown Information protection, labeling, classifying data so that you can treat it programmatically. Data loss prevention, so you’re always taking a look to see if something matches a pattern or data, you know, like pie data, personal information data. 0:27:56.176 –> 0:28:7.696 Jim “JB” Brown that you just definitely want to keep track of and be able to prove that you have controls in place for the inadvertent miss that you can handle any kind of circumstance like that. 0:28:8.976 –> 0:28:28.96 Jim “JB” Brown And there’s deeper functionality to see what happened, to see, maybe do an investigation because something occurred or someone sent something that they shouldn’t have. There’s a whole layer of security and the AI is being embedded in these tools for security. 0:28:28.416 –> 0:28:38.256 Jim “JB” Brown to help protect the agent itself, to help protect, look for interesting things going on that shouldn’t be going on. And 0:28:39.216 –> 0:28:46.376 Jim “JB” Brown You know, finally, there’s Agent 365, which is a little bit different, but you can think of that as, you know, Entra ID manages IDs. 0:28:47.56 –> 0:29:2.496 Jim “JB” Brown Agent 365 is what manages agents. And so that’s relatively due, but provides you a lot of things that you need to be able to control the agent and operationalize. And then there’s a whole slew of other tools. So a lot of these tools, 0:29:3.456 –> 0:29:23.136 Jim “JB” Brown how you use them depends on how you’re doing things within the Microsoft ecosystem. So, but they all work in concert and respect each other, if you will. And then, you know, I put this note down here, but it probably makes sense to check to see what tools are available, maybe at least quarterly. 0:29:23.176 –> 0:29:34.896 Jim “JB” Brown Because there are changes happening all the time, not just when Microsoft has build or their regular shows. There’s just constant changes and improvements. 0:29:39.696 –> 0:29:50.96 Jim “JB” Brown So, Corey, you know, I’m definitely banging the drum on process and categories. You know, so let’s say you want to get going and you want to start doing… 0:29:51.936 –> 0:30:10.176 Jim “JB” Brown agents and you’re a little bit concerned about your data estate and you want to do some checking. I mean, some simple things that you can start right away with is number one is look, look at the data, get that visibility, look for oversharing. SharePoint Advanced Management is a tool. SharePoint is classically 0:30:10.336 –> 0:30:24.816 Jim “JB” Brown unstructured data, lots of stuff going on there. So there are tools to search, discover, to look, and let’s just see the extent of what you find. Are there issues or are you very clean? But that visibility is like step number one. 0:30:26.16 –> 0:30:44.96 Jim “JB” Brown classifying the data that you do find, making sure that things are classified properly. If you haven’t done classification before, you know, understanding what the tool is and starting to use it so that you can protect data. And some of these tools will recognize data and do some auto classification. 0:30:46.96 –> 0:30:59.616 Jim “JB” Brown Then protect the data, make sure you’ve got your security settings for sure set properly and information protection policies assigned, but also that data loss prevention layer. 0:31:2.576 –> 0:31:21.456 Jim “JB” Brown Then once you kind of got your data and you understand it, you can start looking at understanding what your risk is. And Purview has some tools for that to help you look and see. And that can drive your agenda in terms of the next steps that you need to do. 0:31:23.56 –> 0:31:37.296 Jim “JB” Brown Of course, secure access. These aren’t always linear, obviously, but you know, doing these things and then doing them on a regular basis puts you in a spot where you can prove compliance, where you’re ready for Elmer Flood to come in and have the conversation. 0:31:41.616 –> 0:31:48.576 Jim “JB” Brown So I’ve enumerated these steps and they’ll be in the online presentation, but discover and understand. 0:31:49.616 –> 0:32:6.336 Jim “JB” Brown Get visibility before velocity. That will really help you understand your risk or maybe your advantage, because if this has been something that you paid attention to, you’re probably going to be in a good spot to move forward. 0:32:7.416 –> 0:32:8.576 Jim “JB” Brown Classify and label. 0:32:9.736 –> 0:32:31.376 Jim “JB” Brown Classification is where governance becomes actionable. So if that data is labeled and is classified, you can now programmatically search, look at it, avoid it. You can prevent copilot from accessing it or using it based on that. Plus all the other benefits that come from having that classification in an organization. 0:32:34.656 –> 0:32:35.216 Jim “JB” Brown So… 0:32:36.896 –> 0:32:57.576 Jim “JB” Brown Data is an interesting thing within your environment. You’ve got structured data, you’ve got unstructured data, you’ve got different departmental and so forth. And so you can ground your agent in different ways on sets of data. It doesn’t have to be everything that you have. The first step is just making sure that you’ve got it secured. 0:32:57.696 –> 0:33:17.176 Jim “JB” Brown so that you’re reducing the risk of accidental data use by an agent. And Corey, you mentioned this before. I mean, the point isn’t to stop people from doing stuff. It’s to create a safe platform where they can be very effective and innovative without, you know, crossing the crossing the line. 0:33:17.856 –> 0:33:18.656 Jim “JB” Brown inadvertently. 0:33:24.336 –> 0:33:44.96 Jim “JB” Brown So step 4, govern and approve. It’s important to organize and develop that common language that we talked about before. So an AI charter, talking about risk, developing approval workflows for new agents to avoid, well, costs of agents, right? 0:33:44.416 –> 0:34:3.536 Jim “JB” Brown but also to avoid a duplication, to be more strategic about implementing agents within your organization. So there’s tons of tools, tons. Well, there’s a lot of tools now available, Agent 365 and AF Foundry, if we go outside of copilot, where you can 0:34:5.696 –> 0:34:10.736 Jim “JB” Brown You know, get visibility into your AI agents. 0:34:12.416 –> 0:34:26.96 Jim “JB” Brown We talked about compliance evidence, so that’s just beating the drum. And this is an ongoing process that you’re implementing, just like all of your other compliance processes would be, you know, depending on your schedule, how you would do it. 0:34:28.336 –> 0:34:49.376 Corey MacDonald Yeah, and I’ll even add back on the journey step 4 is we’re hearing a large shift in conversation. I know we talked about it maybe last week or the week before about tokenomics. And I think it’s really important to highlight in that ongoing review of understanding how is the cost structure changing. And that’s really, I would say, a question we’re getting 0:34:37.296 –> 0:34:37.696 Jim “JB” Brown Yeah. 0:34:49.856 –> 0:35:14.256 Corey MacDonald more than any other, because people are worried once we scale, now do they pull the rug and change cost on us? How do we manage that effectively? How do we do that appropriately? And really staying up to the times and having a constant lookout for any updates as they come, as more things become, whether it’s generally available, the credit cost goes up, or even with some of these newer models, it’s 0:35:14.296 –> 0:35:20.896 Corey MacDonald It’s one of those things you need to keep an eye on, and it is it is difficult to stay on that bleeding edge, but… 0:35:20.176 –> 0:35:20.576 Jim “JB” Brown Yeah. 0:35:21.776 –> 0:35:33.456 Corey MacDonald Nonetheless, it is one of those really important pieces to keep an eye on. And I would say a topic we’ll probably talk about in a couple of weeks, maybe a month from now, how things change and how that shifts. 0:35:30.816 –> 0:35:31.216 Jim “JB” Brown Yeah. 0:35:34.536 –> 0:35:52.256 Jim “JB” Brown Yeah, and Brian gave a great example of how things spin. You didn’t mean to be running that agent. Nobody knew it was there because you didn’t have visibility on it. And it actually got quite expensive in a hurry. And so being able to detect those things through visibility by implementing the right tool sets and layers of tools. 0:35:53.376 –> 0:35:54.736 Jim “JB” Brown Keep you happy and healthy. 0:35:55.296 –> 0:35:56.496 Corey MacDonald Have the guardrails, yeah. 0:35:58.576 –> 0:36:16.816 Jim “JB” Brown So, you know, journey step 5, we’ve done our due diligence, we’ve cleaned our data, we’ve got controls in place, we’ve got policies, we understand accountability, ownership, have at it. Start enabling Copilot with confidence. Identify workflows. You’ve got user trust, security trust, leadership trust, 0:36:16.896 –> 0:36:27.856 Jim “JB” Brown all that you can build on. So in this case, you know, security is not a stopper of AI. It’s an enabler of AI. It’s what makes it okay to reap the benefits from AI. 0:36:32.496 –> 0:36:40.576 Jim “JB” Brown So as you start to roll out and maybe scale AI, keep measuring, not just… 0:36:41.656 –> 0:37:2.256 Jim “JB” Brown you know, the performance of the AI, because you definitely need to do that. You need a human in the loop to make sure what you thought was going to occur is actually occurring and control that. But, you know, who’s running the agent? What does it have access to? Being able to answer questions about the box you put it in and whether it’s 0:37:2.576 –> 0:37:21.136 Jim “JB” Brown you know, able to go outside or it shifted for some reason. Sometimes agents will actually, you know, change over time. I think it’s called drifting is the word I guess I’ll use here. But so it’s very important to implement operational controls. 0:37:21.256 –> 0:37:22.816 Jim “JB” Brown on AI beyond security. 0:37:27.56 –> 0:37:38.336 Jim “JB” Brown So back to workflows, Corey, we’ve got some examples of some good workflows. And is there one that’s coming to your mind right now that you’re thinking about? 0:37:39.536 –> 0:37:40.816 Corey MacDonald A few. 0:37:43.296 –> 0:38:4.456 Corey MacDonald I would say one of the biggest ones that we’re actually hearing apart from that finance and accounting workflow is it can be in the HR onboarding space. That constantly is a changing requirement. And it’s actually interesting enough, a lot of HRIS organizations are trying to implement AI into that onboarding process to create a workflow that makes it easier for their team. 0:38:5.456 –> 0:38:17.136 Corey MacDonald So it’s understanding what tools do we already have, where is the bottleneck in a workflow, and how can we effectively… 0:38:19.456 –> 0:38:31.936 Corey MacDonald manage that workflow to reduce that bottleneck. Because every organization knows, hey, we have a few workflows to improve, but not everyone gets too excited about certain policies that might be in place to improve that workflow. 0:38:33.376 –> 0:38:54.656 Jim “JB” Brown Yeah, and AI is an adoption of a new technology. We’ve done a lot this year to have conversations with clients where we’ve facilitated leadership discussion about personal AI improvement, which just gives them, you know, lights the fuse about what you could do, what’s possible so that they can understand it and get a consensus on 0:38:55.96 –> 0:39:13.376 Jim “JB” Brown what the organization is going to do with AI. We’ve also done that at the department level, like accounting or finance or HR or whatnot, where we get groups together, let them experience and touch it, which just starts to set off a whole bunch of ideas. And then we will do workflow 0:39:15.56 –> 0:39:32.736 Jim “JB” Brown visioning exercises where we prioritize, we define ROI. Every workflow could have a different ROI. Sometimes it’s saving hours, sometimes it’s customer satisfaction or employee satisfaction. And so developing that ROI, prioritizing based on ROI, but also 0:39:33.56 –> 0:39:42.576 Jim “JB” Brown maybe ease of use or value. We do a lot of exercises like that to help organizations get started with, you know, a vision of how they’re going to use AI. 0:39:43.456 –> 0:40:2.656 Corey MacDonald Yeah, what’s high value or high volume? What’s a high friction item? What’s a really high value process that we can improve? Because every organization, stakeholders will sometimes not agree on what is the return that they’re looking for. You know, if we can save time, what does that mean for the organization? Does that mean we can focus on other higher impact items? 0:40:3.56 –> 0:40:22.256 Corey MacDonald You know, if we can improve our HR onboarding process, does that mean that they can get to work quicker to start contributing to the organization? Does it mean they can work on other projects? Because the last thing they want to do is, hey, we improved the process for them and now they’re, you know, sitting at their desks, twiddling their thumbs. 0:40:22.536 –> 0:40:27.936 Corey MacDonald where, okay, let’s find out where can this take us and how can we do it effectively. 0:40:28.736 –> 0:40:47.296 Jim “JB” Brown Yeah, a lot of organizations are budgeting, you know, for AI next year because they’ve touched it this year. They got, you know, an understanding, they’ve prioritized their workflows, and they’re ready to roll. And it’s going to start out with the same process as usual, which is discovery, design, 0:40:48.96 –> 0:40:55.696 Jim “JB” Brown a build phase, a pilot or a POC and a pilot. Sometimes it’s just pilot and then they can scale that out and reap the benefits. 0:40:59.376 –> 0:41:17.136 Jim “JB” Brown So this is just piling it on, but I like this phrase. Thank you, copilot. copilot did come up with this phrase for me. You know, govern AI like a portfolio, not like a pile of pilots. A lot of people have a bunch of pilots that they’ve run that they got stuck. 0:41:17.976 –> 0:41:36.736 Jim “JB” Brown A lot of times it’s because of their data readiness and the data just wasn’t in the shape that they thought it was. And so, you know, approaching this as a deliberate process, which is totally auditable, can help you break out of that pile that 0:41:36.816 –> 0:41:38.656 Jim “JB” Brown The pile of pilots purgatory. 0:41:42.456 –> 0:42:1.776 Jim “JB” Brown So last kind of here is, well, okay, action items, you know, what are you going to do with this new information and thought process? How about a 90 day plan? So first 30 days, do some digging, get some visibility into what’s going on, do some reporting. 0:42:2.576 –> 0:42:21.936 Jim “JB” Brown And then talk about AI from a business standpoint with departments about what workflows they’re thinking about. You’re looking for those low hanging fruit or those high value items that you can plan to take a look at. Then in days 31 through 60, okay, 0:42:22.416 –> 0:42:38.816 Jim “JB” Brown you’ve found some things, perhaps, in your environment, or there’s some tools you just simply haven’t had the time to turn on yet, turn those on, you know, within the framework of understanding what’s important to get, you know, get you to that point where days 61 through 90, 0:42:40.416 –> 0:42:59.776 Jim “JB” Brown you know, launch some, your maturity review, whatever that is on your journey map. Definitely connect some metrics to agents. Make sure every agent should be owned within the organization and start to find some reusable patterns. 0:42:59.856 –> 0:43:12.816 Jim “JB” Brown that you can copy elsewhere within the organization. And understand that it’s something you manage going forward, just like you’d manage, you know, real world employees. You’ve got to manage your digital employees. 0:43:14.96 –> 0:43:27.856 Jim “JB” Brown And then if you do have to pass an audit, which many of you do, just be prepared. You know, you’ve got the tools, make sure you’ve got the evidence and for whatever compliance or audit you’re preparing for. 0:43:34.896 –> 0:43:42.96 Jim “JB” Brown So I think we’re getting towards the end of our presentation. Corey, were there any questions in the chat so far? 0:43:43.296 –> 0:43:44.256 Corey MacDonald No, we’re good. Go ahead. 0:43:44.576 –> 0:44:4.16 Jim “JB” Brown Sweet. You know, audit thyself. Having been in organizations, there’s a lot of that. Where you prepare for the audit, internal audit helps you. They map the controls. Make sure and have those conversations so that they know it’s coming. Matter of fact, have them participate so that they’re right there with you. 0:44:4.336 –> 0:44:23.216 Jim “JB” Brown to make it easier for when the time comes so that no one’s taken by surprise. And I think you’ll find a lot of controls already map to what we talked about here. And so I wouldn’t be surprised if you already have, you know, a half a leg up on a lot of these activities. 0:44:23.616 –> 0:44:33.616 Jim “JB” Brown But now you can kind of think of this in terms of the journey map and understand that there is some work to do to prepare beyond just creating the agents. 0:44:36.576 –> 0:44:38.576 Jim “JB” Brown So, we’ll leave you with these. 0:44:39.616 –> 0:45:0.856 Jim “JB” Brown Trust is the enabler for AI adoption at scale. Governance is a good thing. It will enable the ability to unleash this powerful new capability, AI. AI does tend to reveal data risks, so do the due diligence 1st and get good at it so that it’s a non-issue. 0:45:1.136 –> 0:45:16.176 Jim “JB” Brown for your adoption. AI owned tasks. Remember the human in the loop. Humans own outcomes. Every workflow in production should be assigned an owner. And then govern your agents like a portfolio, not like a pile of pilots. 0:45:17.296 –> 0:45:19.856 Jim “JB” Brown Corey, you can have the last word here. 0:45:21.56 –> 0:45:42.56 Corey MacDonald Yeah, I would say, you know, taking that first step towards understanding, hey, how can we build this readiness map? Do we want it to be for a six month approach? Do we want it to be for a 12 month, 18 month approach? And I would really try to say that the Auditor is not the villain in this story. The Auditor is that constant reminder that. 0:45:42.656 –> 0:46:1.416 Corey MacDonald As long as AI will matter, governance will matter just along with it, to work to build that control story before somebody starts asking for it. And this really can differ across different industries. You know, what does an audit look like for somebody in the manufacturing space? It looks a lot different. 0:46:1.496 –> 0:46:23.16 Corey MacDonald than somebody in the healthcare space or in the financial space. It’s understanding what rules do we need to adhere by and AI is not an exception. I think it just gives us an opportunity to, you know, trust the data that we’re putting in, trust the workflow. We know that it’s honest work and it takes a bit to do it. 0:46:24.16 –> 0:46:29.296 Corey MacDonald But you can build repeatable processes to help improve both and then scale AI responsibly to be effective. 0:46:31.536 –> 0:46:31.936 Corey MacDonald So. 0:46:32.336 –> 0:46:51.616 Jim “JB” Brown Great. So thanks for your attendance today. Really appreciate it. If you have any questions, please reach out to us. Here’s A LinkedIn. And of course, this will be available on our website. You know, one next step we could recommend is doing a zero trust assessment. We can go in and accelerate your, you know, the first steps of your data readiness. 0:46:53.696 –> 0:46:58.176 Jim “JB” Brown With that, I think we’re at a close, Corey. Any other comments in the chat? 0:46:59.16 –> 0:47:8.976 Corey MacDonald No, just let’s just hope Elmer Fudd doesn’t doesn’t catch up to us too quick, but yeah, audit thyself. So, no, thanks Jim. It was it was nice talk talking. 0:47:7.936 –> 0:47:8.336 Jim “JB” Brown All right. 0:47:9.496 –> 0:47:10.656 Jim “JB” Brown Yep, all right, bye-bye. 0:47:9.856 –> 0:47:11.136 Corey MacDonald Awesome. Thanks.