View Recording: Real-World Applications of Microsoft 365 Copilot Agents

Corey Milliman 0:22 Alright, we’re gonna go ahead and get started today. Today we are going to be talking about how to really enhance productivity and talk about these new agents that we have available in Microsoft Copilot. So before we get started though, I do. I’m going to look off. I want to see a show of hands. How many of you already have copilot deployed in your organization, or using copilot? See I see 57. 10. Numbers keep going up. That’s great. All right, awesome. I just wanted to kind of get a gauge based on the audience that we have today to understand who has copilots and and who doesn’t. So all right. Thank you for taking the time to do that. So let’s just jump right in here. So copilot agents, right? So copilot agents allow us to. Really enhance our work in three different areas, right? So we’re looking at enhancing productivity and efficiency, looking at improved accuracy and personalized assistance. So what this really means is copilot agents can take a mundane and repetitive task and free us up to focus on our strategic or other items that allow us to be more creative or have that human touch. And by automating some of these routine activities, we can focus and we can really focus our time on what is adding value. We’re really looking at these agents as being something like an assistant, an AI autonomous kind of agent that can help us with our daily tasks. So we are going to spend a lot of time. Directly in sharing screen today, looking at how we’re making some of these agents, but there are. Agents available that are available as templates directly in copilot chat right now. So you can get started with some of these. These are up here. We have a writing coach that helps you with feedback on your writing. We have an idea, coach that can help you brainstorm and organize ideas about new projects or new things you’re working on. We have a prompt coach that helps use AI to help you but write better prompts for copilot to help you get better information out of it. Career coach can really help you with career development suggestions, understanding your role, understanding your goals, your objectives and also help you make sure you’re aligned those goals and objectives to what you’re working on for the year. We have a learning coach that is going to help you break down complex complex. Concepts and then we have a visual creator that’s going to help you generate images. So there are some ways that we can also start using these agents that we can create in copilot chat. So today we’re going to be focusing on agents that we create in copilot chat or in SharePoint. So we can do things like an on boarding buddy that’s going to assist us with our onboarding process, right. So it can actually we can create a chat bot or an agent that can help users or new members of our team find access to training information, get answers to questions. We are going to go through this resume reviewer today as one of our live on purpose. But. This is going to help us evaluate resumes against job descriptions, and we’re going to highlight this today. To talk about the responsible use of AI. We can also do some contract and legal review. We’re going to do this Live Today. So what we’re going to do is take our standard terms and conditions and we are going to compare a contract that we receive and get guidance on how to edit that contract. And then we can also look at research assistance and policy searches and the other one we’re going to do today as well is a policy search. And we’re going to use this in the context of an hour chat bond. So we are going to provide a chat bot that looks at our HR policies and procedures and allows employees to get information quickly. So building an agent in copilot, I have to screenshot here and at the end of the presentation there you can all request a copy of this deck, but it is really eight steps, and so we are going to browse over to copilot. We are going to use natural language to define the purpose of this agent. We are going to ground. Oh, Jason, I see that you have a question. Or not. All right. I’ll continue, Jason, if you want, just drop it. And Chad, I can’t hear you right now. We can also add our organization data so that is going to be grounding this agent or this chat up with specific information. So that could be documents from SharePoint that could be an entire SharePoint site. We’re allowed up to 20 different knowledge sources. And then we are going to define our agents behaviour. So how should it assist users? How should it respond? What kind of tone should he use? Should be professional. Should it be friendly? Are there certain topics that we want the agent to maybe avoid talking about, or really hone in on and then we have the ability to customize that and adding different knowledge sources, adding further actions, and then we can actually publish this and share it across the organization if. We want to so we can create agents that we can keep private that are going to help us in our day-to-day tasks. And we can also create agents based on knowledge that were sharing with other people. That is going to help them access our knowledge. So one thing I want to point out though about these, when they are shared is regardless of who you’re sharing them with, it’s going to look at the security around the knowledge source that were using. So if you’re in accounting and you want to share something out, and your knowledge source is something that only two or three of you on the accounting team have access to. And then you share out this copilot chat bot or agent with other people that don’t have access to that source data. This agent is not going to surface that data to those individuals. So our copilot agents are going to respect the existing governance and security controls that we have in place to make sure that we’re securely sharing information across the organization. OK. So we also have agents that can be created directly from a SharePoint site and there are some different ways different use cases that we put out here. So we can really understand how different people that are close to their knowledge can actually start using these agents to impact their daily activities. So I’ve broken these out into customer service. So we have our service Q&A or service knowledge base and this is going to help somebody in customer service get faster answers for customers maybe. It’s a product manual or looking up different information. So we have this document library with all of this information. So this customer service Rep can then use an agent against that information without having to do searches. Same thing with sales. We can actually help with product catalogs or proposal templates, and we’re looking at using this to maybe increase our sales conversion rates. So we are going to have the ability to create agents that can. Maybe do different things to help along with sales cycle help with recommending alternative products to customers or actually help pulling information to build up proposals. Finance can really help with accounting and general policies on Q&A. Getting copies of our expense report information or getting copies of our expense report policies and putting that out there so that you have something that’s more of a self-service. Where you as in accounting don’t have to answer those questions. Then we have marketing, HR, legal and it. But the big ones we’re gonna cover today. We are gonna cover HR and legal another way that we can use this as an. It is having all of these knowledge bases or log files or different things we wanna look at and again using this agent to use AI against a very targeted piece of information to. Help us do our job. So the SharePoint agents are built almost identically. These steps are outlined here, but we’re going to browse over to SharePoint site. We’re going to click create agent again. We’re going to find our knowledge sources, our behaviour, and then we’re going to be able to share that agent over to our teams or to our SharePoint site. So I’m going to stop the slide presentation now. And we are going to go directly into some different use cases here. I’m going to just look over here and see if we have any questions I need to address before we get going. All right. No questions there. So first one of the ways we can use this is with an employee handbook. And what I’ve done here is I’ve gone ahead and actually used copilot to help me create a kind of a fake employee handbook that we can use today. You can see it has company name standard working hours, dress code, just some different information about how we handle P. Requesting time off and different pieces of information. So there are a couple of different ways we can do this. We can create new agents directly in copilot chat or in bizchat. You can see that we have the Microsoft Office. Plus 365 rebranding that’s rolling out. So this is Microsoft Office 365 is being rebranded to Microsoft 365 Copilot. Just so everybody’s aware. So when I open up this chat, I can go over here on the right and I can actually just choose create an agent. Now we will have another session next month. I just wanna bring this up preemptively though, on where we will talk about all of the different governance controls. And considerations we can have put into place. To control who can actually create these agents, but by default when you roll up copilot, everybody in the organization does have the ability to create an agent against the knowledge that they are working with. And so you can see we have some of those that I talked about today as kind of starting points to help us. But I’m actually going to write a prompt here and I tell it what we actually want this to do. So I want to create a chat bot. That is going to help employees get answers about our policies and procedures based. So we’re going to go ahead and copilot’s going to start a conversation with us asking what we want this to do. Great. The agent set up. Now it wants to know a name. It’s going to call it HR assistant. I’m going to use something a little bit different. I’m going to say demo, which are handbook. Please don’t make fun of my typing skills today. All right, so we give it the name and it’s actually in the back end invoking copilot studio to help us with this. So I don’t need to know how to do any coding. I don’t need to know anything special. I am just chatting with this to create this functionality so you can hear. See here that already understands what we’re talking about in HR chatbot, so it’s already come up with some different types of questions that it thinks are going to be important for that. So it updated the name. Now let’s refine the instructions. What should be emphasized or avoided? Please only answer questions that are directly listed in the grounding mission. If we will provide. If you do not know an answer. Please do not try to answer and direct the user to contact HR directly for clarification. Sometimes AI can be make some educated guesses or try to understand the context a little bit more than we want to. And we’re dealing with an HR policy and procedure. We don’t want HR. We don’t want this chap out to do anything outside of that manual. So by giving it that small guardrail, it’s not going to answer a question about something that doesn’t exist in the documentation. So now it’s asking how do we want to communicate it should. Be friendly. And then it’s going to tell us that we’ve gone ahead and wrapped up most of the information we need to start working with this chatbot. So I’m just gonna wait for this response and then we’re going to go to configure and I’m just gonna say no. Now I’m going to jump over to configure and from here you can see I can change the icon. I can give a description that if I choose to share this with people, this is going to be a description of what they will see. You can see it took my natural language and it put together a list of instructions on how this chatbot should actually interact. And now we’re going to browse over to a knowledge source to actually ground this information. So here let me go over and pick our knowledge source. I already have a SharePoint site. And you can see here I went over to the site and I have some different document libraries. I’m gonna drill down and I only want to pull from this employee handbook folder that I’ve already created that has a copy of the Employee Handbook. So now you can see here actions are coming soon. So with these types of chat bots that we’re creating in copilot, we do not have the ability to take more advanced actions. Say interact with an API get automatically. Send out an e-mail automatically takes some kind of action, whether that’s updating data in another system, invoking an action, an office product. So we wanna do some of these more advanced things. That’s where we can transition these agents over into a copilot studio kind of scenario. But for you can also turn on whether or not we wanna add a code interpreter which is going to give us the ability to actually interact with code. If we’re doing something from an IT perspective. Effective and again the image generation capabilities as well. This is an HR policy document though, and you can see based on this it came up with some starter prompts for us about policies, how our request time off. I’m gonna leave these all as default right now, and then I’m gonna go ahead and click create. Now it’s going to go ahead and start creating this agent, and I’m not gonna share this out with anybody initially. But we’ll wait for this to spin for a moment. And while we’re waiting, I’m also going to open up our Q&A here. So I have this on our other screen. So there is a question while we’re waiting for them, and this is a perfect time for this. So when you’re sharing an agent to all the people, you share it with require copilot license. So there are some different promotions going on with Microsoft right now where you do have the ability to create a SharePoint based agent and share that with people who do not have a copilot license for a limited amount of time. I believe that. Promotion is running until I think it’s 90 days right now. It might be longer. Watch your emails for that. Generally everybody that is interacting with this does require copilot license. There are no other metering or there’s no other usage fees like you have a copilot studio. Copilot Studio uses a consumption based model where we’re actually looking at the number of messages, the number of interactions with each of our agents or chatbots, and that goes against the usage where we could have 25 or 50 or 100,000 messages a month. Copilot ag. The only licensing we need is our copilot license, and these are unlimited use. So we’re not tracking, there’s not a meter involved and or anything like that like we have a copilot studio. So you can see here this agent has been created and I’m going to go ahead and right now this only works for myself if I want to I can send this to anyone in the organization specific users via groups or different e-mail addresses or only myself. So right now this is only my I’m going to say this. Actually everybody in my organization. And now it’s saving me sharing settings and then I’m going to go over to the spot here. Alright, copy my link. Just gonna click an update here because I made one change Click to mouse. And then another window. I’m going to get this agent open. Alright, we’re gonna go ahead and go to our agent. So you can see here we have our demo HR handbook chatbot. And. What is our TTL policy? So showing that we received 20 days annually and it’s also giving me a reference over to the document that it used to provide this information. So if I have a SharePoint document library where I’m answering multiple kinds of questions from an HR perspective, maybe I’m asked going to be answering some benefit questions. How do I contact? Maybe Cigna or United Health? Or what’s our group number or different things like that? Who is our benefits manager for this product? All of those pieces of information that could be stored in that SharePoint document library. Now this agent is going to be able to answer all of those questions. So again, this is requiring. This is not the same though, as a copilot studio agent, whereas maybe in certain scenarios I want to publish this agent over to another website or something where maybe my users are logging in or we have different kiosks that are around the organization where maybe. We have people that don’t log in, but we’re controlling the physical access they have and allowing them access to something. So in those kinds of scenarios, we’re publishing the different channels, different websites or e-mail or even voice. That’s a copilot studio scenario, but here we are actually using this directly in teams or this chat. So this is an example of exactly how we can do this and this can be done with any type of documents. So we can do something very similar. For it, so we have a knowledge base in it. We want to publish out maybe information. How do I reset my password and it we have a paragraph in there that says this is the process for resetting your password so we can use the documents that we have and allow ourselves to share these chat bots or agents with people in the organization to kind. Of create a self-service scenario here. So I’m looking back at our Q&A because that got closed when I open this up. Any other questions so far still see you. All right, so now we’re going to move on into another scenario. We’re going to go over to our SharePoint site to do this and here I have. A list of resumes and I’ve gone ahead and I put together a job description. And I have a list of resumes here and I’m actually going to say, create an agent and you can see here this looks a little bit different in SharePoint. I click create an agent. It knows it’s called resumes agent based on where I am and now I’m going to go ahead and edit this. So here’s our description. This agent will compare resumes to job descriptions. HR staff. Look at our sources and here we can see it’s already because I launched it from this folder within this document library. It’s already grounded to this folder called resumes, so I have a list of all these resumes here and what this is going to allow me to do is if I were in HR or and I had a list of all these resumes, I could actually upload a job. Description As part of my prompt and then say I would like to look for. For all of the candidates that meet certain criteria and the reason we’re actually doing this one, this one is not going to work. And the reason I’m bringing this up is so that we can talk about responsible use of AI. So here. Let’s see. I have a data analyst job description. And please compare resumes. To find people that have five years of relevant experience. Yes, that makes it. So it’s actually going to tell us we have a responsible AI issue and there have been some changes. To Office 365 copilot lately you can see this article was updated on the 14th. And if our employees are potentially doing things where they are creating agents that could introduce risk to the organization or creating scenarios where AI could be introducing bias, it is going to. Bring up a problem and bring you over here to talk about responsible AI. So right now, even though this is a agent that is actually listed in Microsoft documentation, this is not something that is currently supported. So this will flag a responsible AI violation and tell you how to handle. You know, using AI in a responsible way. So this is in the deck. That is a scenario that was brought up. Microsoft had actually published that scenario, I believe three days before that article that I just showed you. So I did want to bring up that this one will not work. So we’re going to not. We’re going to go ahead and close this out and that’s just one thing that Microsoft does have some guardrails in place so that as we’re letting our users. Deploy these agents. There are some guardrails in place to help us mitigate risk, so another way we can use this is contract review. So here I have a list of our standard terms and conditions, and in this directory I also have a contract sample and I am actually going to pull the contract sample out. So just a moment. I’m gonna bring this over, and I’m gonna bring this file over to our desktop. And I have the standard this contract that I’m going to wanna review here. Go ahead and delete this. So now we’re going to create a new agent that contains our standardized contract terms and this way anybody interacting with this agent can go ahead, upload a contract and then compare that and decide you know what they need to look at from a contract perspective first. So demo content to the end of this. Go ahead. We already have our contract review as a source. We can add additional document libraries or folders from here. I’m just CL clicking this so you can see the interface that we can do that as we have in the. In copilot itself in chat. So we have our contract and. This is giving a welcome message that users can see. Of course we can summarize this or change that, and I’m going to say summarize deviations from standard terms and conditions. Create a table of deviations from standard terms and conditions. Compared and now we’re going to give the agents instructions here. And a. A lot of these instructions are in the deck, so they’re they are in great details, but I thought it was more valuable to actually walk through these than to just look at a slide that says this is how to do something. So for this agent, we’re going to say the knowledge source. We provided as our standard terms and conditions for contracts. Please advise users of deviations from our standards. And they upload contracts as part of the initial review process and go ahead and save this. And now I’m going to go ahead and attach a file that I want this to use. Oh, I made a mistake. So now it’s going to go ahead and look at deviations from our standard terms and conditions and help us identify what we need to look at for this contract, what we might need to redline. So you can see here I’m going to let this scroll through because it’s going to keep doing this while we’re while it’s actually looking through here. So let this go through. Because it’ll keep scrolling as I try to do this. It’s actually gotten down to governing law indemnification. And this contract that I originally uploaded, I’ll go ahead and pull that up while we’re waiting for this to finish and this will work on a really large contract when we have our this one is one that was also generated by copilot. I did not pull an actual. Contract. So this is not a Microsoft contract, just so everybody’s aware. But you can see here that I do have these different paragraphs that are out there. Through here. So this document is 11 pages and is covering all 11 pages of that going through each section to figure out what deviates from our standards. And here it is summarized that and if we can actually start going through here and see our payment terms are different. Payment is due in 180 days. Our standard terms is 90 and then it actually shows us and summarizes the deviation for us. So it’s also giving us our contract. It’s also giving us a link over to our standard contract terms so we can go view as well and these standard contract terms are very. Very direct and I have a paragraph in here that says these are our standard terms and conditions. Please compare this giving it a little bit more context inside of this document and then listing out the big things that I want this to our compare for us. So you can see here we went through payment terms, our termination clause confidentiality and we have some and then these are our key different key deviations from standards and from here we can copy this over and we could also get this with the same buttons, summarize deviations create. A table and ask some questions of that as well. So you can see that we can add advanced customizations in copilot studio. In Copilot Studio is going to help us take this to another level and maybe automate this. So maybe with copilot studio I could have a document library and every time a new contract came it would go ahead and scan that contract. Use copilot studio in the Microsoft Word API. Automatically read that red line that contract and then save a new version of that contract with those red lines so that somebody has a ability to jumpstart that process. So this is a really good way for individuals, though you can see here I have an agent that was created. I can actually share this agent. This is a file now, so instead of sharing it through a team like I did here in biz chat, now create a file that actually I can actually share with different people as well. And you can see when I click share I have my same sharing screen that used to SharePoint. But what we’re doing here is creating an agent. We’re also looking at a file, so we have security around the file that we’ve created that has the instructions for the agent, and then we have our security around the source knowledge that we have over here. So I’m looking to see. I see some different bubbles here. I want to see if we have any. Questions here let me scroll up a little bit. So there’s one question. Are the answers the agent provides based on what’s in the source document at the time of agent creation? That’s also a very good question. So I pointed our first agent at a document library. So if the knowledge source changes, new documents or uploads that document library, somebody goes into a document and makes a change. Somebody decides to go into this document library and our payment terms have changed for 2025. Those changes will automatically be recognized. In that agent. So it’s not based on just what existed at the point of agent creation. The agent is going to be smart enough if we’re pointing it at directories and not specific files, it will see anything that comes across in that folder. If we pointed at a specific file and that file was sitting out on a team site or SharePoint site, somebody changes. That file, the copilot agent will be reflecting that updated knowledge. So we also wanna have start thinking about how we govern. Our knowledge sources right. So if we have a lot of people that have access to the HR policies and procedures that are being used to share a chat bot, we wanna make sure that the correct HR policies and procedures are there and that we have the appropriate controls and processes around Dec. What is the source of truth for an agent? So again, we have the ability for users to create these agents and the reason these are so powerful is those users are. Closest to that data. So they’re generally the ones that can make the good decisions about whether or not that data is correct, but that is something to consider when sharing certain types of agents. Even though we can do certain types of things, do we really want to share these with everybody in? The organization keep them departmental and we have to be aware of our knowledge source. Excuse me. So I hope that answered your question. Christopher and Lance and now. There is a question from Sean. I noted that you aren’t setting any permissions for use in teams. This still needs to be completed there as well, correct? So I already had a team created. Which is only going to give me the ability to share that with those people in that team. Now the one that you saw me, the other item is you saw me create one in biz chat. That initially was created for myself. I changed the settings to share with anybody in the organization so that one is going to respond in teams. So we do have the ability to set the permissions to explicit Office 365 groups. So those are tied to explicit teams. Tie them to explicit SharePoint sites, but you’re absolutely correct if we don’t want to share with everybody in the organization. We would need to specify the Office 365 groups where we’re gonna share that or the individuals. Amy Cousland 33:51 We also have a bunch of questions in the Q&A section. Corey Milliman 33:54 All right, jumping over there, I saw Chad. So all right, we’re going to start from oldest to newest. Let’s see. Will the recording of this presentation be available? Published. Amy, I’m going to let you answer that question. Amy Cousland 34:15 Yes, it’ll be. It’ll be on our website and on our YouTube channel this afternoon and we’ll be sending it out as a link for everybody who attended. Corey Milliman 34:23 All right. And then we have another question. Are there starter problems coming directly from what you have in your policy document? So you could when I was creating that agent I did not have a knowledge sources associated with that agent when I created it, so that initial round was using AI to understand the context of. The prompt and the language I was adding to understand what might need to be there. When I added my knowledge source, those questions did change a little bit. To reflect what was in that document. So it does. Look at the document and try to make some best guesses about different ways or your knowledge source. Excuse me? Make some best guesses about. What those starter prompts should be to help you with those, but you can definitely customize those. But. You will do a little bit better taking those first and then you know probably rewriting those a little bit. I’ve seen varying levels of success with letting it go automatically. It’s really going to depend on how much data is there and. How articulate your prompt was when you are writing this to talk about how it’s going to help, how it’s going to use the knowledge. So the more information you give in natural language, the better your results you’ll get as well. With those starter prompts. I hope that answered your question. If not, throw up another one. Then we have another one. If users are able to create agents, is there a way for administrators to monitor how many agents are created, managed, monitor, adjust, remove absolutely. So we are going to be going through. That in a separate webinar next month as far as the governance of these agents, so we can see that from a power platform perspective where these deployed, how are they being used? How many are out there and we can have those controls? Some place to start removing them if we need to, and we can also use different copilot security groups to determine who actually has the ability to start creating these documents as well. Very good question. That’s a whole nother hour long seminar. Webinar that we can actually go through on that. All right, I think I’ve caught up on all of our questions. Amy Cousland 36:40 It looks like you have. Corey Milliman 36:41 So. All right, very good. Thanks for the help. So you can see here that. Let me actually zoom in a little bit because I like doing things a lot more live than just relying on PowerPoint here so. The building a SharePoint agent. We went through that. Those steps are here in detail to walk you through that. Again, you do need to copilot license. You can share those agents with people that do not have copilot licenses. The term and the number of people you can share it with, I’ve seen. Different customers receive different offers from Microsoft. On that I had one customer that I believe was allowed to use that through June. But with that scenario for that customer, even though the copilot agent was being deployed, there was a metering that was associated with that. So just be cognizant of what Microsoft is sending over. They’re definitely sending over a lot of. Ways that you can try to leverage this internally without having copilot rolled out to the entire organization. We have the writing coach, idea, coach and prompt coach. These are really good ways again, just to get familiar with how you can get started with these and you can actually modify those and use those. In different ways, one way I’ve used the prompt coach was I modified that because a lot of our customers were going through copilot deployments, right. We’re looking at personas and we’re looking at how do different people in different types of businesses use copilot for different roles. So a marketing person that works for a manufacturing company could be very different for somebody in marketing at a social media company. So what I’ve done here is change the prompting a little bit to talk about categories of information as opposed to personas here. So everybody in the organization creates things. We all have questions to ask. We all need to catch up. Maybe on meetings or documents or meeting notes. We all are learning, I hope. We’re all learning something new, especially with copilot out there, and so this is grounded to a prompt library that I’ve put together for a certain type of customer as a source of knowledge. So it builds on the prompt coach and then we have the ability to refine those prompts and bring things up, maybe for a certain type of customer. They’re not doing project work. The same way that somebody else is, or they use a different set of terminology. Things like that. So that’s another way we can start curating prompts for your organization. In addition to the view prompts that Microsoft keeps expanding here to help you with things like that so you can see here, these all have the concurrency icon on that, because this is something that I’ve been. Working on for a while so you can use those as a base for other items though. You can get those as well through yet an agent. That’s going to open up in another window that I have off the screen here, and when you click get an agent. You can see some of the Microsoft ones. You can see Microsoft 365 certified, so there are a lot of agents out there as well that can help you get started with certain types of things also. So there are some pre built ones you can go ahead and again to get over to build your own agents. You’re going to go to copilot and just create an agent. One other thing I wanna bring up since we’ve created a couple of agents is me as a user. Once I’ve created some of these, how do I get back and edit these and change these and refine these? There isn’t a really good way to see that. The best way to do that is to go create an agent. And now I have a list of all the agents available from here and we can see all the different agents I’ve already created. Excuse me. And then I can go back, click the pencil. Change the knowledge source. Different things like that also. Maybe I want to enable content from the web or to augment my internal knowledge. So just different ways that you can get back to those, edit those. Change any of your starter prompts. Or actually start deleting those as well if you need to make any refinements after you make any of those changes, you can see it’s actually auto saved and those changes have already been published out. So based on everything we’ve covered today, I think I do wanna go back to the PowerPoint and just make sure I haven’t missed anything while I’m doing that. Does anybody else have any other questions or thoughts or even concerns about copilot agents? Amy Cousland 41:39 It looks like you have another question in the Q&A. Corey Milliman 41:43 Switching over. Amy Cousland 41:48 Maybe 2? Corey Milliman 41:50 How do I go back to editing? Oh, I think we accidentally covered that, Matt. So to edit that. Go back over to. Copilot. I have an agent up right now. I’m gonna switch over to copilot, go over to create an agent. And then here on the upper left you can see my agents. My copilot agent is showing me this blank one. Click this down arrow. Do view all agents. And then you have the ability to go back. Click the pencil icon to edit that and you also have the three dots. You can, once you’ve actually published these, there’s a zip file that you can download that gives you some information on how that was created as well. Hope that answered that you mentioned using natural language to ask your queries. Let me expand this. So going what we’ve experienced with another LLM asking the same question but worded differently, the results responses are different. How susceptible is copilot to this? Since copilot is in essence based on the ChatGPT, right? I would say the same level of potential for hallucinations exist. So when you are going through and creating content. We still have the responsibility to ensure that the output is valid to actually. You own the content that is being created by copilot and Copilot Studio. So there is still the possibility that when you are using natural language to create something based on your usage history with copilot and the context of copilot in your organization, because copilot is building this database right on how you use copilot, how you are using it your organ. Is looking at the knowledge in your organization. If I ask the same question of copilot at 5 different companies, they’ve been using copilot 5 different ways in five different industries. I very well could get different answers. So even in your own scenario, as copilot continues to learn more and more about your organization and yourself and how you’re using it, you’re absolutely correct. Asking the same question over the course of 90 days in the same scenario could give you a different answer. So you do. Need to make sure that you are looking at those kind of scenarios where we’re using natural language telling copilot to do something. There is the possibility that asking the same question same way twice could give you different output depending on the knowledge that copilot has for you. See another question. How can you clean up the erroneous info in the database? So tell me, are you talking about erroneous information in the maybe in what copilot is learning and how you’re using copilot? I think that’s what you’re talking about, the semantic index. So the semantic index is that 3 dimensional kind of database, right? That is understanding context. And copilot is going to do a better job of understanding context. The more and more you use it. So when you tell copilot that it gave you the wrong answer. That is going to move through that database and it’s going to associate that and move that through a different spot. So we’ll start understanding. So copilot does it. You don’t just go in and clean up the database. Copilot is going to start understanding the context of your questions, answers, responses, and usage to start making educated. Kind of. I don’t want to say guesses, but. It’s going to go through operations where it’s going to start understanding. That different things at your company maybe aren’t correct and start moving those out of the results. So another thing to keep in mind is that in different organizations, different documents have different value depending on the year they were created. If you are selling a product and you have connected copilot to a SharePoint site and we have some old data out there, I have a. Have a priceless from 2020. And it’s 2025. It’s highly possible the prices or services have changed since then. So from that aspect you also wanna make sure that when you are grounding copilot or providing copilot with information, wanna make sure that we’re providing with the cleanest the best information possible. And we do a lot of that kind of work when we look at how we’re ground. It what data sources we’re using, how we’re cleaning up data and how we’re actually looking at it from a governance perspective. So a couple of different ways of cleaning up data. We want to make sure our knowledge is clean and as we use copilot, it will start understanding. If we say no, this is wrong. That the information that it has could be incorrect. See more questions when you share an agent. Is it integrating it into that is a really good questions as well. When you share an agent, is it integrating it into every copilot prompt or everyone you share it with? So that is, I’m going to read the rest of this. Or could you isolate a particular agent to only a particular context so that it only functions when they ask questions saying a certain page? So you can do that a couple of different ways. One way I can do this is we can call agents by their name from copilot and other applications. So across the stack. So I’ve created these agents and when I hit the ampersand sign now you have AI have a list of agents that are available to myself. So here I’m going to say I’m only talking with prompt coach. So now it’s telling me that I’m chatting directly with that agent from copilot. So that is how we are directing our conversations even outside of biz Chat or SharePoint site, I can call on an agent by name from copilot in other application to direct that conversation. That output and that action. I hope that answered that. I think I accidentally skipped 1. We saw you creating an agent using static data. How can we get an agent to work with transactional data like time to onboarding an employee? So we start looking at copilot agents being triggered by specific actions. That’s where that agent is going to be better suited for something like copilot studio. So if we’re part of a workflow and on day seven we want to do this. Day six, day five, day three pre and post. That is where Copilot Studio’s going to come in, because it’s going to have that extra ability to take actions and look at other triggers where something in copilot created this way is much more focused. The trigger is user interaction, whereas with copilot studio a trigger could be a new file showing up a date passing. In API change we have a lot of different ways we can impact it that way. Let’s see. So I hope that answered that I’m scrolling up more. Another question about. Sorry, I just realized I have zoomed way out and some in because these glasses only use so much. I’ve been struggling to see your questions here for this whole session. So from my experience in low code versus copilot versus Azure AI Foundary, is there any way to look at specific user queries and agent output? The use case would be to ensure safe and responsible AI usage as well as adding different content safety layers. To our agents, that’s a great question. So we can see this in our governance section for AI in purview. We can start looking at. That’s where we can actually look at the types of information that’s being asked and maybe information that’s in that would fall under, say, insider risk or or different policies like that. I cannot see the actual. Full input and output so. Case in point, I’ll use myself as an as an example. I want to look up something I had from a health perspective going on. I use copilot one day. And it is our enterprise copilot, you know, license from from the governance dashboard. I was able to see that I asked a medical question, but I could not see the the actual details around that medical question. It just flagged. I asked a question about health and that could be flagged somehow. So I would say if you are wanting to look at get more into specific queries, once you find copilot agents that have broad enterprise value, that’s a really good candidate to bring into copilot studio and a really productive that and bring that and evangelize it to the entire. Organization. I hope that answered that that might be something we can answer more in depth in another session as well. When using this chat will automatically use available agents to respond. If I ask regular copilot something, will it go into my policy agent and answer questions related to policies? That’s another great question. It will not. Copilot will not automatically connect up to your agents and start asking questions. So if I were to ask a question of what’s our HR policy on this? Or HR policy on that after I created an HR policy bot or agent, it’s not going to automatically query that agent. So these are targeted because if we also think about we have 50 employees that decide they can all create agents and they decide create them and share them with everybody. So we have somebody from it, somebody from HR, somebody from accounting. We’re all taking these kind of questions that we get on the day-to-day. The Interruptor workflow cost, contact shift. That could be messy if it automatically answered from all of those. Sources some doubt you do have to explicitly interact with the agents at this point that you want to, because there’s not really an orchestrator to understand which agent should actually hit which question. So I think I’ve covered everybody’s questions so far. Amy, have I missed any others? Amy Cousland 52:45 I haven’t seen any others. Corey Milliman 52:46 All right. Oh, new post. Amy Cousland 52:48 Yeah. Corey Milliman 52:49 Are you a copilot agent? That’s awesome. No, but. We can create one, right? That would be interesting. Start taking all my meeting notes and things like that. I use them a lot but. That’s that’s my job, so. One other thing I did wanna get back to or mention here, though I did highlight resume reviewer in red and again this I did want to highlight this. This did this scenario did come from Microsoft and this one does not work. You can also see here the workflow for creating these in SharePoint. Different use cases there as well for customer service, sales, finance, marketing. Nhr. This is more related to onboarding to go back to somebody else’s question. One way that I do this in copilot studio is I also integrate with power automate and things like that. So we have those automation tasks that are happening. So I’m kicking off AI agents and saw some AI autonomy through entire workflow or scenario. So I know we have a lot of people that are using copilot. So if you are, make sure you take a look at the survey that we’re going to send out. We do have a lot of information available on copilot agents. We have our copilot momentum program to help with ROI calculations and adoption. We do have an accelerated copilot agent program, and of course copilot studio, and we can also help with the value discovery session if you’re not on copilot yet, we can look at maybe some Microsoft funding opportunities to help understand. The value that copilot brings to your organization. So just make sure that I think Amy shared that survey and make sure you do hit that and if there’s anything new you’d wanna see next month, I was thinking of covering covering a lot more about the governance. How do we look at these from a holistic kind of overall organizational view once we deploy these, how do we keep our hands around these? What is the support process look like? Where does the support the help desk? The IT support stop for copilot agents and how do we do this without creating? Support nightmare for ourselves, so I appreciate everybody’s time today. I will look for more questions by chat and turn it back over to Amy. Amy Cousland 55:15 Yeah. So I put the survey link there. Yes, if there’s any follow up you’re interested in or if you do want to hear about a certain topic, we have a feedback and comments section. You can add any kind of questions and or topics you’d be interested in hearing. The future in that space otherwise. Do we have any more questions? Otherwise, you’ve had a lot of questions during this. Looks like we don’t have any more, but feel free to reach out and I can get you in touch with the right people. Thank you so much for coming in, Corey. Thank you so much for so much information. This was a great hour packed with a lot of great insight on this. Corey Milliman 55:51 All right. Thanks everybody. Have a great day.