View Recording: AI Apps & Agents in Action

Good afternoon and well, as of now, everybody, it’s been a few seconds into noon, so good afternoon. My name is Matt Kravitz. I’m I’m super happy to host you guys for our last session of the day, AI Apps and Agents in Action. A little bit about myself. I’m a senior software engineer here at Concurrency. I’ve been here for, well, almost five years now. On the day-to-day. I’m just an individual contributor and one of the tech leads on our projects. Feel free to connect with me on LinkedIn. I checked that QR, so I expect at least one or three of you guys to to connect with me throughout here today. And then a little bit fun fact about me, I’m actually in Europe. I had some preplanned travel and Monday night I flew out to Poland. So I’m in Poland hanging out and I’m super glad to be with you guys. And with that, I’ll hand it off to Anne from Microsoft, who’s one of our guests here today. Ann Britt 1:10 Hey everyone, Ann Britt, I’m super happy to be here. I’m a Senior Digital Solutions Engineer at Microsoft specializing in AI workforce. So really in low-code solutions, agent to capabilities and co-pilots. And through that, I really Dr. innovation and focus on digital transformation for our customers. I’m really happy to be here to support this conversation and talk to you all about this awesome topic. Mac Krawiec 1:37 Yep, thank you. And I hope you guys can see my little laser pointer. Can you? I don’t know, Ann, if you see my little laser pointer flying around. Awesome. So for today’s agenda, we just went through the intro so we could check that off. Ann Britt 1:49 Yes. Mac Krawiec 1:54 We have a bit of a shameless lug here coming right up. Then we’re going to get into a bit of an overview of Copilot Studio. Some of the audience may or may not be familiar with how it works, what it is. We’re going to talk through some use cases, some of the challenges that. Come around with Copilot Studio and Azure AI Foundry and just in general that entire stack, both organizational that I’ve seen, but also my own. We get to talk a little bit about that. We’re actually gonna spend most of the time here and just building. And we’ll get to what we’re gonna be building next. We’re gonna have another shameless plug and then just talk. So I hope to have us engaged, have us have some conversations. I know it’s now the last one, but I wanna make sure that we make the best of it. So with that shameless plug, we do offer a free complimentary set or. Complimentary. That seems doubled up, but we offer a complimentary session to talk all things AI apps, ideation, anything like that. So feel free to go and chat and if you’re interested in it, our team will be happy to hang out and talk to you about it. With that said, really quickly, what is Copilot Studio? It is Microsoft’s tool for building custom AI copilots. Well, what is copilot? Copilots are effectively assistants. In that are fully employed with with artificial intelligence and they’re able to take conversational instructions to then integrate and work with apps, websites, teams and more. And when I first started using it, the whole conversational aspect of it really kind of. Was hard to grasp and blew my mind a little bit, but the more you work with it, the more, more, more comfortable you are. Really you get to integrate AI within your business and and the the plethora of connectors and and and now MCP servers that are available make that super easy, super seamless. And in general, one of the focuses of this of this talk is really automating workflows, making it such that AI isn’t is within grasp of your organization’s tool belt. That said, though, there’s also the things you have to be careful about with the advent of AI. Everybody has a hammer and everything is a nail. So you have to also be cognizant of where to employ it. So that’s a little bit something that we’re gonna talk about a little bit. Now within Copilot Studio, there’s a studio. And what is that, right? It’s basically a. A UI that’s gonna help us design and manage the environments that we have our Agents in. It’s very akin to Power Platform. As a matter of fact, it’s augmented inside of Power Platform. The same elements of environments that you have in Power Platform, pipelines, managed environments, unmanaged environment, all of that applies. With Copilot Studio and building your Agents in a low code fashion within Copilot Studio is just as seamless as it is to build a canvas app and certain things like that. So a lot of the authoring, the integrations, the managing super easy to come to terms with and super easy to get a grasp on. This is. So I saw this image at Build. I went to Build. I had the great pleasure of going to Build this past year and I saw it and I I thought I would talk about it a little bit. We’re here in this in this left most box and we’re going to stay within this box for the most part. But we’re gonna give honorary mentions to others throughout this presentation. Now, the reason why we’re here and we’re gonna stay here is because I only have 15 minutes, really 30 between the demo and I could talk about this for hours on end and I also could problem solve and go through my things as we. Integrate Copilot Studio. We create an agent in Copilot Studio. We might create an Azure AI Foundry with an agent inside of that project using GPT 40 or something like that. We’re not going to get into all of that. We’re really going to focus on Copilot Studio. Just know that throughout our use cases and the more enterprise designed. Implementations. Really you might have all or elements of these. Oftentimes you really have at least at least one or two, right? I mean, actually you might have all four of these, right? Where if you’re using. Copilot Studio to surface your team’s agent and give some instructions there. Well, then on top of that you’re using likely an Azure function to add some more pro code, more custom, more lower level kind of work, and obviously you’re using Visual Studio for that. You’re using the Foundry SDK. And you’re surfacing that all through an agent in Azure AI Foundry. So this stack I didn’t realize at the time when I was sitting there at Build hugely impactful and hugely relevant today and we are literally living this image if you are building AI Apps and making anything within that and doing it in Microsoft. Tech space, you’re more than likely living this stack and and and did you have anything to add to that? Ann Britt 7:12 No, I would just say, Mac, that, you know, I know you’re going to get into some of this later, but when you look at the platform story, which is what you’re telling here, there’s real cohesion in your builds and in your security, your data structure. So yeah, I would just say great, great. But the nod to this, it’s super relevant and timely, so excited you how you tease this out. Mac Krawiec 7:36 Yeah, for sure. So let’s talk a little bit about some of the real use cases you you might see. The first one being is the most common, just because all business needs to make money and in order to make money, for the most part, all business needs to have sales. And part of the sales process is going back and forth and quoting and negotiating well with the advent of AI, the automation of a sales quote process and and general CPQ. Has been made hugely streamlined and within grasp, where previously we would work with clients that would have to curate their data for years to sometimes years even to build enough data to train custom models and use machine learning to build now with. AI being effectively somewhat of a commodity, it is within grasp to accelerate the sales process of your business using sales quote automation. And just to let you know, our demo is gonna be about that. So really big focus here. Some of the things that we’ve done and I’ve seen is field technician support. So imagine. If your company has a ton of field technicians and they need to reference documentation or schematics or diagrams, and they don’t wanna carry a huge textbook with them when they’re servicing some sort of equipment, for example an AC unit or whatever, we’ve built things that accelerated Agents in the field. Using AI that’s trained on documentation, manufacturing, IT and operations where you’re using something akin to the site reliability engineer, which is an AI augmented into the Azure portal that allows you to monitor what Azure. Performances you all like and what errors you have and even employee fixes right away within seconds. Logistics, right? Logistics is fast-paced and the more AI you can. I have a father and he’s an owner operator so logistics is pretty close to my heart and so I know how I can always think about what you can what the real life use cases are but. Even sales quoting where the dispatcher goes to the broker and you’re starting to negotiate all relevant finance. We’ve actually worked with a few companies where we’ve reconciled accounts payables using AI and accelerated those processes. There’s a ton of different use cases there and retail and e-commerce, right? I’m sure you’re seeing it today. Today, I mean, even just the fact that you’re saying something now when your phone is nearby and you’re talking about a new phone case and all of a sudden your Amazon store has phone case suggestions, that’s one of those examples. So those real life use cases all applicable. Ann, did you have any other thoughts, comments or anything that you wanted to add to this slide? Ann Britt 10:29 Yeah, I love that you hit so many different industries. You know, we have case studies in so many of these. So for example, manufacturing, it’s a great one. We’ve seen a 40% reduction in incident response time through autonomous monitoring on manufacturing lines. It’s pretty crazy what we’re seeing and when sales is another really good one. At Microsoft, we did a study when copilot was released to our sales groups and we see that our sales reps are recouping on average about 90 minutes a week. And at scale you can imagine what that looks like in terms of ROI and just efficiencies that are gleaned. So great examples that you pulled out. Maybe I’ll give one more. You mentioned financial services and this one sticks out to me. 60% faster loan processing time via intelligent document processing. I don’t know about all of you, but for me, dealing with a bank in any way, shape or form is tedious. So if you could reduce the time, I’m excited about that. Mac Krawiec 11:27 I’m living it right now. I’m currently waiting for my house to be built and I just went through the mortgage process and I could tell you that having I I know my parents went through this and when they bought their house and it is much quicker now. So I hope that this company is also using using AI so. Certainly applicable. I did hear of AI being used in manufacturing to your point in fault detection or predicting faults. So for example, if a company is using drills, they’ll use AI coupled with IoT to determine that a drill is going to. Ann Britt 11:55 Mhm. Mac Krawiec 12:02 So bad. And so so a lot of that. And then what that allows you to do is to never have downtime because you’re going to preemptively replace those drills. And a lot of this stuff is is hugely impactful to the to the bottom line. So absolutely. So yes, I need to give the sneak preview. We’re going to focus on sales code automation. So I just wanted to to to let you guys know what we will build. I a lot of my presentations are all F1 themed. I’m a huge fan of Formula One. So what we’re gonna build is a Formula One sales quoting tool, and we’re gonna use Copilot Studio to detect new quotes, find the requisite product, and we’ll talk through that. We’ll assemble a quote in some human readable format, and before that, within the product section, we’re gonna determine what the pricing is, what the. And inventory that is available and we’re gonna send the quote to a client and I have that you can see I have quite a few of asterisks there and part of this and we’ll get to that in a greater fashion here in a second kind of speaks to what Todd and I don’t know how many of you guys were at the keynote kind of speaks to what Todd. Todd was talking about and the confidence rating and some trust. One of the things that we can do in Copilot is send emails directly. We can create drafts, we can reply directly. So we’ll talk to those capabilities. Now the reason why this is asterisked is because I’ve worked with companies where sending a quote directly from AI. At least at the beginning is not something that they are willing to risk. It is business, it is sales, it is money. Especially as as the adoption rate grows, you want to make sure that you you have that human in the loop, right? Todd did mention that human in the loop. So the reason why there’s an asterisk here is that we’re going to be sending that e-mail for quote just for demo purposes. That said, though, I would not recommend doing that straight away. That’s something that you probably get to after you’ve built enough confidence and you’ve got enough test data to prove that you have, you know, 90 some accuracy or whatever. Threshold a company may feel comfortable with, but I did want to throw that out that you do want to be cognizant of those things so as to not erode trust. And with that said, organizational challenges, I have the first one, what I think is one of the most. Prevalent ones and ones that are because I’m on the implementation team, on the delivery team, most often this is the one I run into first is data readiness and quality. A lot of companies want to get into AI and we’ve done this where we’ll have some low cost. Dip your feet in the water solution just to prove out the use cases work for Company X or Company Y. That said, though, the ultimate benefit is somewhat stunted, or at least there is a law of diminishing returns that applies when your data. Is not curated and not clean enough for AI to operate on. We’ll get to that in the demo where you’re going to start to where we’re going to show our data source for this information. Well, if every database and every company used data that was that clean and that simple to understand, AI would have a much easier job. But that’s not the real world. So that is one of the my, my chief challenges, legal compliance and governance. And we’re just talking about this maybe and I’ll let you, I’ll let you talk to speak to that a little bit since you’re you’re familiar with that a lot more probably than I am. Ann Britt 15:38 Yeah, I think, you know, this is a topic that everyone wants to double down on. You you hit the, I think number one part of the concern is data, right. We’re concerned about our data being leaked, whether it’s, you know, things that are under NDA or just very, you know, a PHI or PII, there’s critical data that can be stored. Stored in some repositories. And so with the advent of agentic capabilities, our LLM sitting over these things and just all that can be exposed, it becomes a bigger concern. But Mac, you touched on this. What it really comes down to is being smart about our overall security posture and Satya said it best when he. With. Without security and our zero trust architecture, the rest of it doesn’t really even matter. We could be the most innovative AI company on the planet, and we are. But without having that security posture, without having proper data protection and governance, without properly looking for technology and tools that are gonna help you like. W or DLP, it really can become a little bit of a nightmare situation, but luckily we do have all of the tools to bring to the table to help with that and really there is then that business process behind that of making sure you clean up your data. In two ways, that’s important, right? If you’re moving towards agent in agentic world, you’re right, you want to have that really clean data. But also agents learn differently and the way that data is searched is different and it’s in vectors. And so the data needs to be clean everywhere or you’re really getting half the story. It’s like playing a game of telephone and the message getting. And kind of like watered down as you go along, if you can imagine that. So and the same then is true for the security of that data if we don’t have things properly classified or shared. So I I hope that perspective helps Mac and I’ll turn it back over to you. Mac Krawiec 17:30 Yeah, thanks for that. And with with the legal and compliance aspect of it, I wanted to make sure I give you a chance to talk just because again, we were talking about this. I’ve had clients implementations were slowed and and put to a stop because the legal department just wanted to make sure that they do their due diligence and rightfully so. To make sure that their business is secure and the data is there too, but it is definitely one of the challenges that I want to draw attention to that. That is why. And if I actually go back to one of the slides here that showed the stack, This is why you’re seeing this box around Azure AI Foundry and. Specifically, the security and governance Azure AI Foundry is at top of mind is that security aspect and making sure that you can take all these aspects within the stack and still leverage the things available to you in Azure to ultimately secure it better. So I just wanted to give a mention that trust. Trust is a big one. That’s why you don’t send quotes right away. That’s why you might want to check them. That’s why you kind of set thresholds within your organization to ensure that they’re. The AI is doing what it’s supposed to, and that is exactly why there are certain specific automated tests that you can run against your Azure AI Foundry agents that you can code. And you know if you create an agent in Python and part of it is deploying that to to Azure AI Foundry within ADO and within certain. Specific implementations. You can test a previous deployment against a new deployment to ensure that the answers are what you expect them to be, both from a safety perspective in terms of making sure that the AI isn’t doing something nefarious, but also from just the output in general. Culture is another big organizational challenge where first of all, you don’t see this in in in in consulting companies where you definitely see this in in some in some industries where things just don’t need to change. Or at least that’s the perception of of some of the some of the members, some of the members feel threatened. So that’s a challenge. And a perceived cost and scalability. So AI is is such a huge topic and a lot of conversations around is like oh it’s expensive and a lot of money and tokens cost and and to some of that to some of the degree it’s it’s true. However, with the features that are rolling out within Azure AI Foundry, for example, that chooses the best model for the best price that can solve the problem at hand, which that surfaced in Build or more recently what I’ve seen in Copilot Studio that’s still in preview is the efficiency. Capabilities, which we’ll get to, which again chooses the best model within the copilot studio stack to ensure that the questions answered are cheap. That’s one of the one of the considerations. And then the other one is really just start small. Don’t go big. The projects I’ve worked on that have led to the most success within the AI space always started with a very small proof of concept within a given use case for the business that solved the problem in a very simple way. But really, it started to show that, hey, this is possible and gain confidence. So all of these challenges exist. All of these challenges, all the companies that we’re talking about or seeing or hearing, they’re all going through them. All of them are solvable. Um, so that’s definitely a big one. Personal challenges. We’re about to get into the demo here. Hang on. Personal challenges for me is really at the beginning when I started that I’ve been, I’ve been coding for, you know, a while now. I’ve always been pro code. Front end, back end data, everything. The whole conversation of low code versus pro code is wrong. This versus should not be versus. This is low code with and or and pro code with the advent of Copilot Studio and how it’s growing and how. Agentic apps and agentic development goes in. This is no longer a versus. This is totally with. You should get used to it as developers and you should get used to it as architects. And the sooner you do, the better off any product that you’re working with will be because it is here to stay. And and really, it is when when those two things work together, the solution is overarchingly tons better. I’m seeing this. I’m literally working with a client today where we’re building something that has elements of low code and elements of pro code, and that is becoming more and more prevalent. So I I will definitely say that that’s been a personal challenge for me to realize this is not a versus, this is a with certain preview features are not yet ready for like the enterprise level stuff that that I would like to to see be more more available. Just for example, some of the testing capabilities, certain features that are fundamental are still in preview. So for example, deploying with pipelines, some of the deployments within pipelines using pipelines of Copilot Studio agents from one environment to the next still aren’t. And functioning to the absolute way that you’d like. Well, it’s because it’s in preview and we’re still using it and it’s still great, but it’s still being worked on. One of the things that isn’t fully yet ready I think was I recently tried to use a managed identity authentication within an agent flow and I started running into issues and. I started researching and I found out that there is no documentation and MVP is like, well, just use this. And so we’re all working on this. We’re all in this together. We’re all figuring it out. And the more we do that, the better it’s gonna get. But it is something that I’ve that I’ve had some challenges with, but nothing that you can’t get over. We’re engineers to solve problems. So that’s that’s our job. That’s why we’re here. And then just fundamental changes in approach. I’ll tell you this one of the things that pained me the most. Is understanding that now not everything is going to be exactly the way that I coded it, where you know if I’m if I’m writing a method that’s doing a certain thing, that method is going to do the same exact thing every time, unless I make a change to it and tell it to do something else and my tests are all going to succeed in the same way every time. Getting over that mental hurdle that, hey, I’m a developer and not everything is going to be kind of the way that I would always want, that’s a big change. For example, the responses may vary just a tad. Between one another. And we’ll see that an AI is going to take some liberties and you might have to refine it. And there’s this whole prompt engineering thing, which again, when I first heard prompt engineering, I thought it was this fancy thing. Then I was like, Oh my God, I could never do this. And then I realized, OK, it’s just a fancy word for being very specific with your instructions. And just understanding how all that works. So those are my personal challenges. Those are things that I ran into. Any comments? Ann. Ann Britt 24:55 Yeah, for sure. I think, you know, it’s interesting. So I come from the customer side. I’ve been with Microsoft for almost 2 years now and I used to get immensely frustrated and I still do even internally on the availability of some of the features set, what’s in preview or maybe why things are taking longer. But I mentioned before, right, that security is first. A lot of what you’re seeing now in some of these, I would say delays in release to core product are really because we’re hardening our posture as we’re learning and as we’re growing in this space. The other thing I’ll say is, and Mac, I’m sure you’ve seen this too. In my career, I have never seen a technology move this fast ever before. So I say that to say one of my personal challenges is learning, right? The speed with which this changes, the integrations that are available, the APIs that’s that exist that never existed before. You know, we’re learning about whole new protocols and ways to share and orchestrate these agents. So I think the learning is maybe. Something else that I would add, but that also excites me because it means we certainly won’t be bored for a very long time. Mac Krawiec 26:00 Yeah, for sure. I mean, it is exciting. I mean, the fact that we even get to play with these things in preview is awesome. One of the things that has come in really handy and you mentioned learning, so I just thought of it is I actually had an agent that I built that was connected to the Microsoft Learn MCP server, which is free to use. Ann Britt 26:08 Yeah. Mac Krawiec 26:19 And that makes that a lot easier because that MCP server is obviously going to give you the latest information and all the embeddings are built on the data that’s in there most recently. And so with that said, to anybody here, if you’re dabbling with Copilot Studio, maybe the first one might be to build yourself just a very quick agent that’s answering questions. Questions related to Microsoft documentation that uses the MCP server. That way you don’t have to search through tons and tons of articles. It’s super helpful. But yeah, we’re in this together. We’re learning, we’re doing, so definitely recommend that. Ann Britt 26:51 Yeah, no, that’s awesome, Mac. Build yourself agents, right? Don’t wait for a big enterprise agent. You could automate your whole entire day. It’s a beautiful thing. Mac Krawiec 26:59 Yes, I have co-workers that have built agents that are scheduled to pull the latest information from certain popular blogs to make sure that they have the latest learnings. Use this in your day-to-day like it’s like this is just a more personal since we were talking about use cases, personal use case for developers, not necessarily enterprise grade stuff. But totally, totally helful to our to our day-to-day. But with that said, let’s build O. We’re going to go ahead and jump in. Let me share. Let me change my screen here. I apologize. I am down to one monitor now that I’m across the pond, so I have to. I have to. I’m not nowhere near as professional of a setting as I would normally have. So bear with me here, friends, but we’re. Where we are is copilotstudio.microsoft.com, which if you navigate to that on your own tenant, you’ll probably land in the default environment for your company. As I was mentioning, right, what we’re gonna be building is an F1 sales. Automation agent Really. Where we start is we go straight into agents and let’s just build along. We’ll talk through it. If you guys have any thoughts or anything that you’d like to add, just throw that in the chat. Or any questions that you have, feel free to just fire away or hold them to the discussion and that’s OK too. But we’re just going to go ahead and start with our new agent. If you’ve never seen Copilot Studio before, there’s two ways that you can create an agent. One is where you actually use. Conversational language to configure your language and you could tell it to do, hey, you know, I’d like my agent to do X and do Y or you can configure it straight away. I am a little bit more stringent, so I would like I prefer to configure on my own. So we’re going to see, we’re going to call this the F1 sales agent. And this agent is going to assist us with creating quotes for F1 team parts. Now I have some instructions prepared for us that I think will work best for us. And so we’ll go through them here briefly and then we’ll throw them on into our screens. So here, just one second, you are an agent and we’ll go through this. One of the things that I think I, well, actually I know will improve with due time is the ability to expand. Oh, this. This was not here. I swear to you this was not here like 3 weeks ago. So yeah, so I’m glad that I can expand the chat or the the the the input box because that was pretty decent and available and that was something that I had a a problem with. Ann Britt 29:33 Speed of light. That’s that’s what I’m saying, Mac. It’s like it’s crazy. Mac Krawiec 29:47 But anyways, you are an agent designed to interpret emails. When an e-mail comes in, you should do the following, right? So our agent, the way the workflow is going to work is the workflow of actually many companies. So some of the companies I’ve worked with told me that a lot of their business comes in through e-mail and so and as a matter of fact, a lot of their business, the first. Person to respond gets the business, which is huge because if I build an agent that’s gonna respond to you within say 20 or 30 seconds, which this is the kind of accelerated response times we’ve seen for our clients, then you’re gonna win the business and that’s monumental for making sure that the sales process goes smoothly. Go smoothly and that the ROI is there because typically a sales representative for a lot of companies in the best case scenario takes 15 minutes to get a quote. But if you’re on PTO or if you’re on lunch or whatever, it might be an hour or two and by that time somebody else will respond. Well, if your agent can respond to you within 3030 seconds. That’s a lot better than even 30 minutes. So we’re going to go through the emails and we’re going to scan my e-mail box and we’ll see how that goes. O when an e-mail comes in, you should do the following. Determine if the e-mail, subject and body hint that the e-mail is about a request. For a quote in Formula One parts, if it is, send an e-mail with the quote information including a list of all the quoted parts using the send an e-mail tool which we will build. If not, do nothing and the fact that I can write this two sentences. Again to my develoer mind, like it blows my mind because when I wrote it and it did it, it was just completely mind boggling. The next one is use the knowledge from a SharePoint folder F1 parts to determine what the cost of a given part is. If a part is not on the list or the on hand is 0, please let the customer know that the part is not available. We’ll see that. We’ll add that knowledge base. We’ll add that directory. This is an example. I actually put this instruction here on purpose. Get us thinking about the integration, right? So if we’re thinking about the way that we commonly do things right now, a lot of API is flying around and not everybody’s using MCP servers yet. Well, Jason, if payloads are flying around the world all the time, everywhere, a ton of it. You can have your agent spit out the results, and I’ve done this before for companies, and if you integrate it in this way, you can have it spit out specific Jason schema, which is hugely important given that you can basically integrate with just about anything because you can create a topic or create a. A tool which uses an agent flow that then calls an API and you can send that API request provided this payload and oh by the way, the agent will know to provide that payload automatically and just do it. So that’s the third instruction. And then the 4th one when invoking the send in e-mail tool, the to should be the from. Of the trigger e-mail. So that’ll make a lot more sense when we get through. We’re not gonna be using the reply, we’re not gonna be using the draft, we’re just gonna send a new e-mail to the person. So those are the instructions that I wanna give it and these are very specific. And as you build agents, this is a lot of these instructions whether you’re. Working on them within the configuration of your tool and we’ll get to what the tool is or within the configuration of your agent. Very prevalent for you to get there. So the next thing that we’re gonna do is use knowledge. I actually have a document prepared and I’ll show the team here. Here very simple CSV file. I kept it pretty simple just to save us some time. A very simple CSV file that shows part numbers, the manufacturer, whether it’s Sauber or Ferrari, what the part is and. I mean a front wing, there’s there’s a ton of parts that goes into it. So again, we’re keeping it simple. A clutch is not just a clutch. It happens to be thousands of other little subcomponents. And then then the units of units of measure, some of them are a little bit off, but that’s OK. The price per part, which they’re really low balling Formula One here, trust me. And then the on hand available. So that’s a little bit. Of a sneak peek at the data. And So what we’re going to do and let me share my screen here is we’re actually gonna add and so that file already exists. I have it uploaded in SharePoint and you can add knowledge and choose to add knowledge from Dataverse from D. Those are the typical Microsoft ones, but there’s a ton of other ones that you’ll see once I create this agent, and you can obviously add a knowledge from public websites. What does doing this do? The agent when it attempts to answer questions or attempts to interact with the user. They will go ahead and use whatever sources you specify here to provide the answer. So in our case, we’re gonna use SharePoint and I’m going to navigate to a SharePoint site that we have along with a directory, so. Uh. One second here and I have a F1 parts list here available and I’m going to go ahead and add that and I’m going to say you you really want to be intentional and you want to make sure that some descriptions back when I was developing kind of were like optional. Well, not back. I’m still doing that, but you know, they were optional. Well, now they’re really not. You really have to be very intentional with what you say. And so this document houses holds all the parts that are available to be sold. When a when a customer requests a quote. So that’s the description and that’s the the the context that the agent’s going to have for for this for this file. And so the next thing that we’re going to do, we’re actually not going to do any suggested prompts for now is we’re actually going to create this is doing a lot of the stuff for you and and taking care of of things. The next thing that we’re going to work on is actually triggers. So you can, there’s a few ways that you can surface an agent, right? There’s actually purely and entirely automated, kind of like an automated flow in Power Automate. Or a logic app, but there’s also channels. So if you really wanted to expose your agent to the outside world and to users, these are all the channels that are made that are available. And believe me you, this used to not be this expansive list of four. So it’s it’s moving and one of the most common ones that we’re doing today for clients oftentimes is Teams and 365 copilot. Everybody wants an agent in teams, so this is very common, but you you put it just about anywhere else that you’d like. So with that said, we’re actually not gonna surface this to anybody just because we’re gonna keep it internal. The one thing I wanted to show you guys is now that this agent is created, there’s some other knowledge sources that you can use like Azure SQL, ServiceNow, Salesforce. So we’re really going beyond just the. The typical Microsoft Dynamics ERP kind of with Dataverse or anything like that, we’re expanding and our wings are opening. So there’s a lot more. We’re bringing everybody in and that was one of the key things that Satya mentioned during build is that we. You know, Microsoft aims to be the everything for everybody within opening and adding more and more things to their catalog, so. Ann Britt 37:23 Yeah, Mac, I would just sorry, I would just say one one point to add to that just to maybe punctuate a little bit is really what that allows you to do is maximize on ROI for tools you already have. Mac Krawiec 37:41 Are we there, Ann? Ann Britt 37:42 It’s truly going to allow for our customers to maximize on their investments. Mac Krawiec 37:46 Yep, perfect. Sorry, I just thought. I think I had a lapse in in connectivity and I thought it was you, but it was me. Amy Cousland 37:53 No, yeah, I think, I think, Ann, I think you you for momentarily, we couldn’t hear you. Just want to kind of repeat what you said, Ann. Ann Britt 37:58 Yeah, essentially that those integrations are allowing our customers to maximize on their current investments. Mac Krawiec 38:07 Yes, absolutely. Being able to connect and integrate with more things is if if you’re being siloed to use Azure SQL, just Azure SQL and nothing else, or just D365 sales and nothing else, then you’re less likely to use this. So the fact that we can use this for everything and everyone and the toolbox is expanding. It’s the more better. So one of the things that I just want to talk about really quickly is you’re seeing several cards here. We’re seeing tools, triggers, agents, topics, and suggested prompts. What is a tool? A tool is effectively an integration piece, so you can add various tools, you can add flows that are built. Built in Power Automate, you can add MCP servers that are made available, whether it’s Salesforce or whatever else, and we’ll look at that. You can use agent flows, which are flows that you can create specifically in Copilot Studio. A plethora of different things you can call. APIs directly through those automated flows. Triggers are the extensibility points to making this happen. Believe it or not, on a trigger. So received an e-mail or new opportunity created or new record. Again, various triggers. We’ll take a look at that agent. So I’m sure if you’ve attended other sessions, there was this talk of multi agents, right? And everybody hears multi agents and it’s like, OK, well great really if you think about it, this is kind of a change where an agent almost becomes a method. Or a function in code where that function, if written correctly, is a single purpose and the perfect architecture is always made available and reusable and it’s very modular and kind of like a Lego block. Well, this is the agentic answer to that and now you can have. Agents that for all intents and purposes serve a single function and are a single function that is then called by different agents and using in this case low code, you can actually leverage AI orchestration which lets the agent decide based on the context of the question. And based on the instructions that we provide, let’s the agent decide what other agent to call upon. So that is a low code approach to doing that. The more pro code approach that sometimes is employed for a little bit more. Granular or more specific implementations is the use of orchestration within semantic kernel, which allows you actually now it’s being integrated with Agent One. You can use more specific orchestration, sequential orchestration and such. Which arguably for more enterprise implementations of a sales process, a sequential orchestration within semantic kernel may be the more appropriate approach, but for the intensive versus of a demo, we’re going this way. So that’s just to touch on the fact that really agents can now become functions rather than entire implementations in another. So think about it that way when you approach multi agents and then there’s topics. So if your agent is surfaced to a channel, be it Teams or Slack or whatever, you can add topics that are very procedural and very step by step. And you can maintain those because every agent gets their own set of default topics. I oftentimes, believe it or not, find myself for my use cases to be disabling some of these basic ones or some of these system ones because. That’s just noise that not necessarily everybody wants, and so if my agents are very specific to solve a very specific business problem, I’ll disable some of the default ones or delete them altogether and then enable the ones I’m interested in, which have that more rigid approach. And then suggested a prompt. So if if you’ve ever worked with, I mean Chad, GBT, now Copilot has it where if you start writing there is like little blurbs that pop up above your input box to kind of get what you’re thinking. If you wanted to provide that for the end user, this is where you do that. This is where you add those suggested prompts. So with that said, we’re going to move along a little quicker because I want to make sure that we have enough time. There’s a time check we have. Oh my goodness, we don’t have enough time. We’re going to build this quickly. So for our tools, we’re going to actually go ahead and. Really quickly add a tool and our tool is going to be an Outlook 365 connector. This is very similar to Azure Functions and it’s going to be send an e-mail and so this tool we’re going to get to configure it and we’re going to create it as my connection so you can use. Other types of connections, but in this case we’re going to use my connection that’s going to be acting on behalf of me, Mac at concurrency.com and we’re going to go ahead and configure that really quickly. And I’m going to go ahead and dig into my tool. I I apologize. I do have a bit of a thing prepared just to make sure that we can move along faster. But and this is again where descriptions and this is this was the gripe. I know Ann, this one doesn’t expand yet. That’s the one I want to expand. Ann Britt 43:24 I’ll call a guy, Nick. Mac Krawiec 43:24 So we want thanks. Actually, for all intents and purposes, I’m gonna skip to one. I learned from the build presenters. I have the tool we were building already built, so we’re gonna go through that just to save time. I know that. I want to just get through and hopefully save a little bit time for questions and we’re going to talk through the tools that we have available and the triggers. So I’ve just selected in this agent pane the agent that I’ve built alongside to kind of mimic this approach. One second and it’s thinking. Let me hard refresh here just to make sure that my. Maybe now I’m having technical difficulties from the Internet perspective. And I sure was nothing that a hard refresh control R or command R can’t fix. So we’re gonna go ahead and enter the Mac test bot. And the Mac test bot, for all intents and purposes, is exactly what we were building here today, so. The knowledge that we chose was the exact same file. It’s it’s the CSV file that we had there. The instructions for the agent are the exact same and we’ll talk a little bit about the tool. So we we can configure a tool which I called send an e-mail and the instructions I gave it again very specific when drafting emails in Outlook align neatly and make it easy to read. Kee a rofessional tone for all formatting. Use HTML. This is important. If you’re surfacing it in teams, you’re surfacing in emails, it might be different in teams. I literally just try to make it nice and readable using markup, and then if you do that, it’ll make it nice and readable using markup for teams. But in in for all and and verbs and emails we want it in the HTML. We want it to be nice and clean for quoted items, create a table displaying all the details and it’s going to do that using HTML at the end. Think for the request for the quote. The company that are generating in the emails is concurrency if there are more than one, if there are more than one. Quantity of an item include a total column for a given row. That’s to make it specific that each row might have more than one quantity. At the end of the e-mail, state the total sum of dollar amount. So very specific. And the inputs of this tool are the two, the subject and the body. And notice I told the agent that the two. To for this tool is the from of the trigger and I chose dynamically filled with AI and when you do that the agent will know on its own what the input is. And that’s crazy to me that it just it just does that so. That’s the actual implementation of sending an e-mail, and again, much easier than anything that I’ve ever had to do. You no longer have to use graph or anything like that. And then the trigger, right? So the trigger is actually just in Power Automate. So our trigger is a very conventional trigger. I did add some tidbits to it just from a safety perspective and you guys will see that here in a second. And that is when a new e-mail comes in to my inbox, that’s the trigger and then for each two recipient I actually check to make sure that the sender. Isn’t also the recipient, and that’s because when I was testing this, I was testing it for my concurrency e-mail, sending it to my concurrency e-mail, and then I was in just a recursive hell. So I added that there to make sure that we’re good. And then what we’re gonna do if it’s not and everything is hunky Dory, we’re gonna send a prompt to the specified copilot for processing. And that specified copilot is obviously our bot, and I’ve already disabled all these other topics, which I didn’t really need to do because we’re not surfacing this anywhere. I did not add an agent. This is a very single purpose demo kind of agent. So what I’m going to do. Is actually show you guys a few of the examples that we have. So here let me give me just one second. I have created just in a garbage e-mail and it’s primarily because I. I’ll be sending myself an e-mail and I don’t. Here you are. So I have a proton e-mail. I’m checking it out. I’m going to send it to my concurrency.com e-mail and I’m going to call title it RFQ request for quote and I’ll say I need one Alpine intake, one Red Bull MGUH and one Mercedes wheel rim and I’m going to go ahead and. Send this e-mail to myself and the one thing I wanted to show the team here is if I open Outlook. Just so you know that I’m not lying, I don’t have an e-mail for myself just yet. I’m going to have an e-mail soon as it comes in and what we can do in terms of observability. This is the really nice part is you can go into your flow, your trigger flow. I got the e-mail and I just saw that on my phone, but if I refresh here RFQ right, it came in from me@proton.com. This is exactly what I need. You can actually see the run of the flow, kind of like in Power Automate where the flow is triggered. And it was triggered 8 seconds ago. It ran for three seconds and it called the agent within 3 seconds. Very simple folks. I’m not even going to enter it. But then the next thing you can see is activity for an agent and an activity for an agent you could actually see. Where your agents are being executed last. Now I’ve been getting emails while we were on the phone and because I gave them instructions not to do anything with them, it hasn’t done anything. This was my test right before today just to make sure everything is still working. But this is our run 1249 and I’m gonna access this run and this is cool because for observability sake. You can retrace the individual steps of an agency, the inputs and the outputs. I’m a big inputs, outputs guy and we have our trigger. It got our e-mail content, it used the knowledge source and it actually used in Jason output. It used the CSV file to determine that the price is this and there’s this certain quantity. And by the way, it determined that the Mercedes wheel rim is not available. There’s only an Alpine wheel rim. And then I went ahead and sent an e-mail. So I’m going to look at my mailbox and I got an e-mail from Mac at concurrency. And as you can see, the instructions were pretty clear. It was used nice HTML formatting. Thank you for the quote. Very professional. The Mercedes real room is not available. The only real room is risked is Alpine, not Mercedes. Kind of the same exact response that we were talking about. We have one quantity of this, one quantity of that with a total. The sum total is 43 grand. Thank you for the request. Guys, if I didn’t take us long to build this and when I was building this the first time, it didn’t take me long either. But I have a pretty rudimentary approach for sales processing and if you know for the more complex approaches, obviously. Obviously it’s it’s a little bit more work and you you might have different implementations, but it’s done, it’s done and it’s doing it’s using the data. It’s it’s in and I was, I mean amazed at how how great that was so. That was so huge kudos to the copilot team, but that’s basically the end of our presentation here. I do have just a few more slides now that we’ve built the next slide. I did say that we’re gonna have a second shameless plug. So again, if you wanted to talk more use cases and talk a little bit more about how we can, how we can solve these problems and the more enterprisey approaches, here you are, there’s a chance. And then with that, without further ado, we have a discussion. Don’t just hide, don’t just go away unless you have to, but if you want to stay back and chat. I’m here for it and anything that I missed? Any thoughts, questions, comments if you wanted to start the conversation? Ann Britt 51:29 Yeah, awesome demo. Thank you so much. You know, one of the things you touched on is that right out of the gate you have like better than an MVP. I’m curious, maybe others on the call, like who’s who’s doing this already? What’s the maturity level? Anyone want to throw something in chat? Mac Krawiec 51:31 Um. Ann Britt 51:53 We’ve seen so many really interesting use cases coming through. Maybe I’ll I’ll throw one or two of those at you, Mac, while we’re we’re waiting for people to warm up. But one of the ones that I I think is really neat is an internal use case. Again, we had an AB group in customer service, those with and without co-pilots. And our customer service folks with co-pilots are 12% faster on average and that’s cross 11,500 folks in that customer service area. So it’s not a really cool one, just kind of watching agents handle entire cases. Mac Krawiec 52:27 Yes. And and so that’s the customer approach, customer service approach. There is the SRE. So again more pro code perspective, there’s the SRE agent. I’ve literally seen it revert bad builds after a deployment. So if if something was bad and and it needed to swap the deployments, it went right back to the to the previous build. That worked. I use Copilot or GitHub Copilot every day. Matter of fact, I’ve done webinars on GitHub Copilot and I can tell you the ROI on GitHub Copilot for developers when they’re using it right, because if they’re only using it for completions, it’s really not the whole shebang. The. Huge. And and it saves even even if it think about it, even if it saves you 1/2 an hour every day. That’s a lot of time in the perspective of a year and that’s a lot of time for both you, for your team, for your, for your company, so and and your clients, it’s huge. Abraham had one thing. Would these agents be able to differentiate between spam versus normal emails and how does the flow look like? So yes, they can and and I think we’ve seen an example of that even just today. Where I give it the I give the instructions to look in the body and the subject to look for contextually clues that make this a quote. So that’s already kind of filters the the the output there. Where it’s not gonna bother with the e-mail I got from my coworker double checking that I responded to an e-mail, literally what you guys saw in the inbox cuz I was there and then it didn’t process. But then as far as spam goes, you know, I think with some intentional instructions like let’s say somebody’s spamming you to try and give you fake quotes. We can certainly work that in and say with some more specific prompt engineering, have some more detection. I think really what you might have in a full-fledged implementation is an agent dedicated to just that. That’s detecting spam and and based on certain criteria it’s it’s doing that really bot handling. I have my brother is a data engineer for a huge travel third party company and they handle bots for behavior and activity on their website. You can totally detect that. It’s very difficult. But not not not impossible. So I think that would be more of a bit of a pro code approach just because you might want to use some more interesting approaches. But AI definitely at a POC stage could totally at least solve the 90th percentile of those I think. Ann Britt 55:00 Yeah, I would agree. And it’s gonna learn from the context around your organization as well. I think is is something to consider. You know, yes, we can give it very specific things like it, but we can also ask it to use those context clues. So that’s awesome. Mac Krawiec 55:15 Yeah, especially given that the flow is triggered on basically what’s a flow. If you really wanted to have that more deterministic style rather than just AI, your flow could say what’s our DNB list and add that DNB list, do not do business with list and ultimately filter out those. Emails, but then more instructions that are more agentic could be targeted at a little bit more specific use cases. Great question though, Abraham. Thank you. Any other questions? Any other comments, tears, fears, or maybe experiences that you’ve had? Excuse me, feel free to chime in. I know we’re over. A few people have already left us, but that’s OK. All right guys, well it seems like that’s it. If you need anything again, we have those sessions, but also you saw my QR code for LinkedIn. Feel free to connect, feel free to talk, feel free to chat. If you want my e-mail, I’m also happy to give to give that out. Always happy to talk and and talk shop with anybody. But again, just thank you. First of all to Ann, thank you for being here and thank you for talking with us. But then to you, the audience as well, thank you for attending, taking the time out of your day to listen to me speak. Ann Britt 56:37 Thank you, Mac. 56:39 Thank you.