View Recording: GitHub Copilot Agent Mode: The Future of Autonomous Coding

Mac Krawiec 0:05 Morning, everybody. Welcome, Welcome, welcome. I see some people are still joining in, but we’re gonna go ahead and get started. So welcome to our webinar on GitHub copilot agent mode. We’re calling it the future of autonomous coding. I’ve heard a lot of. Contro like controversy around it like it’s gonna replace us, and it’s gonna do this, and it’s gonna do that, and it’s not gonna do this. It’s not gonna do that. Let’s just talk about it. Let’s see what it is and and and go from there. So before we we dive into it. My name is Matt Kavitz. I’m a senior software engineer at concurrency. I’m part of the delivery team. We’re working with clients delivering these products. Often times using copilot to help us in doing so. And strengthen the the product that we ultimately the that we ultimately deliver. And and and so a little bit about me. I’m a huge Formula One fan and you’re gonna see some of that today. I got married. Last year and it’s our first anniversary’s coming up here in a week. So that’s why there’s the highlight there. So shout out to my wife. And as we continue, let’s talk a little bit about. What we’re gonna be discussing today. So first of all, a quick intro and evolution of copilot. We’re not gonna spend a very long a very long time on that. We really wanna understand the coding agent which is code name, project Padawan. We’re gonna get a deep dive into agent mode within Visual Studio code. I’m sorry. Visual Studio. We’re gonna talk about the differences between that and Visual Studio in Visual Studio code, and we’re gonna actually look at the coding agent, which is the the, the. The the next step in autonomous coding. We’re gonna talk a little bit about Azure site reliability engineer. That is an example of the autonomous. Aspects from a DevOps perspective, I think it’s it’s hugely valuable to to look at that there. We’re gonna talk. We’re actually gonna see the coding agent in action. We’re gonna see the agent mode in action, and we’re just gonna talk about how where it’s good, where it’s bad, how we can make it better. I’ll talk a little bit to my experience with it given how I use it and what I’ve seen in excel like. Oftentimes you see on, you know, blog posts. Here’s what it’s good at. And then you actually use it and you find out it’s not. I want to talk to you a little bit about where, where I’m coming from. And then we’re going to talk about the impact on project management. So how we can integrate GitHub with project management tools or just the fact that we can and then dive into some questions? Some commonly asked questions and then if he if the if the audience here has any we can we can definitely dive into those as well. So Hope island. It started in 2021. It came out in 2021 as a peer programmer. It was just auto complete and I’ll be honest with you, it was. I didn’t like it that much. I didn’t find it to be really exciting. I thought it was somewhat cumbersome with the with the chat completions, especially when you had to add the comments. I did like that. It was somewhat helpful in predicting and what you want to do next as the years went on. And it got better. But now, today it’s a whole different story. And if you’re using it just for the chat completions, it’s really you’re actually probably using it wrong, or you’re using it right, but not to the extent that you should. Or you could, but in a few years since its launch, copilot have copilot capabilities have grown tremendously. It’s in use by over 50,000 organizations. And in recent in the recent build. Satya Nadella actually said, hey, it’s no longer your pair programmer, it’s your peer programmer. These kinds of advancements are a fundamental shift. In copilot, being more than just a suggestive helper, it is your autonomous coding assistant. It’ll work with you. And and and we’re seeing that more and more apparent now the the two things that we’re going to be focusing on today is copilot. The chat and the agent modes, which is inside of your IDE or your code editor. It’s kinda like that rubber ducky, but it’s also starting, so copilot chat is your rubber ducky. And then agent mode is your rub rubber, Duer. So it’s actually gonna go out and and make changes for you, and you’ll see that we’ll actually showcase some of that and we’ll showcase the importance of of being very clear. When you wanted to do whatever you wanted to do. And then the next thing that we’re gonna look at is copilot coding agent. It is. It is an agent that works in the background on issues. It submits pull requests. It is that again that peer programmer. It’s goes beyond the code editor. We have an example that that we’ll get into some that my colleague helped. There is a kegerator, so at our office we have. We have some kegs. And the team a few years ago has decided to implement. Some IoT and ultimately allow us to use the keg using that IoT that has, you know, been left unmaintained and we’re gonna take a look at some of the some of what we can do. Using copilot coding agent here soon. Now I think every one of us here has heard of copilot chat and agent mode for the most part. It is in Visual Studio. It is in Visual Studio code and we’ll talk about that. But what I do wanna talk about is copilot coding agent and. It’s and which is this new preview feature or this new preview product in GitHub now? It was nicknamed Project Padawan. Sounds kind of cool. I prefer to call it that, but officially it’s called copilot coding agent. They probably should have stuck with the Star Wars reference. It is capable of that full functionality. It writes the code, it writes the test, it writes the documentation, it creates the PR and responds to your PR feedback. You can talk to it like a human being, almost in a PR. Except you don’t have to wait for 20 minutes for somebody to get back to you. The the big advantage here is is that you can have your repository create issues and ultimately assign your agent to it. You’re gonna start seeing that. I’ll we’ll showcase that here. Here in a second. Now if we go ahead and head over to. GitHub. Can everybody see this? OK, let’s make this just a tad. Tad bigger for everybody. So I have here a very quick. Issue in GitHub that that we created. It’s part of that part of that repository that I told you about and what I found is that the README was was pretty bare. Actually it was like 2 sentences. And as somebody that’s going into this project, I didn’t find that very helpful, but I would like a starting point. Point. So I created an issue here. That said, the readme is completely empty. Doesn’t tell me anything about the project or how it started locally and I I basically asked it to do just some very basic things to give me a product overview what the app does, blah blah blah. I also am interested in some outstanding issues and areas for improvement now. In order to do this, I would have needed a developer that you know was familiar with the code base and was able to do this well now. What I can do? Is use copilot my a my AI pair programmer and assign copilot this task and once it is assigned this task it actually says that hey I assigned the task to copilot. Once copilot acknowledges that it will put the little eye emojis here and ultimately tell me hey I. Working on this, it takes a little bit. So we’re gonna. We’re gonna get away from this for just a second and before. And let this work. But and then one thing that we’re gonna look at. Is. Brian Hayden, one of our solution architects, was went ahead and told copilot hey, we’re using original Face API service. It’s no longer supported. Let’s refactor this to Azure AI Foundry vision service. And this was a while ago and I hope Brian doesn’t mind me showcasing this a little bit ahead of time. I’m assuming he’s going to want to see it for a different webinar, but ultimately. It Co filer went ahead and created a a PR. This is all generated by it. It says this PR refactors the face service from a deprecated Azure Face API and and starts to. Basically what it did is it ultimately. Is still allowing us the use of the the the of the old API while while giving us the Azure AI foundry which is the now recommended. Implementation or solution rather and it went ahead and. I’ve never been in this detail in a PR. Description Unfortunately, but it went ahead and made the code changes all about 17 hours ago it took. It took it took it about, I think 10 or 15 minutes to to get through all this work. And 1st it created an initial plan for the issue. Then it had an analysis for the for the refactoring and then ultimately. It refactored the base service to support Azure AI found revision service, which we’re gonna take a look at now. And some of the changes that it made are are pretty significant. It just added some new end points. And add face using Azure AI Foundry and announce it’s using Azure AI Foundry and making HTTP calls. To our endpoint to create the the the face for the kegerator and then the next thing that it did is it completed the face service, refactored tests and it added documentation for what it did and it did. All of that in about 20 minutes, I think even less so. Now that’s just one issue and it’s just one issue that Brian and just you know created. The fact is, is if you have five issues, you can take those five issues and assign each one of them to copilot and work them in parallel and allow copilot to do this work. Now we assign copilot to start doing some of this stuff. It link to pull request. So let’s let’s take a look at this at this pull request. If we look here at the at the right development is in progress. So it started 2 minutes ago. It was thankful for that. We assigned this issue. And it said, hey, the copilot assign copilot and myself as a as a PR reviewer and started work on behalf of me two minutes ago. So let’s let it. Let’s let it do its thing. We’re gonna jump into. Back into our our presentation here for just a second and then go from and then we’ll we’ll come back to this. So ultimately, the vision behind these agents is that we have different agents for different things. And some of that is already available, so in public preview today in Visual Studio code, specifically where. You can create custom. Agents. So you can have, I think one of the OR or modalities. So one of the things that you’ve seen, I’m I’m sure is there is ask functionality chat and then agent functionality modes. You can create your own mode with specific instructions to get it to do certain things. So for example, you can have a refactoring mode, or you can have a new implementation mode and ultimately use that. I’ll show you exactly how that what that looks like in Visual Studio code here later. But then the the fundamentally the goal is to use these agents and understand the code base. At some point they’re gonna fully plan the implementation and and to some degree they already are, right? You saw the the implementation of that phase service. It planned that out and it knew what it was gonna do. And then it was gonna it went ahead and and and did all the work now. One other one that is super powerful is the Azure SRE agent, the site reliability. Agent and we’ve talked about this a little bit. This is something I’m I’m super actually excited about from from build. We saw some of this coming out of build. This isn’t public preview. You can sign up. There’s a wait list. There’s an SRE agent application that you can you can sign up for effectively. It helps us to respond to incidents quickly. And what it does is it you’re able to ask it questions within your Azure space. So the question is like what changed in my app? Or, you know, visualized requests in 500 errors? You can ask it these questions within your within the context of your Azure subscription, and it’ll ultimately yield those answers. Now one of the other things that it does is it monitors those issues and and and is able to plan around it in real time. So. An example of that is here. We saw that our SRE agent detected a new. Who monitor Azure monitor alert it. It saw that something is wrong and ultimately it started investigating. On the right it started investigating and forming a hypothesis. It started determining and reviewing logs and just analyzing a bunch of different things that that you could be the root cause of of the problem. Now ultimately, what it came up with is a root cause analysis and said hey, I think it’s this or I think it might. Be. That and then based on that, I’m gonna go ahead and create a work item in GitHub, create a GitHub issue that a developer can reference and look at and work with. Now it’s doing this. It is doing this now. These are just screenshots obviously, but it’s doing all of this in minutes sometimes, and you’re able to respond to these incidents. Super fast. Rather than having to wake up in three in the morning and see these alerts. And ultimately not really know what’s going to happen. It will not only detect the issue, it’ll remediate the issue if it’s tied to a deployment, it’ll actually undo the deployment to get back to the previous working version and then create an issue as you’re seeing here to say, hey, you know this deployment failed. Let’s go ahead. And fix that and then we can. We can come back to it. Pretty cool, right? So. We’re gonna take a break from from the slides. Again, let’s check on. OK. So I was gonna say let’s check on our agent, but it finished. So remember, we ask it to just do some documentation. This is a simple use case. Frankly, this is not nearly as complex as what Brian asked it to do earlier, but this is a fundamental first step for me being able to go in there and start doing stuff in the absence of a different developer. Now basically they went ahead and let’s take a look at the files. So if we look at the commits. It updated the README and I’m gonna. I’m not gonna put this in. A. In the. Yeah, let’s let’s look at it in this case. Now some of the things that it added is a a description of the core features. So some some of the features of this of this application is cagline monitoring. You’re able to manage the users manage the faces who can use this and who can’t. It is integrated with Azure. It is old, right? Keep in. Bear with me here. This is this is I think the last commit was five or six years ago. It’s it’s, it’s pretty. It’s getting there. So it’s it’s on using Core 3.1 it’s using Autofac which is interesting. We are using seriloc, so that’s impressive. Now and it tells me based on what I asked. I asked it to, you know, how do I actually set this up? How do I run it locally? It told me all that which is great. Now it did give me some outstanding issues. It it it told me of some some missing documentation, some some lapses in error handling. Now there are some some areas for improvement. Enhance the DevOps pipeline at security. The one thing that I I saw this before all as well is give me the the fact is, tell me frankly that the fact that you’re still on on three, one, that’s a security risk and just start upgrading and. It’s it’s interesting because you could probably use the agent itself to upgrade. We’re not gonna do that now. I don’t. I don’t necessarily have that part ready, but in in just a few minutes of time it documented this for me. It told me exactly how to run it. It told me what the core features of this are and it submitted all that in a very good PR that I can just go ahead and and approve, right? And it’s doing all of that on behalf of me. So it’s it’s the this underlines one of the the key features key important facets of this is this is not leaving you behind. You still need to be there. It’s doing things on behalf of you, and it’s just a way for us to be more effective and more productive. Now. The next thing that we’re going to move on to is a little bit of of the agent mode. So what we’re going to do? Is create a new Azure function. And we’re going to call it the Formula One. API. Just because I like forming A1. And I went ahead and I prepped a bit of a prompt. So we’re gonna use.net eight for now with an HTTP trigger. Function level authorization assign. Now what I did is I prepped a little bit of a prompt and I’ll show you why this is important because one of the most important things that here let’s get this up here on this screen. Beautiful. Can I hope everybody can see this if this is too small, feel free to just. Spam me in chat and I will adjust as needed. Now this is just a baseline Azure function that we’re looking at here, and one of the things that I’ve done is I’ve prepped a very quick. I’ve prepped a very quick prompt. Now this is Visual Studio 2022 and. About a month ago they released in full the agent mode for Visual Studio. It was a thing in Visual Studio code for months where an agent mode existed in in public preview for since the beginning of the year. Now I was really upset for the first half of the year that it wasn’t a thing. In Visual Studio and on the first day of build. It came out and I was super excited and I couldn’t wait to get back to start using it and this is it. This is it. This is here, now and So what you’re looking at here is on the right side. We have copilot chat. But if you look, we can actually enable on the top right. We can enable copilot edits, and when we do that, we can start to. Go ahead and and give it some props now. Like I said, I have given it. I have a prompt already prepared. I didn’t wanna spend too much time riding it with with with the audience here on board, but it’s not nearly as it’s not a fancy prompt and and and but we’ll go through it using the latest C sharpand.net coding standards, create an Azure function to. Maintain Formula One teams. Will maintain. Will maintain the following. Entities A-Team, team principal and drivers A-Team can only have one team principal and two drivers, and we’re gonna be we’re gonna want to just maintain these. It’s just gonna be a very quick crud API. We’re gonna want a crud teams CRUD team principles and then crud drivers. And then because this is a demo, I’m gonna be I’m gonna be very specific. And I’ll tell you here. Why in a second for the first Gener. Use EF core in memory to maintain the data. And this is important because I did this once in a demo and didn’t tell it this. And it went really fancy and it expected a whole database to be ready and stood up and and, well, actually, I didn’t even give it this much of A prompt. I just said it created an Azure function to maintain Formula One teams and interestingly enough what it did is it. It did all of the stuff using EF core. It took a little bit of some liberties with regard to the entities themselves, but here we’re going to narrow it down. And then for the again. So for the first iteration, we want to use EF core memory. And then we’re going to use asp.net. That core integration and. Yeah. So we’re using the 4.1 model and this is important because you have various models here at your disposal. Again, true for Visual Studio and true for Visual Studio code. The model you choose. Is or largely up to you. And it. And it may very well depend on your use case. It’s difficult to know. There’s not one model that’s good for everything. I’ve seen some posts like hey use uh Claude for refactoring and then use Gemini for new. Yes, there are. There is. There is cases for one being better over the other, but in but really it is. It really depends on what you’re using it for. So we’re gonna stick it with four. One for now. And we’re in H mode, so I’m gonna go ahead and and and and send it this prompt now. Again, it’s a it’s a brand new clear solution that you saw. So what’s telling us right now is it’s gonna add EF core memory and asp.net core integration. It’s gonna create entity classes for the team, team, principal and driver. Modeling their relationships because I told it that a team can only have one team principal and two drivers, it’s gonna create a DB context. It’s gonna implement the appropriate Azure functions. Now you know the caveat here is is depending on your architecture you you probably don’t want all this functionality. You want Azure function but. For simplicity sake, we’re gonna keep it in one for for now. And then it’s gonna register DB context and configure DI and it’s gonna go from there. Now let’s take a look at file by file. What the results are so here it creates the team. And well, actually, let’s start with the project structure. It went ahead and created a models folder, a functions folder for driver, team and team principle functions. And data where we have our DB context and so if we look at these files, we have our team files. So we’ve created the the team model. I’m gonna accept that with tab. I like team principal. So we hit tab again for that. I like the driver. We’re gonna hit tab for that. It went ahead and created the DB context so it created the DB set for all of the team principals. Now I’m getting errors because. I’m I’m missing Microsoft entity framework. More core. So I’m not gonna use copilot to add that. We’re gonna keep it simple. There we go. We’re gonna. And then here it created all of the team endpoint functions. We have a function for getting teams, getting a team, creating A-Team, updating A-Team and deleting A-Team. So standard crud. And so nothing special there. But I think that the messaging there isn’t that this is doing it and doing it in this way, the fact that it’s setting all this up for us with the click of a few buttons and a few messages and we’re taking and we. Done. Now the next thing that it did is it created an endpoint for team principal functions. And driver functions. And it’s using again EF core to do all that and it’s set up my ultimately my my start up and I can fix all this later. It’s not, not not really. What I’m what I’m interested in and left function one alone or left out left that boilerplate alone, so we can just go ahead and delete it. Now, what is the value proposition? It is not that I can go ahead and Vibe code my way through an endpoint. Through an API for for managing Formula One teams. It is the time to creating APOC. It is the time that, first of all, I mean this is crud functionality for an API. So so there’s there’s value there and it and it did all that in seconds for the most well actually. Now that’s that’s pretty cool. We’re gonna move on from this. Let’s talk a little bit about GitHub. No. The difficulty I had is starting out is really just getting grips on where this stuff is and what I found is that. GitHub doesn’t really do a great job in communicating, at least to me. What the? New stuff is and and what it’s doing. So and and the example I have of that is actually as part of part of our part of our organization is the fact that. If you look at our copilot here, one second, let me let me get into here. An organization’s copilot policies. Actually models. Models didn’t exist as a as an individual page. A few. A few months ago and then when it did, it didn’t auto enable all these models for you. And if you don’t, you won’t get EM in copilot. And then you won’t get EM in agent mode. So here you see a full list of all of the of all of the models that you have available for you in Asian mode that you can enable. And ultimately. I’m saying this more as a PSA for you to know that you need to look in here to enable this stuff ’cause. If you don’t, you’re missing out on half. Features that that are available to you now in copilot policies in GitHub, there is an important copilot metrics API. If you’re trying to monitor. And see metrics and who’s using it. Who’s not using it? This is a useful API, but really enabling copilot coding agent comes from here, so you need to go in here whether you’re an organization or whether you are an enterprise and come here now, there are some. Uh. In order to be able to enable copilot agent, you need to have either an enterprise with coding agent enabled, which is I. I believe there’s a a cost of $30 per seat. Or you can have an individual pro plus license and that’s important because without that the the this entire work that it did is not available to you. You need that. Fundamentally, I think that all I just wanted to to get across is look in here. More often than not. And look at specifically billing and copilot settings, because a lot of this stuff is not publicly like, hey, we have this now. You just find it one day. The one interesting thing I I found is these custom instructions. So previously one of the preview features of Visual Studio code insiders is you could. Visual Studio copilot agent is you could stick instructions into your repository. And you could have instructions that are repository specific. And recently they added this view which is adding custom instructions organization wide for your agents. So for example, if organizationally everything you’re implementing it in pythonor.net, whatever, you can add these custom instructions and tell it that, and it’ll ultimately know that, hey, hey, this is the this is what I’m supposed to use and then these are the the kinds of. Dependencies to use and not to use. You could do that organization wide here or. As a repository level within within your repository. Now one of the things I wanted to talk to you about is Visual Studio Code Insider or Visual Studio code and Visual Studio. And one of my biggest gripes is is the fact that this is a this experience is different now. Like I mentioned before, Visual Studio code had copilot agent mode for quite a few months before. Visual Studio, but then even from a from a user interface perspective, these are the modes that I was telling you about. We can what’s in public preview now is the is the ability to create your own mode. Like I said, a refactor mode where you specify. Hard hard instructions and say do this. Don’t do that. And you can actually again publish that to your organization and then the other, but then the experience between that and Visual Studio. Is different and it’s even more different with ssms because Ssms has copilot. But you need to actually specify an openingi deployment and yada yada yada. It’s not nearly as integrated as this. My hope is is in due time we’re going to see that become unified. And and just All in all be more be more uniform across across the platform. So let’s jump back into the the presentation here for just a second. I wanted to talk to you about efficiency gains and and and their use cases. The fact is, is. One of the strengths of copilot is is make is the fact that it’s it’s follows predictable patterns. So for example, if if you’re setting up a a new React component or writing an API client like we just did, right? It it is. It is using the predictable patterns and the standards of Azure functions and it’s using the if core and and it’s. That it’s very good in that and the the strength is is in my opinion is the first draft. It’s the first draft of some functionality. It’s the quick POC. And and getting it out there. Now where I’ve used it in production, which is different than POC is generating unit tests and I can actually tell you that it is good for that. There’s caveats, though, and the caveat is that you should probably. Consult closely with your QA team. For those unit tests and the reason why is is it can all sometimes be wrong. It’ll generate unit tests that are. Not necessarily. Testing all the edge cases and where and at some points it’ll it’ll actually make you think you’re right. You will think that you tested this, but ultimately you haven’t. And I think it’s it’s strong to come together and discuss all those cases and and incorporate that into the prompts documentation. I think you’ve seen a great example of that today. Refactoring and improving code. Again, I think that the the example with the Phase API and then using Azure AI Foundry is great. It is refactoring and improving that code and then one of the things that. They marketed as is is translating code between languages and I’ve seen. I’ve seen that actually happen, but not between languages, but between frameworks where. A different framework was used with. I believe this was Airbnbs, a different framework for for testing all of their the entire front end. And they estimated that rewriting all the unit tests for their front end. Take roughly about three years and copilot, or in General AI was able to reduce that to. I believe it took him three months. Which is a huge value proposition if you think about it. So all these use cases are real, and I think that they’re applicable now. The the next thing I wanna talk about is GitHub and and how it drives the project management integration now. Not all of us are using GitHub and for for work, and I have a feeling most of us actually don’t. I have a feeling most of us are using ado and if you are using GitHub great you have a leg up. Now the the fact is is you do not need to lose all of your work. You do not need to lose all of your project planning. All of your project management, all of your work items. You do not lose any of that by migrating and using GitHub. You do gain the copilot agent mode and or coding agent to be able to help you with these tasks. If you do use it in GitHub, but the most important thing is is you can integrate GitHub with ado. You can integrate GitHub with Gira and ultimately sync GitHub pull requests with things like Azure board work items. You can track the development. And then you can actually automate the transitions using GitHub actions and ado. AP is. All of that is is rather easy to set up from within ADO and GitHub respectively, and. This is going to help us to transition to use GitHub more for the development aspects of things and use potentially ado or Gira for the product management side of things. And keep those two separate and ultimate. But but at the same time working seamlessly and integrated. Now this is important because I know I have plenty of clients who are like, Oh my God, we have everything in ado and what you’re telling me is, is we’re moving to GitHub. And do I think the focus is on GitHub? Absolutely. Do I think that? The GitHub project management is gonna get better. Absolutely not. So when I was actually at at build this year, I spoke to the GitHub team and they very, very flatly told me that the GitHub projects area, which does give you some aspect of project management, is more for smaller teams and it’s more for like just everyday us. Rather than enterprises and organizations trying to manage projects, that’s what ado and DRI is for. And that’s integrations exist so that you can still do that on on. While keeping the work in GitHub and making use of these features that they’re going to keep on, adding all of these agents. The coding agent and and more now. With that, there are some common questions that I often get. First of all, how do we access this? I I talked a little bit about that. Fundamentally you have you have plenty of that in, in, in GitHub. Copilot chat is enabled and in. It’s now publicly available, so is agent mode in both Visual Studio code and Visual Studio and Visual Studio. You do need to have a license, right? Right. So either you’re an organization or an enterprise Oregon, just a team, but you’re paying for the copilot seats and that’s enabled. There is copilot free. There are certain limitations and throttles on it, so if you really want to open the floodgates. Having the the the pro version on an individual licensing perspective or having a seat within a team, organization or enterprise is the way to go. From a privacy and security perspective, that is something that is extremely important. And it’s always asked. GitHub copilot does not upload your entire codebase to any public server nor share it. It is always within your, within your, within your repositories within your organization. So anything that’s private stays private. The AI models themselves were initially trained on the World Wide Web, on public open source code. Now businesses take that to another level, and copilot for business. Has enterprise level privacy, so it ensures that your prompts or your completions are not reused to train the models thereafter and the the the big thing is is that. The in this case, copilot for business is also covered under the privacy SLA that GitHub has, and so it guarantees that it will not retain or leak any code. And obviously it is designed for for developer privacy in mind. And so I say that because Microsoft and GitHub has these contracts and agreements to that effect for enterprise users, they would lose the trust on day one or day zero of this happening hour 0 if if some of this was leaked or some of this went out so. Can you use it safely and privately? Microsoft and GitHub tells you yes. Will AI like copilot replace developers? I don’t think so, no. I I heard something interesting that somebody said that within 10 years. All. You won’t write any lines of code. I was scared when I heard that at first. Brian Haydin said that to me. And not necessarily scared, but it’s like, whoa. Like, are you serious? Like, do you really think that? I don’t, I don’t. I think I don’t think we’ll be replaced. I think that the jobs that we’ll do will change. I I think that some of these tasks will will be more architectural for us, more design driven rather than implementation driven. So I don’t necessarily think they will be fully replaced. Now what if Copilot’s suggestion is wrong or suboptimal? Umm. Can it produce bugs? Absolutely. And I think from personal experience, it does so not even necessarily during bad prompting. It just does it. It creates bugs. It helps me solve bugs every day, but it also will lead you astray if you let it. And so I think it’s it. It all goes down to the whole question up above, which is will it replace us and I think. Because of the fact that it creates. Some of these bugs then. No, I actually don’t think that that’s the case. And then the most important one I think is how do we get the most out of it? How do you use it effectively? I think that. There is. There is plenty of ways to use copilot. I use copilot in teams to help me debug. I use copilot agent to help me write code. I use copilot edits to to to make some changes, but I actually do find myself more often than not using the chat functionality for production code. I don’t like agent mode for for production. Code I I think it it does a little too much that I personally I prefer to use it in chat. And and get more specific with how I want to do things. I use it every day to help me debug and help me. It is really that rubber ducky for me. So how do I get the most out of it? I I use it by by having it by being that conversation by allowing me to not necessarily have to engage with another developer. It is that. It is that peer programmer that I can talk to to solidify an implementation and then come to the table with the rest of the team. So I think that that’s that’s important. And then the key takeaways really. It’s it’s not been long. It’s been. Three or four calendar years, but really it’s been more like 3 1/2 years and it and it went from just this eight. This copilot AI that can perform some code completions now implementing entire APIs for us, which you saw. Taking care of issues in GitHub and and and which again you saw then it’s gonna get better and better and better. Now looking ahead. There is Asian mode and project panel on which obviously coding agent, but I prefer project Padawan. They are the future of autonomous coding. But at the end of the day, it’s also important to remember that copilot is a copilot. It’s in the cockpit with us. It’s it’s. It’s along for the ride. We’re we’re we’ve we’ve got our hands on on. Well, I said cockpit. So we’re we’re flying the plane. And so. Well, the idea is is to is to empower developers. Not necessarily. Remove them. Now the world of software development. Changing completely at a swift pace and so embracing these tools. Will give you not the edge it gave you. The edge. Maybe three years ago. Now it’s gonna be the utmost necessity, where without it, you are actually gonna fall behind. So use copilot, use AI. Use it as your assistant. And if you don’t, you probably get left behind. So I really encourage you to to embrace it. Now, with that said, how can we help? First of all, there is a survey that our team posted in the chat. If you wanna, if you wanna take that survey, we’ll get in touch. But we have a few things that we can help from from our organizational perspective. We have GitHub copilot usage metrics envisioning session, so if you’re trying to get the best of or better understanding of how your developers are using this or or or they’re not using it or what language they’re using it for. And then ultimately be able to track that more and report on it. We have a session for that where we could. We could talk through that. I think the the more important one almost is copilot agent use case discovery session. So this session is is centralized around your business and and determining hey, where can you use copilot agent. Where is it helpful for? Is it gonna help you in this refactoring? Is it gonna be helpful in writing these tests and how do we make that happen in the best way possible? And then there’s of course, our executive AI envisioning session and it’s positioning. It’s above my pay grade, but it’s positioning. At an executive level, how can we best position a business? To embracing the AI now, shameless plug some of our team members have some other upcoming webinars. Real results with Microsoft Copilot. What’s new and what works? This is Microsoft Copilot, not copilot, not GitHub copilot. So this is more of the the the low code implementations and then pro code meets low code navigating the full stack seamlessly. That’s going to be really interesting because. I’ve used. Copilot agent and I code everyday but then based on the engagements I’ve been at, I’ve been on. I’ve also needed to move on to the more low code area and using copilot studio and and Azure AI Foundry and all that, and so I think that both of those are great. I’m actually personally going to attend the last one. Because it’s gonna, it’s gonna really bridge the gap and help us imagine as developers. How can we embrace low code? Because I know. Hope that one year ago me would have been scoffing if you told me that I gotta use a GUI for stuff. So that’s that. I do wanna thank you for for attending. I do wanna thank you for your time. If you have any questions or if you wanna talk about anything, I’m happy to hang back. Amy Cousland 46:35 Mac, I see you did have a question in here. Mac Krawiec 46:35 And. Oh here. One second. Let me stop sharing because I don’t even know the question. I apologize, Rob, along the lines of replacing developers, perhaps not fully replacing experienced developers. Have you seen instances where existing development teams have experienced resources, are utilizing less experienced developers and having to spend significant time reviewing code of less experienced developers be able to delegate those tasks to? Copilot themselves. Yes. So I think that. And we’re even seeing that in the job market. I think that a perfect example is, is the fact that. This coding agent and even agent mode. It’s doing stuff that you would typically ask. An entry level developer to do so. Yes, 100% it is. It is impacting the the entry level developers. I think that from a market perspective it’s gonna change because an entry level developer is gonna is not gonna be just an entry level developer in the sense that we understand it today. It may very well be. It’ll be more thought provocative. Is is entry level is gonna become closer to. Design. Now that’s gonna be an issue for me because some of that design comes from experience. And how do you get that experience without without first, you know, putting your, your, your, your keys on the your your fingers on the keyboard and coding stuff away and being wrong. I do think that. For sure. Less experienced developers are impacted. And so yeah, there. Fundamentally, it’s it’s it’s gonna happen. How can we register for the next webinar? Great question. Amy Cousland 48:28 Yeah, I putting that in the chat right now. Mac Krawiec 48:28 I think Amy’s got you there. Amy Cousland 48:31 I have that exact link, just put it in there right now. There you go. Mac Krawiec 48:34 Rob, I know. Amy Cousland 48:34 It looks like like. Got another question there about Bitbucket. Mac Krawiec 48:40 Yeah, I’m looking so. So first of all, Rob, if I answered your question in like a very political way, just hit me up like I’m happy to talk about it. Fundamentally, I think that less experienced developers are impacted. I think that. Again, these PRS are evidence of that we’re we’re able to do more with less and and I only showed you the. Doing 1 task or or the other task that Brian did all at once. We haven’t really talked about the fact that again, you can create five of these issues and assign copilot to all five of these issues at the exact same time. And so how that’s going to impact velocity and how that’s going to impact the fact that hey, maybe I. Don’t need 2 developers to do these five tasks in, you know three hours each. Maybe I can use copilot to do it and then use a senior developer to to review, because the other thing is you can have a conversation with coding AG. Inside of the PR, so you saw the PRI can actually and I wish I showed you this. I apologize. You can actually say, hey, you know you please change it to this or this isn’t right or why would you do this and in the PR, the agent’s actually gonna make the change and actually gonna do whatever it needs. So I think it really impacts how we. How we’re gonna work. It’s notesh. Can we use copilot? Asian with big bucket, I don’t know. I don’t believe so, but I can find out for sure. I believe that this is only only is not in in in big bucket, but I can totally find out and get back to you. So if you just leave me your e-mail in that survey and Amy and I will be sure to get back to you. If are there any other questions, I see that Rob is is feverishly typing away. Maybe, maybe some others wanna join in on the fun. Yes. And that’s ultimately what we want to do and that’s ultimately what we want to use AI for. So. Any other questions before we we call it a day? Cool. Amy Cousland 51:05 Looks like we’re all good. Yeah. If anybody has any additional questions, feel free to put them in the survey like you said or you know, Mac or somebody from our team can get in touch with you. If you put a note in there. And thank you everybody for coming. Thanks, Mac. It was a great, great, great session. I appreciate it. Mac Krawiec 51:20 Thank you, Amy, and thank you everybody. Thanks for attending. Have a great day and take care.