[0:00] Welcome. My name is Chris Lindsey. Today we are talking about the art of vibe coding and staying secure. We’re going to talk about a practical guide. We have a lot of different aspects that we’re going to talk about today. But what I’d like to do is start off with introducing our panel....
[0:00] Welcome. My name is Chris Lindsey. Today we are talking about the art of vibe coding and staying secure. We’re going to talk about a practical guide. We have a lot of different aspects that we’re going to talk about today. But what I’d like to do is start off with introducing our panel. Dustin, please introduce yourself.
[0:17] Yeah, first of all, happy to be here. I’m really looking forward to this discussion. I have a lot of respect for my co-panelists here. So I was a software engineer for 13 years. I spent a lot of time writing code, designing applications, and this was pre vibe coding of course, but definitely learned a lot about the process and so forth. And then I moved into appsec, so I was an appsec leader for multiple companies. And from there I basically do a handful of things. I started a company, Katilyst. We focus on security culture and security champions. I also work for Security Journey, they do secure code training. So I’m helping customers basically build appsec programs and security training programs. And then I do a few other things too. I have a meetup that’s an open discussion every month. That’s a ton of fun. I do little short videos that are kind of silly and sort of bring a little bit of humor, I hope, into the very difficult industry that we’re in. And then I do a lot of speaking and podcasts and discussions like this as well, thought leadership, keynote talks, this sort of thing. So I’m around. I’m a big fan of appsec and basically quality, right? I really care about building high-quality products at the end of the day. So that’s a little bit about me.
[1:52] Very cool. Rob, how about you? Please introduce yourself.
[1:55] Thanks for inviting me, Chris. It’s really great to be here. I’m Rob van der Veer, a chief AI officer at Software Improvement Group, and I help companies around the world with creating better software by looking at their code and by looking at how they work. I’ve been a programmer myself since the age of 10. I was one of the first of my generation. 33 years now in AI and security. Very active in standardization now for AI. I was main author of the ISO 5338 on the artificial intelligence life cycle. I founded the OWASP AI Exchange, which is sort of a central hub for a body of knowledge for AI security, and from there we collaborate with ISO writing the ISO 27090 on AI security and the standard for the European AI Act. This may all sound very exciting, but writing standards is super boring, very political, and takes a very long time. So you should feel sorry for me, but it’s for a great purpose and I feel blessed that I can play my role here. Thanks.
[3:03] Awesome. I’m glad that both of you guys are here. And I am Chris Lindsey, and my background real quick, I’m a field CTO for OX Security. I have 35 years of software development background. Done a lot. Last application I was in charge of was a medication prescribing system, so security had to be paramount. I then went on to build from scratch an application security program for a large enterprise. Was very successful. Had a lot of great times, a lot of fun. A lot of events happened, and it’s kind of one of those things where, when you’re in the trenches, you learn a lot. So it was fun, enjoyable. Like most people in appsec, you get burned out after a little while. And so after three years I moved on and I have been on the vendor side where I go around and I speak at conferences and do a lot. So, Dustin, Rob, amazing guys. This is going to be a great panel today. A lot of fun. And so I am going to be the moderator. I’ll ask questions but I’m definitely going to be jumping in. I am, as Dustin said, I’m not a quiet guy. So the goal of today’s webinar, our goal is for you to learn best practices, increasing development velocity, understanding vibe coding, the good the bad the ugly, because really there’s a lot of value that comes with vibe coding, but a lot of companies really don’t look at vibe coding properly. And so our hope is to get a better understanding, learn some techniques, get some strategies and guides, and understand the importance of developer education as part of this whole process. Our agenda today: we’re going to talk about what is vibe coding, why are companies adopting this development practice at high rates, and what practices we can use to increase development velocity but also validate and get good security exposure throughout this whole process. We’ll talk about prompting techniques, strategy, guidelines for leveraging AI, and the importance of developer education. So with that, I’m going to start off. When we hit the topics, I’ll hit one of you guys first and then we’ll go from there. So, Dustin, what is vibe coding? Softball question.
[5:28] Actually, this has become a very loaded term, and I think there’s two definitions honestly. I think vibe coding is using an AI assistant to help you write your code. But the other side of it is doing that without necessarily understanding what it’s writing, right? And I do want to dig into this, I think a lot of this webinar is going to be focused on this. There’s sort of a right way to do this and a wrong way to do this in my view. You can certainly just copy-paste whatever the AI assistant generates, run it, test it, tweak it a little bit, but not necessarily try to understand what it’s actually doing, taking a shortcut, if you will. This is an opportunity for maybe non-developers to write applications. But the issue there is, not learning what it’s doing means you might be introducing all kinds of issues, security issues and so forth. And I think that’s the bad side. That’s the concern that I have when I see vibe coding happening everywhere. And I think it’s going to have a pretty huge impact on the landscape in terms of the code that’s being produced. So that’s how I would define it, again, sort of two definitions. And I’m sure we’ll unpack more as we talk through this.
[6:50] Absolutely. If I can add to that. So the original term was meant to define that one meaning of vibe coding, like you explained Dustin, which is sort of the extreme form of AI-supported programming, like not bothering about the code at all, just chatting with your LLM and getting a system that hopefully works. That’s sort of an extreme form of AI-supported coding. We’re going to be talking about more than that today, right, how do you use AI to create code. But that’s the way I look at it. So you have vibe coding, which in its final spectrum is somebody who’s just talking to an LLM that doesn’t know how to code. And then the more you know how to write code, typically the less you rely on the LLM to make sure that things end well. But we’re going to touch on that.
[7:50] Yeah, absolutely. And I’ll throw in my two cents. So for me, vibe coding, one of the things that I love about it, as you know there’s people who look at it positively and negatively at the same time, but I’m going to share the positive side right here. As Dustin said, you have groups of people who simply don’t understand or know how to do development, such as a network engineer. They may need a script. When we think of vibe coding, a lot of people think of C, a lot of your common languages, Python scripts and all that. But you may have somebody on the networking team or somebody that simply doesn’t know how to program but needs something. And so they can ask it, “Hey, I need a script that does X, Y, and Z, please create it.” And as Rob said, it’s a conversation with an LLM. “Please create this ability, here’s what I need.” And a lot of cases it will actually generate code for you. And hopefully it’s something that actually does what you’re looking for. And from a software development standpoint, it’s great for getting something started out of the gate. And you’ve got a lot of tools out there such as Cursor that a lot of people are familiar with, Windsurf, Route 44, among others. So there’s a lot of great tools out there. All right, let’s move on to the next question. Rob, I’ll throw this to you. Why are companies adopting this development practice at such high rates?
[9:18] So what I see in daily practice are larger organizations with development teams, enterprises, government organizations, and the extreme form of vibe coding I don’t see being adopted apart from using it to create prototypes. That’s a great use. You can really quickly put something together without a lot of effort. It doesn’t have to go into production, it doesn’t have to be perfect, but it needs to demonstrate a point. Now I don’t work a lot with other parts of organizations, but what I see there is that people that are not equipped to do programming are embracing vibe coding as a way to make things happen, because then they no longer need the IT department. Wouldn’t that be great? So of course the reason organizations adopt it is the promise: more productivity, more fun, and autonomy, especially for people that don’t know how to program. You get a lot of freedom, you don’t depend on your developers anymore. That’s at least the promise.
[10:29] Yeah, Dustin.
[10:31] Yeah, I’ll add to that. I do think there’s a pretty significant efficiency and productivity boost when you use an AI assistant when you’re coding. I think one of the biggest benefits here is getting past writer’s block. I do think one of the hardest things to do as a software engineer is to start, and having an AI assistant there to give you that first prototype, like you were saying Rob, I think is extremely helpful, because then you go from a mode of “I don’t even know where to start, what should I do first” to “okay, well now I have something and I can tweak it and review it and sort of have this conversation with the AI assistant to help me refine it.” That’s a much different mode. So I think there’s very much an efficiency boost that companies recognize. But I do think there’s a little bit of an over-expectation when it comes to what AI can do for folks, and I think we’re in a little bit of this hype stage right now where people are seeing new technology and saying “oh cool, this is finally going to solve a bunch of problems,” and they’re setting expectations a little too high. So I think obviously that will come down. We’ll get past the hype stage. We’ll get back to something that’s actually practical. But I do think a lot of it is just that it’s a trend right now and people are trying to jump on board. So that’s why it’s all happening at a very high rate.
[12:14] Yeah. To me, I’m just going to drop the word sexy, right? You have people out there working at companies, they’re using these tools and it’s throwing code together really quickly. And management sees this, the board of directors sees this, and they see the speed. They see code being generated. Now the question becomes, when a board member looks at this from the outside, they’re like, “Hey look, he’s typing in just a couple of things, it’s generating code really quickly, and it does what he’s asking it to do. So do we really need junior developers? Can we remove some of our staff to augment in this really cool technology called vibe coding and AI-assisted development and really quickly put some stuff together?” And so this is really a cautionary tale of, we need to be careful for what we ask for. I see the value is massive when you look at what AI can do in vibe coding. But we also need to be cautious, because in some cases policies will prevent us from uploading our existing codebase into AI and asking questions, because people forget about the fact that most AI vibe coding tools actually use external sources, and just like any application it saves in their system, and whatever you put in there, whatever you talk about, is stored there even though it’s marked as removed or deleted. As a developer, we know that background databases never delete the data. It’s just not very performant. And so what happens is it’s much easier just to mark a field or record as “this is deleted” and simply exclude it from the queries versus actually deleting it. It creates a lot of things, but we won’t go there. All right. So let’s talk about topics. One of you guys can kick it off: best practices to increase development velocity while reducing security exposure. What are some practices that we can do to increase the development velocity?
[14:38] Yeah, this is something I’ve spent a lot of time thinking about. Like I mentioned before, I was always very quality and efficiency focused when it came to developing, not just for myself but helping my teams become more efficient. And one of the issues that we constantly ran into is rework, or even just troubleshooting in production. That was ultimately caused by things like technical debt. And if you’re not familiar with this term, it’s essentially choosing to intentionally skip over some of the best practices, some of the quality elements, for the sake of speed. You need to get something out the door, so you take shortcuts is what it comes down to. But the problem is, if you don’t go back and pay down that debt that you created, you’re essentially shooting yourself in the foot over the long term, because anytime you have to go back and touch that piece of code, it’s hard to work with, it’s buggy, and it causes all kinds of issues. So what I’ve found, when there’s a production issue that I have to troubleshoot, a lot of times I would tell myself, “if we had just wrote this correctly, we wouldn’t be in this mess.” So when it comes to the question here, what is a best practice to increase development velocity, it comes down to making sure that you’re writing high-quality code from the start. Making sure that your code is maintainable, making sure it’s efficient, it runs well, etc. And a lot of leadership thinks that code writing is just sort of turnkey, that anyone can do it if they have it on their resume that they’re a developer. But there’s a range of talent here. There are folks who are really passionate about building high-quality software. Those are the people that you need on your team if you’re going to create more velocity over the long run.
[17:01] Yeah, we did a lot of research on this, focusing on maintainability and its effects on velocity and on security. And it shows that if you improve your maintainability, you just need a much smaller team, slash you can achieve much more. And also the correlation between high maintainability and a low number of security issues is clear from our research. So the answer would be indeed: create great code. And sometimes you have to wing things, but you need discipline, and the best way to do this is to monitor and to measure and to set some clear and fair objectives. And sometimes you can calculate that it’s better to throw a piece of software away and start over again than to keep on trying to maintain it until everything comes to a grinding halt. And we’ve seen that a lot. We do a lot of due diligences on systems. Sometimes when we look at the code, our conclusion is that this is just simply practically unmaintainable, and so error-prone that it’s like you’re moving around in a piece of code as a developer and it’s so easy to tip something over and either create a reliability problem or a security problem.
[18:17] Yeah, I think that comes up a lot too when you’re not staying up to date. There’s a certain point, and I’m sure we’ll talk about this more, but if you’re not consistently updating your frameworks and libraries and so forth, you’re getting further and further behind. And then over time you have to sometimes make the decision to just scrap the entire thing. And I’ve actually been through several iterations of that. It’s sad when that happens, especially if you’re forced to, like we’ve come to this point where we’re just not efficient anymore and we just have to basically start over.
[18:58] Yeah. Now, some practices that I’ve run into with vibe coding: you can actually give it examples. If you give it examples of code that you’re aiming for, that does increase the quality of the code that’s being generated. The other thing too is smaller pieces, a little bit more bite-sized, don’t try to go for the big bang. Try to go for something a little more bite-sized in the prompting, “hey look, make sure that you write this securely” and other things. Another good practice that I would strongly recommend is you can use AI to secure AI, but you still want to at least use an enterprise tool at the end to review the results, asking “does this look right?” And you’ll find that AI does a fairly decent job at creating good-quality, readable code. Sometimes a little too verbose. For example, one of the talks that I’m doing right now that I’m speaking about, it creates 80,000 lines of code for something that should really be more like 4,000 lines of code. And so sometimes the code quality is too good, and it’s using the SOLID principles, but it does a really good job at generating what you need. You just need to still make sure that, to what Dustin and Rob were saying, review the code, make sure the code looks right, make sure that it’s within the fit. But at some point sometimes the code quality or the code project itself needs to be thrown out and restarted. I’ve seen that too.
[20:44] Yeah. Just a quick note, you’re in the driver’s seat as a developer. AI is a tool and should be used like a tool. If you guide it, or if you don’t guide it, to build secure code, then you’re not going to get secure code. You have to know and be educated in high-quality software engineering practices essentially to guide your AI assistant effectively.
[21:09] There’s a point in the chat that says that coding is not software engineering, coding is only a small part of a developer’s day. This is true. Our research shows that 15 to 20% of development work is coding. So if you simply look at the activity of coding, the amount of velocity increase is mathematically limited, you can only improve the speed of that part. However, and it’s been said at the beginning of this session, vibe coding offers you more than just more efficient coding, because you can quickly create prototypes, you can bounce ideas off this LLM and try things. So I argue that it helps in many other ways, including refactoring, writing test code, writing code documentation. But when it comes to the activity of creating code and making that more fast, you’re limited by the fact that it’s a small percentage of the total work.
[22:17] Yeah. Hey Stephen, I like what you talked about, checking the fit, the code quality. One of the conversations I had last night: realistically your junior developers, in my opinion, should probably stay away from vibe coding. And the reason for that is, if the code’s actually going to end up in production, if your junior developers are using AI to generate code, they’re not going to know that “hey wait a second, this is a multi-tenant system, did it actually put the proper code in there to ensure that the multi-tenancy is working?” They’re not going to be able to recognize some of the things. One of the examples is the code that’s generated for my talk, for SQL injection it actually creates a validation method using regexes to validate “is this SQL injection,” and that’s the wrong approach to actually resolve SQL injection. We all know it’s command parameterization and other methods, but you would never just run a validate method to go “is this right or wrong?” That’s the wrong approach. And so a junior developer, when they’re looking at this, may not know that the methods or the things that are created are right or wrong. And Ashwin, you’re right, how do you guide it? I’ll let the guys speak and then I’ll answer that too.
[23:40] I’ve seen great work being done, for example by Jim Manico, on sort of whispering into the LLM’s ear how to write secure code, giving relevant instructions regarding for example SQL injection. You would think the LLM would have this expertise by itself, but you can in your prompt, in your system prompt, provide it with instructions specific to the language that you want to use and the context, that give you extra assurance that these requirements are applied, but no guarantee, of course. So that’s one of the things that you can do.
[24:22] Yeah. And you can even go as far as creating your own custom assistant that you preload with a lot of these prompt engineering best practices when it comes to security. And that’s something, Jim Manico like you said talks about this a lot, in terms of having companies adopt those profiles and have their entire development workforce use a profile that is preloaded with security best practices. This is one of the best ways I think that you can take control of this at a mass scale level. And going back to Chris’s point earlier, I just wanted to say I think it’s so important for not just junior developers but any developer, even if you’re senior, to make sure that you’re honing your craft. You are responsible at the end of the day for the code that you write. So take that seriously. Be professional about it. Make sure that you’re understanding what code you’re producing, whether you use an AI assistant or not.
[25:29] Good point. I mean, if you’re compromised, a customer is not going to accept “well, it wasn’t our fault, it was the AI LLM’s fault.” So realistically, at the end of the day, just like Dustin mentioned, you are liable for the code that is being generated, regardless if it’s a developer working on your staff or the AI that is generating your code.
[26:00] Yeah, as far as example prompts, Ashwin is right on target here. I think it does come down to simply asking it to produce code that is secure. I don’t know how many times I’ve asked an AI assistant to produce code and then turned around and said “okay, but can you now make it secure?” And it just does it. So it understands the best practices when it comes to writing secure code, you just have to be aware enough to ask it and guide it in the right direction. But even simple things, like input validation: “this particular input comes from an untrusted source, can you validate it, here are the terms, here are the things I want to make sure this input conforms to, write a simple method or function that’s going to validate the input according to those parameters.” And you can just ask the AI assistant to do that and it will create that function for you for that variable.
[27:08] Yeah. Guys, we’re going to move on to the next topic simply because we could be here the whole time and we have more to cover. So let’s talk about advanced prompting techniques for generating secure, high-quality code. This continues to feed into the previous question. What do you guys think, what are some advanced prompting techniques that we can use?
[27:34] We were indeed talking about it. It’s the work that Jim Manico is advocating a lot, which is including instructions that basically cover cheat sheets that are relevant to the work as part of your system prompt or as part of your prompt. That’s a key technique. I see a question here in the chat also talking about perhaps using security-aware models. That’s not a popular research direction. There is some work going on, I believe, I don’t know the details, but the most popular research direction is to use a specific sort of standard LLM and then do something smart with retrieval-augmented generation: picking the right instructions, giving the task and the code at hand to guide an LLM to either write the code correctly or review it. You can basically use the same instructions again, and then sometimes you see that other LLMs are used for the review, although no guarantees of course for the quality of this review, but we’ll get to that later.
[28:51] Yeah. I think Rob covered it for the most part. But it comes back to what I was talking about before, being knowledgeable enough to guide the AI assistant to more secure coding practices. Now you can find OWASP cheat sheets and use the ASVS, let’s say, to understand what secure code looks like, and then take a portion of that and make sure that your AI assistant is aware of it as well before you get started. And I think this comes down to prompt engineering, the order matters. “Generate the code” is different than “here is an example of a piece of secure code, can you generate code that is like that,” or “uses the same practices that this piece of code uses.” That’s much different, and you’re going to get a much better outcome if you go about it that way.
[29:55] Yeah, it’s important for the listeners to understand that LLMs are not perfect. So you can give them all the instructions you want but in that sense they are just like humans. They will use some of it and they typically don’t focus a lot on the middle of the text. That’s where prompt engineering comes in, knowing the tricks, where the most important instructions go. And that’s an ongoing topic of research and trial and error.
[30:30] Yeah. And I think using vetted sources too. There’s a question here from Martin around “can you rely on the AI assistant to generate good advice?” Not completely. I would go to a vetted source created by professionals, like I mentioned, OWASP cheat sheets, ASVS. There’s a lot of sources of really good information that you can use to help guide your bots.
[30:54] Yeah. I’d like to point people to OpenCRE, which, I’m a founding father of it, opencre.org. It has key security standards in there and all the text of them as well, including all cheat sheets and work from NIST and from MITRE and you name it. And there’s an LLM in there, and you can ask questions and it will use the vetted information from those standards to help you answer those questions. And I’ve seen work going on where people are integrating that resource to feed into an LLM, where you have a vast combination of resources on specific topics, feeding that into a prompt to make the code better or to do a proper review.
[31:41] Exactly. And Anthony, when you’re talking about guardrails, one of the things that you can do, and a lot of people don’t think about it: people do their input, they prompt, they say “hey I need this, blah blah blah.” But what they’re not doing is taking the output that is generated from the LLM and asking it, “Did you meet the requirements? Did you meet what I asked you to do?” Because in some cases your model may hallucinate and give you something that looks right, you might think is right, or maybe the hallucination is so far off key that you’re like “no, this isn’t even close.” But taking the results coming back from the request and re-feeding it back in and asking “hey, does this actually do what I asked it to do?”, in some cases you may find that it actually doesn’t meet what you’re asking for. And so then you can re-prompt or ask it again. And just like Rob and Dustin said, giving it examples, pushing it, or giving it documentation: “here’s our coding standards within our company, and when you’re creating the code I want you to follow these standards.” It doesn’t always get it right, and sometimes it’s terrible at it, but that does definitely help build in better results.
[33:00] Yeah, it gets into a loop sometimes. I’ll share some experience here playing with ChatGPT, in this case with my nine-year-old son. We got obsessed with Einstein puzzles, or zebra puzzles, where there’s a series of facts that you can use to figure out the answer to the puzzle. And they all have to be consistent, self-consistent, so that they’re all true. So we were like, cool, that was really fun, let’s have ChatGPT generate one for us. So it generates a series of rules for us to solve the puzzle. And the problem was those rules weren’t consistent. And when we tried to direct the AI assistant to fix the rules so that they would become consistent, it got into this endless loop. It would fix one of the facts and then that would now become inconsistent with another fact. And we were just amazed that it just could not figure out how to actually fix the set of facts. And that applies, in my view, directly to the way that code is generated by the AI assistants too. They’re not necessarily going to create code that is self-consistent either. And that comes down to you being knowledgeable enough to figure that out. So it’s worth educating yourself. It’s worth really reviewing and analyzing the output and making sure it’s accurate.
[34:28] And this really says something about the risk of vibe coding without knowing how to code, because then you’re on your own. You need to be aware that the thing that you’re getting may be very helpful and nice and exploratory, but don’t put it in production.
[34:46] Right. And one of my favorite things is, when you have a chatbot, give it only access to what it needs access to, the columns, the data itself. My favorite was some guy created a chatbot, put it out there, pointed it to a database. The account that it was using didn’t have read-only access, it had full-blown access, and people were able to actually modify and get outside the guardrails and get the system to update the data in the database itself. And kind of the funny thing too is he was pointing to some documentation that was sitting in a folder, but he gave it basically root access from the root, and it was on a Linux box. So you could actually ask it to give you the contents of /etc/passwd, and poof, there it is. So when you’re putting this stuff together, you really need to think about it from a zero trust standpoint. As I mentioned before, vibe coding and creating the code is just one aspect of the whole process. You need to put the requirements together and really, just like a regular developer, an architect, a software architect putting this thing together, lay out what you’re aiming for, what you’re looking for, and make sure that you’re looking at the different security aspects of it beyond just the code quality and the code security issues, but the functionality security as well.
[36:17] Oh, go ahead, Dustin.
[36:19] No, go ahead, Rob.
[36:21] Yeah, what you just said, Chris, highlights a nice approach to allowing people that have no or very little programming skills to build something great: create sort of a sandbox for them. Provide them with the stuff that they need to build what they want to build, but make sure that the data is indeed read-only or that some of the sensitive data is obfuscated or minimized, and then they have an environment where they can really go ahead. Just provide those guardrails and then they can fulfill their wildest dreams.
[36:54] Yeah. And you actually have attackers working actively against you in these cases as well. I want to bring up this idea of slop squatting, which is literally creating libraries that are referenced from AI code that are not accurate. So basically, if an AI bot makes up a library, attackers are actually creating those libraries, so that if you end up using that code with the hallucinated library link, then you’re now pointing to malicious code directly. So I say that only to share that it’s worse than just “hey, I created something that’s maybe low quality.” You can actually be attacked if you generate inaccurate code.
[38:00] Yeah, I think package hallucination, or squatting, what you just described, is a great illustration of LLMs making mistakes that a person wouldn’t make. Nobody would make up a name of a package, but LLMs do. And it highlights the importance of review. And some people underestimate the problem of this. They think, “oh, if it comes up with a package that doesn’t exist, I’ll get an error.” No, you won’t, because adversaries may have reserved that name and put in some code there that messes everything up.
[38:35] Yeah, very true. Guys, we’re going to move on to the next topic. This is a great topic, but I want to make sure that we have enough time to capture everything. So the next topic: let’s talk about strategic guidelines for when to leverage AI tools and when human expertise is really necessary. Who wants to take this?
[38:59] I think this ties into a pet topic of mine, which is that if developers increasingly use AI to generate code and they just occasionally change code, they’re practicing their coding skills less, which introduces, I think, a responsibility for organizations to keep developers actively involved in the act of coding, simply for the purpose of keeping them skilled, maintaining their skills, and building their skills. And it’s counterintuitive, because what you’re doing is you’re dumbing down the AI just to make sure that developers are actually coding more, because you’re going to need those skills. In the coming years there’s not going to be a solution where code is being generated and it all works perfectly. You need developers to review, and in order to review you need to be able to code, and you still need developers to make the changes, because at some points the AI will get lost. And that will be the situation for some time to come.
[40:08] Yeah, I think it comes down to investing in your people. I want to share a story, a real story, where I was working for a company who decided it would be a good idea, without any loss of efficiency, to swap out the entire QA team. So they thought at a leadership level that they could simply remove the QA folks, bring in a third party, and just have them now perform the QA tasks. Obviously that backfired. There was certainly a lot of loss of efficiency because the new people coming in had to learn the context of how to work with the company, how the code works, what the gotchas are, all of that. So this has been going on for a long time, and I’ve heard this too, I worked for the video game industry and they had a very similar mindset: “you’re a developer, you can be swapped out for another developer tomorrow, and another developer could come in and just do the stuff that you were doing immediately.” Not true. It takes months and months to learn the codebase, to learn what you’re doing, all the context, to learn all of those gotchas. And the same is true whether or not you use AI bots, you’re going to need that context, that core knowledge to be efficient as a developer. Period.
[41:38] Well, and you look at, go ahead, Chris.
[41:42] Go ahead, Rob.
[41:44] So you also need the domain knowledge. Think about it this way: AI only has the visibility to what it has access to. In some cases it’s just a specific aspect. But the reality is, for example, previously, the medication prescribing system, we had over 15 different EHRs within our environment that talked to my system, and so the reality is AI is not going to understand the domain knowledge you do. A great example: IBM fired 8,000 of their developers, replaced it with AI. The product that was created was a colossal failure. They actually reached back out and tried to rehire all 8,000 people back. This was about three weeks ago. And they realized that leaning heavily on AI for all generation didn’t work out well. The other thing that I heard at SANS was conversations around companies that focused in on using AI initially early in the stage, they found themselves actually falling behind their competitors. And so again, it’s all about when to properly leverage AI, when to put it into the process, how to augment it in there. It’s a development tool that will assist you with certain things when you actually know how to use it properly. There’s a lot of value with AI, but you just really got to know when and where to use it.
[43:21] And it’s a search, right? This question is bigger than programming. It’s about everything. We’re all searching for the best augmentation. And sometimes, just like you described, it’s a bit overhyped and we have to recalibrate our expectations. But sometimes you try a workflow and it can be almost entirely AI, and sometimes you need to just dissect it into tasks in which the humans excel and where the AI can support it. And I’ve learned that sometimes this division into subtasks is not something that you would typically expect when you distribute a workflow over people. You need to identify those elements in which the AI excels, and I think the world is getting better and better at that.
[44:05] Yeah. All right. Let’s move on to the next topic simply because we’re running out of time. The importance of developer education and accountability for AI tool output and software quality. Dustin, you want to kick us off on this one?
[44:17] Sure. I think we’ve already talked about this in terms of investing in your own education as a developer, but also a message to leadership to also invest in your people and make a concerted effort to educate them, to reinforce high-quality, secure code output. Set up incentives, make sure that you are checking that people are educating themselves and learning about what the code is actually doing that they’re producing. So beyond policy, we were talking about policy earlier, go way beyond policy and find ways to measure, build metrics against high-quality code, so that people are more or less measured in terms of their performance and their ability to create high-quality code, and that will require them to be educated. So providing the tools, providing training for your developers will help them to that end, but at the end of the day you’re basically setting a bar. You’re saying “you need to be at this level when it comes to your code output.” I think that’s extremely important.
[45:36] Yeah. I’ve seen developers blame their AI because it made a mistake. No, dear developer, you chose to use AI instead of doing the work yourself, so you are accountable. And the same goes for review, and review is my biggest concern in this whole thing, because it’s not just static analysis and dynamic analysis and measuring maintainability. People have to look at the code, and more code is being produced, so there’s more pressure on review. The skills that are required to review a piece of code are higher than the skills that you need to write that piece of code, which means that technically only a small part of the team is able to do the best review, and all the pressure is going to be on that small part. People don’t like it, it’s a lot of work, and it’s easy to wing, because the thoroughness of review can’t really be measured objectively. So my biggest fear is that review will be winged, people will just say “oh, it looks okay,” also because AI mistakes are relatively rare. So if there’s a mistake in 1% of the code, at some point when doing the review you simply zone out and you get review fatigue. So review discipline, and really taking that seriously, is I think the most important thing to focus on if you want to get these vibe coding systems secure.
[47:12] That’s an excellent point. For me, so I live in Kansas. Next to us is Missouri. Their motto is the Show Me State. And so for senior tech review, the process that I had for my team, and that I shared out, was: you show me that it works. Show me the screen’s good, show me all aspects of it. Then let’s review the code. Because when you start looking at the code, when you know that it works, you look for the different things. And to your point, if we start getting sloppy and lazy, companies are going to come out and say “hey guess what, I have an AI tool for your senior tech review.” I’ve already seen them, they’re already popping up. And the question becomes: okay, you use AI-generated code, you start using these AI-generated systems, you have it automatically create your AI-generated fixes, you have AI-generated QA, AI-generated push to production, where is the human in the process? Because when you start looking at the whole thing, you really do need that developer in there. Again, the importance of developer education: if we start putting too much emphasis on AI, we have brain rot. And I look at the MIT report that they created a couple months back, where they took three teams of students in a programming class. One used ChatGPT, one used LLMs, and the other used the old-school way of Google. After the projects were done, it’s what you expected, the GPT and LLM teams were done first, followed by a long distant third, Google. But then the instructor took the students into different rooms and asked them questions about what they did, and then asked them how they would fix this or that. The LLM guys, the ChatGPT guys, could not answer it. They copied, they pasted the code, they didn’t learn anything. The guys who did Google actually could answer the questions. And so realistically, when you’re generating code, you really still need to review it, review it during the creation stage and again during senior tech review, because senior tech review is not necessarily about code. It’s really about: does the code match? Does it fit? Are there any obvious bugs? And again, when AI is generating stuff, it creates security issues, and there’s a lot of developers, even senior developers, who don’t understand some of the security issues that are out there. So they’ll review the code and say “that’s great, cool.” But when you have an enterprise security tool that’s reviewing the code at the end, it’s a good training module back to the developer: “hey, SAST tool found this issue, figure it out, you’ve got to figure out how to resolve it.” Now a lot of the SAST tools these days, the next-gen versions, have an AI-driven resolution, but at least look at it, try to understand what it corrected, what it suggested, or better, if it found something, do the lookup yourself. Keep your brain fresh. You don’t want the brain rot for yourself.
[50:30] Amen. Very cool. Anything else we want to throw, or shall we move on?
[50:37] Well, maybe a quick analogy with self-driving cars. Self-driving cars, autonomous cars, are not here yet, which is why we dumb them down to a level so that the drivers are actively engaged, because in that case they’re constantly paying attention if something goes wrong. In a situation where a car would do 99% of the things right, you’re creating the programming situation that we’re in right now. So again, dumbing down the AI, making sure that developers are doing more, is I think essential, because if we don’t do that, then the current senior programmers are going to be the last of their kind.
[51:20] Exactly. There was such a great question from Joshua, if I may. Do you think there will be a shift in what developers will need to understand? There’s this concept of generations of languages, right, lower-level languages versus third-generation languages like Python. Is AI going to bring us into the realm of fourth-generation languages, where maybe we don’t have to think about the details of the actual code being written, we can essentially guide it to write the code? I think there’s a lot there in terms of possibilities and opportunities. The problem though is we’re not there yet. And I think people are thinking we’re there when we’re not. They’re trying to use AI assistants to write code without recognizing the lack of quality that is being written. But I do think that will be corrected over time to a large degree. I think by default a lot of the code generated will be secure and higher quality as we mature this space.
[52:34] Well, and I was on my flight coming here to Minnesota and we were talking about AI, and the funny part about it is, think about it this way. AI is like a kid. You tell a kid “go draw a picture of a car,” you end up with Elon Musk’s Cybertruck. Here’s a truck, you know, just a sketch. But the reality is, if you ask AI “build me a car, here’s wheels, here’s a steering wheel,” see what it produces. So having the full picture, giving it all the details, how it all comes together, is absolutely critical and vital to make sure that you’re really getting a product that matches. And I love the conversation in chat, this is great. To your point, Stephen, 21st century, “what you see is what you get” couldn’t be more accurate.
[53:27] Can I add one more thing to that? Because I think “what you see is what you get,” that way of programming, that super vibe programming, doesn’t work for complex systems. Because complex systems, many modern systems in enterprises and government are super complex, and in order to specify what they need to do, you need a complex specification, which, if it needs to be maintainable, requires layers, requires all kinds of aspects that are part of coding. It’s code. It’s an unambiguous specification that you can maintain. So it requires programming skills to maintain them. So yes, vibe coding in the extreme form, you can do that for simple systems, but complex systems still require programming skills. And one scenario I’ve been thinking about, if you can call it a silver lining or a doom scenario, I don’t know, is that this may result in us just simply building only simple systems, which has been a large goal of IT for a very long time, but we never succeeded. This may be the way. But there’s a question of business value there too.
[54:38] I do think there are differentiators. And this is where the capitalist system will help us refine this over time, because maybe a complex system is creating a certain level of business value that the simple systems just can’t produce. So we’ll see how that goes. And I think it does come down, like you said Rob, to knowing what you want in terms of defining the specifications for the system that you’re trying to build, which is going to take technical skills at the end of the day. Maybe not knowing all the details in 10 years in terms of how the actual code is produced, but you’re still going to have to know best practices and be able to create the specs that you want to get the desired result.
[55:42] Yes, very good. Time, guys, we are two minutes before the end. I have a spot for Q&A, but really I think honestly we’re out of time. I wish we had more time. So what I’d like to suggest to the guys watching this: if you have questions, feel free to forward them to me. You can forward them to chris.lindsey@ox.security. I will gladly take them, answer them, or forward them on to Dustin and Rob. Please look us up on LinkedIn, I know that the three of us are very active, we post stuff all the time on LinkedIn. I would encourage you to follow us and feel free to ask questions. Rob, Dustin, thank you so much for coming today. I really enjoyed this talk. It was one of those topics that I knew we could probably go a couple hours, and I was trying to make sure that we covered all the different aspects of it. So I really do appreciate your time and thank you guys for coming.
[56:50] Yeah, thanks for having us. Great conversation.
[56:55] Was a blast. Thank you guys so much.
[56:58] All right, to the rest of you guys, thank you for showing. We’ll see you later. Thank you.
