Key insights from the Webinar featuring Dustin Lehr, Co-founder at Katilyst, and David Kosorok, Director of Information Security Programs at Toast. Moderated by Boaz Barzel, Field CTO at OX Security
Let’s face it—AppSec isn’t just about finding vulnerabilities; it’s about building a foundation and culture across your engineering org so that security becomes second nature. But getting there? That’s a journey, and it’s rarely a smooth one. Most companies either go too big too early and crash into complexity, or they under-invest and end up with a patchwork of tools that don’t talk to each other.
We turned to Dustin Lehr, Co-founder at Katilyst, and David Kosorok, Director of Information Security Programs at Toast, to understand the fundamentals of AppSec:
- How to approach an AppSec project?
- What are the must-have tools in your AppSec toolbelt?
- How do you measure success?
Both speakers bring complementary expertise to the AppSec field: David’s background spans development and security, transitioning to AppSec after witnessing how quality-focused processes missed critical security vulnerabilities. Dustin combines over 13 years of software engineering experience with leadership in building AppSec programs at organizations of various sizes, along with founding a company focused on security culture and champion programs.
Here are the key takeaways from the session:
How to Approach an AppSec Project
Start with Stakeholder Conversations
- Build relationships before implementing tools
- Ask each person you meet: “Who else should I talk to?”
- Focus on partnerships rather than dictating security requirements
Understand the Technical Environment
- Create an asset inventory to understand the deployment landscape
- Map repositories to applications and products
- Identify critical assets based on:
- Public-facing vs. internal
- Data classification/sensitivity
- Interconnectivity with other services
- Business criticality
Prioritize Based on Risk
- You can’t secure everything at once – prioritization is essential
- Start with the most critical applications (public-facing, handling sensitive data)
- Demonstrate actual risk to justify budget and resources. Example: use – pen testing to build a business case using evidence of existing vulnerabilities
Partner with Other Teams
- Collaborate with compliance/GRC teams to leverage regulatory requirements: use compliance requirements as a starting point, but aim higher.
- Work with legal teams on privacy and data classification
- Engage with sales teams to understand customer security concerns
Avoid Common Traps
- Don’t purchase tools without understanding if they fit your tech stack
- Avoid the “bright, shiny object” syndrome with new technologies
- Match security efforts to the company’s maturity stage: consider MVPs (Minimum Viable Products) differently than established products.
Must-Have Tools in Your AppSec Toolbelt
Core Security Tools
- Penetration testing (possibly the first to implement)
- Static Application Security Testing (SAST)
- Software Composition Analysis (SCA)
- Dynamic Application Security Testing (DAST)
Program Management Tools
- Asset inventory and prioritization tools
- Vulnerability management platforms
- Data aggregation/visualization tools to create dashboards
People and Process Tools
- Security champion programs
- Developer training resources
- Threat modeling frameworks
Selection Criteria for Tools
- Involve stakeholders beyond the security team in tool evaluations
- Ensure tools match your technology stack
- Consider open source initially to prove value before investing
- Start with a POC (proof of concept) before full implementation
- Tune tools before rolling out to prevent overwhelming developers with false positives
How to Measure Success
Effective Metrics
- Focus on metrics that demonstrate risk reduction
- Track SLA compliance for vulnerability remediation
- Measure vulnerability escape rate (how far issues progress before detection)
- Monitor trend lines showing progress over time – rather than pointing to the unsurmountable mountain of open issues
- Set incremental targets rather than aiming for perfection immediately
Metrics to Avoid
- Avoid metrics that don’t help lower business risk
- Don’t measure the volume of vulnerabilities without context
- Avoid metrics that encourage bad behavior (like paying for bugs found)
- Don’t focus solely on what’s left to fix rather than progress made
Communicating Success
- Ensure clear ownership of vulnerabilities
- Share regular updates with leadership
- Emphasize positive trends to build momentum
- Make the business case by connecting security efforts to business outcomes
- Demonstrate the value of security investments through risk reduction
The session emphasized that AppSec is not a one-size-fits-all approach and should be tailored to each organization’s unique needs, culture, and business objectives. Most importantly, building trust through relationships and demonstrating value were repeatedly emphasized as the foundation for any successful AppSec program, regardless of company size or technical environment.
)