SCIoT 2025 Keynote 1
Test or Trust? Why LLM Applications and Generative AI Need Bulletproof Security
Abstract
As Large Language Models (LLMs) rapidly transform software development and deployment, a critical question emerges: Can we trust these powerful AI systems, and how rigorously must they be tested? This talk explores the urgent need for specialized security testing in LLM applications. An example for testing will be AI-ALT (Automated LLM Testing), a novel platform designed to detect and prevent vulnerabilities in generative AI systems. This platform was sponsored by Secure IVAI over the past year as part of SE4485.
We'll dive into the unique security challenges posed by LLMs, including the OWASP Top 10 vulnerabilities specific to AI applications, from prompt injection attacks and data poisoning to unauthorized information disclosure and excessive agency risks. Through live demonstrations and real-world examples, you'll see how seemingly innocent interactions can expose critical vulnerabilities that traditional security testing misses.
Perfect for software engineering students interested in the intersection of AI, security, and software architecture, this talk will equip you with practical knowledge about:
- Understanding emerging LLM-specific security threats
- Designing automated testing frameworks for AI systems
- Building enterprise-ready security solutions with modern architectural patterns
- Implementing responsible AI practices through continuous security validation
Join us to discover why "move fast and break things" doesn't work when the "things" are AI systems that millions depend on, and learn how the next generation of security tools can help build a safer AI-powered future.
Speaker
Dr. Mark Bentsen USA
Director of AI
Secure IV AI
Mark Bentsen leads AI adoption and quality assurance, drawing on 20 years of experience in industry and academia. At Secure IVAI, which he co-founded, he guides companies in responsible AI implementation, building on his success integrating generative AI and leading large engineering teams at FedEx and with a finance and healthcare solutions company.
He shapes academic AI research through university partnerships while advancing AI literacy through conference presentations and corporate training. He holds the CTAL (Full) certification from the International Software Testing Qualifications Board (ISTQB) and is a certified Project Management Professional (PMP).
