#028 - Cyber AI Chronicle - AI Red Teaming

PRESENTED BY

Cyber AI Chronicle

By Simon Ganiere · 28^th July 2024

Welcome back!

Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.

Table of Contents

• What I learned this week

  • TL;DR

  • Understanding AI Red Teaming

    • Importance of AI Red Teaming

    • Getting Started with AI Red Teaming

    • FAQ: Key Questions about AI Red Teaming 

• Worth a full read

  • Bugcrowd: 2024 Inside the Mind of a CISO

  • Threat Modeling with ATT&CK

    • Key Takeaway

• Wisdom of the week

• Contact

What I learned this week

TL;DR

• Discover the critical steps to implement AI Red Teaming in your organization, a process that not only safeguards against harmful outputs but also builds trust for your users and stakeholders. » MORE

• Crowdstrike is obviously still in the news. They have started to share a preliminary post incident review here. The only thing we can hope for is continuous transparency. Microsoft is also going through their analysis of the incident. The whole debate about security product having access to the kernel is just starting!
  Everybody is still going around with their views and theory. Take some of those with a pinch of salt, I have read some commentary that are borderline conspiracy theories 😀 Would however recommend for you to read Kevin Beaumont and Bruce Schneier posts. By the way George Kurtz, Crowdstrike CEO, benefited from one of the fastest news cycle change I have ever seen. Joe Biden dropping from the US election basically removed Crowdstrike from the front pages of newspaper instantly!

• KnowBe4 (a company active in the Security Awareness and Training space), has published an interesting blog post where they have spotted and stoped a fake IT worker from North Korea. The actor managed to pass through all of the HR checks and once they started working malicious activities was quickly identified. This includes the manipulation of fake picture in the CV. This is not the first instance of North Korea worker, the US have recently disclosed a scheme where more than 300 U.S. companies have been been infiltrated. Better increase those background check and those insider threat program!

• In the world of AI another busy week with the release by Meta of Llama 3.1 with 3 models of respectively 8B, 70B and 405B parameters! The 405B model is the one that everyone talks about as it’s the biggest open source model ever released. Mark Zuckerberg did an interview with TheRundown.ai - This is a must watch. Zuckerberg did make a couple of interesting comments on security topics such as the advantage of open-source vs. close and touch based on how he thinks that open-source model can also counter the misuse of AI by nation state.

• If you are using LangChain you might want to ensure you have patched CVE-2023-46229 and CVE-2023-44467. Those two vulnerabilities can lead to code execution and data leakage. PaloAlto is sharing a detailed analysis of those vulnerabilities here. With no surprises, AI software and library are not immune to bug!