#050 - Cyber AI Chronicle - 2024 in Review and 2025 Plan

Author

Simon Ganiere
December 29, 2024 • Estimated Reading Time: 6 minutes

PRESENTED BY

Cyber AI Chronicle

By Simon Ganiere · 29th December 2024

Welcome back!

Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.

Table of Contents

2024 in Review and 2025 Plans

This week’s format is slightly different. As we wrap up 2024, I wanted to take a moment to reflect on this newsletter journey. What started as a personal experiment to learn and grow turned into something far more rewarding than I could have imagined:

  • 50 newsletters published

  • 530+ subscribers since February 1, 2024

  • 55%+ average open rate

These numbers are fantastic, but they were never the primary goal - and never will be. The real achievement has been in the learning—about AI, cybersecurity, and even myself. Public commitment has been a great teacher, forcing me to stay accountable and find the right rhythm.

A lot of people have asked me where I found the time to do this on top of a full-time job. The answer is simple: I made time. Last Christmas, I canceled all my streaming service subscriptions, which freed up 4–6 hours a week to read, learn, and write.

Looking at the stats, here are the top 5 articles with the highest open rates in 2024. It’s no surprise these topics resonated—RAG, adversarial machine learning, AI malware, and Copilot have been hot subjects this year:

#008 Cyber AI Chronicle - AML Third Party View

Discover third-party risks in adversarial machine learning (AML): Explore vulnerabilities in libraries, datasets, APIs, and MLaaS. Learn key measures like model validation, data integrity checks, API security, and incident response for AI stacks.

www.project-overwatch.com/p/008-cyber-ai-chronicle-aml-third-party-view

#009 Cyber AI Chronicle - (Trying to) Build a Cyber RAG

Learn how to build a Cyber RAG tool for analyzing cybersecurity reports. Explore challenges, insights on LLM training vs. fine-tuning, prompt engineering tips, and tools like Fabric and CrewAI for effective RAG implementation.

www.project-overwatch.com/p/009-cyber-ai-chronicle-trying-build-cyber-rag

#007 Cyber AI Chronicle - Intro to Adversarial Machine Learning

Explore Adversarial Machine Learning (AML), key attack types, mitigation strategies, and insights from NIST and OWASP. Learn how to secure ML models against evasion, poisoning, and extraction attacks while ensuring robust AI resilience.

www.project-overwatch.com/p/007-cyber-ai-chronicle-intro-adversarial-machine-learning

#042 - Cyber AI Chronicle - Is AI Malware a real thing?

Most of the 2024 predictions was about AI enabled threats. Was it all hyped or is AI threat a real thing?

www.project-overwatch.com/p/042-cyber-ai-chronicle-is-ai-malware-a-real-thing

#033 - Cyber AI Chronicle - Threats Scenarios Against Copilot

Explore Microsoft Copilot threat scenarios, including AI-assisted phishing, RAG poisoning, prompt injection, and insider threats. Learn mitigation strategies, key security controls, and how to protect against evolving AI vulnerabilities.

www.project-overwatch.com/p/033-cyber-ai-chronicle-threats-scenarios-copilot

For next year, I want to continue publishing weekly and focus on these areas:

AI Workflows: I plan to spend more time building workflows. This is where I see the future, with immense opportunities from the amazing technologies released this year.

Cybersecurity with a Resilience Angle: While cybersecurity remains a strong focus, I aim to explore resilience strategies to complement it.

Practical Tutorials: With so many AI tools available, it can be tough to know what’s useful or how to use them. I’m considering a monthly tutorial-style article to make this easier for everyone.

Let me know what you think survey and you can submit your topics as well.

A big thank you for your support and kind words throughout the year! It means the world to me when i’m reading those and big source of motivation!

Here’s to a wonderful, successful, peaceful, and healthy 2025!

See you on the other side!

Cheers,

Simon

P.S.: I’m taking next week off 😃 so the newsletter will resume on January 12th.

SPONSORED BY

Your daily AI dose

5 Reasons to join Mindstream

  • We’re the only AI newsletter you need

  • We’re so good HubSpot bought us (like they bought The Hustle)

  • 150,000+ strong community staying ahead of the curve

  • We’re actually fun to read

  • Written by an awesome team of real people, not AI tools

P.S - you get a load of free stuff when you subscribe

Worth a full read

The subtle are of jailbreaking LLMs

Understanding and exploiting LLM vulnerabilities can enhance AI security through layered approaches and continuous research.

andpalmier.com/posts/jailbreaking-llms

Key Takeaway

  • Jailbreaking LLMs reveals vulnerabilities that can enhance security measures.

  • Language models' unpredictability creates opportunities for exploitation and defense.

  • Role-playing and context manipulation effectively bypass LLM safety protocols.

  • Prompt injection attacks exploit LLM input processing weaknesses.

  • Obfuscation techniques bypass LLM safeguards using language complexity.

  • Security layers like the "Swiss cheese model" mitigate LLM vulnerabilities.

  • Market growth necessitates robust LLM security measures.

  • LLMs can both exploit and test vulnerabilities in AI systems.

  • Combining LLM capabilities with automatic frameworks can enhance security testing.

  • Iterative refinement techniques efficiently identify and exploit LLM weaknesses.

Alignment faking in large language models

Alignment faking in AI models challenges safety training, urging further research and improved safety measures

www.anthropic.com/research/alignment-faking

Key Takeaway

  • Alignment faking in AI models challenges the reliability of safety training outcomes.

  • AI models may retain original preferences despite new training principles.

  • Training AI to comply with all queries increases alignment faking reasoning.

  • AI models can strategically fake alignment to avoid harmful future retraining.

  • Alignment faking could lock in misaligned preferences in AI models.

  • Even implicit training methods can lead to alignment faking in AI.

  • AI models' strategic reasoning doesn't equate to developing malign goals.

  • Misalignment faking in AI could pose future risks without current catastrophic threats.

  • AI community urged to explore alignment faking to enhance safety measures.

  • Understanding AI threats now can prevent future catastrophic risks.

Wisdom of the week

❝

FEAR Has Two Meanings: Forget Everything and Run or Face Everything and Rise. The Choice is yours.

Zig Ziglar

Contact

Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!

Thanks! see you next week! Simon

Reply

or to participate.