I’m playing a CTF with my bestie, as one does, and she pointed out that challenges that use generative AI violate the cardinal rule of CTFs, which is that everything that is in there is there because someone put it there on purpose
As a longtime player and author, that is genuinely really upsetting
CTFers: really sit down and think about that! That’s actually bad! Like messing with the laws of physics bad
Like, trying to find an exploit in a LLM for malicious prompting?
I'm ignorant to all the nuances but I can't shake how AI thrashes security in a bunch of ways. Hadn't considered CTF but knowing an LLM is trained means that the content and subsequent vulnerabilities could be clear, no?
It's like seeing cookie cutter sites spit out with the same holes to exploit.
it's really dumb too. sucks the fun out of it. having created small CTFs in the past, a lot of the fun of it comes from the author thinking "this is the silliness required to solve" or "they're going to love THIS"
