Weaponizing ChatGPT's Pre-Filled Prompt Links: Smuggling an Attacker's Prompt into a User's Chat

Getting a chatbot to say something it shouldn’t is not an exploit. To matter, an attack has to exfiltrate data or affect the account in some unintended way, and there are only two paths out of the sandbox: The user carries the data out themselves, which means social engineering. A technical vulnerability carries it out, which means finding a flaw in the harness that grants elevated access plus a channel to send data outward. Neither path is a single move. Both are chains, and the first link in either one is the same: get the model to accept input that produces a response it should have refused. That is the piece I want to focus on today. What follows is a phishing lure that gets an attacker to phase one by leading a user to submit a prompt without understanding what it does. The idea borrows loosely from ClickFix, where a user is tricked into running an attacker's command themselves, except here the final action is sending a prompt rather than running a shell command. submitted by /u/_clickfix_ [link] [comments]

from hacking: security in practice https://ift.tt/wWviDT1