It’s ok for most encrypted protocols (In salty you can fetch other messages but can’t decrypt). Btw i think recipient can be removed so if someone seen message they tried to decypt, if can’t - its not message to you
It is not possible to remove it, otherwise you do not know that the message is for you. With that information you can’t decrypt.