The core of the scene is fantastic! I definitely think you can keep it, and part of writing is sometimes learning that what we first imagine can be reimagined into something even better or stronger for your characters.

That, or just don't worry about people thinking it's not realistic, and write something like what you wanted to write all along, and deal with the details later. Ultimately it's your story, if the scene you're imagining is the only way something can possibly take place, then write it that way. Authors handwave reality all the time, that's literally what we do best!