Recently I revived my old project, “machine to be a cursed other”, inspired by this incredibly cool VR-powered body swap experience.

Back in 2017 I was still a student and was trying to recreate a shoestring budget version of the original. I was in way over my head, not knowing anything about VR (or the relevant 3D graphics knowledge), how to put together hardware, or how to get a camera setup with the right optics.

I wasn’t really up to the original task. But I kept playing with the problem and ended up making something different but weirdly compelling - an AR passthrough system with left/right mirrored vision.

When you put it on, the mirror effect messes up your hand-eye coordination and sense of direction, so even basic things like navigating to the bathroom become really difficult. Actually, even turning your head to look at something generally takes a few tries, during which you might contort your neck while getting further away from what you’re trying to orient toward.

All this to say, a decade later it’s still fun to pass around at parties, so I tasked Claude with getting it to build again and cleaning it up a little, which worked fantastically.

How to have fun with it

I brought two headsets out to Ontario Rationality Weekend and a lot of people tried it out, which gave me a lot of data on what sorts of activities work well with it.

IMO the magic happens when you surprise yourself with your own errors. When someone puts on the headset, the first thing I do is offer to shake their hand. This usually resulted in them confidently sticking out their left hand, and then realizing what went wrong a second later.

It’s important to stay in the sweet spot. Avoid fast movements and things that need depth perception, since that’s too difficult. Avoid things that you could do with your eyes closed, because your muscle memory will steal the fun.

Jenga turned out to be interesting, but the hard part wasn’t removing blocks, it was stacking them on top afterward. Similarly, I tried playing Scrabble at home and placing my tiles without messing up the board was excruciatingly difficult.

Tracing a star shape on paper is also good - the results speak for themselves - although struggling to pick up the pen and find the start point might be a more interesting challenge than the drawing itself.

One interesting note is that doing this standing up greatly increases the difficulty.

My favourite challenge, although it’s a bit high stakes, is to pick a landmark like the exit or the bathroom and ask people to navigate there. Walking with the headset on is pretty disorienting with all the conflicting sensory information, and then getting around obstacles like a chair or desk takes some persistence. There’s some risk of injury so it’s definitely best to do this with a helper.

What makes it so difficult?

Some of the challenge stems from the terrible visual experience - the phone camera is offset to one side of your face, you have no depth perception, a narrow field of view, and a bit more zoom than your natural vision.

But the central challenge is that it turns your brain’s hand-eye coordination system into an error-amplifying feedback loop. Your hand looks like it’s too far left, so you move it right, which visually makes it go even further left! Maybe it’s a little similar to those bicycles with inverted steering?

---

Anyway, it’s not on any app store, but check out the Github repo if you have an Android phone and want to play with it.