Doing buttons in VR is hard.

One of the goofy restrictions we placed on ourselves in Beach Ball Valley is that your hands must be pizza paddles at all times. No putting the paddles away, no using a point-and-click mechanic. Your pizza paddle hands are the only way you can interact with the world.

Designing a UI for pizza paddles is a topic of its own, but the main thing we needed was buttons. Big, squishy, reactive buttons that felt great to press. We figured it would be easy—just put a physics object on an axis and call it a day.

How wrong we were.

Attempt 1: Just Use Physics

It's a pretty simple concept. All we need is a box constrained to an axis, some colliders at the top and bottom, and an overlap volume at the bottom to trigger the button's action.

Unreal will do all the hard work of handling collision and movement for us. Because games are good at handling collision! Right?

Problem 1: Constraining to an Axis

We ran into problems immediately. First of all, while Unreal allows you to constrain movement to a single axis, that axis is not arbitrary—you can only constrain movement along the world's X, Y, and Z axes.

That's annoying, but not a deal-breaker, since we can easily move the physics object onto the axis ourselves. The vector math is quite simple. Let \(\vect{v}\) be the object's position, \(\vect{a}\) be the axis, and \(\vect{p}\) be the root location of the button:

$$ \proj_\vect{a} (\vect{v} - \vect{p}) + \vect{p} $$

Problem solved!

Problem 2: The Constraint Sucks

This mostly worked, but unfortunately, Unreal would resolve collisions before rendering the frame, meaning you could push the button slightly off its axis. This looked and felt sloppy, but there wasn't any way to avoid it because of the order in which Unreal did things each frame.

Our workaround for this was to make the actual physics box invisible, and to track the visible button to the projected location of the physics box. See the picture below.

Problem 3: The Button Sometimes Leaves

This was sort of working, but when we got more aggressive, we found that the physics object would sometimes glitch outside of the colliders that were supposed to contain it. And of course, once it was outside, we couldn't get it back in.

Our fix for *this* was to just reset the button if the physics object ever strayed too far from its home location.

Things were starting to get really messy.

Problem 4: Spamming Inputs

The button's motion finally looked stable, but we found that holding the button down would continuously trigger button presses. This was because the actual physics object was glitching around at the bottom, triggering tons of overlap events.

We tried using the visible, stable button for overlap instead of the physics box, but the physics box was so energetic that even that didn't fix it. We also tried just making the overlap region bigger—this kind of worked, but it didn't fix the problem completely, and it made the button activate too early.

We also tried debouncing the input signal so that you couldn't trigger successive events too quickly, but at this point it was apparent that these were just Band-Aids over a much larger problem.

Problem 5: You Can't Hit the Button Quickly

After all that work, we found that the paddle would almost always miss the button when we were swinging it quickly. This was unfortunate, but it made sense why this was happening.

Basically, the physics engine checks for collisions once per frame (technically, once per physics timestep). It sees if any objects are overlapping, and applies constraint resolution to push objects apart. But since it only does this once per frame, it is possible for fast-moving objects to miss each other. Our paddles are big, and the ends move very fast, so misses were actually more common than successful hits.

We didn't even try to fix this.

(If you want to learn more about physics engines, go watch this excellent talk by Bennett Foddy, the creator of QWOP.)

Problems 6+: Everything Is Bad

Besides all the functional issues, there were a host of smaller problems that we couldn't fix (or didn't fix):

• When the paddle went below the button, the button would pop back up (which felt bad).
• Stabbing the button from the side instead of pressing it resulted in wild, unpredictable vertical motion.
• The button blueprint had become incredibly complicated and difficult to deal with.
• Buttons felt touchy, unpredictable, and generally bad to use.

At this point it was clear that we were polishing a turd. We needed a more robust solution, and the more we thought about it, the more we realized that we hadn't understood the limitations of our approach.

Attempt 2: Forget Physics!

We had our lightbulb moment when we questioned why we were using physics in the first place.

I realized that glitchy physics only occur when the physics engine cannot resolve an impossible situation. Freely-moving physics objects are quite stable, but once they get stuck in the ground, or smashed between two objects, they freak out because the physics engine cannot solve the physical impossibility it's presented with.

But in VR, impossible situations happen *all the time*. In VR, a player can push his hand straight through a tabletop, or pull a doorknob straight up, or throw objects around with alarming force. In our case, the player could stab a pizza paddle straight through a button, causing the physics engine to helplessly try to push the paddle and button apart.

Our approach was fundamentally bad. We needed a system that was comfortable with the impossible situations VR provided. That's when we cracked it.

Box Tracing!

Instead of checking for "collisions", why not just do a box trace up from below the button until we hit the paddle?

This idea is even simpler than the last. We simply start a box trace from way below the button and finish somewhere above the button. If the trace collides with something, we move the button to the height of the collision.

This immediately solved our problems. The glitchy behavior was gone, and we never missed a quick hit. Plus, it felt really good to use!

From that base, we made a couple small modifications:

1. We wanted to be able to stab the button from the side without the button snapping down to below the paddle. To accomplish this, when the paddle first overlaps the button, we find and save the offset between the collision height and the button's current height. We then use that to offset the button's position in the future. The diagram below explains this more clearly.

   

2. Because of modification number 1, quick hits were not always triggering the button, so we added a special case. If the paddle is moving downward very quickly when it overlaps the button, we immediately trigger a hit without doing any tracing.

With these modifications, the buttons did everything we wanted, and felt great to boot. You can see a side-by-side comparison of our old and new buttons in this video:

And what's more, we programmed them without regard to orientation, enabling us to do wacky stuff like this:

These buttons are alive and well in Beach Ball Valley right now, so if you have a Vive, go try them out!