Real-time 3D Autostereogram with FaceID

Some months ago, I was introduced to a book series called Magic Eye. The books have no text, only some strange repeating patterns throughout the page. But when you stare at a page for long enough and let your eyes unfocus, a 3D image “pops out of the page”. When I first heard this I wasn’t quite convinced, but after trying it and seeing it myself I was surprised by how well it works.

I really wanted to know how it worked, so I set out to write a program that could generate these kinds of pictures. I knew that the FaceID camera of my iPad uses 3D to map your face to make it more secure. I was wondering then if it would be possible to combine this somehow. What would it look like, if you could see yourself “pop out” of the iPad screen?

What is an Autostereogram?

I quickly discovered that this effect is called an Autostereogram and that Wikipedia has a great article for it. If you don’t know what it is, I’d highly recommend giving the article a read, but here is the gist of it.

Using a repeated pattern with pixels shifted in the right way, it’s possible to generate an image where if you make your eyes focus behind the image,¹ a 3D figure appears to pop out of the plane in quite a convincing way.

Random Dot Stereograms

When researching how this effect works, I first encountered a method to achieve this effect called Single Image Random Dot Stereograms (SIRDS). I found it in this paper which also proposed an algorithm for generating these images. This method was the easiest for me to understand intuitively but I ended up using the next method for my implementation since that method was much easier to implement for the GPU.

The key idea of this method is that the if you show two dots on a piece of paper. You can focus your eyes behind the paper in a way such that the two dots converge to one, that is when the left eye looks at the left dot and the right eye on the right dot. The longer apart the two dots are, the further behind the paper you must focus your eyes. Then, by sampling random spots on the paper and then drawing two dots equally spaced apart from the spot. By changing the spacing between the dot pairs slightly, the eyes will be able to match the dot pairs at different depths. This can be used to trick the eyes to see a 3D illusion. This may be better shown by the diagram below.

In this diagram, the distance between the two points (shown by the red line) change how long behind the screen the focus point is.

Depth maps

TODO Write section

Shifting method

An alternative to the random dot method, where pairs of dots are scattered randomly across the image, is to repeat a pattern across the image. Then, by shifting the pattern slightly on the horizontal axis based on the desired depth, the same 3D illusion can be made.

The algorithm goes something like this:

1. Generate a pattern with a certain width and use it to fill out the first rows of pixels in the image.
2. Going from left to right, fill in the blank pixel by sampling the pixel, offset by the width of the pattern to the left of the current pixel, but shifted slightly based on a depth map.

Outline

• What is an Autostereogram?
  • Original paper
    • why doesn’t it work for GPUs
    • Alternative (shifting) method
• Depth maps
• The Algorithm
• Making it run on the GPU
• Combining it with the depth map of FaceID

Demo time