Moving to Android (part two): the fold animationMoving to Android (part two): the fold animation

The thing I envied most about the iPhone Duo wasn't the home screen. It was what happens when you open it. The cover screen behaves like a pane of glass. As you unfold the phone, the glass swings away, frosts over and darkens toward its far edge, and the inner screen is revealed behind it with everything already in place. Close it and the same thing plays in reverse. It makes the two screens feel like one object.

The Galaxy Z Fold8 does none of that. The cover screen goes dark, the inner screen lights up, and the layout jumps to a new size. So I tried to build the Duo's animation for the Fold. It's called Fold Light, and this post covers how it works, where the published maths needed changing for a real phone, and its limits.

This is part two of the series. Part one covers the home screen, notification centre, app switcher and icons.

The article that made it possible

Around the Duo's launch, Brandon Ballinger at Empirical Health wrote Building a 3D simulator for iPhone Duo, about simulating the animation in a browser so they could design for it without the hardware. It's the clearest explanation of the effect I've found, and Fold Light is built on its ideas.

The key idea is that both screens are windows onto one flat picture. Imagine the image lying flat behind the phone. Each pixel on either screen shows whatever part of that picture sits behind it, from where you're looking. When the phone is flat, that's just the normal screen. As the glass tilts, each pixel lines up with a different part of the picture, so the content appears to stay still while the glass moves over it.

To find which part, you trace a ray from the eye through the pixel and see where it meets the picture. The article writes that as an inverse camera projection:

Q = C + ((z₀ − C_z) ÷ r_z) × r
 
C   the camera (your eye)
r   the direction from the camera through the pixel
z₀  the depth of the flat picture
Q   the point on the picture that the pixel shows

The second idea is the frost. The further a point is from the hinge, the further the glass has moved away from the picture, so the article blurs more toward the free edge:

blur strength = 96 × sin(fold angle) × distance_from_hinge³

The blur is zero when the phone is flat or closed, because sin(0) and sin(π) are both zero. It's also zero at the hinge and strongest at the free edge. The article renders 14 blur levels ahead of time and blends between neighbouring levels for each pixel.

From simulator to a real hinge

The article simulates a phone on a screen. I was putting the effect on a real phone in my hand, and that changes the maths in a few ways I didn't expect. Most of these took several attempts and a lot of folding to find.

The real glass already rotates

In a simulator you have to draw the rotation, because the phone on your monitor is flat. On a real Fold, the cover glass physically tilts away from you, and your eye foreshortens it for free. All the shader has to do is the remapping: make each pixel show its point Q of the flat picture.

My first versions also drew the squeeze, so the content got squeezed twice: once by the shader and once by the real tilt. At 60° it showed about a quarter of the width it should have, cos² instead of cos. It read as the content being sucked toward the hinge.

Where your eye is

The simulator's camera sits over the hinge. But when you open a Fold you're holding it by the cover screen and looking at the middle of it, so for the cover your eye is over the cover's centre. For the inner screen's swinging half, the hinge is the better guess. The eye sits about five screen-widths away in both cases: 5.17 cover widths, or 5.06 widths of one inner half.

For a pixel at distance u from the hinge (0 at the hinge, 1 at the free edge), tilted by θ, with the eye at horizontal position e and distance D, the ray maths comes down to one line:

q = e + (u·cos θ − e) × D ÷ (D − u·sin θ)
 
cover: e = 0.5, D = 5.17
inner: e = 0,   D = 5.06

Moving the eye matters more than it sounds. At 45°, the cover's far edge should show the picture from 74% of the way across, not the 82% you'd get with the eye over the hinge. That's a noticeably stronger pull outward.

Rows stay level

The full projection also moves content vertically. As the free edge lifts toward you, the far edge should get slightly taller. That's only true if your eye is exactly square-on, about 34 cm from the screen. Held in a hand, it read as the whole picture being dragged down toward the bottom right corner. I dropped the vertical part. Every row stays level and full height, and only the horizontal position follows the maths.

No counter-swing, and a cap

At the very start of a fold, the free edge lifts toward you before it moves away. The true projection makes the content swing slightly the wrong way for a moment, which looks like a stutter. The result is clamped so content only ever moves outward from the hinge.

Near 90° the glass is almost edge-on, and the maths sends the whole picture into a sliver. The compensation stops at 86°. By then the glass has frosted and darkened so much that you can't tell.

The inner screen does the opposite

When you open the phone, the inner screen's left half swings toward you, not away. So the true projection spreads its content outward instead of squeezing it. At a hinge angle of 100°, its left edge shows the picture from 39% of the way across, then 20% at 120° and 2% at 150°. Nothing shrinks, so no black appears around it. I expected a black border at first and assumed the maths was wrong, but the black only appears on the side moving away from you.

Frost that starts sooner

The article's cubic keeps the area near the hinge sharp until about halfway out. On the phone, that read as a sharp picture with a blurry edge rather than frosted glass. I went with a gentler curve, distance to the power 1.35, plus an even frost that grows with the tilt. The curve started from the shader in Duo Fold Live, another take on this effect covered below, and was tuned against videos of the real Duo. The frost and darkening run as one gradient across the whole inner screen. When each half had its own, the halves looked like different colours with a seam down the middle.

On top of the blur, the swinging glass darkens toward its free edge, and a soft front of black sweeps in from the far edge as it steepens. It never goes fully black, so frosted content still shows through.

The blur itself is a pyramid of seven pre-shrunk copies of the frame, blended by the blur radius, with five samples per pixel. An early version looked blocky until I found that Android's shader inputs default to nearest-neighbour sampling, not smooth sampling.

Getting the hinge angle

All of this needs the hinge angle, many times a second. That turned out to be the hardest part.

Android has a public hinge sensor, but on my Fold it only reported three values while folding: 180°, 90° and 0°. Samsung's precise folding sensor needs a permission that only Samsung's own system apps hold.

The workaround is a Samsung fold wallpaper that reacts to the hinge. It writes the angle to the system log as it goes. A helper reads that log, and a second log line from the sensor service fills in when the first goes quiet. The readings are poor, though:

• They're whole degrees, about 25 a second.
• Each one is 60 to 100 ms old when it arrives.
• They stop completely for 230 to 380 ms while the screens switch over.

People fold a phone at 300 to 400° a second, so at 80 ms late the picture lags the hand by 20 to 30°. To fix that, the app fits a line through the last 120 ms of readings and predicts the angle 55 ms ahead. It coasts through the pause and catches up gently afterwards. The sensor service's log line arrives every 11 to 15° moved, so it also sets a hard limit on how far a prediction can run ahead when your hand stops.

Using Samsung's fold wallpaper has a side effect: it replaces yours. So the helper puts your own photos back as layers just above the wallpaper while the phone is unlocked. Locked, it swaps your real wallpaper back in, so you never see Samsung's for a split second on unlock.

The screen swap

The other hard part is the moment the phone hands over from one screen to the other. On the Fold that's a heavy operation. I traced it in the system's own logs:

• The system dims both screens to zero brightness for about 280 ms.
• For about 145 ms neither screen is given anything to show.
• Twice during the switch, no frames reach either screen for 70 to 140 ms, whatever is drawing.

Each of those showed up as a black flash or a jump in the picture. The helper deals with the first two directly. It keeps both screens powered through the switch. It answers each of the system's zero-brightness commands within milliseconds by setting the brightness back. And it pins each screen to the content it was already showing until the system has finished reassigning them.

The frame freeze can't be drawn through, so it has to happen somewhere you can't see it. My first version switched at 85°, halfway through the fold, where a 100 ms freeze is a jump of 30° or more. Now opening switches at 165°, and closing at 172°. Near flat, the inner picture only changes by about 3% in width over the last 15°, so the freeze doesn't show.

There's one more trick. The animation needs a picture of the screen it's revealing, before that screen is even on. So the helper captures one frame at the start of each fold, and the app keeps a recent picture of each screen's home layout. Once the real screen has taken over and settled, the animation fades out and reveals the real thing underneath, pixel for pixel.

How it fits together

There are two pieces. Fold Light is a normal app. It predicts the angle, runs the shader and draws above everything else on both screens. The helper does what an app isn't allowed to do: read the system log, hold brightness, steer which screen is in charge and take screen captures. It runs with the same permissions as a USB debugging session, so a computer has to start it over USB.

The helper never touches the network. It only accepts calls from the Fold Light app, and every change it makes is undone when it stops: wallpapers are restored, layers removed and display requests released. With the screen off, the helper uses about 0.5% of the CPU and the app about 0.2%.

How it compares

There are other attempts out there.

The Empirical Health simulator is the reference for the maths, but it's a design tool. It renders the animation on a monitor, with a perfectly placed camera and no hardware in the way. Its later renders use path tracing for realistic lighting, which a phone can't afford in real time.

Duo Fold Live is an open-source app that does the same job on the Fold8, and it's further along in some ways. It has several animation styles and a settings screen. It runs through Shizuku and needs overlay and accessibility permissions. Its own notes list a black fade during the screen handoff and wallpaper flashing on unlock as known issues. I studied its source, which is MIT licensed, and briefly used its projection as a starting point before rewriting the effect.

Fold Light differs in a few ways:

• The maths is for a held phone. The eye position, level rows, no double squeeze and the outward-only clamp are all tuned for a real hand and a real hinge, not a camera over the hinge.
• It predicts the hinge. Drawing the latest reading leaves the picture 20 to 30° behind a quick fold, so Fold Light draws where the hinge is about to be.
• The handoff is handled in the system, not hidden. Brightness is held, the screens keep their content, and the switch happens near flat, so the black flashes are gone in almost every fold.
• Nothing else changes. It keeps One UI Home and your own wallpaper, and needs no accessibility permission or Shizuku.

Limitations

• It needs a computer after every restart. The helper has to be started over USB each time the phone reboots. Removing that would mean Shizuku or a wireless debugging client built into the app.
• It's built on things Samsung could change. It reads log lines, uses a wallpaper as a sensor and calls internal display functions. A One UI update could break any of them. It has only been tested on my Fold8, on One UI 9 and Android 17.
• The picture is a still frame. The animation folds one capture of the screen, so anything moving, such as a video, freezes for the length of the fold. Secure screens, such as banking apps, aren't captured at all.
• It isn't perfect. In one of five recorded switches, both screens still hit zero brightness for one 8 ms frame. The prediction can overshoot slightly when your hand stops suddenly.
• You can't screen-record it. The effect's layers are hidden from screenshots and screen recording, so recording the screen shows nothing. Any video of it has to be filmed with a camera.
• It isn't released. It's a personal prototype for now.

Next

Part three covers the keyboard.