For those curious, here is prior art from 12 years ago, capturing light in a coke bottle with a single streak camera.

https://www.youtube.com/watch?v=EtsXgODHMWk

https://web.media.mit.edu/~raskar/trillionfps/

I remember when this dropped and where I was when I watched it and read the paper. One of the coolest things I'd ever seen.

This seems like a great extension of the work. I'm okay with trading accuracy for crude usefulness as a model. Making this interactive and putting it in the hands of curious minds is the next step.

https://arxiv.org/pdf/2404.06493 is the paper. If I'm understanding it correctly, the camera isn't actually capturing a pulse of light. Instead, it's recording single pixels from a 10MHz series of pulses using a single pixel camera that rotates around the object. Then uses this time-series of data to render a video of a "single" virtual pulse via a NeRF.

The "AI" in the title appears to be click bait since the paper doesn't mention AI, and a NeRF isn't really AI in the colloquial sense even though it uses a DNN.

Hmm, if AI is involved I'm always wondering whether what I see is realistic or not.

In the raw footage the wavefront will almost always appear to move from the closer points to the further points simply because it takes time to propagate, and we see that in their example here. This is now somewhat misleading, as they're synthetically moving the camera in post (but not the pulse propagation). I point this out because the team have also developed an "unwarping" technique to cancel it out, and they demonstrate it at the bottom of their website. Note how the scattered light now propagates outward intuitively.

https://anaghmalik.com/FlyingWithPhotons/

https://anaghmalik.com/FlyingWithPhotons/media/moving_videos...

Why does the refraction appear instantly below the bottle rather than taking time for the light to propagate there?