A standard digital camera used in a car for stuff like emergency braking has a perceptual latency of a hair above 20 milliseconds. That’s just the time needed for a camera to transform the photons hitting its aperture into electrical charges using either CMOS or CCD sensors. It doesn’t count the further milliseconds needed to send that information to an onboard computer or process it there.
A team of MIT researchers figured that if you had a chip that could process photons directly, you could skip the entire digitization step and perform calculations with the photons themselves, which has the potential to be mind-bogglingly faster.
“We’re focused on a very specific metric here, which is latency. We aim for applications where what matters the most is how fast you can produce a solution. That’s why we are interested in systems where we’re able to do all the computations optically,” says Saumil Bandyopadhyay, an MIT researcher. The team implemented a complete deep neural network on a photonic chip, achieving a latency of 410 picoseconds. To put that in perspective, Bandyopadhyay’s chip could process the entire neural net it had onboard around 58 times within a single tick of the 4 GHz clock on a standard CPU.