Google Pixel 3: This is how the Super Res Zoom works
Google has introduced the new series of Pixel 3 smartphone that still points to a single rear camera. But this does not mean that they are not technologically advanced, that’s how the Super Res Zoom works in detail.
The new line of Pixel 3 smartphones was presented by Google, but details and features continue to emerge. One of the latest news concerns the Super Res Zoom that allows the two new models to have a digital zoom higher than the competition. After a simplified explanation during the announcement, now thanks to the official Google blog dedicated to Artificial Intelligence it is possible to go into details of its operation.
With the Super Res Zoom, we eliminate the need to have a double rear sensor (and relative lenses), leaving the AI to improve the quality of the image, which we know is one of the strong points of Google.
According to the US company, the new digital zoom is better than the solutions used in the past because it does not exploit the upscaling and cutting of a single image but a different approach. With the Super Res Zoom, different frames are combined to obtain a single image at a higher resolution, achieving a performance similar to an optical 2X zoom of other smartphones, according to Google.
Google Pixel 3 and Super Res Zoom: the detail of the operation
In the techniques that use a single image to obtain the resolution necessary for the zoom, the missing information is ” rebuilt ” through the data obtained from the pixels in the vicinity (in this way, up to two-thirds of the image has been recreated without a real own basic information).
With the Super Res Zoom, instead, we aim to use a series of standard resolution images, capturing them in such a way as to compensate for the lack of information by exploiting a different type of digital reconstruction, a technique called ” drizzle “.
As can be seen in the example diagram below, sensor movements (due to the natural tremor of the user’s hand) can be exploited horizontally, vertically and horizontally and vertically. This system can be used with 2X magnification or at most 3X in good light conditions, and it is not a solution that can cover all situations, but it is a step forward compared to traditional digital zoom.
In Mountain View, they also thought about when supports are used to fix the smartphone, it will be in this case the OIS to make movements to the camera module (allowing the sensor to frame slightly different areas).
Always the optical stabilization allows to eliminate the effects of coarse movements (from 5 to 20 pixels) while for the remaining uncorrectable movements from the OIS, we will proceed to align the sequences of images with a reference image (taken from the series of shots) and join them together following the above scheme, interpolating between adjacent pixels in each new frame captured.
Thanks to the algorithm studied by Google it is also able to use this technique with images that show smoke, reflections on glass, leaves moved by the wind and other phenomena difficult to predict in their movements.
Super Res Zoom uses a burst that can reach 15 shots (on Pixel 3) and also reduces noise thanks to the comparison of individual images, as can be seen in the image below. Another problem that has been dealt with is that of ghosting that can occur when a sequence of shots, united, presents a moving subject. In this case, a reference image is always used, and the others are processed only after having ensured that within them, there are the same characteristics of the sample used as a base.
In order to see the potentialities of the Super Res Zoom it is possible to go to the official Google album where you can compare the images taken with Pixel 3 without zooming, with an intermediate zoom and with a zoomed zoom.
