In 2019, Google launched the Nest Hub Max, a big brother to the already popular Nest Hub. While a bigger screen, better speakers, gesture control, and a front-facing camera were all notable additions, there was one trick the Nest Hub put into play early on that looks like we could. start seeing on some Chromebooks in the future: the ability for the camera to follow your face during video calls.
Follow me with face detection
You can see in the video above an example of all of this in action. It’s simple and makes a lot of sense, but how does it work? Honestly, the concept is simple in theory and probably a lot harder to execute. With a solid wide-angle lens, you usually get a lot more in your video than you want. In a video call, you are the main subject and all of this extra information can be removed if needed.
This is precisely what happens with the Nest Hub Max. While it looks like the camera is panning, tilting, and zooming, it’s actually completely software-based. The Nest Hub Max’s camera is a fixed wide-angle camera that captures much more than what you need for a video call, giving you all kinds of room to move around the frame.
Google’s software simply finds your face and digitally crops the excess items in real time to make the camera appear to be moving. With a camera that’s wide enough and a lens / sensor that collects a high enough pixel count, cropping like this actually works quite well. In fact, this is exactly what phones and cameras do to take advantage of Electronic Image Stabilization (EIS) when Optical Image Stabilization (OIS) is not available.
Face tracking is also coming to Chromebooks
Thanks to a few commits found in the Chromium repositories, it looks like this exact thing is on the way for Chromebooks as well. With many devices that have front-facing wide-angle cameras, the same cropping, pan / tilt / zoom tricks can be done on Chromebooks with the right software in place.
You can probably see why this commit caught my attention. Looking at what the aforementioned auto-framing might be like, I had a suspicion that this kind of camera functionality might be for the exact same thing we’re seeing in the Nest Hub Max. Looking at the files surrounding this commit and for other commits on auto-framing, I kept seeing files that looked like this:
Notice the names of these files? Face tracking and flow manipulation are the highlights, and those kinds of actions are just what you would need to get the same face tracking functionality on Chromebooks you get on the Nest Hub Max.
We don’t know what devices this will eventually work on, but I imagine Google can only select Chromebooks with webcams above 720p. The first commit was being tested on a âBryaâ device. As noted in the validation post, the GPU is a big part of that framing, so there’s a chance that only certain Chromebooks will get it in the future.
Finally, I wanted to point out that there is already an indicator for this option in the Canary channel of Chrome OS 99. While the flag does nothing at the moment, it is a sign that this effort is already being implemented. work, so it seems like a safe bet that it will get here before too long. I really hope it will be implemented in a way that includes lots of users, but either way, I can’t wait to see this in action in the future on a Chromebook.