machine learning – frans goes blog

Currently in the Canon camp, my only item from their “Lexus” line – of the more high-quality professional L lenses – is the old Canon EF 70-200mm f/4 L USM (pictured). The second picture, the nice crocus close-up, is however not coming from that long tube, but is shot using a smartphone (Huawei Mate 20 Pro). There are professional quality macro lenses that would definitely produce better results on a DSLR camera, but for a hobbyist photographer it is also a question of “good enough”. This is good enough for me.

The current generation of smartphone cameras and optics are definitely strong in the macro, wide angle to normal lens ranges (meaning in traditional terms the 10-70 mm lenses on full frame cameras). Going to telephoto territory (over 70 mm in full frame terms), a good DSLR lens is still the best option – though, the “periscope” lens systems that are currently developed for smartphone cameras suggest that the situation might change in that front also, for hobbyist and everyday photo needs. (See the Chinese Huawei P30 Pro and OPPO’s coming phones’ periscope cameras leading the way here.) The powerful processors and learning, AI algorithms are used in the future camera systems to combine data coming from multiple lenses and sensors for image-stabilized, long-range and macro photography needs – with very handy, seamless zoom experiences.

My old L telephoto lens is non-stabilized f/4 version, so while it is “fast” in terms of focus and zoom, it is not particularly “fast” in terms of aperture (i.e. not being able to shoot in short exposure times with very wide apertures, in low-light conditions). But in daytime, well-lighted conditions, it is a nice companion to the Huawei smartphone camera, even while the aging technology of Canon APS-C system camera is truly from completely different era, as compared to the fine-tuning, editing and wireless capabilities in the smartphone. I will probably next try to set up a wireless SD card & app system for streaming the telephoto images from the old Canon into the Huawei (or e.g. iPad Pro), so that both the wide-angle, macro, normal range and telephoto images could all, in more-or-less handy manner, meet in the same mobile-access photo roll or editing software. Let’s see how this goes!

(Below, also a Great Tit/talitiainen, shot using the Canon 70-200, as a reference. In an APS-C crop body, it gives same field of view as a 112-320 mm in a full frame, if I calculate this correctly.)

Talitiainen (shot with Canon EOS 550D, EF 70-200mm f/4 L USM lens).

Google Clips camera (image copyright: Google).

The main media attention in applications of AI, artificial intelligence and machine learning, has been on such application areas as smart traffic, autonomous cars, recommendation algorithms, and expert systems in all kinds of professional work. There are, however, also very interesting developments taking place around photography currently.

There are multiple areas where AI is augmenting or transforming photography. One is in how the software tools that professional and amateur photographers are using are advancing. It is getting all the time easier to select complex areas in photos, for example, and apply all kinds of useful, interesting or creative effects and functions in them (see e.g. what Adobe is writing about this in: https://blogs.adobe.com/conversations/2017/10/primer-on-artificial-intelligence.html). The technical quality of photos is improving, as AI and advanced algorithmic techniques are applied in e.g. enhancing the level of detail in digital photos. Even a blurry, low-pixel file can be augmented with AI to look like a very realistic, high resolution photo of the subject (on this, see: https://petapixel.com/2017/11/01/photo-enhancement-starting-get-crazy/.

But the applications of AI do not stop there. Google and other developers are experimenting with “AI-augmented cameras” that can recognize persons and events taking place, and take action, making photos and videos at moments and topics that the AI, rather than the human photographer deemed as worthy (see, e.g. Google Clips: https://www.theverge.com/2017/10/4/16405200/google-clips-camera-ai-photos-video-hands-on-wi-fi-direct). This development can go into multiple directions. There are already smart surveillance cameras, for example, that learn to recognize the family members, and differentiate them from unknown persons entering the house, for example. Such a camera, combined with a conversant backend service, can also serve the human users in their various information needs: telling whether kids have come home in time, or in keeping track of any out-of-ordinary events that the camera and algorithms might have noticed. In the below video is featured Lighthouse AI, that combines a smart security camera with such an “interactive assistant”:

In the domain of amateur (and also professional) photographer practices, AI also means many fundamental changes. There are already add-on tools like Arsenal, the “smart camera assistant”, which is based on the idea that manually tweaking all the complex settings of modern DSLR cameras is not that inspiring, or even necessary, for many users, and that a cloud-based intelligence could handle many challenging photography situations with better success than a fumbling regular user (see their Kickstarter video at: https://www.youtube.com/watch?v=mmfGeaBX-0Q). Such algorithms are already also being built into the cameras of flagship smartphones (see, e.g. AI-enhanced camera functionalities in Huawei Mate 10, and in Google’s Pixel 2, which use AI to produce sharper photos with better image stabilization and better optimized dynamic range). Such smartphones, like Apple’s iPhone X, typically come with a dedicated chip for AI/machine learning operations, like the “Neural Engine” of Apple. (See e.g. https://www.wired.com/story/apples-neural-engine-infuses-the-iphone-with-ai-smarts/).

Many of these developments point the way towards a future age of “computational photography”, where algorithms play as crucial role in the creation of visual representations as optics do today (see: https://en.wikipedia.org/wiki/Computational_photography). It is interesting, for example, to think about situations where photographic presentations are constructed from data derived from myriad of different kinds of optical sensors, scattered in wearable technologies and into the environment, and who will try their best to match the mood, tone or message, set by the human “creative director”, who is no longer employed as the actual camera-man/woman. It is also becoming increasingly complex to define authorship and ownership of photos, and most importantly, the privacy and related processing issues related to the visual and photographic data. – We are living interesting times…