artificial intelligence – frans goes blog

The right camera lens?

Currently in the Canon camp, my only item from their “Lexus” line – of the more high-quality professional L lenses – is the old Canon EF 70-200mm f/4 L USM (pictured). The second picture, the nice crocus close-up, is however not coming from that long tube, but is shot using a smartphone (Huawei Mate 20 Pro). There are professional quality macro lenses that would definitely produce better results on a DSLR camera, but for a hobbyist photographer it is also a question of “good enough”. This is good enough for me.

The current generation of smartphone cameras and optics are definitely strong in the macro, wide angle to normal lens ranges (meaning in traditional terms the 10-70 mm lenses on full frame cameras). Going to telephoto territory (over 70 mm in full frame terms), a good DSLR lens is still the best option – though, the “periscope” lens systems that are currently developed for smartphone cameras suggest that the situation might change in that front also, for hobbyist and everyday photo needs. (See the Chinese Huawei P30 Pro and OPPO’s coming phones’ periscope cameras leading the way here.) The powerful processors and learning, AI algorithms are used in the future camera systems to combine data coming from multiple lenses and sensors for image-stabilized, long-range and macro photography needs – with very handy, seamless zoom experiences.

My old L telephoto lens is non-stabilized f/4 version, so while it is “fast” in terms of focus and zoom, it is not particularly “fast” in terms of aperture (i.e. not being able to shoot in short exposure times with very wide apertures, in low-light conditions). But in daytime, well-lighted conditions, it is a nice companion to the Huawei smartphone camera, even while the aging technology of Canon APS-C system camera is truly from completely different era, as compared to the fine-tuning, editing and wireless capabilities in the smartphone. I will probably next try to set up a wireless SD card & app system for streaming the telephoto images from the old Canon into the Huawei (or e.g. iPad Pro), so that both the wide-angle, macro, normal range and telephoto images could all, in more-or-less handy manner, meet in the same mobile-access photo roll or editing software. Let’s see how this goes!

(Below, also a Great Tit/talitiainen, shot using the Canon 70-200, as a reference. In an APS-C crop body, it gives same field of view as a 112-320 mm in a full frame, if I calculate this correctly.)

Talitiainen (shot with Canon EOS 550D, EF 70-200mm f/4 L USM lens).

Photography and artificial intelligence

Google Clips camera (image copyright: Google).

The main media attention in applications of AI, artificial intelligence and machine learning, has been on such application areas as smart traffic, autonomous cars, recommendation algorithms, and expert systems in all kinds of professional work. There are, however, also very interesting developments taking place around photography currently.

There are multiple areas where AI is augmenting or transforming photography. One is in how the software tools that professional and amateur photographers are using are advancing. It is getting all the time easier to select complex areas in photos, for example, and apply all kinds of useful, interesting or creative effects and functions in them (see e.g. what Adobe is writing about this in: https://blogs.adobe.com/conversations/2017/10/primer-on-artificial-intelligence.html). The technical quality of photos is improving, as AI and advanced algorithmic techniques are applied in e.g. enhancing the level of detail in digital photos. Even a blurry, low-pixel file can be augmented with AI to look like a very realistic, high resolution photo of the subject (on this, see: https://petapixel.com/2017/11/01/photo-enhancement-starting-get-crazy/.

But the applications of AI do not stop there. Google and other developers are experimenting with “AI-augmented cameras” that can recognize persons and events taking place, and take action, making photos and videos at moments and topics that the AI, rather than the human photographer deemed as worthy (see, e.g. Google Clips: https://www.theverge.com/2017/10/4/16405200/google-clips-camera-ai-photos-video-hands-on-wi-fi-direct). This development can go into multiple directions. There are already smart surveillance cameras, for example, that learn to recognize the family members, and differentiate them from unknown persons entering the house, for example. Such a camera, combined with a conversant backend service, can also serve the human users in their various information needs: telling whether kids have come home in time, or in keeping track of any out-of-ordinary events that the camera and algorithms might have noticed. In the below video is featured Lighthouse AI, that combines a smart security camera with such an “interactive assistant”:

In the domain of amateur (and also professional) photographer practices, AI also means many fundamental changes. There are already add-on tools like Arsenal, the “smart camera assistant”, which is based on the idea that manually tweaking all the complex settings of modern DSLR cameras is not that inspiring, or even necessary, for many users, and that a cloud-based intelligence could handle many challenging photography situations with better success than a fumbling regular user (see their Kickstarter video at: https://www.youtube.com/watch?v=mmfGeaBX-0Q). Such algorithms are already also being built into the cameras of flagship smartphones (see, e.g. AI-enhanced camera functionalities in Huawei Mate 10, and in Google’s Pixel 2, which use AI to produce sharper photos with better image stabilization and better optimized dynamic range). Such smartphones, like Apple’s iPhone X, typically come with a dedicated chip for AI/machine learning operations, like the “Neural Engine” of Apple. (See e.g. https://www.wired.com/story/apples-neural-engine-infuses-the-iphone-with-ai-smarts/).

Many of these developments point the way towards a future age of “computational photography”, where algorithms play as crucial role in the creation of visual representations as optics do today (see: https://en.wikipedia.org/wiki/Computational_photography). It is interesting, for example, to think about situations where photographic presentations are constructed from data derived from myriad of different kinds of optical sensors, scattered in wearable technologies and into the environment, and who will try their best to match the mood, tone or message, set by the human “creative director”, who is no longer employed as the actual camera-man/woman. It is also becoming increasingly complex to define authorship and ownership of photos, and most importantly, the privacy and related processing issues related to the visual and photographic data. – We are living interesting times…

New Touran: The Era of AI-Human Hybrids Is Here?

2016-04-01 14.58.26 Due to some practical reasons, we had to update our personal car this spring. Our previous car was a VW Touran that we got in 2011 (when it was necessary for us to fit three kids or baby seats into the back row – Touran is among those very few cars that are sold in Finland that can handle this). Volkswagen’s reputation got really badly tarnished in the pollution cheating scandal in 2015, and it took us some time to consider our options. Finally we nevertheless ended up with a Touran again – there are just too few car models that get right most of the essential specs that our family needs, in the price range we still can afford. There has been some interesting changes in how cars – or at least this particular, German car model – have evolved during the five years that have passed between spring 2011 and 2016, so here are some quick notes.

Firstly: it is a bit disappointing to notice that the fuel economy, climate control or energy consumption elements of Touran have not apparently been at the top priority of VW’s R&D efforts. The excellent TSI motor of VW group has been around for a long time already (I think 1.4 liter TSI was introduced in 2005), and it was ahead of its curve at the time; the emissions were clearly lower than comparable other petrol motors, the power efficiency was so good that VW could make family cars / MPVs (multipurpose vehicles/minivans) that could be moved with 1.4 or 1.2 liter petrol motor. Coupled with the DSG automatic, VW cars have been easy to drive and provide balanced behaviour with well designed interiors that fit need of families with flexible, separate seat arrangements that can fold completely flat when moving cargo of various kinds – all this kind of details help to explain their success. The 2016 Touran model differs from our previous 1.4 TSI model by delivering a bit more power with same cylinders (150 vs. 140 horse powers) and it also has the start/stop system that automatically switches off the motor when the car is stopped in traffic lights, for example.

The key differences that the Touran driver notices are in the assistive, “smart systems” that have taken considerable leaps and entered regular, mainstream cars during the last five years. I am interested enough in futuristic technologies to order this Touran with most (but not all) available “smart” options; these included adaptive cruise control (ACC, utilizing a radar sensor), Front Assist, Park Assist 3.0, Side Assist Plus, Lane Assist, Emergency Assist, and the Traffic Jam Assist. There is also a Traffic Sign Regocnition system that uses camera to keep track of speed limit signs. The Discover Media infotainment system is also from a completely new era, as compared to the basic car radio we had in our 2011 Touran. This one has large capacitive touch screen, Europe-wide navigation system, support for Apple Carplay, Android Auto and MirrorLink standards for getting smartphones with their apps and services linked with the car (though I think Android Auto is not yet working here in Finland?) We also got some extra fuel economy, comfort and security services installed (DEFA WarmUp Link system, with its apps). On the other hand, the “Volkswagen Car-Net” service with its smartphone apps appears to be a work-in-progress; we have not been able to get either iOS or Android versions working at all.

After a couple of test drives, the new car does indeed seem to combine the traditional German stable driving and quality feel with a more informative and “alive” layer of new technologies. (In the mechanical parts side, though, we had a bit too stiff gas spring holding up the hood of our new Touran, so we could not get the hood closed without some expert assistance – there seems to be some holes in VW quality control.) The “operating system” of the new Touran takes considerably more effort than before to really learn and understand. There are both the mechanical controls and switches, the physical buttons in the dashboard that relate to certain key functionalities, and then the touchscreen controls that go deeper into setup of all those multiple smart assisting functions, as well as into controlling the infotainment system. (There is also Voice Control, but that is not supported in Finland/Finnish.) This car also has a multifunction steering wheel, which our old car did not have, and there are loads of more buttons now to learn in there, too. While driving, one can of course ignore most of all this new tech, and just concentrate on the essentials of traffic, but one will notice that the “Assistants” will every now and then engage with the steering, breaks or launch a warning or notice signal of some kind. If one wants to understand and make best use of all assistants, it is necessary to spend some time reading and testing to learn their abilities and also limitations. Driving in the regular country road, one could in principle use now the combination of Lane Assist to steer the car and keep it on the road, and rely on the ACC with its radars to maintain regular speed and safe distance to other road users. However, the systems will flash warning signs if you try to take your hands off the steering wheel for too long, or if you show other signs of losing control – or even consciousness: the Emergency Assist system can show the car and stop it at the side of the road warning lights flashing, if it fails to get proper response from the driver.

I think we are currently at the early stages of hybrid and symbiotic systems in everyday use, and car technology is at the forefront of this evolution. As the learning algorithms, data analytics and artificial intelligence gets better, it is clear that some things really suit better for humans to understand and decide upon, whereas the speed, sensing range and problem-solving capabilities of artificial intelligence systems suit better other kinds of challenges. Learning how to do this basic task division is the key element that a smart car (or smart house, smart environment etc.) user needs to handle first. The new Touran apparently tries to learn the driving habits of its human partner, and adapt to them – but currently I think the human is the one who needs to do more of the adapting. Smart systems are not yet that smart.

(More info, see e.g. VW pages about ACC at: http://www.volkswagen.co.uk/technology/adaptive-cruise-control-acc )