Photography and artificial intelligence

Google Clips camera
Google Clips camera (image copyright: Google).

The main media attention in applications of AI, artificial intelligence and machine learning, has been on such application areas as smart traffic, autonomous cars, recommendation algorithms, and expert systems in all kinds of professional work. There are, however, also very interesting developments taking place around photography currently.

There are multiple areas where AI is augmenting or transforming photography. One is in how the software tools that professional and amateur photographers are using are advancing. It is getting all the time easier to select complex areas in photos, for example, and apply all kinds of useful, interesting or creative effects and functions in them (see e.g. what Adobe is writing about this in: https://blogs.adobe.com/conversations/2017/10/primer-on-artificial-intelligence.html). The technical quality of photos is improving, as AI and advanced algorithmic techniques are applied in e.g. enhancing the level of detail in digital photos. Even a blurry, low-pixel file can be augmented with AI to look like a very realistic, high resolution photo of the subject (on this, see: https://petapixel.com/2017/11/01/photo-enhancement-starting-get-crazy/.

But the applications of AI do not stop there. Google and other developers are experimenting with “AI-augmented cameras” that can recognize persons and events taking place, and take action, making photos and videos at moments and topics that the AI, rather than the human photographer deemed as worthy (see, e.g. Google Clips: https://www.theverge.com/2017/10/4/16405200/google-clips-camera-ai-photos-video-hands-on-wi-fi-direct). This development can go into multiple directions. There are already smart surveillance cameras, for example, that learn to recognize the family members, and differentiate them from unknown persons entering the house, for example. Such a camera, combined with a conversant backend service, can also serve the human users in their various information needs: telling whether kids have come home in time, or in keeping track of any out-of-ordinary events that the camera and algorithms might have noticed. In the below video is featured Lighthouse AI, that combines a smart security camera with such an “interactive assistant”:

In the domain of amateur (and also professional) photographer practices, AI also means many fundamental changes. There are already add-on tools like Arsenal, the “smart camera assistant”, which is based on the idea that manually tweaking all the complex settings of modern DSLR cameras is not that inspiring, or even necessary, for many users, and that a cloud-based intelligence could handle many challenging photography situations with better success than a fumbling regular user (see their Kickstarter video at: https://www.youtube.com/watch?v=mmfGeaBX-0Q). Such algorithms are already also being built into the cameras of flagship smartphones (see, e.g. AI-enhanced camera functionalities in Huawei Mate 10, and in Google’s Pixel 2, which use AI to produce sharper photos with better image stabilization and better optimized dynamic range). Such smartphones, like Apple’s iPhone X, typically come with a dedicated chip for AI/machine learning operations, like the “Neural Engine” of Apple. (See e.g. https://www.wired.com/story/apples-neural-engine-infuses-the-iphone-with-ai-smarts/).

Many of these developments point the way towards a future age of “computational photography”, where algorithms play as crucial role in the creation of visual representations as optics do today (see: https://en.wikipedia.org/wiki/Computational_photography). It is interesting, for example, to think about situations where photographic presentations are constructed from data derived from myriad of different kinds of optical sensors, scattered in wearable technologies and into the environment, and who will try their best to match the mood, tone or message, set by the human “creative director”, who is no longer employed as the actual camera-man/woman. It is also becoming increasingly complex to define authorship and ownership of photos, and most importantly, the privacy and related processing issues related to the visual and photographic data. – We are living interesting times…

Sony RX100: pocket, meet camera

Photography is an interesting thing – many interesting things. Take cameras, for example. For some people, cameras and lenses appear to mean perhaps more than the actual photographs they are supposed to use those equipment for. The global growth of revenue from digital camera industry continues its upwards trend, and by some estimates is expected to reach $46 billion by 2017. There are cameras for multiple uses, and the strengths of one system in one context turn into weaknesses in another. Compare DSLR “systems camera” to a cameraphone (or smartphone), for example: the versatility provided by multiple, interchangeable lenses combined to large image sensor and powerful image processing is unbeatable when the pure technical side of photography as a form of expression is being considered. On the other hand, in everyday daily lives, few people go about hauling their professional DSLR system everywhere. Having a good camera integrated into the mobile phone is your best bet to have camera at hand when the spontaneus opportunity for an interesting photo presents itself. Though, the limitations of small lens and small image sensor inevitably set its limits to what one can achieve with a smartphone camera.

I am going to experiment next by acquiring a compact, “pocket camera” that hopefully would be small enough to actually be feasible to carry around daily in my overcoat pocket, while also having better optics and more versatile feature set than a smartphone camera.

My choice (balancing budget and wish list) concluded into Sony Cyber-shot DSC-RX100 model. This is a compact camera that was introduced already in summer 2012, and there are already several more feature-rich, upgraded versions of RX100 available (Mark II, III, and now also IV, released in summer 2015). My priority here though was to focus on the essential aspects of solid optics combined with decent image sensor and build quality, and the original RX100 ranks high in that department, and the price is pretty competitive by now.

There are few things that smartphone cameras do really well, and extensive app ecosystem, strong computing power in compact form factor and excellent touch screen interfaces are among the key such elements. If the lens and sensor are priority in a compact camera, to get that high quality shot, and you are carrying a powerful smartphone also with you everywhere, it does not make sense to try to duplicate smartphone functions in the camera itself. It is enough to be able to get the photo from camera to the smartphone, and then do the post-processing and possible social media sharing, or archiving from there (or, via a cloud service and/or a PC, for that matter). RX100 does not have a built-in WiFi or other wireless functions, so I have now equipped my new Sony with the Eye-Fi Mobi Pro 32 GB SD memory card, which has the WiFi, and can connect to e.g. iPhone Eye-Fi Mobi app, where from you can take the editing and sharing business as far as you want.

I also invested to some other small add-ons: the official camera LCD screen protector (PCK-LM15) and the Sony AG-R2 Attachment Grip. The latter affects the slim, flat design of RX100 a bit, but is really good for getting reliable hold of the camera so that you can confidently work through multiple positions, without fear of dropping the camera.

RX100 is one of the most popular cameras in the relatively new “enthusiast compact” category, that I guess emerged out of Darwinian adaptation process, where mobile phones took over most of the “snapshot” market, and the compact camera manufacturers were forced to evolve and differentiate their offerings from the most basic and casual photography needs. The manual of RX100 is a rather thick volume, so it has fair number of various options and functions, and this camera has also a rather large, one-inch image sensor (of 20 megapixels), a Carl Zeiss Vario-Sonnar T lens (28-100mm equiv., f/1.8-4.9), image stabilization, automatic face recognition, customizable controls and the ability to shoot recording RAW images – something that the more professional (or nerdy) photo tweakers can value.

It is still too early to say whether the idea of having a daily pocket camera available actually makes any real sense, so that the extra 240 grams of weight in my jacket pocket really pays off. But I guess that in those conference trip breaks this would allow one to jump on and off the “tourist mode” with a bit more expressive range available than just a mobile phone camera would allow. We will see.

4K Ultra HD monitor

Samsung U28D590D
Samsung U28D590D.
Sharper is better. I just booked the last remaining unit of Samsung U28D590D, an Ultra HD, 4K monitor from the local PC store (a display unit) at nice, 320 euros price. This is probably the most budget-consious alternative in 4K, 28″ monitors you can find; there are better, IPS screens (this is a high quality TN), and particularly professional models have better ergonomic in the stand (this is a completely fixed thing, and no VESA mounting either). But the colour reproduction, brightness are excellent, and particularly having 3840 x 2160 resolution at 60Hz, with 1 ms speed (over the Display Port 1.2) makes this pretty much what I have been looking for my gaming and photo editing needs. I am also regularly plugging in several computers (PC/gaming workstation, Macbook Pro Retina, Chromebook) to the same display at my desk, and there is interesting PIP (picture-in-picture) mode in U28D590D where you can keep an eye on the second PC while simultaneously working full screen on the other (let’s see how useful this will be in reality, though). If you think there is a better deal available from somewhere at the 300 euros price range, let me know. More information: http://www.samsung.com/levant/consumer/it/monitor/uhd-monitor/LU28D590DS/ZN.

Edit: this is the thread with instructions for getting 52 Hz at 4k on the retina MacBook Pro 13: http://forums.macrumors.com/threads/4k-display-and-retina-macbook-pro-13.1741440/

HDR in iOS 4.1 – yes or not?

There has been reports around that show how you can shoot HDR (High Dynamic Range) photos with iPhone that has been updated into iOS 4.1. What is weird, since I have the update and looking into the Setting > Photos, I cannot see that option. Yet, e.g. in this video it is there – getting curious… http://www.mobiiliblogi.com/2010/09/15/videolla-iphone-4n-uusi-hdr-kuvausominaisuus/