Photography and artificial intelligence

Google Clips camera
Google Clips camera (image copyright: Google).

The main media attention in applications of AI, artificial intelligence and machine learning, has been on such application areas as smart traffic, autonomous cars, recommendation algorithms, and expert systems in all kinds of professional work. There are, however, also very interesting developments taking place around photography currently.

There are multiple areas where AI is augmenting or transforming photography. One is in how the software tools that professional and amateur photographers are using are advancing. It is getting all the time easier to select complex areas in photos, for example, and apply all kinds of useful, interesting or creative effects and functions in them (see e.g. what Adobe is writing about this in: https://blogs.adobe.com/conversations/2017/10/primer-on-artificial-intelligence.html). The technical quality of photos is improving, as AI and advanced algorithmic techniques are applied in e.g. enhancing the level of detail in digital photos. Even a blurry, low-pixel file can be augmented with AI to look like a very realistic, high resolution photo of the subject (on this, see: https://petapixel.com/2017/11/01/photo-enhancement-starting-get-crazy/.

But the applications of AI do not stop there. Google and other developers are experimenting with “AI-augmented cameras” that can recognize persons and events taking place, and take action, making photos and videos at moments and topics that the AI, rather than the human photographer deemed as worthy (see, e.g. Google Clips: https://www.theverge.com/2017/10/4/16405200/google-clips-camera-ai-photos-video-hands-on-wi-fi-direct). This development can go into multiple directions. There are already smart surveillance cameras, for example, that learn to recognize the family members, and differentiate them from unknown persons entering the house, for example. Such a camera, combined with a conversant backend service, can also serve the human users in their various information needs: telling whether kids have come home in time, or in keeping track of any out-of-ordinary events that the camera and algorithms might have noticed. In the below video is featured Lighthouse AI, that combines a smart security camera with such an “interactive assistant”:

In the domain of amateur (and also professional) photographer practices, AI also means many fundamental changes. There are already add-on tools like Arsenal, the “smart camera assistant”, which is based on the idea that manually tweaking all the complex settings of modern DSLR cameras is not that inspiring, or even necessary, for many users, and that a cloud-based intelligence could handle many challenging photography situations with better success than a fumbling regular user (see their Kickstarter video at: https://www.youtube.com/watch?v=mmfGeaBX-0Q). Such algorithms are already also being built into the cameras of flagship smartphones (see, e.g. AI-enhanced camera functionalities in Huawei Mate 10, and in Google’s Pixel 2, which use AI to produce sharper photos with better image stabilization and better optimized dynamic range). Such smartphones, like Apple’s iPhone X, typically come with a dedicated chip for AI/machine learning operations, like the “Neural Engine” of Apple. (See e.g. https://www.wired.com/story/apples-neural-engine-infuses-the-iphone-with-ai-smarts/).

Many of these developments point the way towards a future age of “computational photography”, where algorithms play as crucial role in the creation of visual representations as optics do today (see: https://en.wikipedia.org/wiki/Computational_photography). It is interesting, for example, to think about situations where photographic presentations are constructed from data derived from myriad of different kinds of optical sensors, scattered in wearable technologies and into the environment, and who will try their best to match the mood, tone or message, set by the human “creative director”, who is no longer employed as the actual camera-man/woman. It is also becoming increasingly complex to define authorship and ownership of photos, and most importantly, the privacy and related processing issues related to the visual and photographic data. – We are living interesting times…

Tech Tips for New Students

Working cross-platform
Going cross-platform: same text accessed via various versions of MS Word and Dropbox in Surface Pro 4, iPad Mini (with Zagg slim book keyboard case), Toshiba Chromebook 2, and iPhone 6 Plus, in the front.

There are many useful practices and tools that can be recommended for new university students; many good study practices are pretty universal, but then there are also elements that relate to what one studies, where one studies – to the institutional or disciplinary frames of academic work. A student that works on a degree in theoretical physics, electronics engineering, organic chemistry, history of the Middle Ages, Japanese language or business administration, for example, all will probably have elements in their studies that are unique to their fields. I will here focus on some simple technicalities should be useful for many students in the humanities, social sciences or digital media studies related fields, as well as for those in our own, Internet and Game Studies degree program.

There are study practices that belong to the daily organisation of work, to the tools, the services and software that one will use, for example. My focus here is on the digital tools and technology that I have found useful – even essential – for today’s university studies, but that does not mean I would downplay the importance of non-digital, informal and more traditional ways of doing things. The ways of taking notes in lectures and seminars is one thing, for example. For many people the use of pen or pencil on paper is absolutely essential, and they are most effective when using their hands in drawing and writing physically to the paper. Also, rather than just participating in online discussion fora, having really good, traditional discussions in the campus café or bar with the fellow students are important in quite many ways. But taken that, there are also some other tools and environments that are worth considering.

It used to be that computers were boxy things that were used in university’s PC classes (apart from terminals, used to access the mainframes). Today, the information and communication technology landscape has greatly changed. Most students carry in their pockets smartphones that are much more capable devices than the mainframes of the past. Also, the operating systems do not matter as much as they did only a few years ago. It used to be a major choice whether one went and joined the camp of Windows (Microsoft-empowered PC computers), that of Apple Macintosh computers, those with Linux, or some other, more obscure camp. The capabilities and software available for each environment were different. Today, it is perfectly possible to access same tools, software or services with all major operating environments. Thus, there is more freedom of choice.

The basic functions most of us in academia probably need daily include reading, writing, communicating/collaborating, research, data collecting, scheduling and other work organisation tasks and use of the related tools. It is an interesting situation that most of these tasks can be achieved already with the mobile device many of us carry with us all the time. A smartphone of iOS or Android kind can be combined with an external Bluetooth keyboard and used for taking notes in the lectures, accessing online reading materials, for using cloud services and most other necessary tasks. In addition, smartphone is of course an effective tool for communication, with its apps for instant messaging, video or voice conferencing. The cameraphone capabilities can be used for taking visual notes, or for scanning one’s physical notes with their mindmaps, drawings and handwriting into digital format. The benefit of that kind of hybrid strategy is it allows taking advantage both of the supreme tactile qualities of physical pen and paper, while also allowing the organisation of scanned materials into digital folders, possibly even in full-text searchable format.

The best tools for this basic task of note taking and organisation are Evernote and MS OneNote. OneNote is the more fully featured one – and more complex – of these two, and allows one to create multiple notebooks, each with several different sections and pages that can include text, images, lists and many other kinds of items. Taking some time to learn how to use OneNote effectively to organise multiple materials is definitely worth it. There are also OneNote plugins for most internet browsers, allowing one to capture materials quickly while surfing various sites.

MS OneNote
MS OneNote, Microsoft tutorial materials.

Evernote is more simple and straightforward tool, and this is perhaps exactly why many prefer it. Saving and searching materials in Evernote is very quick, and it has excellent integration to mobile. OneNote is particularly strong if one invests to Microsoft Surface Pro 4 (or Surface Book), which have a Surface Pen that is a great note taking tool, and allows one to quickly capture materials from a browser window, writing on top of web pages, etc. On the other hand, if one is using an Apple iPhone, iPad or Android phone or tablet, Evernote has characteristics that shine there. On Samsung Note devices with “S Pen” one can take screenshots and make handwritten notes in mostly similar manner than one can do with the MS Surface Pen in the Microsoft environment.

In addition to the note solution, a cloud service is one of the bedrocks of today’s academic world. Some years ago it was perfectly possible to have software or hardware crash and realize that (backups missing), all that important work is now gone. Cloud services have their question marks regarding privacy and security, but for most users the benefits are overwhelming. A tool like Dropbox will silently work in the background and make sure that the most recent versions of all files are always backed up. A file that is in the cloud can also be shared with other users, and some services have expanded into real-time collaboration environments where multiple people can discuss and work together on shared documents. This is especially strong in Google Drive and Google Docs, which includes simplified versions of familiar office tools: text editor, spreadsheet, and presentation programs (cf. classic versions of Microsoft Office: Word, Excel, and PowerPoint; LibreOffice has similar, free, open-source versions). Microsoft cloud service, Office 365 is currently provided for our university’s students and staff as the default environment free of charge, and it includes the OneDrive storage service as well as Outlook email system, and access to both desktop as well as cloud-hosted versions of Office applications – Word Online, Excel Online, PowerPoint Online, and OneNote Online. Apple has their own iCloud system, with Mac office tools (Pages, Numbers, and Keynote) also can be operated in browser, as iCloud versions. All major productivity tools have also iOS and Android mobile app versions of their core functionalities available. It is also possible to save, for example, MS Office documents into the MS OneCloud, or into Dropbox – a seamless synchronization with multiple devices and operating systems is an excellent thing, as it makes possible to start writing on desktop computer, continue with a mobile device, and then finish things up with a laptop computer, for example.

Microsoft Windows, Apple OS X (Macintosh computers) and Linux have a longer history, but I recommend students also having a look at Google’s Chrome OS and Chromebook devices. They are generally cheaper, and provide reliable and very easy to maintain environment that can be used for perhaps 80 % or 90 % of the basic academic tasks. Chromebooks work really well with Google Drive and Google Docs, but principally any service that be accessed as a browser-based, cloud version also works in Chromebooks. It is possible, for example, to create documents in Word or PowerPoint Online, and save them into OneDrive or Dropbox so that they will sync with the other personal computers and mobile devices one might be using. There is a development project at Google to make it possible to run Android mobile applications in Chrome OS devices, which means that the next generation of Chromebooks (which will all most likely support touchscreens) will be even more attractive than today’s versions.

For planning, teamwork, task deadlines and calendar sharing, there are multiple tools available that range from MS Outlook to Google Calendar. I have found that sharing of calendars generally works easier with the Google system, while Outlook allows deeper integration into organisation’s personnel databases etc. It is really good idea to plan and break down all key course work into manageable parts and set milestones (interim deadlines) for them. This can be achieved with careful use of calendars, where one can mark down the hours that are required for personal, as well as teamwork, in addition to lectures, seminars and exercise classes your timetable might include. That way, not all crucial jobs are packed next to the end of term or period deadlines. I personally use a combination of several Google Calendars (the core one synced with the official UTA Outlook calendar) and Wunderlist to-do list app/service. There are also several dedicated project management tools (Asana, Trello, etc.), but mostly you can work the tasks with basic tools like Google Docs, Sheets (Word, Excel) and then break down the tasks and milestones into the calendar you share with your team. Communications are also essential, and apart from email, people today generally utilize Facebook (Messenger, Groups, Pages), Skype, WhatsApp, Google+/Hangouts, Twitter, Instagram and similar social media tools. One of the key skills in this area is to create multiple filter settings or more fine-grained sharing settings (possibly even different accounts and profiles) for professional and private purposes. The intermixing of personal, study related and various commercial dimensions is almost inevitable in these services, which is why some people try to avoid social media altogether. Wisely used, these services can be nevertheless immensely useful in many ways.

All those tools and services require accounts and login details that are easily rather unsafe, by e.g. our tendency to recycle same or very similar passwords. Please do not do that – there will inevitably be a hacking incident or some other issue with some of those services, and that will lead you into trouble in all the others, too. There are various rules-based ways of generating complex passwords for different services, and I recommend using two-factor authentication always when it is available. This is a system where typically a separate mobile app or text messages act as a backup security measure whenever the service is accessed from a new device or location. Life is also much easier using a password manager like LastPass or 1Password, where one only needs to remember the master password – the service will remember the other, complex and automatically generated passwords for you. In several contemporary systems, there are also face recognition (Windows 10 Hello), fingerprint authentication or iris recognition technologies that are designed to provide a further layer of protection at the hardware level. The operating systems are also getting better in protecting against computer viruses, even without a dedicated anti-virus software. There are multiple scams and social engineering hacks in the connected, online world that even the most sophisticated anti-virus tools cannot protect you against.

Finally, a reference database is an important part of any study project. While it is certainly possible to have a physical shoebox full of index cards, filled with quotes, notes and bibliographic details of journal articles, conference papers and book chapters, it is not the most efficient way of doing things. There are comprehensive reference database management services like RefWorks (supported by UTA) and EndNote that are good for this job. I personally like Zotero, which exists both as cloud/browser-based service in Zotero.org, but most importantly allows quick capture of full reference details through browser plugins, and then inserting references in all standard formats into course papers and thesis works, in simple copy-paste style. There can also be set up shared, topics based bibliographic databases, managed by teams in Zotero.org – an example is Zotero version of DigiPlay bibliography (created by Jason Rutter, and converted by Jesper Juul): https://www.zotero.org/groups/digiplay .

As a final note, regardless of the actual tools one uses, it is the systematic and innovative application of those that really sets excellent study practices apart. Even the most cutting edge tools do not automate the research and learning – this is something that needs to be done by yourself, and in your individual style. There are also other solutions, that have not been explored in this short note, that might suit your style. Scrivener, for example, is a more comprehensive “writing studio”, where one can collect snippets of research, order fragments and create structure in more flexible manner than is possible than in e.g. MS Word (even while its Outline View is too underused). The landscape of digital, physical, social and creative opportunities is all the time expanding and changing – if you have suggestions for additions to this topic, please feel free to make those below in the comments.

All-in-one: still not there

HP-elite-x2
HP Elite X2 1012 press photo (image © HP).
Some time ago, I blogged about tablets as productivity devices, and then I also have written about some early experiences as a user of Microsoft Surface Pro 4: a Windows 10, 2-in-one tablet PC that relies on combination of touch screen, pen computing, and keyboard and touchpad cover (plus Cortana voice assistant, if you are a US/English user). It just might be that I am restless and curious by nature, but these days I find myself jumping from Microsoft to Apple to Google ecosystems, and not really finding what I am looking for from any of them.

When I am using an iOS or Android tablet, the file management is usually a mess, external keyboard and mouse inputs are not working reliably, and multitasking between several apps and services, copy-pasting or otherwise sharing information between them all is a pain.

When I am on a regular Windows laptop or PC, keyboard and mouse/touchpad usually are just fine, and file management, multitasking and copy-pasting work fine. Touch screen inputs and the ease of use lag behind tablet systems, though. (This is true also to the Apple OS X desktop environment, but I have pretty much given up the use of Macs for my work these days, I just could not configure the system to work and behave in the ways I want – as a Microsoft OS/PC user who has hacked his way around DOS, then Windows 3.0 etc., and thus has certain things pretty much “hard-wired” in the way I work.)

Surface Pro 4 is the most optimal, almost “all-in-one” system I have found so far, but I have started to increasingly dislike its keyboard cover. Surface Pro 4 cover is not that bad, but if you are a touch-typist, it is not perfect. There is still slight flex in the plastic construction and shallow key movement that turns me off, and produces typing errors exactly when you are in a hurry and you’d need to type fast. I am currently trying to find a way to get rid of the type cover, and instead use my favorite, Logitech K810 instead. But: I am not able to attach it to Surface Pro in solid enough way, and there is no touchpad in K810, so workflow with all those mouse right-clicks becomes rather complex.

I really like the simplicity of Chromebooks, and this blog note, for example, is written with my trusty Toshiba Chromebook 2, which has excellent, solid keyboard (though not backlighted), and a good, crisp Full HD IPS screen plus a responsive, large touchpad. However, I keep reaching out and trying to scroll the screen, which is not a touch version. (Asus Chromebook Flip would be one with a touch screen.) And there is nothing comparable to the Surface Pen, which is truly useful when one e.g. reads and makes notes to a pile of student papers in PDF/electronic formats. Also, file management in a Chrome OS is a mess, and web versions of popular apps still respond more slowly and are more limited than real desktop versions.

So, I keep on looking. Recently I tested the HP Elite X2 1012 (pictured), which is pretty identical to the Surface Pro systems that Microsoft produces, but has an excellent, metallic and solid keyboard cover, as well as other productivity oriented enhancements like the optional 4G/LTE sim card slot, USB C port with Thunderbolt technology, and a decent enough screen, pen and kickstand design. However, Elite X2 falls short in using less powerful Intel Core M series processors (Surface Pro 4 goes for regular Core i5 or i7 after the entry-level model), by being rather expensive, and according to the reviews I have read, also the battery life of Elite X2 is not something a real mobile office worker would prefere.

Maybe I can find a way to connect the Elite X2 metallic keyboard cover to the Surface Pro 4? Or maybe not.

(Edit: The battery life of Elite X2 actually appears to be good; the screen on the other hand only so-and-so.)

Blog comment spam

For a long time already, pretty much the only comments that my blog sites receive are coming from spammers. I have the setting that all new commenters’ comments go for moderation, and I manually delete them all, so they do not serve any purpose for a spammer, either. The original character of blogs as social sites of discussion have long been replaced by social network services, most notably by Facebook, Twitter and Google+. These days, the comments that a note like this one receives, take place in these various services, where original content is being linked to, “liked” and circulated. This is not a particularly good thing if you consider gathering together the various discussion threads, or would like to return to those comments at some point in the future. All those comments will be lost in the constant status update stream of social media, unfortunately. I am now seriously considering closing the comment function altogether from my blogs, and will most likely implement this change in the near future. It will be possible to send me comments via email, of course, and my preferred social network site for public discussion today is Google+ (there are links to my profile in this blog), but any comments, in any platform, are really welcome.