Cognitive engineering of mixed reality


iOS 11: user-adaptable control centre, with application and function shortcuts in the lock screen.
iOS 11: user-adaptable control centre, with application and function shortcuts in the lock screen.

In the 1970s and 1980s the concept ‘cognitive engineering’ was used in the industry labs to describe an approach trying to apply cognitive science lessons to the design and engineering fields. There were people like Donald A. Norman, who wanted to devise systems that are not only easy, or powerful, but most importantly pleasant and even fun to use.

One of the classical challenges of making technology suit humans, is that humans change and evolve, and differ greatly in motivations and abilities, while technological systems tend to stay put. Machines are created in a certain manner, and are mostly locked within the strict walls of material and functional specifications they are based on, and (if correctly manufactured) operate reliably within those parameters. Humans, however, are fallible and changeable, but also capable of learning.

In his 1986 article, Norman uses the example of a novice and experienced sailor, who greatly differ in their abilities to take the information from compass, and translate that into a desirable boat movement (through the use of tiller, and rudder). There have been significant advances in multiple industries in making increasingly clear and simple systems, that are easy to use by almost anyone, and this in turn has translated into increasingly ubiquitous or pervasive application of information and communication technologies in all areas of life. The televisions in our living rooms are computing systems (often equipped with apps of various kinds), our cars are filled with online-connected computers and assistive technologies, and in our pockets we carry powerful terminals into information, entertainment, and into the ebb and flows of social networks.

There is, however, also an alternative interpretation of what ‘cognitive engineering’ could be, in this dawning era of pervasive computing and mixed reality. Rather than only limited to engineering products that attempt to adapt to the innate operations, tendencies and limitations of human cognition and psychology, engineering systems that are actively used by large numbers of people also means designing and affecting the spaces, within which our cognitive and learning processes will then evolve, fit in, and adapt into. Cognitive engineering does not only mean designing and manufacturing certain kinds of machines, but it also translates into an impact that is made into the human element of this dialogical relationship.

Graeme Kirkpatrick (2013) has written about the ‘streamlined self’ of the gamer. There are social theorists who argue that living in a society based on computers and information networks produces new difficulties for people. Social, cultural, technological and economic transitions linked with the life in late modern, capitalist societies involve movements from projects to new projects, and associated necessity for constant re-training. There is necessarily no “connecting theme” in life, or even sense of personal progression. Following Boltanski and Chiapello (2005), Kirkpatrick analyses the subjective condition where life in contradiction – between exigency of adaptation and demand for authenticity – means that the rational course in this kind of systemic reality is to “focus on playing the game well today”. As Kirkpatrick writes, “Playing well means maintaining popularity levels on Facebook, or establishing new connections on LinkedIn, while being no less intensely focused on the details of the project I am currently engaged in. It is permissible to enjoy the work but necessary to appear to be enjoying it and to share this feeling with other involved parties. That is the key to success in the game.” (Kirkpatrick 2013, 25.)

One of the key theoretical trajectories of cognitive science has been focused on what has been called “distributed cognition”: our thinking is not only situated within our individual brains, but it is in complex and important ways also embodied and situated within our environments, and our artefacts, in social, cultural and technological means. Gaming is one example of an activity where people can be witnessed to construct a sense of self and its functional parameters out of resources that they are familiar with, and which they can freely exploit and explore in their everyday lives. Such technologically framed play is also increasingly common in working life, and our schools can similarly be approached as complex, designed and evolving systems that are constituted by institutions, (implicit, as well as explicit) social rules and several layers of historically sedimented technologies.

Beyond all hype of new commercial technologies related to virtual reality, augmented reality and mixed reality technologies of various kinds, lies the fact that we have always already lived in complex substrate of mixed realities: a mixture of ideas, values, myths and concepts of various kinds, that are intermixed and communicated within different physical and immaterial expressive forms and media. Cognitive engineering of mixed reality in this, more comprehensive sense, involves involvement in dialogical cycles of design, analysis and interpretation, where practices of adaptation and adoption of technology are also forming the shapes these technologies are realized within. Within the context of game studies, Kirkpatrick (2013, 27) formulates this as follows: “What we see here, then, is an interplay between the social imaginary of the networked society, with its distinctive limitations, and the development of gaming as a practice partly in response to those limitations. […] Ironically, gaming practices are a key driver for the development of the very situation that produces the need for recuperation.” There are multiple other areas of technology-intertwined lives where similar double bind relationships are currently surfacing: in social use of mobile media, in organisational ICT, in so-called smart homes, and smart traffic design and user culture processes. – A summary? We live in interesting times.

– Boltanski, Luc, ja Eve Chiapello (2005) The New Spirit of Capitalism. London & New York: Verso.
– Kirkpatrick, Graeme (2013) Computer Games and the Social Imaginary. Cambridge: Polity.
– Norman, Donald A. (1986) Cognitive engineering. User Centered System Design31(61).

Paljon pelissä -loppuseminaari 10.10.2017

Lämpimästi tervetuloa projektin Paljon pelissä -loppuseminaariin!

EHYT ry:n aikuisten rahapelihaittoja ehkäisevä Arpa-projekti järjestää projektin tuloksista ja rahapelihaittojen ehkäisyn ajankohtaisista aiheista Paljon pelissä -loppuseminaarin. Tilaisuus järjestetään tiistaina 10.10.2017 klo 12-16 G Livelabissa osoitteessa Yrjönkatu 3, 00120 Helsinki. Tila on esteetön.

Ilmoittaudu mukaan loppuseminaariin 25.9. mennessä täällä:


12:00 Tervetuloa, Kari Vuorinen, talous- ja hallintojohtaja, EHYT ry

12:15 Rahapelihaittojen ehkäisy Suomessa – mahdollisuudet ja haasteet, Saini Mustalampi, johtava asiantuntija, Terveyden ja hyvinvoinnin laitos

12:50 Arpa-projektin tulokset, Tapio Jaakkola, Arpa-projektin projektipäällikkö, EHYT ry

13:20 Kokemuksia rahapelihaittojen ehkäisystä työpaikalla, Milena Kaihari-Virtanen, työsuojeluvaltuutettu, Lempäälä

13:40 Kokemuksia riskirahapelaamisen puheeksiottopilotista

14:00 Kahvitauko

14:30 Onko rahapelihaittojen ehkäisy Suomessa kunnossa? Janne Nikkinen, sosiaalietiikan dosentti, Helsingin yliopisto

14:50 Keskustelupaneeli: Miten aikuisten rahapelihaittoja pitäisi jatkossa ehkäistä?

Pekka Ilmivalta, vastuullisuus- ja henkilöstöjohtaja, Veikkaus Oy

Susanna Raisamo, tutkimuspäällikkö, Terveyden ja hyvinvoinnin laitos

Mari Pajula, yksikön päällikkö, Peluuri

Tapio Jaakkola, Arpa-projektin projektipäällikkö, EHYT ry

15:45 Loppupuheenvuoro, Kari Vuorinen, talous- ja hallintojohtaja, EHYT ry

Tilaisuus streamataan EHYT ry:n YouTube-kanavalla ja tilaisuudesta jää kanavalle myös tallenne.

Loppuseminaarin keskusteluihin voi osallistua sosiaalisessa mediassa käyttämällä tunnisteita #arpaprojekti tai #paljonpelissä.

Arpa-projekti tuotti tietoa ja välineitä rahapelaamisen käsittelyyn. Projekti kehitti osaamista rahapelihaittojen tunnnistamisesta, puheeksiottamisesta ja ehkäisemisestä sekä tuki pelaajia oman pelaamisen hallinnassa. Kohderyhmiä olivat aikuiset rahapelaajat sekä ehkäisevää päihdetyötä tekevät ammattilaiset. Projekti toteutettiin vuosina 2015-2017 sosiaali- ja terveysministeriön tukemana Veikkauksen tuotoilla.

Lisätietoja projektista:

Lisätietoja tilaisuudesta saa projektiasiantuntija Salla Karjalaiselta ( tai p. 050 514 7658).

Viestiä saa levittää eteenpäin.

Drone from China:

Some experiences from international trade: in late June, I ordered a “drone” – a remote controlled quadcopter – from Chinese seller The drone in question was MJX Bugs 2 model, with GPS, 1080P camera, altitude hold, and other nice features, and GeekBying was advertising the best price.

GeekBying changed the delivery company from TNT that I had asked to DHL, but I finally got the drone, at 10th July. It appeared to be a fine little device and worked fine – for two minutes. Then it run out of battery, and a key problem emerged: the battery did not charge with the provided charger. The drone remained dead.

I contacted GeekBuying and their “After Sales Service”, and they responded by asking photo or video evidence of the problem. I made a video where I showed how connecting the charger to the battery does nothing. There was a wait (of ten days) after which they said that they had “contacted the manufacturer” and that are convinced that this is a battery problem. However, a battery is small and “easy to lose during the way” so they wanted me to make another order, where the replacement battery could be combined. This sounded a bit odd. I said that thank you, but I am not interested to order something else at the moment, but I would appreciate if they could just send me the replacement battery.

Another long wait. Finally, in 11th August, I got another small package from China, with the replacement battery Geekbuying had sent me. There is a photo below, showing the original Bugs 2 battery, and the “replacement”.

Batteries: original Bugs 2 battery, and the Geekbuying "replacement part"
Batteries: original Bugs 2 battery, and the’s “replacement part”. Make a guess, which is which?

I mailed the GeekBuying After Sales Service again, explaining that the replacement battery was a completely wrong one, and that they had made a mistake. I did send them photos of both batteries, side by side, and explained that the replacement battery was of wrong capacity (750 mAh vs. 1800 mAh of the original), and that it was also of wrong shape, as the original Bugs 2 battery is specially designed to lock into the battery compartment of the drone. The whole deal was starting to smell fishy, and I asked for instructions to return the drone, and get a refund.

GeekBuying responded by email “We are sorry for that the battery is not original, as there is no original battery in manufacturer. We confirmed it and the battery can work on this drone as well, pls try it first.”

I checked their website, and they actually  were themselves advertising the original, 1800 mAh capacity Bugs 2 battery to be sold as a spare part (link here). In my response, I explained this, and said that I am not willing to “try” using a drone with a battery that is not designed for it: even while with a right voltage and connector, the drone might operate for a couple of minutes, this small battery does not lock into the Bugs 2 battery compartment. It would be dangerous to fly a drone with it, as the battery might just disconnect, and the drone could drop on something – or someone. I also considered it fraudulent practice to mail me a wrong battery, and claim that the manufacturer has no suitable battery, as they themselves openly advertise and sell the correct, original battery.

At this point I escalated the issue in into a claim. I had used PayPal in online shopping, because they advertise certain level of buyer protection.

Even after this, the only responses I got from were emails asking me to use the wrong, small battery, and send them some videos showing how it is operating. Even a single look at the photo (above) would be enough to point out that this makes no sense.

I thought that most obvious rotten practices would had been rooted out from online shopping – at least with big online stores, but this experience at least suggest otherwise. GeekBuying as a seller has been trying me to make further orders, so that it would make better financial sense for them to post the replacement battery to the faulty product they had sold and shipped. And, as I refused to make further orders, they deliberately posted a wrongly designed, smaller battery as a replacement – something that might even put the persons using the wrong battery while flying a drone into physical danger.

It will be interesting to see if I will get any refund from the drone, in the end. There is the added complication that products with lithium ion batteries can usually be shipped from China, as they come in cargo planes. But – as the kind lady in local post office today explained to me – an individual might have trouble shipping them back, due to the tighter safety regulations of regular airmail. I tried disabling the batteries (using sticky tape) and got the drone and both batteries submitted as a post package back into seller in China, but if the delivery company refuses to carry them, then I will not get a “confirm receipt of the merchandise” from GeekBuying, and it is unclear if PayPal will cover me, in that case. Also, even while PayPal advertises “Refunded Returns”, with free shipping worldwide, the actual claim notice I got from them says that I am personally responsible for all shipping costs.

At this point: the “cheap price” I got from has grown quite a bit:

  • drone price: 106,81 euros
  • shipping (from China): 22,43 euros
  • Finnish customs & DHL service fee: 31.04 euros
  • return shipping fee: 43,00 euros
  • total = 203,28 euros.

And: all the used time, energy and peace of mind for all of this? Priceless?

Edit: finally in October, after initially failing to verify that I had indeed returned the drone to the seller, PayPal in the end (after me resubmitting the claim with photographic evidence) concluded that yes, there was indeed faulty product and wrong replacement battery, and that I had returned it to the seller, and they returned me the drone price. I had lost all the other costs, and all time and energy required.

%d bloggers like this: