Explorations into the edges of human

Robots and genetical engineering were dominant topics at this year’s ars electronica, entitled human nature. “So, nothing new…” you might think disappointedly, considering that the latest developments were broadly discussed in their own domains already. But then, this is only the first view. On the second, it appeared that “the arts” (as seen in Linz) weren’t surprised by what today’s science makes possible, either. Some artists added scientific laboratories, complete with staff and researchers, to their toolbox where the general public might still expect brushes and pencils.

Next generation of bio toys?

Next generation of bio toys?

Biotechnological Palettes

The best and most outstanding example for this is Eduardo Kac, this year’s winner of the Golden Nica in the (never more applicable) category of Hybrid Art. Under the cryptic title The Natural History of the Enigma, he had a part of his genome combined biotechnologically with a regular petunia flower. This plant now shows fine red veins in its otherwise pink face (that the upper/inner part of a blossom is called a “face” appears as a helpful coincidence for Kac). It was also Kac who had the first “glow in the dark” bunny produced in 2000, which had fluorescent fur due to flyfish genes smuggled into its DNA.

In his talk, Kac put special emphasis on the fact that the extracted part of his genome usually is responsible for detecting alien material in human blood. So, not only was part of “his blood” now making the flower’s “blood” transportation system visible, it also sneaked into the plant as an alien (with a little help from the biotechnological researchers). The result was then defined as a new life form called “plantimal”, and this particular member baptized (not without wink, as it seems) “Edunia”.

There were a lot of finely considered details, which all together make clear that the artist didn’t want to show (only) what is technologically feasible today. He merely used the potential of today’s technology, which also becomes more and more an everyday procedure, to pursue his aesthetic goals.

This was made even more obvious (or compelling), as this year’s ars electronica gave each prize winner’s talk an accompanying lecture from a “real” scientist. Josef Penninger (Director of the Institute of Molecular Biotechnology, Austrian Academy of the Sciences) explained his work on so called knock-out mice and how he was struggling to find the genetical causes of arthritis.

From both talks resulted the strong impression that the genome is just a set of bricks, and that you can design any property or appearance of a creature by the right combination of these biobricks. The audience put this into question and Penninger also conceded that all of this is less stable than we might think: “motherly love [can even] change the genome”. Still, this remark appears more like a side note. For this piece, the deliberate expression of the artist in his final work was described as central, and less the initiation of processes one can’t quite control or (yet) fully understand.


You can even find a S(ecurity level) 1 laboratory as part of the permanent exhibition in the basement of the new ars electronica center.

Robot in the mirror/Uncanny Robots

Gigantic metal monsters, stampeding over planet Earth, that’s a well known and sort of old skool techo-apocalypse. On the one hand, these monsters are already available on the market (less gigantic but at least as leathal as you might expect). On the other, research as well as the arts are often more interested in finding acceptable counterparts for humans, Sociable Robots as the MIT dubs them.

The Geminoid by Hiroshi Ishiguro (from the JST Erato Asada Project), this year’s featured artist, is an exact copy of the body of the artist in the form of a motor driven puppet. The Geminoid is not a robot (or Android) in the classic sense, because it has almost no sensors, world perception or decision making circuits, it can’t even walk. It is controlled by an external, remote operator.

The artist’s goal is to form a puppet that serves as a credible stand-in, e.g. in a discussion at a table, providing a perfect form of telepresence. Showing a certain amount of small, involuntary movements, as it is typical for humans, is among his strategies to bridge The Uncanny Valley, his “ultimate benchmark”, as Ishiguro put it himself. And while you couldn’t tell who is who on a photo, the puppet’s movements are still too slow and uneven to be accepted as humanlike. The ultimate uncanny feeling caught me (in the ars electronica center’s exhibition) when I touched the puppet, feeling the half-soft, half-rubberlike skin, not cold but also not at body temperature.

Ishiguro also reported that he wants to send his Geminoid to “give” his lectures at the Osaka University. It would be still him who talked and he doesn’t expect his students engaging him in fierce discussions, anyway. The university declined his wish so far, and it appeared pretty much as if this caught Ishiguro by surprise.

The artist is present (through the Geminoid)

The artist is present (through the Geminoid)

While most of us will smirk about this anecdote, this really comes to the central point of these efforts: Why do we think we need a “real” person to give a lecture? And what qualifies a “really present” person over a remote controlled puppet that performs all necessary tasks, one that might even be undistinguishable? Which then extends the question to how we could tell apart human and puppet, anyways (especially in everyday life where we usually don’t pay so much attention)?

Additionally to what you could see in the exhibition, Ishiguro is also looking into self-controlled robots. And because it turned out to be very complicated to program every possible move into a machine beforehand, his CB2 starts out as a “baby”. Just as human babies, CB2 starts out with very little knowledge about his motor capabilities and how to use them. It has to “learn” everything, by trial and error, by repetition, with external assistance (the “mother”). While it is entirely grey and has a far fainter visual relationship to the human body than the Geminoid, this mimicing of a central human behaviour leaves you with uncanny feelings, just as well.

Just as a human baby, this robot can’t stand up in the beginning. It needs to learn it by combining random movements, remembering previous successful efforts, and by following its (up to now human) teachers. In this context, Ishiguro also pointed out that human brains are more powerful than supercomputers, but operate on a considerably higher level of noise (i.e. not everything computes logically correctly). He speculates that this noise might be particularly key to the human brain’s learning capabilities.

Robot research has become more human, obviously. Not so much or not only in trying to copy humans, but in arriving in the same research areas as anthropologists, cognitive scientists, and brain researchers. And, besides all nerdiness that surrounded Ishiguro, this is also his declared goal: Building robots to learn more about humans.

Social Conditions

To me, the Digital Communities category always has been one of the wonderful aspects of ars electronica. This year, a whole conference day was dedicated to the topic of Cloud Intelligence. Unfortunately, the Nica winners from Wikileaks were not part of the panels, even though they provide a very important service for intelligent societies, transparency.

The first part of the Cloud Intelligence Symposium looked at online communities from a scientific or meta level. Ethan Zuckerman (Global Voices) set out to talk about mapping online communication but ended up with the Digital Divide.

Surprisingly, he started with stories about the Marshall Islands that barely rise more than four meters above sea level. That means you can’t go from one island to the next on sight. Old maps used by indigenous people therefore depicted certain distortions in the rhythms of the ocean waves, which are caused by the islands, and can thus guide experienced navigators.

Zuckerman used this as an explanation on how communication mapping can work: not observing what is there (infrastructure), but what happens (emergence). Apparently and to little surprise, the USA, Europe, Japan and south-east Asia all were bustling places, and they are also wealthy regions. Some other countries also were in the bloggers’ focus, the ones which were devastated by military conflicts.

World map distorted by the number of cell phones in use - by Worldmapper

World map distorted by the number of cell phones in use – by Worldmapper

This approach surely provides better results on the “intelligence potential” than just counting registered users or the bandwidth installed in fibre cables. But looking at the installed or rather mostly missing high-speed infrastructure e.g. in Africa can also tell you that there haven’t been huge efforts so far to connect these parts of the world. On the other hand, and this might turn it into a hen-egg problem, it might have been due to a lack of demand from a wider audience which then kept the infrastructure suppliers from building. Speaking out loud what you think has also less of a tradition in these countries, most of which had or still suffer from authoritarian regimes.

One of Zuckerman’s findings was also that most of the communication, interlinking between blogs, or facebook friendships happen on a domestic scale. “Flocking with the same” is obviously an anthropological constant which stays true in a (technologically) globally networked world. So even internet infrastructure tells you something about “human nature”.

Transcending human imagination

Besides high-tech and deeply researched artefacts, you could also find the very calm ones that aren’t less thought provoking. Perfect example is the machine with gears and concrete by Arthur Ganson: … While you can see that it is moving at its “origin” (motor), after 12 gears of reduction, no movement is perceivable at the other end. We can calculate the movement because we know the mechanics. But also this will just give us some numbers that we can not relate with on a human scale. In fact, the final gear will make a full turn in a trillion years or so which is why Ganson can “savely” attach it firmely into concrete. Quite an interesting link of mechanics and philosophy…

Machine with Gears and Concrete

Machine with Gears and Concrete


Next Generation Social Networking?

On this year’s Microsoft Research Faculty Summit, Liz Lawley complained that

Many social-networking sites essentially force users to become part of a huge community, or they force users to choose whether someone else is a friend or not, with no other subtleties defining that relationship

Of course, this direction fits perfectly to my thesis. But more specifically, I get the impression that more “subtleties” are nice and essential but also require a lot of effort by the user. Maybe similiar to metadata that was/is supposed to establish a “semantic web” but needs very simple interfaces to come to real use (delicious’ tag auto complete might serve as a good example). But while “bad” tagging might just mess up your knowledge base, getting the subtle interpersonal relations adjusted wrongly will get you in deeper trouble with your friends (light friends/good friends/best friends).

via experientia via macworld

[this is just a fast article that will be extended later on, hopefully]


Buddyguard on Stage

buddyshow teaser

Finally, my studies at FH Potsdam come to an end. I will give the presentation of my Master’s Thesis and projects on

Tuesday, March 25th, 2008 at 15 h
in the FH Potsdam Casino.

It has been a tough time untill my book went into press and I’m still quite busy preparing a decent show for you. But, hopefully, you will enjoy it and I will succeed in gaining a proud and honourful Master’s degree.

Buddyguard is helping me with making up a proper guestlist. But you are invited now already, as a reader of my blog!


Scanning for Buddies


A public version of my home grown “buddyscanner” is now available! It is a visualisation tool that I built in order to analyse communication log files of my group of test persons. This data can be usually found as a part of your phonebill or it can be extracted out of email archives.

You can give it a try right away: Start the buddyscanner

Of course, the visualisation is not too meaningful until you use your own data. But with this anonymized version you can get an impression of how it looks and works. (If you would like to have a private visualisation (where private means your data and a safe, password protected place) just let me know: blog [at] emotisys.net)

The visualisation can be rearranged to reflect different aspects of the data. It offers items that can be found in the raw data directly (such as the overall duration of communication), as well as computed values like reciprocity. The final value (from the perspective of my thesis), relevance, is available, too. Relevance is similiar to a kind of “rank” or “importance” of that person as it is seen by the machine. Although I’m using rather simple scoring methods, the results were quite meaningful to my test persons, already.

Some additional explanations:

  • In order to rearrange the diagram you need to click into the select boxes at the end of each axis. There is a third box available that is used for the “third” dimension, which is mapped onto the size of each square.
  • Hovering over a data point will load a flyout window with a more fine grained diagram. To keep it opened, you can click onto the according square.
  • In the flyout, a bar for each call/mail is displayed at the day of the year when it took place. The height is related to the duration/size of the event. Light blue means it occured during (usual) work times, dark blue is for the evening and medium for the weekends.
  • You can make some remarks for other users in the comments field if you like to.
  • If you want to keep track of some points across differnt sortings, you can highlight them with the button at the bottom of the flyout.

If you want to see more, express your doubts or have some remarks, don’t hesitate to make a statement below!


buddies and business

L'épicerie in Lyon

The more I get into relation detection via communication data, the more services come to my mind. But of course, I don’t invent this wheel for the first time (Pete Warden’s blog brought a lot of evidence to me): In an article two years from now (already!) ZDnet UK has a nice portrait about the emerging business of email analysis. A positive focus is put on Clearwell Systems because of their special (unique?) ranking algorithm (oha! — I bet Google pays very close attention). Its software

weighs the background data and content of each email for several factors, including the name of the sender, names of recipients, how many replies the message generated, who replied, how quickly replies came, how many times it was forwarded, attachments and, of course, keywords.

Well, so do I… But in the light of a fully grown business, ranking emails gets away from a personal (autonomous) assistant that is just nice to have, handy and good for reflection. With the huge amounts of email produced every day and about every topic relevant to any business process, corporate email archives contain pretty any information a manager, and — more delicately — a prosecutor can desire:

Email has come to be viewed as a source of truth. If you want to know what really happened, you look at the email.

As it became clear to me, too, during my research, collecting and archiving (intercepting?) all electronic conversations improves the the basis for statistical analysis and heuristics and hence the quality of the ranking a lot. A lot of entities (Google, security authorities) are after our data, consequentially.

Pete Warden has to receive an honrable mention once more because his position of “trying to generate a useful index with no human intervention” resonates with my basic motivation, too. I find his blog to be imensly interesting and very relevant for my thesis: Like expoiting the time information inherent to email that I thought of using in some kind of “contact profiling”, all the privacy issues entangled, especially in business context, and drawing profit from the knowledge that accumulates often unnoticed in a company (or workgroup). And he complains about the missing Gmail Api, too. All written in a very comprehensive manner.


User Testing

paper prototype

interacting with paper prototype

sketches after testing session

. .

How to pick friends

myfolks selector

Imagine, you’re back from a trip abroad and want to tell your friends about all the fascinating experiences that you have made (And you either don’t have a blog for that purpose or don’t want to publish it publicly). Usually, that means you have to go through your entire address book and select the appropriate persons. However, if your computer knew about your relationships it could help you a lot with this task.

How could an interface for this case look like? Here are some propositions (and some problems to discuss!).

. .

Autonomous Assistants reloaded

Here comes the all new and sparkling abstract of my Thesis (old stuff). You might want to have a look at it and give it some comments!

In my thesis I propose the idea of a socially aware computer. In order to get to know the user‘s circles of friends, it will mine and analyse the data that is left as traces by her communication, mainly phone call logs and email archives. As a result, a value for personal or subjective importance can be computed for each person in the user‘s network.

This allows for a new arrangement of the personal address book so that more relevant persons can be found more easily – an important feature regarding our ever expanding and globalized personal networks.
Moreover, tasks that require knowledge about the user‘s personal relations can be handled automatically: One is turning the user‘s attention towards old friends that tend to be neglected when he is burried in work or because he is always on the run due to our mobile and flexible times. Another one is managing access to her personal data that she stores online, like photos, travel plans or her activity stream that gets created by recent software like Jaiku or Twitter.

Handling friends and acquaintances in such an environment opens up new challenges that are explored by means of a visual prototype. Different types of displaying, managing, and enriching information about related persons are developped. Results from a user testing will be provided.
As a preliminary study, the data sets of several people have been analysed and plotted into an interactive diagramm in order to investigate the potentials of the communication data given. It also offers the possibility to look for the relevant parameters that determine different types of relations (e.g. best friend or old friend).

To provide a conceptual background, existing social network theories are explored and related to personal, ego-centric ones. I take a closer look onto the whole process of operationalisation, i.e. turning human behaviour into quantifiable data by statistical methods. Finally, implications and problematic consequences of both, the software itself and the concept of the „network society“ in general, are discussed. The felt need to turn our friendships into „social capital“ is one of the most remarkable shifts in the functioning of our societies. Others can make draw profits from this capital if they collect detailed data to establish profiles of us and our relationships. Thus, the whole field of privacy is entangled.
And across all these dynamics, computers become so inseparably intermingeld into our daily social life that borders between our (extended) self and the machine is often hard to determine.

. .

auto-pilot for your relationships

For a healthy relationship, you should leave a short notice for your friends at least once in a while. But — huh — somtimes, days are really packed and you tend to forget things like that anyway…

Why not let your digital companion take on some routine care taking? Computers are well versed with keep alive customs:

The “keep-alive” keyword […] allows the sender to indicate its desire for a persistent connection.

Here is how it works:
keepalive scenario
With the socially aware address book (from my Master’s Thesis) it will know who your friends are–and you will be able to describe your social aspirations, too. You then only need to define where you store interesting images or your latest writings or what you’re currently occupied with and the system will start sending off short notices every now and then.
If your friend happens to implement the same digital assistant (second line in the picture), your digital sidekicks might end up in a circle of automatic messages and reponses, chatting along on their own. Bypassing your human existence altoghther…
But your assistant can also propose some more personal messages that require your contribution (as depicted in the last line).

After all, some digital support is better than neglecting your remote friends too much, isn’t it?
(discussion declared open…)

. .

Steven’s Social Fabric

My Social Fabric as conceptual screenshotimg

The mobile phone as a truely social device makes an ideal plattform for social network visualisations. This gets demonstrated in a very inspiring way by Steven Blyth in his master thesis project at the now discontinued IVREA. As I found out he researched a question in the immediate neighbourhood of mine:

How can the softness and ambiguity of our social worlds be visualized within the computational and binary context of a mobile device?

The Social Fabric is a representation of your social world, displayed as a single visual array [of avatars] on your mobile phone. It does not replace your address book or calendar but keeps you subtly informed [via the body posture or the avatars!] about which relationships are prospering, which you have neglected, and the overall state of your social fabric.

social fabric - posturesimg

In lots of his ideas and writings I found good arguments for what I want to further investigate. A very good point is that ambiguous metaphors can avoid the impression that a computer system could be truely accurate about something that is vague by its nature: social networks. As I am following a rather number based approach at the moment, this is something I will consider (with this Paper by Thomas Erickson from IBM).

He also revived another fascination (deep inside of me and, actually, my thesis proposal) for agents and avatars. In his opinion they are not discarded by history, as one can hear often, but depend on the proper design and sometimes sophisticated technology. The more the latter flourish the more the first can emerge as useful companions.

In contrast to his work, visualisation is supposed to be only one facett of my thesis with further applications building on insights gained by them.
Something left unclear to a certain extent in his text is his profiling method, what I used to call the “long term relation records”. Especially when considering “old friends” and “family members” a good balancing between current communication behaviour and long time habbits can offer new possibilities to deal with the less active parts of our “circles of friends”.

Thanks to one of his co-students at IVREA, Myriel, for poking my nose into this work! It found some good resonance over different media: WMMNA (relates it to GORI), Wired, LIFT 07