PixelCarpet paper accepted!

A while ago, I got the opportunity to take part in a large, EU-wide research project on computer network security and data visualization. The goal of SASER is to lay the foundation for more reliable, efficient, and secure communication networks (I guess the recent revelations about the NSA infiltration might have played a role in the decision to work with substantial effort inside the EU on advanced networks).

Our part as a team of researchers at the Interaction Design Lab at FH Potsdam was to investigate data visualization, visual analysis tools, and dashboards as a support for computer security engineers. While watching traffic and server activity, they often have to sift through loads of data to filter out the suspicious traces of attacks and other malicious activity. Data visualization would certainly help them exploring data, especially bringing patterns to the light that they would have not expected (and therefore wouldn’t have looked for with their data mining tools).

We wrote a paper about our work and one of the resulting demonstrators (which we call Pixel Carpet) – it now got accepted to the IEEE VAST conference in Paris! (yeah!)

The following brief overview is taken from our team’s website, complexdatavisualized.

It builds on the observation that security engineers know their data and the requirements of their work very well. However, they might not be acquainted with advanced visualization techniques. Visualization researchers, on the other hand, know methods to visualize and analyze data but usually lack insight into the specific requirements of computer network security. The paper revolves around two main contributions:

  • results and learnings from a co-creative approach of jointly developing visualizations
  • a pixel oriented visualization technique that graphically represents multi-dimensional data sets (such as computer log files), reflecting ideas from the collaboration

You can get and read the full paper here:

Landstorfer, Herrmann, Stange, Dörk, Wettach (2014): Weaving a Carpet from Log Entries: a Network Security Visualization Built with Co-Creation. in Visual Analytics Science and Technology (VAST), 2014 IEEE Conference on, 2014 (to appear)
at 27MB or a smaller file (4 MB) without embedded video.


Co-creative Approach

User centered approaches are well known in the visualization community (although not always implemented) [D’Amico et al. 2005, Munzner et al. 2009]. Jointly developing the visualizations themselves, however, is rather rare. As we have very good experience with co-creative techniques in design and innovation, we wanted to apply them to the domain of data visualization as well. For example, we tried to experiment with data sets during a day-long workshop with a larger group of stakeholders (a session we called the “data picnic” because everyone brought his/her data and tools).


For this paper, we focused on a pixel oriented technique [Keim 2000] to fullfill requirements such as visualization of raw data or a chronological view of data to preserve the course of events. We stack graphical representations for various parameters of a log line (such as IP, user name, request or message) so that we get small columns for each log line. Lining up these stacks produces a dense visual representation with distinct patterns. This is why we call it the Pixel Carpet. Other subgroups of our research group took different approaches that can be found at other places in this blog.


Snapshot of the Pixel Carpet interface. Each “multi pixel” represents one log line, as it a appears at the bottom of the screen.

Data and Code

Our data sources included an ssh log (~13.000 lines, unpublished for privacy reasons) and an Apache (web server) access log (~145.000 lines, unpublished), and ~4.500 lines (raw data available, including countries from ip2geo .csv | .json ).

We implemented our ideas in a demonstrator in plain HTML/JavaScript (demo online – caution, will heavily stress your CPU). It helped us iterate quickly and evaluate the idea at various stages, also with new stakeholders. While the code achieves what we need, we are also aware that computing performance is rather bad. If you want to take a look or even improve it, you can find it on github.

To bring it closer to a productive tool, we would turn the Pixel Carpet into a plugin for state-of-the-art data processing engines such as ElasticSearch/Kibana or splunk (scriptable with d3.js since version 6).


Design at Linux Tag 2014

Going to a Linux conference as a designer might sound like an exotic idea on first sight. While this should receive a second thought (see later), I have to admit that I had a special approach, too, when I went to this year’s Linuxtag at Station, Berlin: I went there as part of the SaSER research team at IDL which is (on the top most level) concerned with defining new more efficient, reliable, and secure communications on the internet. So we were in the sys admin domain already (although Linux is far more than for admins but that might be what you think of).IMG_9091

Visual Analytics for Security Admins

We gave a talk on our interim results, which are visual analytics tools to investigate huge log files (i.e. text files): Opening Treasure Boxes. Exploring log files with visual data analysis to detect security breaches (slides). As part of the “Tracing and Logging” track, we talked to system administrators and security analysts and everyone else interested. It was a great opportunity for us to extend our contact with users, get feedback, acquire new use cases. We assumed that we would also “evangelize” a little in favor of visual analytics and visualizations beyond bar and line charts. We were quite right with that: our ideas were partially seen as strange and  unusual but we also received quite some thankful feedback by analysts who said our ideas opened new opportunities for them.Screen-Shot-2013-10-31-at-3.45.06-PM___elasticsearch-org

A view of the Kibana interface (image from Elastic Search)

A view of the Kibana interface (image from Elastic Search)

But I also want to point out that a couple of tools in the same track provided a very decent user interface and good visualization support. This is especially remarkable as log files are abstract things and enormously large, which makes providing a grip on them a real design challenge. Lennart Koopmann of Torch presented greylog2, with an interface to query large logfiles, get an overview over values in the file, and also get visual support for results in the form of time lines. Even more dedicated to a graphical user interface was Kibana (which builds on logstash and ElasticSearch), presented by Bernd Erk of Netways. I was impressed by the visual support for building and modifying queries, the ease of building graphs, and the clean overall interface. In many regards it reminded me of Splunk, which is also a great but not an open source software. As we found during the preparation of our talk, also the event monitoring system Icinga2 starts including interesting visualizations – Markus Frosch (of Icinga) just didn’t put a huge focus on the new interface.

Design for Open Source Software (needed!)

Coming back to the (suspected) design – Linux repulsion or even design – open source repulsion: open source software gained a bad reputation for having ugly or “just enough” user interfaces, with little help for users to find a workflow or just please the eye (things like Firefox or Fritzing are an exception, of course, but they are also rather recent offsprings). It seems as if open source is much more appealing to developers than to designers. I have no instant explanation for that – if you do, please let me know! I have to admit that a lot of the software I saw during Linuxtag unfortunately confirmed this prejudice. The more delighted I was when I saw how well crafted things like Kibana were.

It might be worth noting that Edna Kropp and Nicole Charlier of akquinet gave a basic introduction into user centered design and how they work as “on-site UX consultants“. While it was pretty basic for a designer, it was probably new and remarkable for many of the developers (hopefully) listening. I think much more talks like this are necessary to get to a common understanding between developers and designers in the open source scene.

Further notes

The bare crypto stick (it has a modest but nice casing in the final version)

The bare crypto stick (it has a modest but nice casing in the final version)

For the real paranoid cautious people, there is a Crypto Stick: looks like a thumb drive but actually hosts a micro-processor, a smartcard, and an SD card. You can use it to establish secure connections from untrusted systems (like internet cafés), store your passwords, and other things. You can even transport documents “plausibly hidden”, e.g., in case you get searched at an airport – and you don’t have to think of Snowden to understand how relevant that can be. I liked the idea to have a “security thing” that is really strong but also makes it easier for people to stay safe online/digitally.
Btw: it’s open software and open hardware, so you can build it at home (although the small form factor makes it complicated)

UDOO: Standard PC interfaces for the "Linux part" seen at the front here, with pin headers in Arduino due format at the back

UDOO: Standard PC interfaces for the “Linux part” seen at the front here, with pin headers in Arduino due format at the back

Even physical computing was a topic and the only other presentation given by an interaction designer: Michelangelo Guarise presented UDOO, which combines an Arduino Due-derived board with a Linux system running on a powerful quad-core ARM chip. This “natural” combination pops out in various flavors at the moment, combining the sensor-friendly, real-time interaction capable Arduino architecture with high-performance computing. I hope they will soon add their platform as a part to the Fritzing library and I’m curious about the projects building on that single board computer!

And I got a trusted certificate from CAcert to (soon) sign my email and ssl server connections – yeeha! I was impressed by how serious they take the process, with several people checking my ID cards separately. Trust on the internet is a delicate thing and digital signatures can help a lot here.