magnet

Mag­net is a mod­u­lare client/server soft­ware sys­tem capa­ble of visu­al­iz­ing sim­i­lar­i­ties between enti­ties in any struc­tured dataset, but in par­tic­u­lar for media libraries. Some use cases, a first pro­to­type and a usabil­ity eval­u­a­tion of it has been con­ducted as part of my com­puter sci­ence degree dis­ser­ta­tion (“Diplom Infor­matik FH”, a mas­ter equiv­a­lent). Have a look at the writ­ten part (Ger­man ver­sion only, sorry).

major aspects at a glance

• Mag­net visu­al­izes enti­ties of any kind and their sim­i­lar­ity among each other
• When launch­ing the pro­to­type, you will be asked to choose a movie. Please chose one your like and are famil­iar with.
• The data set in use is based on data from IMDb.com
• Enti­ties are ordered and posi­tioned accord­ing to their sim­i­lar­ity to each other
• The user can inter­fere the sim­i­lar­ity cal­cu­la­tion process lively by adjust­ing weight­ings of each facet/criteria, caus­ing the visu­al­iza­tion to rep­re­sent the changes instantly, intro­duc­ing the weighted multi-faceted brows­ing paradigm
• The sys­tem is kept mod­u­lar and should be usable for any kind of data, though it was intended for media libraries and social net­work applications
• An intu­itive inter­face allows users to look “behind” the data by see­ing — with­out the need of study­ing large data tables — coher­ences between data set or entities.

prob­lem, idea and solu­tion approach

unman­age­able size and diver­sity of my media collection

First ideas about mag­net came up, hav­ing my per­sonal media col­lec­tion in mind. The prob­lem, I have been fac­ing, was due to the unman­age­able size and it’s diver­sity. As, for exam­ple, I have been lis­ten­ing to an album I liked, I do not want to hear it again, but some­thing related or sim­i­lar would fit my mood. Now that I do not know all of the artists and their music styles in my col­lec­tion by heart, I will most prob­a­bly miss some good choices and maybe be unsatisfied.

This is a very typ­i­cal use case as one can find sit­u­a­tions like this every­where, e.g. search­ing new movies by telling which kind they should be, or by com­par­ing them to known movies.

• I would like to see a movie like “For­est Gump”, because Tom Hanks is great and I am in the mood for this kind of movie.
• I lis­tened to Louis Arm­strong for the first time and I really loved it, is there some­thing else like that out there? Maybe same genre, from same time or also played with trum­pet. (Depend­ing on the avail­able meta information.)
• I found a nice pic­ture of mine taken in Berlin, where else have I been at that time? Have I taken some other pic­tures of this place on other trips to Berlin?

gen­er­al­iza­tion: an inter­ac­tive data coher­ence visualization

As soon as I began to develop use-cases, it turned out to be gen­er­al­iz­able to some kind of inter­ac­tive data coher­ence visu­al­iza­tion. This find­ing has been proven by state­ments from par­tic­i­pants of the usabil­ity study (see below). Google Maps con be con­sid­ered a visu­al­iza­tion of geo­graph­i­cal meta­data of enti­ties. Mag­net is sup­posed to be a wid­get, too, that can be embed­ded as easy as Google Maps, but it is not bound to a sin­gle infor­ma­tion dimen­sion. Instead, it is capa­ble of visu­al­iz­ing any kind of rela­tion on-the-fly and is fur­ther­more adjustable to the needs of the user by him– or herself.

Using mag­net fol­lows four basic steps:

1. Pick some­thing I know, like or have inter­est in
2. Browse/review the results
3. Optional: adjust weights of cri­te­ria if the results are not suit­able yet by decid­ing which cri­te­ria are more or less impor­tant to me
4. Review changes, maybe pick one of the results, if inter­ested, and con­tinue the brows­ing expe­ri­ence from point 1.

See a short essay (in ger­man) briefly describ­ing the idea of magnet.

mod­u­lar­ity and flexibility

Major fact con­cerned by set­ting up the server side archi­tec­ture was to keep mag­net ver­sa­tile usable and integrable.

sep­a­rated server application

The archi­tec­tural choice of pro­vid­ing a stand­alone server appli­ca­tion brings three advan­tages. First of all, the appli­ca­tion to become mag­net enabled can be writ­ten in any lan­guage and it does not need more depen­den­cies. By mak­ing use of the Data Access Object pat­tern, the data pro­vid­ing appli­ca­tion can be con­nected by Web­Ser­vices or any other stan­dard like JSON (JavaScript Object Nota­tion) or AMF (Action Mes­sage For­mat) over TCP con­nec­tion. And finally the mag­net appli­ca­tion can be run eas­ily on a dif­fer­ent machine, which for sure makes sense for big data bases and/or high work­load. Future devel­op­ments should also allow mag­net to be a dis­trib­uted application.

Addi­tion­ally mag­net has been equipped with an open client inter­face. To use the visu­al­iza­tion as described below, an AMF ser­vice inter­face is pro­vided as default, but could be replaced or extended by addi­tional ser­vices for cus­tom build client applications.

The mag­net server archi­tec­ture, show­ing core (green) and the inter­faces plus data exchange objects (yellow)

The green part is the heart of the server appli­ca­tion of mag­net and can be per­ceived as black box, the yel­low parts are shared to the out­side world. To the left, there is the client inter­face, which could be imple­mented by a Web Ser­vice, AMF gate­way or directly by any kind of GUI, if mag­net is run on a local machine. The appli­ca­tion inter­face is shown on the right, allow­ing mag­net also to con­nected directly to an appli­ca­tion or request­ing any kind of ser­vice, addi­tion­ally this approach makes it pos­si­ble to use e.g. LDAP for user authen­ti­ca­tion. All com­po­nents share the same data exchange objects given in the data package.

skin­ning

The visu­al­iza­tion part as described below fea­tures most broad skin­ning fea­tures to make mag­net inte­grate seam­less into any web appli­ca­tion. The final ver­sion of a GUI should be mod­u­lar, so that the three dif­fer­ent parts can be placed as required.

visu­al­iza­tion

goals

• reduce space for con­trollers and detail view
• give more space for entity visualization
• make user find con­trollers easily
• and make her or him under­stand their func­tion with­out exten­sive explanation

Nor­mal client lay­out, for using mag­net medium or fullscreen.

Com­pact client lay­out, for using mag­net as small wid­get. Show­ing typ­i­cal read­ing direc­tion in shape of a Z, hit­ting all impor­tant GUI objects.

Min­i­mal­is­tic client lay­out, with­out info box, e.g. to use mag­net as a nav­i­ga­tional layer

My mayor goal was reduce space for con­trollers and detail view and give more space for entity visu­al­iza­tion, where the user can get a “feel­ing” of the data, he or she is deal­ing with. That is why the enti­ties are mov­ing around, pulling each other closer or push­ing away. This ani­ma­tion the user an imme­di­ate under­stand­ing of sim­i­lar and dis­sim­i­lar enti­ties as shown in the test below. By reduc­ing dis­trac­tion and putting the ele­ments in places typ­i­cally hit by the user eye by scan­ning the GUI, the user real­ized the con­trollers at the lower left very quickly, which he or she needs to ajust the weightings.

Some sum­ma­rized infor­ma­tion about an entity can be placed in the info box on the right side, but any fur­ther, detailed infor­ma­tion should be pre­sented out­side. The com­pact vari­ant can be used to make mag­net become a nav­i­ga­tional overlay.

graph visu­al­iza­tion

Col­lect­ing sim­i­lar­i­ties between enti­ties and stor­ing this infor­ma­tion ends up in a form of a graph, where each entity is con­nected to each other and the length of each edge equals the rec­i­p­ro­cal sim­i­lar­ity between the two enti­ties con­nected. This reveals two issues.

The more enti­ties con­tained in the graph, the less is the chance to find a sit­u­a­tion, where the defined edge lengths hold for all enti­ties in a two or three dimen­sional visu­al­iza­tion. There­fore I uti­lized the spring graph par­a­digm, where each edge is flex­i­ble. Addi­tion­ally, each entity gets a weight assigned which influ­ences its grav­ity, i.e. the force that tries to make every edge exactly as long as desired. The weight can be defined by the data pro­vid­ing appli­ca­tion, e.g. to make some enti­ties stand out more. As the user starts from a spe­cific entity, the weight of this entity gets mul­ti­plied with a con­stant fac­tor (ten by default), which makes the other enti­ties posi­tion them­selves around and keep the “impor­tant” entity always in focus.

A sec­ond issue is the square cor­re­la­tion of the amount of enti­ties and the edges to be cal­cu­lated. The pro­to­type imple­mented so far is there­fore restricted to 10 enti­ties at the same time. The ver­sion, I am cur­rently work­ing on, is using an mod­i­fied ver­sion of the spring graph frame­work for flex by Mark Shep­herd, which itself is based on the Touch­Graph imple­men­ta­tion for Java, instead of my own imple­men­ta­tion, hop­ing to improve per­for­mance. As a next step I will also make use of clus­ter­ing algo­rithms to reduce the num­ber of edges in the graph.

UI devel­op­ment

The first inter­face has been devel­oped for the demo appli­ca­tion. Cur­rently a sec­ond ver­sion, imple­ment­ing find­ings of usabil­ity and tar­get user group stud­ies, is about to be set up, see “inter­face improve­ments para­graph” below for details.

usabil­ity test

pro­ce­dure of testing

• The par­tic­i­pant have been intro­duced to the test itself and the speak aloud method, as many of them did such a kind of test for the first time and there­fore usu­ally tend to be shy. Hence I tried to make clear, that the soft­ware is about to be tested, not the par­tic­i­pant and thus all errors in han­dling it are due to soft­ware errors and not any kind of insuf­fi­ciency of him or her and there­fore can speak out load, what ever they are think­ing. Though they have not been pre­pared in using mag­net in any kind, just have been informed, that it is some kind of movie database.
• A pro­file had to filled, ask­ing for e.g. age, what kind of movies they like, how they get informed about new ones and finally, they had to pick one of three highly dif­fer­ent movies (“For­est Gump”, “The Matrix”, “Shrek”) which they know and like most.
• They had to ful­fill rep­re­sen­ta­tive tasks and in the mean­while telling what there first impres­sion was, what they liked/disliked at first sight or what they would have liked to do first. Each task has been placed on a dif­fer­ent page and the par­tic­i­pant was instructed to only turned the page after fin­ish­ing it com­pletely or giv­ing up. It has been tested whether the tex­tual infor­ma­tion about each movie is rec­og­nized, whether they can tell which movies are most sim­i­lar (with­out telling that sim­i­lar­ity is rep­re­sented by show­ing movies closely to each other). Finally they should find 10 movies star­ing the same actor, there­fore they needed to find out how to set up the cor­rect weight for the facet “cast” and turn down the other facets, addi­tion­ally they needed to switch focus by dou­ble click­ing a dif­fer­ent movie, to see more movies.
• Con­cluded by a post test talk.

Have a look at the ques­tion­naire (ger­man).

results

The seven par­tic­i­pants had an aver­age age of 27 (min. 23, max. 32), five have been male, two female and they answered to watch 7 movies per month in aver­age (min. 3, max. 10). Six of seven par­tic­i­pants told to use Inter­net resources to inform them­selves about new movie releases and/or read movie reviews.

All par­tic­i­pants could instantly tell which movies shown, are sim­i­lar to their selec­tion, i.e. they auto­mat­i­cally asso­ci­ated cor­rectly the close­ness of two enti­ties relates to their similarity.

While try­ing to find movies with the same lead actor, six out of seven found out the mean­ing of the slider con­trols to adjust weight­ings of the cri­te­ria, four of them by directly try­ing the slid­ers first of all. How­ever, six of the par­tic­i­pants men­tioned in the post test talk, that they missed label­ing for the slid­ers explain­ing their mean­ing, which was the most com­mon neg­a­tive state­ment, by con­trast, four com­mended the clar­ity of the inter­face in general.

Please try the demo appli­ca­tion your­self first, before see­ing the test record­ing high­lights (ger­man).

inter­face improvements

The strongest and most promis­ing sug­ges­tions by test par­tic­i­pants have become future requests and part of the cur­rent development.

High­light an attribute and all enti­ties in which it appears also, get high­lighted, too.

Allow­ing enti­ties to be clus­tered by their sim­i­lar attrib­utes, hid­ing the indi­vid­ual entity in a cloud, show­ing this attrib­utes as cap­tion; clus­ters can be dis­solved by click­ing them.

Sup­port drag and drop to move the stage; yel­low beams indi­cate enti­ties off stage.

A dummy ele­ment with user defined attrib­utes can be used to start brows­ing instead of an exist­ing entry picked from the data base.

A dust­bin in the lower right can be used to throw away enti­ties via drag and drop, caus­ing this entity not to appear again.

First efforts have been made towards devel­op­ing a new, tar­get user group focused graph­i­cal user inter­face as you can see below. The exten­sive user sta­tis­tics of IMDb.com have been used as a basis for an poten­tial mag­net user, i.e. 90% male, aged between 20 to 45. There­fore the inter­face should be appeal­ing for young and for older peo­ple, find­ing a com­pro­mise of a bit way-out style and reduced, dis­creet design focus­ing on read­abil­ity and con­vey­ing infor­ma­tion. The screens to fol­low show drafts help­ing to find ways to visu­al­ize enti­ties in a clear and space sav­ing man­ner, same holds for con­troller ele­ments in the upper right and lower left corner.

Col­ors and improved read­abil­ity. Show­ing ideas about how to han­dle long cap­tions by hover effects on the right side and inline infor­ma­tion pre­sen­ta­tion as an entity gets selected in the lower right corner.

Reduc­ing con­troller sizes fur­ther more by using sym­bols instead of text.

using mag­net — a wide vari­ety of ways

sug­gest­ing media to customers

• in a video rental store
• for online music plat­forms like last.fm or iTunes
• using mag­net on basis of the user’s own music or video col­lec­tion on a pc

mag­net at flickr.com, a pos­si­ble application.

After fin­ish­ing my the­sis, youTube.com launched a visu­al­iza­tion look­ing par­tial sim­i­lar to mag­net. It showed at the end of a clip, visu­al­iz­ing the cur­rent video and its related videos by a thumb­nail in a cir­cle posit­ing them­selves around the cur­rent one. But no con­trollers allowed to infer the selected related movies and the visu­al­iza­tion was miss­ing any infor­ma­tion like title or rat­ing, addi­tion­ally, the ani­ma­tion was quite uneasy and did there­fore not really help the view­ers. I assume that have been the rea­sons that led to the removal of it in the end.

visu­al­iza­tion for data min­ing or net­work­ing portals

• e.g. find­ing cus­tomers with the same hob­bies to define new tar­get groups for mar­ket­ing campaigns
• a human resource man­ager might like to view peo­ple with the sim­i­lar skills, about the same age on xing.com
• or I would like to find pho­tog­ra­phers on flickr.com using the same cam­era located close to my town to share lenses

work done so far

• com­puter sci­ence degree dissertation
• prototype/demo appli­ca­tion
• usabil­ity eval­u­a­tion of the client inter­face pro­to­type by seven par­tic­i­pants — test record­ing high­lights (german)
• sev­eral inter­face improve­ments based on dom­i­nat­ing state­ments given by test participants
• new tar­get user group focused user interface
• improved server per­for­mance by replac­ing the cost inten­sive pre­com­pu­ta­tion of a tri­an­gu­lar matrix of all sim­i­lar­i­ties between all enti­ties with an  lin­ear algo­rithm fast enough to be per­form this cal­cu­la­tion on-the-fly within the data­base, thus sav­ing space and prepa­ra­tion time
• re-engineered pro­to­type as basis for fur­ther development

future work

Right now, a dummy server imple­men­ta­tion is serv­ing a file to the client, which includes a fixed set of 300 enti­ties and their sim­i­lar­ity. It will be replaced by a PHP pro­to­type with an AMF inter­face using amf­php as fast comu­ni­ca­tion chan­nel between server and client. Amf­php has been cho­sen, as its source is open, free and sta­ble, easy to install and to extend, well doc­u­mented though, does not have any depen­den­cies and sup­ports direct object trans­fer in a binary form which is there­fore smaller and faster than XML or JSON, but JSON is any­way sup­ported in par­al­lel by this frame­work, which would allow to cre­ate an AJAX client as well.

Also the work on the skin­ning part needs to be con­tin­ued by real­iz­ing the pro­posed drafts, which will also include extend­ing the skin­ning capa­bil­i­ties of the client prototype.

With new server and skin­ning done, it would be time to set up another user test, to ver­ify the new inter­face on the one hand and to find direc­tions for fur­ther improve­ments on the other.

As Flash and Flex still depend upon the pro­pri­etary Adobe Flash Player, I will con­tinue search­ing for ade­quate alter­na­tives. I am espe­cially hop­ing to make use of the new HTML can­vas object, but I will not put effort into devel­op­ing an AJAX solu­tion unless the Microsoft Inter­net Explorer starts sup­port­ing it, too.