Book Review: “What You See Is What You Feel” by Koert van Mensvoort

A fellow MoM’er already wrote a good review on the PhD thesis What You See Is What You Feel by Koert van Mensvoort. Read the MoM review here and download the full book here. I would like to give an extension to this review with my own thoughts.

Although the thesis is done at the Technical University of Eindhoven (TU/e) the thesis provides a “social-cultural context” which provides the reader a lightweight but still refreshing introduction to media theories in relation to simulation, virtuality and reality in the era of the homo desktopus (p.20) – though I think the homo desktopus is increasingly less suitable due to ubiquitous computing (more on that later). When I read the prologue, Baudrillards concepts of Simulacra and Simulations and McLuhan’s technological determinism in The Medium Is The Message were the two main lines of thought. Mcluhan’s views could be found in where Van Mensvoort states we increasingly do not perceive media as media anymore but part of our experience interaction; media is the message. We have become passive to media, as we are not actively breaking down mediation as it has become inherent. But I thought it was strange they were not mentioned anywhere. It was when reading the epilogue when it became clear that Van Mensvoort actually shares McLuhan’s definition of media (p. 123). Overall the theory provides an interesting read for everyone as simulation and reality is very interesting indeed, and provides a superb example of combining media theory with technical research and application.

The thesis provides next to optically simulated haptic feedback experiments a search for the best suitable development platform to put the new haptic methods in practice. Flash is used as base for haptic feedback development, but since flash is becoming more controversial these days with html5 around the corner it would be a good aspiration to implement haptic feedback techniques instantly and cross-platform without the need for extra software.  In a decade where technology, both in hardware and software, change rapidly it is hard to become universal, even for a moment, as is the case with haptic feedback Koert van Mensvoort proposes. The mouse is since 1967 a main HID but is increasingly losing ground as interface to voice commands and physical touchscreens plus gestures on mobile computers and smartphones. Steve Jobs publicized his thoughts on this matter in relation to Apple’s strategy: Thoughts on Flash, where he states that “Flash was designed for PCs using mice, not for touch screens using fingers. For example, many Flash websites rely on “rollovers”, which pop up menus or other elements when the mouse arrow hovers over a specific spot. […] If developers need to rewrite their Flash websites, why not use modern technologies like HTML5, CSS and JavaScript?”. In this situation optically simulated haptic feedback could be taken over by physical haptic (tactile) feedback (example). Van Mensvoort does give a reaction to these kinds of potential arguments on pages 22-24 and states that “We seem to have more of a ‘both/and’ situation instead of and ‘either/or’ situation” (p. 25), a good observation. But I believe that differences in haptic feedback systems will not help to prevent compatibility differences in ubiquitous computing where user’s attention is spread across both WIMP and touch/speech devices.

For readers with a humanist background in media the discussion by Van Mensvoort are not provoking as the thesis does not offer new insights to the discourse. The theories discussed to give the thesis a social-cultural context do feel like revisiting concepts once read before. Perhaps this isn’t strange as this context is discussed mainly in the prologue, introduction, conclusion and epilogue of the thesis. The thesis itself is based on technical and empirical research. Here I would recommend watching The Woods Smell Like Shampoo as it is more of an artistic expression by Koert van Mensvoort in collaboration with others. But what is most interesting is the outcome, and concluding from Van Mensvoort’s thesis (optically simulated haptic feedback increases productivity in WIMP environments) and my own experience, optically simulated haptic feedback does invoke rather strange but realistic experience. Check it out on powercursor.com, it is pretty cool.

Full Details:

What You See Is What You Feel
By Koert van Mensvoort, 2009
ISBN 978-90-386-1672-8