PDF version of this document

Research Proposal. March 2007

Andrew Green.

M.A. Design

The interactive potential of comic books in human-computer interface design: 'Implied' interaction.

Figures

Figure 1: McCLOUD. S. 1993. What are you really seeing? Understanding Comics: The Invisible Art. New York: HarperCollins. p31.

Figure 2: McCLOUD. S. 1993. Now you die! Understanding Comics: The Invisible Art. New York: HarperCollins. p66.

Aim of the project

The research has the domestic software interface design community as its audience. It will explore the potential of illustrative and temporal comic book techniques, which will be described as 'implied', and discover whether these might be of benefit to human-computer interface design. The project will take the form of an 'Action Research Case-Study'. It is anticipated that the data this case study will produce may be used as the basis for the further development of a formal approach to 'implied' interface design.

The research acts on the hypothesis that current prevailing trends in software and web interface design (interface design hereafter) have resulted in artefacts which may be described as "hot" (McLuhan) and that this is at odds with the way in which such artefacts may best function relative to human perception. The research examines 'implied' aspects of interaction fundamental to the success and longevity of comic books in light of ideas drawn from media theory, interface design theory, Phenomenological philosophy and Gestalt psychology.

The research will explore the potential for 'implied' comic book devices in reversing the trend towards visually "hot" artefacts currently to be observed in the practice of domestic software interface design.

Objectives

The objectives of this 'Action Research Case-Study' are to: (i) Produce a web application as a model of an 'implied' interaction. (ii) Test this web application using established web development reflective practice. (iii) Produce data from this reflection which may be used to form the basis for future research.

Context of research

McLuhan (1964) defines a model of media types which can be discussed with reference to two categories. (i) Media which employ intense focus on any one sense resulting in a passive and unidirectional relationship of meaning with a human participant are described as "hot". And (ii) media requiring the conscious or subconscious participation on the part of a human participant in order to complete the experience are described as "cool". McLuhan suggests the photograph as the archetypal "hot" media. The details are largely present in the image. No act of memory is required on the part of the viewer to complete the scene or make sense of the composition. The viewer absorbs the photograph needing to add very little. By way of contrast, McLuhan suggests television is a "cool" media. Through the interaction of the changing of channels and the selection of viewing material in an entirely personal way the viewer creates an experience unique only to themselves.

McLuhan's model is simplistic and his categorisation contentious. However, these concepts of "hot" or "cool" artefacts, and their properties of unidirectional or bi-directional interaction offer a position from which to begin a discussion of some of the implications the use of "hot" and "cool" artefacts have for interface design.

Professional experience suggests that the prevailing "hot" model of interface design is due to marketing pressures, the ongoing attempt to create the "wow factor", development time constraints, and the desire on the part of software developers to demonstrate their own prowess and to explore the possibilities of the medium of computation, the kind of developmental attitude discussed by Cheng (2006). A full and frank examination of the history of interface design and these forces is beyond the scope of this research, what is not is the belief that domestic interface design has lost track of the people for whom it is being designed.

To provide an example, I will analyse the hard-drive icon from the Mac OSX operating system. This artefact will be discussed as a specific case of current "hot", domestic interface design. This hard-drive icon appears on screen as a near photo realistic representation of the physical appearance of the piece of hardware it has been conceptualised as affording access to, the computers internal hard disk drive. This artefact may be considered "hot" on a number of levels.

Through an examination of several forms of progressive interface design Dourish (2001) draws together commonalities of approach which he relates to a common foundation in Phenomenological philosophy. These are codified as a series of desirable interface design principals which Dourish describes as representative of an "embodied interaction". These principles take a very much more participant centred stance in their attitude toward the computational medium and it is this position which is at the heart of this research project.

Dourish argues that designers need to focus on the ways in which an interface participant might understand and employ a tool, on how they may work on or through an artefact. When Dourish states that "user, not designers, create and communicate meaning" (2001, p.170) and "users, not designers manage coupling" (2001, p.170) he is calling for a multi-directional relationship of meaning between artefacts, their representations, the human participant and the culture of practice in which the use of these artefacts takes place. This relationship is "cool" in nature as it requires input from each of these factors in order to make sense. Dourish describes the importance of a close examination of the processes for which any artefact stands in. This includes any real-world processes which the control is attempting to stand in for and the actual computational process which will be set in motion when this particular control is called to operation.

In light of these principals, the Mac OSX hard-drive icon may be considered as "hot" due to the unidirectional nature of its representation. This representation appears to be attempting to loosely stand in for the computational process which its artefact helps control. Had it succeed in doing so, this representation would have been considered much "cooler". However, the metaphor created between this process and the representation is inconsistent in a number of ways which will now be described.

The hard-drive control exists as a means by which the human participant is able to access shortcuts to collections of data, to network resources, to explore the information saved to the computer's hard disk and to launch installed software applications.

Information stored for later retrieval is a concept which may well be associated with a storage device. However, the metaphor does not extend well to the actions of launching applications and accessing network resources. When considering the latter as an extension of the hard drive metaphor the computational process which is being described quickly becomes confusing. Despite utilizing hard-disk hardware, the computational process involved in retrieving data from network resources is considerably more complex than that of accessing data from a local hard-drive. Retrieval speed, remote access authentication and network uptime are all factors which have little impact in a local disk access process but can dramatically shape a network data access request. Similarly, the use of this control in launching a software application can be considered inconsistent with the subject of its representation. This inconsistency resides within the meaning created by the use of these applications. Whilst in use software applications assume the mantle of tools each with a set of specific purposes. Tools are ubiquitous to humans. When compared with such an instinctive and intrinsic human process as tool selection, the computational process of launching an application is likely to be entirely overwhelmed by the more basic human metaphor.

Further, advertising for the Macintosh has been consistently aimed at the home computer market. Upon its launch in 1984 the Mac was championed as "The computer for the rest of us" (Johnson, 1997), pitting itself against the command-line based interface of IBM. A more recent campaign has featured the cool and fun Macintosh favourably compared to an unreliable, business focused PC (Brooker, 2007). The kind of buyer the Macintosh is being marketed at seems unlikely to largely be also the type of computer enthusiast who carries out their own repairs and would naturally understand the visual appearance of a hard-drive. These Mac buyers are unlikely to understand the reference this control is making.

All of these aspects of the hard-drive icon involve the creation of meaning being either inhibited or codified in a hard and fast manner by the designer. It is proposed that this leaves the human participant isolated from this aspect of their interaction with this control. And it is this which is being described as an example of "hot" interface design.

Lakoff and Johnson (1999) argue that human perception employs a process of categorisation at what they describe as a "basic level". This is described as the simplest form any object can be reduced to and would include concepts such as "car" or "tree". Basic level categorisation, it is argued, allows humans to act with agility. Lakoff and Johnson argue that there is no need for a person to study the detail of a car speeding towards them, nor to recognise even what type of car it is. The basic level information of "car" and the concepts of "fast" and "this way", understood as integral elements of the human embodied condition, are sufficient for action to be taken. Lakoff and Johnson argue that gaps left by the basic level of information gathered are inserted whenever the event is recalled. Details such as red car, hatch-back and so on, are provided by the prior experience of the participant. In effect, this is the human equivalent of the way a DVD player guesses picture elements based on the last stored index frame it has passed.

A recent conversation with a colleague came around to the subject of the "save" control. When it was described as an image of a "floppy disk" my colleague stopped, considered for a moment and exclaimed "yes, I suppose it is". Being young enough to have had scant first hand experience of actual floppy disks my colleague naturally did not link this meaning with the "save" control representation. Where the floppy disk control differs from the Mac OSX hard drive control is that as the floppy disk image is represented in a very simple style, a basic level. It is proposed that the nature of this representation allowed my colleague room to make the image fit her interpretation of what happened when she used the control. She filled in the details with elements from her own experience. This simple style of representation aided her creation of meaning within the interaction environment and it formed a basic level category image for her. As a result of this she had been able instantly to recall the control image as easily as a basic level concept image such as "car". This clearly describes the kind of bi-directional interaction being defined as representative of "cool" interface artefacts.

Gestalt psychology demonstrates the instinctive need of the human participant to make sense of any environment in which they are placed (Kohler, 1970). This process is another example of a "cool" interaction.

The interaction between reader and comic book functions similarly. McCloud (1993) describes the way in which the detail omitted from the comic book page is included by the viewer. Presenting the image of a simplified face (Figure 1) McCloud asks "what are you really seeing?" With this simple illustration, McCloud suggests that a circle, two dots and a straight line have taken on far more meaning than the sum of their parts. Gestalt psychology suggests that the viewer's perception seeks to make sense of the image and so assigns attributes to it from past experience. The resultant image can be viewed as an example of the basic level category of "face". Some informal demonstrations of this example conducted by myself with professional colleagues have yielded a fascinating response in which some participants even assign specific emotions to what they perceive as a face.

McCloud states that Comic books which employ this strategy greatly enhance the immersive power of their story, details of which are all familiar as they are inserted sub-consciously by the reader from their individual experience. He contends that the resultant interaction is both unique and familiar to each reader. This is an 'implied' interaction.

McCloud suggests that comics also employ these devices of omission and implication in a temporal, narrative manner. He describes the majority of comic books' narrative as taking place in the space between the panels, the gutters.

"I may have drawn the axe being raised in this example," (Figure 2) "but I'm not the one who let it drop or decided how hard the blow, or who screamed, or why. That, dear reader, was your special crime, each of you committing it in your own style. All of you participated in the murder. All of you held the axe and chose your spot." (McCloud, 1993, p. 68)

This research takes the position that a consideration of human perception is vital in achieving the kind of interface described by Dourish (2001) and that comic books are able to utilise human perception in order to better communicate and function as a medium. As a result, it is proposed that an exploration of interface design from the perspective of the comic book could help develop an environment in which the creation of meaning could occur more easily between interface and human participant. This would constitute a far more efficient, natural and sympathetic interaction than presently is experienced in environments populated by "hot" representations.

Very little contemporary research in this area exists.

Although Merholtz (2003) points to McCloud being widely read in the interface design community, the usage to which comic book theory has been put in this area of practice is limited to such accounts as of a way of clarifying diagrams (Manning) or of understanding symbols (Dourish).

Cheng (2007) has codified a methodology intended to enhance the comprehensibility of technical, interface design concepts to a non-technical audience. Despite Cheng's use of the comic medium as a tool by which to present and demonstrate the function of interface concepts rather than inform their nature, the justification for this usage is similar to that postulated by this research project.

It is apparent that this is an area in which an extensive and ongoing study may be conducted. It is anticipated that any success enjoyed by this project will form the basis of further research which may become an important contribution to the area of interface design knowledge.

Research Methods

Knowledge gaps

In order to bring this projects reflective stage to a successful conclusion it may prove necessary to form an understanding of statistical analysis. If this indeed proves to be the case then I intend to avail myself of the UWCN facilities and staff who will be able to complement the wealth of information contained in the research methods literature I presently possess.

Plan of work

The project will run from January to December of 2007.

Jan - Mar 2007:

• Initial software development:
  
  • Architecture design.
  • Development of initial functionality.
  • Continue to add resource data.
  • First draft of development practice report.
• Begin the review of research practice.
• Begin the comic and cartoon investigative practice.
• Continue contextual review.

Mar - Aug 2007:

• Continue comic and cartoon investigative practice:
  
  • A body of practice with which to become sufficiently familiar with the medium.
  • Traditional and web based comic navigational styles.
  • Ethnographic differences in approach to the reduction of detail.
  • Begin comic and cartoon investigative practice report.
• Application development:
  
  • Begin interface design work.
  • Finish up any remaining core development.
  • Add remaining resource data.
• Continue contextual review.

Sep - Oct 2007:

• Finalise the comic book investigative practice.
• Complete contextual review.
• Complete practice reports.
• Finalise, designs and implement the resultant interface.
• Design the specifics of the reflective strategy.
• Recruit participants from the interface design, web development and software development communities.

Nov - Dec 2007:

• Conduct reflective strategy.

Dec 2007:

• Collate results of the reflective strategy employing a trend analysis methodology.
• Prepare project presentation material.

Ethical considerations

Fully informed consent will be sought of all participants in the reflective strategy. Interviewees will be presented with a consent form and online participants will be presented with a control by which they may electronically consent.

References

BROOKER, C. 2007. [WWW]http://www.guardian.co.uk/commentisfree/story/0,,2006031,00.html. I Hate Macs. Last Accessed: 24 Feb 2007.
CHENG, K. 2006. [WWW]http://www.ok-cancel.com/archives/article/2006/01/design-trends-and-design-patterns.html. Design Trends and Design Patterns. Last Accessed: 23 Feb 2007.
CHENG, K. 2007. [WWW]http://kevnull.com/2007/01/communicating-concepts-through-comics.html. Communicating Concepts Through Comics. Last Accessed: 23 Feb 2007.
DOURISH, P. 2001. Where the Action is. USA: MIT Press.
JOHNSON, S. 1997. Interface Culture. New York: Basic Books.
KOHLER, W. 1970. Gestalt Psycology. New York: Liveright.
LAKOFF, G. JOHNSON, M. 1999. Philosophy In The Flesh. New York: Basic Books.
MCCLOUD, S. 1993. Understanding Comics. New York: HarperCollins.
MCLUHAN, M. 1964. Understanding Media. UK: Routledge.
MANNING, A. 1998. [WWW]http://cmbsd.cm.nctu.edu.tw/~icm5201/spr2003/manning.pdf. Understanding Comics: The Invisible Art. Last Accessed: 23 Feb 2007.
MERHOLZ, P. 2003. [WWW]http://www.peterme.com/index102698.html. Interface Design Recommended Reading List. Last Accessed: 23 Feb 2007.
VAN DUYNE, D. LANDAY, J. HONG, J. 2003. The Design of Sites. Boston: Addison-Wesley.

Works referenced in Research Methods Section

CHENG, K. 2006. [WWW] http://www.ok-cancel.com/archives/article/2006/01/design-trends-and-design-patterns.html. Design Trends and Design Patterns. Last Accessed: 23 Feb 2007.
DOURISH, P. 2001. Where the Action is. USA: MIT Press.
ENNIS, G. DILLON, S. 1996 (Originally published in magazine form 1995). Preacher: Gone to Texas. New York: DC Comics.
MCCLOUD, S. 1993. Understanding Comics. New York: HarperCollins.
MIGNOLA, M. 2002 (Originally published in magazine form 1993-1994). Hellboy: Seed of Destruction. London: Titan Books.
MILLER, F. 2002 (Originally published in magazine form 1986). Batman: The Dark Knight Returns. New York: DC Comics.
MILLER, F. 2005 (Originally published in magazine form 1993-1995). Sin City: A Dame to Kill For. USA: Dark Horse Books.
MOORE, A. GIBBONS, D. 1987 (Originally published in magazine form 1986-1987). Watchmen. New York: DC Comics.
MOORE, A. LLOYD, D. 1988 (Originally published in magazine form 1982-83). V for Vendetta. New York: DC Comics.
VAN DUYNE, D. LANDAY, J. HONG, J. 2003. The Design of Sites. Boston: Addison-Wesley.
SCHULTZ, C. 1950-2000. [WWW] http://www.snoopy.com/comics/peanuts/archive/index.html. Peanuts Strip Library. Last Accessed: 30th April 2007.

PDF version of this document