Building Upon Interaction Gestalt Research

Master’s Thesis, North Carolina State University, 2016

My Master’s thesis builds upon fundamental Human-Computer Interactions research to expand the vocabulary interaction designers have to work with. By combining attributes that are inherent to interactions, certain experiential qualities emerge. This research gives designers an expanded toolset to shape interactions based upon content.


Graphic designers have long used associations with physical materials to tie form to content. With the general de-materialization of our discipline, we need other methods to express content. Up to this point in interaction design, much attention has been devoted to has been on usability and the builtstructure of digital artifacts. Aesthetics, the visual and behavioral qualities of an interaction, often are viewed as a surface component dealt with at a later stage of the design process, but not integral to the overall functioning of the artifact. The literature in Human-Computer Interactions is moving towards the notion that aesthetics aren’t merely the way that something looks or behaves, but are important to a person’s understanding and experience with an artifact. This work puts fundamental HCI research into designed scenarios towards the goal of expanding the interaction design language for working designers.


… interaction designers in practice are not fully trained to conceptualize and manipulate such immaterial expressive design space compared to the case of other design areas in which designers deal with materials with concrete and tangible attributes.

Lim, Lee, and Kim, Interactivity Attributes for Expression-Oriented Interaction Design
The palette of the designer working with materials is deep and robust. Materials are rich with meaning and associations for designers to exploit. This physical material and its embedded meanings expresses content, helping people understand the nature of the artifact. When working with digital artifacts, designers need a different vocabulary to express content.

Pragmatic Aesthetics

Pragmatic aesthetics enables designers to bring context into the mix of usability and system structure. Designers can shift a focus back towards how people relate to an experience, and what that experience means to them. Though designers can’t themselves create meaning for people, they can create conditions for people to derive meaning from interactive experiences.

Pragmatic Aesthetics does not regard man and world as separate things but focuses on the integration and interrelations bound to context. Designing for aesthetic experiences invites people to actively participate in creating sense and meaning.

Petersen, Graves, Iversen, Krogh, and Ludvigsen. Aesthetic Interaction: A Pragmatist’s Aesthetics of Interactive Systems.
The three Vitruvian principles of Good Architecture, structure, commodity (usability), and delight (aesthetics), can help conceptually organize the characteristics and elements that make up an interactive experience.

… functionality and clarity is not enough to meet human needs and desires when engaging with interactive systems.

Petersen, Graves, Iversen, Krogh, and Ludvigsen. Aesthetic Interaction: A Pragmatist’s Aesthetics of Interactive Systems.

Interaction Gestalt

The framework of Interaction Gestalt allows designers to combine interactivity attributes, such as movement speed, reaction speed, or range, to create overall experiential qualities such as lightness, complexity, or simplicity. These qualities become apparent through use of the interactive artifact.   

[Interaction Gestalt is] the aesthetics of interaction that does not have tangible properties because it emphasizes that the aesthetics is not intrinsic to the artifact itself, but to the way people experience it.

Lim, Stolterman, Jung, and Donaldson. Interaction Gestalt and the Design of Aesthetic Interactions.

Interaction Gestalt provides a new way to describe interactive experiences. Instead of only relying on user experience qualities such as ease of use, or artifact properties such as layout, hierarchy, and other such visual properties, interaction gestalt describes the totality of behaviors and the affect that has to a person’s experience.

Interactivity Attributes

In Interaction Gestalt and the Design of Aesthetic Interactions Lim, Stolterman, Jung, and Donaldson suggest that interactions are composed of certain fundamental attributes.

Lim, Stolterman, Jung, and Donaldson. Interaction Gestalt and the Design of Aesthetic Interactions

Experiential Qualities

In Interactivity Attributes: A New Way of Thinking and Describing Interactivity Lim, Lee, and Lee conducted user tests involving pairing interactivity attributes (i.e. expectedness, proximity, movement speed) with what they called “emotional qualities.” These successful tests provide a roadmap for exploring these emotional qualities (experiential qualities) in designed scenarios.
Lim, Lee, and Lee. Interactivity Attributes: A New Way of Thinking and Describing Interactivity


The investigation was largely based on understanding the nature of interactivity attributes and interaction gestalt through designed scenarios. The scenarios explored the concepts through combining attributes, combining experiential qualities, testing amplitude of attributes, and where qualities become neutral.

Combining Attributes

Using data visualizations as context for exploring interactivity attributes, these examples show sequences of movement. Combining
between 2–4 interactivity attributes created the desired experiential qualities.   
The user in this interaction pushes around the various bit of data represented by ellipses. Fast speed, prompt response, and wide movement range combine to create interactive elements which feel light.
Fast, sequential, and approximate motion. The transition between data parameters caused each bar to quickly shift positions sequentially. The motion had a bounce to it, so that when it reaches its destination, the bar snaps back just slightly. These attributes come together to create an exciting, fast quality.

Artificial Experiential Quality
Discrete and narrow range of motion creates an artificial quality in these hover states.

(Data represents percentage of male and female artists in the Tate’s collection).

Sympathetic Reaction
Slow, sequential motion that is imprecise and delayed in reaction creates an almost friendly, sympathetic quality.

Unsympathetic Reaction
Likewise, fast, concurrent motion that is precise and prompt creates a cold, efficient, unsympathetic quality.

While seemingly obvious, the research provides a user-tested roadmap to creating certain qualities like simplicity in this instance. Expected and precise motion, all happening at once (concurrency) are the keys to a simple interaction.

Likewise, although this example shows exaggerated behaviors, pairing unexpectedinteraction attributes with sequential and delayed behaviors creates complexity.

Exploring Attribute Amplitude

This study explores how the amplitude of an interactivity attribute defines an experiential quality. Opposite qualities are put along a gradient, the variables mapped to mouse position. When the cursor moves from one side to the other, attributes change from one extreme to its other extreme.
Heavy and Light Gradient
The heavy end of the spectrum is defined by the ellipses moving in unison, precisely and slowly, with a similar look between them. The light end of the spectrum is characterized by the ellipses expanding and contracting widely, an after-image remaining in their size shifts. The changes move at a swift pace.

Combining Experiential Qualities

Could experiential qualities be combined? For these experiential qualities, some attributes define the quality more than others. By selecting qualities that had many  differences, and selecting their dominant attributes, hybrid qualities began to emerge.
Complicated / Simple Hybrid
Combining experiential qualities need not be limited to just behavior. By considering the qualities of the visual properties of an interactive object along with it’s behavioral properties, hybrid qualities emerge.

Exploring Neutral Qualities

These explorations asked whether the average of two experiential qualities produced what could be considered “neutral.” And if the average wasn’t neutral, the study explored what attributes lead to a neutral experiential quality.

Based on a definition of neutral which is “having no strongly marked or positive characteristics or features,” a neutral interaction is concurrent (happens simultaneously), continuous, rather expected, somewhat narrow in range, a medium speed, and pretty precise in its accuracy.


What this investigation has shown is that interaction requires holistic thinking to create a larger experience for people. Every single decision is important. These principles aren’t effects that you turn on, they are ways of thinking about interaction. It truly is a gestalt that sets the mood, the tone, the feel, however you want to describe it.

The interactivity attributes that I worked with are adaptable. They can be stretched in their meaning. A sequential attribute isn’t just how something moves, or how it behaves, it can be a fundamental organizing structure of an interaction. It can relate to the information architecture, or the user experience. A discrete attribute could relate to an animation, or a transition, or how type moves towards us in a 3D virtual environment. We are at a beginning of truly taking advantage of the affordances of interactive experiences, and this is the start of a vocabulary that has the potential to grow into a sophisticated, complex language.

To develop a sense of quality more or less requires a development of a language. A designer needs a well-developed language in order to be able to express design ideas and design qualities. A sense of quality is not enough if that sense cannot be expressed in some kind of language.

Löwgren and Stolterman. Thoughtful Interaction Design: A Design Perspective on Information Technology