Brett Oppegaard
University of Hawai‘i at Mānoa
brett.oppegaard (at) hawaii.edu

Download the full article in PDF format

INTRODUCTION

Communication takes place, per the idiom about it. That phrase refers to the information-exchanging action happening somewhere. In the communication process, a discourse territory gets defined and bounded by inscriptions that manifest within it. These moments are realized, and reflected upon, as a part of a spatial and temporal context, embodied with some sort of a physicality, shaped by sociocultural and historical frames of reference that each participant contributes. Conceptualizations of space and place can conjure many ideas, including of a spectrum that theorizes undifferentiated and transitional space on one end and highly particularized and inscribed place on the other (Casey, 2013; Cresswell, 2004; Massey, 2013; Tuan, 1977; Wilken, 2008). This place-making moment— where communication happens—therefore is of fundamental concern to communication designers and researchers (St.Amant, 2018a), even though it routinely is taken for granted, ignored, or simply unrecognized.

The who, the what, the when, the how, and the why of communication all have to emerge and instantiate somewhere, if they are to occur beyond intrapersonal theoretics. Where this materialization happens—if that makes a difference whatsoever in the message or even use of the message—therefore matters. This experience report, in turn, offers insights from direct interventions and related field studies—with mobile technologies—over a recent five-year period. Its academic products include pragmatic documentation of experiments and their designs, user-testing results, observations, and a collection of lessons learned.

Such investigations of corporeal contexts especially need more attention from applied scholars incorporating mobile technologies (Robinson, Lanius, & Weber, 2018). Such locative-media exigence transcends disciplines, particularly for those engaged in aspects of user-experience design, who are striving to understand relationships between location and usability, as a way to improve communicative connections in the smartphone era (St.Amant, 2018a, St.Amant, 2018b, Fan, 2017, Frith, 2015). The “triple revolution,” which combines a roughly decade-long era of related developments in the Internet, wireless connectivity, and social media (Rainie & Wellman, 2012, p. 19), privileges and expands communication mobility. Similar to the ways in which the advent of linear perspective tilted public interests toward visual culture (Jay, 1993), the advent of smartphones has drawn societal attention to fluidities in communication contexts or to the perceived potential of such a mobility, even if it is unenacted (Campbell, 2018; Campbell 2020; Wilken & Goggin, 2017).

An intent of this work was to establish insider positionality in this realm of locative media and to investigate the “wicked problem” of designing for mobility, in which user attention rapidly toggles between physical and mobile interfaces and competing yet intertwined contexts, many outside of a designer’s purview, control, or interest (Verhulsdonck, 2017, p. 56). As Farman (2012) envisioned such an information-exchanging dynamic, mobile technologies, such as smartphones, create an interface in the world that can transcend digital and physical environments to create a mixed and hybrid media ecosystem. Users can emphasize the physical of this hybrid by putting their phones in their pockets (and responding only to sounds or vibrations). Or emphasize the digital by putting on headphones and positioning smartphone screens close to their faces. They can seek external information, even deeply into the recesses of the Internet. They can be prompted by programming and geolocation hardware to pause and engage in an activity at a specific spot. All of these situations, and many more, coinhabit within mobile experiences. So how can such experiences holistically be studied?

For initial resources to marshal toward this effort, I had just a special-topics class to offer, a small grant to cover programming costs, and a lingering interest in creating prototypes of new forms of locative media. I knew I wanted to share durable information that led to deeper learning about places and better interactions with them. I knew I needed multiple sites, to test transferability of findings, and each site needed a physical landmark to draw interest and to signal that something informationally richer might be there, if a person just were to look carefully and seek it.

During this contemplation and preproduction period, I would walk by a broken fountain in the middle of my campus just about every day and wonder why that had never been fixed (more details below). After asking myself that question on several occasions, I thought about other people possibly wondering the same thing. I also walked by many different pieces of public art around campus and thought of the questions I had, when in the vicinity of the artwork, based on the scant information present, the foundational information missing, and my personal curiosities. I wondered what kinds of meaningful interactions could be prompted by smartphones at these sites. As I kept coming back over and over again to various public art sites on campus and in my community, I knew I had found fertile locations for my field laboratories. But to discover what I really wanted to know, without the necessary research instruments already at hand, I also knew that I had prototypes to build and important research questions to investigate and pursue, such as:

RQ1: In what ways can place matter in user-experience-design prototyping?

RQ2: In what ways can developing prototypes of place around public art generate novel forms of locative media, or even more specifically of locative journalism, worthy of further development and study?

LITERATURE REVIEW

Prototypes have been used for centuries to help produce innovative and effective representations of ideas that build connections among stakeholders, including between designers and their audiences (Isa & Liem, 2020). User expectations often relate to context, and prototypes of place can help to identify aspects that could create or affect perceptions of affordances or constraints offered by a physical setting (St.Amant, 2018b).

Despite the ascent and increased ubiquity of mobile technologies in societies throughout the world during the past decade, usability and user-experience studies of these phenomena have not kept pace, with frustrated users abandoning more than half of the mobile media sites encountered in roughly three seconds or less (Knight, 2019). The field of user-experience-design research meanwhile has shown growing acceptance of studies in all phases of product development, including early stages. Besides more of an emphasis on prototyping, that expansion has encompassed human responses and attitudes about technologies as well (Robinson, Lanius, & Weber, 2018). Empirical research into locative media therefore seems to fit well in this field of communication design. Conceptual and physical prototypes can lessen uncertainties about a design while under development. But more research needs to be done about how mixing conceptual and physical prototypes can be useful, especially in the design of locative media.

St.Amant (2018b) explicitly connected user expectations to context and argues that the better that user-experience designers understand a particular place of use, the greater the chances of creating designs that are useful there. He posited that a conceptual prototype of place represents elements you expect to encounter in that specific context. By surveying dozens of places of public art in my community, for example, I knew I could expect to find the artwork either in an open public setting, such as a courtyard, or in a public building, open to visitors during regular business hours. In either type of location, the artwork typically would be accompanied by a small label that shared the name of the artist, the name of the work, the year it was placed, and a reference to the funder, usually the Hawai‘i State Foundation on Culture and the Arts. Beyond those basics, though, the contexts varied widely, such as the contrasts between being situated in an easily accessible and open grassy field, near a busy road, or being housed in a special room on the top floor of a university library, which required staff permission to enter.

Locations of public art typically are places that are intended to spark conversations throughout the world and across cultures. Communities generally come together to commission, create, and place a piece of public art in order to inspire people who encounter it—in the physical environment—into some sort of thought or action. Public-art places therefore are felicitous sites for experimentations into locative-media design.

To broaden and enrich academic perspectives on such mobile media, at least some scholars are going to have to design, build, and test apps as a way to examine them from an internal positionality (Brown, 1992; Crompton, 2013; Design-Based Research Collective, 2003; de Souza e Silva & Delacruz, 2006; Dunleavy & Dede, 2014; Henze, 2012; Kearney, Burden, & Shuck, 2019; Martin, & Ertzberger, 2013; McMillan, et al., 2010). St.Amant (2018b) wrote that a key to this type of research is collecting information that allows the user-experience designer to develop, test, and revise according to feedback from users, in a cyclical nature. He suggested a process with three general steps: 1. Context Identification, 2. Prototype Mapping, and 3. Prototype Testing. Even though this project began before St.Amant’s (2018b) publication, general concepts—like he described—already had been circulating in the research community and in my research project. For his part, St.Amant put them in a particular order, provided depth to the ideas, and gave them specific boundaries, outlining a process that retrospectively is reflected in this report. In that respect, he provided a procedural framework for conceptually structuring this type of research, which was used to classify and categorize descriptions of what happened during this project as a post-mortem organizational and reporting tool.

The Mānoa Public Art app was built for this research purpose. As a free and open-source app, it has clear community benefits, widely sharing information about the large public-art collection on or near the University of Hawai‘i ’s flagship campus, in a Honolulu neighborhood, just north of Waikiki, called Mānoa. The first group of students who worked on it, as part of a graduate class on “Locative Journalism,” helped to design the structure and the content focus. The groups of students that have contributed to it since have been trained to think of it as a research tool first, a platform for innovative mobile-media content second, and a community resource third, created primarily to study user-experience design and interaction with locative media in-situ.

This functional example, available by searching for “Mānoa Public Art” in both iOS and Android markets, illustrates behind-the-scenes decisions and the underlying pragmatics of creating content and a platform for exploring issues related to user-experience design of locative media. It also provides a replicable framework (see Figure 1) for pursuing similar projects at other campuses or communities, integrating people, place, and information digitally embedded in a location, via geolocative technologies, and triggered by sensors that come standard on smartphones.

This schematic shows the three primary cycles of research incorporated into the Mānoa Public Art project’s process as stacks of steps. Each step includes Planning, Action, Observation, and Reflection. The first step focuses on development of a paper prototype. The second step focuses on the development of a mobile-app prototype, and the third step focuses on field tests of the mobile-app prototype.
Figure 1: The first three iterative cycles of the Mānoa Public Art design-based research project.

Context Identification: Mānoa Public Art

For Context Identification, St.Amant (2018b) recommended an examination of the context or contexts where users perform a process. In this case, I considered the process an interaction among people, place, and smartphones, primarily focused on a physical artifact of that place. When starting this project, I considered many types of potential test sites for such a process, including those of significant cultural, social, historical, and natural importance. I knew I needed a landmark that could signal to a passerby that the potential for deeper place engagement existed, as a way to trigger a mixed reality of digital and physical information. Ideally, I thought, the landmark would be the place’s primary artifact of interest.

I also wanted to work across multiple test sites, in order to examine the transferability of any findings. Each test site needed a similar type of prompt. Most cities—and college campuses—have public- art collections. They typically are underused and underappreciated. They also usually are information poor in context about the art, with little more than a small label of background details. In such contexts, I could expect that the artwork would have that label, and the general proximity of the artwork could be accessed by almost anyone, including those with mobility impairments, based on Americans with Disabilities Act standards generally regulated in public places, including on public college campuses. In addition, human sensory organs, such as eyes and ears, could be extended and amplified (McLuhan, 1964) through their smartphones, which most of them would have, per current statistics provided by the PEW Research Center about adoption rates of mobile devices by American adults (“Mobile Fact Sheet,” 2019).

To put a scope on this effort, and to identify our corpus of possibilities, students and I focused only on public art within the official boundaries of the Mānoa neighborhood of Oahu, most of which— but not all—had been identified by the university as a part of the state’s public-art collection in this vicinity. As another limitation, the experiences we were to design had to exist only in mobile-app form, without any physical-environment enhancements, meaning our intervention had to be through mobile media only. A mobile-app intervention is useless without the user having a smartphone that our program had been designed for (a supported version of Android or iOS). But the user also would have to be willing, or willing to be enticed, to open that app’s mobile interface and at least partially experience their present place through it for more than a few seconds. That means the phone would have to be connected to the Internet or wireless network. It would have to connect its GPS sensors with GPS satellites, to locate the place and locate the user and bring those two together.

To make this happen, the user would have to download the free app, open the app, and engage with the mobile media in the app, including those users with varying levels of sensory abilities (some blind, some visually impaired, some color-blind, some deaf, some hard-of-hearing, etc.). Therefore, media accessibility was an important consideration. In addition, the battery on the device would have to be charged enough to complete the provided information exchanges, and, maybe most importantly, the user would have to willfully want to participate in this activity, which was not a trivial design concern. College students typically are not wandering around campus musing about the meanings of nearby public art. In our studies, we found that many of them did not even notice the art—even if they walked by it daily—unless our app, or us, pointed it out to them.

But if we could get them to notice the art, and if all of the technical processes went right, and the user was motivated and engaged enough to give it a chance, only then would our design team really have an opportunity to affect the experiences that person had in that place. From the researcher perspective, if we really did our homework about the place and the public art, and understood the dynamic already existing in the physical environment between people and context, like we wanted to do, and if we designed an engaging-enough experience for that particular place, then we might just learn about the peculiarities of locative media. In other words, there were a lot of background steps to take, and a lot of connections to make, just to get to the part of the research we really wanted to do.

To this end, in the fall of 2015, I first created a “special topics” class on Locative Journalism with five master’s level graduate students. This class was 15 weeks long, meeting once a week, for about three hours per session. Collaboratively co-creating this new class, my students and I took opportunities in our formal First Space learning settings to consider ways—through emerging affordances of mobile technologies—that our university campus could become both a more dynamic learning place and a multilayered physical learning space, with a porous boundary between student designers, community audiences, and informal Third Space learners.

We started this process by imagining the wildest possibilities we could (one student, for example, envisioned an elaborate production of street theater around an oddly out-of-place statue, intended to prompt a variety of responses both physical and digital, that would be captured by smartphones in the crowd, to be streamed live and also available for replay). But we eventually settled into a pragmatic mindset about productive ways to investigate ideas of locative media, with our initiative’s boundaries gradually formed by the constraints of our technological skills, project resources, and the rigid time frame of a semester-long production cycle.

We also considered a variety of content options, including locative media about the many exemplary plants and trees on campus, for which the university has a designated tour, or developing overlaps with the neighborhood’s sponsored tour of historic homes. But we chose instead to explore mobile-mediated interactions between people and public art, for a variety of reasons, including the passive nature of this activity in general, open access to the raw materials, and the latent learning potential of such an art collection and its generally disconnected passersby. Hawaii also was the first U.S. state to enact a Percent-for-Art in Public Places program, dedicating 1 percent of the construction costs on new buildings for the commissioning or acquisition of art. The choice of content therefore had historic, journalistic, and experimental foundations.

The basic structure of this class was similar to other graduate classes, in that we covered weekly assigned readings about the core topics, as in mobile media in general, and locative media in particular. What differed greatly in structure, though, was the open pursuit of a buildable idea that could be envisioned, planned, and executed during the course of a single semester. The interactive mobile experience, and all of its parts, was the final product of the course. Because none of these students had studied mobile media before, and only one of them had any type of professional media- production experience, the first five weeks of the course were devoted to building a basic level of knowledge about these topics, through an extensive literature review, including examining how they overlapped, or could overlap, and trying out potential ideas as a part of those discussions. We also brought in via videoconferencing five prominent scholars in the field, to get a global sense of the direction of this type of scholarship and to brainstorm with external actors.

That class turned out to be just the first of five—both undergraduate- and graduate-level courses—to significantly contribute to this experience report and this research project’s academic products during the five-year span, plus a couple of additional classes that contributed students as user-experience testers as well.

Prototype Mapping: Mānoa Public Art

When Prototype Mapping, St.Amant (2018b) suggested determining what features or characteristics users associate with the related prototype of place. Once such factors are known, he wrote, user-experience designers can create models of those prototypes for that particular place. This is a two-step process identifying both prototypes of context and prototypes of objects (items or individuals). The goal of this part of the process is to develop representations of what features or characteristics the context should have, including through the mobile interface. This information is critical to successful modeling of prototypes.

Through such discovery steps, user-experience design has been found to be a continual conversation with the external world and its artifacts, with oneself, and with other people, including a design’s intended and unintended audiences. The most successful interactions of this type usually have been found to happen when the user is in control of the activity, can test ideas by performing experiments, and can ask questions, collaborate with others, seek out new knowledge and plan new actions (Rudman, Sharples, et al., 2008, p. 149). Responsive interaction design, in turn, seeks to establish a dialogue between products, people, and physical contexts, in order to anticipate how the use of in-situ content will affect comprehension, and to determine appropriate forms. The complexity of such a design process, the unpredictability of usage, and the rapid development of technologies as design materials make concrete examples of good interaction the only stable foundation for future prototypes and models of place-based experiences (Gottleib, 2008, pp. 167–175).

The origins of this study actually started thousands of miles away from Hawaii, on the Blackfeet Nation’s tribal grounds in northwest Montana, around Browning, just east of Glacier National Park. In that vicinity, a research team composed of freelance journalists, filmmakers, and scholars from Washington State University, University of Oregon, and the University of Hawai‘i (also this paper’s author) worked together in the early 2010s on a documentary film and experimental locative-media mobile app. This free app— available for both Android and iOS—commemorated the 50th anniversary of the worst natural disaster in Montana’s history, an enormous flood that killed 30 Blackfeet tribal members, many of whom were children. The ‘64 Flood app, through its short films and reporting, told a story of the effects of the flood, as a retrospective but also from the previously neglected Blackfeet perspective. Most relevant to this paper, it also digitally embedded all of that media into particular GPS locations around the tribal grounds, based on either where the stories took place, or where the storyteller shared the information. The ‘64 Flood app was released in 2014, and its code and simple interface designs also were aimed at constructing a flexible new platform, from which these types of interactive locative-media stories could be delivered elsewhere but also as a springboard for further innovation and research on this topic area. A few months later, I applied for a neighborhood grant of $2,000 (from Malama Mānoa) with the idea of repurposing the code by stripping the content from the ‘64 Flood app and adding new types of journalistic content, of an undetermined nature. From there, the five students in that original special-topics class in 2015 generally followed the creative process later outlined under the umbrella of St.Amant’s categories of development (2018b) through iterative action-research cycles outlined in Figure 1. Action-research cycles typically follow a progression from idea to planning to action/ intervention, which is observed and recorded through empirical means, followed by a reflection period, in which the original idea and plans are modified based on the findings, and a new cycle of action research begins (Elliott, 1991; Herr & Anderson, 2005; Anderson & Shattuck, 2012; Jones, Scanlon, & Clough, 2013; Liaw, Hatala, & Huang, 2010; Sharples, Taylor, & Vavoula, 2010).

I worked with the students in the class, in the formal First Space of the classroom, to conduct the following prototype-mapping steps:

Step 1: Identify a piece of public art in Mānoa that could serve as the hub of an interactive piece of locative media.

Step 2: Spend significant time, at least a couple of hours, just in the presence of the piece, getting to know it, and examining it carefully, including its setting, taking notes about who happens to pass by and what people typically do while in the vicinity of this artwork.

Step 3: Furthermore, watch carefully how people interact with the artwork (or not) now, and think about how a mobile device and locative media could enhance this experience in significant ways. Interview representative users, about their potential interests in the public art, when possible.

Step 4: Determine the news values of those ideas (Caple & Bednarek, 2015), in terms of what makes each experience “journalistic,” and develop the journalistic questions it will ask of sources as well as the story it will tell audiences. In this case, in the information neighborhoods we explored, including entertainment, public relations, cultural critique, etc., we pursued what we thought were the most journalistic of the ideas generated.

Step 5: Determine precisely how spatiality—and the interplay between digital and physical data gathering at this location—is important to this experience and refine the plan to emphasize that element.

Step 6: Determine how sociability—and the interplay between people present and not present—is important to this experience and refine the plan to emphasize that element, too.

Step 7: Learn more about the history and context of the piece. Who made it and why? How does this piece fit into the Mānoa Public Art collection, but also how does it fit into the broad art scene, in the area, in the state, in the country, and in the world?

Step 8: Create the learning objectives for what audiences are supposed to take away from this experience.

Step 9: Create a detailed production plan for how and when this work will be done, and by whom, setting expectations, roles, and hard deadlines.

Step 10: Identify any significant challenges that the team might have in completing this production plan, creating a risk analysis of those challenges.

Step 11: Research the story as a journalist would, including gathering observational and testimonial data as well as other types of documentation to triangulate the validity of the work.

For example, in front of the University of Hawai‘i’s pillared administration building, Hawaii Hall, is a traffic roundabout, called Varney Circle. In the middle of that traffic circle is a fountain. This fountain has been dry for at least a decade, growing black mold on it, and weeds around it. Instead of serving as a source of pride, and as a symbolic fount of wisdom and vitality, this overgrown central campus architectural figure fades into the weeds. Students on this project, curious about the fountain’s degradation, like I had been, decided to investigate the history of it and discovered that students at the school about a century earlier had pooled their money to build the fountain in honor of a beloved teacher, Ada Susan Varney. They also found other interesting footnotes, such as about its tiki motif, which is not native to Hawaii and might be the oldest existing example of the symbology in the state. Those were leads to developing both prototypes of context (articulating historical connections to the place) and object prototypes (like how this tiki design compares to others in the area) that could be synthesized into a broader prototype of place, where we might expect a certain collective experience, including water flowing out of the fountain.

Another novel prototype was developed around The Founders Gate, which consists of two cast-stone arches, with built-in benches sculpturally creating curved arms, welcoming visitors to the area. This gate—dedicated in 1933 and placed on opposite corners of University Avenue, a main campus thoroughfare—was created as a memorial to the founders of the university and intended to serve as an entryway to the campus. Yet the development of the university grounds did not happen as projected, and the gates now, almost a century later, seem out of place, disorientingly leading to nowhere in particular. A prototype of context in this case would suggest that a person finding these gates would also expect to find a purpose for the gates, as in a portal to a new area. With that purpose absent, an opportunity for locative media emerges. Taking these types of foundational prototype-mapping steps can make low- fidelity instantiations and user testing more efficient and effective by theoretically predicting areas of richest opportunities.

Prototype Testing: Mānoa Public Art

For Prototype Testing, St.Amant (2018b) suggested focus groups and interviews as potential methods of assessing the effectiveness of a prototype of place. For our work here, we have done both of those types of inquiries about place at the conceptual level, where St.Amant aimed his attention, but also one step beyond, with the creation of physical prototypes as well, which realized these prototype ideas in complementary abstract and concrete forms. We used both types of prototypes (conceptual and physical) for interviews and in focus groups. Those physical prototypes allowed for representative users to interact with authentic instantiations of mobile media in the settings for which they were designed, which we thought was a valuable extension in a prototyping process. Those developmental steps in this prototype-testing process thereby included:

Step 1: After interviewing representative users on site, about their potential interests in the public art, create a paper prototype to reflect those interests. Build the user-experience story and its related mobile interactions on index cards, representative of the size of a typical smartphone screen. These cards are intended to show exactly what the designer envisions the interface and interactions to be, as in what the user will see and do. That prototype includes sketches of text blocks, images, embedded video boxes, buttons and even gesturing functionality (such as the ability to swipe or pinch and zoom). The cards later will get moved by hand, by the user, based on how the user understands and interacts with the prototype. This prototype process is a critical step in developing an understanding of what the locative-media experience will be and how the audience will experience, understand, and respond to it. Test the paper prototype on yourself and colleagues multiple times first, before sharing with others, until confidence is high that it will work in the field setting. Then, test the prototype on less- connected others, such as friends outside this research area, before taking it into the field, to get a better sense of how it will work with strangers. Practice the testing process until it seems adequately controlled and purposeful.

Step 2: At the location of the artwork, next give that prototype to five representative users—people who you would envision as a part of your primary audience, per rapid prototyping techniques pioneered by Nielsen (2000)—and have each one try out the app, on paper, without any coaching or direction beyond, “What would you do next?” Use think-aloud protocol to learn why the user made each choice, with prompts roughly limited to: “Why did you do that?”

Step 3: Refine the paper prototype based on the feedback and then test again, with a new set of five representative users. Repeat as many times as necessary, until the data received is familiar and mostly routine, not providing any significant new directions for improvement.

Step 4: Refine the plan, again, and build the prototype digital media for the mobile app, with clear instructions about when particular media is pushed to the user and when other media is pulled from the user, and install this plan and its media into the prototype mobile- app platform. Create an analytics plan and suggest other types of research metrics that can be used in mixed-media experiments to gather data about how the plan is working. Then, test the experience again on site with the prototype mobile app this time.

Step 5: Repeat as many times as necessary, or pragmatically possible, until the data received is recurring in patterns and familiar. When the story is ready for further study, release it to the public via the Mānoa Public Art app to see how it fares.

The first Mānoa Public Art-related class, in the fall of 2015, generated six interactive mobile stories that were implemented in the debut version of the public app (Figure 2).

Left: This screenshot shows the landing page of the Mānoa Public Art app, which has the name of the app at the top and a horizontal high-level image of Oahu underneath, including a view of the city, the ocean, and its landmark Diamond Head mountain. Below the Hawaii image is a brown bar that reads “Pick Your Path,” and below that bar are square thumbprint-sized images used to select the mobile story. They are stacked two to a row, with two columns. The first row, from left to right, shows “Varney Circle” in the first square, and “Air” in the second. The Varney Circle button shows a circular fountain with water pouring out of spouts on its rim. A tiki motif decorates the fountain’s walls. The Air button shows a section of a mural, and in this section, two gigantic hands surround a brown-skinned person, with his back to the viewer. He is nude, except for the traditional Hawaiian malo, or loincloth, that he wears. The two buttons below show “Earth” and “Fire” murals, also featuring symbolic Hawaiian imagery, of people with mythological creatures. This screen shows the original version of the app, which is contrasted in the second image by the current version. Right: The contrasting image in this pairing shows the same basic interface screens, only with different buttons highlighted this time. The buttons shown in this current version of the app show a “Chinese Zodiac Sign” sculpture, a “Mayan Warrior” mural, the “Stevenson Hut,” and “Spirit of Manoa,” with measurements after each artwork’s title, such as 2764 Feet S, 4724 Feet SW, etc., indicating how far away the user is from the artwork and in what direction.
Figure 2: The original Mānoa Public Art app home screen (left), compared to a more-recent version of the home screen, which shows how story tiles since have been geolocated based on the user’s location, individually sorting the artwork to the user by proximity.

As of this writing, in the spring of 2020, about 20 more undergraduate students have volunteered to work on this project, outside of their normal coursework, in three different media-production classes. In addition, the latest group of students to work on this project in a class were nine graduate students in the fall of 2019. They not only each added a new locative experience to the app, one student converted an Architecture School exhibit on “Tropical Modernism” into 10 additional pieces of mobile media, connecting art to architecture. Malama Mānoa also contributed another $2,000 grant to the project, its fourth, bringing its total support to the project to $8,000, all spent on subcontracted programming costs. The media- production labor of this project was in-kind, either by faculty or students. At this point, we have published 100 mobile-media stories through this app, of varying depths and purposes, but all designed as research instruments.

Highlights, findings, and lessons learned from those efforts will be described in more detail in the next section of this paper, including descriptive details about the novel locative forms generated.

FINDINGS

Based on the research questions used to shape this inquiry, findings will be reported in two sections. The first, based on responses to RQ1, will focus on ways place can matter in user-experience- design prototyping. The second, based on responses to RQ2, will focus on novel forms of locative media, and what can be learned from experimenting with those, in terms of opening more or wider paths of further research inquiry in this area.

Responding to RQ1: About place and prototyping

One of the key findings in this case is that what appeared to be a separation and solid boundaries between a designer and a user actually, in effect, becomes a dynamic series of reflections and adjustments based on feedback, both ways. The designer responds to users in place, and the users seem to grow in curiosity about the locative-media form as it gets closer and closer to what each person intuitively wants. In other words, the design context is more fluid than fixed and certain, requiring a reflective and adaptive design paradigm to address and guide choices. Mobile media, with its responsive nature, appears to be well-suited for such work.

Despite a curricular emphasis on stretching boundaries of this field, through experimental prototypes, though, these design experiences all eventually fell into somewhat predictable patterns, constrained by early design choices in the app and also by resources (support funds, technical skills, time, etc.) available to do this work. In addition, by limiting our work to what could be provided through a smartphone, and having no possibility for impact on the physical setting, we were limited to roughly half of the design space. In an ideal context, we would be able to adjust both the physical and the digital, based on user responses.

In the digital frame, for example, once we determined that a menu of thumbnail-sized tiles would show the artwork available in the app, each artwork then had to be represented in a small square image. Once the mobile-mediated story was chosen and opened, we kept coming back to contemporary publishing traditions of starting the experience with a photo, a headline, and a block of text for orientation. Sometimes, the story became nothing more than a couple of photos, with accompanying texts, that either accentuated aspects of the art or layered historical images onto the present view of the piece, to create a then-and-now effect.

Most of the truly experimental progress, testing and challenging the locative nature of the media, came through one of two means, in communion with the public art: Site-specific multimedia (audio, video, 360-degree video, etc.) designed explicitly to be experienced in place, with prompts to interact directly with the particular place, or calls for specific interactions (such as “take and share this” type of photo) related to the place. All of these instruments are available for public viewing through the free apps. But a couple of the examples, related to the call for action of RQ2, are detailed in thicker descriptions below.

Responding to RQ2: About novel prototypes and lessons learned

The locative-media experience created in the Mānoa Public Art app around Varney Circle, for example, included historic photographs, aligned with what the person physically could see as well, creating a layering effect common in augmented reality. One particularly powerful photo sequence in the module showed the fountain as it is today, empty, cracking, peeling, followed by an image that most people on campus have never seen before, showing the feature, on a sunny day, with students happily lounging around it, while water pours off its spouts (Figure 3). The sociability aspects of this experience then provided the kicker, offering a link to the user to sign an online petition to ask the administration to fix this fountain. The app also offers other interactive options, for sharing photos, video, audio, or text about the situation through social media.

Left: This black-and-white photo shows three young women, with hairstyles and clothing indicative of the 1950s, lounging around the Varney Circle’s fountain as water flows out of its spouts. Two of the women are seated in front of the fountain, on its curbing, reading a book together. The third is looking down at her friends. One of the tiki symbols is clearly visible. Hawaii Hall is in the background. This image shows a time when the fountain was a beloved gathering place on campus. Right: This screen shows a horizontal view of the Varney Circle fountain, illustrating what it looked like with running water and blooming flowers around it. The text on the page asks the user of the app if “you want water running out of Varney Circle again” and then provides a prompt to sign an online petition. The screen also offers ways to take and share a photo, a video, an audio, and a text.
Figure 3: Varney Circle module screens.

Additional lessons learned during the Varney Circle module development and testing process, included: From a traditional news-value perspective (Caple & Bednarek, 2015), the story of a longtime broken fountain is potentially newsworthy—depending on the publisher and the intended audience. That said, though, if an audience member is standing near Varney Circle, with smartphone in hand, how much more interesting might that story be to the proximate individual (Oppegaard & Rabby, 2016)? Likely much higher than when that same person is elsewhere, but traditional understandings of news and media do not encompass ideas about how emerging technologies can change the scale and proportions of information, as a type of place-based learning, calibrating information so precisely to the individual that even turning a body 90 degrees could open up new learning possibilities via digital technologies. This latent power to engage and motivate an audience member, based on proximity, deserves much further attention.

After the Varney module was uploaded to the public version of the app, I asked one of my graduate-level research assistants, who is blind, to try it. She had spent several years on this campus, and the Office for Students with Disabilities, which she frequented, is adjacent to the traffic circle. But she was puzzled by my suggestion that there was a fountain located in that area. She said that she had never heard any running water, and she also explained that when she crossed streets, she went from corner to corner, not across the middle of wide roads. She had no idea a fountain existed in the middle of that traffic circle, or that anything of significance was there. So, for her, it wasn’t. Which is a lesson for all media design, if a user cannot see or find the media, it might as well not exist. In addition, the media in this module was not designed with a visually impaired person in mind, having no sound elements. The experience for a blind person, then, becomes just text and alt-text descriptions of the images being read aloud by a machine-voiced screen reader, without any acoustic aesthetics. The field test with the research assistant therefore showed weaknesses in our design plan in terms of media accessibility.

When a community asset, like this fountain, has become functionally invisible due to neglect, it also does not just naturally prompt people to look for a mobile app about it, either. Our site investigation revealed that smartphone use in this area was very high, with buses stopping regularly, and dozens of people regularly loitering around this area, looking at their phones while waiting for transportation. We did not see anyone, though, just randomly using our app during many site visits. So unless a significant marketing plan was in place, including physical prompts to download and try the app, this type of learning opportunity likely would go unrealized in its potential, sort of like the fountain itself, because it would not automatically register with passersby as an available learning opportunity.

Students working on The Founders Gate module, as another example, decided to experiment with the idea that the gates might not be moveable physically (they are large and made of stone and on the historic register), but they could be relocated virtually, by taking a mobile photograph of a new location and with the app automatically superimposing the gates on top of the picture, to see what they would look like elsewhere (Figures 4–5). Users then can share the image of the gates via social media and compare ideas. As an artwork representing transition, the gates are meant to designate leaving one place and entering another. In a similar way, this module was intended to shift around a sense of space.

Left: This screen shows a black-and-white aerial image of the University of Hawaii campus in the early 1900s. There are few buildings and structures on the campus. But in the lower-right corner, at the intersection of two roads, there is a red box around two dark specs that are labeled as the Founders Gate. It shows the position of the gate in the early days of the campus. Right: This interface screen has a horizontal image in the top third that shows part of a Founders Gate lamp, illustrating its glass panels and metal work. The lower two-thirds of the screen contains instructions for how to take a photo on the current campus and to let the mobile app overlay representations of the gate, to show what they would look like in another location, an example of which is shown in Figure 5.
Figure 4: Founders Gate module screens
This color photograph shows the outcome of using the app tool in Figure 4, which is an image taken on campus (of the pillared Hawaii Hall landmark), only with half of each Founders Gate on each side of the image, giving the impression that the gates have been moved to this location and walking through them, across a manicured lawn, will take the app user to Hawaii Hall.
Figure 5: The Founders Gate overlay on a photo of Hawaii Hall, showing how the gates would look here.

Other significant lessons learned during the design of The Founders Gate module, related to RQ2, included: The idea that physical positioning matters greatly in these types of locative-media situations. The Founders Gate has two mirror-image gates, one on each side of a busy six-lane road. These therefore are two related pieces in different locations, within sight of each other. But to get from one to the other requires crossing heavy traffic and walking a significant distance. Since Hawaii is sunny most of the time, this crossing area is in the direct sun, and one gate has far less shade than the other. When considering what we were asking the audience to do, which is stand by a gate, look it over, think about the digital media, in comparison to the physical setting, etc., we had to decide if both gates would have the same experience, or if one would be different than the other. We also had to decide where the content would be triggered by the smartphone in this location. Too small of a prompting target would mean many people might miss the content altogether. Too large and the proximate effects of being near the gates would be diminished. In some of our early paper prototypes, representative users also ended up uncomfortably close to the road, or standing in the sun for long periods of time, reading the texts and looking at the images, while trying to avoid glare on the screen. Comfort became a consideration, and so did safety. Also, because of the traffic noise, some times of day, when the traffic was heavy, were much more unpleasant than others.

IMPLICATIONS FOR COMMUNICATION DESIGN

Emerging technologies—many supported by, or connected via, commonplace smartphones—dramatically have changed affordances for scholars to create and analyze communication. Prototypes unbuildable or even unimaginable 15 years ago now can be in-hand in a semester’s worth of work. Viewed from an insider’s positionality, designs can transcend original intent and address complexities of locative media otherwise unapproachable. For example, other faculty, staff, and community members have found this project’s app useful for diverse learning purposes that illustrate its potential as a multi-faceted research instrument. One of the most surprising examples of its unintended use was for a 400-level Second Language Studies class. The interdisciplinary researcher leading that class, Dongping Zheng, also studies locative mobile apps as language-learning tools at UH. She learned of the Mānoa Public Art app and wanted to try it with her students. To test it, there were 16 of them who traversed the campus with the app in hand, stopping at Varney Circle and Founders Gate, among other modules in the program. Several of them remarked that even after years of attending University of Hawai‘i, they had barely noticed the gates before. One wrote in a report afterward, “I never really cared about the Founders Gate and did not even know it was called that, but I learned that the Founders Gate was originally intended to lead people into (the) University campus rather than the Mānoa neighborhood. Because I now know this, there is significance in the gate being there, and (I) take a different perspective on it.” In just that short response, the student suggested concrete learning had happened and that many possible variables in this sort of experience potentially could be operationalized, measured, and analyzed, including engagement, motivation, and contextual knowledge.

The reports the SLS students wrote about their experiences (Figure 6) afterward offered much evidence that learning was happening in complex and dynamic ways, worthy of further research. At another location, for example, the app offered GPS-triggered context about four murals spread throughout the chemistry building, representing the elements of air, earth, fire, and water from a Hawaiian mythological perspective. One student remarked, “I spent a lot of classes in Bilger Hall, not even aware that four murals were painted on the walls. Amazingly enough, during my final semester at UH, it was this app that not only directed me to each of those four murals, but gave me the history of each one, their meanings and even the artist who created them. Needless to say, I did have a good time using the app with my classmates.” Another student, when documenting his experience in front of a specific mural, wrote, “I see so many different connections between one another (of the images), thinking of questions and thoughts that would not have been (brought) up in a regular conversation. For example, looking at the Earth painting of Ku, one student asked, ‘Why was he planted upside down? At that moment, several of us were wondering as well and languaging to each other of the possibilities as to why.”

Left: This vertical color photograph, taken during a field test of the Manoa Public Art app, shows a half-dozen students in the foreground and the Varney Circle fountain and Hawaii Hall in the background. The students are looking at their smartphones and engaging with the locative media that the app provides in that place. Right: A horizontal color photograph, this image shows two students, different students from the first image. They are seated on a bench with the “Air” mural in the background, conducting another field test of the Manoa Public Art app. The male student in the middle is holding his smartphone and talking with a female student to his left.
Figure 6: Second Language Students interacting with the Mānoa Public Art app, near Varney Circle and in Bilger Hall.

IMPLICATIONS FOR SCHOLARS

While every research project like this will be somewhat different from any other, in some ways, this one had key infrastructure in place to insulate it from some of the basic technical, organizational, and production issues. No. 1, this was not my first mobile-app research project (having focused my dissertation on one as well, plus other small app experiments, including with students as collaborators), and I was supported by professional programmers who also work daily in the mobile-development environment. They were paid to do this programming job. Their expertise and professionalism absorbed and made invisible many potential obstacles that might have otherwise been in our path. In addition, many of the technical issues of the core Mānoa Public Art app interface and functionalities actually were worked out during the earlier ‘64 Flood app project, over a couple of years, and in earlier research projects, so any appearance of a relatively rapid learning curve for Mānoa Public Art might be misleading. This project really has been integrated into a nearly decade-long research agenda based on inquiries into locative and mobile media, including through analysis of related ethical issues (Oppegaard, 2020), on top of two decades of experience working as a professional journalist.

At its foundations, though, this study created a replicable intervention that illustrated benefits of community-engaged scholarship as well as the multi-faceted and multi-directional learning potential of locative media (journalistic or not) in informal Third Space contexts. This case also has a durable and long-term development model that is relatively inexpensive to maintain and flexibly designed to be able to easily adapt to emerging locative- media ideas, which then can be conveniently examined in-situ (just outside the buildings where the researchers work), with authentic audience responses and plentiful representative users to choose from (anyone in the vicinity of the artwork, willing to give the app a try). In turn, the Mānoa Public Art research project has reached a certain level of maturity and robustness that indicates its potential without realizing all of it.

FUTURE PLANS

At this point, most of the locative media in Mānoa Public Art is silent. There are few audio or video files available, because of production costs and technical skills necessary to make them, although the app regularly asks audience members to make their own audio or video clips in response to the art (and app prompts) and share those via social media. A better balance of media types might enrich the app, in which the app, for example, provides an audio clip about a particular nearby artwork, then, as a social-capital exchange, asks for a place-made audio clip in return. Besides the inclusion of more types of media, the app could offer more accessible media as well, with audio description instead of alt-text, for example, or sound options as substitutes for text.

For all of the things that the Mānoa Public Art app does well, which have been documented at some level of depth here, the research project also has many opportunities for improvement. Most commonly, the criticisms of the app from users are focused on the lack of a global storyline, other than just “learning about public art,” and its nonlinear and noncompetitive structure. While there are benefits in this structure for independent learners with high curiosity and a willingness to wander, other types of learners might reach deeper engagement levels through the application of gamification techniques or the implementation of linear tours or some kind of explicit and connective underlying storyline. This learning experience also might benefit from more designs that try to prompt offline but related discussions, among people using the app in small groups, such as the “why is he buried upside down” question that generated a lively group discussion.

The complex dynamic among people, places, communication, technologies, mobilities, etc., clearly needs more scholarly attention from applied-research perspectives, including from an insider’s positionality. Studies of mobile media in communication contexts that ignore embodiment and physical aspects of digital media— such as the environmental settings in which they are emplaced—are chasing ghosts of sorts, seeking to focus on ungrounded phantasms lacking forms or features or footings. If designers succeed through deeply understanding their audiences, those audiences must be understood in context, and in place. Locative technologies, such as those available in smartphones, have emerged with sensors and tools for building and studying prototypes like never before. Researchers can envision contexts and audiences through these prototypes. But first, they need to build them.

MOBILIZATION LINKS

The Mānoa Public Art app is available for free in both the Android (https://goo.gl/V1k2Tb) and Apple (https://goo.gl/Yb9x1L) markets.

ACKNOWLEDGEMENTS

Thank you to Malama Manoa for the financial support of this project, UH Associate Professor Dongping Zheng for her classroom collaborations, and to all of the students who have contributed original content to the MPA app over the years, including: Winlove Almazan, Marc Arakaki, Cynthia Arata, Anna Barrett, Leslie Betancourt, Katie Boon, Kapiolani Ching, Chanel Dias, Bronson Doria, Eunica Escalante, Kaainoa Fernandez, Mari Galiher, Ashley Gomez, Jacob Hensley, Jessica Homrich, Tracy Kim, Shannon Manamtam, Ivy Martin, Andreas Miguel, Harrison Patino, Godwin Polendey, Brandon Rivera, Jim Bea Ildefonso Sampaga, Aven Santiago, Nicole Tam, Melissa Tiogangco, Glenn Ugalde, and Laura Young.

REFERENCES

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178. https://doi.org/10.1207/s15327809jls0202_2

Campbell, S. (2018). From frontier to field: Old and new theoretical directions in mobile communication studies. Communication Theory, 29(1), 46–65. https://doi.org/10.1093/ct/qty021

Campbell, S. (2020). Cutting the cord: Social and scholarly revolutions as CMC goes mobile. Journal of Computer-Mediated Communication, 25(1), 101–110. https://doi.org/10.1093/jcmc/zmz021

Caple, H., & Bednarek, M. (2015). Rethinking news values: What a discursive approach can tell us about the construction of news discourse and news photography. Journalism, 17(4), 435–455. https://doi.org/10.1177/1464884914568078

Casey, E. (2013). The fate of place: A philosophical history. University of California Press.

Cresswell, T. (2004). Place. Blackwell Publishing.

Crompton, H. (2013). A historical overview of m-learning: Toward learner-centered education. In Berge, Z.L., & Muilenburg, L.Y. (Eds.) Handbook of Mobile Learning (pp. 3–14). Taylor & Francis.

Design-Based Research Collective (2003). Design-based research: An emergingparadigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005

de Souza e Silva, A. and G. Delacruz (2006). Hybrid reality games reframed: Potential uses in educational contexts. Games and Culture 1(3): 231. https://doi.org/10.1177/1555412006290443

Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In Spector, J. M., Merrill, M., Elen, J., & Bishop, M. (Eds.). Handbook of research on educational communications and technology (pp. 735–745). Springer.

Elliott, J. (1991). Action research for educational change. Open University Press.

Fan, L. T. (2017). Writing while wandering: material and spatial contingency in locative media narratives. Convergence, 23(1), 5–19. https://doi.org/10.1177/1354856516679635

Farman, J. (2012). Mobile interface theory: Embodied space and locative media. Routledge.

Frith, J. (2015). Smartphones as locative media. Polity.

Gottlieb, H. (2008). Interactive adventures. In L. Tallon (Ed.), Digital technologies and the museum experience: Handheld guides and other media. AltaMira Press.

Henze, N. (2012). Hit it! An apparatus for upscaling mobile HCI studies. In CHI’12 Extended Abstracts on Human Factors in Computing Systems (pp. 1333–1338). Austin, TX: CHI’12. https://doi.org/10.1145/2212776.2212450

Herr, K., & Anderson, G. (2005). The action research dissertation: A guide for students and faculty. Sage Publications, Inc.

Isa, S., & Liem, A. (2020). Exploring the role of physical prototypes during co-creation activities at LEGO company using case study validation, CoDesign, https://doi.org/10.1080/15710882.2020.1715443

Jay, M. (1993). Downcast eyes: The denigration of vision in twentieth-century French thought. University of California Press.

Jones, A., Scanlon, E., & Clough, G. (2013). Mobile learning: Two case studies of supporting inquiry learning in informal and semiformal settings. Computers & Education, 61(1), 21–32. https://doi.org/10.1016/j.compedu.2012.08.008

Kearney M., Burden K., & Schuck S. (2019) In L. Daniela (Ed.) Disrupting education using smart mobile pedagogies (pp. 139–157). Springer Cham.

Knight W. (2019) UX for developers. APress.

Liaw, S., Hatala, M., & Huang, H. (2010). Investigating acceptance toward mobile learning to assist individual knowledge management: Based on activity theory approach. Computers & Education, 54(2), 446–454. https://doi.org/10.1016/j.compedu.2009.08.029

Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: An experimental study on the use of mobile technology. Computers & Education, 68(1), 76–85. https://doi.org/10.1016/j.compedu.2013.04.021

Massey, D. (2013). Space, place and gender. University of Minnesota Press.

McLuhan, M. (1964). Understanding media: The extensions of man. McGraw Hill.

McMillan, D., Morrison, A., Brown, O., Hall, M., & Chalmers, M. (2010). Further into the wild: Running worldwide trials of mobile systems. In International Conference on Pervasive Computing (pp. 210–227). Springer Berlin.

Mobile fact sheet. (2019, June 12). Pew Research Group. Retrieved from: https://www.pewresearch.org/internet/fact-sheet/mobile/

Nielsen, J. (2000). Why you only need to test with 5 users, Alertbox: Current issues in web usability, March 19, 2000, retrieved Jan 15, 2019, from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Oppegaard, B. (2020). Locative media ethics: A call for protocols to guide interactions of people, place, and technologies. Journalism & Mass Communication Quarterly, 97(1), 13–15. https://doi.org/10.1177/1077699019898773

Oppegaard, B., & Rabby, M. (2016). Proximity: Revealing new mobile meanings of a traditional news concept. Digital Journalism, 4(5), 621–638. https://doi.org/10.1080/21670811.2015.1063075

Rainie, H., & Wellman, B. (2012). Networked: The new social operating system. MIT Press.

Robinson, J., Lanius, C., & Weber, R. (2018). The past, present, and future of UX empirical research. Communication Design Quarterly, 5(3), 10–23. https://doi.org/10.1145/3188173.3188175

Rudman, P., Sharples, M., et al. (2008). Cross-context learning. In L. Tallon (Ed.) Digital technologies and the museum experience: Handheld guides and other media. AltaMira Press.

St.Amant, K. (2018a). Editor’s introduction: Reflecting on and re-thinking usability and user experience design. Communication Design Quarterly, 5(3), 4–9. https://doi.org/10.1145/3188173.3188174

St.Amant, K. (2018b). Reflexes, reactions, and usability: examining how prototypes of place can enhance UXD practices. Communication Design Quarterly, 6(1), 45–53. https://doi.org/10.1145/3230970.3230976

Sharples, M., Taylor, J., & Vavoula, G. (2010). A theory of learning for the mobile age. Medienbildung in Neuen Kulturräumen, 87–99. https://doi.org/10.1007/978-3-531-92133-4_6

Tuan, Y.F. (1977). Space and place: The perspective of experience. University of Minnesota Press.

Verhulsdonck, G. (2018). Designing for global mobile: Considering user experience mapping with infrastructure, global openness, local user contexts and local cultural beliefs of technology use. Communication Design Quarterly, 5(3), 55–62. https://doi.org/10.1145/3188173.3188179

Wilken, R. (2008). Mobilizing place: Mobile media, peripatetics, and the renegotiation of urban places. Journal of Urban Technology, 15(3), 39–55. https://doi.org/10.1080/10630730802677939

Wilken, R., & Goggin, G. (2017). Mobilizing place: Conceptual currents and controversies. In J. Farman (Ed.), Foundations of mobile media studies: Essential texts on the formation of a field (pp. 50 – 69). Routledge.

ABOUT THE AUTHOR

Brett Oppegaard is an associate professor in the School of Communications at the University of Hawai‘i at Mānoa. His research interests converge at the intersections of Technical Communication, mobile technologies, and media accessibility. He is especially interested in how accessible mobile media can inscribe and create a perception of “place.” He is the principal investigator on The UniDescription Project, www.unidescription.org.

Prototyping and Public Art: Design and Field Studies in Locative Media
Tagged on: