University of Central Florida
Sonia H. Stephens
University of Central Florida
In this project experience report, we describe our experience working as researchers specializing in technical communication that informed the risk communication decisions for an interdisciplinary, grant-funded, risk communication website called HazardAware. We first discuss how content audits serve as a website design research method. Next, we provide our Content Audit Selection heuristic in a process flowchart format to enable communicators to understand how practical application of content audits serve as a formative tool to streamline the decision-making processes for complex website design content. Finally, we describe how we used the Content Audit Selection heuristic to inform the risk communication decisions for HazardAware.
When designing a website with complex scientific content, professional and technical communicators (PTCs) on a project team are often responsible for determining information usability. In interdisciplinary team projects, PTCs usually face additional challenges evaluating and communicating complex information designs both within the project team and with target audiences. This experience report describes how we used web content audits to inform the design of “HazardAware”—a website that is being developed as part of a three-year funded interdisciplinary project to provide information on natural hazards to help U.S. Gulf Coast renters, homeowners, and homebuyers increase their hazard resilience as it relates to housing. We describe the methods that we used for our content audits, provide a heuristic for using content audits in multisector or complex communication contexts, and discuss implications for our project and for communication design in general.
This project was developed in a risk communication context, specifically communication about natural hazard exposure and risk mitigation related to personal housing in the U.S. Gulf of Mexico states of Texas, Louisiana, Mississippi, Alabama, and Florida. This region is not only exposed to multiple natural hazards, most significantly severe storms, hurricanes, and floods, but also extreme heat, wildfire, and sinkholes (Gall et al., 2011). Per capita financial losses continue to increase over time despite government investment in mitigation (Gall & Friedland, 2020). Beyond direct financial losses, natural hazards affect people’s long-term wealth capacity, community ties, and mental and physical health. A community’s resilience to hazards measures its ability to “prepare and plan for, absorb, recover from, and more successfully adapt to actual or potential adverse events in a timely and efficient manner” (Cutter et al., 2014, p. 65). Importantly, individuals’ hazard resilience is mediated by personal, community, and governmental decisions, as well as historic social factors and the physical environment (Summers et al., 2018).
This project’s purpose is to gather, provide context for, and develop an index that summarizes hazard information for residential buildings in the Gulf region. Individual risk-related decision-making is affected by risk perception, which includes personal confidence in scientific understanding of a hazard, relative discomfort with a hazard, and the hazard’s potential impact (Fischhoff, 2009). In addition, it is important to note that “people’s risk perceptions are determined by real and localized situations” (Grabill & Simmons, 1998, p. 419), including their personal experience of risk (Stephens & Altamirano, 2021) and potential socioeconomic marginalization. The primary audience for the HazardAware website is individual residents who are interested in learning about and mitigating their risks; a secondary audience includes community planners, facilitators, and other organizational users who might use the information in a professional capacity. The website will provide information on the past hazard history and potential future hazard changes for residences in the study area. Website users will be able to look at a property, its overall risk, and how different hazards (e.g., wind or flooding) contribute to its risk. Then, they are able to see which options exist in terms of costs and benefits for mitigating hazards.
One challenge for the project team has been to determine how best to represent and provide context for different types of hazard data to make this information useful and engaging for users. The datasets the team members are working with vary in spatial and temporal scale; can be represented in text, numeric, or geospatial formats; and have broad thematic diversity. For example, the data includes historical information about past natural disaster locations and costs, current local building codes and state-level insurance information, and potential future changes to flooding due to sea level rise. In similar complex information settings, researchers have pointed out the importance of understanding possible transfer effects, such as users forming procedural expectations based on a new website’s similarity to a familiar one (Albers, 2009); carefully evaluating how to represent uncertainty, realism, and hazard impacts (Kostelnick et al., 2013); and balancing open user exploration with functions that support meaningful information seeking and decision-making processes (Richards, 2019).
Environmental risk communication involves diverse individual and social risk exposures and experiences that necessitate careful attention to communication design. The overall HazardAware design and development process includes several components to help the team understand both (1) user needs and preferences and (2) how online interactive data tools with a similar level of complexity have been structured. As the PTCs on the project team, our contribution focuses on helping the website development project team and the other investigators conduct formative and summative user-centered design (UCD) activities and conceptualize the design of the website. We used content audits for formative expert evaluation and combined this with scoping interviews with target users (Stephens & Altamirano, 2021) and summative usability testing to better understand audience needs, as suggested by Spyridakis et. al. (2005).
This experience report focuses on practical application of web content audits intended to inform our design understanding of similar interactive data visualization tools. During our search of the literature, we were challenged to find a large body of heuristics solely dedicated to content audits. In addition, the complexities of the project required us to leverage multiple content audits to achieve our goals. Therefore, this paper explains how our experience can serve as a framework for other PTCs in similar situations. Our primary goals in this paper are to: (1) provide an overview of content audits and their practical application in website design; (2) suggest a heuristic to help technical communicators incorporate content audits into complex website design; and 3) demonstrate how project teams can use content audits to inform website design for risk communication.
CONTENT AUDITS FOR WEBSITE DESIGN
Content audits are a research method that allows website design teams to systematically evaluate and analyze messaging and characteristics of a website or other product (Still & Crane, 2017), and can also be used as a decision-making tool to influence product design (Jones, 2009). Content audits are often positioned within a broader strategy for guiding website design. For example, Spyridakis et. al (2005) describes four types of methods for deriving and assessing web-design guidelines: (1) expert evaluations (including content audits) and automated evaluation tools, (2) usability tests, (3) surveys of users’ perceptions, and (4) true experiments conducted either in laboratories or remotely via the internet. When we perform a content audit to support a website design project, we review and document either specific components or everything on one or more websites. Miller (2019) points out that “the content audit is likely to be perceived as the most time- intensive process for improving usability, mainly because it involves looking at everything on the website” (p. 214). The time PTCs invest in performing content audits can be considerable. However, they provide valuable details that allow website development teams to discuss which services or information should be included on the website and why (Miller, 2019). The literature often uses the terms “content audit” and “content analysis” interchangeably (Detzi, 2012; Jones, 2009; Still & Crane, 2017). In this paper, we make a distinction between these terms as they suggest, in terms of informing website design, different uses. For example, in this study we conducted two separate content audits of websites that had different thematic characteristics, and then analyzed these findings. Therefore, this paper uses the term “content analysis” referring to how we synthesized findings from two content audits. For example, in this study we conducted two separate systematic evaluations of website content (i.e., two content audits), and then analyzed these separate results by synthesizing the findings into an overall report. We therefore use “content analysis” to describe this overall synthetic reporting process.
There are multiple types of content audits. For example, Sperano (2017) identified 23 types and concluded with the strategic recommendation that “the determination of a type of audit will vary depending on the objectives to be achieved through the audit of content” (p. 6). For this project, we chose different types of content audits to align with the primary goal that we were trying to achieve during that phase of the web design project. Table 1 highlights common types of content audits and a brief description of the purpose or focus for each.
|Comparative content audit (Rockley et al., 2003)||Content audit in which you compare like information among multiple websites (e.g., website content about a single product or service).|
|Competitor analysis (Still & Crane, 2017)||Like a comparative content audit, but instead centers on understanding competitor products with a goal to directly compete via product design.|
|Content quality (Land, 2014) or Qualitative audit (Halvorson & Rach, 2012)||Assesses various content-related criteria including relevance, consistency, messaging, and whether easy-to-read or scan to determine if it represents brand standards, guidelines, or personas.|
|Quantitative audit (Halvorson & Rach, 2012)||Provides an overview of website content including what type of content (text, images, video, etc.), how it is organized, and where it is located (e.g., URL and/or menu).|
|ROT (Redundant, outdated, or trivial) analysis (Bloomstein, 2012)||Helps you create more valuable content by identifying content that is redundant, outdated, or trivial.|
Guidelines for content audits are commonly found in checklist or heuristic format. For example, Brinck et. al (2002) provides a detailed usability-focused checklist consisting of four main audit categories, each containing a subset of questions that are designed to allow the reviewer to comprehensively review a website: (1) architecture and navigation, (2) layout and design, (3) content, and (4) forms and interaction. Nielsen (2020) offers a more general list of ten usability heuristics for user interface design, noting “they are called ‘heuristics’ because they are broad rules of thumb and not specific usability guidelines” (n.p.): (1) visibility of system status; (2) match between system and the real world; (3) user control and freedom; (4) consistency and standards; (5) error prevention; (6) recognition rather than recall; (7) flexibility and efficiency of use; (8) aesthetic and minimalist design; (9) helps users recognize, diagnose, and recover from errors; and (10) help and documentation.
When strategizing content audits for complex web design, PTCs need to focus on more than general content audit guidelines. While guidelines describe how to conduct a content audit, they do not describe how to integrate multiple types of content audits into the product design cycle. We argue that overall website design strategy requires a mixed methods approach for integrating various content-based elements, including usability, user interface design and information architecture (IA). We found that a mixed-methods approach supported the interdisciplinary, complex, and dynamic research for this project, which also encompasses a content domain that “often can be better informed by the use of multiple methods” (Johnson & Onwuegbuzie, 2004 p. 15). For similar complex projects, we recommend that PTCs implement content audits as a mixed-methods approach that complements a comprehensive iterative website design strategy. At each respective project stage, PTCs can select a content audit type that aligns with their website design goal(s). These findings then inform the next stage of testing in a “holistic approach” (Miller, 2019, p. 214) toward addressing problems in different stages of the project.
Examining the IA components of a website is key to understanding the features and functionality that designers can apply in their own project. Content audits provide communicators with a means of identifying, describing, quantifying, and assessing content (Getto et al., 2019), which they can use to examine website IA components including overall shape of information, information “chunking,” and navigation (McCool, 2006). While content audits are valuable at any time during the life of content (Halvorson, Rach, 2012), website design teams will typically conduct content audits during one or more website design stages, including during a website redesign or the initial stages of website wireframing. For the HazardAware project, our goals for the content audits were to inform our risk communication strategy by understanding (1) how risk was visually represented on websites whose communicative purpose was like that of HazardAware and (2) how risk was contextualized for target audiences. This included using the content audit to evaluate websites’: (1) informational content, (2) balance of images and text, and (3) interactive features (Detzi, 2012). The user experience (UX) design process is all about ensuring that no aspect of the user’s experience with the product happens without the designer’s conscious, explicit intent (Garrett, 2010). Content audits fit in to a broader strategy of UCD as an analytical tool to evaluate a website and are a first step in capturing important insights into UX that will impact the UCD choices that we will make on the project website.
CONTENT AUDIT SELECTION HEURISTIC FOR COMPLEX WEBSITE DESIGN
As described in the introduction, we identified several published heuristics for website design, but none focused specifically on how to incorporate multiple types of content audits into different stages of project development. Here, we present the Content Audit Selection heuristic (Figure 1), which is a process flowchart that recommends communicators working on complex website design projects: (1) identify goals, (2) determine the type of audit, (3) prioritize goals, (4) conduct the audit, (5) assess audit results, and (6) synthesize audit results.
The intent of our heuristic is to help professionals streamline the complexities involved with planning and/or evaluating content to inform website design. Focusing on one major project goal or initiative for each content audit (e.g., in Part I we focused on interactive mapping tools) yields several benefits for the entire project team. First, this helps to simplify the project’s complexities while prioritizing and fulfilling the website design project deliverables. Second, this makes it easier for PTCs to target interdisciplinary areas that intersect with website design (e.g., our content audits focused on usability, user interface design, web design and IA). By centering the decision on which of the project goals to prioritize first, this heuristic should facilitate the design of content audits to provide actionable insights that answer important questions for website design strategy. Finally, communicators can apply this heuristic during various stages during the website design production cycle simply by implementing a new content audit strategy. Ultimately, this heuristic offers a simplified problem- solving approach for communicating complex web-design content that supports the goals of both the internal and external project stakeholders (e.g., research team members, funding agencies, members of intended audiences). In the next section, we discuss how we applied this heuristic to our project.
USING THE CONTENT AUDIT HEURISTIC FOR RISK COMMUNICATION WEBSITE DESIGN
We used the Content Audit Selection heuristic and its corresponding six steps to simplify the risk communication complexities for the HazardAware website design. In the first step, we identified that our goals for the HazardAware content audits were to (1) examine how websites with a similar purpose visually depicted risk and (2) understand how (e.g., with visual or text annotations) these websites contextualized map-based information for target audiences. We conducted content audits during the formative stages of website design.
As we were concluding the initial content audit, other members of the project team shifted design focus. Rather than primarily focusing on interactive risk maps, they developed a communication strategy primarily focused on individual building-based information with, as a secondary component, community-level maps of flooding (and other hazards). Therefore, Part I of the content audit focused on risk information depicted on interactive risk maps, and Part II focused on textual and other types of visual communication conveyed on risk maps and real estate websites.
We implemented two types of content audits. Part I was a content inventory audit of flood risk websites that allowed us to perform quantitative and qualitative assessments of content (Martin et al., 2012). Part II was a comparative content audit of flood risk and real estate websites that allowed us to compare content features and functionality between flood risk and real estate website genres.
When choosing comparable websites to audit, we prioritized those that had the same intended audience as the HazardAware website, which included (1) property owners, renters, and prospective homebuyers and (2) community decision makers and policy makers. For the Part I audit, we focused on flood risk websites, both because flooding is one of the main hazards in the project area and because there are multiple existing websites that display flood- related information (Richards, 2019; Stephens et al., 2014, 2015). The Part II audit compared flood risk websites and real estate websites. Real estate websites have developed various strategies for exploring the features and functionality during the content audit; and, many individual residents (e.g., homeowners, homebuyers, renters) are likely to be at least somewhat familiar with using real estate websites to research purchasing and/or renting properties.
The Part I audit involved 11 flood risk websites, and Part II involved two flood risk websites and five realtor websites (see Table 2). Two flood risk websites were examined in both Part I and Part II; in Part I we focused on those sites’ interactive flood maps, and in Part II we focused on built-infrastructure features (primarily buildings). We collected, in accordance with the appropriate content audit, extensive data for each website. This included content audit website names, developing organizations, genres, the part of the content audit in which they were examined, and URLs.
|Website name, developing organization||Website genre||Content audit part used in||URL|
|Aqueduct Floods, World Resources Institute (WRI)||Flood risk||Part I: Content Inventory Audit||http://wri.org/applications/aqueduct/floods|
|Buyers Be-Where, Texas A&M University||Flood risk||Part I: Content Inventory Audit||https://portal.texascoastalatlas.com/buyersbwhere/landingpage|
|Coastal Resilience, TNC and various partners||Flood risk||Part I: Content Inventory Audit||http://maps.coastalresilience.org|
|Digital Flood Insurance Rate
Map of City of Galveston, City of Galveston, Texas
|Flood risk||Part I: Content Inventory Audit||http://arcgis.com/apps/webappviewer/index.html|
|FEMA’s National Flood Hazard Layer (NFHL), FEMA||Flood risk||Part I: Content Inventory Audit||http://fema.gov/flood-maps/national-flood-hazard-layer|
|Flood Factor, First Street Foundation||Flood risk||Part I: Content Inventory Audit & Part II: Comparative Content Audit||http://floodfactor.com|
|Flood Smart, FEMA & National Flood Insurance Program||Flood risk||Part I: Content Inventory Audit||http://floodsmart.gov|
|Louisiana FloodMaps Portal, LSU Louisiana State University AgCenter||Flood risk||Part I: Content Inventory Audit||http://lsuagcenter.com/floodmaps|
|National Storm Surge
Hazard Maps, NHC/CPHC
|Flood risk||Part I: Content Inventory Audit||http://nhc.noaa.gov/nationalsurge|
|Sea Level Rise Viewer, NOAA Coastal Service Center||Flood risk||Part I: Content Inventory Audit||http://coast.noaa.gov/slr|
|Surging Seas Risk Finder, Climate Central||Flood risk||Part I: Content Inventory Audit & Part II: Comparative Content Audit||http://riskfinder.climatecentral.org|
|Realtor, National Associations of Realtors||Real estate||Part II: Comparative Content Audit||http://realtor.com|
|Redfin, Redfin Corporation||Real estate||Part II: Comparative Content Audit||http://redfin.com|
|Trulia, Zillow, Inc||Real estate||Part II: Comparative Content Audit||http://trulia.com|
|Walk Score, Walk Score Inc.||Real estate||Part II: Comparative Content Audit||http://walkscore.com|
|Zillow, Zillow, Inc.||Real estate||Part II: Comparative Content Audit||http://www.zillow.com|
After completing a content audit, the next step for researchers is to perform a content analysis, which allows researchers to “analyze relatively unstructured data in review of the meanings, symbolic qualities, and expressive contents they have and of the communicative roles they play in the lives of the data’s sources” (Krippendorff, 2013, p. 49). After we completed each audit, we analyzed the findings, which allowed us to examine patterns and unique attributes within the data sources and how they structured their communication. We then synthesized our findings into a written report that we distributed to the HazardAware project team.
Part I Content Inventory Audit: Understanding Flood Risk Websites
PTCs have the flexibility to use a variety of strategies for content audit design (Sperano, 2017). Our initial strategy was to use the Part I content audit as a first step to capture important insights into UX that would impact the UCD choices that the website development team would make for the HazardAware website. We chose a content inventory audit because it allowed us to index content from natural hazard risk communication websites, identify the potential challenges that our target audience(s) could encounter while using a risk communication website, and understand the organizational structure of websites with similar target audiences, which we wanted to use to inform the IA of HazardAware. This approach helped us identify major topical content areas and explore possible organization schemes that we could use as a model for providing access to that content (Rosenfeld, Morville, & Nielsen, 2002). We had two goals for examining these data sources: (1) to understand how they visualize risk and (2) to evaluate their contextual value.
We selected 11 websites (Table 2) that visually depicted for users flood risk using interactive maps and other tools. These sources were selected because they all include the Gulf region in their spatial extent, and because they encompass a range of types of risks, intended purposes, and risk visualization techniques. We also aimed to find data sources that enabled users to make their own choices on how to engage with the scientific data on the website. We selected data sources by drawing from an initial list of flood risk tools previously examined by Stephens et al. (2014, 2015), and then asking for additional recommendations for flood risk tools from members of the overall project team. The Part II audit compared flood risk websites and real estate websites. Real estate websites employ various strategies for displaying information about individual buildings that the project team wanted to build upon. Additionally, many individual residents (e.g., homeowners, homebuyers, renters) are likely to be at least somewhat familiar with using real estate websites to research purchasing and/or renting properties.
Our first goal was to evaluate the direct risk representation features and functionality of the flood risk websites so that they could help us consider how we could enhance user engagement and promote insight into these risks in HazardAware. Therefore, we focused on three main subcategories: (1) types of risk displayed on the website, (2) how risks were operationalized, and (3) how risks were visualized. For the first category, types of risks, we audited data sources that used interactive maps to depict flood risk, but some included specific categories of flooding, such as sea-level rise, and/ or additional risks. The second category, risk operationalization, was concerned with how risk was numerically or categorically described (e.g., whether described as the percent chance of flooding vs. a high, medium, or low chance of flooding). The third category we explored was how these sources chose to visualize risk on maps, (e.g., by color shading, numeric values). Fourth, we looked at the described target audiences for each data source (e.g., the general public, risk management decision-makers, insurance professionals). The subsequent table (see Table 3) summarizes data categories and items that we reviewed for the content inventory audit, along with a representative example of the types of features we evaluated.
|Category||Item/Description||Example (Source: Aqueduct Floods)|
|Source details||Website name, affiliated organization, URL||Affiliated organization: World Resources Institute (WRI), Netherlands|
|Risk visualization||Type(s) of risk, how risk is operationalized (e.g., % risk of flooding, high/medium/ low chance, etc.), how risk is visualized (e.g., color shading, numeric, “stoplight”), intended audience (e.g., homebuyers, sellers, professionals, etc.)||Type(s) of risk: Coastal flooding and riverine flooding|
|Mapping and search functionality||Map resolution (e.g., house level, neighborhood level, city level), Ability
to search by address (yes or no), default/ starting view, disclaimer, annotations (e.g., help text), additional map views/features
|Resolution: Neighborhood level|
|Additional features/content||Sharing features (e.g., ability to share maps on social media), external links, multimedia and/or printer-friendly features, downloadable content||Downloadable content: The Download icon allows users to embed the widget or download as CSV, JSON, image, or report|
|Additional notes||Other features (features not previously categorized that may be of interest), notes (additional miscellaneous observational notes)||The Aqueduct Flood tool allows users to understand and identify current and future water risks to agriculture and food security|
Our second goal was to understand features that might help our intended audiences use or interpret via context-related features the risk information. These features give users context so that they can understand risk. Map-based visualizations have varying levels of risk communication effectiveness, which can create issues for website users’ ability to understand and interpret the scientific data. Specific issues that relate to map-based visualizations of SLR information include context of use, probability and risk, and geographic location (Stephens et al., 2017). We wanted to ensure that the visualization tools we developed for the HazardAware website were effective; the project design goal was to communicate complex science-related issues that include risk-related decision making. Therefore, we aimed to use this content inventory audit to capture specific map-based visualization features that we would consider as being either effective or problematic for website users. We captured information on the following nine subcategories of risk context features: (1) disclaimers, (2) annotations, (3) default/ starting views, (4) additional map views/features, (5) sharing features, (6) external links, (7) multimedia and/or printer-friendly features, (8) downloadable content, and (9) other features.
After tabulating our data in a spreadsheet, we performed an analysis of each website’s content, including their textual, visual, and interactive mapping features. Within each of the subcategories of data described above, we looked for patterns of similarity and major differences between websites. We evaluated trends in risk representation and contextual factors, summarized them, identified novel or unique features, and provided recommendations for features that the HazardAware project team might consider to inform our website design. The results of the content analysis were presented to the rest of the HazardAware project team in a report format. Our findings influenced website design discussion in several project areas, which included (1) possible website features, (2) the methods and data that these sources used, (3) how we could develop future flood risk information for the HazardAware website, and (4) appropriate timelines for depicting future flood risks (e.g., 15 years, 30 years).
Our focus was to determine the strategies for depicting risk and contextualizing risk that would be the most effective to enable our target audience to better understand the information on the HazardAware website. For example, we found that flood risk websites differed in the types of risks that they chose to depict on interactive maps. Most of the sources focused on either only flood risk or a combination of flood risk and sea-level rise. Three sources included additional risks in interactive maps: Buyers Be-Where (flooding, hurricane, and hazardous waste), Coastal Resilience (flooding, sea-level rise, and storm surge), and Louisiana FloodMaps Portal (both flood and wind hazards). We wanted to ensure that the website visualizations we chose for this project communicate risks effectively and in such a way that they support accurate mental models of risk dynamics (Stephens et al., 2017). Therefore, this content audit allowed us to document and analyze features that influenced the decision-making process for implementing features for the HazardAware website design.
Part II Comparative Content Audit: Comparing Flood Risk and Real Estate Websites
As we were completing the Part I content audit, the overall project team shifted how they envisioned communicating risk in HazardAware from a primarily map-focused strategy to one that combined text, graphs, and some map-based information similar to the types found on real estate websites. In response to this shift, we identified a new goal for the Part II audit: to compare non-map based communicative features of selected flood risk and real estate websites. We selected this goal to provide possible models for depicting building-level information on housing and risk, in contrast to the community-level scale of information that is depicted in most flood risk websites. We identified two flood risk websites from our Part I audit that displayed flood risk at a building-level scale, as well as five real estate related websites that included community and building specific details (see Table 2). Our strategy was to compare how these websites primarily communicated building- level information, as well as community-level information that provided context about specific homes.
Our content audit process was like that in Part I: after selecting websites to analyze, we gathered information on specific subcategories of information, performed an analysis of content within each genre of website (flood risk and real estate), summarized results, and provided recommendations for the project team. When evaluating the flood risk websites, our primary focus was on features other than interactive maps, as we had previously analyzed those during the Part I audit. We looked at three subcategories of features: (1) community risk types and population, (2) other additional community-level details, and (3) building-level features. For the real estate websites, we explored seven subcategories: (1) nearby features on interactive maps, (2) interactive transit maps, (3) additional community details, (4) building-level details, (5) home-ownership costs, (6) additional home specific features, and (7) other building-level features. In addition, we made notes relating to how multi-metric scores were depicted (e.g., a “Walk Score” that measures neighborhood walkability). The subsequent table (see Table 4) provides examples of community-level features and building-specific features that we documented using our comparative content audit.
|Data Source Name/Genre||Community-Level Features||Building-Specific Features|
|Realtor (Real Estate)||Closest grocery stores, transit map information included: traffic, public transit, and bike lines||Default map view of the home included
an image of the front of the home with a 360-degree view. Listed the last sold date of the home.
|Surging Seas (Flood Risk)||Community population defined as Caucasian, Hispanic, etc. and depicted in social vulnerability (e.g., low, medium).||Includes home value and provides state level flood risk resources.|
When analyzing the Part II data, we evaluated the data sources and compiled our findings in two parts: flood risk and real estate data sources. For each part, we detailed our findings independently for individual building-level and community-level features. This section briefly highlights the categories and standout features that we reported on for the Part II comparative content audit.
Capturing building-level information was important because the project team envisioned our audience primarily being interested in researching hazard information for their own home or for a home they are considering purchasing. While exploring building- level details, we looked at two subcategories of features: risk communication details and data sources. Risk communication focused on the way that the website communicated risk and data sources referred to the sources that these websites used to depict risk. When performing a comparative content audit for the HazardAware website, we reviewed several building-level categories and specific items for each website genre (see Table 5).
|Website genre||Main Category||Subcategory||Item/Description|
|Real estate||Building specific details (documented in Y/N format)||Risk communication||Status (e.g., on market, sale pending, etc.), building type, building square footage, lot square footage, # of beds and baths, stories, year built, year renovated, county, last sold, estimated home value, mortgage payment, comparable nearby homes, HOA dues, utilities
and maintenance, property taxes, homeowner’s insurance, neighborhood median price
|Real estate||Building specific details||Risk communication||Default home view, other home vies, default map view, other map view(s), other home ownership costs, interior features, exterior features/ taxes/assessments, property/lot details, property/price history, real estate sales, median rent, additional building specific details|
|Real estate||Additional features||Risk communication and data sources||Other features, notes|
|Flood risk||Building specific details (documented in Y/N format)||Risk communication and data sources||Future flood risk to homes, home value, state-level resources|
|Flood risk||Building specific details||Risk communication||Future flood risk to home types (e.g., description of the types of risks they depict risk)|
|Flood risk||Additional features||Risk communication and data sources||Other features, notes|
We also envisioned that the HazardAware audience would be interested in community-level risk information, such as the scope of flooding beyond their own home or the history of wind damages in their neighborhood. While exploring community-level details, we looked at three main categories of features: community details, community risk, and data sources. Table 6 provides an overview of the community-level categories and specific items that we reviewed. By researching these website features, we were able to gain a better understanding of how these two genres depicted community risks. When performing a comparative content audit for the HazardAware website, we reviewed several items within each main category for each website genre (see Table 6).
|Website genre||Main Category||Subcategory|
|Real estate||Community details (documented in Y/N format)||Crime, noise, nearby: schools, grocery stores, coffee shops/cafes, restaurants, parks, gas stations, pharmacies|
|Real estate||Community details||Transit, Additional Community Details|
|Flood risk||Community risk||Community risk types, population, buildings, additional community details|
|Flood risk||Data sources||Data sources|
|Flood risk||Community risk||Other features, notes|
Our findings first helped us to maximize our communication effectiveness with our target audience by helping us to understand the characteristics of flood risk and real estate websites. We also identified specific organizational strategies and features that we could adapt for the HazardAware website. For example, flood risk websites communicate community-level risk using different strategies, and our audit helped inform the team’s discussion about how HazardAware might address this issue. Other features from real estate websites, such as the ability to generate a custom report about a home or look at a multi-metric index like a Walk Score, have also informed the project team’s conversations about analogous features for the HazardAware site.
As our case study illustrates, it can be a complex task to incorporate different types of content audits into the evaluation and design of a new informationally complex website. Our heuristic is intended to provide guidance to PTCs to help them work out how to integrate content audits into the formative stages of website design. By breaking the content audit process into primary goals, identifying the type of information needed to address each goal and type of audit that would be appropriate to obtain it, conducting iterative audits, and synthesizing results, PTCs can simplify this process.
With the project’s overarching goals in mind, our heuristic design emphasizes these areas as they intersect research and practical application methodologies that drive content audit strategy. As Sperano (2017) suggests, a content audit can provide valuable information to the people who design and maintain information within a digital information ecosystem. Therefore, we hope that our Content Audit Selection heuristic provides guidance for practitioners to create a content audit strategy that addresses each of their website design project goals. In addition, as the literature suggests, content audits can serve as an evaluation method at various stages of the website design process (Land, 2014; Sperano 2017). Practitioners can apply this heuristic in such various scenarios, such as initial design stages or during a website redesign. This practical application creates an agile goal-oriented project strategy that allows practitioners to update content audits to align with new or reimagined project goals.
We were able to use content audits to inform the design decisions for the HazardAware website. Ultimately, we found that using content audits allowed us to simplify the complexities of the HazardAware website design and streamlined our ability to make risk communication decisions for the project. Content audits enabled us to (1) communicate website features and functionality to the broader project team, (2) articulate how other designers have addressed challenges related to website features and data limitations, and (3) develop systematic report results to streamline collaboration with the scientific and creative project teams. In addition, we found that the comparative content audit strategy can be especially useful for complex website design and for projects that have an interdisciplinary focus, as it can help us make connections to different genres within the website scope.
We recommend that PTCs integrate content audits into a broader development strategy that incorporates input and response to users, as described by other researchers (Miller, 2019; Spyridakis et al., 2005). This is particularly important in risk communication contexts, in which individual experiences and social factors lead to very different experiences of risk (Grabill & Simmons, 1998; Stephens & Altamirano, 2021). As we present the Content Audit Selection heuristic, we also recognize that performing a content audit is not the only thing professional communicators need to do for complex website design, but it is one tool that they can use. Content audits do not replace talking to users and understanding what they want; therefore, we recognize the importance of talking to users during formative stages and suggest that researchers can also use content audits to accompany other participatory design methods.
Research reported in this publication is supported by the Gulf Research Program of the National Academies of Sciences, Engineering, and Medicine under Grant Agreement #200010880. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Gulf Research Program or the National Academies of Sciences, Engineering, and Medicine.
Albers, M. J. (2009). Design for effective support of user intentions in information-rich interactions. Journal of Technical Writing and Communication, 39(2), 177–194.
Bloomstein, M. (2012). Content strategy at work: Real-world stories to strengthen every interactive project. Elsevier.
Brinck, T., Blythe, D., Gergle, D., Bunyan, J., Wood, S. D., McReynolds, T. (2002). Designing web sites that work: Usability for the web. United Kingdom: Morgan Kaufmann Publishers.
Cutter, S. L., Ash, K. D. & Emrich, C. T. (2014). The geographies of community disaster resilience. Global Environmental Change, 29, 65–77. DOI:10.1016/j.gloenvcha.2014.08.005
Detzi, C. (2012, March 20). From content audit to design insight: How a content audit facilitates decision-making and influences design strategy. UX Magazine. https://uxmag.com/articles/from-content-audit-to-design-insight
Fischhoff, B. (2009). Risk perception and communication. In R. Detels, R. Beaglehole, M.A. Lansang, & M. Gulliford (Eds.), Oxford textbook of public health (5th ed., pp. 940–952). Oxford University Press.
Gall, M., Borden, K. A., Emrich, C. T., & Cutter, S. L. (2011). The unsustainable trend of natural hazard losses in the United States. Sustainability, 3, 2157–2181. DOI:10.3390/ su3112157
Gall, M., & Friedland, C. J. (2020). If mitigation saves $6 per every $1 spent, then why are we not investing more? A Louisiana perspective on a national issue. Natural Hazards Review, 21(1), 04019013. DOI:10.1061/(ASCE)NH.1527- 6996.0000342
Garrett, Jesse James (2010). The elements of user experience: User-centered design for the web and beyond. Pearson Education.
Getto, G., Labriola, J., & Ruszkiewicz, S. (2019). A practitioner view of content strategy best practices in technical communication: A meta-analysis of the literature. Proceedings of the 37th ACM International Conference on the Design of Communication, 1–9. https://doi.org/10.1145/3328020.3353943
Grabill, R. T., & Simmons, W. M. (1998). Toward a critical rhetoric of risk communication: Producing citizens and the role of technical communicators. Technical Communication Quarterly, 7(4), 415–441. DOI: 10.1080/10572259809364640
Halvorson, K., & Rach, M. (2012). Content strategy for the web (Second). New Riders.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. DOI: 10.3102/0013189X033007014
Jones, C (2009). Content analysis: A practical approach. UXmatters.com.
Kostelnick, J. C., McDermott, D., Rowley, R. J., & Bunnyfield, N. (2013). A cartographic framework for visualizing risk. Cartographica, 48, 200–224.
Krippendorff, K. (2013). Content analysis: An introduction to its methodology. United Kingdom: SAGE Publications.
Land, P. L. (2014). Content audits and inventories: A handbook. XML Press.
Martin, B., Hanington, B., & Hanington, B. M. (2012). Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.
McCool, M. (2006). Information architecture: Intercultural human factors. Technical Communication, 53(2), 167–183.
Miller, J. (2019). The design cycle and a mixed methods Approach for improving usability: A case study. Journal of Web Librarianship, 13(3), 203–229. https://doi.org/10.1080/19322909.2019.1600451
Nielsen, J. (2020). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/
Rockley, A., Kostur, P., & Manning, S. (2003). Managing enterprise content: A unified content strategy. New Riders.
Rosenfeld, L., Morville, P., Nielsen, J. (2002). Information architecture for the world wide web. Cambridge: O’Reilly.
Richards, D. P. (2019). An ethic of constraint: Citizens, sea- level rise viewers, and the limits of agency. Journal of Business and Technical Communication, 33(3), 292–337. DOI:10.1177/1050651919834983
Sperano, I. (2017). Content audit for the assessment of digital information space: Definitions and exploratory typology. Proceedings of the 35th ACM International Conference on the Design of Communication, 1–10. https://doi.org/10.1145/3121113.3121227
Spyridakis, J. H., Wei, C., Barrick, J., Uddihy, E. C., & Maust, B. (2005). Internet-based research: Providing a foundation for web-design guidelines. IEEE Transactions on Professional Communication, 48(3), 242–260. https://doi.org/10.1109/TPC.2005.853927
Stephens, S. H., & Altamirano, A. (2021). Understanding user expertise through lived experience: Making natural hazard risk and mitigation Information more accountable to users. Proceedings of the 39th ACM International Conference on Design of Communication, 326–330. DOI: 10.1145/3472714.3473660
Stephens, S. H., DeLorme, D. E., & Hagen, S. C. (2014). An analysis of the narrative-building features of interactive sea level rise viewers. Science Communication, 36(6), 675–705. DOI:10.1177/1075547014550371
Stephens, S. H., DeLorme, D. E., & Hagen, S. C. (2015). Evaluating the utility and communicative effectiveness of an interactive sea-level rise view through stakeholder engagement. Journal of Business & Technical Communication, 29(3), 314–343. https://doi.org/10.1177/1050651915573963
Stephens, S. H., DeLorme, D. E., & Hagen, S. C. (2017). Evaluation of the design features of interactive sea-level rise viewers for risk communication. Environmental Communication, 11(2), 248–262. https://doi.org/10.1080/17524032.2016.1167758
Still, B., & Crane, K. (2017). Fundamentals of user-centered design: A practical approach. CRC Press. https://doi.org/10.4324/9781315200927
Summers, J. K., Harwell, L. C., Smith, L. M., & Buck, K. D. (2018). Measuring community resilience to natural hazards: The Natural Hazard Resilience Screening Index (NaHRSI)—Development and application to the United States. GeoHealth, 2, 372–394. https://doi.org/10.1029/2018GH000160
ABOUT THE AUTHORS
Amanda Altamirano is a graduate research assistant and graduate student in the Texts & Technology Ph.D. program at the University of Central Florida. Her research examines technical communication in information architecture, gender and technology, and user- centered design.
Sonia H. Stephens is an associate professor of English at the University of Central Florida. Her research examines scientific communication in digital and visual media, visual risk communication, and user-centered design.