So, how did you like the show?

Muse October 2007

Museum visitor surveys come in all types, formats and sizes—front-end, formative, evaluative, exit—but whatever the category, they’re critical to keeping ‘em coming through the door. It’s feedback; no different than a restaurant wanting to know how you enjoyed your sirloin or a hotel asking you if the check-in staff was courteous and efficient.

“It’s really important to us to get a good understanding of what our visitors want and need,” says Angela Williams, Director of Business and Operational Services for the Royal British Columbia Museum Corporation. “And, what they expect from us.”

The whys

Surveys can eat up time and money. Are they worth it? Karen Graham, Director of Audit and Evaluation at the Canadian Museum of Civilization Corporation thinks so. “In a nutshell, we want to make sure that our visitors have the best, most engaging experiences they can have from a museum visit,” she says. “Getting feedback from visitors is a fundamental part of the trend towards moving from a museum-centred approach to a visitor-centred approach. Visitor research bridges the gap between visitor expectations and needs, and the museum’s goals.”

The RBCM’s Williams says they survey for a whole variety of reasons. “We want to know why visitors come, we want to know what we can do to improve the facility. We want visitors to have an exceptional experience here for all the right reasons—did they have a grumpy experience with security or a bad experience in our café for example.”

Williams offers another important incentive for surveys. “We also do them to find out the economic impact of our operation on local tourism,” she says, “and to document our financial contribution to the local economy.” She says that helps put into perspective why they do what they do.

Some museums have additional special reasons for surveys. Gabrielle Trepanier, Audit and Evaluation Officer for the Canada Science and Technology Museum Corporation, which includes the Canada Agriculture Museum and the Canada Aviation Museum, says that because they’re a crown corporation reporting to Heritage Canada, “we need [surveys] for reporting to government for planning programs. Like any other government body, we have to know who we’re serving.”

What kind of information is important to gather in a survey? The CMCC’s Graham starts with needing to know people’s motivations for visiting. “We need to know what topics will interest them, what titles will attract them, why they DON’T come to the museum,” she says. “And behaviours, values, how they found out about the museum and what they like and don’t like about their visit.” She says other useful data are how they feel about price hikes, membership benefits, way-finding to and inside the museum, how they perceive staff and programs. “We need to know how we compare with other museums and attractions in the region and whether or not other museums present real competition or opportunities for cooperation with each other.” And, she adds, “we want to know if they will come back.”

Formative surveys have visitors test interactive exhibits and text panels. “We want to know if visitors understand our labels and if they can manipulate our interactives,” says Graham. “We need to know whether or not they are getting the message of an exhibition and if they are having the satisfying experiences they anticipated they would have.”

Williams says in addition the RBCM likes to know things like: Did you have fun here? Did you learn something? “We ask visitors about aspects of our entire organization,” she says. “Including our archives and research services.”

The hows

To elicit those data from patrons, Williams says the RBCM uses what’s called an Event Corp Survey, offered through a third party contractor. “These are electronic kiosk surveys,” she explains. “They’re touch screens; really easy to use.” Questions are developed with the company and are based on a logic system because not all of the questions are pertinent for all respondents. “For instance if you ask, ‘have you seen an IMAX movie today?’ and they say yes, then we’ll ask what they thought about the film. If they say no we’ll ask them ‘what kind of movie would you like to see?’” Williams says they’ve had great results, that have helped operations as well as deciding future exhibitions, but laments that the kiosks can’t poll people who do not come.

The RBCM has also done in-person surveys at fixed points like the base of the escalator. Others using this technique report a need for close control of sampling in order to have a truly representative data group. “The people who do these have to follow a proper sampling protocol,” cautions the CSTMC’s Trepanier. “The easiest way is to have an imaginary line on the floor and you must ask every sixth person that walks across it to do a survey. Otherwise you can’t trust that your data are representative.” She says they contract out larger surveys.

That sampling protocol goes for front-end evaluation surveys too. Front-end surveys gauge visitors’ interests in potential upcoming shows. “They’re mostly qualitative,” says Trepanier. “We talk to people about the theme of an upcoming exhibition, to see if they think its interesting or not, and what they would expect to see. Sometimes we show them pictures of artifacts.” One of her favourite methods is card-sorting. “We’ll write out themes for an exhibition on cards and ask visitors to sort them from the most to the least interesting.”

Getting it right choosing upcoming attractions is critical, says Trepanier. “If the director of the Science and Technology Museum wants to pick topics for the next five years, knowing that that’s how it’s going to be used is going to have a huge influence on what I do.” Pulling together new exhibitions can take a long time, and the potential return from getting it right is very high but the potential damage from getting it wrong is pretty serious. But she adds sometimes directors will take a decision that goes contrary to what the evaluation has said. “It’s still the right decision because of other factors,” she says. “But at least the audience has some sort of voice. And the decisions are taken with the data in mind.”

That audience voice is a fundamental part of the decision; it keeps decision-makers’ thinking in perspective. “Front-end research results remind exhibit developers of how non-experts approach, think about and understand an idea,” says the CMCC’s Graham.

The visitor ‘non-expert’ is a critical voice for formative evaluations too. Trepanier says at the CSTMC they develop all of their interactives in house. “We’ll test three or four prototypes before the final design,” she says. “They’re built so that we can roll them out on the floor and ask visitors to come and test them. Because they get to play with prototypes they feel part of the museum and part of developing exhibitions.” She says she understands why people are much more receptive to these than to surveys. “Would you rather fill out a form or play with this contraption over here?” she says. And formative samples don’t have to be as scientific either. “If six people in a row have problems with it, you fix it.”

The CMCC’s Graham shows how useful formative surveys can be. “We used a formative evaluation to determine how best to use Inuktitut language,” she says. “And we’ve used this convention now in other exhibitions that have incorporated syllabic alphabets (Inuktitut in Roman alphabet, pronunciation, Inuktitut (syllabic) and definition of word). We never could have figured this out without talking to visitors.”

What kind of information to gather? “Typical surveys that anyone would do in marketing,” says Trepanier. “You want to capture some demographics, satisfaction with different services, ask them what else they’d like to see. We’re in the process of refining our visitor standards document that spells out how we are to serve our visitors, so that’s one of our big monitoring tools through our summer surveys.” She says data are most useful where there’s a question you’re trying to answer or a clear purpose for the data collection. “You don’t do surveys just because you think it’s neat.”

Graham adds that it’s not only the topic that’s important but also the type of experience you’re offering, e.g. learning and discovery, social, object, introspective, entertainment or restorative. “So not only do we consider the variety of topics but also the variety of experiences we will be offering visitors,” she says. “Exhibition teams will now be asked to consider different experiences in exhibits and not just simply the information that is offered.”

Graham also feels the type of question is important. “Open-ended questions allow visitors to describe their experiences in their own words, using their own language as opposed to having them fit their experiences into the pre-determined, museum-generated responses that usually appear on standardized surveys,” she says.

Front-end surveys can sometimes surprise; and that shows how useful they are. Trepanier says with her previous museum employer they once put together a very successful exhibit on nursing. “One of the surprises at the front end was that visitors had a staunchly positive impression of nurses so they really didn’t want to hear anything negative about them,” she recalls. “We thought the section on more controversial aspects would be very appealing to visitors, but in the end it was the least appealing.” That section she says was completely re-thought as a result.

Williams says the RBCM recently looked at a number of exhibitions and found people love mummies, or anything to do with natural disasters and, surprisingly, Antarctica. “We’ve found our visitors have a fascination with the human story and how the natural environment affects that,” she says. “We find that people respond to stories that reflect themselves.”

Graham says that after doing some topic testing, her museum was shown a few surprises too: “We were thinking of doing an exhibition on disco and our visitors really hated that topic, so it was dropped,” she says.

Demographics

The disco surprise may have something to do with demographics—important data to gather on surveys. Demographics isn’t just for reporting purposes, it helps to anticipate what shows might attract visitors.

Logically, demographics vary with museums. Trepanier says there’s no such thing as a typical visitor; each of the corporation’s three museums has its own profile. “Two of our museums are family oriented so most of those are visited with children,” she says. “The Canada Agricultural Museum is a working farm so it attracts people with the really little ones—four and under.” From there the age ranges differ. “At Science and Tech the children are a little older,” she says. “We get a few more women at the Agricultural Museum and a few more men at the Science and Tech.” She adds that the Canada Aviation Museum generally attracts an older crowd, much like war museums. Despite all the differences, similarities prevail too. For instance, she finds that museum visitors are generally more educated than the average population.

Williams says demographics are changing at the RBCM. “Our previous demographic would have indicated mid-40s to early 50s, well educated, and high income,” she says. “I don’t think that’s reality any more. The kiosks are showing now the average age of the visitor is probably in the 30s and 40s.” She says their typical visitor depends on the time of year too “We did one survey at spring break where we had a higher family demographic. Also over Christmas we had a more locals and found the demographics were all over the place.”

At the Canadian Museum of Civilization, Graham has fairly well defined their average demographic: 38 years old, slightly more females than males, 69 per cent English, 57 per cent some university or higher, 28 per cent come from the National Capital Region and over two-thirds come with family members to spend an average of three hours and 24 minutes at the museum.

Small museums heed

Whatever the demographic, Williams’s previous comments on doing surveys to gauge a museum’s community contribution ring particularly true for smaller museums. “Especially for a smaller organization who might be applying for funding from a local government,” she says, “it would be good for them to know how they contribute to the overall well-being of their communities, so that they can make that healthy argument for support from their local government organizations.”

For smaller museums, Trepanier emphasizes the importance of front-end surveys, so that “what does get on the floor is of the highest quality you can manage,” she says. And some of the easiest things to do are tracking studies, she adds. “You don’t need a lot of skill, basically just a floor plan and people to follow visitors around and observe what they’re doing. With formative evaluations again you don’t need a lot of specialized training to take out a prototype and see how people react to it.”

With exit surveys Trepanier again cautions about proper sampling. “You have to be aware of what you’re doing in terms of the sampling,” she says. “Make sure that the person collecting the information knows how to sample properly and understand why they have to ask every sixth person because otherwise your data won’t be terribly trustworthy.” She advises not to use the cash counter point for asking questions. “The problem with that is you’re not getting a random sample of visitors, you’re just getting a sample of people who are buying the tickets.”

Overall, suggests Trepanier, “try and be clear about what is most helpful to you and tailor your goals to what you can manage. It doesn’t need to be complex. Even with one volunteer one day a week there’s lots you can do.”

Graham has some valuable suggestions about sourcing outside help and resources. “Become a member of the Visitor Studies Association, this is a huge resource for any museum just beginning in visitor studies,” she offers. “If you’re just getting into visitor research take a course that tells you about quantitative and qualitative methodologies, survey design and how to sample a population—you can make some very serious mistakes conducting surveys that make them virtually useless if you don’t do things right.” She adds that Marker Research and Intelligence Association, or MRIA, offers excellent, inexpensive and comprehensive one- and two-day courses across Canada.

Graham says it’s very important to test your survey instruments. And to find out from your exhibition team what they’re willing to change and what they’re not willing to change, as it will have a big impact on the research design. She reinforces the other research managers’ emphasis on front-end evaluations before doing an exhibition. And fully agrees with the need to test an interactive, “before it’s too late to change it.”




VIEW ARTICLE ONLINE


 
Copyright 2017 Graham Chandler | All Rights Reserved | Contact | Site by Element Media Ltd