One of the 2018 goals for our email outreach efforts was to learn more about our audience. Though in the past we had performed different exercises to do so -- email-level data analysis, content testing, and persona development to name a few -- we decided nothing beats being able to ask questions directly to our subscribers.
So we began to explore how to survey our email audience of 915,000.
We worked with GSA’s Office of Evaluation Sciences (OES), a group of behavioral science and analysis experts, to harness their expertise of behavioral science best practices and learn more about the choices and behaviors our specific audience takes. These were our goals:
- Create a survey that clearly asks our subscribers for their interests and goals when it comes to following our brand.
- Learn more about the business subscriber list we inherited a few years ago from the Department of Commerce.
- Test if different templates of an email influence a subscriber’s decision to take or not take a survey we send.
To survey the general public like this, you must go through Paperwork Reduction Act (PRA) approval process. This ensures information collected from the public is handled well. It requires getting clearance from the Office of Management and Budget (OMB).
Surveying our email subscribers involved this unfamiliar approval process and new tools and approaches to managing data. OES helped us with context around the PRA process, develop questions to ask our email subscribers and optimized the survey through Qualtrics, a mobile and desktop friendly data analysis system. Additionally, the team helped us randomize our email subscribers into lists through our marketing automation tool. After the survey development was complete and the approvals were in place, we had a strategy that included:
- A 7-question bilingual survey
- 8 distinct email templates (four in both English and Spanish) with different formats, images, and Call-to-Action (CTA) buttons
- 16 randomized lists with varying engagement, language, and business list subscription
We sent this survey out to our subscribers in September 2018 and got thousands of responses in both English and Spanish. In an upcoming blog, we will discuss the high level trends, what we learned, and how we will use this anonymous data to share better information with the public.
