A/B Testing: Voice and Tone Analysis

TECHNICAL SKILLS

Qualtrics XM Platform

Duarte design template slidedocs

HyperWrite AI writing tool

Microsoft Teams

Conceptual Skills

A/B testing

Linguistics charactersistics Style/Voice/Tone

Survey writing best practices

Participant selection

Identifying the purpose of the About Us content

First, we needed to agree on the purpose of the content in terms of the client’s purpose or intended result. We determined that the purpose of the “Our Story” is to give customers a first impression of the company, which impacts conversions. We also determined that customers need to be able to trust organizations they invest their money with.

Identifying distinct tones of voice for control treatment and test treatment (Versions A and B)

We used the Original Text for Version A (Casual Tone of Voice) and created a Revised Text for Version B (Formal Tone of Voice). We used artificial intelligence tool HyperWrite to develop Version B by manipulating the text with linguistics characteristics of a more formal tone, such as passive voice, longer and more complex sentences, and academic vocabulary.

Identifying content performance metrics

Our content performance metrics involved both qualitative and quantitative metrics. Our surveys tested user reactions and interest in Nada content. We also measured whether the current content could be improved through word-of-mouth recommendations, potential customer engagements, and conversions such as downloading an app or reaching out to customer service. these metrics helped us determine whether the “About Us” content accurately engages the audience and encourages a sense of trust and respect from potential customers.

Recruiting test participants

We recruited 32 participants for this study who were potential Nada customers or users. Participants were assigned randomly to read the 2 versions. 16 read Version A and 16 read Version B.

Building tools

The content and test questions were delivered through 2 versions of a Qualtrics survey. We chose to use Qualtrics surveys for ease of administration and the ability to test participants remotely. Users were assigned randomly to one of the two surveys and each was sent an anonymous link to one of the surveys. Our surveys consisted of demographic questions, screening questions, and questions testing user reactions and interest in the content. Another benefit of this survey method is that the data provided by participants was captured in a spreadsheet by the Qualtrics program.

Delivering survey

Our A/B versions of copy for testing were delivered online through the Qualtrics surveys. Each team member recruited at least 10 participants aged 18 or older.

Analyzing test results and making recommendations for style/voice/tone

Using the spreadsheet of responses created by Qualtrics and the graphics provided for each survey questions, we were able to analyze test results and make recommendations to Nada. We identified and outlined both strengths and opportunities for Nada.

Presenting data and recommendations with slide decks

Finally, we created a Duarte slidedoc to share recommendations with Nada, Inc. The slidedoc includes the background, strengths, opportunities, additional results, and appendices containing the 2 content versions and the survey questions.

We used Microsoft Teams for group communication and document sharing.