Lessons on Designing Scalable Financial Coaching

Background

Financial interventions for consumers and employees rely heavily on financial education, but financial education has demonstrated virtually no impact on behavior change, accounting for only 0.3% of the variance in financial behaviors. While financial coaching has been found to be effective at improving financial outcomes, it’s expensive to offer and suffers from high drop-out rates.

The Center for Advanced Hindsight (CAH) partnered with Moneymap, a high-tech, high-touch financial coaching platform, to experiment with ways to make financial coaching more effective at a more accessible cost.

We wanted to know:

● Does changing the tone of financial coaching affect outcomes? Specifically, we decided to vary the levels of:

_____Self-efficacy, or users’ feeling that they have the ability to meaningfully impact their _____ financial situation.
_____Accountability, or users’ awareness that their coach expects them to complete tasks.

● What contact method works best for financial coaching? Specifically, we wanted to study the effectiveness of two different methods of coach-client contact:

_____Phone calls, which are higher-cost, must be scheduled for a specific time and allow _____ for more direct, personal contact.
_____Self-guided surveys, which are lower-cost, can be completed at any time and result _____ in a more autonomous experience.

We also wanted to know how these different contact methods and tones combine to affect outcomes. To test these questions, we designed edits to Moneymap’s coaching program.

In Moneymap, participants have two touchpoints with their coach. Between these touchpoints, they accomplish three To-Do’s. After the two touchpoints, they enter into a sustain phase where they can contact their coach as needed.

Experiment Design

CAH implemented two interventions into the program design.

Call versus Survey

To test contact methods, we randomly assigned participants to either talk to their coach on scheduled phone calls or to take self-guided surveys that their coach would later review. Participants were defaulted into one of the two contact methods, but could switch from one contact method to the other.

Participants had two touchpoints with their coach – one at the beginning of the program and one after 4 weeks. For the first touchpoint, reviewing a survey required about 5 – 7 minutes of a coach’s time, while calls required about 20.  We wanted to understand whether experiencing the more efficient, lower-cost surveys would result in different outcomes for participants – specifically, whether condition would have an effect on completion rates for their two touchpoints and their three assigned To-Dos.

Examples of commonly assigned To-Do’s are included below.

Self-efficacy versus Accountability

To test tone, we customized language throughout the program to create two parallel experiences designed to either (1) build self-efficacy, or (2) increase accountability for completing tasks. Then, we randomly assigned participants to receive one of the two types of messaging.

In the field

Participants were recruited for the 6 month program between July 8th and September 6th, 2020, gathered from three small-to-midsize employers and one large online savings platform. Out of 2,795 people who began signup, 1,560 finalized their signup and are considered our participant pool. Attrition was evenly distributed across the four conditions. Because all but 143 participants were recruited from the online savings platform, we did not distinguish between recruitment sources in our analyses.

Results

Call versus Survey

207 participants chose to switch from the call condition to the survey condition, while only 7 participants chose to switch from the survey to the call condition. Significantly more participants in the survey condition completed the first touchpoint (p < 0.001). There was no significant difference in completion rates for the second touchpoint (p = 0.15).

The difference in completion rates between conditions seems to be driven by timing. Most participants in the survey condition chose to fill out the first survey right away (585 vs. 162), and those participants were more likely to complete it than those who delayed (98% vs. 36% completion).

While there was no significant difference in completion of the first or second To-Do based on contact method, participants in the survey condition were more likely to complete the third To-Do (p = 0.018).

Self-efficacy versus Accountability

When analyzed alone, messaging tone had no significant effect on outcomes. When we analyzed the combination of contact methods and tone, survey + self-efficacy outperformed the other three conditions, particularly in 1st (p = 0.018) and 3rd (p < 0.001) To-Do completion.

Income Breakdown

To better inform financial inclusion efforts within financial coaching and other initiatives, we also analyzed the effect of household income on touchpoint and To-Do completion.

Of participants who completed signup, 89% had a household income under $75,000.

Not surprisingly, touchpoint completion rates and To-Do completion rates increase with household income. Households earning under $25,000 annually (815 participants) were significantly less likely to attend touchpoints (p < 0.001), and significantly less likely to complete any To-Do’s (p < 0.001).

Implications of Findings

While this research had limitations, the largest being the short-term nature of the outcomes measured, these findings still unveil some valuable insights for those designing financial coaching programs:

Design for Time Scarcity

In this study, communicating via a self-directed survey was more attractive for participants, less costly for coaches, and resulted in similar or higher completion rates compared to a traditional introductory phone call. Consider developing a financial inclusion coaching program with less rigid touch points so that participants can complete them on their own schedule. Recent research on flexibility showed that flexible scheduling influenced both participation and longer lasting habit formation. While this recent work involved more than just flexible scheduling, providing flexibility remains important when designing for limited bandwidth.

Build on Momentum

Respondents had much higher touchpoint completion rates when they were able to take action immediately. Consider developing a self-directed option to take the place of one or more face-to-face touchpoints, particularly at junctions when participants may need to wait to speak with a person. The CFPB financial well-being questionnaire includes examples of perception-oriented questions that can be supplemented with specific financial data questions to give coaches a solid foundation for planning prior to ever sitting down with anyone.

 

__

This research was conducted in collaboration with Moneymap through the financial support of Prudential.