Evidence-Based Evaluation Tools for Small Steps to Health and Wealth

Evidence-Based Evaluation Tools for Small Steps to Health and Wealth™

Barbara O’Neill
Rutgers University

Martie Gillen
University of Florida

Jennifer Hunter
University of Kentucky

Abstract

Cooperative Extension family and consumer sciences (FCS) programs frequently address subject matter related to nutrition/health and personal finances. This article describes tools designed to collect evidence about the impact of the interdisciplinary Cooperative Extension Small Steps to Health and Wealth™ (SSHW) program. It begins by describing SSHW program components and recent research linking health, personal finances, and individual behavioral practices. SSHW is a signature program of NIFA-USDA that encourages people to make positive behavior changes to simultaneously improve their health and personal finances (e.g., more frequent physical activity and increased savings). The article then presents a description of available program evaluation methods and how they can be used to build the evidence base that SSHW is improving the health and financial well-being of individuals and families. It concludes with two examples of evaluation results for SSHW programs using specialized curricula that were developed for youth and older adults.

Keywords: Health, wealth, personal finance, behavior change, impact evaluation

Many Cooperative Extension family and consumer sciences (FCS) programs address topics related to health and personal finances (North Central Cooperative Extension Association 2015). Instead of addressing these two key life domains separately in “silos,” however, the Small Steps to Health and Wealth™ (SSHW) program encourages participants to make positive behavior changes that simultaneously improve their health and personal finances (O’Neill and Ensle 2014).  SSHW was developed by Rutgers Cooperative Extension (O’Neill & Ensle 2004) following a shift to family and consumer sciences (FCS) programming focused exclusively on health and nutrition.

Many Americans have “issues” including overweight/obesity, physical inactivity, low savings, and high debt. SSHW developers recognized parallels between health and personal finance and created a research-based program that incorporates health and personal finance research findings, similarities between both aspects of people’s lives, and expert recommendations. A need exists to demonstrate the effectiveness of SSHW with evidence-based research to assure the continued delivery and future expansion of this program.

SSHW Program Components

A wide variety of SSHW program delivery tools are available online for Extension educators’ use. The public SSHW website http://njaes.rutgers.edu/sshw/ includes archived monthly health and financial messages, links to PDF files of chapters in the 132-page SSHW workbook, an online SSHW Challenge and daily motivational messages, animated videos, a blog, posters, a list of SSHW scholarly publications and workplace wellness program newsletters. The internal SSHW website for educators http://njaes.rutgers.edu/sshw/internal/contains instructor guides for each of the 25 SSHW behavior change strategies, marketing pieces, lesson plans, specialized curricula for youth and older adult audiences, class activities, PowerPoint presentations, and evaluation tools.

Health and Personal Finance Research Findings

Recent studies provide justification for interdisciplinary health and personal finance programming. For a summary of research findings, see O’Neill (2015) and O’Neill & Ensle (2014).  One study (Sun, Webb, and Zhivan 2010) found that the “cost” of better health is the need for greater wealth due to more years of out-of-pocket expenses and an increased likelihood of chronic disease and/or long-term care in later life. Another found that even relatively small financial incentives influence individuals’ health-related behaviors (Sutherland, Christianson, and Leathermann 2008). Kosteas (2012) found a positive relationship between engaging in regular physical activity and labor market earnings. Specifically, regular exercise (or the discipline that it represents) was associated with a 6% to 10% wage increase.

Some researchers have noted that personal qualities may potentially underlie positive health and financial practices. In one study, employees received a workplace health screening and were followed for two years. 401(k) plan contributors showed improvements in health behaviors about 27% more often than non-contributors despite having few health differences prior to the program (Gubler and Pierce 2014). Time discounting preferences (inclination toward current spending versus future saving) and conscientiousness were believed to be related to similarities between workers’ retirement savings patterns and health improvement behaviors. Carr et al. (2015) found that individuals who engage in cognitive health information search activities, such as reading nutrition labels, are more likely to engage in retirement planning activities. Regular physical activity and healthy eating, however, were not found to be associated with retirement planning.

SSHW Evaluation Tools

While SSHW is research-based (i.e., it incorporates health and personal finance research findings), it has not been particularly evidence-based. Efforts are underway to change this. A program is considered evidence-based if evaluation research shows it produced expected positive results that can be attributed to the program itself and not extraneous factors. The evaluation is peer-reviewed and the program is endorsed by a federal government agency or other respected organization as an effective program (Cooney, Huser, Small, and O’Connor 2007).

Evaluation methods are often guided by a program logic model that describes planned inputs (e.g., funding, personnel, publications), outputs (e.g., training activities and outreach methods), and outcomes (e.g., knowledge gained and actions taken by participants). Templates to develop a logic model are available through the University of Wisconsin-Extension (Logic Model 2014). The logic model for SSHW is available at http://njaes.rutgers.edu/sshw/internal/.

The following five SSHW evaluation tools were discussed at the 2015 AFCPE Extension Pre-Symposium meeting in an effort to encourage Extension educators to collect impact evaluation data that can be aggregated nationwide. Each tool is available online for easy retrieval and use.

Personal Health and Finance Quiz- The quiz consists of 10 questions about health practices and 10 questions about financial practices. It is available via Rutgers Cooperative Extension at http://njaes.rutgers.edu/money/health-finance-quiz/. Respondents indicate one of four frequencies for performance of the 20 activities (never, sometimes, usually, always) and receive a health score, financial score, and total score indicating their frequency of performing activities that health and financial experts recommend. In addition to providing users with personalized feedback, the quiz collects data for research about the health and financial practices of Americans to inform future Extension outreach and can also be used as a pre-/post-test to evaluate the impact of SSHW programming (O’Neill and Ensle 2015). In an analysis of the first year’s data, collected from a sample of 942 respondents, there was a positive, statistically significant, and moderate (.463) correlation between indices of health and financial behaviors (O’Neill, Xiao, and Ensle 2016). A master’s thesis is underway using the quiz as a SSHW program evaluation tool. It will be administered before a series of SSHW classes and once again upon program completion.

SSHW Class Evaluation Forms– A suggested post-class evaluation form for use at the end of a SSHW presentation (any length and format) is available at http://njaes.rutgers.edu/sshw/internal/. This evaluation focuses on learners’ self-assessment of the value of a SSHW program, the amount of knowledge gained, general program comments, and planned actions (from a list of 11 items to check off), including “increase daily exercise” and “set one or more specific written health or financial goals.” A follow-up evaluation form, also available for downloading from the SSHW internal web site, asks respondents to check off actions taken and progress made toward their health and financial goals as well as success stories, comments, and questions about SSHW, and perceptions about how their health and finances have affected one another. Both evaluation forms collect qualitative and quantitative data that can be aggregated within an individual state and across state lines.

SSHW Pre- and Post-Test- On the internal web site http://njaes.rutgers.edu/sshw/internal/is a 20-question test of subject matter content covered within the SSHW workbook, lesson plans, and PowerPoint presentations. A master copy can be downloaded for printing with or without the answer key. Like the Personal Health and Finance Quiz that can be used before a SSHW educational intervention (e.g., class series, workplace seminars) to assess changes in behavioral practices, the pre- and post- test can be used to assess changes in knowledge about topics covered in the program.

SSHW Challenges- Another way to assess action taken by learners to improve their finances is to have them compete in a SSHW Challenge consisting of five health-related activities (e.g., perform at least 30 minutes of physical activity) and five financial activities (e.g., save a $1 bill or more and/or loose change) on a daily basis. Local program organizers may wish to provide prizes as an incentive to participate. The online SSHW challenge can be accessed at http://rutgers.ancc.net/.  Ten points per day are received for each of the ten activities that are performed and challenges typically last five or six weeks. Each year, a non-competitive challenge is set up where people can track their points for their own use or to submit to a competitive Challenge sponsor. “Paper and pencil” tracking forms for individuals and teams are also available at http://njaes.rutgers.edu/sshw/challenge/. Challenge evaluation metrics include changes in point totals over the duration of a Challenge, especially if a Challenge is combined with one or more SSHW presentations and/or other supportive outreach methods.

Success Stories- There is, perhaps, no better way to describe the impact of SSHW (or any educational program) than with the words of people whose lives were improved by it. In addition to using the evaluation forms described above, Extension educators should look for opportunities to engage program participants in informal conversations that provide useful anecdotes. It is also helpful for educators to undertake a personal health or financial behavior change of their own to experience how difficult it is for program participants to change and to generate a personal success story to share as a role model for others.

Two Examples of SSHW Impact Evaluation

SSHW Program for Youth

A survey was conducted to measure knowledge gained and outcomes of Building a Healthy Wealthy Future (BHWF), the SSHW youth program.  BHWF is a six-week workshop series designed to offer personal finance and health education, along with leadership and life skills, to students in sixth through eighth grade. The survey instrument with 15 questions included closed-ended and open-ended response questions. The outcome evaluation was based on a BHWF Logic Model and the KOSA assessment model (knowledge gained, opinions changed, skills acquired and aspirations heightened) (Rennekamp nd). A retrospective pre- then post- test instrument was used to assess self-reported knowledge gained as a result of participation in the BHWF program.

Table 1 displays the pre/post-course assessment of knowledge level. Participants responded to a statement which read: “For each topic listed below, circle the number that shows how well you understood each topic BEFORE completing the Building a Healthy Wealthy Futureprogram, and then circle the number that shows how well you understood each topic AFTERthe lesson.”  Responses were presented in a Likert fashion ranging from 1 = No Knowledge to 5 = Expert. Paired sample t-tests were calculated to compare the mean pre-test score to the mean post-test score for each topic. A statistically significant increase in knowledge gained was found from pre- to post- programming for all measures.

Table 1. Paired t-Tests for Retrospective Pretest (n=193)

In addition to measuring a change in knowledge, participants were asked to respond to a series of seven action statements, focused on intended behavior changes. The response matrix is presented in Table 2. Key intentions included: begin working on one goal related to positive health behaviors, increase my physical activity, and paying closer attention to my environment and how it influences my actions.

Table 2. Intended Behavior Change as a Result of Education Program

SSHW Program for Older Adults

The older adult component of the SSHW program is designed as a series of seven lessons to motivate older adults to improve their lives through behavior change strategies (small steps) for both health and financial management (Gillen 2014). The curriculum is available at http://njaes.rutgers.edu/sshw/internal/  and includes PowerPoint presentations, lesson plans including a leader’s guide, activities, and evaluation. An evaluation is administered immediately following each lesson. The evaluation assesses the level of knowledge gain (very little, some, or a lot) a participant perceives receiving and if they intend to make a behavior change after participating in the lesson. Educators are encouraged to follow up with participants to determine if behavior change occurred.

In 2014, the program was pilot tested in Duvall County, Florida. Seventy-six older adults participated in the program. Results indicated that 66% of the participants learned a lot from the lessons, 37% of the respondents planned to share the information with family and friends, and 92% of the participants planned to make a positive change to their health and/or finances using information received from the program. For example, participants indicated they planned to change the following behaviors: walk more, shop more wisely at the grocery store, check nutrition labels on food before purchasing, eat more fresh food and vegetables, add grains and more colors to meals, reduce sodium intake, reduce intake of soft drinks and sugar, cut down on cookie eating and not smoke, and use salsa instead of ketchup.

Summary

This article provided a description of the Small Steps to Health and Wealth™ program, available SSHW program components, recent research indicating a variety of health and personal finance relationships, and five types of SSHW program evaluation tools. Borrowing a phrase from Jim Collins’ best-selling book, Good to Great (2001), SSHW is currently a “good” program but won’t become “great” until it achieves a larger scale and the educators who deliver it use common evaluation indicators that can be aggregated nationwide. For additional information about SSHW, a 140-slide training presentation for professionals is available at http://njaes.rutgers.edu/sshw/internal/ (under “Other Materials”).

References

Carr, N.A., Sages, R.A., Fernatt, F.R., Nabeshima, G.G., and Grable, J.E. 2015. Health information search and retirement planning. Journal of Financial Counseling and Planning, 26(1), 3-16. http://afcpe.org/journal-articles.php?volume=395&article=505.

Collins, J. 2001. Good to great: Why some companies make the leap…and others don’t. New York: HarperBusiness.

Cooney, S.M., Huser, M., Small, S., and O’Connor, C. 2007. Evidence-based programs: An overview. University of Wisconsin-Madison. http://www.human.cornell.edu/outreach/upload/Evidence-based-Programs-Overview.pdf.

Gillen, M. 2014. Small steps to health and wealth^TM for older adults. The Forum for Family and Consumer Issues, 19(1). https://ncsu.edu/ffci/publications/2014/v19-n1-2014-spring/gillen.php.

Gubler, T. and Pierce, T. 2014.” Healthy, wealthy, and wise: Retirement planning predicts employee health improvements.” Psychological Science, 13(3), 219-224.  http://students.olin.wustl.edu/~gublert/Research_files/401k%20paper_submit.pdf.

Kosteas, V.D. 2012. “The effect of exercise on earnings: Evidence from the NLSY.” Journal of Labor Research, 33, 225-250.

Logic model. 2014. University of Wisconsin-Extension. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html.

North Central Cooperative Extension Association. 2015. “Analysis of the value of family and consumer sciences (FCS) in the north central region.” http://www.nccea.org/multistate-activities/fcs-battelle-report-2015/.

O’Neill, B. 2015. “The greatest wealth is health: Relationships between health and financial behaviors. Journal of Personal Finance, 14(1), 38- 47. http://www.iarfc.org/documents/issues/Vol.14Issue1.pdf.

O’Neill, B. and Ensle, K. 2015. “Personal health and finance quiz: A tool for outreach research, and evaluation.” Journal of Human Sciences and Extension, 3(1).  http://media.wix.com/ugd/c8fe6e_25e2da24fc874dc58b88bd525acbb340.pdf.

O’Neill, B. and Ensle, K. 2014. “Small steps to health and wealth™: Program update and research insights.”  The Forum for Family and Consumer Issues, 19(1). https://ncsu.edu/ffci/publications/2014/v19-n1-2014-spring/oneil-ensle.php

O’Neill, B. and Ensle, K. 2004. “Small steps to health and wealth.”  The Forum for Family and Consumer Issues, 9 (3), https://ncsu.edu/ffci/publications/2004/v9-n3-2004-december/fa-1-small-steps.php.

O’Neill, B., Xiao, J.J., and Ensle, K. (2016). Propensity to plan: A key to health and wealth? Journal of Financial Planning. 29(3), 42-50. https://www.onefpa.org/journal/Pages/MAR16-Propensity-to-Plan-A-Key-to-Health-and-Wealth.aspx.

Rennekamp, R. (n.d.). Using end-of-meeting questionnaires to assess KOSA change. The Program Evaluation Toolbox. Informally published report, Kentucky Cooperative Extension Service, University of Kentucky, Lexington, KY. Retrieved from http://www.ca.uky.edu/agpsd/toolbox1.pdf .

Sun, W., Webb, A., and Zhivan, N. 2010. “Does staying healthy reduce your lifetime health care costs? Center for Retirement Research at Boston College, Number 10-8. http://crr.bc.edu/wp-content/uploads/2010/05/IB_10-8.pdf.

Sutherland, K., Christianson, J.B., and Leatherman, S. 2008. “Impact of targeted financial incentives on personal health behavior.” Medical Care Research and Review, 65 (6), 365-375. http://mcr.sagepub.com/content/65/6_suppl/36S.short.

Return to Table of Contents
http://www.ncsu.edu/ffci/publications/2016/v20-n2-2016-fall/index-v20-n2-fall-2016.php

Back to table of contents ->https://www.theforumjournal.org/2017/08/29/summerfall-2016-vol-20-no-2/