Case Study: Best Friends Mentoring Program

Overview

Good projects are data driven projects. Data is needed to validate the need for a project. It should be gathered throughout the project to evaluate the effectiveness of the project and continued need. Most grantors now require an evaluation process be included in the grant application. Evaluation is a way of measuring whether a project is accomplishing its intended goals. It is also a means to understand why it was successful or why it was not successful. Through this process you are able to develop and improve upon your project.

Beginning in 2005 the Western Wellness Foundation contracted with DLN Consulting, Inc. to conduct program evaluations. DLN Consulting, Inc. in collaboration with Western Wellness Foundation developed a series of evaluation tools to measure effectiveness of the Best Friends Mentoring Program activities.

The mission of the Best Friends Mentoring Program is "Making a difference in the lives of children and families, one at a time. "The Best Friends Mentoring Program matches adult role models with children who need positive adult guidance and friendship. The School-Based Mentoring Program matches adult volunteers with students to meet one-on-one during the school day.

The program serves school-aged children ages 6-16. The Best Friends Mentoring Program is a project of the Western Wellness Foundation, Inc., a non-profit organization formed to promote support of children and families through mentoring and other programs across southwest North Dakota.

Objectives

The Western Wellness Foundation Inc. set out to accomplish four goals for the youth served by the AmeriCorps project and the Best Friends Mentoring Program. These goals were to:

1) increase academic skills

2) participate in structured social and service activities

3) increase social skills

4) recruit and support volunteers

Activities

DLN worked with the Western Wellness staff to develop a series of tools, designed to provide results for the measures required by Best Friends Mentoring Program. The Western Wellness staff determined the survey recipient selections. In 2005 the overall response rate was 58%. In 2006 the overall response rate was about 62%.

DLN conducted all data entry and ran quality assurance reports. The data was then compiled into spreadsheets for tabulation and cross-tabulation. Charts, graphs, and tables were extracted for the purpose of analysis.

Program Evaluation

After comparing observations and recommendations made in 2005 to information analyzed in the 2006 report, the following observations and recommendations are made for further program evaluation for the Best Friends Mentoring Program.

In the 2005 report, a recommendation was made that referral information was documented for all students, not just those enrolled at the beginning of the school year. This method would ensure a more accurate analysis of whether students increased their social skill levels. This recommendation was implemented in 2006 and great improvement was made in this area. It is recommended that attention be continued to this practice in the new school year.
It is recommended that teachers, when given the end-of-school survey for students, also be given their ratings provided for the students at the beginning of the school year. Evaluation in certain areas such as social skills can be subjective, based on many factors including the attitude of the teacher toward the student on the day the rating is made. If teachers are reminded of how they rated the student at the beginning of the school year, and given the opportunity to indicate why they gave a higher or lower rating at the end of the school year, a more well-rounded evaluation and analysis could be conducted. This recommendation was not considered in the previous year because ratings for students were not conducted at the beginning of the school year, and the issue was not anticipated in 2006.
A revised Student Progress Report was implemented to generate the desired results for math and reading levels. However, a few teachers still had difficulty indicating reading and math levels for students. Some still provided grades (A, B, C, etc.) rather than providing test scores for grade levels. This was a problem in 2005, and for the most part, was rectified in 2006 through the revised survey tool. However, a few students could not be evaluated because teachers misread the instructions. If student reading and math levels are to be rated for improvement in 2007, some method must be developed to ensure that all teachers fully understand this evaluation component.
A self-reported survey of alcohol use was administered to students in junior and senior high. This was a recommendation in 2005 and was met in 2006.
The 2005 report recommended that the tutoring program be continued at the Southwest Community High School. Students in this school were given the opportunity for tutoring in 2006. Hopefully, this program can be continued, despite the lack of AmeriCorps funding in 2007, as excellent results were again demonstrated in 2006.
Teachers who responded to the survey outnumbered parents/family members in both 2005 and 2006. It had been recommended that Western Wellness find a method of gathering more parent surveys to allow for a more accurate analysis and comparison of results reported between teachers and families. It is unknown what was done to increase the parent responses, but similar efforts should continue.
Host sites completed surveys at the close of each activity, rather than at the end of the year as was done in 2005. This improved the host site evaluations dramatically in 2006.
A recommendation to include a student evaluation of the program was made in 2005. This was not done in 2006, and the recommendation stands for 2007. A student evaluation would allow for an additional comparison of the program success and also measure the satisfaction of the student with the program. Written parental permission to conduct such a survey should be acquired prior to its undertaking.