The Foundation for Youth Development (FYD) manages the nationwide Kiwi Can, Stars and Project K programmes. These all aim to inspire school-aged children to reach their potential as confident, healthy individuals who contribute positively to New Zealand society. In 2009, the programmes helped over 18,000 young people to build self-esteem, and learn valuable life, education and health skills.
FYD puts great emphasis on evaluating the results for all three programmes. Evaluation is comprehensive, purpose-designed, and ongoing. Results are reported back to FYD’s government funders and sponsors, supporting continued investment. The findings also assist FYD in evolving the programmes to match the needs of the participants. In the following case study, Jo-anne Wilkinson, FYD co-founder and Executive Trustee, explains what’s involved in evaluation and why it matters so much.
Background & partners
FYD was founded in 1994 by Graeme Dingle and Jo-anne Wilkinson. As Jo-anne notes, outdoor physical activity teaches young people about their bodies and minds. “It helps them to understand their capacity to extend themselves - which can then be linked to their everyday lives. Outdoor activity also teaches them about the physical environment - weather, flora, fauna, waterways, air - and the link between them and our capacity to impact on them either positively or negatively,” she says.
The outdoors is an excellent classroom for learning ecological understanding and knowledge; teamwork and communication; resilience and perseverance; and goal setting and achievement. Further, in New Zealand, the outdoors is very accessible and affordable, offers great variety, and has generally good weather. The actual risks are low, meaning safe learning. There’s also good access to highly skilled instructors.
FYD has a foundation commitment to supporting young people. While it is a registered charity, it’s run like a business. The national office has 19 paid staff. Around the country, there are 750 FYD volunteers and paid employees or contractors.
The Lion Foundation is FYD’s principal partner. Other key sponsors are: Air New Zealand, Chapman Tripp, Deloitte, HP, The Southern Trust, Hotel Grand Chancellor, New Zealand Community Trust. The Ministry of Education funds FYD’s Kiwi Can programme with $300,000 pa, distributed to the regions. The Ministry of Youth Development (MYD) funds Project K with $450,000 pa.
Kiwi Can, Stars and Project K all aim to give young people the opportunity to overcome challenges and achieve their full potential. More specifically:
Project K is about inspiring 14 to 15-year-olds to maximise their potential. The 14-month programme consists of three core components: a wilderness adventure, a community challenge and mentoring. Students learn self-reliance, team building, self-confidence, perseverance, goal setting, good health and life skills.
Stars is about helping students make a safe transition into high school. It supports Year 9 (13 to 14-year-old) students through five integrated phases: an adventure camp, a community project, community adventure, careers expo and peer mentoring. Stars aims to give students purpose, to feel connected and have friends, and feel good about themselves.
Kiwi Can is about growing confident Kiwi kids. It’s a life skills and values programme for 5 to 12-year-olds, delivered once a week in participating primary and intermediate schools nationwide. It aims to equip children with a sense of self-worth and self-confidence, the ability to take responsibility, valuable life skills, and a can-do attitude.
Defining evaluation terminology:
- Quantitative research is about data in the form of numbers and statistics. It seeks precise measurement and analysis for statistical purposes.
- Qualitative research is about data in the form of words, pictures or objects. It seeks individual interpretation of events through participant observation, interviews etc.
- A self-efficacy measure is about finding out how people feel, think and act. A low sense of self-efficacy is associated with low self-esteem, depression, anxiety, and helplessness.
“When we started out 15 years ago we decided we wouldn’t do anything where we couldn’t assess the results. It’s about proving quantitatively as well as qualitatively that what we are doing is working,” says Jo-anne.
Of the three FYD programmes, Project K has the most scientific evaluation process, because a control group setting has been possible. Project K evaluation starts before each course, with a self-efficacy measure of the school’s entire year-10 cohort. Every student is evaluated, unless parents opt out. Students fill out a questionnaire, as do two teachers.
Results are cross-referenced to come up with a score. Places are then offered on the course (there’s an exclusion policy for issues such as drug/alcohol abuse, recent suicide attempt or violence). There are more questionnaires during and after the course, for up to three years. “It gives a comprehensive picture of the results [of participation],” says Jo-anne.
Until 2008, Project K evaluation included a control group: young people who didn’t do the course. This ended once the Ministry of Social Development assessed the FYD evaluation process, and found it scientifically robust. “There’s an ethical problem in using a control group because you’re identifying kids that could benefit from support but you are not providing it,” she observes.
Stars has the same self efficacy measure run across a school’s entire year-9 cohort. The measure is utilised every year with this cohort, for their entire school career. This is more about developing an overall picture of New Zealand high school students’ self-esteem, as it’s not possible to define quantitatively which positive gains are due specifically to Stars, versus other factors.
“There’s also qualitative research to test how kids [doing Stars] feel about themselves. It shows they do feel connected, are well networked, and have friends. That’s incredibly important to self-esteem,” says Jo-anne. The results show that the more connected the young person feels the more likely they are to stay at school and engaged in education.
Kiwi Can sees FYD ask the principal of each participating school what their goal is in having the programme (for example, to reduce bullying). The programme modules reflect that goal. At the end of the year, the school gives feedback against the goal, and on how else children have been influenced by the programme. Children complete a questionnaire.
A questionnaire for parents is also being developed by a research student. “It’s qualitative research. We ask the children things like ‘what has Kiwi Can taught you?’ and ‘What would you change?’ We get some great stories from them,” says Jo-anne.
Across all the FYD programmes, it’s crucial that the evaluation regime is right for the environment being tested. The self-efficacy measure was designed by a Psychology Masters student in conjunction with Auckland University. It reflects New Zealand’s cultural and social mix. The language and question set was scientifically tested across an entire school.
The FYD National Support Office has specialist staff to run, track and follow-up on evaluations. There’s an in-house research/evaluation manager, and a coordinator to help compile results. An IT specialist designed and manages the databases which help measure outcomes. “Plus we have fantastic volunteers who come and feed in the data [into the database].”
FYD also welcomes academic support and has relationships with the University of Auckland, AUT (Auckland University of Technology) and Massey University. “We had a PhD student develop a workshop model to help evaluation. And we had a clinical psychologist research student looking into the application of exclusion criteria [to support research accuracy],” notes Jo-anne.
It’s all a lot of work, but essential to the success of the organisation. “We have both corporate and philanthropic sponsors. They all say, we are only into supporting organisations that track the results of what they do. We report back according to each sponsorship agreement. For some it’s annual, for others it’s six-monthly or three-monthly,” she explains.
“Without the strong emphasis on evaluation we could only be in a position to ‘believe’ we are getting good outcomes, rather than being able to show evidence of those outcomes. With corporates and government all saying they want to put their money where there’s best value for dollar, being able to prove we are achieving great outcomes for young people stands us in good stead.”
“In addition, we’re able to keep improving the programmes and involve our young people in ways that are meaningful to them, their teachers and parents. [Evaluation] helps make sure we stay relevant to our audience of young people and stay at the cutting edge in youth development.”
Key benefits of evaluation
- Strong sponsor relationships. “If a sponsor is not communicated with or the information is not robust, they won’t continue a sponsorship. It’s not just about written reporting, it’s about caring for the people you have relationships with; meeting with them, keeping them in touch with how things are going and involving them where you can; keeping the relationships positive and alive,” says Jo-anne.
- Ongoing Ministry funding. Without being able to track results, government funding would never be implemented. FYD evaluation enables comprehensive and robust analysis of return on investment. “Without being satisfied with return on investment, contracts wouldn’t be renewed,” says Jo-anne.
- Focused programme development. FYD has an advisory group for each progamme. The groups use young people’s feedback to continually refine the programmes. In 2009, for example, the outdoors handbook was updated with more artwork, and more blank pages were added. The Stars Peer Mentor manual was significantly rewritten to reflect the need for more activity-based learning between peer mentors and Year-9 students. The Project K Wilderness Facilitator manual was abbreviated for fieldwork.
- Expert support. By having a commitment to research, FYD opens the door to partnerships with universities and other sources of expertise. In Jo-anne’s view, “we couldn’t be where we are without this support.”
- The knowledge to be an advocate. The comprehensive evaluation FYD undertakes gives the organisation credibility to be an advocate on youth issues. FYD regularly releases statements and is seen by media as an authority in the area.
- Quality personnel. FYD continues to attract highly qualified and high profile personnel, both at Trust Board level and staffing. This reflects its standing in the Youth Sector and the corporate world – a position which evaluation has helped build.
To know more about Foundation for Youth Development, visit www.fyd.org.nz or contact Jo-anne Wilkinson by calling 09 477 6249 or emailing Jofirstname.lastname@example.org.