Earthkeepers Evaluation Report

Executive Summary of the Report on Evaluation of the Teaching Responsible Earth Education's Earthkeepers Program

Prepared by the Earth Education Research and Evaluation Team, University of Arizona, July 2005

Introduction

An evaluation of the Earthkeepers program offered by Teaching Responsible Earth Education (T.R.E.E.) was conducted from September 2004 through May 2005 by the Earth Education Research and Evaluation Team at the University of Arizona. The evaluation consisted of pre- and post- tests and questionnaires completed by participating students, teachers and parents; in-person and phone interviews with classroom teachers and T.R.E.E. staff; observations while programs were being conducted and in classrooms afterwards; and document analysis. This summary highlights the main findings.

During the 2004-05 school year. T.R.E.E. offered the Earthkeepers program 13 times with 472 4th grade students from 7 different schools and 1 after-school group from two other schools. All but one of the participating schools are public elementary schools in the Orleans Parish and St. Charles Parish districts; the other school is a parochial elementary school in New Orleans. Many of the students at these schools live in high poverty areas; rates of students who qualify for free and reduced lunch at these schools range from a high of 99% to a low of 24%.

The evaluation resulted in four conclusions; each is described in the full report and summarized here.

T.R.E.E.'s Earthkeepers program fits well into participating schools' curriculum.

Analysis of the Louisiana state standards documents, teacher surveys, and interviews with teachers provided substantial evidence that the Earthkeepers program meshes well with state standards and helped students accomplish many important curriculum objectives. Both teachers and students have even commented that concepts learned during their camp experience were included on the LEAP test. Teacher comments included:

Our curriculum (FOSS) is great, but it doesn't cover everything we need to teach. That's one of the great things about Earthkeepers, they cover so much [of the district standards].

All the curriculum is covered, and made so real, so connected....This is going to help [students] so much on the LEAP test.

The children learned so much, not just science but social studies, math, language arts, poems, journals. The children...are still talking about it.

Students' understandings of ecological concepts increase as a result of participation in T.R.E.E.'s Earthkeepers program.

The Earthkeepers program helps students learn fundamental ecological concepts, based on what is known about good learning. The ecological concepts are abstract ideas and are difficult for students to truly understand. The programs use experiential learning to make the abstract concepts concrete using the I-A-A (Inform-Assimilate-Apply) Learning Model. All of the activities take place outdoors in a natural setting, providing students with a real world context and with plenty of opportunities for further examples and reinforcement. The ecological understandings of students were assessed using the Ecological Concept Questionnaire (ECQ). The ECQ-EK was administered as both a pre-test at school before students participated in the Earthkeepers program and as a post-test back at school afterwards. Overall scores increased by 43% from pre-test to post-test. Results varied for different schools, from an increase of 15% for a school at which scores started out fairly high to a high increase of 95% for another school. Scores also increased substantially for each of the four concepts taught in the program.

Students' personal environmental actions change as a result of participation in T.R.E.E.'s Earthkeepers program.

We asked students about their personal environmental actions both before and after participating in Earthkeepers. We also individually interviewed 20 students who had participated in the Earthkeepers program nine months earlier (during the previous school year). At the same time, we individually interviewed 8 of their parents. Finally, we asked parents and teachers about changes they saw in the actions students were taking. Almost all (92%) of participating students reported that they undertook new environmental actions after participating in Earthkeepers. After participating in Earthkeepers, most students reported that in the winter they regularly make sure the heat in their room is not on too high (71%), turn off the light when they do not need it any more (85%), and save water by taking shorter showers or by turning off the water when brushing their teeth (73%). All 20 students we interviewed were able to describe specific environmental actions they do. All but 2 said they do these more after attending Earthkeepers than they did before. When asked why, they gave reasons such as:

Because it made me think more about how life works and how it can affect things. So I just started doing it after that.

I learned that if we do one thing, it triggers another thing and it can mess really a lot of the environment.

All teachers (100%) reported that their students have lessened their impact on the environment since participating in the Earthkeepers program. In written surveys, over two-thirds of the parents (68%) said they had seen changes in their children since the program. In interviews, 7 of the 8 parents noted changes in their children's behavior (saving energy and materials), in their awareness of environmental issues, and in their interest in the natural world.

Students, teachers, and parents are highly satisfied with T.R.E.E.'s Earthkeepers program.

One hundred five (105) students from five schools completed satisfaction surveys for Earthkeepers. All students were in the 4th grade at the time they did the program. These responses represent a cross-section of demographics that T.R.E.E. typically serves (low and middle income urban children and families).

Over 90% of participating students reported that they enjoyed their Earthkeepers experience. Similarly, ninety-six (92%) of the students would like to do more outdoor activities like these and ninety-eight (95%) would like to return to do another program like Earthkeepers.

I really liked all the activities and I learned a lot. Please never stop teaching kids about saving Earth. Thank you!

I love how we were always on an adventure.

Earthkeepers is the best field trip I've ever had.

I liked EK a lot. Mainly because I got to get in touch with the earth.

That was the best 3 days I have ever had in my life. I hope we can come again.

Teachers also felt the program was valuable. When asked to explain, teachers mentioned the amount of content covered; the hands-on, engaging nature of the activities; the excitement of the students, and the high level of organization.

It far exceeded my expectations in terms of the amount of curriculum that is covered.

It's a great way to teach critical thinking and science.

I think it's fantastic...top notch. The TREE folks do such a great experience. All the curriculum is covered, and made so real, so connected....This is going to help [students] so much on the LEAP test.

An outstanding example of how students should be taught, all hands-on, interconnected, exposure to concepts that are real to them. Many students have no experience with being outdoors, that in itself is really important—you don't get that from a textbook. I think I've never seen anything that's as good, not even close.

All but one parent agreed that their children enjoyed the Earthkeepers program and 96% would recommend that their child's school continue to participate. The two most common reasons given, listed by most of those responding, dealt with how much their child had learned and how much their child enjoyed the program. Several parents mentioned the value of hands-on learning and the importance of learning about helping the environment.

Overall Conclusion

T.R.E.E.'s Earthkeepers program provides unique opportunities for city children to experience the natural world. The program is exciting, adventurous, and magical. The students are fully engaged in active learning in ways that cannot be duplicated in the classroom. Data gathered from pre- and post-tests; observations; interviews with students, parents, and teachers; and questionnaires provide evidence that students learned important ecological concepts, were exposed to new environmental perceptions, and changed their own environmental actions. It is clear that all of this took place not only because of powerful, integrated, and complete educational program but also due to the talent and hard work of T.R.E.E. staff, classroom teachers, and parents.