Carleton Curricular Community Service Learning (CSL) Pilot Project Assignment 1: Program Profile, Logic Model, Performance Measurement Framework PADM 5420 D POLICY AND PROGRAM EVALUATION Fall 2010 Professor Robert Shepherd Group Members: Graeme Innes Greg Kaminski Peter Stanton November 4, 2010
Table of contents List of Acronyms... 3 Introduction... 4 Background... 4 Alignment with Institutional Priorities... 5 The Challenge... 5 Project Profile... 6 Project Description... 6 Objectives... 6 Stakeholders... 7 Carleton University... 7 Community Engaged Pedagogy Group (CEP)... 7 Carleton Initiative for Community-University Engagement (ICUE)... 7 Carleton Student Experience Office (SEO)... 7 Project Reach... 7 Approach... 8 Resources... 9 Expected Results... 11 Project Logic Model... 13 Performance Measurement Framework... 14 2
List of Acronyms CSL CEP SEO EDC ICUE CACSL RMAF Community Service Learning Community Engaged Pedagogy Group Student Experience Office Educational Development Center Carleton Initiative for Community-University Engagement Canadian Alliance for Community Service Learning Results Based Accountability Framework 3
Introduction This document presents the Project Profile, Logic Model and Performance Measurement Framework for the Carleton University Curricular Community Service Learning (CSL) Pilot Project. It is intended to provide guidance to the Carleton Community Engaged Pedagogy (CEP) Group in the pursuit of strengthening curriculum-based, community service learning and community-based research at Carleton University. Background Community Service-Learning (CSL) is a model of experiential learning that combines classroom learning with community service and research work designed to achieve community goals and to instill in students a sense of civic engagement. It started developing across North America in the early 1990s. Recently, community service learning in Canada has evolved significantly. The Canadian Alliance for Community Service Learning (CACSL), an organization headquartered at Carleton University, supports post-secondary institutions, faculty and community organizations to ensure the effective growth of CSL in Canada. Its vision is students, educators and communities learning and working together to strengthen individuals and society. CACSL defines CSL as an educational approach that integrates service in the community with intentional learning activities. Within effective CSL efforts, members of both educational institutions and community organizations work together toward outcomes that are mutually beneficial. Much of the genesis of support for CSL at Carleton springs from the McConnell Foundation a philanthropic organization based out of Montreal. In 2005, the McConnell Foundation issued a call for proposals for a grant it created to fund CSL programs and initiatives at Canadian universities. While Carleton s funding application was ultimately unsuccessful, the application process confirmed the desire to move forward with CSL initiatives at Carleton. A keystone learning objective from the McConnell Foundation CSL strategy is as follows: 4
Many students have a strong appetite for hands-on experience that is professionally relevant and personally rewarding. Effective CSL placements can have a profound effect on these individuals, improving their academic performance, nurturing a sustained commitment to volunteer activity, increasing their job prospects and stimulating civic engagement. Alignment with Institutional Priorities Engagement with the community is one of the four key pillars of Carleton s new strategic plan entitled Defining Dreams. The following excerpt from that plan demonstrates how the CSL is aligned with Carleton s overall strategic objectives: As a designer and custodian of the future, Carleton must serve our academic and all the communities and the local and international environments with which we interact. The university has the responsibility and the intellectual resources to lead our local, regional, national and international communities as stewards not only of community well-being but of healthier, more sustainable communities. Furthermore, four of the five courses in the pilot project have a specific focus in the field of environment and sustainability another of the key pillars of Carleton s strategic Plan. The Challenge Carleton University historically has a strong attachment to community engagement initiatives. Specifically it has a well developed co-op program and co-curricular community service opportunities that are managed from the Student Experience Office (SEO). However, CSL at Carleton University like most other post secondary institutions in Canada is at a genesis stage. In 2009, a group of academic staff and communityminded individuals, chaired by Peter Andrée from the Department of Political Science, formed the Community Engaged Pedagogy (CEP) Group. CEP members share an interest in promoting student learning and engagement through a variety of means that connect students and their community. The immediate challenge for CEP is to grow CSL at Carleton from an introductory level with limited institutional and financial resource support. In an effort to build a strategic foundation for CSL at Carleton University the CEP committee launched the CSL Pilot Project in the fall of 2010. 5
Project Profile Project Description The Carleton Curricular Community-Service Learning (CSL) Pilot involves five professors teaching courses in each of Carleton s five faculties (Business, Public Affairs, Arts and Social Science, Engineering and Science) over the 2010-2011 academic year, working together with the Student Experience Office (SEO), to provide CSL opportunities in the context of their courses. These courses range from first year undergraduate to graduate level. CSL is a method of experiential learning that has students working alongside community organizations (mostly in the non-profit sector) to fulfill specific tasks or answer research questions. Unlike traditional volunteerism, CSL has a learning outcome attached to the service, as well as a reflection component to help students draw connections between their service experience and their education in the classroom. Four of the five courses in the pilot have a specific focus in the field of environment and sustainability, one of the key pillars of Carleton s new strategic plan. The primary goal of the Carleton Curricular CSL pilot is to develop an understanding of best practices for course-based CSL at Carleton, as well as a growing level of expertise among staff and faculty members in this area. Objectives The CSL Pilot Project had three primary objectives: 1. Achieve tangible progress in improving the awareness of course-based CSL and gain institutional support for CSL initiatives at Carleton. 2. Develop an understanding of best practices for course-based CSL at Carleton, as well as a growing level of expertise among staff and faculty members in this area. 3. Foster a multi-disciplinary approach by bringing in students and faculty from varying disciplines to share their perspectives and CSL experience in a collaborative way. 6
Stakeholders Carleton University Engagement with the Community is one of the four key pillars of Carleton s new strategic plan. The CSL Pilot Project is in alignment with this overall strategic objective. Community Engaged Pedagogy Group (CEP) CEP comprises a group of academic staff and community-minded individuals who share an interest in promoting student learning and engagement through a variety of means that connect students and their community. The CSL Pilot is an intervention from this group. Carleton Initiative for Community-University Engagement (ICUE) A committee chaired by Ted Jackson from Carleton s School of Public Administration with a mandate to foster community engagement as a pre-condition to the development of social innovation. CSL is part of the academic conditions required for a successful community-university partnership. Carleton Student Experience Office (SEO) The SEO develops and facilitates co-curricular programs that encourage community service. Examples are the Alternative Spring Break and Days of Service programs. The curriculum based CSL pilot will be complimentary to the SEO activities and both will serve the larger strategic objective of community engagement. Project Reach This section provides an overview of the target groups and immediate beneficiaries of the CSL Pilot Project. The dimensions of CSL which will be fostered in each target of the pilot are also outlined below. 1. Community Participants: community organizations involved in the CSL pilot program (e.g. charities; non profit organizations; non governmental organizations; church service groups; community associations, etcetera). These groups benefit 7
directly from the Pilot Project by having the students provide human resources and skills for short term projects and research assignments. 2. Student Participants: undergraduate and graduate students enrolled in pilot project courses. CSL-based courses provide context to classroom learning through practical experience and exposure to citizenship leadership opportunities. Students are able to provide feedback on their CSL experience through structured reflection exercises. 3. Faculty Participants: the faculty members delivering the CSL courses in the pilot and those attending community engaged workshops, mentoring sessions and focus groups. These faculty members will have their course quality enhanced by incorporating CSL into their curricula. Participants will be recognized as practitioners and CSL champions. Approach The pilot project is an attempt to establish a foundation of CSL at Carleton University. At the end of its two-year time frame the university will have migrated from a genesis stage to an intermediate level of service learning. The hallmarks of this transition will be the following: Institutional support for CSL; Students will feel that CSL enhanced their skills and knowledge; many will develop a personal commitment to engaged citizenship; Faculty support for design and implementation of CSL courses; Measurement of course quality and impact; The start of a formal community partner network. The objectives of the pilot will be achieved by the implementation of three activity streams: 8
1. Promotion of CSL and CEP at Carleton will increase awareness of CSL and through networking and published success stories grow the number of faculty conducting CSL courses. Furthermore, community partner presentations will establish their needs leading to a proper alignment of the student resources and ultimately to a reciprocal relationship of community university engagement. The CEP group will develop a dedicated web site to disseminate most of the CSL information. Existing university vehicles like the Carleton Now electronic newsletter will also be accessed to showcase CSL initiatives. 2. Developing and Delivering CSL Courses will create a benchmark of the courses out of the pilot gate and allow the measurement of course quality and impact both for the students and community participants. The pilot will set the groundwork for gauging both community partner and student satisfaction with CSL projects. At this time the pilot has five faculty members actively delivering CSL courses. We anticipate the number to increase during the pilot s lifespan. 3. Research and Continuous Improvement will deliver a wide range of faculty support activities such as EDC workshops, focus groups and CEP mentoring. The development by the CEP of best practice tools and case studies will increase faculty knowledge and competence in CSL. The development of collaborative tools to support CSL course development will also increase faculty participation by instilling confidence and some mastery in CSL course delivery. Finally, all of the collaborative efforts in support of CSL during the pilot will reach an action threshold. At this point we should see a shift within the Chief Academic Officers from an intrigued and investigatory stance to institutional recognition of the value of course-based CSL. Resources Community Based: There is a wide array of community organizations with diverse needs in Ottawa, as well as in neighbouring municipalities and rural communities. The 9
opportunities are vast for partnering with local organizations that could benefit from the knowledge, expertise and a work ethic of Carleton students enrolled in CSL pilot project courses. In addition to the pilot participants there are a number of faculty using a community service component in their teaching and are already aligned with community organizations. These contacts can be added to the formal network. Throughout the pilot project an effort will be made to actively solicit new community partners as well as receive enquiries and organize presentations by the community-based organizations where needs can be assessed. University Based: The keystone resources are the students. The pilot draws from both undergraduate and graduate students across all five of Carleton s faculties. The faculty members and their teaching assistants along with the following university-based stakeholders in the pilot are pivotal to its success. The Community Engaged Pedagogy Group is currently the driving force behind all aspects the pilot project and CSL at Carleton. The Educational Development Center will provide resources related to the research and continuous improvement stream of the project. The Student Experience Office will provide administrative support through the dedication of one half-time equivalent project coordinator. The Carleton Communications department can help with CSL promotion and awareness by aiding in dissemination of success stories and building a CSL web site. The Canadian Alliance for Community Service Learning, while not under the auspices of Carleton, is housed at the university. It is the repository of a vast amount of CSL support materials and literature which can be easily accessed. CACSL has collected expertise in CSL development across Canada and is a willing resource for the pilot. 10
Financial Resources Budget Assumptions Service Learning Coordinator Community Organization Honoraria CSL Workshops/Roundtables Curriculum Development Promotion Salaries Program Costs 0.5 FTE Base salary and benefits assumed to be additional 18% Stipends for supervisors, overhead costs, participation in monitoring and evaluation Total $1000 Year 1 and 2 delivered by coordinators at no additional costs. Costs associated with the space and delivery of the sessions (possibly through EDC) 0.75 workload credit for faculty in year one to design core CSL courses Web site development costs; associated printing fees; community partner presentations. Monitoring and Evaluation Evaluation Design Provided graduate students enrolled in PADM 5420 D Final Evaluation 25 days at $900 per day this cost can be offset by using in house resources Annual Reflections Preparation and event costs of $1000; associated space and participant expenses Expected Results The expected results of the activities undertaken by the CSL Pilot are as follows: During the course of the Pilot Project: Participating faculty are able to share experiences and co-ordinate the exchange of lessons learned and best practices. Participating students are able to recognize they have a community service element to their learning and understand its value. They will reflect on their experiences offering invaluable feedback to the pilot project courses. 11
Participating community organizations will benefit from student research or services. They will be able to provide valuable feedback on how CSL projects impacted their ability to satisfy the organization s mission. The chief academic officers (i.e. Academic Provost) are intrigued and learning about the value of CSL. 12 Months after Pilot Project Completion: There will be more participating faculty members from a diverse range of disciplines once the pilot has branded CSL at the university and provided adequate course start up support. CSL course quality and impact are measured. Students will feel that CSL enhanced courses increased their skills and knowledge. They will develop a personal disposition associated with engaged citizenship. There will be a growing network of community organizations with a clear understanding of the reciprocal relationship between the university and community partner. The strengths, assets and needs of these organizations are captured to properly align the student resources. The chief academic officers are supportive and recognize how curricular based CSL can be reflected in the university mission along with possible financial support. 12
Project Logic Model Inputs Activities Outputs Outcomes Faculty / Instructors Teaching Assistants Students (undergraduate and graduate) Community Organizations Community-Engaged Pedagogy (CEP) Group SEO Office (Project Coordinator,.5 FTE) Carleton Communications Department Carleton Administration Educational Development Center (EDC) Canadian Alliance for Community Service Learning (CASL) Promoting CSL and CEP at Carleton Developing and delivering CSL courses Delivering EDC workshops and roundtable sessions Mentoring activities Building faculty collaboration tools to support development of CSL CSL articles and project success stories Community partner presentations CSL courses CSL term projects and community research assignments Student reflection exercises CSL best-practice tools and case studies CSL lessons-learned Resource and incentive recommendations to university administration Inventory of CSL initiatives, research and community partners Immediate Intermediate Long-term Increased awareness of CSL courses and projects Increased awareness of community organizations needs Increased community partner satisfaction with CSL projects Increased student engagement and satisfaction with course material Increased faculty knowledge and competence in CSL Institutional recognition of value of curricular, coursebased CSL Faculty recognition of value of curricular, course-based CSL Increased communityuniversity engagement Continuous improvement of CSL courses Increased network of satisfied and participating community partners Increased institutional resources for CSL and CEP at Carleton University Increased Faculty participation in CSL and CEP at Carleton Create a sustainable culture of community engagement at Carleton
Performance Measurement Framework Logic Model Box Elements Performance Indicator Data Source Measures CSL articles and project success stories Increased promotion of CSL and CEP at Carleton Carleton Publications o Carleton Web site o Carleton s Dedicated Community Web site (URL TBD) o Carleton Now o Charlatan Number of articles and project success stories published Types of Measures Output Y/N Goal Displacement? Target Assumption 2-3 articles per semester Outputs Community partner presentations Increased promotion of Community Initiatives Pilot Project records Number of presentations delivered Number of registrations (if applicable) Output 1 community partner presentation per semester CSL courses Increased number of CSL course options available to undergraduate and graduate students Carleton Academic Calendar Pilot Project Records Number of CSL courses offered in Pilot Courses from each of the 5 faculties Output At least 1 one course offered in each of Carleton s five faculties 14
Outputs Elements Performance Indicator Data Source Measures CSL term projects and community research assignments Student Reflection exercises Increased number and variety of CSL projects and community research assignments Number of CSL courses that have student reflection component integrated into the learning experience CSL course syllabuses Pilot project records Pilot course records Number and variety of CSL projects and community research assignments offered in Pilot Written or presentation reflection component (i.e. field journal, project report etc) Types of Measures Output Output Y/N Goal Displacement? Target Assumption 2-3 new community partner projects or research opportunities introduce into Pilot each semester 90% of CSL Pilot courses have reflection component CSL best-practice tools and case studies Increased levels of CSL and CEP mentorship and training Pilot project data stores (i.e. network drive) EDC workshops records Number of presentations Number of research reviews Output Creation of a basic tool kit of materials on CSL to be distributed to all on-boarded Faculty 15
Outputs Elements Performance Indicator Data Source Measures CSL lessons-learned Resource and incentive recommendations to university administration Increased levels of collaboration and knowledge sharing Increased commitment of Carleton Administration Pilot Project Records Pilot Project Records EDC workshop registration numbers Number of case studies Increased membership in CEP Group Number of committees served by CEP members Types of Measures Output Output Y/N Goal Displacement? Target Assumption 10% increase in EDC workshop registrations 15% increase in CEP Group membership by Pilot close Yes Annual recommendations are made to the office of the Academic Provost Inventory of CSL initiatives, research and community partners Increased ability to report and track CSL activities at Carleton CSL initiative and community partner inventory Faculty Survey Faculty Focus Groups Level of knowledge of CSL activities at Carleton Output 75-90% of all CSL related activities at Carleton are known and tracked by CEP Group 16
Elements Performance Indicator Data Source Measures Increased awareness of CSL courses and projects Increased awareness levels Faculty Survey Student Survey Change in Level of faculty and students reporting increased levels of awareness of CSL activities Types of Measures Immediate Outcome Y/N Goal Displacement? Target Assumption 15% increase in awareness of CSL activities from baseline to Pilot close Immediate Outcomes Increased awareness of community organizations needs Increased community partner satisfaction with CSL projects Increased student engagement and satisfaction with course material Increased awareness levels Increased satisfaction levels Increased satisfaction levels Community organization presentations exit survey Project evaluation survey Project exit interviews Course evaluation and feedback survey Level of faculty and students reporting increased levels of awareness of community organization s needs Satisfaction rating scale Satisfaction rating scale Feedback comments Immediate Outcome Immediate Outcome Immediate Outcome 15% increase in awareness of community organizations needs from baseline to Pilot close 80% of community partners express satisfaction with CSL projects 80% of students express satisfaction with CSL courses 17
Elements Performance Indicator Data Source Measures Increased faculty knowledge and competence in CSL Increased knowledge and competency levels EDC Workshop intake and exit surveys EDC Workshop participant interviews Self-reported gained knowledge and competency levels Types of Measures Immediate Outcome Y/N Goal Displacement? Target Assumption 10% increase in reported knowledge and competency levels Immediate Outcomes Institutional recognition of value of CSL course component Increased commitment of Carleton Administration Carleton Faculty Deans and Provost interviews Number of formal recommendations accepted, institutionalized Immediate Outcome Yes Course-based, curricular CSL becomes part of Carleton s Academic Plan and Provost Performance Measure Faculty recognition of value of CSL course component Increased commitment of Carleton Faculty Community Engaged Pedagogy (CEP) Group Pilot project records EDC workshop registration numbers Increased membership in CEP Group Immediate Outcome 10% increase in the number of faculty running courses in CSL from baseline to Pilot close 18
Elements Performance Indicator Data Source Measures Types of Measures Y/N Goal Displacement? Target Assumption Intermediate Outcomes Increased communityuniversity engagement Continuous improvement of CSL courses Increased number of community organizations partnering with Carleton on CSL initiatives Increased number of courses with CSL component Increased satisfaction levels Pilot project records Student and Faculty community engagement surveys Student and Faculty community engagement focus groups Project evaluation survey Project exit interviews Course evaluation and feedback survey Pilot project records Engagement rating levels Satisfaction rating scale Feedback comments Intermediate Outcomes Intermediate Outcomes Yes 10% increase in the number of students and faculty engaged in CSL from baseline to Pilot close 80% of students and community partners express satisfaction with CSL projects Increased network of satisfied and participating community partners Increased number of community partnerships Pilot project records Baseline Pilot milestone Intermediate Outcomes 10% increase in the number of faculty engaged in CSL from baseline to Pilot close Increased institutional resources for CSL and CEP at Carleton University Increased commitment of Carleton Administration Teaching Agreements and contracts Number of resources and incentives offered to CSL Faculty and Instructors Immediate Outcome Yes CSL courses receive special incentives and resources (i.e. TAs, teaching credits) 19
Intermediate Outcomes Long term Outcome Elements Performance Indicator Data Source Measures Increased Faculty participation in CSL and CEP at Carleton Create a sustainable culture of community engagement at Carleton Increased participation levels Increased student, faculty and community organization satisfaction levels with CSL projects Increased number of community partnerships Increased commitment of Carleton Administration Carleton Academic Calendar CEP Group records Pilot project records Pilot project records Student and Faculty community engagement surveys and focus groups Number of Faculty offering CSL-based courses Number of CEP Group members Number of EDC Workshop registrations Number of CSL based teaching Agreements and contracts Increased awareness levels of community organizations needs Increased participation in CSL activities at Carleton Types of Measures Immediate Outcome Long Term Outcomes Y/N Goal Displacement? Yes Target Assumption 10% increase in the number of faculty engaged in CSL from baseline to Pilot close CSL and CEP become recognized and viable components of Carleton s Academic Plan and external reputation 20