Link
Print Friendly

CGS33In 2009, Boston University’s College of General Studies (CGS) received funding from the Davis Educational Foundation to assess its general education program using Digication, an ePortfolio platform. Since 2009, each CGS student has maintained a single ePortfolio for all CGS courses, and our college’s Writing Committee (chaired by C2L member Gillian Pierce) developed a rubric of learning outcomes for our program and an assessment score sheet. During the summers of 2012 and 2013 we conducted an assessment of over 100 student ePortfolios each summer and now have collected and evaluated two years of data about our program. The results are encouraging in that they seem to suggest that student improvement in our program is above the national average.

Authors:
Natalie McKnight, Interim Dean of the College of General Studies; Gillian Pierce, Assistant Professor of Rhetoric; Amod Lele, Educational Technologist; John Regan, Senior Lecturer, Department of Rhetoric.

 

Our Scaling Up Story

Current Status

Currently all 1,100 students at Boston University’s College of General Studies use ePortfolios to enhance and archive their learning in our two-year, core-curriculum program. Assessment methods include a rubric and scoring methodology to collect data and chart student progress. In 2009, with support from the Davis Educational Foundation, our college began a program-wide use of student eportfolios. These ePortfolios have encouraged students to make connections among their classes, reflect on their learning, enhance their academic projects with videos and other images, and archive their writings and extra-curricular activities. Our experience with Digication indicates that ePortfolios offer a rich means of assessing student progress during their two years at our college. In 2011 we received additional funding from the Davis Foundation. Using a common rubric based on the AAC&U VALUE rubric , we have currently completed a full-scale assessment of 118 student eportfolios to gather quantitative and qualitative data that will indicate patterns of student progress. At this point, we would consider our work on assessment successful and are pleased with how the use of ePortfolio is facilitating student-centered learning.

Regarding ePortfolio as a connector and catalyst for change on our campus, from the beginning of our project, we have had support from the School of Education for our assessment work at the College of General Studies. Katie Schooler, a graduate student in the School of Education, provided support by coordinating our activities and was an invaluable team member until she graduated in 2012.  We have also reached out to other colleges at our university to promote the wider use of ePortfolio on our campus. Last year we worked with our Dean to host a luncheon for BU administrators and faculty from other colleges to promote ePortfolio use and showcase our students’   work. We have also reached out to and have met with administrators, faculty, and support staff from various colleges, departments and programs including the College of Arts and Sciences and the School of Management and Government. We are also planning to present our work at a workshop sponsored by the Center for Excellence and Innovation in Teaching.

The most significant challenge we face is faculty buy-in (see Next Steps below).

Catalyst and Connector

    In terms of Randy Bass’s concept of ePortfolio as a catalyst for change, we have seen some progress in the “the shift to a student-organized  view of learning, bridging curriculum and co-curriculum” (Bass “Disrupting Ourselves: The Problem of Learning in Higher Education“). In the last few years since the introduction of ePortfolio, we have seen a degree of movement towards learning in which “learners pull from knowledge resources and offerings” (Bass Disrupting“)–for example, ethnographic Rhetoric assignments that involve student observations of campus clubs or Science assignments in which students observe a Boston neighborhood, collect data, and analyze and evaluate the data in visual formats. Eportfolio (and our rubric) has clearly encouraged many of our faculty to think beyond course goals to consider broader learning contexts.

We have also seen more evidence of “an institutional conversation on student learning, moving towards a learning-centered culture and structure” (Bass “Disrupting“) and the fact that our students create a single ePortfolio for all of their courses has helped this conversation.  When a viewer or user enters a student ePortfolio and sees the range of student course work and other activities, one gets an immediate reminder of the interconnectedness of the student learning experience—that the parts connect to a larger whole. Since our faculty have had greater and more immediate access to what is going on in each other’s classrooms, institutional conversations on student learning can now be informed by a deeper, more vivid sense of the “whole” of the student’s experience.

Finally, Randy Bass’s observation that “ePortfolio practices must straddle both established and emerging paradigms of higher education” is perfectly illustrated in the structure we have adopted for the student ePortfolios. While we ask students to organize their ePortfolios in a traditional and linear way though subdividing by course, (i.e. Rhetoric 101, Social Science 101, etc.), a more integrative and iterative approach one  would be to organize by learning outcomes (our rubric categories)—but we’re not there yet!

Developmental History

The history and key stages of our ePortfolio work at the College of General Studies is as follows:

September 2007-May 2008:  Work on a rubric begins in college committees.

September 2008-May 2009: Awarded Grant from Davis Foundation to train faculty in ePortfolio.

Rubric informed by AAC & U VALUES models.  First draft of rubric introduced to faculty.

Summer 2009:  Freshman faculty trained in Digication platform.

Sept 2009-May 2010:   All first year students use ePortfolio. Rubric modified based on faculty input.

Summer 2010:  Sophomore faculty trained on Digication.

January 2011:    Join Connect to Learn (C2L) project.  All first and second year students (approximately  1100) now use ePortfolio.

Summer 2011: Assessment Score sheet developed.

September 2011-May 2012: Awarded a second grant from Davis Foundation. Assessment committee holds norming sessions.

Summer 2012:  Full-scale assessment of 118 student ePortfolios begins.

Sept. 2012-May 2012.  Analysis and evaluation of first set of assessment data begins. Committee continues norming sessions.

Summer 2013: Full-scale assessment.

Sept 2013-present. A second year of assessment data collected and analyzed. Formal advising component added.

During our project, we strategically connected with and received support from BU’s School of Education and office of Information Services and Technology.  One key means of support that helped our project immensely was the creation of a full-time position in educational technology, and this professional has done outstanding work in providing help in nearly all aspects of our project. This support includes the creation of a Portfolio about Portfolios designed for faculty use that goes beyond the instructions given by the platform provider and offers comprehensive, easy-to-follow instructions with BU-specific screenshots. In addition, the creation of the Center for Interdisciplinary Teaching  based at Boston University’s College of General Studies, has provided great support for our work.

Regarding key decisions, we decided early on that students should maintain a single ePort for all of their CGS courses instead of individual ones for each course.  This decision required us to develop a format for student ePorts and to create and distribute instructions for students on how to follow this format.  While this decision created more work for us and for the students as they set up their ePorts, overall there were no major problems.  As we crafted our instructions for students, we made another major decision regarding privacy settings. Much of our faculty’s resistance to ePortfolio centers around concerns over plagiarism and academic integrity—in short, the fear that ePorts give students access to each other’s work.  Therefore, our students were instructed to use privacy settings that restricted viewership to their professors, advisors, and themselves.  While we realized that the impact on the pedagogical potency of ePortfolio might be diminished in some cases, many of our faculty would have never agreed to have their students post their materials if viewership of the ePorts was less restricted.  We do plan on revisiting and perhaps revising this decision as we move forward, but at the time we are convinced that our decision was crucial for full faculty compliance.  Finally, perhaps our most significant decision was to seek (and receive) additional funding from the Davis Foundation; this funding allowed us to maintain our project’s momentum and expand our assessment committee to include even broader faculty participation.

Connection to Core Strategies

The two strategies that have most advanced our use of ePortfolios are 1) connecting to high-impact practices such as first-year experiences and capstones, and 2) building strategic connections to outcomes assessment.

High-Impact Practices

We have implemented a self-reflection assignment at the end of the students’ freshman year that asks them to use our rubric in examining the work they have archived on their ePortfolios over the course of the freshman year. Students are asked to comment on which assignments show progress in the competency areas noted in the rubric. This assignment has been illuminating for both them and for us. Now that we use it, it seems hard to believe that we had never systematically asked students to reflect on their development before , but we had not.  Eportfolios make it possible for the students and the faculty to see student progress in terms that transcend grades and credit hours and that is a significant advancement for all of us. The more we can get students to think in terms of their development as thinkers and communicators the deeper their learning will be.

We also implemented the use of ePortfolios and self-assessment assignments in the sophomore Capstone project.  The Capstone is a group-written, 50-page research proposal that offers a real-world solution to a contemporary problem.  Two years ago we started asking students to post in their ePortfolios all the individual work they do for Capstone and to keep a log of their weekly activity on the project. This has been an enormous asset to professors as it gives us the ability to ascertain individual’s contributions to the group project. In the past we always gave an individual and a group grade on the Capstone, but many of us felt we didn’t have enough evidence for the individual grade. Eportfolios have solved that problem.  Sophomore professors, many of whom were originally very skeptical about ePortfolios, have been persuaded of their usefulness because of their ability to offer a window into the individual’s input on group work.

We also ask students to post a self-assessment essay along with their Capstone. In the essay, students reflect on what they learned from the project and how the Capstone relates to their other coursework over their two years in our program.  These essays have been very illuminating for us:  they reveal what students feel they have benefitted most from and where they feel less well-prepared—information we can use as we contemplate curriculum reform.

Outcomes Assessment

Using ePortfolios for outcomes assessment for our program has probably been the most important practice in the “scaling up” of our ePortfolio project.  Even though “assessment” can rub many academics the wrong way, most can see the necessity of trying to determine whether our program accomplishes what we say we accomplish. Obviously we are being asked by accreditors to show what we have in place for assessment, but beyond that, analyzing the effectiveness of our program is something we should want to do for its own sake and for the sake of our students. Eportfolios, and the rubric we’ve developed to use across all disciplines in our general education program, enable us to analyze students’ development in various competency areas, such as writing skills and analyzing and documenting data, in a way that goes beyond their performance in individual courses. Our initial data from our analysis of 118 ePortfolios in 2012 shows that, on average, students make progress in all rubric areas, although not always as much progress as we’d like to see, and our second year of assessment shows similar results.  This is information we can use as we embark on curricular reform, and the data will enable us to compare student progress before and after implementing certain reforms. We’ve simply never had this kind of information before, and many constituencies around and beyond our campus are interested in the assessment system we’ve established since they are looking to establish assessment projects of their own.

Next Steps

We will be continuing our ePortfolio assessment project, meeting monthly to discuss how to interpret data from our assessments and continuing to assess student ePortfolios as a group to maintain inter rater reliability. We’ve already hired a student peer mentor for ePortfolios to help students and faculty with any problems they might have in setting up or adding on to ePortfolios, and we have hosted an ePortfolio contest and showcase that gave students more incentive to work on their portfolios. We will also be continuing our efforts to meet with faculty and staff across campus to encourage them to have their students use ePortfolios. We presented a workshop on ePortfolios through the Center for Excellence and Innovation in Teaching in December 2012. In 2013 we made numerous presentations at regional and national conferences including the AAC&U in Atlanta and the national AEEBL confernce in Boston and we have more planned moving forward. We will also begin publishing some of the results of our assessment project.

One of the key challenges we face is faculty and student buy-in, a topic that often comes up in discussions with our C2L peers. In assessing 118 ePortfolios this summer, our assessment committee of 11 faculty noticed many incomplete portfolios. Whole classes—even whole years—were not represented in some (our students have one ePortfolio in which they have sections for each of their courses in our program, and they are supposed to post at least a couple artifacts for each course).  Obviously not all are doing that, and equally obvious, not all faculty members are asking them to do so. We are hoping that the core strategy of “engaging students” (through peer mentors and the spring contest and showcase) will yield more complete portfolios for us to examine next summer. The contest may also help with faculty buy-in as well, since faculty will not want to jeopardize their students’ chances of winning a prize and receiving accolades.

 

Connections to Other Sectors of the Catalyst

Pedagogy

Overall, the response to ePortfolio pedagogy has been mixed. In a positive vein, ePortfolio pedagogy has fit very well with our sophomore Capstone project and now is an indispensible part of that project.In addition, many course, especially sophomore Science and first-year Rhetoric courses, have through ePortfolio enhanced their use of visuals and videos in assignments, leading to deeper student engagement and learning. We also require that all freshman do a reflection assignment at the end of the first year; this assignment allows them to evaluate themselves and their learning in relation to the CGS rubric. The sophomore Capstone project has added a similar reflection component, and we do see smaller-scale examples of professors using elements of reflection and social pedagogy in a number of assignments; for example, some social science professors assign readings in social stratification and have students research and document neighborhoods in the Boston area.

Professional Development

In both Fall 2012 and Fall 2013 we added new members to our assessment committee, which helped to “widen the circle” of our efforts. Interestingly, some of these faculty had misgivings about ePortfolio, but bringing them into the loop  has been instrumental in spreading a more positive word about ePortoflio and assessment throughout of our college and helping to alleviate the fears and misgivings of resistant faculty.

Outcomes Assessment

Beginning in the Summer 2013 we are participating in a university-wide assessment initiative in which all campus undergraduate and graduate programs and colleges must identifying demonstrated learning outcomes and make plans to assess student learning within the next three years. One C2L member (John Regan) is the CGS representative on the university committee, and through our CGS assessment project we seem further along in identifying learning outcomes and assessing student learning than many other programs and colleges at Boston University. Therefore, through our experience with assessment, C2L members may be ideally positioned to offer invaluable insights to assessment committee members from other programs and colleges, a fact that may enhance the reputation of our college with our university.

Technology

The use of Digication as an ePortfolio platform has been essential for our project. Overall, the use of Digication has enhanced our project because although some users find it a bit cumbersome  (e.g. multiple steps such as Add, Edit, Save, and Publish are needed to post a document, adjusting the privacy settings is convoluted), on the whole it is user-friendly; nearly every if not every student is able to understand how to make an ePortfolio and how to post material. Moreover, given that some faculty showed resistance to ePortfolio, our chosen platform had to be as user-friendly as possible, and Digication appeared easier to use than other platforms that we looked at, especially open-source ones.

 

Conclusion

Overall, we believe our ePortfolio initiative has had positive results in the following areas:

  • Supporting Outcomes Assessment
  • Enhancing student learning & success
  • Advancing integrative learning—building connections across boundaries
  • Helping Students become more reflective learners
  • Deepening student understanding of course content
  • Strengthening High Impact Practices (such as FYE or learning communities)
  • Educational development–helping students developing their identities as learners

Despite resistance from some faculty members, our initiative has been successful because of our strong institutional support, ability to secure grant funding, and a close-knit, hard-working group of talented individuals led by Natalie McKnight. With the advent of a formal advising component to go along with our continuing efforts, our hope is to further deepen the use of ePortfolio at our college and encourage others throughout Boston University to give it a try.