Boston University – Professional Development Practice – Establishing a Structure for Assessing Student Learning

Print Friendly

CGS1This practice engages faculty representatives of the four Divisions of the College of General Studies in ongoing collaborative work evaluating student work collected in e-Portfolios and representing four semesters of coursework in the program.  The practice establish norms for rating student work for assessment purposes but also allows the faculty to reflect on course objectives as they relate to assessment outcomes.

Description

Ultimately, the committee’s work will produce a more streamlined assessment practice of outcomes assessment that we hope will take hold at CGS but also influence practices across the BU campus.  Gathering significant data about student learning in our program is also an important goal of the practice.

Type of professional development

Our practice functions as a two-year assessment committee/workshop funded by the Davis Educational Foundation.  The group meets once a month during the school year; work continues during the summer months.

Framework

At the end of the grant period, the group will continue its work as part of a regular cycle of inquiry, reflection, and integration connected to the ongoing work of assessment.

Participants and seminar leadership

The intent of the practice was to gather representative faculty members so that stakes would be shared in the assessment work and so that standards could be established across disciplinary and divisional lines.  The ten participants represent the Humanities, Rhetoric, Natural Science, and Social Science Divisions of CGS and include both freshman and sophomore faculty.  Most are full-time faculty members; the group leader is an Assistant Dean.  Three of the members of this group are also part of the campus e-Portfolio team.

 Successes

The practice is still in its initial phase, so we are still working with the first group of ten faculty members.  After reviewing and assessing multiple student porfolios both individually and as a group, it is clear that the group is arriving at a shared sense of standards and expectations that will allow us to assess student work going forward with a high degee of reliability as documented in scores collected from a sample group of portfolios rated by the committee.  Just as importantly, we have been able to identify common pedagogical goals for the use of e-portfolio that will guide our collective work as a college.  Because of our collaborative work, we see the need to establish common practices for assigning and posting work that will give students the opportunity to demonstrate their progress in all areas of the rubic across four semesters and will be able to implement these changes as we move forward.  Our work has also opened the way for discussion of curriculum change and key revisions to the jointly-developed rubric we have been using in our work.

Challenges

Implementing the changes we hope for will be a challenge, but working as a team with faculty from across the college certainly increases the sense of “buy-in” across the divisions.  Stipends to faculty participants for assessment work with e-Portfolios have also been important for sustaining faculty engagement.

Connections

Our earlier reflective practice (“Student Reflection Assignments”) is directly related to this Professional Development practice.  Many of the same faculty who developed and implemented our reflective practice remain involved in professional development.  Our reflective practice asked students to use the rubric of student outcomes at the end of the school year to identify work in the portfolio that demonstrated their progress in each category.  Now, faculty are using the same rubric and student portfolios to assess student learning.

Resources

The CGS rubric is a key resource for our Professional Development Practice.  Through working closely with sample student ePorfolios and this rubric, faculty familiarize themselves with these learning outcomes in ways that will allow them to think carefully about designing course materials and assignments that will give students the opportunity to demonstrate their strengths in these areas.  We have collected data on student progress over four semesters in our assessment during the Summer of 2012 that identifies clear growth in most areas of the rubric; in those areas where progress is slower, we are able to consider how to make meaningful changes to the curriculum.

The following rubric is to be applied to student electronic portfolios at various critical junctures during their four semesters at the College of General Studies.  Students will gather samples of their work to reflect each of the dimensions of their coursework at the College listed in the left-hand column of the rubric.  The fourth scale level represents outstanding or excellent accomplishment in a given dimension, the third indicates competence, the second represents a developing or nascent skill, and the final scale level,“no mastery,” indicates that a student is not yet performing in a way that meets the criteria set forth in the rubric in a given dimension.  It is important for the College to communicate the goals described below to students, so that their work at CGS is purposeful.  Students should be able to articulate what they are working towards and where they are in their performance at any given stage of their coursework.

Over the course of four semesters, students at CGS develop 1) the ability to communicate in writing and orally, 2) the skills needed to gather, analyze, document, and integrate information, 3) a detailed understanding of historical processes, literary and aesthetic movements, and specific cultural contexts, 4) an understanding of the different “ways of knowing”—modes of thought, concerns, and methodologies—in the humanities, social sciences, and natural sciences, 5) the ability to use quantitative methods in the natural and social sciences, and 6) the ability to integrate knowledge and modes of thinking drawn from two or more disciplines to produce an interdisciplinary understanding of complex problems and engage in perspective-taking.

The AACU “VALUE” rubrics ( see http://www.aacu.org/value/rubrics/index.cfm) for critical thinking, integrative learning, written and oral communication, information literacy, and inquiry and analysis all apply directly to the skills students gain in our program.  Students not only develop disciplinary knowledge pertaining to course problems or themes an an understanding of modes of thinking (“ways of knowing”) in discrete disciplines, they also are able to make connections among their courses and purposefully employ “ways of knowing” and specific disciplinary knowledge across disciplinary boundaries.

CGS Assessment Rubric—Draft version

level 4excellent level 3competent level 2developing level 1no mastery
Written and oral   communication Demonstrates   detailed attention to and successful execution of a wide range of conventions   particular to  a specific discipline   and/or writing task (including organization, content, presentation,   formatting, and stylistic choices); uses graceful language that skillfully   communicates meaning to readers with clarity and fluency, and is virtually   error-free Demonstrates   consistent use of important conventions particular to a specific discipline   and/or writing task; uses straightforward language that generally conveys   meaning to readers.  The language in   the portfolio has few errors. Follows expectations   appropriate to a specific discipline and/or writing task for basic   organization, content, and presentation; uses language that generally conveys   meaning, although there may be problems with clarity and the writing may   include some errors. Attempts to use a   consistent system for basic organization and presentation; uses language that   sometimes impedes meaning or clarity. Contains errors in usage.
Gathering,   analyzing, and documenting information Synthesizes in-depth   information from a range of high-quality, credible, relevant sources that are   appropriate for the discipline and genre to develop ideas and documents these   sources fully using MLA or Chicago style. Consistently   presents in-depth information from credible, relevant sources appropriate to   the discipline and genre to support ideas. Documents sources with few errors   or exceptions using MLA or Chicago style. Demonstrates an   attempt to use credible and/or relevant sources to support ideas and to   document these sources properly using MLA orChicagostyle. Minimally attempts   to use sources to support ideas in the writing; these sources may not be   correctly documented using an acceptable style manual and/or may not be fully   relevant to the task at hand.
Awareness of   specific historical, literary, and cultural contexts Uses appropriate,   relevant, and compelling content and sufficient detail to illustrate mastery   of the subject, including historical, literary, and cultural contexts. Uses appropriate,   relevant, and compelling content to explore ideas within the context of the   discipline(s), but many not yet provide sufficient detail or illustrate   mastery of historical, literary, and cultural contexts. Uses appropriate and   relevant content to develop and explore ideas through most of the work; does   not display a consistently clear or adequately detailed understanding of   historical, literary, and cultural contexts. May use appropriate   and relevant content to develop simple ideas in some parts of the work.
Rhetorical and   aesthetic conventions Demonstrates a   thorough understanding of context, audience, purpose.  Makes skillful rhetorical choices and shows   deep appreciation for literary and aesthetic conventions and their effects. Demonstrates   adequate consideration of context, audience, and purpose.   Understands rhetorical effects and shows   appreciation for literary and aesthetic conventions and their effects. Demonstrates some   awareness of context, audience, and purpose.    Can identify rhetorical strategies and shows some appreciation for   literary and aesthetic techniques and conventions. Demonstrates minimal   attention to context, purpose, and audience.    May not be aware of rhetorical effects of one’s own work or of   rhetorical strategies and literary techniques in works analyzed.
Critical Thinking   and perspective-taking Questions are   examined from a range of viewpoints, taking into account the complexities of   an issue.  Conclusions and related   outcomes are logical and reflect the student’s informed evaluation and   ability to place evidence and perspectives discussed in priority order. Specific position   takes into account the complexities of an issue and acknowledges other   viewpoints.  Conclusion is logically   tied to a range of information. Information is   presented with some interpretation or evaluation, but not enough to develop a   coherent analysis or synthesis.    Acknowledges different sides of an issue, but may be more aware of   others’ assumptions than one’s own (or vice versa). Specific position is   stated, but is simplistic and obvious.    Conclusion is inconsistently tied to some of the information   discussed.  Information from sources is   presented without interpretation or evaluation.
Integrative and   applied learning Makes insightful   connections across disciplines and perspectives.  Draws conclusions by combining examples,   facts, theories or methodologies from more than one field of study to arrive   at a sophisticated interdisciplinary understanding. Makes connections   across disciplines and perspectives by independently combining examples,   facts, theories, or methodologies from more than one field of study. When prompted,   connects examples, facts, or theories across disciplines and   perspectives.  May not show a strong   understanding of how methodologies differ across fields of study or could be   applied in a new situation. When prompted,   presents examples, facts, or theories representing different disciplines and   perspectives.  Shows a limited   interdisciplinary understanding.
Quantitative methods Uses quantitative   analysis of data as the basis for deep and thoughtful judgments, drawing   insightful and carefully-qualified conclusions from this work. Uses the   quantitative analysis of data as the basis for competent judgments, drawing   reasonable and appropriately qualified conclusions from this work. Uses the   quantitative analysis of data for basic judgments, drawing plausible   conclusions from this work. Uses the quantiative   analysis of data for tentative judgments; hesitates to draw conclusions from   this work.