CSU Long Beach College of Education Accreditation
2b.3 How does the unit maintain records of formal candidate complaints and their resolutions?
Following a program improvement model, the college and its programs collect and analyze data related to candidate performance and unit operations systematically and on a regular basis. The full data collection and reporting process is reflected in the figure titled Data Collection and Reporting Process.
All Initial and Advanced programs have candidate performance data reaching back at least four semesters (fall 2007 through spring 2009). The depth and richness of these data have evolved with time and experience, as the college has learned more about assessment, its needs, and the structure of its assessment system.
Programs collect data on candidates for all of the signature assignments and other data sources across all transition points (Admission, Advancement to Culminating Experience, Exit) whenever these experiences are offered, as indicated in the second to last column of each program assessment plan.
While there is some variation, data collection and analysis is largely centralized. Each term the Assessment Office consults with department chairs to establish a timeline for data collection, with the Assessment Office operations guided by a more general calendar as well. The office uses the list of courses being offered that term to determine what data to expect. For programs that collect candidate assignments and data using TaskStream (the choice on whether or not to adopt TaskStream is left up to each program), the courses for the term are set up in this software system. For programs that collect assignments and data using more traditional paper method, the Assessment Office prepares custom-made Excel data templates for each course, populating the information with candidates' names and ID numbers for faculty to then enter candidate performance data (overall and by rubric criteria). The templates are then returned to the Assessment Office for analysis.
The Assessment Office processes the candidate data in two ways. First, the office generates program-specific reports that reflect candidate performance on SLOs for the given term or year and present criteria-level candidate performance on individual signature assignments). The office then aggregates data across semesters within a year (e.g., 08-09) as appropriate. With the advent of our new database, scheduled to be operative within the next year (we are moving to a system that is easier to maintain, provide greater integration of data, and enhance security), we will have a system that will allow us to provide outcome-specific data when programs are scheduled to report on specific outcomes, thus more easily compiling data across several semesters or multiple years. Second, the Assessment Office takes this program-level data for an academic year and aggregates it around both Conceptual Framework and NCATE Standard 1 elements. These data are reviewed each fall by the Assessment Committee.
Data analysis takes place primarily through the use of Excel. The Assessment Office uses Excel to create the bar graphs (described above) for use by programs in their data discussions, as well as to summarize and present aggregated data like that shown in aggregated data tables. Programs that use TaskStream can also access reports on candidates' signature assignment performance at the level of individual criteria and overall score.
Data management is supported by both the COSS and TaskStream. The COSS serves as the final, official home for nearly all college data related to assessment and it continues to be adapted to serve our evolving needs. This database is composed of two "sides". One side serves as the home for administrative data related to assessment. In this area, learning outcomes are listed and mapped to college, state and national standards, and signature assignments are identified for each learning outcome. In addition, electronic versions of candidate work exemplars are stored in this location for use by faculty in data review and program improvement. At this writing, the COSS is being developed to allow for web-based data entry. It is our expectation that faculty will soon be able to enter signature assignment data directly into the database.
The other side of the database houses candidate data, including demographic data and performance data linked to program-specific signature assignments. Data from the Excel data collection templates are imported into the COSS to maintain a record of candidate performance. This structure allows us to manage and report data at the level of the individual candidate, across the program, by learning outcome or signature assignment, and aggregated by college, state and national standards.
TaskStream serves a similar function for programs that elect to use this software. Faculty scores on signature assignments are automatically captured as the faculty use web-based rubrics to score the assignment. Candidates submit their assignments on-line, and TaskStream maintains an electronic version of every submission, meaning that it provides a comprehensive archive of candidate work. To enable us to connect this data to candidate demographic data, we are currently working on building a process for transferring TaskStream data into our COSS system.
Finally, the college uses both the Assessment Office website as well as course management software (BeachBoard, aka BlackBoard) to provide resources and support for assessment. The site (http://www.ced.csulb.edu/assessment/) provides information on a variety of assessment-related activities, including an overview of the UAS, how to plan for assessment, how to collect and use data, and how to report data. The website includes a combination of documents, links to other resources, and timelines to guide faculty activities.
Data are provided to programs on an annual basis to support an interpretation and reporting process described in more detail below in 2c.1.
The data collection process has taken place for data reaching back to fall 2007, but the nature of the data has changed over time, with fall 2008 marking a substantial step forward. In fall 2007, the college refocused its assessment efforts with the hiring of a new assessment coordinator and creation of the Assessment Committee. Because details of the evolving assessment system were as then emerging, the college made the decision to ask programs to continue past practices, which involved collecting single grade scores (A-F) on existing signature assignments, for fall 2007 and spring 2008, even as they worked together to revise and refine their assessment plans for the future. (In some cases, course instructors reported course grades rather than signature assignment grades.) These scores were reported to the Assessment Office using the college's 0-4 scoring scale, with a 4 being the highest score. These 2007-08 data were then analyzed by the Assessment Office and reported back to programs in simple graphs that reflected the percentage of students in the program who earned a particular score related to the learning outcome.
As programs have refined their assessment efforts, they have implemented the use of rubrics, as required by the Assessment Committee starting in fall 2008. As a result, data collected from fall 2008 forward can generally be reported in a more detailed manner, provided programs submit the rubric sub-scores for a given signature assignment. An example of the kind of chart that is now being prepared is provided at the following link, and program data reports reflect these charts where the data have been reported. While there are a small number of cases in 2008-09 where some courses have provided an overall score for the signature assignment rather than the more nuanced data, the majority of data reported now includes rubric-level data on candidate performance.
The Education Specialist, Multiple Subject, and Single Subject initial credential programs do have intern program components that can be considered alternate routes to a teaching credential. These candidates are held accountable for the same learning outcomes and signature assignments as their peers. As noted under Standard 1, the numbers of such candidates has steadily declined in recent years and they reflect a very small portion of overall enrollment.
To date, the data collection process has not allowed us to disaggregate data at this level of detail. However, the advent of the COSS database now enables us to examine the data at this more fine-grained level and future reports for programs will reflect this disaggregation.
Programs follow informal procedures for handling issues that do not rise to the level of "formal complaint." The unit also maintains a suggestion or "complaint" box in the lobby of Education Building 1. Comments are given to the dean who then directs them to the appropriate faculty or staff member. The university has formal appeal processes for students to follow when necessary. Resolution is first attempted at the lowest level, i.e., between faculty and student, before moving through levels that include program coordinator, department chair, associate dean, department/program petition committees, college appeals committee, dean, etc. Formal candidate complaints must be written and signed and must comply with university grievance procedures as described in various policy statements. Programs keep a record of formal complaints and their procedures for handling complaints. Faculty keep records regarding an incident in a confidential file and confer with the Associate Dean as appropriate. For example, if a grade appeal is not informally resolved between the student and faculty, the next step is a formal grade appeal following existing university policy. In the past three years there have been 2 grade appeals (College of Education Grade Appeals, 2006-2009).
N/A
