Note: This page is best viewed on a larger screen; it may be difficult to read on a mobile device or tablet.
How to Complete and Use the
Program Outcomes Assessment Matrix
Use the lefthand tabs or the next/previous links at the bottom of each page to navigate through a self-paced tutorial by the VTSU Center for Teaching & Learning Innovation focused on developing the program outcomes assessment matrix for the purposes of tracking and planning actions that result from annual program assessment activities.
The Continuous Improvement Cycle
We engage in program assessment for continuous improvement. The Program Outcomes Assessment Matrix helps you track multiple steps of the cycle, and especially helps with the planning process for changes that will lead to improved student learning outcomes.
The Program Assessment Process
A first draft of the Program Outcomes Assessment Matrix is submitted in January of Year 3 of the program assessment process. It will also need to be updated when completing the PReCIP Report in Year 5.
Sample Matrix
For each Program Learning Outcome, use the Matrix to track what changes you’ve made during the current cycle as a result of the evidence you’ve collected, and to also make a plan for how you’ll assess the effectiveness of these changes in the next PReCIP cycle.
While the Matrix is drafted in Year 3, the final version will be completed in Year 5 with the PReCIP report. As a result, the reference in the first column header to the “past cycle” can be a little confusing when you complete this during Year 3. Think of column 1 referring to the current cycle, as you fill this in during Year 3. When completing the Matrix in Year 5, “past” refers to the cycle that is ending.
Program Learning Outcomes |
Changes Made During the Past* Cycle |
Evidence to Be Collected in Future Cycle(s) |
Responsible Parties |
Schedule |
Plan for Taking Action Through Next Cycle |
|---|---|---|---|---|---|
|
Outcome 1 [Title] |
Adoption of common assignment rubric by faculty after evidence showed student learning gaps in Outcome 1. |
Scores from shared rubric on assignments in [designated program courses] |
Faculty teaching [designated courses] to use rubric; review of summary data by entire department |
Collect rubric score data every semester within [designated courses]; Year 1 focus for analysis of assessment results |
Program will use its analysis of assessment results to inform faculty development activities. |
|
Outcome 2 [Title] |
Addition of narrative reflection assignment field placement course after evidence showed students not recognizing learning gains in field placement. |
Shared rubric on narrative reflection assignment; Field placement supervisor feedback |
Faculty supervisors of field placement experience |
Collect data every semester within [designated courses]; Year 2 focus for analysis of assessment results |
Program will use its analysis of assessment results to inform outreach with field placement supervisors. |
|
Outcome 3 [Title] |
Program added public presentation capstone assignment to upper-level required course after student feedback identified concerns about having sufficient preparation to succeed in workplace environments. |
Capstone assignment; evaluation rubric |
Faculty and/or other evaluators of capstone assignment |
Collect rubric data for every capstone assignment; Year 3 focus for analysis of assessment results |
Program will use its analysis to inform curriculum revisions. |
|
Outcome 4 [Title] |
Student feedback survey results |
Dept chair to administer survey; entire department for review of evidence |
Annual completion of surveys; Year 4 focus for analysis of assessment results |
Program will use its analysis to inform program outcome revisions. |
*The “Past” in the Column 1 header can be confusing. During Year 3, replace “Past” with “Current.” During Year 5, “Past” means the cycle that is concluding.
Column 1: Changes Made During the Program Review Cycle
Column 1 is essentially a summary of the information you report in the Yearly Learning Outcomes Assessment Report, documenting one program learning outcome per row in the matrix.
In order to complete this column, the faculty must first collect and analyze evidence, then determine appropriate actions, and finally implement identified changes to positively affect student learning.
Step 1.
Collect and analyze evidence.
Step 2.
Determine appropriate actions.
Step 3.
Implement changes to affect learning.
As mentioned previously, the matrix is initially drafted in Year 3 of the program review cycle and finalized in Year 5:
- During Year 3: In Column 1, summarize changes made as a result of using the data/evidence collected and and analyzed during the current cycle. By January of year 3 (when the matrix is due), you will have efforts from the first two years of the cycle on which to report. You will likely have implemented changes from Year 1 and be in the process of implementing changes from Year 2.
- During Year 5: In Column 1, summarize changes made as a result of using the data/evidence of the entire 5-year cycle (building on the draft you started in Year 3). By Year 5, you should be reporting on activity for all program outcomes for a full 5-year cycle*. Additionally, these changes are discussed in narrative form in Section III of the PReCIP report.
This matrix will help ensure that the desired changes you’ve identified not only are put into effect, but then also evaluated for effectiveness.
*Programs are expected to assess each Program Learning Outcome at least one time during each 5-year cycle. For programs with 5 or fewer outcomes, this means assessing one outcome per year; for programs with more than 5 outcomes, multiple outcomes will need to be assessed for at least some of the years of the cycle. Programs on abbreviated cycles through 2028 should assess one outcome per year for the duration of that cycle, and then on the next full 5-year cycle, ensure that all program learning outcomes are assessed.
Column 2: Evidence to Be Collected in Future Cycle(s)
Column 2 is a summary of direct and indirect evidence to be collected in the upcoming program review cycle(s) for each program learning outcome, as part of the a continuous improvement process.
For each change that has been implemented, consider two questions:
Where would you expect to see the impact of the change on student learning?
How will you collect evidence of the change?
In other words, what evidence (direct and indirect) will you utilize to evaluate the actual affect on student learning of this cycle’s interventions?
Include details such as the courses or other programmatic activities (internships, placement exams, etc.) where you would be able to assess the change’s effectiveness. And identify direct and indirect measures that you will likely use to collect evidence of the related student learning. Again, you likely won’t begin collecting this evidence until the next PReCIP cycle, but these plans will give you a solid starting point.
Column 3: Responsible Parties
Column 3 requires program faculty to identify the people responsible for ensuring the planned work is executed in a timely fashion in upcoming years and/or the next program review cycle.
When completing Column 3, answer the following questions:
- Who will be in charge of ensuring the assessment occurs?
- Who needs to know/be involved in collecting evidence?
- Who needs to be involved in analyzing the evidence?
This planning will ensure continuity as one assessment cycle rolls into the next.
Column 4: Assessment Schedule
Column 4 also focuses on planning for the next cycle, this time mapping out a timeline.
Consider: When, in the next 5-year cycle, will it be effective and a priority to assess the impact of the change(s) implemented during this cycle?
Document a schedule for completing the assessment activities identified in Column 2.
Column 5: Plan for Taking Action Through Next Cycle
Column 5 asks you to project into the future and imagine how you might act on the results of the evidence you plan to connect.
Questions to consider when filling in this column are:
- Once you collect and analyze the evidence, how might you imagine using it?
- Who will need to know the results?
Actions you might anticipate taking including shifts to the benchmarks or targets; modifications to the measure methodology (e.g., adjust the rubric); revisions to the assessment, curriculum, or program sequence; adjustments specific to one or more modalities; professional development for faculty; etc.
Evolving the Program Outcomes Assessment Matrix
Drafting the Program Outcomes Assessment Matrix in Year 3 of the program assessment cycle makes the task of filling out the final matrix in Year 5 less overwhelming and daunting. It also helps you track on details from early in the current cycle that may be hard to remember or capture in Year 5.
Year 3:
Draft Matrix
Year 5:
Final Matrix in PReCIP Report
Keep in mind that you will have the opportunity to refine and finalize the matrix when you submit it with your PReCIP Report.
Submission Deadline
When is the Program Outcomes Assessment Matrix submitted?
The matrix is due by January 1st in Year 3 and by June 1st in Year 5 (as part of the PReCIP Report).
The Program Outcomes Assessment Matrix Template
Is the use of a template a requirement?
Yes, and a Word-based copy of the report template is available on the VTSU Program Assessment SharePoint site.
Once you’ve accessed the site, scroll to the Important Links section and select the Assessment Docs – Templates button. Then, download a copy to get started.
Examples of Completed Matrices
Are there examples of previously submitted matrices?
Yes, PDF-based exemplars from prior years are on the VTSU Program Assessment SharePoint site.
Once you’ve accessed the site, scroll to the Important Links section and select the Assessment Docs – Examples button. Then, review examples previously submitted by your faculty colleagues.
Submitting the Report
How is the Program Outcomes Assessment Matrix submitted?
Navigate to the VTSU Program Assessment SharePoint site and scroll to the Program Assessment Documents section. Then, select the Submit Required Documents button.
The resulting form is used to attach the completed PDF document, and the submission will automatically be forwarded to the applicable program coordinator or dean to review. The report submission is part of a structured workflow. Do not submit the file(s) by email or any other means.
Program Assessment Support
Is there support available?
Yes, there is support available for the program assessment process.
Consulting with a group of faculty colleagues is a great place to start and one recommendation is to schedule a formal meeting, series of meetings, and/or departmental retreat to address program assessment.
In addition to faculty colleagues, there are campus partners who are available to provide additional support and program assessment leads are encouraged to:
- Consult with the Center for Teaching & Learning Innovation
- Access institutional data on the Institutional Analytics SharePoint site
- Contact Institutional Research with additional data-related questions and requests
References
You may find it useful to browse these sources that were used in the development of this presentation.
Andrews, A. (2019). A program assessment guide: Best practice for designing effective assessment plans. University of Wisconsin Milwaukee. https://uwm.edu/academicaffairs/wp-content/uploads/sites/32/2019/04/Guide.pdf
Core Curriculum Committee. (n.d.). CSU Core Curriculum Handbook. Cleveland State University. https://pressbooks.ulib.csuohio.edu/corecurriculum/
Massa, L. J., & Kasimatis, M. (2017). Meaningful and manageable program assessment : A how-to guide for higher education faculty. Routledge.
Suskie, L. (2018). Assessing student learning: A common sense guide (3rd ed.). John Wiley & Sons.