Academic Program Review
Entire Document (in pdf format )
Context for Academic Program Review
Purpose of Academic Program Review
Academic Program Review Process
Portfolio Components and Guidelines
Coordination with Other Accreditation and External Review Processes
Timeline for Academic Program Review
Academic Program Review Resources
Office of Institutional Research
Questions Regarding Academic Program Review please contact Miya Simpson
Context for Academic Program Review
Increasing demands for greater accountability within higher education have prompted institutions to more closely examine their academic programs and practices to assess their quality, efficiency and impact on student learning outcomes. Efforts at the federal level along with standards set by the Southern Association of Colleges and Schools Commission on Colleges (SACS-COC), State Council for Higher Education (SCHEV) and various other accreditation and coordinating agencies challenge institutions to provide evidence of a continuous, systematic evaluation process for reviewing academic programs, and an assurance that results are being used toward program improvement.
Academic program review (APR) is an on-going, comprehensive process that enables academic units to engage in a thorough evaluation of the status, strengths and weaknesses of their degree programs; determine the extent to which departmental and institutional goals and objectives are aligned; and, make credible claims regarding educational outcomes. Through careful evaluation and feedback from faculty and other stakeholders, as well as in-depth analysis of quantitative and qualitative indicators of quality, institutions are able to gather essential information that can be used to inform the decision-making process and make modifications and improvements as necessary.
Virginia Tech is committed to educational excellence and the enhancement of its academic programs. As such, the university has developed and will implement an ongoing academic program review process whereby all formally authorized undergraduate and graduate programs will undergo a comprehensive review at least once every 7 years.
Purpose of Academic Program Review
The primary purpose of APR is to promote and maintain high quality undergraduate and graduate programs that are effective, efficient and consistent with the mission and resources of the department and university. APR requires a thorough, reflective examination of the departments’ degree programs and educational practices to assess their status and progress and identify future directions, needs and priorities. Consequently, the review process is inextricably linked to strategic planning, as well as resource allocation and other decision-making at the program, department, college and university levels. Implementation of a systematic process ensures a mechanism for continuous evaluation, provides evidence that a functional process is in place and demonstrates how results will be used to improve programs.
It is understood that many programs regularly undergo accreditation or other external reviews as required by their accrediting and professional organizations; and the university review is not intended to burden departments with an additional review process. However, while issues addressed by external assessment measures are often relevant for program review, the purposes, processes and outcomes are not always identical and additional measures may be necessary. To the extent possible, attempts will be made to coordinate the APR so that it occurs at a time most convenient to the accreditation cycle, as requested by the department. In addition, to minimize the duplication of effort and maximize the value of all review process(es), documentation prepared within the last three (3) years as part of the department/programs accreditation and/or external review processes may be submitted or included in the materials submitted for APR. These reports will be reviewed for completeness and alignment with the university’s APR guidelines. Requests for additional information will be made if necessary.
Academic Program Review Process
APR is intended to go beyond simple data gathering and analysis to a point of expecting departments and colleges to make informed judgments about program quality and effectiveness and demonstrate how the findings will be used to improve undergraduate and graduate programs. The process is designed to provide a uniform method for gathering essential input, process and output data on all academic programs, and to do so in a manner that is both efficient and effective. The final result of each review will be a comprehensive portfolio that provides descriptive and evaluative information about the program, faculty and students; identifies strengths and weaknesses, significant accomplishments and challenges; and suggests areas and plans for improvement.
The Dean of each College will assume responsibility for data gathering and related outcomes of the process for programs under review within the College. At the University level, responsibility lies with the Vice President and Dean for Undergraduate Education (for undergraduate programs) and the Vice President and Dean for Graduate Education (for graduate programs).
The following is a brief overview of the APR process. A more detailed description of each stage is provided in the section titled, Guidelines for Academic Program Review.
- All undergraduate and graduate degree programs will be reviewed on a 7-year cycle. (Review Schedule and Notification)
- Once notified of a forthcoming APR, the department must prepare and submit a portfolio consisting of a detailed program self-study and an action plan for program modification and/or improvement(s). (Portfolio Components and Guidelines)
- The Director of Academic Program Review will review submitted materials to assess the completeness and quality of the APR portfolio and/or external report submitted. An Internal Review Team (IRT) consisting of 2-3 university faculty members may be selected by the Vice President and Dean for Undergraduate Education or Vice President and Dean for Graduate Education to evaluate the academic program review portfolio and prepare a report based on their findings.
- The Vice President, College Dean and department head/program chair will meet to discuss the IRT report and draft preliminary recommendations and program modification and/or improvement plans. The Vice President will prepare and submit the final report to the Provost. (Draft of Recommendations, Improvement Plans and Final Report)
- The Director for Academic Program Review will maintain a list and summary of all completed academic program reviews and reports. (APR Data Management and Dissemination)
Coordination with Other Accreditation and External Review Processes
It is understood that many programs regularly undergo accreditation or other external reviews as required by their accrediting and professional organizations; and the university review is not intended to burden departments with an additional review process. Therefore, in an attempt to minimize the duplication of effort and maximize the value of all review process(es), documentation prepared within the last three (3) years as part of the department/programs accreditation and/or external review processes may be submitted as or included in the materials submitted for APR. These reports will be reviewed for completeness and alignment with the university's APR guidelines. Requests for additional information will be made if necessary.
Timeline for Academic Program Review
- Academic program reviews will be scheduled on a 7-year rotation unless otherwise notified. The full schedule will be updated annually and posted to the APR website.
- Deans/departments will be notified of a pending review at least 2 months in advance.
- The review process will typically follow a 12-month* timeline from preparation of the portfolio through the completion and submission of the final report to the Provost as indicated below (*The estimated 12 month period may be longer in some cases due to unforeseen circumstances or if an external review is conducted):
- Portfolio preparation (8 months)
- IRT review and report (2 months)
- Draft recommendations and final report (2 months
Results of the academic program review will provide evidence of the quality and strength of Virginia Tech’s undergraduate and graduate degree programs and allow for decisions to be made regarding program development, enhancement and continuation. Findings will also contribute to the following efforts:
Planning: To ensure program quality and provide a foundation for sound planning and budgeting decisions to be made.
Quality Assessment and Assurance: As a mechanism for identifying and understanding the strengths and weaknesses of an academic program and establishing agreement on programmatic goals, academic priorities and consistent quality measures.
Academic Profile and Data Synthesis: To maintain an accurate and consistent academic profile for each degree program and allow for the identification and analysis of relevant trends in undergraduate and graduate programs.
External Evaluation: To meet regional and state requirements for a continuous, systematic evaluation process to assess the strength of our academic programs and use results to foster program improvement.
Accountability: To provide data that is clear, relevant and easily accessible to students, faculty, parents, alumni and other stakeholders.
Guidelines for Academic Program Review
- Review Cycle and Notification
All undergraduate and graduate degree programs will be reviewed on a 7-year cycle. Reviews may occur more frequently at the request of the department/college or if deemed necessary by the Provost. The Director for Academic Program Review will work with College Deans and the Vice-Presidents for Undergraduate and Graduate Education to develop and coordinate the master rotation schedule for all reviews. (A schedule of current and future reviews will be posted on the APR website but is subject to changes.) Deans/departments will be notified at least 2 months in advance of a scheduled review. Undergraduate and graduate programs in the same program area will be reviewed concurrently, unless a specific request is made by the Dean to have the programs reviewed separately.
- Portfolio Components and Guidelines
Once notified of a forthcoming APR, the department will have 8 months to prepare a portfolio consisting of a detailed program self-study and an action plan for suggested program improvement(s). The portfolio is the department’s main opportunity to provide detailed information, both descriptive and evaluative, about the academic program and serves as the basis for assessing and identifying the status, accomplishments, needs and future directions of the undergraduate and/or graduate program. The self study elements outlined below are provided as a guideline for APR and indicate the type(s) of information deemed essential for conducting the most comprehensive review. However, given the differences among programs there is some flexibility for departments to determine (in accordance with the Dean) the relevant criteria, critical sources of evidence and appropriate analysis and evaluation procedures that best depict the academic program.
- Self-Study Report
This report is a detailed description and analysis of each undergraduate and graduate program under review, utilizing and drawing conclusions based on qualitative and quantitative data across the preceding 5-year period. Resources that may be helpful in preparing and generating reports for the APR portfolio can be found in the section titled, Academic Program Review Resources.
The self-study should be thorough yet concise. Any supplemental documentation or data summaries should be included as an Appendix (see the section at the end of this document titled Examples of Appendix Data for a list of possible supplemental materials).
The self-study report should address (but is not limited to) the following elements for each undergraduate and graduate program:a. Department: mission, organization, size and purpose.
b. Degree program(s):
1. Name, history and mission of program; goals; number and type of degrees offered, tracks or concentrations available under the program, program location (i.e. Blacksburg, extended campus), learning objectives and outcomes; and name of program chair.
2. Centrality of the program to the department and university goals for undergraduate and/or graduate education;
3. National, regional, state and local need for and impact of program, as well as the name and type of external accreditation agency(ies) and requirements (if applicable)
c. Program Quality:
1. Assessment of the quality of the program in general and as compared to similar programs regionally and nationally (may include ranking).
2. Evidence of the effectiveness and efficiency of the program to: establish and achieve educational goals and objectives; identify the programs successes, strengths and weaknesses; assess what students know, have experienced and are able to do upon completion; evaluate instructional strategies to determine if they meet program goals and student needs and if changes are needed to enhance program quality and effectiveness.
d. Student profile: recruitment plan; admissions and transfer requirements; demographics for applicants, admitted, transfer and enrolled students by class/degree level, race/ethnicity and gender; enrollment trends; number and types of scholarships, assistantships and fellowships awarded; number and type of national scholarships recipients; students’ assessment of climate and learning environment; quality and type of advising and mentoring opportunities.
e. Faculty profile: name, number, rank, gender and race/ethnicity of all program faculty; table or abbreviated CV for all program faculty indicating education/training, employment history, courses taught, list of supervised theses and dissertations and/or committee memberships; research and scholarly productivity, distinguished honors/ awards; research grants/support and number of licenses, start-ups and patents; and professional memberships (A CV template is provided at the end of this document as an example).
f. Facilities: Description and evaluation of office, classroom, lab/research space, equipment, technology and library resources available for program needs/use.
g. Curriculum and Instruction: degree requirements; courses offered and schedule of offerings; number and type of freshman year courses/structured experiences and capstone courses (undergraduates); teaching loads and faculty/student ratio(s) for teaching and advising (undergraduate and theses/dissertations); number of undergraduate and graduate teaching assistantships/opportunities; number and type of academic support services; special/unique initiatives; non-traditional instruction methods; use of technology.
h. Human Resources: number and titles of the programs professional and student staff; faculty recruitment plan.
i. Student progress and productivity:
- Undergraduates – annual progress review; number of students who have been suspended (for academic reasons) or placed on academic probation; number of students who have participated in undergraduate research, study abroad and/or service/experiential learning experiences; persistence and graduation rates; number of bachelors degree’s awarded; average time to degree completion.
- Graduate Students – number of students who have been suspended (for academic reasons) or placed on academic probation; passage rates for qualifying exams, preliminary exams and dissertation defenses, number and titles of completed theses and dissertations; number of masters and doctorates awarded; student authored and co-authored publications; professional presentations; persistence and graduation rates; average time to degree completion
- Undergraduates – skills and competencies developed; number of graduates entering graduate/professional school and the workforce; alumni and employer evaluations.
- Graduate Students – skills and competencies developed; number of graduates entering post-doc programs, teaching, research and industry; alumni and employer evaluations.
- Self-Study Report
- Action Plan
Based on an evaluation and interpretation of the information provided in the self-study, determine the need for any adjustments or improvements to the program. Develop an action plan and timeline detailing how the APR findings will be used toward program modification and/or improvement. Plans should focus on improvements that can be made utilizing existing resources. Improvements and expansions requiring additional resources should be clearly specified and offer sound justification.
- Internal Review Team
An Internal Review Team (IRT) consisting of 2-3 university faculty members may be selected by the Vice President and Dean for Undergraduate Education or the Vice President and Dean for Graduate Education to evaluate the academic program review portfolio and prepare a report based on their findings.
Following a review of the APR portfolio documents, the Vice President may assemble an IRT to conduct a further review of the program. Recommendations for IRT members may also be submitted by the Dean; however, no member of the IRT shall be from the program under review. IRT’s are charged to thoroughly examine and evaluate the APR portfolio to determine if it is complete and how well the APR criteria have been addressed. (A member of the IRT team or the Director for Academic Program Review will contact the Dean regarding incomplete portfolio’s or to request additional information if needed.) The IRT will prepare a report that includes a general assessment of the program (students, faculty, curriculum/course content, etc.), significant strengths and trends, areas for improvement and recommendations. Reports will be submitted to the appropriate Vice President within 2 months following the portfolio review.
- Draft of Recommendations, Improvement Plan and Final Report
The Vice President, College Dean and department head/program chair will meet to discuss the IRT report and draft preliminary recommendations and improvement plans. At this time a decision may also be made about the desirability of conducting an external review. If so, the Vice President and College Dean will consult on the particulars of the external review to include purpose and structure of the external review, committee composition and scheduling. Resources to support external review will be provided by the Provost.
The Vice President will prepare and submit the final APR report to the Provost, who will have the final authority on any future action(s) taken related to APR findings. A copy of the report will be provided to the College Dean and program chair.
- APR Data Management and Dissemination
The Director for Academic Program Review will maintain a list and summary of all completed APR’s. Reports and other information related to the APR process will also be posted to the APR website (under development).
Academic Program Review Procedures presented to Commission on Graduate Studies and Policies (Spring, 2005) and Commission on Undergraduate Studies and Policies (Spring, 2008)
Academic Program Review Resources
The following resources may be helpful in preparing materials and generating reports for the APR portfolio.
WEAVEonline (Office of Academic Assessment, www.aap.vt.edu)
The Office of Academic Assessment offers an on-line assessment management system called WEAVEonline (http://www.weaveonline.net/subscriber/vt/), which can be used to provide descriptive information about the program/department/college, identify educational objectives and learning outcomes, document assessment efforts and track actions to improve programs. A detailed annual assessment report can be generated based on the data entered and can be used to analyze, interpret and make decisions regarding effectiveness.
College and Departmental Profiles (Office of Institutional Research, www.irpa.vt.edu)
The Office of Institutional Research has compiled several databases that include various university, College and departmental statistics and trend data. The website provides data (tables, graphs, maps) related to students, credit hours, class sections, grades and faculty and staff, as well as information about Virginia Tech’s peer institutions and publicly released common data sets. A 5-year profile for each College and department is also available and allows departments to generate reports based on the aforementioned data, as well as admissions and enrollment, majors, facilities and sponsored research information.
Examples of Appendix Data
- Organizational Chart
- Course Catalog and Syllabi
- Policy and Procedures Handbook
- Alphabetical list of all faculty including rank, length of employment, tenure status, gender, ethnicity, major areas of expertise
- Abbreviated faculty CV’s
- Information related to external support and sponsored research
- Statistical summaries related to student application, admittance, enrollment, retention, persistence, time-to-degree and graduation rates(5 years)
- Survey instruments and result’s (students, faculty, alumni, employer’s)
- National rankings information and other metrics of excellence
- Layout/floor plan of physical facilities utilized by program
- Copies of reports and profiles generated through WEAVEonline and Institutional Research, as applicable (should include an analysis and interpretation of data presented)
- Graphs and tables
- External accreditation reports (up to 3 year old)
- Assessment reports (internal, Office of Academic Assessment, WEAVE Online)
SAMPLE ELEMENTS FOR ABBREVIATED FACULTY CV
Faculty Name and Title
Education/Training
(List all degrees and certifications, institutions, completion date/year, and field of study)
Employment History
(Starting with current position, list all positions held and period of employment over last 5 years.)
Courses Taught
(List the title and level of all courses taught in current or other academic program over the last 5 years)
Supervised Theses and Dissertation and/or Committee Memberships
(List all theses and dissertations committees you have supervised or for which you have served as a member over the last 5 years. Include each students name, year of graduation, degree earned and title of thesis/dissertation.)
Research and Scholarly Productivity
(List all peer reviewed publications over last 5 years)
Distinguished Honors/Awards
(List all honors, awards and special recognitions received over last 5 years)
Research Grants/Support
(List all ongoing and completed projects for which you have received federal and non-federal financial support over the last 5 years)
Licenses and Patents
(List all licenses and patents received over the last 5 years)