Assessment of institutional effectiveness and student learning at all levels of the University is guided by clearly identified objectives and learning outcomes (LOs) aiming at implementing the mission of the unit.  This page explains how these are defined at different levels of the institution and how they interact.

The mission statement of a department, unit or program concisely describes its purpose and values and its relation to the institutional mission. The figure below shows how Objectives and LOs are related at different levels.

Figure 1.1.1 Hierarchical alignment structure of the university’s assessment process.

 

Institutional-level objectives (Pillars) and Institutional-level learning outcomes (Core Capabilities) are derived directly from the University Mission and Strategic Plan and form the basis for the assessment of the University’s Institutional Effectiveness.  At the unit and program level, objectives and learning outcomes are derived from the unit and program missions, drafted and regularly revised by the staff and faculty belonging to the unit or program, and in alignment with the Pillars and Core Capabilities of the University. This process continues, with equal attention to coherence and alignment, to lower levels. In the figure above, arrows pointing downward are meant to illustrate that objectives and learning outcomes at higher levels are used to define objectives and learning outcomes at lower levels. Arrows pointing upward show the path of execution of the assessment effort.

The link between higher and lower level objectives and learning outcomes is articulated in alignment matrixes (green boxes). Alignment matrixes are drafted for all units and departments, they are part of the assessment reporting templates and they are collected in the Institutional Alignment Register.

 

Institutional Objectives and Learning Outcomes

The definition of objectives and learning outcomes at the institutional level is part of the Strategic Planning process undertaken regularly by the President, the Leadership Team and the Board in an interactive exchange with Faculty and Staff. Under the current Strategic Plan (2020-2023), the University’s mission is advanced by five distinct Pillars: Community, Curriculum, Campus, Communication, and Continuous Improvement. Each pillar is further divided into a set of Core Initiatives. The success of the strategic plan can be identified with the completion of the Core Initiatives.  Advancement toward completion of the Core Initiatives is measured by a set of key performance indicators, timely achievement of milestones, and collection of deliverables which are linked with specific unit objectives. Institutional learning outcomes are set, and regularly reviewed, by the President and Provost in consultation with the faculty. The current institutional learning outcomes are referred to as the “Core Capabilities”. Every academic program offered at AUP demonstrate coherent, principally through an articulated alignment table, with the institutional Core Capabilities.

 

Unit Objectives and Program Learning Outcomes

Each administrative unit and academic program defines respectively its own objectives and learning outcomes. Note that the choice of using “objectives” for administrative units and “learning outcomes” for academic programs stemmed from the need to align with previous practices at the University while also maintaining the assessment process simple. Objectives and learning outcomes must be measurable and aligned with those defined at the institutional level (meaning that lower-level objectives and LOs should contribute to the achievement of higher-levels ones) and must be agreed upon by all direct stakeholders. As part of the assessment-planning-implementation cycle, objectives and LOs should be subject to continuous revision and improvement (here are some recommendations on the definition of objectives and learning outcomes)

 

Measuring Achievement of Objectives and Learning Outcomes

 

Institutional objectives

Under the current Strategic Plan, Institutional objectives are referred to as “Core Initiatives”. The advancement of the Core Initiatives of the Strategic Plan is measured and monitored through a collection of KPIs, deliverables, and milestones. These measurements are gathered annually either through the regular unit and program assessment reports or through complementary processes such as alumni surveys, LinkedIn studies, ad-hoc reporting.    

Deliverables, KPIs, and milestones for each of the Core initiatives of the Strategic Plan are established by the President and Leadership Team in consultation with the Dean of Institutional Effectiveness and all stakeholder staff and faculty. Deliverables, KPIs, and milestones for each of the Core initiatives of the Strategic Plan are collected annually and archived in the Strategic Plan DashBoard under the supervision of the President and Dean of Institutional Effectiveness.

 

Unit Objectives and Program Learning Outcomes

Assessment methodologies for each one of the administrative unit objectives and academic program LOs are specified in the “Objectives” and “Learning Outcomes” tables of the Assessment Report Templates. Each university Unit or program defines the most appropriate measures and target results for its own objectives and LOs. In assessment of student learning, we distinguish two types of evidence: direct and indirect. Direct evidence looks at student artifacts evaluated through articulated criteria (e.g., by a grading rubric) or at tests where specific items are associated with specific learning outcomes (the ensemble of these links are referred as the “blueprint’ of the test). Indirect evidence is provided by surveys and questionnaires and consist of opinion and perception of learning (e.g.: from student evaluations). Assessment of student learning should ideally be based on a mix of direct and indirect evidence though direct evidence alone will suffice.  In any case, indirect evidence alone is insufficient to demonstrate achievement of learning outcomes. Any instrument supporting measurement methodologies used by the units or programs - such as scoring rubrics, instructions for portfolio creations, qualifying or comprehensive examinations – should be described or included in the assessment plan.

Assessment of achievement of unit objectives should be objective, representative, and cost effective. As a general best practice rule, whenever the unit is responsible for producing specific KPIs, deliverables, or milestones to measure specific Core Initiatives of the Strategic Plan Dashboard, the unit should define objectives that are aligned directly with those Core Initiatives and use the same KPIs, deliverables, or milestones to measure progress toward their objective.