Evaluating

Program evaluation is the systematic collection, analysis, and reporting of information about a program to assist in decision-making. Process evaluation is used to track the strengths and weaknesses of the CCFAP and seek to identify what parts of the program are working and which are not.

Impact evaluation answers questions about the short-term effects or benefits of the CCFAP—as opposed to long-term outcomes, such as patient outcomes. It focuses on questions, such as the following:

• What effects did the CCFAP have?
• Can the effects be attributed to the program?
• Did the program participants' knowledge, attitudes, beliefs, or behaviors change as a result of the program?
• Did the CCFAP achieve it objectives?
• What happened as a result of the core project team's efforts?

The impact evaluation addresses the factors that are believed to precede, and that are linked to, long-term outcomes.

Outcome evaluations might focus on patient outcomes, patient/family satisfaction, change in care team and family relationships, or systems change. Outcome evaluation questions are generally related to the overall program goal:

• What is the current length of stay in the ICU vs length of stay prior to the CCFAP?
• Has there been a decrease in return visits for the same complaint?
• What change in family satisfaction occurred because of the CCFAP?
• What change has occurred in ICU staff perception of the relationship with families vs prior relationship with families (before the CCFAP)?

The steps for successful CCFAP evaluation can be summarized as follows:

• Make a clear plan that includes CCFAP evaluation steps.
• Use evaluation findings to make improvements in the CCFAP.
• Research evidence-based trends in critical care medicine that impact the execution of the CCFAP.

Included in the Replication Toolkit:

• Evaluation planning document that helps you determine evaluation measures and standards for success and clarifies who is responsible for collecting, analyzing, and reporting data.
• Strategies for ensuring the implementation of the evaluation plan.
• Plan for reporting evaluation findings.
• Strategies for using evaluation findings to make improvements in the CCFAP.
• Evidence-based trends in critical care medicine that impact the execution of the CCFAP.