|Year : 2016 | Volume
| Issue : 1 | Page : 32-38
A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement
Prashant Paschal, KK Kalra, Bhupendra Kumar Rana
National Accreditation Board for Hospitals and Healthcare Providers, ITPI Building, 5th Floor, 4-A, I P Estate, Ring Road, New Delhi, India
|Date of Web Publication||4-Aug-2016|
Bhupendra Kumar Rana
National Accreditation Board for Hospitals and Healthcare Providers ITPI Building, 5th Floor, 4-A, I P Estate, Ring Road, New Delhi - 110 002
Source of Support: None, Conflict of Interest: None
Introduction: Feedback is one of the crucial elements of learning and achievement, but its outcome can be either positive or negative. Drawing on findings from different trainings on Program On Implementation (POI) organized by NABH, this study reveals the extent to which effectiveness can be accurately measured. The study argues that the participants are in the best position to judge the effectiveness of feedback, but may not always recognize the benefit it provides.
Aims and Objectives: The aim and objective of this study are: (1) To find out the effectiveness of the NABH training programs and efficiency of faculties related to its effect on learning, improvement, and achievement. (2) To find out ways to improve the NABH form related to delivery, content, and its appropriateness.
Patients and Methods: All the feedback forms were collected from the participants after the successful completion of the program and then the interpretations and comments if any provided by them were viewed and analyzed.
Statistical Analysis: Responses were analyzed using average percentage method which reflected the satisfaction level of participants as well as the appropriateness of NABH training (POI) programs.
Results: The feedback forms obtained by the participants are analyzed in two ways. (1) Detailed analysis of feedback forms of all the programs together. (2) Program wise analysis.
Conclusions: The overall conclusion of the feedback analysis was that NABH needs to take evaluation feedback much more seriously if they are to achieve the dual benefits of greater accountability and more effective learning.
Keywords: Evaluation, feedback, feedback effectiveness, self-assessment
|How to cite this article:|
Paschal P, Kalra K K, Rana BK. A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement. J Nat Accred Board Hosp Healthcare Providers 2016;3:32-8
|How to cite this URL:|
Paschal P, Kalra K K, Rana BK. A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement. J Nat Accred Board Hosp Healthcare Providers [serial online] 2016 [cited 2018 Dec 12];3:32-8. Available from: http://www.nabh.ind.in/text.asp?2016/3/1/32/187764
| Introduction|| |
Much staff time and effort go into producing assessment feedback, but very little effort is made to examine its effectiveness. This study draws on findings from the analysis of feedback forms addressing participant's engagement with training programs conducted by NABH.
Feedback is an information about how we are doing in our efforts to reach a goal. It is a basic teaching method essential for learning and improving performance.  Hence, training and its regular evaluation would make NABH to stand in the limelight in achieving the objectives and imparting the knowledge of quality services being required in health-care industry.
| Methodology|| |
Feedback can only be effective when the participant understands the feedback and he/she is willing and able to act on it. In program-on-implementation (POI) organized by NABH, the likelihood of feedback providing unambiguous, categorical feedback about the effectiveness of the program or how to improve is very clear with most feedback requiring interpretation.
The feedback forms are distributed to the participants as a part of course material. They provide their feedback in the format (feedback forms) prepared by the NABH. All the feedback forms were collected from the participants after the successful completion of the program, and then the interpretations provided by them are viewed and analyzed in regards to the following points:
- The three criteria for the faculties, namely, knowledge about the course content, teaching skills, and engagement with the participants
- The NABH feedback form was based on the understanding and usefulness on different parameters such as appropriateness, venue, food quality, presentation content, timely information, and administrative arrangement which decide the effectiveness and usefulness of these training programs
- Logistics which includes venue arrangement, food quality, and registration counter.
The participants are required to fill the feedback form based on a detailed score matrix which includes 5 for excellent, 4 for very good, 3 for fairly well, 2 for average, and 1 for poor. At last, all the feedback forms are analyzed, and conclusion is drawn out with respect to the number of feedbacks received.
The participants gave their feedback based on a grade matrix system comprising 5-point Likert scale. All the responses were analyzed using average percentage method which reflected the satisfaction level of participants as well as the appropriateness of NABH training (POI) programs. In addition, the analysis is done against each parameter of the NABH feedback form, for which average percentage is obtained for the entire program and each program separately.
Levels of feedback
- Level 1: Participants observe the faculties (without judgment) - Description of the observed behavior - easiest for the learner to hear and accept
- Level 2: Participants' personal reaction (not judgment)
- Level 3: Participants' prediction - Insights based on experience about the appropriateness, correctness, or effectiveness of the training program.
Analysis of feedback forms
This study reveals the analysis of NABH training (POI) feedback forms of different programs. Feedback forms of 16 different programs comprising 559 participants (n = 559) were analyzed. The categorical distribution is as follows: (A) Dental POI: 5; (B) nursing POI: 6; (C) blood bank: 1; (D) and hospital: 4.
| Results|| |
The feedback forms obtained by the participants are analyzed in two ways. (1) Detailed analysis of feedback forms of all the programs together. (2) Program wise analysis.
The analysis clearly shows that 90% of the participants are satisfied and fully agree that NABH training programs are appropriate and useful. Around 81% of the participants agreed that they receive timely information from the NABH regarding the training schedule, venue, timing, etc., some of the participants (around 7.6%) are not satisfied with the criteria of timely information.
In regards to the venue decided by the NABH for different programs, it was observed that 94.6% of the participants are satisfied and feel that venue is appropriate in terms of their approachable location, facilities, etc., More than 93% of the participants agreed and gave their responses in favor of administrative arrangements done by the NABH. The satisfaction level suddenly drops down in regards to registration counter where only 67% of the participants are satisfied, and the same reaction is being observed in case of presentation content where only 48.7% of the participants are fully satisfied.
The majority of the participants felt that the NABH provides quality food at these venues as 89% of the participants gave their opinion in favor of the criteria. The concern area is nonresponse obtained for different criteria. As per the percentage analysis shown in [Table 1] and [Figure 1], each criterion has some percentage of participants who did not gave any response which is observed to be maximum in presentation content and minimum in venue.
Program wise analysis
A total of 5 feedback forms comprising 117 participants (n = 117) were analyzed and their results are calculated based on average percentage method. The analysis was done for NABH training programs as well as faculties who present the course content. The results are shown in [Table 2], [Table 3] and [Figure 2], [Figure 3].
A total of 4 feedback forms comprising 153 participants (n = 153) were analyzed and their results are calculated based on average percentage method. The analysis was done for the NABH training programs as well as faculties who present the course content. The results are shown in [Table 4], [Table 5] and [Figure 4], [Figure 5].
Blood bank program
One feedback form comprising 34 participants (n = 34) were analyzed and their results are calculated based on average percentage method. The analysis was done for the NABH training programs as well as faculties who present the course content. The results are shown in [Table 6], [Table 7] and [Figure 6], [Figure 7].
A total of 6 feedback forms comprising 255 participants (n = 255) were analyzed and their results are calculated based on average percentage method. The analysis was done for the NABH training programs as well as faculties who present the course content. The results are shown in [Table 8], [Table 9] and [Figure 8], [Figure 9].
| Discussion|| |
Understanding about effectiveness may require the use of input, process, and output measures. However, meaningful evaluation requires a common conformity on the purpose of feedback. This is aided by the use of "efficient" tick-box feedback forms which focused on benchmarking and providing analytical justification of the mark awarded. However, those unsure about the purpose of feedback had not given any response and left the column blank. Limiting feedback to justification of the grade system ranging from excellent to poor reinforced the belief among participants that feedback has no feed-forward opportunity. The findings suggest scientific investigation, but this seems to arise from the difficulty of determining effectiveness rather than a reluctance to measure what is assumed to be positive. 
The key components of feedback 
The key components of feedback are tangible, and transparent, actionable, user-friendly (specific and personalized), timely, ongoing, and consistent.
Purpose, measurement, and engagement
The analysis of feedback form explain its purpose of among and between faculties and participants. Hence, while faculties acknowledged that feedback provides an "indicator of performance," it is obvious that it plays a role in learning and development and helps to move participants in the right direction.
Comparison of feedback forms
We have compared the old & modified feedback form. The detailed content are given in [Table 10] and [Table 11].
| Conclusions|| |
The overall conclusion of the feedback analysis was that NABH needs to take evaluation feedback much more seriously if they are to achieve the dual benefits of greater accountability and more effective learning.
From the above feedback, the conclusions that were drawn include: 
- Training environment must favorable
- Course material should include more case studies and practical trainings
- Session should not be lengthy and exhaustive and must include video demonstration and mock drills with free time
- The learning phase is evaluated by conducting test before and after training
- NABH should add facility planning criteria chapter within content, aims, and objectives for further information and clarity.
There was a recognition that more effort is needed in identifying and prioritizing target audiences in specific training program, and that attention needs to be given to this early on in the process, not after the evaluation is completed. It was agreed that "learning by actually doing" is often the most effective feedback route.
We recommend that all faculties videotape their own classes at least once a month. Peer review is another strategy for managing the load to ensure lot of timely feedback; it is essential, however, to train staffs to do small group peer review to high standards, without immature criticisms or unhelpful praise.
Some recommendations for effective feedback are as follows:-
- Choosing the option which offers the best solution
- Identifying possible courses of action
- Weighing up the pros and cons of each alternative
- Defining the nature of the issue to be dealt with implementing the policy.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Hewson MG, Little ML. Giving feedback in medical education: Verification of recommended techniques. J Gen Intern Med 1998;13:111-6.
Hattie J, Timperley H. The Power of Feedback. Vol. 77. America: American Educational Research Association; 2007. p. 81-112.
Turnbull J, Gray J, MacFadyen J. Improving in-training evaluation programs. J Gen Intern Med 1998;13:317-23.
Kunche A, Puli RK, Guniganti S, Puli D. Analysis and evaluation of training effectiveness. Hum Resour Manage Res 2011;1:1-7.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6], [Figure 7], [Figure 8], [Figure 9]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6], [Table 7], [Table 8], [Table 9], [Table 10], [Table 11]