Contents:
The size of the kappa reflects the level of agreement with the p-value assessing the likelihood of chance agreement. Only provider responses that were spontaneous were included. As the OSP evaluation progressed, additional information was given to the providers to allow them to assess the specific features of malaria of interest.
Provider questions that were un-related to malaria were not scored. Only three of the 20 observed real patients were diagnosed to have malaria and treated. For this reason, the analysis compares assessment up to the point of diagnosis. Separately, we evaluated the performance of all 20 providers at providing appropriate treatment for malaria based upon the OSP scenario presented. The providers, nearly universally, performed well at patient history taking and both using and reading the RDT kits of the OSPs.
Providers did worse at asking about or identifying signs of sever malaria, and at taking patient vital signs. Providers most frequently omitted history taking specific to antimalarial drugs. As shown in Figure 3 , there was little difference between average scores for true patients and OSP patients in all areas except antimalarial drug history taking. Only one measure, antimalarial drug history taking, differed by t-test between observed patients and the OSP. Twenty providers were scored on their performance diagnosing and treating malaria, first by direct observation with real patients; subsequently using the OSP methodology.
Figure 3 shows the providers scored for each of the five diagnosis modules.
"The Gist of Emergency Medicine was written for oral board candidates, practicing Medicine: The Management of Real Or Simulated Patient Encounters. The Gist of Emergency Medicine by Micheal O. Hebb was written for oral board candidates, The Management of Real or Simulated Patient Encounters-.
The weights given to each module were determined through consultation with experts in malaria treatment as described in the text. Sub analysis of response areas shows similar rates of correlation between true patient and OSP scores Table 1. By the kappa statistic and Z-test, there was statistically significant moderate agreement for the OSP tool overall Table 2.
By sub-components, there was significant perfect agreement in the critical area of RDT application and reading, significant moderate agreement in taking vital signs, borderline significant fair agreement in the general examination, and non-significant less agreement in the overall and in the drug taking histories. Among the 20 true patients only three had positive RDT tests and were treated for malaria. For this reason, we have not compared the treatment scores between real and OSP patients. The average scores of the providers with OSP patients on all clinical aspects of treatment were high, As was true for the history-taking sub-module, providers scored poorly on prevention counseling of OSP patients, averaging 3.
Our study suggests the high potential a new hybrid quality assessment tool when applied to the treatment of pediatric malaria in Myanmar. We found significant agreement in quality assessment scores among private providers when measuring their performance using direct observation of provider patient interaction and using Observed Simulated Patients.
The overall quality of care provided by doctors in the study was found to be moderately high, averaging 79 out of a total possible score of , indicating broadly appropriate practice in diagnosis and treatment for the simulated patient. Providers lost points on history taking, but all providers scored well on use and interpretation of diagnostic test kits, and prescription of appropriate anti-malarial medicine.
The kappa statistic can appear low when overall agreement is actually high because in the sample there is a high chance of getting correct responses, similar to how positive and negative predictive values of tests are affected by the prevalence of disease in the sample. This is a general limitation of the kappa statistic [25]. For the present study, we emphasize that the agreement is greater than predicted by chance in a context of high likelihood of a correct answer. A budget-driven limitation of this study is the small sample size, and in particular the small number of confirmed malaria cases in the observed patient sample.
Patients may have selected providers based on skills, and so it is possible that the 20 providers included in this study were more qualified than the 17 providers who did not report patients. We have also only compared the validity of OSP for one disease, in one setting.
Caution must therefore be taken in making any extrapolations about the ability of OSP to provide accurate assessments of provider practices in the management of other health issues, or in countries where the norms of provider-patient interaction may be quite different from those in Myanmar and have correspondingly different responses to this methodology. The method of provider identifying febrile patients presenting may have introduced bias in which patients were included in the study.
Providers in the study were scored on their ex-post treatment quality: in other words, they were not scored on treatment practices that were correct, but not related to malaria even where such examination might have allowed the providers to rule out non-malaria illnesses. This is a common gap in quality measurement, but remains unaddressed in this study. We have highlighted the risks of this in OSP as well as other quality assessment methods in Table 4. Providers receive benefits from membership in the Sun Quality Health network and may have provided better-than-normal care while under observation for this reason.
Although this would effect both the Directly Observed patient and the OSP, there may have been differential impact on the first and second assessment. We believe OSP to be cost-efficient compared to other methods based on the short time collecting data at each clinic and the continued use of a small number of trained researchers. Nevertheless, no comparison of the costs of conducting different quality assessment methods has been conducted leaving this issue, important for field implementation, unaddressed in this study.
Despite these limitations the close degree of quality score correlation between OSP measures and observed patient measures suggests that this methodology may provide a valuable option for quality assessment of providers in places, or for health conditions, where other assessment tools are impossible or impractical. While a variety of quality assessment methods may be used in urban areas, measuring the quality of care provided by rural providers is challenging for a range of reasons. For example, the language, ethnicity, likelihood of personal provider-patient knowledge, and the ethical and practical barriers associated with pediatric illnesses make introducing external mystery clients impossible.
Small patient volumes and long distances between providers make direct observation costly. And poor or incomplete paper record keeping in many small clinics obviates record abstraction. OSP has the potential to provide a solution to these challenges. The source of care in developing countries includes a wide range of providers, from qualified doctors to informal providers. It is desirable, therefore, for an assessment tool to be used to assess the presentation of complicated illnesses in a range of settings.
Although not part of this evaluation, we feel that the applicability of assessment tools across a range of provider types deserves study going forward. The value of a quality assessment tool can be conceptualized as a the extent to which they are able to provide information on a broad set of illnesses; b the extent to which they are able to provide estimates that account for confounders and; c the extent to which they measure knowledge versus practice. As described above, a number of current tools are limited in the conditions they are able to assess, or the patient populations they can mimic Table 4.
In this context, the results from this study lead us to believe that OSP offers an advantage on existing quality assessment tools in some instances, and merit a larger pilot of the use of OSP to assess the quality of management of pediatric malaria by rural medical practitioners is merited. The use of this methodology has the potential to provide an accurate and affordable solution to the challenges of rural outpatient quality assessment.
Browse Subject Areas? Click through the PLOS taxonomy to find articles in your field. Abstract Background Assessing the quality of care provided by individual health practitioners is critical to identifying possible risks to the health of the public. Introduction Assessing the quality of care provided by individual medical practitioners is critical to evaluating training, monitoring the scale up of programs delivering new treatments, and identifying possible risks to the health of the public.
Clinical Vignettes are hypothetical scenarios with questions or prompts for the chose course of action, given in stages to medical practitioners, with their responses noted for each stage before adding information [6]. Abstraction of Medical Records is the most common way of evaluating physician practices, however application to outpatient care provided by rural practitioners has been limited, and data collection by trained professionals is expensive [11] , [12].
High fidelity Medical Mannequins are common tools for medical education in developed countries, both during formative training and in continuing medical education, notably in anesthesiology, surgery, obstetrics, emergency medicine, pediatrics e.
Direct Observation, or the observing or recording of a real-life patient, is a well-established method for performance-based assessment of clinical practitioners, and has been proven effective in developing countries for the assessment of outpatient care [17]. Download: PPT.
Table 1. Summary and Sub-unit evaluations scores for directly observed simulated patients: Reception and diagnosis. Data Collection Data on quality of care for children presenting with fever, and for the OSP presenting with fever, were collected from 20 private independent medical practitioners in Mon and Kayin States, Myanmar two largely rural regions with endemic malaria between December 26, and January 12, Data Analysis and Interpretation The evaluation of OSP in terms of its potential as a hybrid quality assessment tool and its reliability rests on three aspects of our study: a its development following national and international guidelines with input of practitioners in the field on the relevance of the scenario, component items, and weights, b the overall score among highly trained, knowledgeable, and experienced practitioners, and c the agreement between OSP and real life patients.
Additional objectives may be included by particular facilitators depending on the overall goals of the session and topics of the post-didactic session. We generally try to have approximately three objectives with an effort to integrate a non-medical expert objective into the case when possible. The ability to manage acute upper gastrointestinal bleeding is crucial for anyone practicing acute care medicine. The simultaneous assessment, investigation and management are challenging even for the experienced practitioner.
Simulation creates an environment that permits learners to make real-life decisions without the possibility of adverse patient outcomes. The development of the case scenario using a stepwise algorithm allows the simulation to unfold according to decisions made by trainees.
Having a facilitator complete the run-through in advance ensures both that the case is of reasonable difficulty for the learner and enables instructors to address any limitations of the scenario. Finally, the use of a formal debriefing coupled with a post-scenario didactic session allows collaborative identification of knowledge gaps and process errors that arise during the simulation. Teaching emergency medicine residents to identify and manage an acute UGIB through the use of simulation promises to be of value. It permits learner development in an environment safe for both patients and learners.
The case also presents flexibility with respect to the degree of difficulty appropriate for each individual group of learners. A stepwise approach for the simulation is developed to facilitate the execution of a scenario and an integrated teaching session incorporating simulation and didactics with components of debriefing is described herein. Cureus is not responsible for the scientific accuracy or reliability of data or conclusions published herein.
All content published within Cureus is intended only for educational, research and reference purposes. Additionally, articles published within Cureus should not be deemed a suitable substitute for the advice of a qualified health care professional.
Do not disregard or avoid professional medical advice due to content published within Cureus. The authors have declared that no competing interests exist.
Consent was obtained by all participants in this study. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. National Center for Biotechnology Information , U. Journal List Cureus v. Published online Jan Author information Article notes Copyright and License information Disclaimer. Corresponding author. Cody Dunne ac. Received Nov 16; Accepted Jan This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Abstract Emergency medicine practitioners frequently encounter acute presentations requiring quick, directed treatment to ensure the best patient outcome. Keywords: simulation, emergency medicine, medical education, simulation based medical education, hemmorhage, shock, upper gi bleed. Introduction Upper gastrointestinal bleeds UGIB , defined as bleeding that originates proximal to the ligament of Treitz, have an annual incidence between 39 and per , [ 1 ].
Recognize and manage a deteriorating hemodynamically unstable patient with a suspected upper gastrointestinal bleed The report will follow the Context-Inputs-Process-Product model to present the data compiled [ 9 ]. Technical report Context The simulation was designed for the training of emergency medicine residents.
Inputs Personnel Two facilitators were present during the simulation. Equipment The training session was conducted in a simulation lab using a Laerdal SimMan 3G human patient simulator. Open in a separate window. Figure 1. Table 1 Summary of initial patient findings upon presentation to the Emergency Department to guide completion of Learning Objective 1.