How To Use Data to Quantify Prehospital Performance

At the end of my first year in my first paramedic job, I got a performance review that looked something like this:     3          3          3          3.

Each “3” was my score on a scale of one to five for allegedly quantifiable attributes like teamwork, judgment and effort. Given the lack of supporting details, I might as well have been graded on cat grooming, too.

I have no idea how my boss “calculated” all those threes. There were no comments accompanying them and no policy governing the assignment of threes instead of perhaps twos or fours. That happened 20 years ago, but I still remember how frustrated I felt to be classified arbitrarily as average.

Constructive Feedback

There are much better ways to evaluate EMS employees; just ask Rob Lawrence, chief operating officer of Virginia’s Richmond Ambulance Authority. During an hour-long webinar that focused on clinical quality improvement, Lawrence and his director of QA/QI, Tom Ludin, showed how their agency uses FirstPass data management software to gauge patient care.

One of FirstPass’s most robust features is its ability to quantify elements of prehospital performance. We’re not just talking about response times, a mainstream measure Lawrence characterized as “only one-fifth of the job.” Lawrence and his staff also task FirstPass with call-by-call analysis of patient assessment and treatment—less conventional, more compelling aspects of quality assurance.

Lawrence, who calls “data” his favorite four-letter word, suggested agencies that buy into total quality management (TQM) can mine gigabytes of street-level details to grade caregivers and improve productivity.

“Our ability to interrogate electronic patient-care records means we’re able to understand exactly what’s going on,” he said.

Ludin offered Richmond’s Provider Protocol Compliance summary as an example of practical information retrieval. The extract, one of many available from the same database, shows the percentage of each paramedic’s calls that are compliant with medical protocols.

Compliance is based on “bundles” of protocol-specific criteria against which ePCRs are scored. For example, an ischemic chest-pain protocol might dictate administration of sublingual nitroglycerine. A medic’s failure to do so for non-hypotensive patients would indicate lack of compliance, subject to review by Ludin. Such manual oversight ensures consideration of mitigating circumstances—a medication allergy, perhaps.

After “raw” scores calculated by the system are examined by Ludin and revised as necessary, employees are evaluated based on their adjusted compliance percentages.

One Not-so-small Step…

For TQM devotees like Richmond Ambulance, justifying the purchase of QA/QI software has a lot to do with man-hours saved by automating review of ePCRs.

“The question becomes, do you want a whole department of checkers?” asked Lawrence, “or would you rather put those people on the street where they need to be?”

Lawrence and Ludin see protocol compliance as one measure of clinical quality, but not the only measure. Sedgwick County (KS) EMS uses FirstPass to compare prehospital care and hospital diagnoses—an intriguing next step for Richmond, according to Lawrence. Once outcomes are added, EMS providers will be able to see not only how closely they followed protocols, but how accurately they diagnosed patients’ complaints.

Sometimes two of a kind beats four threes.

You can view the RAA/FirstPass webinar by clicking here.

Mike Rubin is a paramedic in Nashville, Tennessee and a member of EMS World’s editorial advisory board. Contact him at mgr22@prodigy.net.