What Makes a Good Medical Scoring System?

20050629My good friend Al Cichon wrote the following comments to me about my article on the 10 Point Pain Scale.

Use of a ‘scale’ to measure subjective factors is a true oxymoron (heavy on the moron part). The ‘fifth vital sign’ is a misdirected effort to solve a true quality care problem – in my opinion. Vital Signs are objective (as in measurable) indicators that have been demonstrated to provide consistently valid data for patient care.

JCAHO has imposed the ‘Pain Scale’ in an effort to assure that the assessment of pain will be factored into patient care. Unfortunately, it is a bureaucratic response to a clinical situation – if it can be made to be measurable (regardless of practical validity) it can be enforced. My apologies to proponents and defenders – but no matter how well intended; it is still dysfunctional.

Yet, we are obliged to employ some process of assessment to determine the impact of subjective symptoms in an efficient / effective manner. The many schemes (Pain 1-10; Cardiac pain 1-5; etc.) developed have achieved relative success / adoption / adaptation. It does seem that these processes have provided some benefit – yet they all suffer from the same susceptibility – subjectivity. Whether you are working in corrections (where any subjective report must be verified) or the community (where verification is not always considered) any of these schemes has the same risk – because there is no objective validation.

One possible option to stabilize the use of any such scale is some method of anchoring the initial or end point of the scale. That would then provide a somewhat stable reference for guiding care and transmitting information (about the patient) to other providers. Also, this mechanism can be employed in other subjective areas.

Begin with asking the patient – ‘What is the worst pain you have ever experienced?’ (the response is usually associated with fractures, renal calculi, child-birth, etc.) Now, lets’ label that as a 10 (or 5) and now –‘What is the discomfort you have now in comparison to that prior event?’ Then document the exchange in the record: Worst Pain: fractured wrist – 10); Current Pain: pulled muscle – 7. Now, no matter the number system you have an ‘anchor’ to both the scale and the present condition that can be related to the care process and others. It is important to document the ‘source’ of pain too – as this may be quite useful if you are explaining the case later (DOC, court, etc.)

One variant of this is: If you had a ‘bucket labeled depression and a stick marked off from 0 to 10 what is the worst you’ve ever felt and when”? The response will then provide an anchor (9 when I was admitted to the hospital after trying to kill myself). Additional questions are: How deep is that bucket now? Has it ever been at 0? If I could make it happen (wave magic wand) and get you released and home now – what would it be? (interestingly it can go up with release for many reasons)

When you ‘anchor’ the ‘pain scale’ it establishes the assessment in a relatable setting and can be quite helpful in guiding patient care. The ‘anchor’ can also be quite useful in discerning the ‘historical validity’ of the case as well – ‘Doc when I had that car accident with multiple fractures it was a 10; now my sprained (non-swollen, non-bruised) ankle is a 10 too!’

Clinical judgment is always the most important skill in any patient care situation. The data subjective & objective is helpful but must be understood / applied in the clinical context.

Also confounding the assessment is the different types of acute / chronic discomfort (myofascial, neuropathic, visceral). As important as assessing the level of pain is the type – physiologic source – since it is critical in guiding treatment.

As noted – ‘chronic pain’ is better measures by an assessment of the patients ADL (what daily activities are disrupted and is that new). If a patient can function (nutrition, hygiene, elimination) and participate in some activities then treatment of the physiologic cause is the most important clinical consideration.

Thanks, Al!  Excellent comments.  Let’s summarize:

  1. The 10-Point Pain Scale is not the only scoring system in medicine for subjective complaints. There is the Hamilton Rating Scale for Depression, for example.  Also, the Clinical Institute Withdrawal Assessment for Alcohol (CIWA-Ar) for alcohol withdrawal.   There are scoring systems to rate abdominal pain and chest pain and the likelihood of pulmonary embolism.  I could go on and on.  All of these systems have the same limitations and liabilities, though some do a better job than others.
  2. Just because you assign a number to a complaint, that does not make it objective.  Because each of these scoring systems assigns a number to the complaint, there is a tendency to think of them as objective rather than subjective.  But as Al points out, this is not the case.  There is no real difference between someone who says his pain is “a six” and someone who says that he has “moderate pain.”
  3. In order for a subjective scoring system to work, it needs to be “anchored” in some way.  Al anchors the 10-Point Pain scale on the patient’s own worst experience.  Another commonly used clinical pain scale, the Visual Analogue Pain Scale, anchors the scale onto facial expressions.  The more “anchored” a system is, the better it is.  The less anchored it is, the less useful it will be.Unknown
  4. The more numbers in the scoring system, the less reliable it becomes.  Let’s say instead of a 10-Point Pain Score, we used a 1000-point pain score and were asking patients “would you say your pain is a 671 or a 672?”  Of course, a pain score of “672” gives us no more useful information than a “6.”  In fact, it gives us less useful information because it is more confusing.  In a scientific sense, the more numbers a subjective rating system has, the less “inter-rater reliability” that system has. The simplest subjective scale has only two scores, “None (zero)” and “Some (one),” as in “Do you have pain or not?”  The subjective scale perhaps used most often in daily life (Like when you go to a Thai restaurant and are asked how hot you want your food to be) is a four score scale:  None, Mild, Medium, Severe.  The Visual Analogue score is a 6 point scale.  CIWA-Ar uses 8 points.  And the 10-Point JCAHO Pain Scale uses 11 points (zero plus one through ten). Is the 10-Point JCAHO scale more accurate than a simple “mild-moderate-severe” system?  Probably not. In fact, no “probably” about it.  No.
  5. Subjective scoring systems work better at evaluating changes over time than the initial severity of a symptom.  If a patient says his pain is a “6,” I may not know exactly how that is different from a “7” or even a “4.”  But later, when the same patient now rates his pain as a “5,” I am pretty confident that his pain has decreased, at least by a little.
  6. Subjective scoring systems only work if the patient understands and is cooperative with the process.  Al helps the patient to understand the process with his excellent “anchoring” technique (which I will be adopting, by the way).  But the system still will not work if the patient always, no matter what, says “my pain is a 10.”   That is the main problem I run into in my jails; deciding when patients are exaggerating their symptoms.  For example, if a patient complains of “severe” constipation, what weight do I give to their use of the word “severe?”
  7. There are two types of subjectivity in scoring, the patient’s and ours.  The patient is subjective when rating her own pain or depression or whatever.  Then we clinicians make our own subjective assessments.  How sick does the patient look?  Often, the two assessments do not coincide, as when the patient rates his abdominal pain as a ten while munching on Cheetos and looking bored.  If I do not trust the patient’s own subjective assessment, sometimes I must substitute my own clinical judgment.
  8. Scoring systems for pain perform worse for chronic pain than for acute pain.  For chronic pain, a more useful assessment tool is to evaluate how the chronic pain affects Activities of Daily Living (ADLs).  Is the pain too debilitating to hold a job?  Play golf?  Go to the store?  Walk?  ADLs are usually much easier to assess in a correctional facility than in the outside world.  You can go down to housing and watch the patient.  How easily does the patient sit, stand, walk?  Does the patient go to recreation?  Sit for long periods of time playing cards or watching TV?  This sort of assessment is very useful for gauging the impact of chronic pain.

Any thoughts?  Please comment!

Enhanced by Zemanta

5 thoughts on “What Makes a Good Medical Scoring System?

  1. Karel

    The problem here may be that clinicians seek to pair patients pain scores to treatment plans.
    Your pain is a 6 or more? You get opioids. 5 or less? Ibuprofen or acetominophen must be sufficient for you. Now, regardless of anchoring etcetera, this will not work(morfine for a sprained ankle when patient experiences a 7?)
    Pain scores to evaluate if pain is lessening/worsening, fine, but not for clinical decision making please.

    1. Jeffrey Keller Post author

      Excellent comment, Karel. Patients in correctional facilitates will quickly figure this one out! And since the pain score is what the patient says it is, patients are essentially prescribing to themselves.

  2. Than A. Aw, MD

    I wonder if one day chronic pain management will fall under conditions that is being followed periodically like diabetes, hypertension etc.,. ? And if so, what will the practitioner have to do in those appointments ?

  3. Dolph Druckman

    Sorry, I’m may have misunderstood, but what is the endpoint for therapy here?

    In the community yes, there’s this new paradigm to use smiley faces, but what does this mean? Treat for maximum pt satisfaction (aka: pain relief?, intoxication?) or should our endpoint be something else? (say, mitigation of functional impairments??)

    In the few years since this post was begun, I think there’s all to massive a body of evidence for the collateral damage (to society) that occurrs when persons with health issues are perceived as “consumers” of services that are delivered by Providers of these health services, instead of the decidedly old school concept of patients and physicians…..hey, do no harm works.


Leave a Reply

Your email address will not be published. Required fields are marked *