If you want to know who the best surgeon in the hospital is, ask the surgical nursing staff. If you want to know who does the best job opening up coronary arteries using catheters, balloons, and stents, ask the cardiac catheterization lab nurses and technicians.
Unfortunately these approaches to comparing physicians’ skills are only available to hospital personnel. They are the only people who are in a position to compare the technical performance of many different doctors. This is not information the average patient can easily obtain.
The average patient has to rely on intangibles when trying to select a doctor. Fuzzy data such as bedside manner, self-confidence, board certification, waiting time for office visits and procedures, and word of mouth. Or, worse still, patients are told which doctor they have to see by their insurance company. None of these methods of choosing a doctor is likely to have a high correlation with a doctor’s technical skill. There’s got to be a better way…
Along comes the Surgeon Scorecard, a web application published by ProPublica.
With the scorecard it is possible to see raw complication rates of any surgeon (I’m sure the same data for cardiologists and other specialities is coming soon) in the United States who operates on Medicare patients. Pick any hospital and the individual complication percentages are displayed on a colored spectrum (green, yellow and pink on my screen) indicative of low, medium and high complication rates. An ominous red explanation point appears if one or more surgeons have complication rates in the pink zone. Click on individual surgeons and their data is shown in more detail, including numbers of procedures and 95% confidence limits (which frequently overlap more than one complication zone). Curiously, some surgeons with zero complications still have an adjusted complication rate in the medium range.
The publication of this database has unleashed somewhat of a twitter-and-media-storm, to the point that I’m not sure why I’m chiming in at all. Smarter people than I have complained about the methodology or have bemoaned the impact of all this on the practice of medicine. We are living in the era of “Big Data,” and “Data” is only going to get “Bigger” as it continues to accumulate in the ultimate garbage-in-garbage-out receptacle: electronic health records (EHR).
Unfortunately the subtleties of statistics are lost on the average patient, who just looks at whether a surgeon falls into the green, yellow or pink zone on the complication rate spectrum. Given the negative PR potential of this data, it is likely that some surgeons will refuse to operate on high risk patients, for fear of tainting their outcome data. So, as with quantum physics, the Heisenberg Uncertainly Principal holds in the field of medicine, in that the attempt to measure outcomes may result in changing outcomes. Certainly the numbers will look better if high-risk patients are avoided. But will healthcare actually be better? As was seen with EHR systems, the field of medicine’s square peg continues to be a difficult fit for computer technology’s round hole.
It is hard to argue against transparency, which seems axiomatically to be a good thing. There is no way to put the database genie back into the bottle. The only way to go forward is to make sure data collected is accurate and includes medical and demographic information about the population operated upon. This will allow the data to be normalized as best as possible. All that data collection is a pain and a distraction. But patients want to know how good their doctor is, and right now the Surgeon Scorecard is the only game in town — unless you can corner a surgical nurse and get his or her honest opinion.
