Improvements to FRS risk prediction may be overstated
MedWire News: Studies claiming improved prediction of coronary heart disease (CHD) risk by addition of a novel candidate risk predictor to the Framingham risk score (FRS) may be unreliable, report researchers in the Journal of the American Medical Association.
Ioanna Tzoulaki (University of Ioannina, Greece) found flaws in the design, analyses, and reporting of studies that compared the predictive performance of the FRS with the FRS plus some additional predictor.
Of 79 such papers published from 1998 to 2009, 49 (62%) calculated or used the FRS inappropriately, while 15 (19%) modeled the additional predictor in more than one way but presented only the best-fit results or area-under-the-curve (AUC) results for one model without clarifying if this was the best-fitting.
Forty-one (52%) studies did not examine CHD as an outcome of interest in any analysis, only 23 (29%) had a mean or median follow-up time of 10 years or longer, and 25 (32%) claimed improved prediction in a subgroup whereas only seven (9%) formally tested subgroup differences.
Evidence for independent, incremental information based on multivariable analyses, discrimination, calibration, and reclassification were all under-reported and not adequately documented, add Tzoulaki and colleagues.
Overall, 63 (80%) studies claimed some degree of improved prediction.
Estimated improvements in prediction based on increases in AUC were significantly greater in those studies with suboptimal FRS calculation or use, in studies where evaluation of independence or discrimination was not adequately documented, and in studies where an additional predictor had been modeled in more than one way but only one model reported, in comparison with studies without these methodologic limitations.
Indeed, studies without such limitations showed on average no improvement in AUC with the additional candidate predictors, comments the team.
“Clinicians should cautiously interpret results that claim importance of new risk factors for initial CHD events,” commented Peter Wilson (Emory University School of Medicine, Georgia, Atlanta, USA) in an accompanying editorial.
“Risk assessment with traditional variables works relatively well at the present time: the approach is simple, the factors used to make the estimates are familiar, the cost is low, the interpretation is understandable by clinicians and patients, and results help guide lifestyle and medication recommendations.”
MedWire (www.medwire-news.md) is an independent clinical news service provided by Current Medicine Group, a trading division of Springer Healthcare Limited. © Springer Healthcare Ltd; 2009
By Caroline Price