The current system used to measure the quality of health maintenance organizations (HMOs) is inefficient, researchers at Harvard Medical School found in a study released yesterday.

The non-profit National Committee for Quality Assurance (NCQA), which is responsible for issuing these rankings, disputed the finding and called the current system “effective.”

In NCQA’s current “health care quality report cards,” HMOs gain quality points based on their performance in five areas: preventative health care measures, more complex operations, claims processing, communication with physicians and general “effectiveness of care.”

But HMOs can choose not to disclose their rankings—which undercuts the effectiveness of this system, researchers said.

The study, led by Danny McCormick, an instructor in medicine at Harvard, was first published in yesterday’s Journal of the American Medical Association.

Associate Professor of Medicine David U. Himmelstein, who assisted with the study, said the research team developed the idea for the study after analyzing the quality data for HMOs over several years. The doctors noticed that individual HMOs did not appear consistently in the rankings from year to year.

After further investigation of quality ratings from 1997 through 1999, Himmelstein said the researchers discovered a pattern of HMOs performing poorly one year and the next year withdrawing from the rankings.

Himmelstein criticized the ability of HMOs in the current system to evade public scrutiny of their quality rankings.

“The monitoring quality of healthcare in the U.S. is a joke and a failure,” he said.

Himmelstein suggested that a nationalized health care system would ensure quality healthcare and eliminate irresponsible HMOs, which he referred to as “financial middlemen who contract for care.”

At the least, he advocated more stringent ranking policies or ending the ability of HMOs to withhold their rankings.

NCQA spokesperson Brian Schilling said he agrees shortcomings exists in a system that allows HMOs to withhold their rankings but noted the NCQA changed its policy in 1999 to require all accredited health plans to report their rankings. HMOs that are not accredited, however, still do not have to release the rankings.

Schilling said the data used by the researchers were “outdated” because of this policy change.

With the NCQA’s policy change, the percentage of HMOs withholding rankings has been reduced to 15 percent in the past year.

Schilling said the best way for patients to overcome the shortcomings of the rankings system is to “pick an accredited plan that publicly reports data.”