Baucus Calls on McKinsey to Release Methodology Behind Survey Results

Finance Chairman Formally Requests Information on Methods, Conclusions After Silence from Consulting Company

Washington, DC – Senate Finance Committee Chairman Max Baucus (D-Mont.) today sent a letter to McKinsey Consulting Company Managing Director Dominic Barton, calling on the company to release the methodology behind a health care survey it published in early June.  Despite repeated requests, McKinsey has not disclosed the methodology behind their results thus far, which differed markedly from similar research on the topic.  Baucus today sent the company a formal request for the details on the survey, entitled “How US health care reform will affect employee benefits,” to ensure the public and policymakers have accurate information to evaluate the findings.        

“Honest public discourse requires a standard level of transparency -- one McKinsey simply has not met,” Baucus said.  “The conclusions McKinsey reached differ sharply from results of other reputable, transparent research on the subject.  McKinsey’s findings also counter what actually happened in Massachusetts when similar policies increased employer-sponsored health insurance.  We all want the most accurate information and the ability to evaluate its integrity, which is why McKinsey should answer these basic questions.” 

The results of the McKinsey study differ markedly from a number of other respected, independent organizations that have examined the same issue, including The Rand Corporation, The Urban Institute, Mercer, and The Congressional Budget Office.  Many of these reputable firms have welcomed the opportunity to provide Congress the methodology behind their surveys and research, including research on this topic.  Additionally, the McKinsey survey reached conclusions that counter the results of similar policies in Massachusetts when the state enacted health reform legislation that increased employer-provided health insurance.  Baucus also said it is important for McKinsey to disclose the survey methodology because that the article indicates McKinsey “educated respondents” in advance of asking survey questions.  Baucus called on McKinsey to release information critical for the public and for policymakers to understand how the company reached its conclusions. 

The full text of Baucus’ letter to McKinsey follows here:

June 16, 2011

Via Electronic Transmission

Dominic Barton
Global Managing Director
McKinsey & Company
No. 1 Jermyn Street
London SW1Y4UH
United Kingdom

Dear Mr. Barton:

I am writing to express my concern that McKinsey & Company has thus far declined to publicly release the methodology behind the June 2011 McKinsey Quarterly article titled, “How US health care reform will affect employee benefits.”  I understand that after repeated requests to meet with the staff of the Senate Finance Committee to discuss the methodology behind this survey, yesterday, McKinsey finally agreed to a meeting.  We would like this meeting to be fruitful, so below is a list of questions I expect you to be able to answer completely. 

The American Association for Public Opinion Research Disclosure Standards make clear that it is standard professional practice to release essential information on the methodology of publicly disclosed surveys. I urge McKinsey & Company to publicly disclose the methodology behind its survey consistent with these standards. 

In order to evaluate your survey’s conclusions, I request a full disclosure of the survey and its supporting materials, as well as answers to the following questions tomorrow:

1. Who funded the survey? If McKinsey & Company sponsored the survey, what account did the funding come from? Who are your biggest clients? Do you expect McKinsey & Company to benefit financially from the results of this survey?

2. Have the results of the study been featured in any presentations, whether written or oral, to potential new clients? Have they been featured in presentations to existing clients considering additional consulting work for McKinsey?

3. What was your sampling design?

4. Were the results statistically significant?

5. What was the breakout of the survey responses by:

1. industry
2. geography
3. employer size
4. the type of benefits the employer offers, and
5. the percentage of low-wage workers at the company?

6. Were there any oversamples, and if there were, how did you account for this? What was the margin of sampling error?

7. How were participants chosen? What methods were used to recruit the participants? What percentage of participants were McKinsey clients?

8. Did you use eligibility criteria or screening procedures? What was the participation rate? How were interviewees selected?

9. What position did the interviewees hold within their respective companies?

10. How were the interviews conducted? What script was used to “educate respondents” before asking questions? Could the script used to “educate respondents,” the wording of the questions, or the order of the questions have influenced the interviewees’ responses?

11. What questions were asked, including their exact wording? What was the order of the questions?

12. What was your internal review process for questionnaire? Did any outside experts review the survey and its results? What procedures were taken to verify the data?

13. Deciding not to offer health benefits is a major business decision that will be made by more than one person, how did you account for this?

Determining your methodology is particularly important because the conclusions drawn in this study are counter to the findings of a number of other reputable studies on this topic for which the methodology was disclosed.  And your findings are counter to what actually happened in Massachusetts when similar policies took effect and employer-provided health insurance coverage actually increased.  Providing the public and policymakers with the necessary information to evaluate the integrity of publicly released survey conclusions is essential to honest public discourse.  

Thank you for your prompt attention to this matter.

Sincerely,

Max Baucus
Chairman

# # #

• Print
• Email
• Tweet