Authors of a study just published by the American Journal of Public Health (available online now and in the Dec. issue of AJPH) say that youth anti-smoking television ads funded by tobacco companies are ineffective, and that the spots intended for parents may even have harmful effects. Among 10th and 12th graders, they say, higher exposure to the parent-targeted ads was associated with lower perceived harm of smoking, stronger approval of smoking, stronger intentions to smoke in the future, and a greater likelihood of having smoked in the past 30 days.

"Of course," I can hear you saying to yourself, "we all know that Philip Morris is intentionally sabotaging the ad campaign so that it ends up bringing in more future smokers, or at least is just burnishing its reputation with this campaign as window-dressing." I would have thought so myself. Except that in looking into the campaign, I found out that an old friend and colleague, Cheryl Olson, is on the advisory board for Philip Morris USA's Youth Smoking Prevention initiative.

Cheryl and I met in grad school, and we have since worked on various projects together, including evaluating tobacco prevention programs. She, along with her husband, psychologist Larry Kutner (who is also the chair of the advisory board), founded and co-direct the Center for Mental Health and Media at the Harvard Medical School. I know that Cheryl is no tobacco industry patsy, and that she would not compromise her integrity if she suspected there were any nefarious strategies behind this campaign.

I got in touch with Cheryl and asked her for her take on the research results that were just published. She, not surprisingly, had a lot to say about why this study is flawed and may just be showing what the researchers wanted to find. I invited her to send me her thoughts to post on the blog, which I've reprinted here:

For the past couple of years, I have consulted to Philip Morris USA on smoking cessation and prevention. I had primary responsibility for the content of the QuitAssist cessation guide, and also review and contribute to materials aimed at parents from the Youth Smoking Prevention group.

I work with a group of researchers and clinicians who are affiliated with various universities and hospitals. (We do this work independently from our institutions.) Part of our mandate is to oversee the quality of material content and evaluation, and be vigilant for any unintended negative effects.

Collaborating with a tobacco company can be an awkward and uncomfortable experience for a public health researcher who worked in tobacco control. But since Philip Morris USA is voluntarily committing 100s of millions of dollars to prevention and cessation – going well beyond the requirements of the Master Settlement Agreement - it’s important that a group of independent researchers and clinicians be part of this process to ensure that the resulting materials are honest, research-based, and effective.

My work has included: interviewing parents and former smokers and choosing quotes to use in print and web materials; incorporating research and advice from experts (selected by me) who work with smokers and parents; writing print and web content; and observing focus groups to help formulate and test content (including groups of Spanish speakers with simultaneous translation). I am proud of the brochures and guides I’ve helped develop. Their quality is apparent to anyone who reads them – which may explain why I have received at most a half-dozen phone calls or emails from academics, health workers or reporters asking why I got involved in this.

It is also exciting to be part of a project with such a huge reach. To date, PMUSA has distributed 70 million parent brochures, and hundreds of thousands of QuitAssist guides. The main role of the guide is to encourage smokers to connect with useful government and nonprofit cessation resources; I have heard that the PMUSA web site is the most visited cessation site in the US, and refers more traffic to government web resources than any other source.

I am not involved in developing PMUSA’s TV, radio or magazine ads on smoking cessation or prevention. But I do have some concerns about the article in December’s AJPH. There are some serious flaws in this study’s methodology that make it hard to draw any conclusions about the effects of the ads in question. To describe just two:

1) The “Talk, They’ll Listen” campaign that supposedly harmed children (the one aimed at parents) is based on an estimated exposure to an average of 1.13 thirty-second ads over a four-month period. Let’s take a closer look at this measure of ad exposure.

According to the recent Kaiser Family Foundation national survey, kids between the ages of 8-18 spend an average of 3 hours and 4 minutes per day watching broadcast television. Let’s call it three hours for simplicity’s sake.

There are 122 days in a 4-month period. So kids watch an average of 366 hours or 21,960 minutes of television in a 4-month period. A parent-oriented commercial lasts 30-seconds - or 1/43,920 of their viewing time. If they see 1.13 parent-oriented commercials in 4 months, that means that the commercials comprise 0.000026 (twenty-six one-millionths) of their television viewing content and time. Does it make sense to assume that such an extremely rare event would have the levels of influence on behaviors and attitudes that the authors claim? Based on this exposure issue alone, it’s hard to take the article seriously.

2) The authors should have used a 99% confidence interval (not 95%) with such a big data set [n=103,172]. That is a standard approach to avoid getting significant results just due to a large sample size. The use of 95% CIs raises questions about the odds ratios.

This is pretty disappointing. It’s hard not to think that the authors were determined to find something negative to say. The dramatic statements in the abstract are hedged a lot in the actual paper text, but the abstract is all that many researchers – and most journalists - will read.

This could have been an opportunity to get some useful lessons that could be applied to future media campaigns, and to model state-of-the-art methods for evaluating media-based behavior change materials – methods that could be used by academics and industry alike.

Among other things, it’s too bad that the authors lumped together ads aimed at youth from two companies that used very different approaches. This doesn’t tell us anything about what aspects might have been particularly helpful or harmful.

Given their well-documented past behavior, tobacco companies (and their anti-smoking media materials) must receive ongoing scrutiny from the public health community. But that shouldn't mean checking our common sense at the door.

Technorati Tags: smoking, philip morris, ajph, tobacco, advertising, youth