Best of the Week: They had one job

[View web version]( BEST OF THE WEEK [Cooperate]( And another thing... Welcome to Best of the Week, written on a day off (if writing this counts as a day off…) in Melbourne, ahead of a trip to see David Bowie’s Lazarus at the Arts Centre last night. (My verdict: It’s one for Bowie fans only, but iOTA was tremendous.) Today’s writing soundtrack: SBS Radio 3 - Chill Afternoons. (It’s intriguing what you can find on the outer reaches of a mid-market apartment hotel’s television service.) This week: The problem with political polling Loving the alien Thanks to Saturday’s election result, it’s been a bad week for the reputation of Australia’s research industry. Possibly its worst week ever. They had one job - to figure out which party was in front - and they failed. Unusually for a journo, I’m pretty good with numbers. I’ve got a reasonable grasp of statistics and back in my Hospital Doctor magazine days, I even briefly understood what a confidence interval was. It’s a good skill for journalists to have. Knowing what to look for in a survey can help to dig out a great news line. If the raw numbers are provided, there’s nothing more enjoyable than finding a better angle than the one teased by the press release. At the very least, it helps to avoid falling for a dodgy story being peddled by a PR person. It seems to have gone a bit out of fashion over the last couple of years, but for a while it seemed that every other press release (and indeed, news story) was based on a survey. You’d look at an article in a newspaper about how more men than ever before thought that wearing spectacles added to sex appeal, and inevitably the last paragraph would include the revelation that the survey had been commissioned by a glasses company. Or a survey about favourite snacks, commissioned by apple lobbyists (if there’s such a thing) would demonstrate the growing popularity of apples. And so on. Sometimes an actual research company might be involved in crunching the data, more often it would be based on a cheap online panel. Those who participate in these online panels have usually signed up in order to get airline points or a few cents, or the opportunity to win prizes. I was on one myself for a while. I quickly learned I was in an over-subscribed demographic, so I’d get messages saying the survey was full. So I quickly learned to start give a postcode from regional Australia instead, as that was a harder quota to fill. In 2011 the ABC’s Media Watch program did a devastating takedown of McCrindle Research which had been behind a string of press releases about everything from Australians eating sandwiches at their desks, to tipping habits. Media Watch suggested that sample sizes were small, and based on the relatively unscientific approach of using free online tool Survey Monkey. In the case of surveys organised by PR agencies or brands, they have of course been commissioned with the sole aim of creating a news angle, not with some sort of noble scientific mission in mind. The only thing that matters is generating a number that can generate news coverage. There are of course analysis techniques that will help deliver a more interesting number. How the question is asked is an important one. Eliminating any “don’t know” or “don’t want to say” responses also helps. And multiple options might be grouped together. Anybody who tickets either the “agree strongly” or “agree slightly” box about the sexiness of spec wearers might be characterised in the press release as agreeing with the question. In Mumbrella’s world, a lot of research reaches our news inbox. A specific medium will commission a survey, or sometimes more detailed, expensive research. Sometimes they’ll invest in making the process as rigorous as possible. Nonetheless, that research was still commissioned in order to prove a hypothesis. Inevitably, the results will allow the commissioner to offer “proof” that this particular medium or product is best at this particular thing. It would be a short-sighted research business that accepted the commission without being confident it will deliver the result its client wants. In the unlikely event they don’t, then the work would never get published, and the research company would see no more commissions from that client. In the past, where the methodology looks sound (a standard journalism starting point: is the sample at least 1,000 people), we’ve given the research a run in Mumbrella. But it’s generally been with a slight sinking feeling. We know we’ll be greeted by (correctly) sceptical reader comments that a study commissioned by Brand A will of course prove that Brand A is the best. These days, we generally don’t publish the marking-your-own-homework surveys. (That’s one reason you don’t see much coverage of the NewsMediaWorks-funded Emma survey of print readership in Mumbrella, by the way. It may indeed be world-standard research as they argue, but it’s commissioned by the publishers, with an outcome in mind.) Yet the refusal to publish can offend. Many a trade marketer has got quite cross with us because the implication of choosing not to publish is that we must think they’ve done something dodgy. A smarter approach comes from brands that commission surveys that offer interesting insights into their world rather than themselves. Back in 2013, a survey from out-of-home company Adshel caught my eye just as we were about to send the day’s newsletter. It contained the delicious statistic that only 24 percent of media agency people living in NSW had ever visited Parramatta, while 62 percent said they’d been to the then fashionable North Bondi Italian restaurant. (It’s since closed - medialand is fickle.) It allowed me to write a news story with the headline “Sydney agency people live in a media bubble and most have never even been to Parramatta, suggests study”. It tapped in brilliantly to the issue of the media bubble, and the difficulties the industry has in understanding the lives of ordinary Australians. Six years on, I think that’s why the Boomtown campaign is resonating. So I suspect that I didn’t look too hard at the quality of the research because I really wanted to publish the Adshel piece. I wrote it quickly and it led our daily email newsletter that day. The resulting discussion thread (a respectable 75 comments) featured a mixture of those debating the issue of the media bubble, along with those questioning the rigour of the research. It can become a bit of a conspiracy between publisher and the commissioner of the research when they both really want the story to be published. It’s a bit too easy for the journalist to forget about their duty to do their best to tell the reader what’s really true, not just what’s entertaining. Like Adshel’s work, Junkee is another media owner which has been smart in how it’s used research over the years. If it conducted research that highlighted how many people read Junkee compared to Pedestrian, then nobody would have given the findings much weight. (Rather like AdNews’ occasional media kit surveys of AdNews readers that prove that AdNews readers read AdNews.) But instead, Junkee hit on the model of researching its audience and producing an impressive annual insight into what young Australians were thinking and doing. It provided the basis for an entertaining roadshow presentation in which there was no overt Junkee sales pitch. But it also positioned Junkee as the authority on the youth audience. When the client’s youth brief then came into the media agency, that positioned Junkee as the obvious direction to turn to for the solution. And we’ve generated our own research too. At the end of last year, we presented our “State of the Industry” research at the Mumbrella Next event. And the only time a news story about Mumbrella was top story on both A Current Affair and Today Tonight was when we published our Encore Score survey into what the Australian public thought of various celebrities. (The short version: They love Hugh Jackman and hate Kyle Sandilands.) It was also an intriguing insight into the competitive nature of tabloid TV. We offered the research to one of the two shows (I forget which) and they declined it, so we offered it to the other who took it. As soon as the on-air promos started to run, the first show decided they loved the story after all, and rushed out a spoiler, Which brings me onto the political opinion polls. One of the reasons that news organisations commission political opinion polls is for their PR value. On occasion, I review the newspapers on ABC News Breakfast. That’s the other reason I’m in Melbourne as I write this. You’ll find me on the program - which comes from the ABC’s South Bank studios - at about 6.45am this Monday. I usually do News Breakfast on a Monday, which happens to be the day that polls usually come out. I’ve lost count of the number of times I’ve talked on the show about Newspoll in The Australian or the Ipsos survey for the Fairfax (now Nine) papers. Doing my 4.30am scan before going on air, I’d be looking for two numbers: First, what the two-party preferred vote was for each party. Second, which of the two main party leaders was preferred. It’s agenda-setting. Whenever the polls are published, the ripple starts the night before as they seep out onto Twitter. The next morning the TV and radio networks are following it too. Which means that those newspapers are being talked about and helping set the day’s news agenda for everybody else. Once he was Prime Minister, Malcolm Turnbull’s critics never let him forget that one of his justifications for rolling Tony Abbott was that his predecessor had lost 30 Newspolls in a row. Also not to be underestimated is that newspapers’ most important weapon in retaining subscribers is habit. For some it’s Sudoku. For others it’s cartoons. (I’ve never really forgiven Sydney’s The Daily Telegraph for dropping Dilbeert.) Columns too - imagine the AFR without Rear Window or The Oz without Margin Call. And for the political classes, it’s the polling. Which takes us to last Saturday’s election result, which was an embarrassment both for the polling organisations and the media. Was everybody so deep in the bubble they didn’t understand what was happening in real Australia? With retrospect (and almost nobody called it beforehand) there are lessons from elsewhere. In the 1992 British general election, the polls said that Labour would win comfortably. Instead, incumbent Prime MInister John Major, who had inherited the role a few months earlier after his predecessor Margaret Thatcher was rolled, retained power. Afterwards the Market Research Society held an inquiry into the failure of the polls, and the “shy Tory” factor - those who didn’t want to admit to pollsters which way they’d vote - was blamed This time in Australia, of course, the polls said that Labor would win comfortably. Instead, incumbent Prime Minister Scott Morrison, who inherited the role a few months earlier after his predecessor Malcolm Turnbull was rolled, retained power. This week, [the Association of Market and Social Research Organisations announced an inquiry](, while Morrison’s “quiet Australians” who didn’t want to say they’d be voting for the Coalition, are already being cited. Yesterday we republished an insightful piece from The Conversation, written by statistics academic Adrian Beaumont. He argued that [the polls have been wrong for ages](; this wasn’t the result of some late change in voting sentiment. Part of the problem could be methodology. The mix of automated calls, researcher phone calls to landline numbers and mobile phones, online panels and (decreasingly) face-to-face interviews is unclear. And the question of whether those surveyed are honest about their intentions is another. And potentially, the sampling does not match individual electorates. Given Australia’s 76-seats-to-win parliamentary system, that’s important, because the seat-by-seat distribution of voting matters more than first-past-the-post popularity Beaumont also suggests there was a pack effect, with Newspoll setting a tone and then other polls tweaking their numbers so as not to be too far off. Yikes. So what next? [The Sydney Morning Herald and The Age said yesterday that they would probably be ditching polls](. (Regular polling does not come cheap.) National editor Tory Maguire wrote: “Polling companies are the main reason Saturday night’s result took voters, the media and many political operatives by surprise,” says Maguire. “The implications of our major pollsters making the same mistakes in a consistent way are serious.” But [News Corp plans to stick with it](. Policy boss Campbell Reid told Mumbrella: “To simply declare “the polls got it wrong” is simplistic. We would rather work with our polling partners to continue to improve rather than throw up our hands.” I suspect News Corp’s approach will prevail. The Nine newspapers have thrown their partner Ipsos under the bus somewhat. But even if people doubt the polls more than they have in the past, they’ll remain too agenda-setting a tool for newspapers not to retain. But their credibility is weakened. The fact is, the polls were only a couple of percentage points out. 51-49 one way rather than the other. But, just like Brexit and Trump, that made all the difference in the world. Nearly three decades on in the UK, nobody really trusts polls. The same will now apply in Australia. The man who sold the world And I have just one more job, which is housekeeping. I’ve got three things to mention. First, and most important, have you looked at the Mumbrella360 program yet? It’s the week after next. I honestly think that if you [look at the program](, you’ll want to come. So please take a look. Second, Mumbrella will soon (well, in a few months’ time) be on the move. We’re looking for somewhere bigger. So Mumbrella House is now for sale. [If you’re looking to buy office space in Chippendale, here’s the link](. And finally, we had [a really good Mumbrellacast]( this week, with MCN boss Mark Frain, who’s got one of media’s toughest sales jobs. It’s worth a listen. Time for us to go and find somewhere in Melbourne for breakfast. The coffee may not be as good as Sydney, but I’m sure we’ll get by. And then perhaps the Terracotta Warriors at the NGV. As ever, I welcome your emails to tim@mumbrella.com.au. And our news editor Paul Wallbank - paul@mumbrella.com.au - is on the weekend newsdesk. Have a great weekend. Toodlepip... Tim Burrowes Content director - Mumbrella [Cooperate]( Mumbrella | 46-48 Balfour Street Chippendale NSW 2008 Australia This email was sent to {EMAIL}. If you would rather not receive Mumbrella's Best of the Week email you can [unsubscribe]( or [manage subscriptions](. [Facebook]( [LinkedIn]( [Twitter](