A Smart Play by Wikileaks

Like everybody else, I’ve been following the drama of Wikileaks, billed as “the world’s first stateless news organization,” and what the documents leaked through the service reveal about the U.S. war in Afghanistan. Few stories could more dramatically show how the press is changing. As Jay Rosen notes at the above link, “in media history up to now, the press is free to report on what the powerful wish to keep secret because the laws of a given nation protect it. But Wikileaks is able to report on what the powerful wish to keep secret because the logic of the Internet permits it. This is new.”

A smaller part of what Wikileaks does also strikes me as new — and enormously smart. Back in October, Computerworld’s Dan Nystedt wrote about Wikileaks and talked with Julien Assange, a member of its advisory board. Assange told Nystedt that Wikileaks planned to create a form that publishers could put on their Web sites allowing readers to “upload a disclosure” to the publisher using the online clearinghouse. Wikileaks would take care of protecting the source and any legal risks related to publishing the document, as well as confirming that it’s real. Once confirmed, Wikileaks passes it on to the publisher and gives that publisher an embargo period during which the information is exclusive. After that, it’s available to the world. (I’m trying to figure out if the disclosure form has become a reality yet — Wikileaks, understandably, is a bit overloaded right now.)

As Assange explained last fall, Wikileaks needs that period of publisher exclusivity to guarantee a leaked document gets attention: “It’s counterintuitive. You’d think the bigger and more important the document is, the more likely it will be reported on but that’s absolutely not true. It’s about supply and demand. Zero supply equals high demand, it has value. As soon as we release the material, the supply goes to infinity, so the perceived value goes to zero.”

That’s smart, but what’s even smarter is the clever way this arrangement simultaneously serves a publisher and puts pressure on that publisher. Exclusives are precious commodities — many an organization or source has tried to interest a reporter in something and been asked, “Who else has this?” Wikileaks’ disclosure form allows a source to effectively say, “No one — but that will soon change.” As long as a leaked document proves legitimate, the countdown begins and the pressure for publishers to do something with it starts to grow. Sources can favor publishers, but also automatically have a Plan B, with no jousting or negotiating required.

Think how astonishing the little ecosystem that’s been created is compared to how things worked a generation ago. Not so long ago, newspapers, magazines and their ilk were effectively the only ones who could publish and distribute something to a world-wide audience. Now, sources can use a clearinghouse to release that information, with the clearinghouse employing a savvy understanding of publishers’ priorities to maximize the impact of that information. The publisher that once held all the power now plays a lesser role as part of a bigger bargain between three parties.

Not so long ago, that would have seemed like something snatched from a Neal Stephenson plot. Now, it’s new but makes perfect sense. Before too long, it will just be the way the world works.

Algorithms Aren’t Evil

Recently I’ve been quoted a fair amount about Demand Media and other so-called content farms, and come to accept that my initial description of Demand Media as “how our profession ends” will follow me around forever. (The Web is the end of forgetting, after all.) My views of content farms and what they do has become more nuanced since then, but so be it — I wrote it, after all.

That said, there’s one thing about content-farming I don’t object to in the least, and that’s their use of algorithms to create story ideas. News organizations are beginning to use the same tools and taking flak for it, which is reactionary and silly.

The “algorithm-as-editor” meme started with this article by the New York Times’ Jeremy W. Peters about how Yahoo is using search queries to help guide its writing and reporting for The Upshot. Peters’ take on what Yahoo is doing and how such efforts are shaping editorial agendas is pretty nuanced overall, but that got lost because of his rather breathless lede: “Welcome to the era of the algorithm as editor. For as long as hot lead has been used to make metal type, the model for generating news has been top-down: editors determined what information was important and then shared it with the masses. But with the advent of technology that allows media companies to identify what kind of content readers want, that model is becoming inverted.”

Beyond the fact that that’s not true — the article explores algorithms as a tool used side-by-side with traditional brainstorming — the emphasis angered Upshot editor Andrew Golis and writer Michael Calderone, who took their grievances to Twitter, as chronicled by Business Insider’s Joe Pompeo. (Sample tweet from Golis: “OMG, online journos periodically use data to figure out what readers are actually interested in! PANIC! HANDWRING!”) Today, Lacey Rose of Forbes chatted with Yahoo’s Jimmy Pitaro about the issue. “First off, the algorithm and the automated approach are one component of how we’re identifying topics and programming sites,” Pitaro told her. “We’re sitting on all of this [audience] data where our users are telling us specifically what they want and we need to take all of it into consideration as we program both video and text on our site. The way I look at it is we need to be feeding our users both what they want and what they need. If you cover both then I think users will be kept well informed.”

Pitaro is exactly right. That top-down model of news discussed by Peters — “editors determined what information was important and then shared it with the masses” — is obsolete, a product of an age in which we couldn’t know what readers thought was important in any timely way. And good riddance to it. Seeing that top-down model as a hallmark of journalism instead of as a technological limitation was a trap, as the implicit arrogance of that model won us few friends among readers and obscured plenty of the good work we do.

Journalists worry themselves, sometimes to distraction, about the idea that traffic and studying search queries will wind up driving editorial priorities and assignments, squeezing out substantive stories in favor of, I don’t know, lurid tales, gadget reviews and pet pictures. Peters quotes Perfect Market’s Robertson Barrett as saying that “there’s obviously an embedded negative view [in newsrooms] toward using any type of outside information to influence coverage.” Which there is. But which possibility is more likely, and thus more damaging to journalism: That a news organization given search queries will be run with them and devolve into Funny Cat Pictures Daily Times, or that worrying about that will lead editors to reject valuable insights into what their readers are looking for? It may strike an editor as noble to stop his ears to the guy from some outside service who’s crunched the numbers on search terms, but what that editor is really doing is refusing to listen to his own readers.

Pitaro tells Peters of one Yahoo story that emerged from looking at traffic: Why do Olympic divers shower after they get out of the water? You know what? It’s a good question.

In his Wired portrait of Demand Media, Daniel Roth asked Demand Media’s Byron Reese, creator of the algorithm the company uses to generate story ideas, what Demand’s most valuable query was. Reese’s answer: “Where can I donate a car in Dallas?” Reese didn’t know why so many people in Dallas were looking to donate cars, but if I were a Dallas-area editor, I’d sure want to know — and I’d be glad for whatever mechanism had brought a good story idea to my attention.

In the spring, I spoke on a panel at the MIT Sloan Sports Analytics Conference, and ESPN.com Editor-in-Chief Rob King told a funny story about one of his first days on the job. (You can listen in here, starting at 47:50 in the video.) Listening to the radio in the car on his way to work, King heard India and Pakistan were playing a big cricket match. So he asked about it in the morning meeting — where it hadn’t come up — and said ESPN ought to do something. Later in the meeting, a staffer ran through a list of sports search terms were trending, and something called “20 20” was at the top of the list. What was 20 20? Nobody knew. It turned out to be the India-Pakistan match — in fact, King said, cricket-mad ESPN employees were holding a big party elsewhere on campus to watch. King’s reaction? “That was the moment where I thought, ‘We really need to pay attention to what our audiences are into, because they’re telling us where the traffic is.’ ”

My issue with content farms has to do with their business model, which I think all but ensures the production of low-quality content when they stray from generating simple, straightforward tutorials. But it doesn’t bother me that they comb through search terms or make use of algorithms that point out potential story ideas. Why would it? Those tools work for any newsroom, and using them will help us serve our readers better.

More on ‘Content Farms’

I’m quoted in Dylan Stableford’s very good piece about Demand Media and other so-called content farms over at TheWrap. (There’s some very smart stuff in the comments as well.)

For those who’ve arrived here because of that article, you can read my first post on Demand Media here, and a more-nuanced take looking at articles it supplied to USA Today here. I also did a roundup of posts about content farms here that may be useful for further reading.

Anyway, a few more thoughts.

I don’t think what Demand and its ilk do is “evil” — “unfortunate” is a better word. And my concern isn’t that companies like Demand and Associated Content will drive down writers’ salaries or that the compensation built into their model is “too low,” whatever “too low” means. That’s unfortunate, sure, but it’s just the pitiless economics of supply and demand at work — there would have been some other actor if Demand had never existed. Rather, what bugs me is the quality of the stuff these companies produce, and what it does to search. (See Daniel Roth’s Wired article for a deeper exploration of that.)

Nor does it bug me that content-farm bosses and writers aren’t journalists, which is a charge you’ll find tossed around here and there. First off, a lot of their writers actually are journalists who are trying to get by in extremely trying times for the profession. Second, I think it’s elitist nonsense to say you need some fancy degree or seal of approval to practice journalism. Third, the captains of the news industry don’t exactly have a glittering record when it comes to figuring out their own business.

Demand and AC produce some helpful articles, particularly step-by-step processes and tutorials. I can never remember how to take a screen shot on my Mac, for instance, and inevitably wind up at the same eHow article reminding me how to do it. That’s valuable information that I’m happy to get from them, and that could complement news organizations’ offerings nicely.

My objection is that when you get beyond tutorials and simple how-tos, the quality of the content produced by Demand, AC and others is mostly poor. (That’s my opinion — your mileage may vary of course. Go look at the USA Today travel tips and draw your own conclusions.) Granted, there’s a lot of poor content out there — but content-farm stuff is specifically shaped and molded to game Google and appear higher in search results, which wastes people’s time. And the real problem, as I told Stableford, is that the business model makes it very difficult to produce good content. There just isn’t time to do it profitably.

A commenter on Stableford’s article raised a good question: “Has anyone read copy with an editor’s eye in small- and medium-sized newspapers? With the exception of the NY Times and the Washington Post (the LA Times can’t even compare these days), most of the stories read as first drafts. They’re poorly written and grossly undersourced. The quality of writing and reporting has gone down the toilet.”

Sadly, this is too often true. But I’d say that’s an unfortunate product of years of cost-cutting and an industry in terrible distress — it’s not supposed to be that way. With the economics of Demand and its ilk, though, it’s the logical outcome.