Here’s my presentation at Columbia Business School’s Transitioned Media conference where I talk about “The Wikipedia Revolution:Crowds, Collaboration, Content and Curation Remaking the News.”

Transitioned Media

The new concept I’m introducing is a new way to look at content and curation, and this graphic attempts to distinguish between roles done by the mainstream media outlets/government, and the “crowd” at large. Hope to followup with a post soon with more details.

Understanding Content and Curation

Today’s WSJ has an article by Julia Angwin and Geoff Fowler: Volunteers Log Off as Wikipedia Ages and the associated Digits blog post. It’s one of the best reported stories so far on the dropoff in numbers in Wikipedia (and it’s not just because they quoted me).

The article taps all the right folks: founder Jimmy Wales; WMF’s Sue Gardner and Frank Schulenberg; WMF board of trustees members Sam Klein and Kat Walsh; and many researchers of the project ranging from Mathias Schindler to Ed Chi. It’s hard to argue the plateau is something that can be dismissed lightly.

Perhaps the greatest fear is that Wikipedia will decline not with a bang, but a whimper. Why? Wikipedia has usually made its big strides from reacting to massive public relations “bangs.” Whether it was the Seigenthaler incident that restricted anonymous editing, or upped the requirement for verifiability and reliable sources, Jimmy Wales has been able to push through tough community changes in reaction to obvious public problems.

The alarming thing about a slow decline in Wikipedia’s quality is that there may be no flashpoint to rally around. A slow, low-level infiltration of spam and non-neutral edits may be occurring that the shrinking community may not be able to police.

At SXSW 2010, I’ll be doing a solo talk on this exact topic: “Can Wikipedia Survive Popular Success and Community Decline?” I welcome any and all theories related to this question, either in email or as comments to this post.

Here is an extended video interview I did with the Journal’s Angwin about this.

On August 1, my wife Mei and I boarded a plane in Beijing. We shook the dust off our shoes (literally), took the last bites of some of the best cuisine in the world, and made the big move back to the United States.

Our new home: Venice, California, and a nice modern bungalow near the beach where the sun shines bright and the sea air welcomes you each morning with a cool relaxing breeze. I’ll be taking up a new post as associate professor and director of new media the University of Southern California’s Annenberg School for Journalism.

I’ve always said I like to go where “interesting problems” are. For the last six years in Asia, I got to observe the growth and impact of the Internet in the region, especially while living in China. I also had the chance to write a book, the book, about Wikipedia while traveling the world to meet enthusiastic volunteers contributing their slice of human knowledge for free. It’s been a great ride.

As a journalist and educator, right now, there is no more compelling place to be than the US where the news industry is facing an incredibly tough challenge as it looks for ways to survive an era of digital economics with Craigslist, hyperlocal content, and rapidly changing subscriber and revenue models. It will require smart analysis, enthusiastic practitioners and a unique new role for the academy to help determine how the news industry remains solvent and relevant in the 21st century. I’m thrilled to be with USC to help figure out that future. It’s a city I never imagined I’d live in, but have been impressed and excited at every turn.

It’s July 1, and in China the ominous deadline to implement the Green Dam/Youth Escort internet filtering software has been postponed, to much rejoicing by Internet users in the country.

To outsiders, this must seem quite puzzling. Why would China’s “totalitarian” system need to back down on this?

This should be seen as a case study on how the complexities of China’s decision system is much more nuanced than what a “Communist” regime would suggest, and the role of citizen deliberation in a new, upwardly mobile, aspirational, IT-savvy China.

While the outside world sees the PRC government in absolute control, in reality the heavy handed, top down authoritarian system rides on a delicate balance of, bottom up public consent that supports the state’s legitimacy.

Here’s why Green Dam illustrates this quite well.

China’s Internet filtering is by far the most advanced in the world in terms of precision and scale. But until now, it happened in the “cloud,” in far off intangible spaces through two main vehicles:

• One is through massive domestic Web site content regulation through revokable Internet Content Provider licenses (ICP). Operators have to self-censor through technical or human means to please the authorities regarding general guidelines on taboo topics. Keywords are banned and discussion topics are forbidden. In some cases, explicit timely edicts are required, such as for significant June anniversaries, sensitive political meetings (People’s Congress) or poor construction standards in Sichuan earthquake zones. Even with these, China’s netizens have come up with clever tricks and puns to get around many of these automated filtering systems.

• The other is the Great Firewall, the blocking of what foreign Web sites China users can surf. The implementation is clever, in that restrictions show up as technical errors (connection reset, site not found/unreachable) and curb behavior through uncertainty and doubt about a site’s reach-ability, rather than fear. You don’t know whether it’s the Internet acting flaky, or whether a site is actually being filtered. Tech-savvy users can trivially circumvent this.

But you don’t need perfect censorship to have effective censorship. Both these systems do quite well for the PRC government in keeping the 3T1F topics outside the mainstream, and ensuring that the government is not embarrassed by reporting on its incompetence.

The key, here is that both the domestic and international filtering activities happened in the cloud, the ether, the machines that comprise the Internet. It wasn’t in your home and it didn’t intrude beyond the cable to your desk.

Green Dam suddenly put the specter of restriction, surveillance and control in your home.

With that one stroke, which probably seemed like the next logical innocuous extension of the censorship regime for PRC bureaucrats, the government took the big miscalculation of crossing into the the private space, and the personal property of China’s citizens. And that’s where the outrage came.

This was the camel’s nose into the private tent of Internet users. A poll on China’s major sites (Sina, Netease, et al) all showed over 3/4 of respondents said Green Dam was not necessary or a bad idea.

(NB: China is not the first or the only government wanting to censor Internet traffic for content. Australia’s Clean Feed proposal to covertly filter out sites at the ISP level has been under fire from their netizens, and was unceremoniously put on hiatus as well. Most public schools and libraries in the United States implement content filtering at some level. This is not a uniquely China issue.)

What the authorities in China didn’t realize was how serious that breach of boundary would be.

I knew it was going to be a tough road for Green Dam when it appeared the MIIT initiative was not a unified effort. Before leaving for my travels, I did commentaries with the Associated Press, Deutsche Welle, Al Jazeera and others, making the point that even China’s official news outlets were openly questioning Green Dam’s legitimacy. The new Global Times newspaper, which has been rather frank about other issues, led off with serious questions about the software’s safety.

Then came the big one.

China Daily, the official mouthpiece of the government, was publishing criticisms of Green Dam shortly after it was announced, even publishing Photoshop’ed illustrations of netizens mocking the system. (”Outrage over bid to tame Web“, China Daily, June 18, 2009)

One picture it included with the article was a “Who Wants to be a Millionaire?” multiple choice question describing Green Dam as “spyware” with “systemic flaws” that could be “exploited by hackers.” Another cartoon shows a gray hand of censorship coming from the computer screen and stiff-arming a computer user in the face.

It was clear at this point, the Green Dam initiative was from a smaller portion of the PRC bureaucracy, and not from the highest levels. China Daily would have never published something so critical if it was of the highest-level of agenda pushing.

China’s netizens were speaking, and the media and government were taking notice (and with higher ups looking the other way). So while this was not democracy in action, it certainly was something in action.

At TEDxShanghai last month, I described the phenomenon of Wikipedia and Twitter forming the basis of a new online commons where global netizens come to share and reinforce memes across geographic and social boundaries (SlideShare presentation). For years, enthusiastic faith-based technology enthusiasts hoped the Internet would bring democracy to any place it touched. This has been spectacularly elusive. On the flipside, some viewed the new Web 2.0 social revolution as “socialist”, “collectivist” and at worst, Maoist. That’s been inaccurate as well.

Instead, I describe the new borderless, socially agile, activist associations that crop up on the Internet as a new system of ‘deliberative adhocracy’. Alvin Toffler, and later Cory Doctorow, used adhocracy to describe a new form of rule based ephemeral associations that “capture opportunities, solve problems, and get results.” (Waterman)

Whether it’s as massive as #IranElection to bring global awareness to its politics, or as small as #MotrinMoms to discuss outrage at an insulting advertisement, we now have an online information commons (Twitter) and knowledge commons (Wikipedia) that supports a space for the new distributed Zeitgeist. In China, obviously there are other analogs (Twitter clone Fanfou, Baidu Baike, BBS forums, et al.) but the effect is the same. To see deliberative adhocracy in action look no further than the Human Flesh Search Engine that metes out social justice in the absence of a strong rule of law in China.

Readers familiar with my book will know I described how a Wikipedia Revolution changed forever how we deal with free access to knowledge and its production. I will however, be quite Burke-ian in my pronouncement about the Internet’s effect on China.

Revolutions are sudden overthrows and disruptive repudiations of the status quo. China has a terrible modern history with revolutions, with more of them going bad than good. The rule law is sometimes described as when “reason trumps politics.” To China’s authorities, the Internet is being used in a deliberative process that fulfills that role. It is not perfect, nor prevalent enough to ensure social justice on a large scale. However, it is a huge step forward for a country that is convinced that after a century of turmoil, that any step must take safety and efficiency into account.

The hiatus for Green Dam, is the standard face-saving way for the government to back down. There is a good possibility it may come back in another form, watered down or otherwise. But for now, China’s netizens are having their day.

“Scientists have more faith in Wikipedia than national print media”

That’s one of the takeaways from a recent poll of nearly 1000 toxicologists when they were asked  how various media outlets cover their specialty: the representation to the public of chemical risks. (The poll was conducted by STATS, The Center for Health and Risk Communication at George Mason University, and the Society of Toxicology)

Given the common lament that Wikipedia shuns “experts,” and information is produced by people “off the street,” the results are intriguing when you look at the numbers for other professional and “mainstream” media outlets. From the report synopsis:

WebMD and Wikipedia were seen as significantly more accurate in the way they presented chemical risk than any other media source.

·         56% say WebMD accurately portrays chemical risks

·         45% say Wikipedia accurately portrays chemical risks

·         By contrast, no more than 15% say that leading national newspapers, news magazines, and television networks accurately portray chemical risks.

·         Over 80% say that leading national newspapers, news magazines, and television networks overstate chemical risks

[...]

…only 15 percent described similar coverage in the national print media (i.e., the New York Times, Washington Post, and Wall Street Journal) as accurate. This figure dropped to 6 percent for USA Today and 5 percent for broadcast network news.

At a press conference at the National Press Club to release the preliminary results of the study, Dr. S. Robert Lichter, who conducted the survey described the Wikipedia finding as an indictment of the mainstream media – ” it’s disturbing that someone off the street seemingly can do a better job than the media.”

I’d take issue with the fact that Wikipedia is simply the product of random person “off the street,” but it is a real shift in what we consider authority and how reliable information can be produced.

Even the best performer, WebMD, gained the approval of only about half the toxicologists who were surveyed which should be a bit surprising in itself. My (full disclosure: unpaid, uncompensated) commentary as it appeared in the report:

“This reminds me of the Nature study [link] that was done in December 2005 where it found that on average, Britannica had 3 errors per article, and Wikipedia had 4 errors,” Lih says by email. “It was surprising because Wikipedia did much better than expected, given its foreign work process and Britannica did much worse. People had presumed a certain level of accuracy from Britannica’s reputation, and it was knocked down from that pedestal. To me the WebMD and Wikipedia results here are similar – they’re much closer than what one would expect. Wikipedia doing better, WebMD doing worse.”

But perhaps the most interesting part was not WebMD, but that the daily professional print media came up so short in the eyes of these experts. It seems to reinforce the old adage: “Journalists do a pretty good job of covering things, except for subjects in which you’re knowledgeable.”

The commentary for Columbia Journalism Review contributor Alissa Quart was insightful about why the MSM approach (reporting science as a storyteller for the masses) is perhaps systemically flawed:

“Journalists fall into storylines, because that’s how we write. There are three narratives, that we use, which can make us great but also get us into trouble – one narrative to please our editors, one to please our readers, and one which leans toward our sources, because we identify with them. WebMD and Wikipedia contributors are disconnected from most of those narratives – maybe they are trying to please certain readers, but they aren’t ‘the reader.’ Their model of knowledge doesn’t ask for stories, or sentiment or people.

This is a really good observation that meshed well with my views about the role of public relations and the dangerous media narrative driving scientific reporting. Quart and I arrived at the same conclusion.

In short, argument trumps aesthetics. Lih, an engineer by education, concurs. The clash of narratives “also says something about motivation, in that the mainstream press will be driven by reports, PR bring shoved at them, and also the market and the desire by editors (in a top-down manner) to demand reporters find a story in the latest research, even if in the greater context of the field, it doesn’t warrant so much attention. In that sense, Wikipedia’s motivations are different, in that the ‘crowd’ helps moderate and even dampen the type of ‘recentism’ that is so pervasive in news coverage.”

The overall summary can be found at the Stats.org site, or you can view the full PDF.

Interesting first day at South by Southwest conference.

It started with a panel “User Generated Content: State of the Union” where I asked whether the term UGC was still useful, as we have better more specific terms. I relayed the fact that Wikipedians (specifically, Brianna Laugher of Wikimedia Commons) have come up with the term “community curated content,” as a more meaningful phrase to describe what Wikipedians do. Chris Tolles, Topix CEO, who was moderating, asked whether people knew what that meant. Not many raised their hands. But later on in the Q&A, someone described their company as doing “curated video” and Steve Rosenbaum, on old friend from NYC I ran into, described a trend towards using the term “curation.”

The next session, an excellent talk by author Stephen Johnson, talked about the future of news. On his diagram of the news ecosystem, he described News and Commentary functions, being mediated by a Curation layer, and feeding the Distribution of news. Johnson says this curation is done by various sources: social media groups, professional editors, aggregators, group filters.

I see it as more than a coincidence that the term “curation” is a word being used now.

This is a smart crowd at SXSW. I’m glad to see more accurate, nuanced and thoughtful terminology being used to describe the functions within the Web 2.0 community.

UPDATE: I forgot that on day two of the conference, the curation theme continues, with a session titled: Curating the Crowd-Sourced World: “With all the stuff we weed through online, good filters are crucial. Who’s best-suited to determine what’s best, curators or the crowd? People have their religion about one or the other, however this panel will focus on the overlap, the grey areas and how curating and crowd-sourcing enrich each other.”

UPDATE 2: The Guardian’s Jemima Kiss blogged about this too, but just a slight error in referring to it as “crowd-curated works”

Bloggers vs. journalists. It’s a debate that still continues in the media circles, with only the most cursory examples used to argue the case. So the next time the pajama-clad scribes are cast as the evildoers ruining professional news careers, point them to this post on DailyKos as evidence of how these two sides work together.

Diarist “lollydee” relays a tip from a friend who is a Bank of America human resources employee. The news was “crappy.” There were  layoffs at the bank just weeks after a “three year protection package” expired for many employees, meaning no severance. It has not not been reported by the local or national “MSM”.

The writer reports only what is directly reported from the friend, but it’s important inside knowledge. In the comments section of the busy DailyKos site, others chime in to help triangulate with other data points. Some confirm the story, others point out some clarifications based onother reports. Lollydee posts them all as updates, and a narrative starts to form. But in the end, he/she tosses it to the “real” reporters. Addressing those sending tip offs in the comments:

PLEASE contact your local media, or at least one of the front pagers on this website.  I’m not the MSM.  I’m just an individual diarist/blogger and because I’m unfamiliar with the legality involved in posting certain types of information, I’m not comfortable going beyond this commentary.

I’m not a reporter…

I’ve done this once before, and I like to think I’m giving leads to those REAL reporters out there.

And, because I’m a good community citizen, when I get an updated fact I post it.  When I have a mainstream link, I post it.

Even though their work at the grassroots level takes place in the primordial soup of journalism (proto-news, if you will), bloggers are not out to negate the job of professional journalists. The best bloggers know where their own limits end and where responsibility begins.

Journalism academics and professionals need to understand this dynamic more, rather than engaging in the same simplistic CNN Crossfire-style bloggers vs. journalists debate.

The job of a journalist is hard. New subjects crop up each day, and the task by the deadline is to demystify a topic for the general public. A common technique is to use familiar markers to interpret new ones. Give the reader something they know to help understand things they don’t: “The KitchenAid stand mixer is the Cadillac of cooking equipment,” or “The new Blackberry Curve is the answer to the iPhone.”

But the seduction of this technique poses some serious problems.

This has been the case with Google’s new offering called Knol, the so-called “Wikipedia rival,” which is meant to “highlight authors” creating user-generated content.

The recent general reporting around Google Knol has been rather atrocious. For the lack of any better metaphors, most journalists (including professional “tech journalists”) saw the “user created” part of Knol and lacking any significant understanding about either project, immediately labeled it a “Wikipedia rival.” A quick Yahoo News and Google News search sees an overwhelming number of headlines trumpeting Knol as this Wikipedia “rival” or Google’s “answer” to the free encyclopedia.

But besides simplistically sharing a “user generated” Web 2.0 pedigree, the comparison is flawed in so many ways.

The head-to-head matchup seems obvious because Wikipedia is the only thing that immediately comes to mind to most writers when thinking “user generated.” I contend it’s lazy journalism, where respected tech outlets also fell for this trap. It seems it was too tempting to stick by the story, portraying nonprofit David competing against corporate search king Goliath.

The result, is we’re stuck with this fallacious “media narrative.” And from now on, Knol will always be seen as Wikipedia’s foil. Even when it’s not.

Have you heard of Associated Content, Squidoo, Helium or WikiHow? No? If you haven’t, you shouldn’t be writing about Google Knol. These are exactly the working models that Google Knol is up against, not Wikipedia’s.

To see exactly why Wikipedia is such a bad comparison, consider the main aspects of Google Knol [1] [2]:

• Goal: “first thing someone who searches for this topic for the first time will want to read”
• Articles are controlled by a single author, who has to use a real name.
• Collaboration: at the discretion of the lead author aka “moderated collaboration”
• Opinions are allowed and encouraged in articles, and there can be competing articles about the same subject.
• Knol may include ads at the discretion of the author, and profits shared
• Licensing of content is varied: can be CC-BY, CC-BY noncommercial, or traditional copyright
• “Google will not serve as an editor in any way”
• “So what subjects can I write on? (Almost) anything you like. You pick the subject and write it the way you see fit.”

As a result, most of the content that has emerged so far resemble the “practical” content sites as listed above:
how-to guides, health and medical advice, consumer/buyers guides, business/career pointers. These are exactly the things Wikipedia has insisted it does not want to be.

Don’t take my word for it, see the guidelines at [[What_Wikipedia_is_not]]

• Wikipedia is not a manual, guidebook, or textbook…
  • A Wikipedia article should not read like a how-to style manual of instructions, advice (legal, medical, or otherwise) or suggestions, or contain how-tos. This includes tutorials, walk-throughs, instruction manuals, game guides, and recipes.[4] If you are interested in a how-to style manual, you may want to look at wikiHow or our sister project Wikibooks.

The guidelines of Wikipedia also prohibit: personal essays, advocacy, opinion pieces on current affairs or politics, scandal mongering or gossip columns and self promotion.

Although Knol has been touted as a more responsible, moderated Wikipedia the above policies actually makes Knol much more liberal and uncontrolled than Wikipedia, which has many guidelines about what constitutes an article, what is acceptable content, and how to abide by its neutral point of view policy.

By throttling collaboration through a single lead author, you lose what has been Wikipedia’s hallmaark — the “piranha effect” of people building off each others’ work and evolving content beyond a single author’s knowledge.

Let’s put these criteria up side by side again:

Let me pull out my journalism professor’s ruler and whap all the tech journalists on the wrist who have used this comparison. Let’s please stop pitting these two against each other.
From my feed of Yahoo News reporting on Wikipedia and Knol, here are my three tier ratings of how folks did on this story. I will not even bother hyperlinking to the stories I considered faulty analysis.
Poor

• Google launches Wikipedia rival Knol (ZDNet UK)
• Google Launches Its Challenge To Wikipedia With Wide Release Of Knol (paidContent.org via Yahoo! Finance)
• Google’s Wikipedia rival, Knol, goes public (CNET)
• Google Launches Its Challenge To Wikipedia With Wide Release Of Knol (CBS News)
• Google launches Wikipedia rival (IT World)
• Knol: Google Takes on Wikipedia (ReadWriteWeb)
• Knol, Google’s Version of Wikipedia, Goes Public (PC Magazine)
• Knol: (n.) Google’s version of Wikipedia (BetaNews)
• Google infiltrates the knowledge sharing game (SiliconIndia)
• Google unveils Wikipedia-like tool (Australian IT)

Average

• Google’s Knol Launches: Like Wikipedia, With Moderation (Search Engine Land)
• Wikipedia, Meet Knol (New York Times)
• Google Launches Knol, The Monetizable Wikipedia (TechCrunch)
• Google Makes Knol Publically Available (EContent Magazine)

Good

• Google Knol Launches; More Like Squidoo than Wikipedia (Mashable)
• Google Decides to Make its Knol Site a Moderated Wikipedia (Search Engine Journal)
• Knol is no Wiki (techkeyla.com)

And for those still debating the quality of “blogging vs journalism” issue, consider all the best sources for reporting on the Knol launch are, yes, blogs.

Earlier today on Twitter I commented about the “insanity” that is Robert Scoble following 21,000+ people on the group messaging and microblogging service. Since his bot (software robot) monitors everything on Twitter for mentions of his name, he saw my comment and challenged me.

Scobleizer @fuzheado thinks he knows the point of Twitter and says I don’t. This might be interesting. Might.

Twitter limits you to 140 characters per post, so I had to be succinct:

Twitter is a modern digital commons – nonhierarchical, transparent, open, human-speed. Once bots inhabit it, tragedy perhaps.

Earlier that day I was thinking about what Twitter “was” before the Scoble tweet.

Twitter’s model is simple but powerful — complete transparency. Anyone in the Twitterverse can see what you’re receiving, who you are following and who’s following you. It creates a continually changing set of readers and writers, allowing peer discovery faster than any other SNS. Some other features help.

“Retweeting” an interesting post to your followers effectively bridges two disconnected cliques. The directed “@user” messages send an exploratory “Tweet” to make contact with new peers. It’s great in its simplicity, and the Twitter API furthers extendibility and usability.

The Internet had a lot of naysayers in the 1990s — people complained it was a peer-to-peer system that created ghettos. Terms like narrowcasting, personalized media, customized front pages and the Daily Me all implied compartmentalized lonely existences. Twitter does the opposite, by embracing radical transparency to support serendipitous discovery and social mixing. Perhaps it’s why a number of Twitterers have been noticing they spend a lot less time on Facebook. They’ve hit a wall with their “trusted” network of friends. They’ve tagged photos, thrown sheep, played 100 games of Scrabulous and reunited with old classmates. But they didn’t really broaden their horizons.

For English-language China-oriented bloggers, Twitter has fostered a nimble community whether it’s about Shanghai, Beijing, Guangzhou, Hong Kong, business, arts, pop culture, academic, Olympics, you name it. Suddenly a disparate set of folks are discovering each other, trading 140-character info nuggets faster than ever before. And with recent Olympic torch dramas and Carrefour boycotts, it’s been the only info stream that can keep up with breaking events.

Elliott Ng at CNReviews.com has a metric-heavy analysis of who’s who in this sphere, based on the list created by Christine Lu (the ultimate “connector” in Tipping Point parlance). It’s like a celebrity who’s in and who’s out list.

But as Twitter grows, there is the risk the signal may not keep up with the noise. Spam, bots and scalability are always a problem to new digital commons spaces. There is a very good chance Twitter, a “faddish web app“, could be the CB radio of Web 2.0 if it can’t find a way to scale with its new-found fame.

Let’s hope we don’t refer to it as the PointCast of 2008 and wake up with a bad hangover.

Kaiser Kuo has a good writeup on his Twitter conversion. For China-based users it’s been a particularly useful application the last few weeks:

Pick the right folks to follow and there’s real value: They link to interesting reads — this is to me probably the most useful thing about Twitter — and make trenchant, sometimes insightful comments. During the recent troubles in Western China, I was following Twitter feeds from people on the scene, providing first-hand perspective that was nearly impossible to find in the press.

This is a great example of the power of citizen blogging/microblogging being not just a frivolous act (ie. tweeting: “Sitting here watching paint dry”)

With the rising tensions over the Olympic torch relay in Europe, the boycott of Carrefour, the roughing up of an American English teacher by a mob in Hunan and the takedown of Web sites by pro-China hackers, Twitter has been ahead of the curve by assembling an ad hoc community of folks across different cities, pointing to blog posts, first hand accounts from the ground and  BBS postings reflecting local sentiment. It’s something that all China-oriented reporters should check out and experience first-hand.

I’ll be addressing the Hong Kong Journalists Association next week about China’s Internet and will absolutely talk about Twitter and its China-based counterparts.