Augmented Social Cognition @ PARC, Models Of Authorship And Governance Options Rating

[Jon Awbrey]

So what gives? Is there a right way or wrong way to constructing and compiling knowledge resources? As designers of social systems, what should be the governance model for these systems?

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

These are such important questions — and so resonant with the questions of inquiry's activity in nature and society that have occupied me for umpteen years now — that I'm going to experiment with a somewhat more disciplined style of discussion than we usually engage in here.

Jon Awbrey

[Somey]

Sounds rather ominous! (IMG:smilys0b23ax56/default/ohmy.gif)

Having read that, it seems clear that Mr. Chi here is approaching this from an almost entirely ergonomic perspective, as opposed to an ethical-humanist or societal-impact perspective. That would be in keeping with his professional focus, of course (PARC has been a leader in computing ergonomics for a long time, as is well known).

That isn't to say he's missing the point, but this bit strikes me as the key question he's interested in:

So now the research question is whether you want to design your editing policy to favor the upper class (top editors and administrators), the middle class (the 5000-6000 editors who contribute the middle 50% of all edits), or the lower class (the 15000 editors who contribute the last 25%).

One way to think about this problem is to study the amount of resistance each of these four classes of editors experience on Wikipedia. A metric that we used is the reverts-to-edits ratio. That is, on average, what percentage of edits were reverted, as experienced by each of these four classes of editors? Turns out that the reverts-to-edits ratio for each of these 4 classes of editors were 1.3%, 1.4%, 1.5%, and 4.7%, respectively. Meaning that the lower class of editors clearly experience greater resistance, such that, on average, 1 out of every 20 edits they contribute are reverted.

For good or ill, hardly anyone on Wikipedia thinks in these terms, and the vast majority of them believe that the same rules should apply to everyone, including admins, even Jimbo himself in many cases. That's not necessarily bad, of course, but the real issue for them is how do they define the term "everyone."

In other words, it's obvious now that a clear majority of WP'ers agree that the higher rates of "disruption" and "tendentious editing behavior" by AnonIP's makes it acceptable to treat them as a different, less-privileged class of editor than the "community" of named users. This should have been obvious to all of them from the start, but old egalitarian dogmas die hard. The really unfortunate thing about this isn't that AnonIP's will be treated differently now; the unfortunate thing is that they waited so long to do it, it's now much harder to implement this because of the fear of what's considered negative publicity - most of which is likely to come from the tech media, which is what many of them read almost exclusively.

Moreover, I would think that quite a few Wikipedia admins and "power users," having looked at those numbers, would not think, "gosh, there's a significant disparity there." Instead, their reaction would be, "how is it that admins are getting a whopping 1.3 percent of their edits reverted? And who's doing these reverts? And can they be banned summarily"? And so on.

[Jon Awbrey]

Sounds rather ominous! (IMG:smilys0b23ax56/default/ohmy.gif)

Having read that, it seems clear that Mr. Chi here is approaching this from an almost entirely ergonomic perspective, as opposed to an ethical-humanist or societal-impact perspective. That would be in keeping with his professional focus, of course (PARC has been a leader in computing ergonomics for a long time, as is well known).

That isn't to say he's missing the point, but this bit strikes me as the key question he's interested in:

So now the research question is whether you want to design your editing policy to favor the upper class (top editors and administrators), the middle class (the 5000–6000 editors who contribute the middle 50% of all edits), or the lower class (the 15000 editors who contribute the last 25%).

One way to think about this problem is to study the amount of resistance each of these four classes of editors experience on Wikipedia. A metric that we used is the reverts-to-edits ratio. That is, on average, what percentage of edits were reverted, as experienced by each of these four classes of editors? Turns out that the reverts-to-edits ratio for each of these 4 classes of editors were 1.3%, 1.4%, 1.5%, and 4.7%, respectively. Meaning that the lower class of editors clearly experience greater resistance, such that, on average, 1 out of every 20 edits they contribute are reverted.

For good or ill, hardly anyone on Wikipedia thinks in these terms, and the vast majority of them believe that the same rules should apply to everyone, including admins, even Jimbo himself in many cases. That's not necessarily bad, of course, but the real issue for them is how do they define the term "everyone."

In other words, it's obvious now that a clear majority of WP'ers agree that the higher rates of "disruption" and "tendentious editing behavior" by AnonIP's makes it acceptable to treat them as a different, less-privileged class of editor than the "community" of named users. This should have been obvious to all of them from the start, but old egalitarian dogmas die hard. The really unfortunate thing about this isn't that AnonIP's will be treated differently now; the unfortunate thing is that they waited so long to do it, it's now much harder to implement this because of the fear of what's considered negative publicity — most of which is likely to come from the tech media, which is what many of them read almost exclusively.

Moreover, I would think that quite a few Wikipedia admins and "power users," having looked at those numbers, would not think, "gosh, there's a significant disparity there." Instead, their reaction would be, "how is it that admins are getting a whopping 1.3 percent of their edits reverted? And who's doing these reverts? And can they be banned summarily"? And so on.

Actually, I was thinking of inviting Ed Chi et al. here to discuss the state of their research — as I have long been interested in the more general context of questions they are tackling — but I thought I'd let the Earth take another spin before doing that. Then again, maybe inviting folks over for a Valentine's Day Party would sound even more ominous …

Jon (IMG:http://wikipediareview.com/stimg9x0b4fsr2/1/folder_post_icons/icon9.gif)

[GlassBeadGame]

I think it would be very good, Jon, if you extended the invitation. I'm very happy to see PARC formulating research questions about WP's governance, even if the question are not exactly the ones that most interest us, or at least me. I wouldn't expect the PARC folks to accept most of our views but the interaction would likely result in a wider perspective on what constitutes significant research questions.

I'm also happy to hear about the more disciplined approach. I think if they respond to the invitation you should keep the discussion in your forum and use as much latitude as you feel is needed to facilitate the discussion, even if this is somewhat more restrictive than the usual WR standards.

[Jon Awbrey]

I think it would be very good, Jon, if you extended the invitation. I'm very happy to see PARC formulating research questions about WP's governance, even if the question are not exactly the ones that most interest us, or at least me. I wouldn't expect the PARC folks to accept most of our views but the interaction would likely result in a wider perspective on what constitutes significant research questions.

I'm also happy to hear about the more disciplined approach. I think if they respond to the invitation you should keep the discussion in your forum and use as much latitude as you feel is needed to facilitate the discussion, even if this is somewhat more restrictive than the usual WR standards.

I posted a message to the Augmented Social Cognition Blog inviting them over.

The thing I like about their overall perspective is that they are asking the Big Questions about how we use technology in social contexts to address the problems of "constructing and compiling knowledge resources". And they seem to be taking a Systems Design approach toward the development of social-technical architectures. This is very compatible with the Peircean way of relating all belief and knowledge to the Community of Inquiry that serves as its Interpreter and treating logic as a normative science concerned with the optimal conduct of inquiry.

Viewed within that Big Picture Frame, any particular enterprise like Britannica or Wikipedia is just one of many alternative projects that give us fodder for comparative critique in terms of how well they achieve the commonly espoused objectives of the larger knowledge enterprise.

So let's try to keep our eyes on the prize …

Jon Awbrey

[Jon Awbrey]

The Prize

So what gives? Is there a right way or wrong way to constructing and compiling knowledge resources? As designers of social systems, what should be the governance model for these systems?

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

More information about the Mission and Vision of the Augmented Social Cognition Group can be found in these recent blog items:

• Ed H. Chi (13 Feb 2009), "WikiDashboard and the Living Laboratory", Augmented Social Cognition Blog.
• Ed H. Chi (05 Nov 2008), "'Living Laboratories' : Rethinking Ecological Designs and Experimentation in Human-Computer Interaction", Augmented Social Cognition Blog.

[Jon Awbrey]

The Prize

So what gives? Is there a right way or wrong way to constructing and compiling knowledge resources? As designers of social systems, what should be the governance model for these systems?

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

Here's a nice summary of the Living Laboratory idea:

The Augmented Social Cognition group have been a proponent of the idea of 'Living Labratory' within PARC. The idea … is that in order to bridge the gulf between academic models of science and practical research, we need to conduct research within laboratories that are situated in the real world. Many of these living laboratories are real platforms and services that researchers would build and maintain, and … would remain somewhat unreliable and experimental, but yet useful and real. The idea is to engage real users in ecological valid situations, while gathering data and building models of social behavior.

Ed H. Chi (05 Nov 2008), "'Living Laboratories' : Rethinking Ecological Designs and Experimentation in Human-Computer Interaction", Augmented Social Cognition Blog.

Time has come to do a great more deal of experimentation in the real world, using real and living laboratories.

Ed H. Chi (13 Feb 2009), "WikiDashboard and the Living Laboratory", Augmented Social Cognition Blog.

[Jon Awbrey]

The Prize

So what gives? Is there a right way or wrong way to constructing and compiling knowledge resources? As designers of social systems, what should be the governance model for these systems?

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

From my own point of view — it being no sin on this Review to own such a thing — the confession that follows the 3rd degree interrogation in evidence above has to be taken as the critical reflection on which the whole success of the Augmented Social Cognition project is likely to turn.

For one thing, we still know awfully little about the social dynamics in these large social systems.

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

My Emphasis.

Jon Awbrey

[Jon Awbrey]

In Search of the Lost Chord

After a good night's reflection

a good night is one you do not go gentle into

the dying of the light is not a sight for a neutral point of view

a good night is one you come gentle out of

I have come to view the agon this way:

Protagony : The Prize

So what gives? Is there a right way or wrong way to constructing and compiling knowledge resources? As designers of social systems, what should be the governance model for these systems?

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

Antagony : The Fight

For one thing, we still know awfully little about the social dynamics in these large social systems.

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

My Subtitles.

Jon Awbrey

[Jon Awbrey]

I have singled out the two notes struck above because they recapitulate in a very telling way the ♪ Objective ♪ that we want to achieve and the ♪ Obstruction ♪ that gets in the way. If it makes us edgy to contemplate the tension between the two, then we should not turn too quickly away, or seize too quickly on simplistic evasions, or else we lose the opportunity to resolve the tension in a creative and genuinely satisfying way.

♫ Jon Awbrey ♫

[Jon Awbrey]

For one thing, we still know awfully little about the social dynamics in these large social systems.

Ed H. Chi (26 Jan 2009), "Governing and Authorship Models at Wikipedia and Britannica", Augmented Social Cognition Blog.

The question then arises:

• Why is it that "we still know awfully little about the social dynamics in these large social systems"?

Jon Awbrey

[Jon Awbrey]

I've been chatting quite a bit with Ed Chi on the Augmented Social Cognition Blog, doing my best to stay focused on the Big Tapestry questions that I've highlighted on this thread. The more distance I get from the warpfield of Wikipedia, the more I think I'm beginning to get a handle on the underlying psycho-social-technical dynamics in these net-web-wiki media.

Jon

[Jon Awbrey]

I see there's a few new blogicles at the Augmented Social Cognition Blog, but I haven't had a chance to read them in depth. Just off the cuff, however, they seem as clueless as ever about the heart of dimness in Wikipedia.

Jon (IMG:smilys0b23ax56/default/dry.gif)

[Jon Awbrey]

Re: "What if we can design an expertise finding system that recommends the best articles for you to contribute to in Wikipedia? Would it increase participation rates?"

Ed,

This is so clueless that it boggles the imagination.

You have clearly not spent a lot of time actually participating in Wikipedia.

Do you have any idea how many "expertise enhancing efforts" have gone already gone down in flames on Wikipedian shores?

You need to put aside your wiki-pipe-dreams, spend a little more time talking to people outside the Wikimediocracy, and try to grasp the real dynamics of what goes on in Wikipedia.

Jon Awbrey, 11 Mar 2010, 8:08 AM

I'm afraid I allowed myself to get a little exasperated — I probably should've gone for my walk in the park before commentating.

Still, it's all true …

Jon (IMG:http://wikipediareview.com/stimg9x0b4fsr2/1/folder_post_icons/icon9.gif)

[thekohser]

I added my own comment to Ed Chi...

Ed, is it a scientific process if you have an unwavering viewpoint that the subject matter (in this case, Wikipedia) is inherently a wonderful phenomenon and couldn't possibly be wrongly governed or nefariously exploited?

My viewpoint is that you are studying something like malaria, but you (as the scientist) have chosen an agenda that takes the side of the eukaryotic Plasmodium protist. What a tragedy if this beautiful parasite were to become extinct, you seem to be saying.

[Milton Roe]

Re: "What if we can design an expertise finding system that recommends the best articles for you to contribute to in Wikipedia? Would it increase participation rates?"

Ed,

This is so clueless that it boggles the imagination.

You have clearly not spent a lot of time actually participating in Wikipedia.

Do you have any idea how many "expertise enhancing efforts" have gone already gone down in flames on Wikipedian shores?

You need to put aside your wiki-pipe-dreams, spend a little more time talking to people outside the Wikimediocracy, and try to grasp the real dynamics of what goes on in Wikipedia.

Jon Awbrey, 11 Mar 2010, 8:08 AM

I'm afraid I allowed myself to get a little exasperated — I probably should've gone for my walk in the park before commentating.

Still, it's all true …

Jon (IMG:http://wikipediareview.com/stimg9x0b4fsr2/1/folder_post_icons/icon9.gif)

Chi seems to have a bad case of nerditis. You can always tell a true nerd by their aversion to ever actually tackling any project directly. Instead, they'll suggest tackling it at the next level up.

You say, "Gee, we need somebody to invent a [system/ device/ program] that does X or Y."

Nerd: "No, no, what we need is for somebody to invent an automatic machine or computer that will INVENT systems and devices that do X and Y. Then you just tell them what want, and voila, they do it automatically."

(IMG:smilys0b23ax56/default/ermm.gif)

I think actually what we really need is a machine that does the following: anytime you need a job done, you feed the job-description into the machine, and out comes a suggestion that we need a machine that do that job, and perhaps even a machine that invents machines that can do that job...

You see-- you can replace nerd suggestions with an algorithm. So why do we even need nerds at all? What is needed is a nerd-replacement machine. (IMG:smilys0b23ax56/default/wink.gif)

[Malleus]

I added my own comment to Ed Chi...

Ed, is it a scientific process if you have an unwavering viewpoint that the subject matter (in this case, Wikipedia) is inherently a wonderful phenomenon and couldn't possibly be wrongly governed or nefariously exploited?

My viewpoint is that you are studying something like malaria, but you (as the scientist) have chosen an agenda that takes the side of the eukaryotic Plasmodium protist. What a tragedy if this beautiful parasite were to become extinct, you seem to be saying.

I have rarely seen such pseudoscientic intellectual claptrap as is on display here. There is no "scientific process", there is a scientific method, one that demands testable hypotheses. Your viewpoint is neither here nor there, except insofar as it's a manifestation of your own prejudice. Same goes for Awbrey of course, in spades.

Ja, Ja.

[Somey]

To the best of our knowledge Mr. Chi actually did register a WR account as edchi, but as of now he hasn't posted anything. Sometimes these academic types get to thinking we're all too unruly and end up being scared off, but I think there's still reason to hope in his case.

My own experience with "recommendation engines," at least at the consumer level, has been fairly consistent so far - I'm not impressed by any of them yet. The one I'm exposed to the most, that of eMusic.com, is terrible. The ones for Netflix and iTunes aren't much better, and this is after Netflix put up $1 million in prize money for an improved algorithm. Amazon.com probably has the best one of the major music/movie sites I actually use, but that's not saying much, and they probably have the largest database for mining purposes. (Last.fm seems to be pretty good too, but AFAIK that's almost entirely human-driven, isn't it? I don't go there much.)

I'm not saying the technology won't improve (I'm sure it will), and to some extent my tastes in both movies and music tend to be non-mainstream and therefore (I assume) harder to work out automatically. But it does strike me that music/movies are fairly basic-simplistic applications in comparison to something like what Mr. Chi is suggesting for Wikipedia.

It might be a good idea **IF** they can do it in such a way as to not involve the entrenched WP user community, but without them they might not get widespread adoption, because they won't be allowed to integrate it into WP itself, at least not immediately.

[Malleus]

There is already a "recommendation engine" on wikipedia, but its output is to say the least bizarre. At least judging from the couple of times I tried it.

[Jon Awbrey]

Ed Chi's reply

My follow-up —

Let me rephrase your suggestion:

"What if we can design an expertise finding system that recommends the best articles for you to contribute to in [Ideal System X]?"

That, of course, is brilliant idea, and well worth trying somewhere or other, almost anywhere but Wikipedia.

The catch is that there is already a culture in place at Wikipedia, a culture that is antithetical to every condition that it would take to make such a utility work.

This is true of every attempt to bring Wikipedia up to par with the intellectual virtues usually associated with responsible scholarship and journalism, and automation would not make the slightest difference, since you cannot implement disciplined routines without the consent of the mob.

You could try to follow the 5-year debacle over "flagged revisions" if you wanted an ever-ongoing example.

Jon Awbrey, 11 Mar 2010, 11:14 AM