Content contributors, statistical analysis Options

[Peter Damian]

My blog post for today http://ocham.blogspot.com/2011/10/repetiti...-wikipedia.html on whether there are statistically measurable properties that distinguish 'content contributors' from wiki-gnomes. Conclusion: the statistical difference is strongly indicative of a real difference, discussed in detail on the blog.

Remaining questions: why do content contributors remain on the project, given that they have a lower status than those who perform repetitive and tedious work?

Easily-learned repetitive labour is nearly always paid less in real life than labour which requires either specialised learning, or some innate but scarce skill. The simple reason for this is supply and demand. Rare or difficult-to-acquire skills are by definition in short supply, and will attract a higher price than common, easily acquired skills (at least, to my simple mind - I don't know any economics).

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

This post has been edited by Peter Damian:

[Silver seren]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

[Peter Damian]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

Yes of course there are a 101 ways in which this number could fail to have the meaning it may have. But then Giano tends to edit in his own space in the way you describe, yet he has one of the highest epp's.

All we can say, and all we need to say is that:

1. In general, editors with low epp's tend to perform relatively mechanical low economic value easily learned tasks. We can verify this by looking at their actual contributions. Editors with high epp's tend to be those with lots of FA and GA stars on their page, and who are generally and anecdotally known as so-called content contributors. That proves there is a division of labour in Wikipedia.

2. Low epp's predominate in the admin corps. Hardly surprising, given that the qualities required of an admin are precisely low-value, repetitive tasks, and given that RfA tends to emphasise quantity rather than quality of edits.

3. The theory of crowdsourcing says that this shouldn't happen.

[radek]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

Yes of course there are a 101 ways in which this number could fail to have the meaning it may have. But then Giano tends to edit in his own space in the way you describe, yet he has one of the highest epp's.

All we can say, and all we need to say is that:

1. In general, editors with low epp's tend to perform relatively mechanical low economic value easily learned tasks. We can verify this by looking at their actual contributions. Editors with high epp's tend to be those with lots of FA and GA stars on their page, and who are generally and anecdotally known as so-called content contributors. That proves there is a division of labour in Wikipedia.

2. Low epp's predominate in the admin corps. Hardly surprising, given that the qualities required of an admin are precisely low-value, repetitive tasks, and given that RfA tends to emphasise quantity rather than quality of edits.

3. The theory of crowdsourcing says that this shouldn't happen.

As I mention above, after seeing Jechoman's epp (7.46) I disagree with the third sentence of 1, though I'm not sure how indicative this is on average. Basically you DO have to control somehow for % of edits to actual articles vs. other categories of Wikipedia pages.

If there was data you could do some regressions here:

1. Dependent variable is a 0/1 dummy for whether a person is an admin or a non-admin. Independent variables are epp, % edits to articles space etc. Run this as a Probit or Logit.

2. Construct a measure of whether a person is a "content creator" by, say, counting up their GAs, FAs and maybe DYKs and just non-redirect articles, weighting these in some way (which would be arbitrary but you could change the weighting to do robustness checks). Then correlate that with epp and % edits to article space.

Overall I don't think the idea that there's "division of labor" on Wikipedia is controversial though. And some of that may even be justified. The problem is with the differential awards and over (under) supply of one particular type relative to the other.

Edit: or as another counter example take Baseball Bugs. His epp is 10.63. But we all know that's only because he just edits AN/I more or less. Yet a simple measure such as yours would put him in a category of "content creator"

(As a further aside, in that Dr. Blofeld discussion that was linked, some moron objects to people objecting to Dr. Blofeld's mass creation of one sentence stubs because "we shouldn't interfere with the work of content creators". In other words, lots of these idiots actually think that auto-creating thousdands of one sentence next to useless stubs is "content creation"!)

This post has been edited by radek:

[Peter Damian]

As I mention above, after seeing Jechoman's epp (7.46) I disagree with the third sentence of 1, though I'm not sure how indicative this is on average. Basically you DO have to control somehow for % of edits to actual articles vs. other categories of Wikipedia pages.

I looked at his edits and he has a large percentage of 'blue' (Wikipedia: pages) which suggests he is part of the peanut gallery. I'm not disagreeing - it's an 'in general' thing. I looked at 720 admin editors and tried in each case of > 4 to explain why it was higher. In nearly all cases the person has a hobby of caterpillars or asteroids, or has FA and GA stars. In most cases of <4, this is not the case. In nearly every case of < 2 the person either is a bot, or acts like one.

Interesting that David Gerard got the second lowest score, I should have mentioned that earlier :|

Edit: or as another counter example take Baseball Bugs. His epp is 10.63. But we all know that's only because he just edits AN/I more or less. Yet a simple measure such as yours would put him in a category of "content creator"

Agree again. With all statistical measures, we see if there is broad agreement, look for anomalies, then try and explain them.

I will do this study again some time, but using the tool to check 720 edits take exactly 2 days. Access to the database would be wonderful.

This post has been edited by Peter Damian:

[radek]

As I mention above, after seeing Jechoman's epp (7.46) I disagree with the third sentence of 1, though I'm not sure how indicative this is on average. Basically you DO have to control somehow for % of edits to actual articles vs. other categories of Wikipedia pages.

I looked at his edits and he has a large percentage of 'blue' (Wikipedia: pages) which suggests he is part of the peanut gallery. I'm not disagreeing - it's an 'in general' thing. I looked at 720 admin editors and tried in each case of > 4 to explain why it was higher. In nearly all cases the person has a hobby of caterpillars or asteroids, or has FA and GA stars. In most cases of <4, this is not the case. In nearly every case of < 2 the person either is a bot, or acts like one.

Interesting that David Gerard got the second lowest score, I should have mentioned that earlier :|

Edit: or as another counter example take Baseball Bugs. His epp is 10.63. But we all know that's only because he just edits AN/I more or less. Yet a simple measure such as yours would put him in a category of "content creator"

Agree again. With all statistical measures, we see if there is broad agreement, look for anomalies, then try and explain them.

I will do this study again some time, but using the tool to check 720 edits take exactly 2 days. Access to the database would be wonderful.

I think you have uncovered a certain asymmetric pattern here: low epp --> "gnomish edits" or "useless crap" but certainly not "content". Hi epp --> it depends.

I brought up the counter examples above simply because I'm wondering how much of the pattern that is and if it could somehow be controlled for. High % "blue pages" and % "user's talk" I think would be good indicators that a particular editor with a high epp is in the "peanut gallery" category, not the "content creator" category

This post has been edited by radek:

[Peter Damian]

I think you have uncovered a certain asymmetric pattern here: low epp --> "gnomish edits" or "useless crap" but certainly not "content". Hi epp --> it depends.

It does depend, but if you look at the actual top 20, with very few exceptions, they don't edit 'blue pages'. Hochman is the only one, I think.

Could it or should it be controlled? Only if it occurs significantly across much of the sample. Here, I think we can note it and pass one.

The anomalies are actually in the 2-3 region where content contributors also engage in regular frenetic 'gnoming'.

On Baseball Bugs, I did another study a few months ago of edits over 2 years to ANI. He came out way ahead of anyone else and is, again, probably an anomaly.

This post has been edited by Peter Damian:

[radek]

I think you have uncovered a certain asymmetric pattern here: low epp --> "gnomish edits" or "useless crap" but certainly not "content". Hi epp --> it depends.

It does depend, but if you look at the actual top 20, with very few exceptions, they don't edit 'blue pages'. Hochman is the only one, I think.

Could it or should it be controlled? Only if it occurs significantly across much of the sample. Here, I think we can note it and pass one.

The anomalies are actually in the 2-3 region where content contributors also engage in regular frenetic 'gnoming'.

On Baseball Bugs, I did another study a few months ago of edits over 2 years to ANI. He came out way ahead of anyone else and is, again, probably an anomaly.

Well, the other one that you should include is "purple" pages (User talk). But yes, there is some patterns here.

Here, I made a matrix (and uploaded it to commons (IMG:smilys0b23ax56/default/wink.gif)) which I think sort of describes what is going on, though obviously we haven't got the data to confirm ALL the cells in it:

(IMG:http://upload.wikimedia.org/wikipedia/commons/5/59/DIV_LABOR_WIKI.png)

You could graph some "famous" editors on that matrix like in those libertarian "economics/social values" graphs people put on their userpages. I expect that'd be pretty funny AND informative.

(and on that note, I'm sort of wondering if there's a way to randomly sample editors (say, those with more than 1000 edits) in a way similar to the Random Article feature)

This post has been edited by radek:

[Ottava]

Here, I made a matrix (and uploaded it to commons (IMG:smilys0b23ax56/default/wink.gif)) which I think sort of describes what is going on, though obviously we haven't got the data to confirm ALL the cells in it:

(IMG:http://upload.wikimedia.org/wikipedia/commons/5/59/DIV_LABOR_WIKI.png)

You could graph some "famous" editors on that matrix like in those libertarian "economics/social values" graphs people put on their userpages. I expect that'd be pretty funny AND informative.

(and on that note, I'm sort of wondering if there's a way to randomly sample editors (say, those with more than 1000 edits) in a way similar to the Random Article feature)

My percentage in Articles was less than 30%. I still think you are forgetting WP:DYK, WP:GAN, WP:FAC, which moves edits from "article" or "article talk" to Wikipedia. Nevermind, you mentioned that in your next post.

By the way, Gatoclass writes very little actual content. He is just an admin that latched onto DYK and used it as his little territory. SandyGeorgia does some article work but very little anymore.

This post has been edited by Ottava:

[radek]

Here, I made a matrix (and uploaded it to commons (IMG:smilys0b23ax56/default/wink.gif)) which I think sort of describes what is going on, though obviously we haven't got the data to confirm ALL the cells in it:

(IMG:http://upload.wikimedia.org/wikipedia/commons/5/59/DIV_LABOR_WIKI.png)

You could graph some "famous" editors on that matrix like in those libertarian "economics/social values" graphs people put on their userpages. I expect that'd be pretty funny AND informative.

(and on that note, I'm sort of wondering if there's a way to randomly sample editors (say, those with more than 1000 edits) in a way similar to the Random Article feature)

My percentage in Articles was less than 30%. I still think you are forgetting WP:DYK, WP:GAN, WP:FAC, which moves edits from "article" or "article talk" to Wikipedia. Nevermind, you mentioned that in your next post.

By the way, Gatoclass writes very little actual content. He is just an admin that latched onto DYK and used it as his little territory. SandyGeorgia does some article work but very little anymore.

You are, for once, right on this. I'm actually taking down some of this data for various people and you come up as a "Someone who uses Wikipedia as Facebook" but I don't think you were that - well, not that much - correction, you come up as "Drama Queen"... hmm, maybe not that far off. This is actually very similar to the problem that someone like SandyGeorgia comes up as indistinguishable along these two dimensions from someone like Baseball Bugs. And all of that has to do with the fact that the soxred data does not distinguish between "Posting to AN/I way too much" from "Reviewing GAs and FAs" - it counts both under "Wikipedia" but qualitatively these are very different things.

So... I'm still tweaking it. If anyone can point me to a statistic which would allow me to distinguish "Posting to ANI way too much" from "Reviewing GAs" (or similar) kind of people then I would appreciate it. For some editors who "opted in" into the whole soxred thing you can do it, but most haven't. Other than that, the only thing I can think of is to take an editor's last 1000 or so contributions and see what % were to ANI, AE etc. But that's a buttload of work at this point.

BTW, Malleus is a very clear outlier. Very high % in article space and pretty high % epp. Very clearly a "content contributor". Giano not so much (though still in that cell).

Update:

Here's a bit of what I have so far:

(IMG:http://upload.wikimedia.org/wikipedia/commons/3/34/Div_of_labor2.png)

Again, the basic problem is that given the data, in the "warm colors" category (red and orange) it is impossible to distinguish people who use WP:whatever type pages (the blue pages) for what could essentially be considered legitimate uses (reviewing FAs etc.) vs. people who are fucking around (playing on ANI, politicking on talk pages)

Also, related to the other thread, someone like Dr. Blofeld shows up as a "wiki gnome" because they mass create a lot of one or two sentence stubs. This means their article % is high, but since he never goes back to see what happened to the children he sired he has a low epp. In this case I think "wiki gnome" is not too inaccurate (cough cough), so I'm not bothered by this. Overall I think this illustrates some of the above discussion.

This post has been edited by radek:

[Malleus]

BTW, Malleus is a very clear outlier. Very high % in article space and pretty high % epp. Very clearly a "content contributor". Giano not so much (though still in that cell).

Update:

Here's a bit of what I have so far:

(IMG:http://upload.wikimedia.org/wikipedia/commons/3/34/Div_of_labor2.png)

Are you sure you don't mean "outlaw" rather than "outlier"?

[radek]

[/quote]
Are you sure you don't mean "outlaw" rather than "outlier"?
[/quote]

You're an Outlaw Outlier. Oooooo!