Content contributors, statistical analysis Options

[Peter Damian]

My blog post for today http://ocham.blogspot.com/2011/10/repetiti...-wikipedia.html on whether there are statistically measurable properties that distinguish 'content contributors' from wiki-gnomes. Conclusion: the statistical difference is strongly indicative of a real difference, discussed in detail on the blog.

Remaining questions: why do content contributors remain on the project, given that they have a lower status than those who perform repetitive and tedious work?

Easily-learned repetitive labour is nearly always paid less in real life than labour which requires either specialised learning, or some innate but scarce skill. The simple reason for this is supply and demand. Rare or difficult-to-acquire skills are by definition in short supply, and will attract a higher price than common, easily acquired skills (at least, to my simple mind - I don't know any economics).

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

This post has been edited by Peter Damian:

[SB_Johnny]

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

To be fair, I suspect at least part of that is because administrative buttons aren't really all that useful for content creators. The pool of people who have endless hours to engage in wikipolitics and chase vandals aren't necessarily the ones who have a deep background from which to contribute to actual encyclopedia-building.

[Ottava]

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

To be fair, I suspect at least part of that is because administrative buttons aren't really all that useful for content creators. The pool of people who have endless hours to engage in wikipolitics and chase vandals aren't necessarily the ones who have a deep background from which to contribute to actual encyclopedia-building.

You'd be surprised. Editing protected pages, history merges, moving over redirects, suppressing redirects, etc., are all extremely valuable to editing content.

[Peter Damian]

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

To be fair, I suspect at least part of that is because administrative buttons aren't really all that useful for content creators. The pool of people who have endless hours to engage in wikipolitics and chase vandals aren't necessarily the ones who have a deep background from which to contribute to actual encyclopedia-building.

I didn't understand the 'to be fair' bit.

[communicat]

Peter/Edward, don't know if you've come across this, written by a fellow logician. It might or might not answer some of your questions, and it provides some useful references.
http://knol.google.com/k/carl-hewitt/corru...ip_by_Wikipedia

[Peter Damian]

Peter/Edward, don't know if you've come across this, written by a fellow logician. It might or might not answer some of your questions, and it provides some useful references.
http://knol.google.com/k/carl-hewitt/corru...ip_by_Wikipedia

Thanks, but yes, actually I am familiar with that one.

[radek]

My blog post for today http://ocham.blogspot.com/2011/10/repetiti...-wikipedia.html on whether there are statistically measurable properties that distinguish 'content contributors' from wiki-gnomes. Conclusion: the statistical difference is strongly indicative of a real difference, discussed in detail on the blog.

Remaining questions: why do content contributors remain on the project, given that they have a lower status than those who perform repetitive and tedious work?

Easily-learned repetitive labour is nearly always paid less in real life than labour which requires either specialised learning, or some innate but scarce skill. The simple reason for this is supply and demand. Rare or difficult-to-acquire skills are by definition in short supply, and will attract a higher price than common, easily acquired skills (at least, to my simple mind - I don't know any economics).

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

I think you already answered your own question - supply and demand, while always present, lead to the outcomes you describe (higher paid for scarcer skills) only in functioning markets. Wikipedia is not a market. So rewards are not necessarily related to productivity or usefulness but rather determined through a messy social and political process (who's got what friends).

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

To be fair, I suspect at least part of that is because administrative buttons aren't really all that useful for content creators. The pool of people who have endless hours to engage in wikipolitics and chase vandals aren't necessarily the ones who have a deep background from which to contribute to actual encyclopedia-building.

You'd be surprised. Editing protected pages, history merges, moving over redirects, suppressing redirects, etc., are all extremely valuable to editing content.

Eh, they may be useful but are neither necessary nor even "extremely valuable".

[Peter Damian]

Wikipedia is not a market.

That's interesting because analysis of Wales' early posts to the lists in 2001 suggests a market economy was exactly what he had in mind. That's why he was so heavy on not biasing the outcome by having content committees or editors in chief and so on.

You'd be surprised. Editing protected pages, history merges, moving over redirects, suppressing redirects, etc., are all extremely valuable to editing content.

Eh, they may be useful but are neither necessary nor even "extremely valuable".

Ottava is not given to irony, but I assumed he was here.

[radek]

My blog post for today http://ocham.blogspot.com/2011/10/repetiti...-wikipedia.html on whether there are statistically measurable properties that distinguish 'content contributors' from wiki-gnomes. Conclusion: the statistical difference is strongly indicative of a real difference, discussed in detail on the blog.

Remaining questions: why do content contributors remain on the project, given that they have a lower status than those who perform repetitive and tedious work?

Easily-learned repetitive labour is nearly always paid less in real life than labour which requires either specialised learning, or some innate but scarce skill. The simple reason for this is supply and demand. Rare or difficult-to-acquire skills are by definition in short supply, and will attract a higher price than common, easily acquired skills (at least, to my simple mind - I don't know any economics).

So why is the situation apparently reversed on Wikipedia? The statistics suggest that the majority of administrators use these low-value skills like vandal reversion, template adding, linking to the Estonian Wikipedia etc. Yet their status on Wikipedia is high, whereas that of 'content contributors' is low.

Oh yeah Peter, one thing. Your methodology will overestimate "content creation" by admins for ones who hang out mostly at AN/I and AE. More precisely, their high edits per page will come from them posting frequently to these drama boards, rather than working on articles.

There's probably some bias on the other end too. Someone like Piotrus has a edit per page number of 3.75, which is somewhere in the middle. But that's because the guy creates LOTS of pages and works a LOT on each of them. So there you'd have to control for total number of edits.

Actually, I think you could somehow use the Namespace Totals % which are given to separate out the repeated edits to actual articles vs. repeated edits to drama boards and user's talk pages. That would give a more accurate and relevant ratio for your purposes (I'd have to think for a few minutes how to do it which I might)

[Ottava]

Peter/Edward, don't know if you've come across this, written by a fellow logician. It might or might not answer some of your questions, and it provides some useful references.
http://knol.google.com/k/carl-hewitt/corru...ip_by_Wikipedia

Hewitt doesn't understand that the Stewards haven't respected the Foundation since 2008 and that very few people ever actually listened to Jimbo to begin with. (Not to say that the few who did weren't powerful, but the Commons matter shows that Jimbo was only given some power when he was towing the party line).



Peter

Ottava is not given to irony, but I assumed he was here.

I was being 100% honest. History merges were an annoying thing that was utterly important to me many times. It is annoying to have to go find an admin, link the different pages, and hope they get it right instead of being able to do a bunch of history merges in a row. Remember, I was building a dozen or so articles on average in my user space and then moving them out where many of them had articles. The merging of histories was a valuable addition.

Also, seeing deleted pages is important if you are trying to see what was there before when trying to recreate something. Editing protected pages is good for when you are working on different pages, want to update DYK queue, etc. Importing is another feature that I used. Suppressing redirect was handy quite regularly to me. And it is annoying if you want to move out a page to something that is a redirect already.

This post has been edited by Ottava:

[radek]

Wikipedia is not a market.

That's interesting because analysis of Wales' early posts to the lists in 2001 suggests a market economy was exactly what he had in mind. That's why he was so heavy on not biasing the outcome by having content committees or editors in chief and so on.

Well, he can make whatever crappy analogies he wants to, but it still ain't. I think this just shows that Jimmy doesn't have much of a clue of what a market is or how it functions.

[Peter Damian]

Oh yeah Peter, one thing. Your methodology will overestimate "content creation" by admins for ones who hang out mostly at AN/I and AE. More precisely, their high edits per page will come from them posting frequently to these drama boards, rather than working on articles.

There's probably some bias on the other end too. Someone like Piotrus has a edit per page number of 3.75, which is somewhere in the middle. But that's because the guy creates LOTS of pages and works a LOT on each of them. So there you'd have to control for total number of edits.

Actually, I think you could somehow use the Namespace Totals % which are given to separate out the repeated edits to actual articles vs. repeated edits to drama boards and user's talk pages. That would give a more accurate and relevant ratio for your purposes (I'd have to think for a few minutes how to do it which I might)

Quite correct. That is evident from the pie chart - certain editors have a high 'blue' proportion, which is the WP: prefixed pages. There is no way round that except by selective querying of the database to get only article contributions.

And there are many other ways this figure is skewed. E.g. YellowMonkey has the highest number of FAs, yet a (relatively) low average e.p.p. of 3.69. All I can hope to give is a blunt figure that shows some correlation with our intuitive idea of 'content', namely something that cannot be produced by flitting from page to page, and which requires a long look at a single article, concerning the summary, the meaning of the parts.

Yes, you could use the % of namespace totals as a proxy, but I can think of several reasons why that might be skewed.

At the end of the day, I am trying to give one of many reasons why the concept of 'crowdsourcing' is badly flawed.

But that's because the guy creates LOTS of pages and works a LOT on each of them. So there you'd have to control for total number of edits.

I don't agree with that. If I create 100 pages and give 100 edits to each page, that's a very high e.p.p. of 100. Piotrus is probably contaminating his content work with mechanical repetitive editing. Which I understand well, because I relieve the writer's block doldrums with such activity myself.

Well, he can make whatever crappy analogies he wants to, but it still ain't. I think this just shows that Jimmy doesn't have much of a clue of what a market is or how it functions.

Well, he did publish a peer-reviewed paper on options pricing as part of his Ph.D., so he can't be a complete dunce. I think there are other explanations for why he said those things.

Ottava is not given to irony, but I assumed he was here.

I was being 100% honest.

OK so I was right about the bit before the 'but'.

[thekohser]

Wikipedia is not a market.

For most editors, no, it's not.

Me, though... I just received another $100 PayPal payment for some fairly simple work on Wikipedia.

(IMG:smilys0b23ax56/default/evilgrin.gif)

[communicat]

PeterEdward, in my experience there's another category of wikipedian apart from admins with low-value skills and actual 'content contributors'. I'm referring of course to the category of "supervisor", namely the fact that for every content contributor there seem to be at least three or four extremely tedious and irritating "supervisors", not necessarily admins or productive editors, who constantly nit-pick and tell the content contributor how they think the edit should be done or what should or should not be included. Needless to say, these "supervisors" never, but never, make any edits or content contributions of their own. (Possibly because they already know from some past experience how much shit they'd have to put up with if ever they did try to make a useful contribution).

This post has been edited by communicat:

[radek]

Quite correct. That is evident from the pie chart - certain editors have a high 'blue' proportion, which is the WP: prefixed pages. There is no way round that except by selective querying of the database to get only article contributions.

Well, there's no perfect way of doing it but you could just subtract off the blue to get a probably better estimate.
So edits per article page would be (1-(%wikipedia+%wikipedia talk))*average edits per page

Ideally you'd want to adjust the number of "pages" as well by subtracting AN/I and AE or whatever, but since there aren't that many of these pages it won't get too skewed.

The only possible exception is FAR pages which also count as "wikipedia" (blue) even though a lot of that is obviously content related.

The real difficulty is adjusting for # of edits on users' talk, since there's no way to tell how many different user talk pages a particular person posted to. And a lot of these admins basically spend the majority of their time politickin' on each others' talk pages so that's really something which should be taken into account. For example, Fetchcommons has 28.26% of his posts to user's talk. Sandstein has 24.71%. SarekOfVulcan has 24.09%, Jechochman (who has a pretty high average edits per page - but that's not cause he edits articles a lot) 28.93%, Georgewilliamherbert 34.69% BWilkins 39.26% etc.
For comparison, only 6.62% of my edits are to users talk.

So none of the above have anything to do with average edits per ARTICLE page. Again, the difficulty is in adjusting both the numerator and denominator here.

Still I think the formula above would give a somewhat better picture of actual edits per article page.

And there are many other ways this figure is skewed. E.g. YellowMonkey has the highest number of FAs, yet a (relatively) low average e.p.p. of 3.69. All I can hope to give is a blunt figure that shows some correlation with our intuitive idea of 'content', namely something that cannot be produced by flitting from page to page, and which requires a long look at a single article, concerning the summary, the meaning of the parts.

Yes, and some people will work on articles on their word processor or sandbox and then just post the ready thing. Others (like me) like to do it bit by bit. So the measure is obviously going to miss that.

Yes, you could use the % of namespace totals as a proxy, but I can think of several reasons why that might be skewed.

At the end of the day, I am trying to give one of many reasons why the concept of 'crowdsourcing' is badly flawed.

Well, any statistic summarizes information, almost by definition. And when you summarize information, by definition, you're going to loose some information (the only alternative is to somehow look at every single edit ever made at Wikipedia simultaneously). That doesn't mean that describing data with statistics is useless.

[Peter Damian]

PeterEdward, in my experience there's another category of wikipedian apart from admins with low-value skills and actual 'content contributors'. I'm referring of course to the category of "supervisor", namely the fact that for every content contributor there seem to be at least three or four extremely tedious and irritating "supervisors", not necessarily admins or productive editors, who constantly nit-pick and tell the content contributor how they think the edit should be done or what should or should not be included. Needless to say, these "supervisors" never, but never, make any edits or content contributions of their own. (Possibly because they already know from some past experience how much shit they'd have to put up with if ever they did try to make a useful contribution).

Isn't this similar to the way the Red Army used to have 'political officers'?

[Silver seren]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

[Peter Damian]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

Yes of course there are a 101 ways in which this number could fail to have the meaning it may have. But then Giano tends to edit in his own space in the way you describe, yet he has one of the highest epp's.

All we can say, and all we need to say is that:

1. In general, editors with low epp's tend to perform relatively mechanical low economic value easily learned tasks. We can verify this by looking at their actual contributions. Editors with high epp's tend to be those with lots of FA and GA stars on their page, and who are generally and anecdotally known as so-called content contributors. That proves there is a division of labour in Wikipedia.

2. Low epp's predominate in the admin corps. Hardly surprising, given that the qualities required of an admin are precisely low-value, repetitive tasks, and given that RfA tends to emphasise quantity rather than quality of edits.

3. The theory of crowdsourcing says that this shouldn't happen.

[Ottava]

Well, there's no perfect way of doing it but you could just subtract off the blue to get a probably better estimate.
So edits per article page would be (1-(%wikipedia+%wikipedia talk))*average edits per page

Ideally you'd want to adjust the number of "pages" as well by subtracting AN/I and AE or whatever, but since there aren't that many of these pages it won't get too skewed.

The only possible exception is FAR pages which also count as "wikipedia" (blue) even though a lot of that is obviously content related.

The real difficulty is adjusting for # of edits on users' talk, since there's no way to tell how many different user talk pages a particular person posted to. And a lot of these admins basically spend the majority of their time politickin' on each others' talk pages so that's really something which should be taken into account. For example, Fetchcommons has 28.26% of his posts to user's talk. Sandstein has 24.71%. SarekOfVulcan has 24.09%, Jechochman (who has a pretty high average edits per page - but that's not cause he edits articles a lot) 28.93%, Georgewilliamherbert 34.69% BWilkins 39.26% etc.
For comparison, only 6.62% of my edits are to users talk.

So none of the above have anything to do with average edits per ARTICLE page. Again, the difficulty is in adjusting both the numerator and denominator here.

Still I think the formula above would give a somewhat better picture of actual edits per article page.

Just for curiosity's sake: my user talk page percentage was 30.02%. My article percentage was 25.53%. I made an average of 8.61 average edits per page.

I also participated in over 300 different FAC reviews ("Wikipedia" page) and many DYK related matters (also "Wikipedia" page related).


I have a feeling that you might want to break down where exactly the people are editing. Perhaps a much better way of determining "content" contributors are those who add large amounts of bytes to an article that aren't part of an undo? (It would be hard to remove all of the undos though).

[radek]

How would you account for the people that work on making articles in their user subspace and then submit then whole to the mainspace in a single edit? They may end up being the ones with the lowest number of edits to an article, but actually contributed almost all of the content.

Yes of course there are a 101 ways in which this number could fail to have the meaning it may have. But then Giano tends to edit in his own space in the way you describe, yet he has one of the highest epp's.

All we can say, and all we need to say is that:

1. In general, editors with low epp's tend to perform relatively mechanical low economic value easily learned tasks. We can verify this by looking at their actual contributions. Editors with high epp's tend to be those with lots of FA and GA stars on their page, and who are generally and anecdotally known as so-called content contributors. That proves there is a division of labour in Wikipedia.

2. Low epp's predominate in the admin corps. Hardly surprising, given that the qualities required of an admin are precisely low-value, repetitive tasks, and given that RfA tends to emphasise quantity rather than quality of edits.

3. The theory of crowdsourcing says that this shouldn't happen.

As I mention above, after seeing Jechoman's epp (7.46) I disagree with the third sentence of 1, though I'm not sure how indicative this is on average. Basically you DO have to control somehow for % of edits to actual articles vs. other categories of Wikipedia pages.

If there was data you could do some regressions here:

1. Dependent variable is a 0/1 dummy for whether a person is an admin or a non-admin. Independent variables are epp, % edits to articles space etc. Run this as a Probit or Logit.

2. Construct a measure of whether a person is a "content creator" by, say, counting up their GAs, FAs and maybe DYKs and just non-redirect articles, weighting these in some way (which would be arbitrary but you could change the weighting to do robustness checks). Then correlate that with epp and % edits to article space.

Overall I don't think the idea that there's "division of labor" on Wikipedia is controversial though. And some of that may even be justified. The problem is with the differential awards and over (under) supply of one particular type relative to the other.

Edit: or as another counter example take Baseball Bugs. His epp is 10.63. But we all know that's only because he just edits AN/I more or less. Yet a simple measure such as yours would put him in a category of "content creator"

(As a further aside, in that Dr. Blofeld discussion that was linked, some moron objects to people objecting to Dr. Blofeld's mass creation of one sentence stubs because "we shouldn't interfere with the work of content creators". In other words, lots of these idiots actually think that auto-creating thousdands of one sentence next to useless stubs is "content creation"!)

This post has been edited by radek: