The "holy shit" slide on editor retention, Sue Gardner on Wikipedia's death spiral Options

[timbo]

1. Dittos on the kudos for the content summary by PD. The summary is accurate and on point.

2. I like Sue Gardner. She is sharp and is not oblivious to the big problems of the project. WP could do much, much worse than having her at the helm.

3. I was intrigued that the "Rogue Administrator Problem" is on the foundation's radar. The need for simplification of rules and provision of a WYSIWYG editing panel seems obvious and that's rightfully recognized. But the curse of Rogue Administrators is another big fetter on growth of the project and one that is less obvious at a glance.

4. The use of the amorphous term "Editor" needs to evolve into descriptives of the real functions of participants -- content creation, copy-editing, and quality control being the useful functions. This, of course, omits the non-productive participation of drama hounds.

5. I liked the fact that Gardner comprehends that New Page Patrol is being "played" like a first-person shooter, combined with the recognition that it is an A7 bloodbath that is blowing away "nuns and tourists."

tim

This post has been edited by timbo: Mon 21st November 2011, 8:10am

[Cla68]

1. Dittos on the kudos for the content summary by PD. The summary is accurate and on point.

2. I like Sue Gardner. She is sharp and is not oblivious to the big problems of the project. WP could do much, much worse than having her at the helm.

3. I was intrigued that the "Rogue Administrator Problem" is on the foundation's radar. The need for simplification of rules and provision of a WYSIWYG editing panel seems obvious and that's rightfully recognized. But the curse of Rogue Administrators is another big fetter on growth of the project and one that is less obvious at a glance.

4. The use of the amorphous term "Editor" needs to evolve into descriptives of the real functions of participants -- content creation, copy-editing, and quality control being the useful functions. This, of course, omits the non-productive participation of drama hounds.

5. I liked the fact that Gardner comprehends that New Page Patrol is being "played" like a first-person shooter, combined with the recognition that it is an A7 bloodbath that is blowing away "nuns and tourists."

tim

So, did Sue present a plan of action, with accompanying timeline, to resolve the problems she presented on the "Oh Shit" slide? Does the WMF have a formal plan of action to resolve this problem? This is what Sue is being paid to do, solve the big problems, no?

This post has been edited by Cla68: Mon 21st November 2011, 8:23am

[Peter Damian]

So, did Sue present a plan of action, with accompanying timeline, to resolve the problems she presented on the "Oh Shit" slide? Does the WMF have a formal plan of action to resolve this problem? This is what Sue is being paid to do, solve the big problems, no?

One of the (probably many) things I missed from the summary was where Sue talks about the German Wikipedia and their rejection of image filtering. She said (something like) the Foundation accepts their position for now, but implied that may change in the future.

I'll see if I can find it.

[Cla68]

So, did Sue present a plan of action, with accompanying timeline, to resolve the problems she presented on the "Oh Shit" slide? Does the WMF have a formal plan of action to resolve this problem? This is what Sue is being paid to do, solve the big problems, no?

One of the (probably many) things I missed from the summary was where Sue talks about the German Wikipedia and their rejection of image filtering. She said (something like) the Foundation accepts their position for now, but implied that may change in the future.

I'll see if I can find it.

It appears from your earlier summary that Sue has identified the following problems afflicting Wikipedia:

1. A death spiral of declining editor participation
2. A depersonalization in interaction with new editors
3. Rigidity of certain key processes, such as the FA process
4. Rogue admins
5. Increased complexity of the editing process

Now, after identifying these problems, Sue's job, as chief executive, is only partially complete. She needs to identify the causes of these problems, and thereby validate that these are root problems and not symptoms of deeper issues. Then, she needs to develop and implement a plan of action, with a timeline, for a solution to these problems and how the proposed solution will be measured to determine its effectiveness. The WMF board needs to document and evaluate her performance in executing this process. If she doesn't solve the problems she identifies in a timely manner, she needs to be replaced by someone who will.

This is why I would hire a free-market capitalist. A capitalist would understand that what is first needed is to identify the truly unique and valuable product or service that Wikipedia provides or can provide, define the current obstacles to its fully exploiting its competitive product, and design and implement a plan that will fully develop, market, and "sell" its product for maximum return. Since Wikipedia's stated goal is not profit, but egalitarian distribution of "knowledge", then the WMF needs to determine what constitutes a successful spread of this knowledge and how to achieve and measure it.

If Sue hasn't done this yet, she had better get started. She has already been on the job several years. Visiting the OWS protest to find some ideas is fine as a start, but she should be graded on actually taking the ideas and making them work and soon.

This post has been edited by Cla68: Mon 21st November 2011, 12:04pm

[lilburne]

This is why I would hire a free-market capitalist. A capitalist would understand that what is first needed is to identify the truly unique and valuable product or service that Wikipedia provides or can provide, identify the current obstacles to its fully exploiting its competitive product, and design and implement a plan that will fully develop, market, and "sell" its product for maximum return.

Its USP is as a MMORPG for inadequate wannabe intellectuals. In that regard it is successful. For anything else one needs a different product entirely.

[Peter Damian]

I should have given some more signposting. See below. She begins the 'What WMF is doing' bit around 35:40.

2:00 Editor retention is not not not OK. It's a big problem. It's the thing that needs to be solved.
2:30 "the holy shit slide" New editors aren't making it to their first year anniversary. People are coming in large numbers as they always have, but they are getting rebuffed. Why are they failing to enter community. Warnings have gone up, criticism is 'way up', and praise and thanks have been in decline. People join and then it's warning template after template.
5:20 The templates are well-intended but they make new editors feel like 'Wikipedia hates you'.
6:00 Pre-2006 there were no automated warning. Established editors would talk to new editors and help them through.
6:55 Now, 4 out of 5 messages are bot-delivered.
7:20 Someone asked about the 'goldrush' hypothesis - early gains are over. The low hanging fruit has all been picked.
7:56 "You are completely wrong" "There is very little evidence to support that.
8:35 On the contrary, we see the same pattern on other Wikipedias. Even the Hindi Wikipedia, which has only 5,000 articles.
9:15 After a couple of years a Wiki gets "crufted up" with rules and templates.
9:40 People talking about their first edits. Usually bad, but you used to be able to recover from bad edits. Newbies are having a terrible experience.
11:00 SO! Deletions and reversions stop people hanging around.
11:20 The indicator of good Wikipedian is to make lots of edits in first few days.
11:45 But these are the most disincentivised.
12:20 "People think the website is yelling at them. " People don't distinguish between people on the website, and the website itself. They feel like the website hates them. "Its super super challenging".
13:20 Q: Are there any patterns in established non-admin editors? - They think anecdotally it's harder for everyone. There's a ripple effect that is troubling for everyone. There aren't enough people to do the work. Qualitatively, people are stressed and worked out, and the culture is 'fightier'. It reminds her of how a newsroom operates. Seasoned editors need a desk job. Her impression as a working journalist, was that older guys still acting like junior reporters. "Where are the new generations of people to do scut work [?]". the older editors will age out and there will be no one to replace them. People start editing when at school, and the "forgive me for saying this, but your wife makes you stop". Sometimes parents, sometimes girlfriends "time to put away childish things". "When are you going to make partner?".
19:00 the problem of FA.
20:40 In 2005-6 the wall started to go up. [Mention of how difficult she found Florence Devouard] She discusses FA and how the bar has got so high that certain articles will never go to FA.
22:00 The Saturn article analogy. "I honestly think we have lost our way, I really do right. I sometimes think we have become Nupedia, and we need another Wikipedia to feed into it. I just think we are really really rigid ... We have lost sight of what makes the project special, which is that everybody really does have something to contribute, it's not just a priesthood".
23:20 Someone comments about a user who started an article on their user page, and told that if they did not stop their disruptive editing they will be blocked.
24:00 Sue thinks this came out of the Seigenthaler thing, when Jimmy insisted there must be a focus on quality, also the same time as Essjay. "There was a moral panic created around quality". This gave a whole lot of people license to be jerks.
25:00 People are playing Wikipedia like a video game, shooting down vandals, and every now and then a nun or a tourist wanders in front of the AK47 and just gets murdered. "What we think now is that it's all nuns and tourists. There's a big massacre and there's one vandal in the background running away. And meanwhile everyone else is dead."
25:30 Q: - So what are WMF going to do about it?
26:00 "I deliberately kept out the phrase 'death spiral', because I didn't want to put it in writing. It's too depressing".
26:40 The March 2011 resolution. The WMF rarely speaks to editors (I think "speaks to is US or Canadian for "tells editors what they should do", or perhaps it means 'speak about'). It used to be Jimmy who would proclaim something. As an movement develops matures it needs institutions, not individuals. Jimmy still plays a role as an individual, but the board, the institution plays much more of a role today. The first time was the BLP resolution. The board said on March 2011 that editor retention was its top priority.
27:40 "Some of this is chapter challenge, some of this is individual editor challenge" (At this point they read something which is not spoken aloud).
28:20 Symonds: there are more and more situations where someone has bitten a newbie. They've followed policy and rules. So what if Arbcom tells them otherwise? Does Arbcom de-admin them? "even as arbitrators our hands are tied". "We can't desysop someone over a minor ongoing issue without an awful lot of drama". Someone else comments that they follow them around and revert their edits.
31:00 Someone else moots the idea of a 'topic ban' equivalent, but Symonds objects that at this point they are saying they no longer trust the admin, as a committee and as a community, to carry out certain tasks, therefore the community no longer trusts you therefore [pause] - you should no longer be an admin.
32:00 Yet another person says that as a result of the community becoming a hive [?] the biggest problem is those admins, particularly rogue admins, who don't have a big turnover, who are admins for life, who were elected when it was a lot easier to become an admin etc.
32:30 Sue mentions someone on Foundation-l (or internal l?) who was being 'a real jerk', and a number of people even called him on it.
33:00 He told her "I don't think I was being uncivil, I was being icily sarcastic". Is icily sarcastic what we aim for? Is that success? "I would interpret our requirement for civility as a flaw. You cannot get worse than that". "As though I can be barely civil and that's what we are supposed to do here".
33:20 She then goes on about Occupy Wall St, where she was (at Manhattan) a few weeks ago. The think she liked most was that everyone there took responsibility for maintaining a good tone. She does think that Arbcom should play a role. Not banning, but maybe 'taking a role'.
35:00 She notes that the smaller projects have a big problem, e.g. Wikiversity.
35:43 "What is the WMF doing" First a bunch of analysis (see the pages on meta).
36:20 the 'false tension' between quality and participation. "We fell into thinking that was true"
36:45 The abuse filter does not hurt participation. Wikiproject monuments ditto.
37:45 the attrition pipeline - the problem of the edit interface and other impediments to new editors.
38:40 Maryana's analysis of top editors. "That was where I learned to be nice to Marek69". "Go Marek!"
40:42 - who are the top newbie killers. They are thinking about mechanisms for that. - have they tried to identify who the top newbie killers are? They can't, but they have been thinking about mechanisms. "There are people we call 'moth people'. They are drawn to the flame. And some people are drawn to the flame in a good way (they negotiate, they mediate), others in a bad way, they like fighting. "Its really hard for data to tell you things like that". They had thought about a flag to identify the bad moth people.
41:30 the moth people. They thought about flagging people as 'unhelpful'. The problem is that some admins and arbcom people would be flagged as 'unhelpful'.
42:50 'Dont be a dick' is really unpleasant.
43:30. The building piece. The most important is the visual editor. They can't continue to live in 2001 visual world. "Its a big big big deal". She anticipates lots of upset. She wants the chapters to play a role in creating acceptance. "We cannot continue to move at the pace of 'community acceptance'. The 'bike shed essay'. (A board of trustees can approve the building of a nuclear power station in 5 minutes, but will spend weeks over the construction of a bike shed).
49:00 the volunteers devs tend to increase complexity. She sees the job of the WMF professional devs to reduce complexity, and she sees the chapters as helping.
51:49 Wikilove is the next thing they have done.
52:17 the article feedback tool. Every website in the world has a feedback tool, why not WP? It's part of making it more warm and welcoming.
1:02:45 Other things they are doing. E.g. pink and fluffy templates.
1:03:34 what the chapters should be doing. "Our reach is really important, education is really important". Only Wikipedians can do community work. She can't hire people to work on policy. Chapters have credibility in editing community. The bank account analogy. The UK chapter should spend its credibility on the ideas she has presented. If the chapter stands on the sidelines, that will not be interpreted as 'neutral', it will be interpreted as 'not helpful'.
1:07:00 question about people being aggressive and throwing their weight around. There are people for whom rowing [arguing] is an important part of their life. But they have nothing to do with the chapter.

[Cla68]

I should have given some more signposting. See below. She begins the 'What WMF is doing' bit around 35:40.

She talks about some possible causes of the problems and some potential remedies. That won't cut it. The board needs to give her a deadline for implementing a formal remedial plan of action, with a timeline and ECD, for solving the problems. Until then, she's not making much, if any, progress on fixing what ails Wikipedia.

[Peter Damian]

It appears from your earlier summary that Sue has identified the following problems afflicting Wikipedia:

1. A death spiral of declining editor participation
2. A depersonalization in interaction with new editors
3. Rigidity of certain key processes, such as the FA process
4. Rogue admins
5. Increased complexity of the editing process

Now, after identifying these problems, Sue's job, as chief executive, is only partially complete. She needs to identify the causes of these problems, and thereby validate that these are root problems and not symptoms of deeper issues. Then, she needs to develop and implement a plan of action, with a timeline, for a solution to these problems and how the proposed solution will be measured to determine its effectiveness. The WMF board needs to document and evaluate her performance in executing this process. If she doesn't solve the problems she identifies in a timely manner, she needs to be replaced by someone who will.

I think the longer version of the talk partly answers that. The increased complexity of the editing process will be addressed by the new visual editor (costing $3m, no?). Rogue admins they will address by software to spot rogue admins. The rigidity of processes they will address by software like the abuse filter.

The Wikilove thing (where you can send a picture of a kitten to someone) is another example of how to stop Wikipedia being so 'fighty' and depersonalised. So I think she has ideas about how to fix all these problems. Essentially, you can fix everything with software and fluffy kittens.

This post has been edited by Peter Damian: Mon 21st November 2011, 12:25pm

[HRIP7]

I think the longer version of the talk partly answers that. The increased complexity of the editing process will be addressed by the new visual editor (costing $3m, no?). Rogue admins they will address by software to spot rogue admins. The rigidity of processes they will address by software like the abuse filter.

The Wikilove thing (where you can send a picture of a kitten to someone) is another example of how to stop Wikipedia being so 'fighty' and depersonalised. So I think she has ideas about how to fix all these problems. Essentially, you can fix everything with software and fluffy kittens.

Two other things that were mentioned somewhere along the way were "New Page Triage" rather than "New Page Patrol" and redesigning the sorts of templates that end up on people's user pages.

A good example of the AK47 approach was a prolific new page patroller (now "retired" following an RfC/U, or more likely working under a new user name) whose user page featured the following proud mission statement –

Whack-a-mole: as much as the vandals and the new bands and the essayers and the family-tree scribes enjoy creating worthless articles, I enjoy speedily deleting them more.

He had also awarded himself a bunch of barnstars on his user page, including one that said

You play whack-a-mole with terrible new pages like no one I've ever seen! Awesome!

The trouble is his deletion sprees included articles that cited sources and were created in perfectly good faith, often deleted minutes or seconds after creation (he didn't want another patroller to get there first).

Here on the other hand is a new article by a newbie who can actually write. They don't have a good grasp of WP:V and wiki formating yet (why would they, after less than ten edits?), but that is something an intelligent person can pick up quickly. Learning to write well, on the other hand, is something not everyone can or will learn.

So this is someone Wikipedia should have embraced with open arms. Instead, what happened was that the article was reviewed and turned down (scroll down to see the red "you've failed" template).

That was accompanied by two templates on her talk page, and she hasn't been seen since. (The article ended up in mainspace anyway; I don't quite understand why. And someone has since left a nice comment on her talk page, but it's probably too little, too late.)

So, on that evidence at least, Wikipedia attracts idiots wrapped up in puerile power fantasies, and repels intelligent people who can write.

This post has been edited by HRIP7: Mon 21st November 2011, 2:34pm

[HRIP7]

One of the (probably many) things I missed from the summary was where Sue talks about the German Wikipedia and their rejection of image filtering. She said (something like) the Foundation accepts their position for now, but implied that may change in the future.

I'll see if I can find it.

She said that a category-based image filter is off the agenda, and that she wouldn't go against the German community on that. This is from her slides:

Some German editors have told me that they cannot accept an image filter of the type we originally designed. Its introduction would make them want to leave the projects. I take that very seriously. The Wikimedia Foundation will not impose an image filter on German editors that editors strongly oppose.

Having said that: I am not promising that nothing will ever change on the German Wikipedia without consensus agreement. As a movement, we need to be able to be bold, and to experiment freely.

The image filter is different though: it is not an ordinary feature, and so it required special, serious advance discussion.

Does the Wikimedia Foundation intend to go ahead with its original plan to build a category-based image filter?

The Wikimedia Foundation does not intend to build a category-based image filter.

It was clear in the referendum results and the discussion afterwards that a category-based filter system would be unworkable and unacceptable to many editors.

Therefore, we will not build it.

That leaves WereSpielChequers' proposal, which would seem to have the weakness of being extremely gameable (see what Anonymous did to the Time Magazine poll 1 2) as well as any other non-category-based proposals on the Meta Brainstorming page.

[radek]

-----------------------------------------------
New editors aren't making it to their first year anniversary. People are coming in large numbers as they always have, but they are getting rebuffed. Why are they failing to enter community. Warnings have gone up, criticism is 'way up', and praise and thanks have been in decline. People join and then it's warning template after template. The templates are well-intended but they make new editors feel like 'Wikipedia hates you'. Pre-2006 there were no automated warning. Established editors would talk to new editors and help them through. Now, 4 out of 5 messages are bot-delivered.

I'm also seeing this happen to somebody almost as we speak.

Part of me understands why people are keen to block new-but-disruptive users who are not quick enough on climbing that learning curve. Once a potential troublemaker gets settled in it takes an insane amount of time to do something about them. I'm reminded of the user Varsovian whose sole purpose on en-wiki was trolling Polish editors, and who has been banned for exactly same behavior on other internet forums. He managed to survive the first two weeks and then it took more than a year, half a dozen AE reports, several good editors leaving the project and several others getting stupidly baited into wars with him and blocked, before he was finally topic banned. So yeah, people figure that it's better to be preventive and shoot down potential trouble makers while they're still easy targets.

So the rule of thumb is that when you have a situation A which requires solution X, and a situation B which requires solution Y, the Wikipedia admin corps will implement Y in A and X in B, managing to get BOTH wrong.

It's also a perfect illustration of how one set of problems on Wikipedia (no meaningful dispute resolution process for established editors) creates a whole another set of problems (shooting newbies on sight), which leads to a developing crisis (holy shit slide).

Watch them decide that the way to deal with it is to put more cookies in that image that comes with the {welcome} template. Maybe put a cute kitten in there. And then still block the people after a day or two.

This post has been edited by radek: Mon 21st November 2011, 4:32pm

[SB_Johnny]

The Wikilove thing (where you can send a picture of a kitten to someone) is another example of how to stop Wikipedia being so 'fighty' and depersonalised. So I think she has ideas about how to fix all these problems. Essentially, you can fix everything with software and fluffy kittens.

Yup, it's all about the kitties. Maybe they should hire the guy who designed cow clicker to change the user interface for them.

[gomi]

Essentially, you can fix everything with software and fluffy kittens.

No, no. You have to make the software from fluffy kittens. After grinding them up.

[carbuncle]

Why do I have the feeling that the solution to the problem of new editors receiving critical messages this will be to shut down ClueBot...?

[GlassBeadGame]

There is nothing new in this thread AFAIK. Some of this has been discussed on WR at length (Death Spiral) and some has been given the derision it deserves (imposing civility.) The only thing new is Gardner is now leading the conversation. Which makes me wonder...isn't there a better website to have this conversation?

[powercorrupts]

There is nothing new in this thread AFAIK. Some of this has been discussed on WR at length (Death Spiral) and some has been given the derision it deserves (imposing civility.) The only thing new is Gardner is now leading the conversation. Which makes me wonder...isn't there a better website to have this conversation?

Here we go, it's the 'GBG Has Spoken.' moment.

Sometimes I wonder if you'll ever be happy until you see this website actually closed down. I suppose that since your hatred of Wikipedia is so completely self-consuming, it does make a kind of logical sense.

[Cla68]

There is nothing new in this thread AFAIK. Some of this has been discussed on WR at length (Death Spiral) and some has been given the derision it deserves (imposing civility.) The only thing new is Gardner is now leading the conversation. Which makes me wonder...isn't there a better website to have this conversation?

Here we go, it's the 'GBG Has Spoken.' moment.

Sometimes I wonder if you'll ever be happy until you see this website actually closed down. I suppose that since your hatred of Wikipedia is so completely self-consuming, it does make a kind of logical sense.

GBG can correct me if I'm wrong, but I think his premise is that Wikipedia can't be fixed because it is a fundamentally, inherently wrong concept. The thing is, we can't know for sure if he's right until the WMF actually makes a coherent effort at trying to fix it. Several of you above discuss the remedies that the WMF is attempting to implement. Again, the WMF needs to detail how and when it will assess if the remedies have worked, including what kinds of metrics or measures they will use as part of their evaluation.

[EricBarbour]

Rogue admins they will address by software to spot rogue admins. The rigidity of processes they will address by software like the abuse filter.

And what if the rogue admins are software?......

[thekohser]

Again, the WMF needs to detail how and when it will assess if the remedies have worked, including what kinds of metrics or measures they will use as part of their evaluation.

Regardless of what you may or may not ever hear the WMF say as regards their measurement of remedies, the only TRUE METRICS that they care about are all found once per year on the IRS Form 990.

[GlassBeadGame]

There is nothing new in this thread AFAIK. Some of this has been discussed on WR at length (Death Spiral) and some has been given the derision it deserves (imposing civility.) The only thing new is Gardner is now leading the conversation. Which makes me wonder...isn't there a better website to have this conversation?

Here we go, it's the 'GBG Has Spoken.' moment.

Sometimes I wonder if you'll ever be happy until you see this website actually closed down. I suppose that since your hatred of Wikipedia is so completely self-consuming, it does make a kind of logical sense.

GBG can correct me if I'm wrong, but I think his premise is that Wikipedia can't be fixed because it is a fundamentally, inherently wrong concept. The thing is, we can't know for sure if he's right until the WMF actually makes a coherent effort at trying to fix it. Several of you above discuss the remedies that the WMF is attempting to implement. Again, the WMF needs to detail how and when it will assess if the remedies have worked, including what kinds of metrics or measures they will use as part of their evaluation.

I don't know if WP can be "fixed" or not. If it continues to operate I want it to do so in a responsible manner answerable to significant stake holders not at their table like parents, BLP subjects, Muslim and others with cultural/religious concerns etc. I believe the most backward segment within WMF/Wikipedia is not management Mr.Wales or even admins. It is a demographically narrow and ideologically right wing (although they would deny that) corps of over-involved Wikipedians. "Content Creator" politicians or game playing gnomes... it makes no difference. They are the bulwark of Wikipedia. They are all over WR too. As for Ms Gardner I have said some positive things about her in the past. At least she rid WMF of the gun moll Dorne, Mr. Wales coke and whore credit cards and Danny's silly antics. More importantly she has begun to address the scale of staffing and resources needed to lay a foundation for imposing controls over the sites users and activities, even as she misapplied them into public relations ans silly special projects. Still there was no hope of a responsible site while Danny brought in no grants and whole shebang ran on $200,000 a year. Why she has not rid of Eric Moeller I still can't explain.

So if she wants to revisit some of the topics already explored on WR I wish her well. But if the draw is what she has to add or to educate her then it seems to me a outright WP vehicle would be a better place to engage her in the conversation than spoon feed the conversation from there to back here.