Plagiarism on Wikipedia: The Brandt Project, Light the blue touchpaper and watch the fireworks Options

[JohnA]

By the way Daniel, did you preserve the edit histories of the articles so that we will still know who did what?

[Daniel Brandt]

By the way Daniel, did you preserve the edit histories of the articles so that we will still know who did what?

Nope. I didn't expect that they'd wipe the evidence so systematically. Well, I guess I half-expected it, because I did the presentation on my site so that my pages would survive a deletion. But of course I also wanted to assume good faith.

Not to worry -- we're talking about one percent of Wikipedia's 1.4 million articles that I've scanned for evidence of plagiarism. There's always the other 99 percent if more evidence is needed.

[JohnA]

By the way Daniel, did you preserve the edit histories of the articles so that we will still know who did what?

Nope. I didn't expect that they'd wipe the evidence so systematically. Well, I guess I half-expected it, because I did the presentation on my site so that my pages would survive a deletion. But of course I also wanted to assume good faith.

Not to worry -- we're talking about one percent of Wikipedia's 1.4 million articles that I've scanned for evidence of plagiarism. There's always the other 99 percent if more evidence is needed.

I think you'll need to get the article histories otherwise it'll be "Who do you want to believe: me or your own lying eyes?"

[Somey]

I think you'll need to get the article histories otherwise it'll be "Who do you want to believe: me or your own lying eyes?"

Maybe, but a really cynical reporter would tell you that any electronic document can be forged. Heck, any paper document can be forged, just not as easily... I'd say the best approach for any further article lists is to put them directly into the hands of reporters, confidentially, from the get-go.

At the same time, Brandt really did the ethical thing by making the initial list public - and it was probably necessary anyway, just to see what they'd do. Now that we know, he can include that in whatever goes out to the media. I suppose it might be even more ethical to wait a week or two to see if any changes in policy are proposed, or if there's expanded use of this "Wherebot" thingy... But I doubt we'll see anything of that nature!

[JohnA]

I think you'll need to get the article histories otherwise it'll be "Who do you want to believe: me or your own lying eyes?"

Maybe, but a really cynical reporter would tell you that any electronic document can be forged. Heck, any paper document can be forged, just not as easily... I'd say the best approach for any further article lists is to put them directly into the hands of reporters, confidentially, from the get-go.

At the same time, Brandt really did the ethical thing by making the initial list public - and it was probably necessary anyway, just to see what they'd do. Now that we know, he can include that in whatever goes out to the media. I suppose it might be even more ethical to wait a week or two to see if any changes in policy are proposed, or if there's expanded use of this "Wherebot" thingy... But I doubt we'll see anything of that nature!

I think going to the journos with a press release and a list of further gratuitous plagiarism would be a great idea. I also think that monitoring Wikipedia (with screenshots) especially of the edit histories would be a good idea as well.

As I thought they would, they went for the "Connelley Red Button" with alacrity

[Daniel Brandt]

I am somewhat confused over the role of the Foundation as a legal entity in charge of Wikipedia's servers and software, and the role of admins as so-called independent "users" of an "interactive service."

I know what Brad says. He says the users -- even the admins -- are responsible for their own edits. But what happens when admins fire up the shredding machine and destroy evidence of possible illegality? In the interests of making a distinction between the Foundation's legal responsibility, and the responsibility of individual admins, it seems to me that admins should not be able to delete the history behind the article.

It has been noted that there are one or two errors in my listing. I agree that this is likely. The nature of the error is that the plagiarism went the wrong direction, and I didn't exclude it because I failed to arrive at this conclusion. The reason is that I didn't study the history of each instance of alleged plagiarism, to determine if they were all inserted by a single user at a single moment in time. If they were inserted piecemeal, then most likely the source that I've listed as the original source is just another scraper site that didn't credit Wikipedia. Not only that, but they probably have their own copyright notice on their page, which is entirely inappropriate.

But now that the histories are gone, I guess we cannot reconstruct a sequence of events to make these determinations. By the way, there is no way for a bot to make checks like this. And I'm not about to do it manually in the future. If my error rate for failing to make these checks is under five percent, I'll be happy.

I urge the Foundation to take steps to curtail the shredding of evidence. These are all from the deletion log, and while some of the revisions were subsequently restored in some cases, it still means that the relevant history is no longer available. Any histories that can be restored by the Foundation should be restored, so that we can get on with our research.

07:17, 23 October 2006 Alphachimp (Talk | contribs) deleted "Alonzo M. Clark" (g12)
07:10, 23 October 2006 Alphachimp (Talk | contribs) deleted "William Pepperell Montague" (copyright issues)
02:51, 23 October 2006 DragonflySixtyseven (Talk | contribs) deleted "James Francis Smith" (copyvio)
03:35, 23 October 2006 DragonflySixtyseven (Talk | contribs) deleted "Michael Thonet" (content was: '{{copyvio|url=http://www.914.qc.ca/thonet.html}}')
03:24, 23 October 2006 Geni (Talk | contribs) deleted "Adam Lindsay Gordon" (G12)
01:41, 23 October 2006 Geni (Talk | contribs) deleted "Adam Sedgwick" (G12)
02:03, 23 October 2006 Geni (Talk | contribs) deleted "Albert Bassermann" (G12)
02:17, 23 October 2006 Geni (Talk | contribs) deleted "Albert Fonó" (G12 all derived from copyvio version)
01:33, 23 October 2006 Geni (Talk | contribs) deleted "Benjamin Waterhouse" (G12)
03:07, 23 October 2006 Geni (Talk | contribs) deleted "Charles Robinson" (G12)
02:51, 23 October 2006 Geni (Talk | contribs) deleted "Charles W. Leng" (G12 and derivitives there of)
02:00, 23 October 2006 Geni (Talk | contribs) deleted "Clifford Whittingham Beers" (G12)
02:46, 23 October 2006 Geni (Talk | contribs) deleted "Elwood Haynes" (G12)
01:57, 23 October 2006 Geni (Talk | contribs) deleted "Ernest Everett Just" (G12)
01:28, 23 October 2006 Geni (Talk | contribs) deleted "Ernest Malinowski" (G12)
03:38, 23 October 2006 Geni (Talk | contribs) deleted "Félix Vallotton" (G12)
02:25, 23 October 2006 Geni (Talk | contribs) deleted "George Arliss" (G12)
01:45, 23 October 2006 Geni (Talk | contribs) deleted "George Augustus Selwyn" (G12)
03:02, 23 October 2006 Geni (Talk | contribs) deleted "Gurdon Saltonstall Hubbard" (G12 + derivatives)
01:39, 23 October 2006 Geni (Talk | contribs) deleted "Hans Christian Ørsted" (G12)
01:17, 23 October 2006 Geni (Talk | contribs) deleted "Hans Rottenhammer" (G12 article on commons was a copyvio)
01:43, 23 October 2006 Geni (Talk | contribs) deleted "Harry Beaumont" (G12)
01:34, 23 October 2006 Geni (Talk | contribs) deleted "Henry Nehrling" (rmG12)
03:43, 23 October 2006 Geni (Talk | contribs) deleted "Henry Williams (missionary)" (G12)
03:12, 23 October 2006 Geni (Talk | contribs) deleted "Henryk Arctowski" (G12)
01:20, 23 October 2006 Geni (Talk | contribs) deleted "James Clark Ross" (G12)
01:51, 23 October 2006 Geni (Talk | contribs) deleted "James Reddy Clendon" (G12)
02:39, 23 October 2006 Geni (Talk | contribs) deleted "Johann Zoffany" (G12 plus derivatives of such)
01:53, 23 October 2006 Geni (Talk | contribs) deleted "John Mercer (scientist)" (G12)
03:08, 23 October 2006 Geni (Talk | contribs) deleted "John Struthers (biologist)" (G12)
01:48, 23 October 2006 Geni (Talk | contribs) deleted "Jones Very" (G12)
02:49, 23 October 2006 Geni (Talk | contribs) deleted "Joseph Pitton de Tournefort" (G12)
03:30, 23 October 2006 Geni (Talk | contribs) deleted "Julio C. Tello" (G12)
03:22, 23 October 2006 Geni (Talk | contribs) deleted "Julius L. Meier" (G12 copyvio and derivatives of)
02:08, 23 October 2006 Geni (Talk | contribs) deleted "Leonard Woolley" (G12)
02:14, 23 October 2006 Geni (Talk | contribs) deleted "Ludwig Binswanger" (G12 whole thing derived from copyvio)
01:35, 23 October 2006 Geni (Talk | contribs) deleted "Martin Grove Brumbaugh" (G12)
01:24, 23 October 2006 Geni (Talk | contribs) deleted "Melvin Jones" (G12)
03:23, 23 October 2006 Geni (Talk | contribs) deleted "Murray Seasongood" (G12)
02:32, 23 October 2006 Geni (Talk | contribs) deleted "Nicolaas Witsen" (G12)
03:10, 23 October 2006 Geni (Talk | contribs) deleted "Octaviano Ambrosio Larrazolo" (g12)
03:36, 23 October 2006 Geni (Talk | contribs) deleted "Olive Higgins Prouty" (G12)
02:53, 23 October 2006 Geni (Talk | contribs) deleted "Olive Schreiner" (G12)
03:39, 23 October 2006 Geni (Talk | contribs) deleted "Pieter Post" (G12 and derivatives)
02:58, 23 October 2006 Geni (Talk | contribs) deleted "Thomas Coke (Methodist)" (G12 + derivatives)
03:03, 23 October 2006 Geni (Talk | contribs) deleted "Vilhelm Hammershøi" (G12 and derivatives)
02:13, 23 October 2006 Geni (Talk | contribs) deleted "William Watts Folwell" (G12 problem is that it is all derived from copyvio version)
01:38, 23 October 2006 Glen S (Talk | contribs) deleted "Charles Camsell" (g12)
01:52, 23 October 2006 Glen S (Talk | contribs) deleted "Charles Manly" (g12)
01:35, 23 October 2006 Glen S (Talk | contribs) deleted "Elizabeth O'Neill Verner" (g12 http://www.tfaoi.com/newsm1/n1m656.htm)
01:32, 23 October 2006 Glen S (Talk | contribs) deleted "Frank E. Lucas" (g12 http://wyoarchives.state.wy.us/governor/lucas.htm])
01:31, 23 October 2006 Glen S (Talk | contribs) deleted "James H. Price" (copyvio http://lawlibrary.wlu.edu/alumni/bios/price.htm)
01:33, 23 October 2006 Glen S (Talk | contribs) deleted "James MacLaine" (g12 http://www.stand-and-deliver.org.uk/highwa...d_plunkett.htm)
01:33, 23 October 2006 Glen S (Talk | contribs) deleted "John Glover (general)" (g12 http://gloversregiment.org/historyglover.html)
01:31, 23 October 2006 Glen S (Talk | contribs) deleted "Lewis Tappan" (copyvio http://www.pbs.org/wgbh/theymadeamerica/wh...tappan_lo.html)
01:38, 23 October 2006 Glen S (Talk | contribs) deleted "Lorenzino de' Medici" (g12)
01:36, 23 October 2006 Glen S (Talk | contribs) deleted "Richard Willstätter" (g12 http://www.britannica.com/nobel/micro/641_3.html)
01:34, 23 October 2006 Glen S (Talk | contribs) deleted "Zechariah Chafee" (cg12 http://www.brown.edu/Administration/News_B...p?serial=C0290)
03:28, 23 October 2006 W.marsh (Talk | contribs) deleted "George Formby (Senior)" (removing copyvio stuff from history (requires a temporary deletion))
01:02, 23 October 2006 W.marsh (Talk | contribs) deleted "Gopabandhu Das" (copyvio of http://www.orissatv.com/peoples.asp)
01:08, 23 October 2006 W.marsh (Talk | contribs) deleted "Henry H. Blood" (copyvio of http://www.media.utah.edu/UHE/b/BLOOD,HENRY.html)
03:33, 23 October 2006 W.marsh (Talk | contribs) deleted "Hulbert Footner" (removing copyvio stuff from history (requires a temporary deletion))
03:31, 23 October 2006 W.marsh (Talk | contribs) deleted "Johnny Burke" (removing copyvio stuff from history (requires a temporary deletion))
03:27, 23 October 2006 W.marsh (Talk | contribs) deleted "Nels H. Smith" (all of this is copied from http://wyoarchives.state.wy.us/governor/nsmith.htm)
03:02, 23 October 2006 W.marsh (Talk | contribs) deleted "Sol White" (killing copyvio portions)
03:08, 23 October 2006 W.marsh (Talk | contribs) deleted "Zénobe Gramme" (deleting copyvio versions)

[Ashibaka]

Although Wikipedia should make itself accountable for this, hosting the copyright violations in the edit history is just as much a liability as you hosting them on your server. The possibility of getting sued is real.

[Daniel Brandt]

Although Wikipedia should make itself accountable for this, hosting the copyright violations in the edit history is just as much a liability as you hosting them on your server. The possibility of getting sued is real.

There are ways it could be done with more elegance. The history could be put in a directory that requires a four-day-old username to read it.

The foul deed was the initial copyright violation, and now it's spread over the entire web through various mirrors and scrapers. From my standpoint, it looks like wiping the history of exactly what happened is comparable to destruction of evidence.

Most likely I don't have standing to serve a subpoena on the Foundation because it's not my copyright that may have been violated. But if I see wholesale plagiarism and copyright violation occurring, I believe that I have the right as a concerned citizen and wannabe investigative journalist to expect that the Foundation will not tolerate the immediate suppression of the evidence. This has the effect of stopping my research in its tracks.

[JohnA]

Oh dear they're hitting the "Connelley Red Button" for all its worth.

Daniel can you make sure with the next lot that journalists are briefed first and you take a copy of the edit histories so we know who the guilty are? (or at least their monikers). Plagiarists tend to be repeat offenders.

Also perhaps some of the copyright holders should be informed to see what their reaction is....

This post has been edited by JohnA: Mon 23rd October 2006, 10:19pm

[Daniel Brandt]

Daniel can you make sure with the next lot that journalists are briefed first and you take a copy of the edit histories so we know who the guilty are? (or at least their monikers). Plagiarists tend to be repeat offenders.

I'm still not inclined to do this. It would have the effect of letting Wikimedia Foundation, Inc. off the hook. Jimbo, Brad, and the trustees and about seven employees of the Foundation want all journalists to go after the Brian Chases of the world, and then Jimbo can sit back and accuse journalists of violating the privacy of poor cyber citizens, the way he accused me of violating the privacy of Brian Chase.

I think Brian Chase was less guilty than Jimbo, but maybe that's just one of my "incoherent rants."

I believe that the Foundation is responsible for Wikipedia. In the case of deletions, which can only be done by admins, it's even clearer that the Foundation is responsible. The Foundation controls the software that decides who gets the admin flag. I want journalists to call up the Foundation and ask them to restore the evidence of plagiarism found in the histories that have been deleted, so that they as journalists can investigate further. If the Foundation says, "It's not our problem," then I want that to be a part of the record.

At least one admin is trying to go back and insert the username of the editor who inserted the questionable material. And I asked Geni on her talk page to stop doing what he/she's been doing. I'm inclined to give it a couple days and then post the rest of the article titles I have, and see what happens to those.

[Rufus]

I urge the Foundation to take steps to curtail the shredding of evidence. These are all from the deletion log, and while some of the revisions were subsequently restored in some cases, it still means that the relevant history is no longer available. Any histories that can be restored by the Foundation should be restored, so that we can get on with our research.

All Wikipedia pages, including history pages, are published with a GFDL notice. As w.marsh said earlier, longstanding policy is for admins to specifically delete revisions with copied text from the article's history. To do otherwise would be intellectually dishonest.

The deletion log is ample proof that these articles were at one time copied. The articles and revisions are being deleted and undeleted not to deprive you of evidence, but because it is standard Wikipedia practice. I think that practice is generally a good idea. If you need any specifics of the deleted histories, I'm sure an admin can be found who will make them available.

[Somey]

If you need any specifics of the deleted histories, I'm sure an admin can be found who will make them available.

You are kidding, right?

[Rufus]

If you need any specifics of the deleted histories, I'm sure an admin can be found who will make them available.

You are kidding, right?

No, although I guessed I might get that reaction. I wouldn't have said I was sure if I wasn't. Now, I'm not saying they'll publicly reveal who they are, but you take what you can get, eh?

This post has been edited by Rufus: Tue 24th October 2006, 5:28am

[Somey]

No, although I guessed I might get that reaction. I wouldn't have said I was sure if I wasn't. Now, I'm not saying they'll publicly reveal who they are, but you take what you can get, eh?

I guess you do!

Regardless, this is beside the point. Nobody in their right mind is going to e-mail a Wikipedia admin, asking for complete revision histories on plagiarized articles, just so that they can learn... what, exactly? The IP or account name of the anonymous editor who plagiarized them? Then what? Are they going to help us actually identify those people, when their policies clearly forbid them to do so?

My own attempts to look into the histories of the affected articles before they were deleted (see earlier in this thread) revealed what I felt were some fairly clear tell-tale signs of, shall we say, questionable provenance that probably should have been caught, and in fact, I daresay they almost certainly would have been if they'd been on more interesting subjects. If I'd been allowed some more time on it, I could have maybe come up with some more tell-tale signs - heck anyone could have. But not now, huh?

Still, you have to take what the software gives you, and the admins have to follow the policies they've agreed to follow. The problem is that the software is inadequate and the policies are wrong. It shouldn't be necessary for Daniel Brandt or anyone else who wants to research the plagiarism problem on Wikipedia to ask anyone for permission to see deleted histories, as long as they have a legitimate reason for it. (How would those revisions be delivered, anyway? Snail mail?)

The way it should work (IMO, natch) is that revisions deleted for those reasons should have a special bit-flag applied to them, and people who are interested in the problem should be able to ask for the necessary rights applied to their accounts that would enable them to see those revisions. Revisions deleted to remove personal info about users, or libel, wouldn't have the bit-flag and would therefore not be visible. But that's not a simple feature to implement, and it adds an additional layer of complexity to an already hard-to-use interface. Also, it wouldn't do much good for people like me who don't have accounts at all, would it?

Of course, Brandt's account is blocked, along with some other accounts he's used, and most of his IP ranges. But it's a moot point - we're all perfectly aware that Brandt isn't doing any of this to help Wikipedia solve its plagiarism problem. He's trying to give people an idea of the scope of it, in real, hard numbers, something Wikipedia has so far been completely uninterested in doing. That's hardly surprising, but regardless, if the reaction of Wikipedia's admins is simply to delete the plagiarized material and go back to business as usual, with no attempt to address the problem beyond that, what does that say about how moral and ethical principles are used to guide their policies? Or how they affect actual day-to-day operations? What does it say about them?

I could be wrong, though... we'll just have to see what happens in the next 2-3 weeks.

Ultimately, Wikipedia is a corporate entity. The fact that it isn't a corporation means nothing - it functions as a corporation, and as such, it's capable at any time of taking on the moral caliber of the least moral among its leadership. Which, I would argue, is probably very low, given the number of people involved, and the law of averages.

This is why people like me say you can't put the general public, i.e., just anybody in charge of a major collaborative knowledge-gathering project which, if allowed to continue, could ultimately destroy traditional encyclopedias, dictionaries, and other professionally-produced publications, not to mention undermining traditional standards in academia. That would be a tragedy, and what's even sadder is that most people probably won't realize what a tragedy it is until it's too late, and all of the quality alternatives to unregulated (not to mention plagiarized) content are gone.

[Daniel Brandt]

What they should have done is replace the article page with that big copyright violation box, and maybe wipe out the history after a reasonable statute of limitations. Already the history, the diffs, and the prior versions of every article are all in the en.wikipedia.org/w/ directory instead of the en.wikipedia.org/wiki/ directory. This means they are disallowed to all bots as per en.wikipedia.org/robots.txt:

User-agent: *
Disallow: /w/

Even bots that don't care about robots.txt wouldn't bother with prior versions. All they want is some content to carry AdSense, and there's plenty of that just lying on the surface, waiting to be abused. If these bots come across a big violation box, they'll discard it and move on.

It seems disingenuous to me when admins get righteous about how they are required to wipe out histories because they don't want to get sued. What they're really trying to do is destroy evidence so that culpability is obscured. For the articles I listed on my plagiarism page, most of them have been on Wikipedia for many months. That means each one has probably been scraped about 100 times by now. Remember, my list was purged against 965 scraper domains I compiled that did not credit Wikipedia or have the word "wiki" on their page. That was before I eyeballed what remained and ended up purging even more. Now add the ones who do credit Wikipedia like they're supposed to (which I didn't even see because Google took them out for me), and you get an idea of what it means to lock the copyright barn door after the horses are gone.

It's clear to me that the instant-history-wipeout feature is designed to protect admins asses, and at the same time to give them enhanced feelings of potency -- a typical Wikipedia solution.

[Uly]

I'm not sure I understand the discussion going on now.

After being notified of a crime in progress, the proper thing to do is to let the crime continue?

If these articles were left accessable to the public, I know I'd be pretty amazed by the flagrant disregard of copyright.

As it is leaving them admin-accessable is probably a bad idea: it's still redistributing the content, even if it's to a smaller audience, and it serves to reinforce the 'admins are part of wikipedia' position that the foundation is trying to distance itself from lately.

[Somey]

I think what ought to happen, at some point, is that some sort of colloquium of copyright holders should be convened to discuss the extent to which this is a serious problem, and how best to deal with it.

Let me put it this way: If I were the publisher of a traditional encyclopedia, and I learned (as I'm about to, presumably) that some substantial percentage of Wikipedia is plagiarized from my commercial publications, based on a sampling of just 1 percent of the total article count, how would I want Wikipedia to proceed? Remember, they've already removed the plagiarized content from the main article space, and now it only exists in revision histories that hardly any casual browser is ever going to look at.

In fact, I'd want them to keep the history in there, to help me and other publishers develop better strategies for checking and reviewing edits and entire articles for copyright violations. As for the articles themselves, sure, I wouldn't like the fact that they were still available in some fashion, but at least give me time to do some reseach on them, OK? Don't take away all of my options before I'm even aware of what's going on. After all, like the man says, the horses are already out of the barn. And in some cases, I might even be able to track down the source of the plagiarism.

But it's that last thing that Wikipedia fears most - that a user might be prosecuted for something they did on their website. That would have a certain chilling effect, would it not? It might make people think twice about just how anonymous they really are.

Wikipedia has one overarching consideration in all aspects of its operations: Protect the anonymity of users at all costs. Biography subjects, copyright holders, academics, and people in general who care about accuracy of information can all go screw themselves, as far as they're concerned.

[taiwopanfob]

After being notified of a crime in progress, the proper thing to do is to let the crime continue?

The usual way things proceed is like this:

1. stop crime
2. gather evidence
3. prosecute

The Wikipedia way appears to be:

1. stop crime
2. destroy evidence
3. deny responsibility

Brad Patrick has said that individual contributors are to be held accountable for their actions, not WP. (He also said this is clearly published on all edit pages.) Exactly how are current admin actions consistent with this bit of high-level policy? It seems to most people that rather than cranking the digital shredder into high gear, these articles should have been given big, fat, plagiarism banners at the top, along with the name and IP number of the responsible contributors, a reference to the source material and Brad Patrick's instructions to offended parties it's not WP's fault. Keep it all up until someone complains to Brad -- then take it all down.

Ooops! Looks like WP must retain the evidence!

http://library.findlaw.com/1996/Nov/1/231209.html

"During the mid-1980s, courts in three states (Alaska/California/Florida) began upholding the civil tort of spoliation of evidence, permitting the recovery of money damages from persons or companies who destroyed evidence relevant to a civil claim."

Emphasis added. Heh heh heh.


This post has been edited by taiwopanfob: Tue 24th October 2006, 5:14pm

[JohnA]

I urge the Foundation to take steps to curtail the shredding of evidence. These are all from the deletion log, and while some of the revisions were subsequently restored in some cases, it still means that the relevant history is no longer available. Any histories that can be restored by the Foundation should be restored, so that we can get on with our research.

All Wikipedia pages, including history pages, are published with a GFDL notice. As w.marsh said earlier, longstanding policy is for admins to specifically delete revisions with copied text from the article's history. To do otherwise would be intellectually dishonest.

Huh? Intellectual dishonesty was committed by the plagiarists

[Somey]

Hey, would anybody mind if I changed the title of this thread? When it gets cut off on the home page, it almost like it's Daniel doing the plagiarizing...! I was thinking along the lines of, say, "Plagiarism on Wikipedia: The Brandt Project" or something. You know, something official-sounding.



Hey, this is Post #666 for ol' Somey! I'm really goin' to Hell now...