The Wikipedia Review: A forum for discussion and criticism of Wikipedia
Wikipedia Review Op-Ed Pages

Welcome, Guest! ( Log In | Register )

 
 Reply to this topicStart new topic
> Google.com, Yahoo.com etc reading this board?, What does this mean?
blissyu2
post Fri 10th March 2006, 12:34pm
Post #1


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


I have bookmarked the main page, and when I click on it sometimes it says that there are e.g. 1 member and 2 guests reading, and it says "Blissyu2, Google.com, Yahoo.com". Why does it say this? What does it mean? Does this mean that Google.com and Yahoo.com spiders are reading the forum or someone is doing a search and finding this page via their caches or what? Does anyone have any idea what it means?
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Blu Aardvark
post Fri 10th March 2006, 1:42pm
Post #2


Gone
*****

Group: On Vacation
Posts: 907
Joined: Sun 26th Feb 2006, 10:46pm
Member No.: 2

WP user page - talk
check - contribs


I wasn't aware that non-admins could view that, but yes, that's exactly what that means. A search engine bot is indexing the forum.

Google.com got here first. About a week later, Yahoo showed up. And MSN just now started poking around. But yeah, it's just shows the bot activities.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Fri 10th March 2006, 2:13pm
Post #3


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


Yeah, it's search engine spiders
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Sat 11th March 2006, 1:51pm
Post #4


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


I wouldn't mind it so much if we could click on them. There was a new one there today. Wisenut.com. I'd really like it if they were clickable, so we could spy on them back. http://www.wisenut.com/
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Sat 11th March 2006, 6:46pm
Post #5


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs














































Bot NameHitsLast Hit
Google.com
1497
11th 3 06, 5:19pm
Yahoo.com
814
11th 3 06, 4:28pm
Wisenut.com
11
11th 3 06, 3:11pm
Cobion.com
1
11th 3 06, 10:28am
MSN.com
40
10th 3 06, 11:18am
Nextopia.com
1
9th 3 06, 5:57pm
Ask Jeeves
2
7th 3 06, 3:28am
Google.com
754
5th 3 06, 7:03pm
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Sat 11th March 2006, 7:07pm
Post #6


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


If Googlebot is hitting us 1497+754 times (2 bots?) = 2251, then how come we seem to be banned from Google? What does this mean?
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Sat 11th March 2006, 7:17pm
Post #7


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


No idea

http://google.com/search?q=site:wikipediareview.com
http://google.com/search?q=site:wikipediar...proboards78.com

Maybe it takes a while for it to organise the pages in its database once it's spidered them or something

as for this, http://google.com/search?q=wikipedia+review , I think it's probably because we're a new site we're ranked lower than wikipedia and the other sites

http://google.com/search?q=wikipediareview and http://google.com/search?q="wikipedia+review" still show us tho
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Sun 12th March 2006, 6:40am
Post #8


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


Yes. I think that we want to be searchable on Google. It'd be good to advertise this site and such and for people who are interested in certain topics to be able to find one that is relevant to them.

Of course, we don't want Google Adsense, or any ads on the pages, but I don't think that the two are connected.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Lir
post Sun 12th March 2006, 7:00am
Post #9


Communist
*****

Group: Inactive
Posts: 978
Joined: Sun 26th Feb 2006, 10:27pm
Member No.: 4
I'd rather not see any ads here; as far as advertising this site, I've linked to it from the top of my kapitalism.net article -- and I think most anyone looking for criticism of wikipedia eventually finds my article. Don't try to rush publicity, with time you'll get better results on search engines.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Sun 12th March 2006, 11:57am
Post #10


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


-- http://wikipediareview.com/indexsitemap.xml.gz
----> http://wikipediareview.com/topicsitemap.xml.gz
----> http://wikipediareview.com/forumsitemap.xml.gz

Those have both been up for a while and the site's been registered on Google Sitemaps for a while

The files also automatically get updated via the forum

Maybe Sitemaps is what's actually causing the problem, I dunno
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Sun 12th March 2006, 12:10pm
Post #11


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


A big time no on Google Adsense, or any advertising on here. It lowers the quality of a site if it has ads. My web page used to be ad-free but now is spammed with ads, so much so that silly Malber suggested it had a virus on it. We don't want the same situation here. If we are going to advertise, do it legitimately. Sell coffee mugs etc if we want to, but no spam ads.

When I clicked on Selina's links, I got "montly". Monthly has an "h" in it. Is that what is causing the problems?
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Sun 12th March 2006, 4:37pm
Post #12


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


hmmm I think I'd probably be better off just letting the google bot crawl then and removing sitemaps - since the mod for the forum that makes the sitemap obviously doesnt work properly and it'd be a waste of time to add a link to every new topic every time there is one
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
God of War
post Sun 12th March 2006, 5:03pm
Post #13


New Member
*

Group: Contributors
Posts: 43
Joined: Mon 27th Feb 2006, 6:34am
Member No.: 6
http://search.yahoo.com/search?p=wikipedia...1&cop=&ei=UTF-8

It seems that we are now number 10 on the Yahoo search page.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Mon 13th March 2006, 9:49am
Post #14


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


QUOTE(God of War @ Mon 13th March 2006, 2:33am) *

http://search.yahoo.com/search?p=wikipedia...1&cop=&ei=UTF-8

It seems that we are now number 10 on the Yahoo search page.


Number 5 now, with the old forum at number 4.

We should be number 1 though, at least in searches for the actual site.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Mon 13th March 2006, 10:06am
Post #15


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


Yeah, I think google ranks older sites higher basically - makes sense, since it stops people creating tonnes of spam subdomains and linking each other etc - I've noticed this on google don't know if anyone else has, some things seem to automatically make subdomains with certain keywords and get listed in google but they're nearly always at the very end of searches (unless there's no other results)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Blu Aardvark
post Tue 14th March 2006, 1:39am
Post #16


Gone
*****

Group: On Vacation
Posts: 907
Joined: Sun 26th Feb 2006, 10:46pm
Member No.: 2

WP user page - talk
check - contribs


QUOTE(qwerty @ Sun 12th March 2006, 8:11am) *

No, I guess it's not because of that. I guess it's because of the format of the files. On the Google Sitemaps help it says clearly that the feed file has to be RSS 2.0 if it's RSS. The file also has to have <link> and <pubDate> tags.

The feed isn't RSS, however. It's an XML document using the Sitemap Protocol.

QUOTE(qwerty @ Sun 12th March 2006, 8:11am) *
And I don't know if the .gz extension at the end is also affecting the thing.

Possibly. There shouldn't have been a .gz extension present, because gzip'ing of the sitemap documents was disabled. I've enabled that now; we'll see if that helps.

Editted to add Do we have a google sitemaps account? We need one.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Selina
post Tue 14th March 2006, 9:54am
Post #17


Cat herder
******

Group: Staffy
Posts: 1,513
Joined: Sun 19th Feb 2006, 10:28pm
Member No.: 1

WP user page - talk
check - contribs


We have a google sitemaps account yeah. I disabled sitemapping though to see how it goes - I think quite possibly googlebot will crawl the pages better on it's own than relying on that mod which doesn't make proper sitemap syntax files - when the sitemap was enabled it was set up to read the files and has an account, yeah
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
blissyu2
post Tue 14th March 2006, 10:57am
Post #18


the wookie
*********

Group: On Vacation
Posts: 4,596
Joined: Mon 27th Feb 2006, 12:14am
From: Australia
Member No.: 5

WP user page - talk
check - contribs


Well, while popularity isn't everything, I think that we do want to be listed on search engines. We want people who are interested in talking about the pitfalls of Wikipedia to know that there is somewhere that they can go. And this should include journalists and more well known critics as well as just regular users. Ultimately, we don't want this to be mostly filled with people trying to destroy the forum. I don't know if it will always be 50/50, but long term I'd hope that we are mainly working towards the criticism of Wikipedia element rather than criticism of criticism of Wikipedia! LOL.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Blu Aardvark
post Fri 17th March 2006, 7:31am
Post #19


Gone
*****

Group: On Vacation
Posts: 907
Joined: Sun 26th Feb 2006, 10:46pm
Member No.: 2

WP user page - talk
check - contribs


QUOTE(Selina @ Tue 14th March 2006, 1:54am) *

We have a google sitemaps account yeah. I disabled sitemapping though to see how it goes - I think quite possibly googlebot will crawl the pages better on it's own than relying on that mod which doesn't make proper sitemap syntax files - when the sitemap was enabled it was set up to read the files and has an account, yeah


When the sitemap was set up, it didn't list any forums to index, and therefore, none were indexed. Since that has been changed (you have to list them by forum id, comma seperated, in the ACP options for Sitemapping), google is indexing us. Whether or not it was the sitemap config, or something else, I dunno. But we're there now.

http://www.google.com/search?q=site:wikipediareview.com
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
 

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 

-   Lo-Fi Version Time is now: 19th 6 13, 6:23pm