Still Googlebot problems

[krisp]

Still getting a lot of:

66.249.65.7 - - [23/Jan/2010:04:00:45 +0100] "GET /ja/sv/showthread.php?t=6853&page=2 HTTP/1.1" 200 195783 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I can see Google already indexed such wrong links (from the time when vbseo canonical was disabled because it didnt work) and perhaps sometimes are revisiting some of them. However this particular url dosent seem to be in the index already so Google is getting it from somwhere.

Try and go to the link ( お気に入りの開幕戦-ページ2 ) and mouseover the flags, you will see wrong urls in the page source.

Some links on such a page are normal some are not (search for /ja/sv in source). Going to a double flag link will go to the first language in the double folder and the canonical link is corrosponding to the first language folder in the url = the correct language. So the bug is not so bad - hopefully Google will adhere to the canonical url. But still it is a little messy

1. We need to find the cause how such links arise about and prevent it from happening.
2. We need to correct flag links once Googlebot reach such a page because it already have such urls in the index (meaning resolving 1. will not prevent 2.).

vbetflag- issue seems to be ok

[vBET]

This is not vBET issue anymore. As you wrote Google takes it from its own resources - vBET is not generating such links anymore (if you find it does - please note it and we will correct it, but as we know now this issue is solved).

And the flags are correct - in this link ja is language and the rest is treated as normal URL, so sv is just directory, that is why it is included there.

You have to go to your .htaccess file and add rule which will redirect all such requests to request with single lang code. After a while Google will update it and issue will be gone.

Here is such rule (not tested since on our forum such URLs are not working):

RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(.*)?$ http://%{HTTP_HOST}/$1/$3 [L,R=301]

Please add it before vBET rules and test it immediately. After some while (in Google it can be up to few months) this rule will not be necessary anymore so you will be able to comment it and see does issue comes back or not.

Let us know does it is working

[krisp]

Thanks Rewrite rule redirects to:

/ja/showthread.php

(missing what comes after php)

Using htaccess:

RewriteEngine On

RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(.*)?$ http://%{HTTP_HOST}/$1/$3? [L,R=301]

RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/$ index.php?language=$1 [L,QSA]

RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(.*)?$ $2?language=$1 [L,QSA]

RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

[krisp]

Works without the "?" in the end. Thanks!

[vBET]

Yes - that was just a typo - sorry for that and glad you found it

I will correct it in previous post so others will see good solution immediately