Szukałem i szukają, ale nie może znaleźć wskazówki na temat treści robots.txt. Używam vBET 4.2.3 z Vbseo 3.6.0 RC 2.
Zakładam, że ja muszę go podobny do was korzysta?
Czy to jedna dobra, czy też muszę ogromny wy jesteście tam gdzie jest zablokować dostęp do każdego pliku w języku?
PHP Code:
# Allow Archiver
User-agent: ia_archiver
Allow: /
User-agent: *
Disallow: /forum/*.js
Disallow: /forum/*.jsp
Disallow: /forum/*.cfm
Disallow: /forum/*.asp
Disallow: /forum/*.aspx
Disallow: /forum/*.cgi
Disallow: /forum/images/
Disallow: /forum/includes/
Disallow: /forum/install/
Disallow: /forum/customavatars/
Disallow: /forum/archive/
Disallow: /forum/attachments/
Disallow: /forum/ajax.php
Disallow: /forum/attachment.php
Disallow: /forum/admincp/
Disallow: /forum/calendar.php
Disallow: /forum/cron.php
Disallow: /forum/editpost.php
Disallow: /forum/global.php
Disallow: /forum/external.php
Disallow: /forum/image.php
Disallow: /forum/inlinemod.php
Disallow: /forum/joinrequests.php
Disallow: /forum/login.php
Disallow: /forum/billspaypal.php
Disallow: /forum/member.php
Disallow: /forum/memberlist.php
Disallow: /forum/misc.php
Disallow: /forum/moderator.php
Disallow: /forum/newattachment.php
Disallow: /forum/newreply.php
Disallow: /forum/newthread.php
Disallow: /forum/online.php
Disallow: /forum/poll.php
Disallow: /forum/postings.php
Disallow: /forum/printthread.php
Disallow: /forum/private.php
Disallow: /forum/profile.php
Disallow: /forum/register.php
Disallow: /forum/report.php
Disallow: /forum/reputation.php
Disallow: /forum/search.php
Disallow: /forum/sendmessage.php
Disallow: /forum/showgroups.php
Disallow: /forum/subscription.php
Disallow: /forum/threadrate.php
Disallow: /forum/usercp.php
Disallow: /forum/usernote.php
Disallow: /forum/faq.php
Disallow: /forum/vbseo.php
Disallow: /forum/vbseocp.php
Disallow: /forum/vbseocpform.php
Disallow: /forum/chat.php
Sitemap: http://www.mydomain.com/forum/sitemap_index.xml.gz