Important: This page is using cookies (cookies). Using this website without turning off cookies in browser, means that you agree for using it.
Buy Now! Features Downloads

Earn with us!

If you would like to start earning money with vBET join to Affiliate Program.
Results 1 to 4 of 4

Thread: Robots.txt for vbseo-enabled forum running Vbet 4.2.3

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Senior Member
    Join Date
    Feb 2011
    Posts
    138

    Default Robots.txt for vbseo-enabled forum running Vbet 4.2.3

    I've been searching and searching but can't find tips on the content of my robots.txt. I'm running Vbet 4.2.3 with Vbseo 3.6.0 RC 2.

    I assume that I need one similar to the one you guys are using?


    Is this one okay or do I need the huge one you guys are using where you block access to each file by language?

    PHP Code:
    # Allow Archiver
    User-agentia_archiver
    Allow
    : /

    User-agent: *
    Disallow: /forum/*.js
    Disallow: /forum/*.jsp
    Disallow: /forum/*.cfm
    Disallow: /forum/*.asp
    Disallow: /forum/*.aspx
    Disallow: /forum/*.cgi
    Disallow: /forum/images/
    Disallow: /forum/includes/
    Disallow: /forum/install/
    Disallow: /forum/customavatars/
    Disallow: /forum/archive/
    Disallow: /forum/attachments/ 
    Disallow: /forum/ajax.php
    Disallow: /forum/attachment.php
    Disallow: /forum/admincp/
    Disallow: /forum/calendar.php
    Disallow: /forum/cron.php
    Disallow: /forum/editpost.php
    Disallow: /forum/global.php
    Disallow: /forum/external.php
    Disallow: /forum/image.php
    Disallow: /forum/inlinemod.php
    Disallow: /forum/joinrequests.php
    Disallow: /forum/login.php
    Disallow: /forum/billspaypal.php
    Disallow: /forum/member.php
    Disallow: /forum/memberlist.php
    Disallow: /forum/misc.php
    Disallow: /forum/moderator.php
    Disallow: /forum/newattachment.php
    Disallow: /forum/newreply.php
    Disallow: /forum/newthread.php
    Disallow: /forum/online.php
    Disallow: /forum/poll.php
    Disallow: /forum/postings.php
    Disallow: /forum/printthread.php
    Disallow: /forum/private.php
    Disallow: /forum/profile.php
    Disallow: /forum/register.php
    Disallow: /forum/report.php
    Disallow: /forum/reputation.php
    Disallow: /forum/search.php
    Disallow: /forum/sendmessage.php
    Disallow: /forum/showgroups.php
    Disallow: /forum/subscription.php
    Disallow: /forum/threadrate.php
    Disallow: /forum/usercp.php
    Disallow: /forum/usernote.php
    Disallow: /forum/faq.php
    Disallow: /forum/vbseo.php
    Disallow: /forum/vbseocp.php
    Disallow: /forum/vbseocpform.php
    Disallow: /forum/chat.php

    Sitemap: http://www.mydomain.com/forum/sitemap_index.xml.gz 
    Last edited by norku; 14-03-11 at 21:56.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •