Is default indexing for web crawlers on?

[akamis]

I want Google to index my site so I can use its custom search engine, but my site is not yet in Google's index. I know it takes a while to be crawled, but I thought the problem could also be that the default is to be excluded, e.g., robots exclusion file somewhere. Is the default for indexing by search engine crawlers on (included) or off (excluded)? Thanks.

[sjlplat]

There is no file installed by default to prevent your site from being spidered. If you want to change the way in which search engines index your site, take a look at http://www.robotstxt.org/

[charlesgan]

by default, there will be no blocking to google bots etc.

you will need to seo your site, and submit it to search engines, and also directory. these will helps

[MagicMan]

Hi, I just wanted to confirm something based on the information about the robots.txt file from this URL http://www.robotstxt.org/wc/exclusion-user.html

My question is: I want to confirm that Host Monsters shared hosting plans indeed have the ability for each user to add their own robots.txt file.

Also, will every thing work fine if we put a seperate robots.txt file in the root directory of each of our add-on domains. If so, if there is one in an add-on directory but not on the main domain that our account is set up under, is every thing still okay?

Thanks!
Rick in Phoenix AZ

[r2b2]

Yes to your first question and probably to your second. (I say probably rather than yes for the slight chance it isn't :D)

[sjlplat]

Also, will every thing work fine if we put a seperate robots.txt file in the root directory of each of our add-on domains. If so, if there is one in an add-on directory but not on the main domain that our account is set up under, is every thing still okay?

Yes, each addon domain is treated as a separate domain, in regard to robots.txt.