|FAQs of LitBot by Larsen IT
|1. What is LitBot?
||LitBot is the Larsen IT search engine application which is used to visit pages on the internet, analyze the pages and build information into LitIndex. This will help users decide wisiting the content information on your site based on the LitBot Algorithm.
|2. Will LitBot create additional loading to my servers?
||In order to ensure the search results cover most of your content, LitBot will keep the crawling at a certain level. We have been trying our best to avoid increasing the load to your servers by adjusting the frequency based on your server's capability, the quality and the update frequency of your site. If you find any unreasonable access from LitBot, please inform us at email@example.com
|3. How can I prevent LitBot from crawling my site?
||LitBot works on the robots.txt protocol. You can prevent LitBot from crawling your entire site, or specific content as described in the robots.txt
Below example command will block all crawling from LitBot: