Sitemap and robots.txt files are vital elements of websites structure which is helps for websites search engine optimizations (SEO) to provide information’s to web crawlers. These elements necessary to index a website on search engine because of without sitemap and robot.txt crawler not access or finding the path of webpages. in WordPress website, So many plugins are available which automated generate sitemap and robot.txt for your website and add them on website if you installed the plugin in WordPress admin but if other sites like custom coded which not have option to any plugins that why you need to sitemap and robot.txt file manually. There are many websites owners not know about how to add sitemap and robots.txt in website who can easy access their PHP or custom code websites to add sitemap and robots.txt through this blog post.
What is sitemap and robots.txt?
In simple language, Sitemap is a list of pages which contains all the information of the website like how many pages in website, and what path of images and media.
It’s an xml file which is kept in the root folder of the website host. This is very important for any website because without sitemap Google’s bots can not crawl your web-pages. So any website require sitemap so that when the crawler come into your website to crawl then they can get all the information’s.
Just as a person needs a roadmap to reach his destination, Google needs a sitemap to reach your website
What is robots.txt in SEO?
Site owner can provide the permission to Google by robots.txt which pages to crawl and which not to crawl. The robots.txt instructions to Google crawlers for which web page need to crawl or not.
This is an simple code file that can be created accordingly.
Example: User-agent: *
Disallow: /search
Allow: /
Sitemap: https://www.thetechtweed.com/sitemap.xml
Does sitemap is mandatory in robots.txt?
No, it’s not mandatory in robots.txt file but Google crawlers first check your robots file before crawling then if there sitemap included then you will get extra benefits because they can easily crawl your pages.
How to generate Sitemap and robots for PHP website?
1. Open Sitemap generator website – https://www.xml-sitemaps.com/
2. Put your website URL which for you want to generate and press start. And then view generated file and download it.
3. You will get a sitemap.xml file.
Now open your hosting panel and go file manager >>public.html and upload it. Now you can check your sitemap have been uploaded in root folder
For robot.txt you can simply copy my above robot and replace your domain name with my domain.
The upload process is same as sitemap but you need to open notepad and paste the code. The file name should be robot.txt.
Summary – We explained the process how to generate sitemap or Robots.txt and how to upload them website? hope you understand well if you get any error then you contact us by mails or comment.