You can certainly disallow crawling of directories and data files and instruct the robots how you can crawl your internet site.Thank you greatly on your collaboration that helps me a whole lot in my commencing while in the Search engine marketing environment.Google not just suggests applying Lighthouse To judge your website page functionality, but