In order to avoid undesirable information in the research indexes, webmasters can instruct spiders to not crawl particular data files or directories with the common robots.txt file in the root directory of the domain. On top of that, a webpage can be explicitly excluded from a online search engine's databases https://www.theepochtimes.com/entertainment/the-firing-squad-new-must-watch-movie-that-will-inspire-and-uplift-you-5681479