Web crawlers are computer programmes that browse the World Wide Web systematically, mechanically, or in an organisedway.Crawling the web is an essential method for learning about and keeping up with the ever changing internet. Numerous online sites are updated often. This project presents an overview of web crawling, including onion routing and Tor network utilisation for both the surface and deep webs. This is done to monitor and preserve onion links and to keep up with the large deep web and to keep track of some illegal websites within it.