Last Updated on April 20, 2020
Is there an invisible web?
Did you ever think how much time and effort was required for a web page to be the first when you perform a certain search? When you need information and you seek for it with search engines such as Google, Yahoo, MSN or whatever other, the results are brought to you through an automatic process of matching your search phrase or keyword with sites containing it. But in order for this to happen, those sites need to be visited by robots and indexed beforehand. In order to help robots correctly indexing a site, I, the webmaster, need to put some info there especially for those programs, so called ‘robots’. If I don’t to it properly, you won’t reach my site, regardless how useful it might be to you in that situation. Or if all my relevant info is published as photos, the search engines won’t find my site for you, because robots can’t read photographs, they know only how to handle text information. Nor robots know how to read pdf files. Nor databases. All info you cannot reach by classic searches is the so called invisible web.
How to see the invisible web?
You might want to ask now: so, if there is an invisible web, how can we reach it, though? Besides portals and directories, invisible or deep web, and specialized search engines, social bookmarking sites must be mentioned here.
Invisible web resources
There are many resources available for deep invisible web exploration. For the curious ones, here there are a few of the best ones:
In the article How to Search the Invisible Web, Lifehacker teaches us the basics of a better targeted deep web search.
If you want to know more on specific topics, you can follow the online courses from free-ed. All courses are free and very easy to subscribe to.
The Berkeley University has a comprehensive guide of invisible web: what it is, how to find it and its inherent ambiguity.
At the Web Lens you can find a lot of invisible web research tools which students and researchers might find useful.
Eric Digests present some interesting facts and references for further reading. The article is rather old, but still good.
A thorough, illustrated description of the web search process can be found at JEP (The Journal of Electronic Publishing published by the University of Michigan Press).
This being said, remember to go for the deep web whenever you get frustrated by the surface web search results.
