Deep web is not dark web!

WWW (World Wide Web), A tangled web of opportunities, entertainment, communication, and news etc. Supposedly, there are 362.5 million top level domain names (TLDs) registered [Source: VeriSign]. Yet, sometimes we cannot get relevant information easily. That’s not because it is not there on web but because it is not accessible easily. Certainly, we access information through search engines like Google, Bing, Yahoo, etc. These search engines usually, skim very small part of data that is nearly 0.03% of whole web. That’s the data web we have access through easily. This part of web is called surface web. Rest of the others is part of The Deep Web

Search engines and Deep web

To understand why can’t we access Deep web easily, One needs to understand how search engines work. For different search engine there is different way to access data. But usually, they follow same process. They have their automated bots called spiders which skims data starting from popular sites. The spiders read the content of the sites and follow the hyperlink provided in the site, leaving strings of path and weaving or creating a web or map. This whole process is called Web Crawling.

After web crawling, Search engines store the content which is found during web crawling. This is done by reading content, archives, and meta tags of the site. This process is called indexing.Next, the search engine defines a density to a particular domain name for a particular word. With the help of this density of keywords the ranking of the website is decided. This whole process of indexing and ranking is done after every web crawling. For building a search it uses these rankings for every word search and with the help of Boolean operators it tries to find the relevant pages.

With the huge number of pages and their sub pages, there is a huge problem of Big Data. The data is so huge that even computers of big corporations like Google, Microsoft cannot manage. The whole search analogy is dependent on the bots. Hence, for whatsoever reason, if bots couldn’t get to access the page, the pages will not be indexed and not listed in the search rankings making pages difficult to access.

What is part of Deep Web?

Often people tend to get confused between Deep web and Dark web. These are two different things. As we know, the pages which aren’t indexed goes out of the sight of search engine. Sometimes sub pages of catalogued sites can’t be accessed easily. For instance, especially on government sites we try to find the some sub pages from search engines. But we don’t get it through search engines so when we try to search it from the government site then we get it.

Sometimes, the sites have their pay wall such that one can access the site until they pay for it. Therefore, web spiders don’t get access to it and the site does not get indexed. It is done for the sites which publish research papers. These are the kind of sites which are known by professionals and used by them only, e.g. If we search HIV we don’t get relevant research medical papers but we get reporting of various media channels.

Sites which are not indexed are usually difficult to find. E.g. If someone searches for your bank details, he/she does not find it online as it is password protected. Moreover, there are chances that owners, especially corporations purposely don’t catalogue their pages. This is because these portals are used by company employees for their internal work. The domain names of these portals are known by the employees. With the help of the domain name, they are able to access the management portals.

However, there is gloomy and darker part of deep web where also the data is purposely hidden by the user which takes us to the Dark web.

Deep web vs Dark web

Dark web is subset of Deep web. Consequently, the dark web is formed from the shadow of deep web. The Dark web gives and maintains anonymity to users. It unleashes a lot of power. It is just unbelievable. Before going to the bad aspects of it, let’s talk about the good ones. In this era, surveillance is everywhere just as privacy is not a thing. It helps like-minded whistle blowers of the oppressive and violent countries to communicate. The Dark web has it’s own versions of search engines and social media, accessing these is not criminal activity at all. They don’t track your activity and don’t give you stream of advertisements based on search at all.

The problems comes with the highlighted part of the dark web which everyone talks about. Mostly Dark web is used for illicit and child pornography. Moreover, it is also used for drug supply, human trafficking, and weapon supply. Even credit cards numbers, Netflix accounts, fake passports, fake college degrees, stolen Uber accounts are also for sale on the Dark web. The Dark web cannot be accessed through normal browser. Normally, the dark websites don’t use .org, .com, .co, or other top level domain. Instead they use .onion domain. These sites are accessed by the TOR (the onion project).

Tor Browser

It is a special equipped browser which maintains your anonymity online. It doesn’t track its users by Search Engines, Websites, ISPs, Social media sites, etc. The Tor browser always applies three levels of proxy. It randomly connects to a public node. Thereby, the traffic is sent to other nodes by a random path of three levels and then the nodes connect to the website. Just like layers of onion, if one could peel one layer off, there is yet another layer. Data is encrypted layer by layer, thereby making sites and ISPs to track the user very difficult. The .onion sites are always accessed through the Tor browser as these allow only the private peers and doesn’t connect to the peers of the ISPs. Often people also use VPNs to add another level of security.

There are some disadvantages of tor that you should know before using it. Certainly, it gives anonymity to the user but the data transferred through browser is not encrypted. Therefore, it is very unlikely for a person to use http instead of https website on tor. For this reason, a person should not use tor in internet banking. However, if one has to simply browse internet and don’t want a stream of advertisement based on their search, tor is best.

Bitcoin

As we have known, the dark web involves monetary transactions. The transactions that are carried out using the virtual currencies called cryptocurrencies. Cryptocurrencies like Bitcoin provide obscurity to the user. It exists in the online world of the mathematics equations and encryption and is not managed or issued by any kind of Central Banks of States, which makes it much decentralized. Sending and receiving Bitcoins is as easy as sending emails. Bitcoin works on peer to peer kind of network. It doesn’t have a server, instead it uses its user’s device as a server. This makes it free of regulation and monitoring which makes it favorable for usage.

Deep web and its potential

The dark of the web is illuminated from within.

Anthony T.Hincks

As the data on web is increasing day by day, the Deep web gets deeper everyday. Meanwhile the programmers of search engines are also trying to improve analogy in order to dig out relevant information out of web. This takes us to the problem of big data which is incoherent and unmanageable. This gives great opportunity to the corporation to mine out data from deep web and analyze it. Doing this will help them have an upper hand against their competitor. This gives a great opportunity to evolve technologically in the meantime.