Most people’s concept of the web is the pages they find on Google, social media posts, or their favorite YouTube channels.
Yet, there is a whole other web that’s hidden from view. It’s locked behind passwords and isn’t indexed by search engines. It forms part of the backbone that delivers the surface web we all use. It’s the deep web.
While it sounds mysterious and often conflated with the dark web, most of us access it every day. Surprisingly, it makes up more than 90% of the internet!
Today, we’ll show you how to access the deep web.
What Is the Deep Web?
But first, what is the deep web?
Simply put, the deep web is the private side of the World Wide Web. These pages aren’t indexed by search engines and are often hidden behind passwords or subscriptions. When going directly to them, you might get redirected or see an error message. Even though you already use the deep web, it’s different from the more familiar surface web.
1. Surface Web
Pages that are indexed by search engines are part of the surface web. Examples of this are traditional web pages, blog posts, and forums.
Choosing a video to watch on the YouTube homepage is a wide-open process. Everyone can replicate it. The web address of that video and the page it resides on, for the most part, is the same for everybody.
To access the surface web, you only need an internet connection and a web browser. There might be slight differences in the page layout depending on your geographical location or device. But whether you’re in Japan, the United States, or Sweden (perhaps not always China), anyone can go directly to the video.
And once the web crawlers have visited it, people can find it by entering relevant terms in the search engines.
2. Deep Web
On the other hand, the deep web is not open to everyone. If you’ve got a YouTube account, then you’ll be familiar with your backend dashboard. Here you can change settings, manage playlists, and even set up your own channel.
Imagine for a moment that someone could load these pages simply by entering a web address in their browser or find them after a quick Google search. It would be a security nightmare. People’s accounts and personal details would be routinely stolen.
That’s why deep web browsing is private. Through a combination of dynamic delivery, server permissions, redirections, and password protection, only certain people can access certain deep internet areas.
For example, only you can access your online bank account with your password. Not even high-level bank staff can log in. Your password is encrypted. As long as you don’t share it, nobody else knows it.
On the flipside, bank security staff will have their own deep web admin area that the public can’t access. It may permit them to view your transactions in the case of fraud, but there are all sorts of restrictions. They cannot just add new payees or move your money around as you can.
What’s on the Deep Web?
Common deep web websites include:
- Paywall and membership sites
- User dashboards on social media
- Geo-restricted content
- Internal corporations and organizations sites
- Bank and email accounts
- Scientific and academic databases
- Medical records
Interestingly, the deep web makes up around 96% of the internet. The surface web only comprises between 1% and 4. Some parts of the deep internet have never even been accessed by humans. It’s full of coding and programming languages that talk to servers and process data.
Then there are pages on the border between the surface web and the deep web. If you’ve ever worked with WordPress or other content management systems, you will know that there’s a static login page.
Most people don’t know this exists and would fail to enter the correct login details if they accidentally found it. However, unless there are extra security measures, the page is still available for anyone to see. Search engines even occasionally index pages like this.
Due to poor security and permissions, many private pages or pages with no use wind up on the surface.
How To Access the Deep Web?
Even if you’re wondering how to access the deep web, you probably already have. When you go to your emails or Twitter feed, this is deep web access because it’s reserved especially for you.
People can access the deep web on iPhone or Android via a mobile browser, just as they can on a desktop browser. There is no dedicated deep web browser.
Unless you have permission, accessing someone else’s email or social media accounts is illegal. It’s also virtually impossible unless you steal their password or find a security vulnerability within the site. Companies hire white hat hackers to access deep web pages and test for weaknesses, so black hat hackers don’t exploit them.
Other deep web sites are accessible by anyone but are removed from the surface and can’t be found through search engines. Some local library websites operate this way. To find a book in their catalog, you must first browse the site with an internal search function.
Whatever you search will create a unique query of their database and serve a dynamic page of results. Even the page for a book title does not have a static page and cannot be navigated by clicking through links from the homepage. You can only access it through a deep web search.
Major web search engines like Google and Yahoo can only crawl and index pages through links. These provide the path, while the link’s text (anchor text) helps define the topic. Generally, the more links pointing to a page, the higher it’s ranked in the results.
If there are no links or the site has purposely blocked search engine bots, the page falls into the deep web.
Geo-Restrictions: How to Access Deep Web Content
There are areas of the internet that are considered the deep web for some but not for others. For example, heavy internet restrictions in China have walled off large chunks of the internet that people in most other countries can freely access. Localized search engines don’t serve these links.
A common experience is geo-restriction, where news sites and streaming services serve different content to different users based on their geographical location. Or block access altogether.
Fortunately, this content is not entirely out of reach. Using a VPN service or proxy can make it seem like you are in the required geographical area. Although it’s frowned upon, many people outside the United States use such tools to access American content on Netflix and other streaming sites.
Deep Web vs Dark Web
In recent years the dark web (sometimes called the darknet) has received considerable attention thanks to the growth of illegal marketplaces like the Silk Road and its copycats. These are known to deal with drugs, firearms, and stolen data. It’s fostered the image of an anonymous and lawless wild west, full of criminals and abhorrent content like child pornography.
Unfortunately, there is some truth to this. A 2016 paper found that of 2,723 dark websites analyzed, 57% were based around illicit content and illegal activities.
You’d be remiss to think it’s all bad though, as whistleblowers, journalists, the military, and intelligence agencies also use the dark web for the greater good while protecting their identity.
While the term dark web has been used interchangeably with the deep web, it is technically a subset. The dark web only makes up a tiny fraction of the deep web.
Dark Web Browser
Both the dark web and deep web are not indexed by regular search engines. The critical difference is that most of the deep web can still be accessed by a regular web browser.
The same cannot be said for the dark web, as it uses additional encryption and its own protocols. It also has its own top-level domain, the .onion. The underlying IP address and server location are hidden, and you also need a specific web browsing software called Tor (The Onion Routing) to open it. You cannot access Onion sites without Tor safely.
Furthermore, when you use Tor, there is no comprehensive search engine to find dark websites. Some directories and sites mimic the search experience, but these tend to focus on niche topics. They are often outdated and certainly don’t index as broadly as Google.
Part of the dark web’s nature is to change addresses and move things around for extra security. Maintaining accurate search engines is an enormous challenge.
It’s much more common for users to learn the precise address of an Onion site and then visit it directly. Ironically, you can find many Onion URLs on the surface web, but you can’t use them there.
Tor utilizes a series of proxy servers maintained by volunteers to mask what sites you visit. Your IP address cannot be identified or tracked, and no browsing history is generated. Doing this creates a slow and unreliable browsing experience. But those who use the dark web tolerate the trade-off to protect one’s identity and the ability to partake in illegal activities.
Tor also permits surfing the deep web areas you have access to and the regular surface web anonymously as long as you don’t log in to personal accounts. It’s similar to using a VPN service, and applying both solutions gives you as much security as you can get.
How to Access the Deep Web without Tor
If you want to access the dark side of the deep web without Tor, you can use a dark web proxy site. These are sites that you can visit on a regular browser like Chrome that connects to the Tor network. 4everproxy.com provides this service.
Please note that these browser-based proxies cannot provide the same level of anonymity and can make the process even slower. They aren’t as safe as Tor because they can log who has used them.
Is The Deep Web Illegal?
You may be wondering, is it illegal to access the deep web? Sometimes yes, sometimes no.
Just because a page is not accessible via links or hasn’t been discovered by search engines doesn’t mean it is private or illegal.
You can consider our previous example of a dynamic local library catalog as deep web content, but it is still open to the public. Likewise, unknown and partially hidden login pages are only illegal to access if you attempt to crack the username and password. It’s rare, but you could accidentally stumble on such a page with no mal intent.
When is the Deep Web Illegal?
Accessing the deep web is illegal when you don’t have permission. In effect, this is hacking, though prosecution usually rides on the hacker downloading or changing data on the system. This is especially true if the intention is fraud or blackmail.
For example, It’s illegal to access someone else’s email account. Things get serious fast if you take information from those emails for financial gain.
Of course, if you access your own emails, that’s not illegal, but it’s still an example of using the deep web.
Deep Web vs Dark Web Illegality
Even venturing into the dark web is not inherently illegal. Despite popular belief, there are Onion sites that don’t host or engage in criminal activity. You only cross the line when you actively break the law, such as buying false identity documents in an illegal marketplace.
There are also some gray areas. A whistleblower leaking information to a journalist might be technically breaking the law, but they might be exposing wrongdoing at the hands of a corporation or arm of the state.
Wrap Up
Today we posed the question: what is the deep web? You should now have a deeper understanding of the enormous amount of web content that isn’t known to search engines or the average user. You now know how to access the deep web and why it exists in the first place.
We also explored the difference between the dark and the deep web and how the dark web takes anonymity to the next level. While the dark web is a subset of the deep web, the terms are not interchangeable.
Timeline Of The Article
Sirisha is a networking enthusiast with a bachelor's degree in computer science and engineering. She has developed a keen interest in Virtual Private Networks (VPNs) and has gone the extra mile to gain firsthand experience and knowledge.