Dark Web OSINT With Python and OnionScan: Part One

You may have heard of this awesome tool called OnionScan that is used to scan hidden services in the dark web looking for potential data leaks. Recently the project released some cool visualizations and a high level description of what their scanning results looked like. What they didn’t provide is how to actually go about scanning as much of the dark web as possible, and then how to produce those very cool visualizations that they show.

At a high level we need to do the following:

1. Setup a server somewhere to host our scanner 24/7 because it takes some time to do the scanning work.
2. Get TOR running on the server.
3. Get OnionScan setup.
4. Write some Python to handle the scanning and some of the other data management to deal with the scan results.
5. Write some more Python to make some cool graphs. (Part Two of the series)

Let’s get started!

Setting up a Digital Ocean Droplet

If you already use Amazon, or have your own Linux server somewhere you can skip this step. For the rest of you, you can use my referral link here to get a $10 credit with Digital Ocean that will get you a couple months free (full disclosure I make money in my Digital Ocean account if you start paying for your server, feel free to bypass that referral link and pay for your own server). I am assuming you are running Ubuntu 16.04 for the rest of the instructions.

1. The first thing you need to do is to create a new Droplet by clicking on the big Create Droplet button.
2. Next select a Ubuntu 16.04 configuration, and select the $5.00/month option (unless you want something more powerful).
3. You can pick a datacenter wherever you like, and then scroll to the bottom and click Create.

It will begin creating your droplet, and soon you should receive an email with how to access your new Linux server. If you are on Mac OSX or Linux get your terminal open. If you are on Windows then grab Putty from here.

• On Mac OSX it is: Finder -> Applications -> Utilities -> Terminal
• On Linux: Click your start menu and search for Terminal

Now you are going to SSH into your new server. Windows Putty users just punch the IP address in that you received in your email and hit Enter. You will be authenticating as the root user and then type in the password you were provided in your email.

Due to the large amount of code and the special formatting required please continue reading this post at its original source here.