Pentest tricks for faster, wider and greater engagements

Blog post 6 December 2018, by Rick van Leeuwen, security specialist at Secura
Some of our technical security specialists attended the security conference Hack.lu on 16th- 18th October, to gain knowledge and to learn some new tricks and tools. In this blog post, Rick van Leeuwen reports about the talk 'Pentest tricks for faster, wider and greater engagements' by Thomas Debize.

In his talk, Thomas Debize begins by explaining how the development of tools and techniques has helped pentesters in gathering more data in less time. First of all, we can now scan entire IP spaces in just hours or days. This can be achieved by using tools such as Zmap and masscan. In addition, without even having to actively scan your targets, you can still retrieve loads of information by querying third party services like Shodan or ZoomEye. These services scan all devices on the internet 24/7 and provide a searchable database with all kinds of information about your targets.

Also, the development of OSINT (Open Source Intelligence) has grown and nowadays there are countless tools available to gather information about companies and people. An extensive list can be found here: https://github.com/Ph055a/OSINT-Collection  

Not only information gathering and scanning has greatly evolved. We also see automation of the exploitation and post-exploitation of Windows environments increase. For example, Bloodhound can be used to get a complete overview of the Active Directory structure, users and privileges. This makes is easier for the pentester to direct his attacks to relevant machines and users.

For the exploitation phase we can use Responder to gather Windows credentials or use CrackMapExec to scan and exploit SMB services to compromise Windows machines. After getting the initial foothold, there are tools for post exploitation. In this phase, the pentester tries to gain access to more machines and user accounts, preferably elevating the privileges. Mimikatz is a tool that really helps in this phase, by gathering plain text Windows credentials from memory.

Why do we need to adapt our techniques?

  • More and more tools are being developed and nowadays they are reliable;
  • We are required to cover wider scopes as more (business) processes are automated and more devices are connected to the internet;
  • Large amount of data, gathered by tools and manual testing, have to be analysed.

Companies are scaling their digital assets, so as a pentester you need to scale your tools and techniques.

The power of CSV

Thomas recommends to process all obtained data in CSV format. The three main reasons for this are:

  1. Most tools used by pentesters offer CSV output;
  2. CSV is human readable;
  3. CSV can be easily queried like a database.

Querying CSV can be done with Excel Power Query which allows you to run advanced queries over a dataset. For unix there is 'csvkit'. This is a collection of command-line tools for formatting and analysing CSV files. It can print statistics, use SQL-like join queries to merge CSV files and other SQL queries to search and analyse the data. With these tools, your simple CSV files now have the power of full featured databases.

GNU Parallel

At Secura we often use big lists of URLs, hosts, IP addresses and more which we want to process through tools or scripts. Most of these tools are not really efficiently using the CPU. Also, when you create a quick and simple script for a simple task, you don’t want to spend time on implementing parallel execution or multithreading. This is where the tool GNU parallel comes in handy. This tool can parallelise any command. So instead of processing large lists one by one in your tool, you can save a lot of time by using GNU parallel.

Thomas gives a nice example how this can be useful. He uses parallel on the URL discovery tool 'wfuzz' with a large list of possible URLs. The tool simultaneously checks multiple URLs instead of doing it one by one, greatly reducing the required time. When this is running, he also starts parallel with the tool ‘webscreenshot’ to take a screenshot on the URLs that ’wfuzz’ found. The result is a list of available URLs, with a screenshot of the page.

In another example Thomas described how you can use parallel with native tools to replace other tools which require complete libraries or frameworks to be installed. Using parallel with 'dig' for instance can replace GoBuster for DNS brute forcing.

The mentioned tools and techniques will not give you the most interesting results to obtain the ultimate goal of your pentest. However automating the simple tasks and processing the data in an efficient way means that you have more time to focus on the real interesting and customer specific parts of the pentest.

You can find the recording of Thomas’ talk, with a lot of demos and script examples on youtube: https://www.youtube.com/watch?v=mZ0OUJmkIlA

At Secura we believe it to be important to train our own experts to the highest standards, but also externally share this security knowledge. We offer on a regular basis various training sessions, designed to bring you up to date with the latest knowledge suchs as the workshop by OSINT guru Arno Reuser as part of the OSINT Information Gathering Masterclass.

Other blog post:

@Secura 2018
Webdesign Studio HB / webdevelopment Medusa