Alt3rn4tiv3's Tech Blog

Web Application Penetration Testing (WAPT) Vol. 1 - Discovery

18/01/08

11:32:43 pm, by alt3rn4tiv3

, 2129 words,
Categories: Security

Web Application Penetration Testing (WAPT) Vol. 1 - Discovery

Apart from the highly-inactive DOSAM series that I hope to continue writing soon, I decided to write another series about web application penetration testing (WAPT). I’ve learnt pretty much over the past few months and I’d like to share my knowledge and experiences.

So, let’s start off the core of today’s post. Penetration testing requires a good knowledge of your target. The better you know your victim, the safer you are, and the higher chance your victim will fall. Not only can discovery of information about your target give you valuable information about its internal network structure, it can also give you insights as to what type of attacks can be used and which of those might be more successful and / or useful than others.

The first step in the discovery process is looking for the information about your target’s host. Where is it hosted in? Who runs it? Does it have a load balancer? What might its network structure look like? WHOIS is your friend here. Whois servers belonging to Regional Internet Registries (RIR) can be queried to determine the Internet Service Provider responsible for a particular IP address.

These servers are:
ARIN (North America) - whois.arin.net
RIPE NCC (Europe and parts of Asia) - whois.ripe.net
APNIC (Asia Pacific) - whois.apnic.net
LACNIC (Latin America and the Caribbean) - whois.lacnic.net
AFRINIC (Africa) - whois.afrinic.net
And finally, International NIC.

Checking DNS records is also a great way of finding out more information. You can use the following tools -

dig
nslookup
host

Of course, if you’re lazy to visit so many sites and use so many tools, there’s always SamSpade to do it for you.

The next part to the discovery phase is to look out for filters. Knowing your target’s network infrastructure… whether there is any filtering device between your traffic and the actual target… will go a long way. There’s a project out there by PureHacking called Active Filter Detection (AFD). It detects the presence of Intrusion Prevention Systems (IPS) and other technologies that would directly impact the quality of a security assessment. Of course, you could get nmap to do it for you as well.

A load balancer is often in place to divert massive amounts of traffic to different servers for handling. This load balancer can be detected in a variety of ways. There is no generic way for detection, but some of them maintain session information in cookies and some use custom HTTP headers. In the F5 BigIP series of load balancers, you’ll get a cookie saying something like…

Cookie: BIGipServerHTTP-Servers=374349740.20480.0000

Converting 374349740 to binary, we get 10110010100000001111110101100. Breaking it up into…
10110010 = 178
10000000 = 128
11111101 = 253
00001100 = 12
So, we know that the IP of one of the machines that sits behind the load balancer is 178.128.253.12 - isn’t that easy?

Header-based identification is also possible. Typically, infrastructural-level devices can modify HTTP headers to work their magic. Two common modifications are “nnCoection: close” and “Cneonction: close” instead of “Connection: close".

Now also, if your target uses SSL / TLS, you should find out their versions and what ciphers they support. A good tool to use is THC’s THCSSLCheck. Another one is Foundstone’s SSLDigger.

Of course, the next step is to find out what OS your target is using. Most of you should be familiar with this. There are just so many variety of ways you can do this. You can use NetCraft and nmap amongst all the other tools. However, there are 2 interesting programs I would like to introduce to you guys. The first is passive OS fingerprinting (p0f) written by Michal Zalewski. It has a database of unique TCP/IP characteristics for both SYN handshake initiations and SYN/ACK response packets. Unlike active OS fingerprinting, which includes sending uniquely crafted abnormal packets to the remote host, p0f makes its decisions based on the normal requests, since different TCP stacks consistently provide unique responses. p0f doesn’t need to send any data, thus the “passive” in its name. This way, it doesn’t trigger any Intrusion Detection System (IDS). You can perform analysis on the data with p0f stats or store the p0f data in a database.

Another interesting tool is DMitry (Deepmagic Information Gathering Tool). It taps into numerous sources of data and presents them all a nicely formated page, all in one program run. You gotta love this.

So now you know that your target runs on Windows Server 2008, or perhaps an outdated version of Linux. The next step is to find out what web server it’s running. For you guys who only know about the LAMP / WAMP configurations, please remember that there are other types of web servers out there 88| There’s Microsoft IIS, Google GFE (yes, Google has its own web server too! It’s called Google Front End, or GFE for short), Oversee, lighthttpd and amongst others. At this point, I’d like to make a note about Google. Just as California’s wildfires can be seen from space, Google seems to be becoming more and more significant by the days. Watch out for its probably-coming battle with Yahoo over the TrustRank algorithm soon.

Perhaps a useful tool in this case would be httPrint. It can identify web servers whose identities have been obfuscated in some fashion and is excellent at identifying web servers that are protected by proxies. I’ll leave you to find out how to use it.

Right now, you must probably be getting tired of all the boring stuff. The real stuff comes now. There are a few questions you need to ask yourself to identify what applications are running on your target -

What ports are actively listening?
Are the services for each open port identifiable?
Are there error pages that can leak information about the services?
What file types do this target properly handle?
What resources are verified as “existing” on your target?

Port mapping comes in as an important process here. Of course, nmap is our best candidate again. However, there’s another tool called unicornscan which can uniquely scan UDP-based targets.

To identify applications running on your target, use this extremely powerful application-mapping tool amap. With nmap results from the -oM switch (which outputs XML), amap will attempt to identify applications even if they are running on non-standard ports. It sends out trigger packets and then compares the responses against a list of known and confirmed response signatures. Because this tool is a must-have in all pen-testers’ toolkit, it would be good for everyone to submit unidentified signatures to the project to aid further development of the tool.

A similar project to amap is the Open Protocol Resource Project (OPRP). A PERL wrapper script that couples nmap with the OPRP data has also been written - nwrap. You’ll also need the OPRP DB dump.

Database identification is not easy. THC (again) has a tool to help identify the running database if it’s Oracle or DB2, though - THCDBFP.

Analyzing error pages can you identify services running on the web server as well. Look out for differences between the 404 and 500 error pages. Also look out for other types of errors such as “Proxy Error". Try making different types of requests -

valid requests for invalid resources
invalid requests for valid resources
violate protocol, e.g. GGGG / HTTP / 1.0
tamper with parameters in query strings, HTML forms for POST requests too

A very important lesson I learnt from rustylime’s competition is that resource enumeration is very important. Not only do you need to find resources already shown, you also need to find those which are not listed for public eyes.

Of all programs that you can run, there are …

HTTrack
wget –mirror -r -w2 -p –html-extension –convert-links -P /targetdirectory/ http://www.site.com/
Wikto
PureTest
http://www.softbytelabs.com/” target="_blank">Black Widow

A PERL script written by Andres Andreu I found does the “hidden” resource enumeration just fine. Below is the source.

Code:

#!/usr/bin/perl
#
# This simple script provides very basic HTTP file & dir enumeration functionality
# It requires a target host, a file with resource (file) names
# and a file with a list of extensions (PHP, ASP, JSP, HTML, etc).
# 
# Author: Andres Andreu <andres [at] neurofuzz dot com>
# File:   list_web_resources.pl
# Ver:    1.0
#
 
use strict; 
use Getopt::Std; 
use LWP::UserAgent;
 
#Define initial hash
my (%opts);
getopts('f:e:h:p:d:', \%opts);
my (@extarray, @resarray, @patharray,
    @resresultsarray, @pathresultsarray);
 
#Define initial variables
my ($path, $resource, $extension, $pathfilename,
    $resfilename, $extfilename, $host, $depth, $ua);
 
# Create a user agent object
$ua = LWP::UserAgent->new;
$ua->agent("EnumScript");
 
# Usage Statement
sub usage() {
  print "\nUsage :\tperl $0 -h target -d num -f resource_list_file.txt -e extension_list_file.txt -p directory_list_file.txt\n\n";      
  exit;
}
 
sub dig($) {
 
  chop $_[0];
  my $tmppath = shift;
  my $cnt = shift;
  my $path1;
  
  foreach $path1 (@patharray) {
  
    if ($path1 eq "/") {
      next;
    }
    
    if (!($path1 =~ m/\/$/i)) {
      # cat trailing slash so query is vs directory
      $path1 =~ s/^\s+//;
      $path1 =~ s/\s+$//;
      $path1 = $path1 . "/";  
    }
    
    # Create an HTTP request
    my $req = HTTP::Request->new(GET => $host . $tmppath . $path1);
  
    $req->content_type('application/x-www-form-urlencoded');
    # Pass request to the user agent and get a response back
    my $res = $ua->request($req);
  
    # Check the outcome of the response
    if (!($res->status_line =~ m/404/) && (!($res->status_line =~ m/300/))) {
      my $tmp = $tmppath . $path1;
      # populate array with discovered data
      push(@pathresultsarray, $tmp);
      # make recursive call for any discovered directories
      if ($cnt > 0) {
        &dig($tmp, $cnt - 1);
      }
    }
  }
 
}
 
# open file with directory listing
if (!(defined($opts{p}))) { 
  print "You must specify a resource list file.\n"; 
  usage();
} else {
  $pathfilename = $opts{p};
  open (PATHS, "< $pathfilename") or die "Can't open $pathfilename : $!";
  
  while (<PATHS>) {
    chomp;
    push(@patharray, $_) unless $_ eq '';
  }
}
 
# open file with resource listing
if (!(defined($opts{f}))) { 
  print "You must specify a resource list file.\n"; 
  usage();
} else {
  $resfilename = $opts{f};
  open (RESOURCES, "< $resfilename") or die "Can't open $resfilename : $!";
  
  while (<RESOURCES>) {
    chomp;
    push(@resarray, $_) unless $_ eq '';
  }
}
 
# open file with extension listing
if (!(defined($opts{e}))) { 
  print "You must specify an extension list file.\n"; 
  usage();
} else {
  $extfilename = $opts{e};
  open (EXTENSIONS, "< $extfilename") or die "Can't open $extfilename : $!";
  
  while (<EXTENSIONS>) {
    chomp;
    push(@extarray, $_) unless $_ eq '';
  }
}
 
# we need a target
if (!(defined($opts{h}))) { 
  print "You must specify a host to scan.\n"; 
  usage();
} else {
  $host = $opts{h};
  # if the host string does not start with http ...
  if (!($host =~ m/^http:\/\//i)) {
    # strip starting and ending white spaces
    $host =~ s/^\s+//;
    $host =~ s/\s+$//;
    # cat protocol and host
    $host = "http://" . $host;
  }
}
 
# get or set depth level
if (!(defined($opts{d}))) { 
  print "You did not specify a depth level, defaulting to 2.\n"; 
  $depth = 2;
} else {
  $depth = $opts{d};
}
 
# loop thru the list of directories and
# populate an array with the results
foreach $path (@patharray) {
  # strip starting and ending white spaces
  $path =~ s/^\s+//;
  $path =~ s/\s+$//;
  # if path starts with / but is not /
  if (!($path =~ m/^\//i) && !($path eq "/")) {
    # cat slash and path
    $path = "/".$path;
  }
  # if path ends with / but is not /
  if (!($path =~ m/\/$/i) && !($path eq "/")) {
    # cat path and slash
    $path = $path . "/";
  }
 
  # Create an HTTP request
  my $req = HTTP::Request->new(GET => $host . $path);
  $req->content_type('application/x-www-form-urlencoded');
  # Pass request to the user agent and get a response back
  my $res = $ua->request($req);
 
  # Check the outcome of the response
  if (!($res->status_line =~ m/^404/)) {
    push(@pathresultsarray, $path);
    &dig($path, $depth) unless $path eq "/";
  }
}
 
# loop thru the array of found directories and
# populate an array with the resource hit results
foreach $path (@pathresultsarray) {
  foreach $resource (@resarray) {  
    # loop thru the list of extensions
    foreach $extension (@extarray) {
      # Create a request
      my $req = HTTP::Request->new(GET => $host . $path . $resource . "." . $extension);
      $req->content_type('application/x-www-form-urlencoded');
 
      # Pass request to the user agent and get a response back
      my $res = $ua->request($req);
 
      # Check the outcome of the response
      if ($res->is_success) {
        push(@resresultsarray, "$path$resource.$extension");
      }
    }
  }
}
  
print "Directories discovered on the web server at $host \n";
foreach $path (@pathresultsarray) {
  print "$path\n";
}
print "\n";
 
print "Resources discovered on the web server at $host \n";
foreach $resource (@resresultsarray) {  
  print "$resource\n";
}
print "\n";
 
# clean up
close (PATHS);
close (RESOURCES);
close (EXTENSIONS);

Have fun with it!

Now, the final step to discovery is the one which I’m sure you guys always do - scanning the HTML code for leaks. There can be legacy codes, random comments and hidden directories amongst other clues hidden in there. It’s a real gem. Picture this -

<meta name="GENERATOR" content="Microsoft FrontPage (Visual InterDev Edition) 2.0">. Refer to DOSAM II for more information.

And thus I conclude Volume 1 of WAPT. I hope you enjoyed reading my post as much as I had enjoyed typing it. Have fun discovering!

Permalink 1 comment • Trackback (1)

Trackback address for this post:

http://altblog.searix.net/comtrack/trackback.php/77

Comments, Trackbacks, Pingbacks:

Comment from: Gizmore [Visitor]

Very nice post. I enjoyed reading it.
Many thanks for that and...
carry on your great work :)

Permalink 01/26/08 @ 02:32

Trackback from: I ask about help in promoting blog - consecrated service skis.I have hopes that it will begin develo [Visitor]

I ask about help in promoting blog - consecrated service skis.I have hopes that it will begin developing thanks of your help.I invite to division with your impressions. www.atomicnarty.pl
I ask about help in promoting blog - consecrated service skis.I have hopes that it will begin developing thanks of your help.I invite to division with your impressions. www.atomicnarty.pl

Permalink 03/06/08 @ 19:54

This post has 3 feedbacks awaiting moderation...

Sun	Mon	Tue	Wed	Thu	Fri	Sat
<< <				> >>
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

Web Application Penetration Testing (WAPT) Vol. 1 - Discovery

18/01/08

Web Application Penetration Testing (WAPT) Vol. 1 - Discovery

Trackback address for this post:

Comments, Trackbacks, Pingbacks: