RDNSLOGS - Performs Reverse DNS lookups on Web Site Logs

What is RDNSLOGS?

RDNSLOGS is a windows program that scans files for IP addresses and produces a cache file of their DNS Reverse Lookups.  Originally designed to read files generated by web servers, the program can be used with any text file that contains IP addresses.  Through the use of multiple concurrent reverse DNS lookup threads, the program quickly converts numeric IP addresses to their corresponding host names.  RDNSLOGS is written in C++ for fast native mode execution.  This program was written to complement the Analog Log File Analyzer, but it can certainly be used in other environments.

The cache file produced by RDNSLOGS is a text file whose format is:  timestamp IP_address name  where the timestamp is the number of minutes since the beginning of 1970, GMT (i.e., "Unix time" divided by 60), and the name is just * if the address couldn't be resolved.  This format should be compatible with most log analyzers.

RDNSLOGS has internal support for gzip compression and supports all other external forms of file compression.  As of V6.0, there is also an option to translate the IP addresses in files to their associated domain names.


RDNSLOGS Word Cloud(08/30/2013) RDNSLOGS V6.0 is here! Includes both Win32 and x64 builds! Built with PCRE and ZLIB Code.

This build now has an option to translate the IP addresses in files to their actual domain names! This operation is reported to be lightening quick.  There two additional switches to enable this function:

This build was made using the latest Visual Studio 2012 C++.  Please consider purchasing this version.  V6.0 is extremely stable and offers many features and performance enhancements not found in the older versions.

To purchase RDNSLOGS Binary through PayPal for $20, click this button:

To purchase Source code for RDNSLOGS (includes the binary) through PayPal for $69, click this button:   The source code reveals some great examples of how to perform multithreading and thread synchronization, perform regular expression operations using the pcre library, as well as how to do low level IP packet transmission and reception, all in C++ code.

Upon receipt of PayPal payment, a zip file of the package will be immediately emailed to you.  Installation of RDNSLOGS is straightforward.  Unzip the contents of the zip file, and copy rdnslogs.exe into a convenient directory.


How to Use RDNSLOGS

The command line syntax is as follows:

rdnslogs [options] <configuration filename>
Option Description
-t threads Max count of concurrent DNS lookups (thread count) (default: 20, maximum: 400)
-v Verbosity switch.  Use the -v switch only when you want to monitor the state of RDNSLOGS.  It generates copious amounts of output and generally slows the program.
-r Recursively visit all configuration files specified by CONFIGFILE command
-n NO subnet host lookups.  Default is to perform the subnet host lookups.
-l logfile You specified a log file specification on the command line. i.e. -l c:\logs\*.gz You can specify as many log file specs as can fit on a command line provided that each spec is preceded with a -l. If a log file spec contains a space, surround the spec with quotes. If a log file spec is made, a configuration file does not need to be specified.
-o dnscache You specify the desired DNS cache output filename.
-y DNS server Querying a specific DNS server.  Use this switch to specify a particular DNS server to query.  Not specifying the switch will make RDNSLOGS revert back to the default method of resolving names.
-x max tries Specifies the maximum number of attempts to resolve an IP address.  Specifying 0 forces an indefinite attempt toward resolving the IP address.  Works in conjunction with the -y switch. Default is 3 tries.
-m timeout Specifies the base timeout value for a lookup.  If a lookup fails, the timeout value is doubled, up to a maximum of 60 secs, on the next and subsequent attempts.  Works in conjunction with the -y switch.  Default base timeout is 4 secs.
-c New in v6.0: Convert IP addresses to domain names in files.  Creates *.trans files. Original files are not modified.
-b New in v6.0: Bypass Reverse Lookup.  Used with -c switch.  Use existing dnscache for file translation.

If a configuration file is specified, command Line switches are processed before the configuration file.

Example 1:

C:> rdnslogs -t 100 config.txt

This command line will read the config.txt file in the current directory for DNS related information and use up to 100 threads to resolve IP addresses in log files.

Example 2:

C:> rdnslogs

Displays a complete usage summary including version number and available options.

Example 3:

C:> rdnslogs -v -t 150 myconfig.txt

Read the file myconfig.txt for log file locations and use up to 150 threads to resolve the IP addresses in those files.  Also, generate debugging information on the screen.

Example 4:

C:> rdnslogs -l c:\logs\*.log -l "d:\log files\*.gz" -o mynewdns.txt -t 300 

Reads all the *.log files in the C:\logs directory and all the *.gz files in the "d:\log files" directory to create a mynewdns.txt file in the current directory using 300 lookup threads.

Example 5:

C:> rdnslogs -l c:\logs\*.log -t 300 -y 1.2.3.4 -c

Reads all the *.log files in the C:\logs directory to create/update a dns.txt file in the current directory using 300 lookup threads and directly communicating with a DNS server whose ip address is 1.2.3.4.  After this is done, go through these same .log files and create corresponding .trans files with IP addresses changed to their domain names.

Example 6:

C:> rdnslogs -l c:\logs\*.log -t 300 -y 1.2.3.4 -b

ILLEGAL! You specified to bypass the reverse dns lookup process and did not specify a -c to do a translation.

Example 7:

C:> rdnslogs -l c:\logs\*.log -t 300 -y 1.2.3.4 -b -c

Reverse DNS lookup is bypassed.  The -y, -t switches are therefore ignored.  Only a translation is specified to go through all the .log files in the c:\logs directory and create .trans files  in that same c:\logs directory with IP addresses changed to domain names.

All other lines in the configuration file are ignored. The syntax of these lines can be obtained from http://www.analog.cx/docs/indx.html.

If you are not using analog as your log file analyzer, the following can be used as a sample configuration file:

# Start of Sample Configuration File for RDNSLOGS

# LOGFILE lines specify the directory locations of your log files. 
# You can have multiple LOGFILE commands. 
# RDNSLOGS will process each specified directory. 

LOGFILE D:\WEBSITELOGS\W3SVC1\*.LOG
LOGFILE D:\WEBSITELOGS\W3SVC1\*.GZ

# DNSFILE command specifies the name of your DNS cache file. 

DNSFILE dns.txt

# DNSGOODHOURS specifies how many hours a successfully resolved IP address is considered valid. 

DNSGOODHOURS 2880

# DNSBADHOURS specifies how many hours an unsuccessfully resolved IP address needs to wait before attempting to resolve that IP address again. 

DNSBADHOURS 336

# CONFIGFILE command lines specify other configuration files to be read processed. 
# If the -r command line switch is specified, these files will be processed by RDNSLOGS. 
# The default is to not process this command line. 

CONFIGFILE FileAliases.txt
CONFIGFILE FileExcludes.txt

# UNCOMPRESS specifies the external program to call if you want to uncompress log files that are 
# compressed using a format other than gzip.
# See http://www.analog.cx/docs/logfile.html#UNCOMPRESS for more info

UNCOMPRESS *.zip ("winzip32 -e filename[.zip])

# TO specifies the final timestamp that will be considered in a log file analysis. 
# It is used when the percent specifiers are used in log file names. 
# If no TO command line is found, RDNSLOGS assumes the current time when these programs are executed. 
# See http://www.analog.cx/docs/include.html#FROMTO for more info

TO +00+00+00:0000
# End of Sample Configuration File for RDNSLOGS

It would be advantageous to use the output produced by RDNSLOGS as a way to monitor its progress.   When scheduling RDNSLOGS, use the cmd /c command to allow for output redirection.  For instance, the following command in the windows scheduler:

cmd /c "rdnslogs.exe -t 100 config.txt > progress.txt"

will cause output to be redirected into a file called progress.txt.  You may periodically view the file to see the state of RDNSLOGS.  Note: When scheduling RDNSLOGS, redirection is not possible without the cmd /c.


Credits and Disclaimer

You agree to use it at your own risk.  This software is under no warranty.  The author assumes no liability in the event of any loss of data, time, and/or money.

RDNSLOGS is written and maintained by:

Prof. Antonio C. Silvestri
Department of Engineering and Computer Science Transfer
Springfield Technical Community College
1 Armory Square
Springfield, MA  01105
silvestri@stcc.edu