Support request checklist
This document describes the steps needed to collect the information necessary to successfully resolve a ModSecurity support request. You don't have to go through this document every time you need to ask something. For simpler problems just go ahead and ask a question. But if the information you provided is not sufficient you will be forwarded back to this document.
This is what you need to do:
You are about to submit a support request to a public mailing list. Before you do it you need to think about the data you have included in the report. Not only there are many people subscribed to the list (that's a good thing) but it is archived at many web sites and practically stored for ever. You should not send anything private, sensitive, or confidential.
1. Specify the operating system
Which operating system are you running?
On Unix, the version can be provided using the uname command:
$ uname -a Linux fedora1 2.4.22-1.2199.nptl #1 Wed Aug 4 12:25:07 EDT 2004 i686 athlon i386 GNU/Linux
On Red Hat systems the file /etc/redhat-release provides more information:
$ cat /etc/redhat-release Fedora Core release 1 (Yarrow)
On Windows computers, system information is available from the Control Panel, option System (on the first tab). For example:
Microsoft Windows XP Professional Version 2002 Service Pack 2
2. Specify the Apache version
Which Apache are you running? You can get that information from the httpd binary when you execute it with the -V command:
$ ./httpd -V Server version: Apache/2.0.52 Server built: Jan 16 2005 10:37:17 Server's Module Magic Number: 20020903:9 Architecture: 32-bit Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D HTTPD_ROOT="/home/ivanr/apache2" -D SUEXEC_BIN="/home/ivanr/apache2/bin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf"
While you are there, it is also useful to list the available modules:
$ ./httpd -l Compiled in modules: core.c mod_access.c mod_auth.c mod_include.c mod_log_config.c mod_env.c mod_setenvif.c prefork.c http_core.c mod_mime.c mod_status.c mod_autoindex.c mod_asis.c mod_cgi.c mod_negotiation.c mod_dir.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_rewrite.c mod_so.c
3. Specify the ModSecurity version
You can find out the ModSecurity version from the name of the archive (the version number is part of the name, for example mod_security-1.8.6.tar.gz), or by looking in the source code (file mod_security.c):
#define MODULE_RELEASE "1.8.6"
If you find that you are using an older version of ModSecurity, please check the “Known issues” page (http://www.modsecurity.org/documentation/known-issues.html) to determine if your problem is already known. If it is, you will most likely find the fix already available (in a form of minor version upgrade). If the fix isn't available yet chances it will be very soon.
4. Brief system description
Briefly describe how is the system used. For example:
Apache is used to host a departmental intranet application. The application was developed internally, runs in PHP, connects to a MySQL database, and our IMAP server.
5. Information about the browser
Consider the following questions:
6. Apache and ModSecurity configuration
Please supply copies of your Apache and ModSecurity configuration. In most cases the configuration is only contained in the httpd.conf file. Some administrators choose to separate configuration data logically into more files. If this is the case you can look for the Include configuration directive, which specifies another file to include. Include all such files in your report.
Your Apache configuration may contain sensitive information. If you think it does you should inspect the configuration files and remove sensitive information before you include the files in your report.
7. Filesystem layout
If you are experiencing filesystem problems you should include a listing of a part of the filesystem in the report. If you execute the following on a Unix system, the file fs-listing.txt will contain the list of all files in the /home/ivanr/apache2 directory.
$ ls -alR /home/ivanr/apache2 > ~/fs-listing.txt
On Windows, the same effect can be achieved with:
C:\> dir "c:\Program Files\Apache" /s > c:\fs-listing.txt
Feel free to include dumps of several directories if required.
8. Special circumstances to consider
Are there any special circumstances to consider when requesting support? For example, your Apache installation may be running from a chroot jail, where it has access only to a limited set of files.
The final piece of information needed is contained in the log files. Since logs usually contain large amounts of information you need to take steps to only send the necessary data:
If you are not getting any output from ModSecurity make sure it is configured and enabled with SecFilterEngine On.
9. Apache access log
Each request appears as a single line in the Apache access log. By default the access log is in the file /logs/access_log. When virtual hosting is used each virtual host may have a separate access log file. Examine the Apache configuration if you don't see the information about the test requests you make in the access log (look for the appearance of the TransferLog or CustomLog directives).
The following example shows traces of two requests (both have generated errors, as you shall see later in the text):
192.168.2.10 - - [16/Jan/2005:10:56:49 +0000] "GET /index.html?111 HTTP/1.1" 403 291 192.168.2.10 - - [16/Jan/2005:10:56:49 +0000] "GET /favicon.ico HTTP/1.1" 404 288
10. Apache error log
Apache errors are logged to the error log. ModSecurity will emit relevant warnings and errors to the error log too. By default the error log is written to a file located at /logs/error_log. It is entirely possible for different virtual hosts to have different error log files. To determine if the errors will appear in the file you are monitoring just perform a test request for a resource that does not exist on the server. A line warning about a page not found should appear in the log:
[Sun Jan 16 10:56:49 2005] [error] [client 192.168.2.10] File does not exist: /home/ivanr/apache2/htdocs/favicon.ico
Otherwise examine the Apache configuration to find the correct error log location (look for the ErrorLog directive).
This is what a single ModSecurity error message looks like:
[Sun Jan 16 10:56:49 2005] [error] [client 192.168.2.10] mod_security: Access denied with code 403. Pattern match "111" at THE_REQUEST [hostname "192.168.2.101"] [uri "/index.html?111"]
11. ModSecurity Debug log
The ModSecurity debug log will, when configured properly, provide a ton of information for every HTTP request. To get the information, specify the file where you want the debug output to go and set the log level to 9:
SecFilterDebugLevel 9 SecFilterDebugLog logs/modsec_debug_log
The output will be in the file /logs/modsec_debug_log.
There can be pages of text for each request, especially if a file is being uploaded.
This is how a very short output looks like (one request, one ModSecurity rule only):
[16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] sec_check_access, path=(null) [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Normalised REQUEST_URI: "/index.html?111" [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Parsing arguments... [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Adding parameter:  [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Checking signature "111" at THE_REQUEST [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Checking against "/index.html?111" [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Signature check returned 403 [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Access denied with code 403. Pattern match "111" at THE_REQUEST [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Rule match, returning code 403 [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] find_last_request: start with 9008830 "/index.html" [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] sec_logger: start [16/Jan/2005:10:56:49 +0000] [192.168.2.101/sid#8f519d8][rid#9008830][/index.html] Audit log off here
12. ModSecurity Audit log
The ModSecurity audit log will, when enabled, contain the complete request and the headers from the response. To make sure request bodies are observed configure:
The following configuration will tell ModSecurity to log only violations:
SecAuditEngine RelevantOnly SecAuditLog logs/audit_log
The output will be in the file /logs/audit_log. The example below shouls one audit log entry:
======================================== Request: 192.168.2.10 - - [17/Jan/2005:17:56:30 +0000] "GET /index.html?111 HTTP/1.1" 403 291 Handler: (null) ---------------------------------------- GET /index.html?111 HTTP/1.1 Host: 192.168.2.101:8080 User-Agent: mod_security regression test utility Accept: text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-gb,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive mod_security-message: Access denied with code 403. Pattern match "111" at THE_REQUEST mod_security-action: 403 HTTP/1.1 403 Forbidden Content-Length: 291 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1
Notes on working on a production server
Providing logs from a production server, possibly one used by dozens of people at any given time, can be challenging. For example, switching the debug log to level 9 will instantly slow down the web server. It is still possible to provide detailed logs by increasing the debug log level only for the parts where it is absolutely required.
For example, let us assume you only need to test one script, for example /test.php. To increase the debug log level for that script only do the following (assuming you have previously configured the file name for the debug log and set the default log level to zero):
<Location /test.php> SecFilterDebugLogLevel 9 </Location>
Almost all ModSecurity configuration directives can be put inside Apache context tags. If you need to perform a test but you are afraid your ModSecurity configuration can affect your users you can even enable ModSecurity for one file only. Just put all the directives in a container tag.