ModSecurity Rule for Full-width/Half-width Unicode Evasion Detection

Posted by rcbarnett on May 23, 2007.

You have probably heard it by now, but US-CERT released a Vulnerability Note last week entitled "HTTP content scanning systems full-width/half-width Unicode encoding bypass." The short of it is that many HTTP content scanning systems (think IDS/IPS/WAFs) may not be able to properly decode data that is encoded using Unicode full-width/half-width encoding thus allowing a possible evasion issue for malicious traffic.

This is yet another Impedance Mismatch issue where one host may interpret data a certain way while another interprets in differently. In this case, security devices that use decoding functions may not properly decode the data and cannot therefore apply certain signatures. If this is the case, then you would have a false negative if the request was malicious and the destination host is able to decode the data and process it.

The $1,000,000 question here is does this issue affect ModSecurity? Yes. The %u syntax is a Microsoft-specific extension. While ModSecurity does not decode such encodings by default (meaning you have to explicitly address the issue in your rules if you need this feature) you can choose to decode them using the urlDecodeUni transformation function. In the current version of ModSecurity this transformation function cannot deal with the above-mentioned evasion technique. (On the positive side, the transformation function behaves exactly as documented in the reference manual.) Fortunately, it is quite easy to create a ModSecurity rule that can identify and block any use of this type of encoding. Here is the example rule that you can use:

# Disallow use of full-width unicode
SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS|XML:/*|!REQUEST_HEADERS:Referer \
"\%u[fF]{2}[0-9a-fA-F]{2}" \
"t:none,deny,log,auditlog,status:400,msg:'Unicode Full/Half Width Abuse Attack Attempt',id:'950116',severity:'4'"

This rule is also included in the latest development release of the Core Rules.

Posted by rcbarnett at 01:45 AM

ModSecurity 2.2.0 Development Releases

Posted by brectanus on May 14, 2007.

Hello all. As this is my first official blog entry, let me first start off with a short introduction...

My name is Brian Rectanus (pronounced rec-tan-us, for those curious). As some of you may already know, I was hired by Breach Security, Inc. in February as the primary ModSecurity developer. So far, I have worked on mainly bug-fixes and smaller features in the ModSecurity 1.9.x and 2.1.x branches, but most of my time is being spent on the upcoming version 2.2.0 release. My PGP key for verifying archive signatures can be found on most any PGP keyserver (pgp.mit.edu).

With 2.2.0, Ivan and I will be periodically releasing a development branch that is a stabilized version of the main ModSecurity development branch. This is partially so that users can have access to new features sooner, but is primarily meant to be a means to allow the ModSecurity community to become more involved in the future of ModSecurity as well as to keep ModSecurity as stable as possible for official releases. With the development releases, users will have a chance to preview upcoming changes and provide valuable feedback to the developers before changes, features, configuration syntax, etc. become finalized.

Typically there will be 3-4 development releases followed by release candidates, then a final release. Each development release will enhance upon the previous by adding more features and changes. Once Ivan and I are happy with the changes, I will close the branch to new features and start the release candidates -- with only bug fixes and minor changes -- until the code is stabilized. At that point I will release the final version.

With that, I am happy to announce the general availability of the first development release for ModSecurity version 2.2.0. The source code is available in the download section of the ModSecurity Website. Below is an outline of the new features and changes in this release so far. Please see the documentation included in the release for full details and usage examples. Please direct any comments to the ModSecurity User Support mailing list.

Changes since version 2.1.1

Experimental support for content injection. You can use @prepend value and @append value to inject content into the output.
PDF XSS protection added. Ivan is giving a talk on this at OWASP in Milan this week.
Geographical lookups by IP address/hostname were added. With this you can access the country, city, postal code, etc. -- within a new GEO collection -- from any existing ModSecurity variable by using the new @geoLookup operator.
String comparison operators -- @streq, @beginsWith and @endsWith -- to allow an easier-to-use non-regular expression operator. Values will have macros interpreted prior to a match, so that you can do "@streq %{REMOTE_ADDR}", etc.
A length transformation -- t:length -- transforms a value into a numeric character length.
Whitespace trimming transformations are now available. Use t:trimLeft, t:trimRight or t:trim to trim whitespace from the left, right or both, respectively.
Experimental variables -- RESPONSE_CONTENT_LENGTH, RESPONSE_CONTENT_TYPE, and RESPONSE_CONTENT_ENCODING -- were added.
Added the filename and line number of the rule to the debug log on its execution. I will add it to the audit log in the next development release.
Removed support for CGI style HTTP_HEADER_NAME variables for request headers. The 2.x REQUEST_HEADERS:Header-Name should now be used.
Removed support for %0 - %9 TX variables that expanded to numbered captures. Use %{TX.0} - %{TX.9} instead.
Various other bug fixes and smaller changes. See the CHANGES file in the release for more details.

Posted by brectanus at 05:00 PM

ModSecurity Console Performance Tuning

Posted by rcbarnett on May 09, 2007.

Help, my ModSecurity Community Console is not responding!"

Perhaps you have seen this type of email sent to the ModSecurity mailing list? Unfortunately, it is relatively easy to overload the ModSecurity Conmmunity Console when you combine the following three factors: using the default Console configurations, having the ModSecurity Sensor configured to log alerts for all traffic and if your site has large amounts of traffic. In this type of situation, the Console MUI may become unresponsive. This is due to having too many open/active alert messages. So, what can you do about it? Read on to find steps to help alleviate this problem.

ModSecurity Sensor Configuration Settings

`SecAuditEngine`

The SecAuditEngine Directive can have an impact on the total number of alert messages being created, mainly if it is set to On. The recommended setting for this directive is RelevantOnly.

`SecAuditLogRelevantStatus`

The defualt setting for this directive in the Core Rules is "^[45]". This directive will generate audit logs for all 4xx and 5xx level status codes. The downside of this setting, from a performance perspective, is that this also logs 404 Not Found status codes which can generate a lot of information. If you want to exclude 404 status codes from being logged, use this setting - "^(?:5|4\d[^4])".

`modsecurity-auditlog-collector`

The Perl script that is currently available for transporting the concurrent audit log files to the remote Console host does not perform well under production loads. There is even a disclaimer saying as much inside the script:

# This code is not suitable for non-trivial production use
# since it can only submit one audit log entry at a time, plus
# it does not handle errors gracefully.

So, what can you do? Breach Security is putting the finishing touches on a production quality audit log transport replacement tool called mlogc. This tool will be released to the public in the very near future so keep your eyes out for it. It won't solve all of your performance issues but its will help.

ModSecurity Console Configurations

Avoiding Too Many Open/Active Alerts

The main memory hog in the Console is the number of active alerts (alerts with status open) in the Alert interface. Part of this can be alleviated by properly configuring the various ModSecurity directives mentioned previously as to not generate too many needless alert messages.

Update the Alert Management Settings

Another change that can be made to help avoid a memory problem associated with this issue is to update the Alert Management configuration settings. You can optionally go to Administration -> Alert Management in the MUI and decrease the “Automatic Stale Alert Removal” time setting. This will automatically archive and remove alerts below a specified alert severity from the alert page. This can help to guard against low level Notice/Info alerts from clogging up the Alert interface. You can still get to all of the data, however you would need to use the Transactions Search page to bring them back up.

Getting Access to the MUI When There Are Too Many Alerts

If your Console is being bombarded by alert messages, the initial homepage may not be responsive when you login as the MUI is attempting to display the proper number of Active Alerts under the Sensor Overview section. If this is happening and your MUI is hanging, try accessing another page that does not display the Active Alert data such as the Administration page located here - http://CONSOLEHOST:8886/adminHome. This should allow you to get access to the Alert Management interface (mentioned in the previous section) and possibly flush out the active alerts and archive them to the database.

Allocating More RAM to Java

You can allocate more memory available/allocated to the Java MUI application. The amount of Memory (RAM) that is allocated to the Java application can be increased by editing the modsecurity-console shell script file. You want to edit the -Xms and -Xmx arguments in the following line:

-INSTALL4J_ADD_VM_PARAMS="-Xms128M -Xmx512M - Dderby.storage.pageSize=32768 -Dderby.storage.pageCacheSize=8000"

Here is a quick breakdown of what these parameters mean:

-Xms128M - start with 128 MB RAM
-Xmx512M - use max. 512 MB

If you increase these values, you should take care to still leave some RAM to be used by the operating system and other processes on the box.

-Dderby.storage.pageCacheSize=8000 - each page should be 8 KB
-Dderby.storage.pageSize=32768 - have 32768 pages in cache

In the above example you allocate 32768 x 8 KB = 256 MB. To be on the conservative side, the database cache should be at most 30% of the total RAM dedicated to the Java process. Feel free to experiment to find the optimal value for your case.

When the free Console just won't keep up...

If your site simply generates too much traffic for the free Console to handle, you many want to consider purchasing the soon to be released ModSecurity Management Console provided by Breach Security. This updated version of the Console has much better performance. It will initially function just like the open source Console in that it is used as a centralized alert management host. In future releases, it will also include "Command and Control" functionality that will allow it to actually manage remote ModSecurity Sensors.

Posted by rcbarnett at 07:36 PM