« February 2008 | Main

Web Application Monitoring Data Model

Posted by ivanr on March 27, 2008.

A data model is the foundation of web application monitoring and, thus, key to successful utilisation of web application firewalls. We don't get to design the model; we can only deduct it from the information provided to us from the underlying technology. What we can do is build on it, and, for that reason, it is very important to understand what we have to work with.

An ideal model is one that helps structure the information available to us, allows us to enrich it with additional pieces of data and generally helps us raise events based on the information it contains.

The major parts of a web application monitoring data model are as follows:

• Connection - corresponds to one TCP connection.
• Request - corresponds to one HTTP request.
• Response - corresponds to one HTTP response.
• IP Address - the IP address of the client, retrieved from the TCP connection.
• Session - application session.
• User - authenticated user; in most cases this translates to the application user, but some sites still use HTTP authentication, and some might use both.
• Site - perhaps more accurately called Protection domain, or Application. None of these terms is perfect, but I generally prefer to use Site. In our model, Site refers either to the functionality behind an entire domain name (e.g. www.example.com), or only a subset of one (www.example.com/forums/).
• Country - the country where the request originates.
• City - the city where the request originates.
• Custom - any number of custom attributes. For example, you might want to have different policies for different departments within your organisation. To achieve this, you will map client IP addresses to department names, which you will then use to determine policies.

Most of the components are easy to construct, mapping from the structures used in programming, but there are a few places where the technology does not support the view, or where what we are given is not what we want to see:

• Some work is needed to be able to distinguish sessions. There are different session identifier techniques to consider (e.g. in the URI, in a parameter, in a cookie). While there is a number of platforms that have standardised session management, there is also a large number of applications using their own schemes, so in general some custom work will be needed.
• More so in the case of user identification. Building on session identification one needs to identify a successful login event in the traffic in order to determine the session username.
• The IP address may not be accurate. It may be that of an intermediary, and not of the client itself. Such cases can sometimes be identified (as is the case with HTTP proxies) , but not always (e.g. if a transparent HTTP proxy is used). The problem is that, unless you control the proxy, you can only rely on the IP address you got from the TCP stack; the information extracted from HTTP requests headers is not to be trusted.

Note: This post is part of the Web Application Firewall Concepts series.

Posted by ivanr at 12:10 PM

Web Application Firewall Use Case: Continuous Security Assessment

Posted by ivanr on March 19, 2008.

After some deliberation, I have decided to add Continuous security assessment as a standalone item on my web application firewall use cases list. Although some could argue the functionality is already covered in the Web intrusion detection and prevention section, it is not obvious—you have to understand this field in order to know it is there.

Continuous security assessment is not specific to web application firewalls—it's been used for the network layer for years—but web application firewalls are more useful for web applications (than IDS tools are for network applications), simply because there's essentially one data format to analyse (if you can call a bunch of loosely related specifications used by web applications "one" data format). With web applications, you get to see clear picture of application input and output. Therefore, with proper parsing and analysis in place, it's almost like having a direct communication channel with the application itself.

The focus here is not on the attacks and other suspicious client behaviour, which comes out of the stock IDS/IPS functionality, but on everything else:

• application defects,
  
• information leakage,
  
• configuration errors,
  
• change control
  

and so on. The advantage is that you can detect some very subtle issues, only possible because of the depth of analysis.

Just as an extreme example, there are quite a few web applications out there where SQL injection (less often) and XSS (surprisingly common) exist by design—their clients are allowed and expected to send such low-level stuff and have it executed by the server. These types of problem can be detected early and with little effort, and because assessment never stops, you get the alert as soon as they are manifested.

Note: This post is part of the Web Application Firewall Concepts series.

Posted by ivanr at 02:42 PM

Web Application Firewall Use Cases

Posted by ivanr on March 10, 2008.

There are many reasons to use a web application firewall. Most people tend to focus on prevention and blocking when the term is brought up, but that is just one of the possible uses. Three years ago, almost to the day, I wrote this post to argue how one needs a WAF to serve as a part of the overall defence strategy. My opinion remains unchanged today, but I have since expanded the list of use cases for web application firewalls. Here they are:

1. Web intrusion detection and prevention. Applying the principles of external traffic monitoring (IDS) and prevention (IPS) to HTTP and the related technologies, which are used to build web applications. Through your WAF you will look for signs of generic web application attacks (negative security model), or deploy a learning engine to construct a model of your site and reject all invalid traffic, not just attacks (also known as positive security model).
2. Continuous security assessment. The idea with this case is to emphasize the fact web application firewalls actually understand web applications pretty well. Armed with this knowledge, they can do more than detect attacks; they can observe from signs of weaknesses, information leaks, configuration errors, and similar problems before an attempt to exploit them is made.
3. Virtual (or just-in-time) patching. When you need to deal with a specific problem in your web site, which exists either in your code or in the code of the product you are relying on. The focus in this use case is on writing custom rules to deal with custom issues.
4. HTTP traffic logging and monitoring. Do you know what is actually going on in your web applications? Who are your users and how are they using your systems? Are you being attacked and how?
5. Network building blocks. This use case is not often a primary motivator for WAF deployment, but if you're already deploying a reverse proxy to serve as a HTTP switch/router, making the device security-aware is the way to go.
6. Web application hardening. If you deploy your WAF as a reverse proxy then you can get it to modify the traffic stream to fix some of the design faults of your application or the HTTP protocol.

I will expand on each use case in my future posts.

Note: This post is part of the Web Application Firewall Concepts series.

Posted by ivanr at 04:20 PM

Web Application Firewall Concepts

Posted by ivanr on March 10, 2008.

I went through all my ModSecurity Blog posts yesterday, partly to admire myself for blogging consistently for almost 5 years and partly to understand what is that I talked about during this time. While I knew that most of my posts were pretty technical (after all, I did start my new blog to focus on other things) imagine my surprise when I realised I didn't properly cover the one thing this blog is supposed to cover: web application firewalls! The emphasize is on the word "properly": I provided a great deal of technical information but not enough content that would explain why one would deploy a web application firewall and how. This stuff had went into my conference talks and the Web Application Firewall Evaluation Criteria project, but I forgot to discuss the topics here.

In an effort to fix this I am starting a series of blog posts called Web Application Firewall Concepts. Each post will be reasonably brief and cover one aspect of the technology, and I will continually update this post to serve as a table of contents.

Posts in this series:

1. Use Cases
   1. Web intrusion detection and prevention
   2. Continuous Security Assessment
   3. Virtual (or just-in-time) patching
   4. HTTP traffic logging and monitoring
   5. Network building blocks
   6. Web application hardening
2. Deployment models
   1. Inline
   2. Out of line
   3. Embedded
3. Data Model
   1. Model construction
   2. Persisting information across requests
   3. Distinguishing sessions
   4. Distinguishing users
4. Analysis Model
   1. Negative security
   2. Positive security
   3. Anomaly scoring
   4. Learning
   5. Evasion
   6. Impedance mismatch
5. Traffic logging
6. Special protection techniques
   1. Cookie protection
   2. Cross-Site Request Forgery
   3. Brute force attacks
   4. Denial of Service attacks
   5. PDF UXSS protection

Posted by ivanr at 03:38 PM