Re: ModSecurity: First integration patches

[Thierry Fournier]

> On 23 Apr 2017, at 15:19, Aleksandar Lazic  wrote:
> 
> Hi Thierry
> 
> Am 20-04-2017 15:05, schrieb Thierry Fournier:
>> Hi,
>> After a quick private brainstorm, Willy propose to me a new binary encoding
>> for the headers. It is useless to give the numbers of headers contained in
>> the block, so the end of headers is marked by an empty header (header name
>> and value have a length of 0).
>> The new patches in attachment (first patch is 0002).
> 
> I have now created a docker image
> 
> https://hub.docker.com/r/me2digital/haproxy-waf/
> 
> The build was successfully and haproxy starts.
> 
> I have also try to check if the modsecurity starts and yeah it starts ;-)
> 
> ###
> 
> 1492953204.290643 [00] ModSecurity for nginx (STABLE)/2.9.1 
> (http://www.modsecurity.org/) configured.
> 1492953204.290705 [00] ModSecurity: APR compiled version="1.4.8"; loaded 
> version="1.4.8"
> 1492953204.290730 [00] ModSecurity: PCRE compiled version="8.32 "; loaded 
> version="8.32 2012-11-30"
> 1492953204.290739 [00] ModSecurity: YAJL compiled version="2.0.4"
> 1492953204.290743 [00] ModSecurity: LIBXML compiled version="2.9.1"
> 1492953204.290747 [00] ModSecurity: Status engine is currently disabled, 
> enable it by set SecStatusEngine to On.
> 1492953209.298799 [03] 0 clients connected
> 1492953209.298937 [04] 0 clients connected
> ...
> ###
> 
> Now we can go the next step and create a complete solution ;-)
> 
> It would be interesting to go the same way as mod_sec & nginx is going.
> 
> https://www.trustwave.com/Resources/SpiderLabs-Blog/Announcing-the-availability-of-ModSecurity-extension-for-Nginx/
> 


Hi aleksander,

Thanks for the news. I integrated ModSec using the same way than NGinx, so it is
absolutely a good idea to follow the same way than ModSec/NGinx. I will write an
howto and publish benchmark when I found a lot of time.

A feedback on your test is welcome.

Thierry

>> Thierry
> 
> Regards
> Aleks
> 
>>> On 19 Apr 2017, at 15:34, thierry.fourn...@arpalert.org wrote:
>>> Hi,
>>> There is a new lot of patches for the spoa/modescurity contrib.
>>> Thierry
>>> On Wed, 19 Apr 2017 11:24:36 +0200
>>> Thierry Fournier  wrote:
> On 19 Apr 2017, at 09:16, Aleksandar Lazic  wrote:
> Am 19-04-2017 05:51, schrieb Willy Tarreau:
>> On Tue, Apr 18, 2017 at 11:55:46PM +0200, Aleksandar Lazic wrote:
>>> Why not reuse the upcoming http/2 format.
>>> HTTP/2 is *easy* to parse and the implementations of servers are 
>>> growing?
>> Are you kidding ? I mean you want everyone to have to implement HPACK 
>> etc ?
> Well there are a currently lot of http/2 servers out there, but I 
> understand your concerns.
>>> I know this is still on the todo list but I think this is worth to go
>>> instead to add another protocol.
>>> Or we can take a look into grpc, which is on top of http/2 with 
>>> protobuffer.
>>> http://www.grpc.io/
>>> As I take more time to think in this direction why not add a grpc filter
>>> like the spoa filter?
>>> Opinions?
>> Code generators are always a pain to deal with. It reminds me the old 
>> days
>> when I was using rpcgen. And grpc doesn't provide a C implementation so 
>> that
>> kills it :-)
> Yeah I have also seen that they prefer c++, c#, go and other languages 
> but not offer a c lib.
> This is strange just because the core is written in c, but I don't need 
> to understand everything ;-)
>> Thierry told me he's going to implement a simple TLV-like list which will
>> be much easier to implement and easy to code on both sides.
> @Thierry:
> Ah something like netstrings ?
> https://cr.yp.to/proto/netstrings.txt
 Hi thanks for the link. I do not like this encoding because it is not easy 
 to
 parse. I must read ascii representation of numbers and convert-it. I prefer
 simple binary format, with length (encoding in the same way than the SPOE
 protocol) and value.
 Hi, the main goal is providing:
 - a format easy to decode
 - not adding lot of dependencies to haproxy
 So, after brainstorm, I propose:
 - remove all data already accessible via existing sample-fetches from the 
 new
  sample-fetch introduced by my previous patch. These data are:
  method, path, query string, http-version and the body. All of these data 
 are
  available in some sample fetches and it will be great to reuse it. Note 
 that
  the only data which keep are the headers.
 - For the headers I propose two format: The first format is for general 
 use. It
  is simply the header block in the same format that we received (with the 
 final
  \r\n for detecting truncated header bloc)
 - For header I propose also a binary format like this:
  
  [
 
 
  ]…
  This format is easy to decode because each length is encoded 

Re: ModSecurity: First integration patches

[Aleksandar Lazic]

Hi Thierry

Am 20-04-2017 15:05, schrieb Thierry Fournier:

Hi,

After a quick private brainstorm, Willy propose to me a new binary 
encoding
for the headers. It is useless to give the numbers of headers contained 
in
the block, so the end of headers is marked by an empty header (header 
name

and value have a length of 0).

The new patches in attachment (first patch is 0002).


I have now created a docker image

https://hub.docker.com/r/me2digital/haproxy-waf/

The build was successfully and haproxy starts.

I have also try to check if the modsecurity starts and yeah it starts 
;-)


###

1492953204.290643 [00] ModSecurity for nginx (STABLE)/2.9.1 
(http://www.modsecurity.org/) configured.
1492953204.290705 [00] ModSecurity: APR compiled version="1.4.8"; loaded 
version="1.4.8"
1492953204.290730 [00] ModSecurity: PCRE compiled version="8.32 "; 
loaded version="8.32 2012-11-30"

1492953204.290739 [00] ModSecurity: YAJL compiled version="2.0.4"
1492953204.290743 [00] ModSecurity: LIBXML compiled version="2.9.1"
1492953204.290747 [00] ModSecurity: Status engine is currently disabled, 
enable it by set SecStatusEngine to On.

1492953209.298799 [03] 0 clients connected
1492953209.298937 [04] 0 clients connected
...
###

Now we can go the next step and create a complete solution ;-)

It would be interesting to go the same way as mod_sec & nginx is going.

https://www.trustwave.com/Resources/SpiderLabs-Blog/Announcing-the-availability-of-ModSecurity-extension-for-Nginx/


Thierry


Regards
Aleks






On 19 Apr 2017, at 15:34, thierry.fourn...@arpalert.org wrote:

Hi,

There is a new lot of patches for the spoa/modescurity contrib.

Thierry


On Wed, 19 Apr 2017 11:24:36 +0200
Thierry Fournier  wrote:



On 19 Apr 2017, at 09:16, Aleksandar Lazic  
wrote:




Am 19-04-2017 05:51, schrieb Willy Tarreau:

On Tue, Apr 18, 2017 at 11:55:46PM +0200, Aleksandar Lazic wrote:

Why not reuse the upcoming http/2 format.
HTTP/2 is *easy* to parse and the implementations of servers are 
growing?
Are you kidding ? I mean you want everyone to have to implement 
HPACK etc ?


Well there are a currently lot of http/2 servers out there, but I 
understand your concerns.


I know this is still on the todo list but I think this is worth to 
go

instead to add another protocol.
Or we can take a look into grpc, which is on top of http/2 with 
protobuffer.

http://www.grpc.io/
As I take more time to think in this direction why not add a grpc 
filter

like the spoa filter?
Opinions?
Code generators are always a pain to deal with. It reminds me the 
old days
when I was using rpcgen. And grpc doesn't provide a C 
implementation so that

kills it :-)


Yeah I have also seen that they prefer c++, c#, go and other 
languages but not offer a c lib.
This is strange just because the core is written in c, but I don't 
need to understand everything ;-)


Thierry told me he's going to implement a simple TLV-like list 
which will

be much easier to implement and easy to code on both sides.


@Thierry:
Ah something like netstrings ?
https://cr.yp.to/proto/netstrings.txt


Hi thanks for the link. I do not like this encoding because it is not 
easy to
parse. I must read ascii representation of numbers and convert-it. I 
prefer
simple binary format, with length (encoding in the same way than the 
SPOE

protocol) and value.

Hi, the main goal is providing:
- a format easy to decode
- not adding lot of dependencies to haproxy

So, after brainstorm, I propose:

- remove all data already accessible via existing sample-fetches from 
the new

  sample-fetch introduced by my previous patch. These data are:
  method, path, query string, http-version and the body. All of these 
data are
  available in some sample fetches and it will be great to reuse it. 
Note that

  the only data which keep are the headers.

- For the headers I propose two format: The first format is for 
general use. It
  is simply the header block in the same format that we received 
(with the final

  \r\n for detecting truncated header bloc)

- For header I propose also a binary format like this:
  
  [
 
 
  ]…
  This format is easy to decode because each length is encoded with 
the SPOE
  numeric values encoding method, so it is always embedded in SPOE 
client.


The SPOE message description will become

  spoe-message check-request
 uniqueid args method path query req.ver req.hdrs_bin 
req.body_size req.body


uniqueid is a string which allow to do matching between the mod 
security logs and
the haproxy logs. I propose the uniqueid, but this string can 
be another
think like an header containing a uniqueid previously 
generated.


req.hrds_bin is the header bloc in binary format.

req.body_size is the advertised size of the body and it is used to 
determine if the

 body is full or truncated.

Thierry


I personally
think that adding dependencies on tons of libs to implement simple 
stuff
is more 

Re: Certificate order

[Aleksandar Lazic]

Hi.

Am 21-04-2017 11:02, schrieb Sander Hoentjen:

[snipp]


Well, in essence certificate "weights" is what I implemented in the
patch, with the config order determining the weight. While unfortunate
for me, I can understand your reasoning. I do still think my approach
offers more flexibility. Anyway, I attached a new patch which I hope 
you

will consider :)


[snipp]

I was able to patch successfully the 1.8 & 1.7 with your patch ;-)

Regards
Aleks