Nice!
I’m fine with anything! The beauty of YAML is that it’s so easy to parse and
generate that it’s trivial to change over to another format. If you like to
work in Python, that might have the added advantage that it can be switched
over from making HTTP requests to libmodsecurity’s Python bindings in the
future. It would be most cool to run it with libmodsecurity on a ruleset, and
assert that specific rules must be triggered:
- test:
input:
method: 'GET'
url: ‘/?id=1%20OR%20...’
output:
rules: [901234, 912345]
status: 403
This functionality is present in the old regression-tests too, but I haven’t
actually run it. That old testing framework also has another advantage which
Christian suggested in a private mail (and which would be hard to do with the
Go http client): you can really mold the complete request text, so you can
generate HTTP protocol violations and other advanced things. I don’t know if
the Python Requests library allows to just send a custom blurb over the socket.
If so, it would be very cool to copy over the old tests from regression-tests/
too. They could take a format like:
input:
raw: ‘GET / HTTP/1.0...'
Cheers!
WH
> On 27 Feb 2016, at 18:56, Chaim Sanders <csand...@trustwave.com> wrote:
>
> Walter,
> I Love this concept. So much so that I have started constructing my own based
> on a very similar concept
> (https://github.com/csanders-git/owasp-crs-regressions). I have expanded the
> idea of the yaml script such that it has input and output (so that I can
> support Christians content comparison). Additionally, I have started
> designing it so that it will be able to call both python Requests (similar as
> to what you've done) and output curl commands for research purposes. I'll use
> Christian's list as a glossary of features to add. I would have adopted it
> from GO but since the formats are similar, I figured it might be possible to
> just have an alternative GO client, and I'm not cool enough to GO :-). I'll
> keep developing but mine should have no problem taking a superset of yours
> (or being quickly adapted to taking your yaml scripts) because of the
> defaults. This should make them fairly interoperable. Thoughts?
>
> Chaim Sanders
> Security Researcher, SpiderLabs
>
> Trustwave | SMART SECURITY ON DEMAND
> www.trustwave.com
>
> -----Original Message-----
> From: owasp-modsecurity-core-rule-set-boun...@lists.owasp.org
> [mailto:owasp-modsecurity-core-rule-set-boun...@lists.owasp.org] On Behalf Of
> Christian Folini
> Sent: Saturday, February 27, 2016 12:13 AM
> To: Walter Hop
> Cc: owasp-modsecurity-core-rule-set@lists.owasp.org
> Subject: Re: [Owasp-modsecurity-core-rule-set] Paranoia Mode: Forgotten
> controversial ...
>
> Walter,
>
> This is nice. I have tested many http checkers through the years, but somehow
> they all sucked. Your choice of YAML looks valid to me, I might give it a go
> next week or so.
>
> A few feature-related questions :
> - It says it has the ability to consume cookies. Can you delete them as
> well?
> - Can you load data from an external file (passwords spring to mind)?
> - Can I extract a token from a page and use it on a subsequent request?
> - What is the behaviour with multiple http request headers of the same
> name?
> - Can you control the order of http headers?
> - It follows redirects. But I can't check for the redirect status code?
> - I can't test for the content of a http response header, can I?
> - Can I test for status code _and_ content?
>
> Sorry if this is just nitpicking. I know these are advanced features, but
> they are the reason I come back to ugly shell/curl hacks after a week with
> any http checker.
>
> Ahoj,
>
> Christian
>
>
>
> On Fri, Feb 26, 2016 at 10:47:40PM +0100, Walter Hop wrote:
>> I use a little home-built thing to do these kinds of regression tests
>> on our own web services. It walks through a YAML file with tests:
>> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgx
>> vS3v2gRV6ZDw&s=5&u=https%3a%2f%2fgithub%2ecom%2flifeforms%2fhttpcheck
>>
>> I started to write tests for CRSv3, but I didn’t get very far, because
>> I got stuck on the PHP rules issue. Here is the start of my httpcheck
>> manifest with some SQLi tests:
>> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgx
>> vS3q3-FVzIWQ&s=5&u=http%3a%2f%2flf%2ems%2fcrsv3%2eyaml
>> I just do 'httpcheck -i crsv3.yaml -u http://localhost/' to run them on a
>> server.
>>
>> It’s in Go though, because I wanted to test lots of servers
>> concurrently and I wanted it to be fast, so it was a good excuse to
>> try Go. Maybe it’s even a good excuse to learn about its C bindings
>> with libmodsecurity in the future :)
>>
>> Ultimately it could be any tool, I’m not really familiar with the Python
>> world, but I’m in favor of anything that makes it very easy to write a test.
>> Something like just adding a simple line to a file, but with options for
>> doing advanced stuff like POST data, cookies and custom headers. I
>> personally hate writing correct JSON (that trailing comma…) or XML. But I
>> find YAML pretty nice for this case.
>>
>> Cheers!
>> WH
>>
>>> On 22 Feb 2016, at 21:47, Chaim Sanders <csand...@trustwave.com> wrote:
>>>
>>> You’ll note that I have a ticket for updating the regression tests.
>>> I think that this is something that we should look into more.
>>> Potentially, a framework that leverages Python’s Requests Library?
>>> My strong preference is towards python in these types of things, I
>>> might be convinced into using bash curl. Perl need not apply :-P thoughts?
>>>
>>> On 2/22/16, 3:03 PM, "Christian Folini"
>>> <christian.fol...@netnea.com>
>>> wrote:
>>>
>>>> On Mon, Feb 22, 2016 at 07:57:50PM +0000, Chaim Sanders wrote:
>>>>> Actually I¹m really shocked about this. But this is why we have a
>>>>> community :).
>>>>
>>>> We desperately need QA for the core rules.
>>>>
>>>> Alternatively, we could also get hold of Walter and dump his brain
>>>> into an analysis engine. His familiarity with all the individual
>>>> rules is most exceptional.
>>>>
>>>> Christian
>>>>
>>>>
>>>> --
>>>> Croyez ceux qui cherchent la vérité, doutez de ceux qui la trouvent.
>>>> -- André Gide
>>>
>>>
>>> ________________________________
>>>
>>> This transmission may contain information that is privileged, confidential,
>>> and/or exempt from disclosure under applicable law. If you are not the
>>> intended recipient, you are hereby notified that any disclosure, copying,
>>> distribution, or use of the information contained herein (including any
>>> reliance thereon) is strictly prohibited. If you received this transmission
>>> in error, please immediately contact the sender and destroy the material in
>>> its entirety, whether in electronic or hard copy format.
>>> _______________________________________________
>>> Owasp-modsecurity-core-rule-set mailing list
>>> Owasp-modsecurity-core-rule-set@lists.owasp.org
>>> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGF
>>> gxvS3q32SAudAw&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2fl
>>> istinfo%2fowasp-modsecurity-core-rule-set
>>
>> --
>> Walter Hop | PGP key:
>> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgx
>> vS3q_1QQ3MDA&s=5&u=https%3a%2f%2flifeforms%2enl%2fpgp
>>
>
>> _______________________________________________
>> Owasp-modsecurity-core-rule-set mailing list
>> Owasp-modsecurity-core-rule-set@lists.owasp.org
>> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgx
>> vS3q32SAudAw&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flisti
>> nfo%2fowasp-modsecurity-core-rule-set
>
>
> --
> mailto:christian.fol...@netnea.com
> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgxvS3qijFA3GDg&s=5&u=http%3a%2f%2fwww%2echristian-folini%2ech
> twitter: @ChrFolini
> _______________________________________________
> Owasp-modsecurity-core-rule-set mailing list
> Owasp-modsecurity-core-rule-set@lists.owasp.org
> http://scanmail.trustwave.com/?c=4062&d=pbbR1ldGV12U_iMDiY8XX8Z4CqGFgxvS3q32SAudAw&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flistinfo%2fowasp-modsecurity-core-rule-set
>
> ________________________________
>
> This transmission may contain information that is privileged, confidential,
> and/or exempt from disclosure under applicable law. If you are not the
> intended recipient, you are hereby notified that any disclosure, copying,
> distribution, or use of the information contained herein (including any
> reliance thereon) is strictly prohibited. If you received this transmission
> in error, please immediately contact the sender and destroy the material in
> its entirety, whether in electronic or hard copy format.
_______________________________________________
Owasp-modsecurity-core-rule-set mailing list
Owasp-modsecurity-core-rule-set@lists.owasp.org
https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
