Just realized that the ASSERT() in binpac's pac_type.cc is only effective for
DEBUG build, the non-debug
version will accept &exportsourcedata for incremental inputs happily. But I
need DBG_LOG() macro, so I
just comment out 2 ASSERT() in pac_type.cc, everything goes smoothly so far :)
diff --git a/src/pac_type.cc b/src/pac_type.cc
index 1b827ea..eb8b868 100644
--- a/src/pac_type.cc
+++ b/src/pac_type.cc
@@ -837,7 +837,7 @@ bool Type::AddSizeVar(Output* out_cc, Env* env)
if ( StaticSize(env) >= 0 )
return false;
- ASSERT(! incremental_input());
+ //ASSERT(! incremental_input());
ID *size_var_id = new ID(strfmt("%s__size",
value_var() ? value_var()->Name() : decl_id()->Name()));
@@ -854,7 +854,7 @@ bool Type::AddSizeVar(Output* out_cc, Env* env)
string Type::EvalLengthExpr(Output* out_cc, Env* env)
{
- ASSERT(!incremental_input());
+ //ASSERT(!incremental_input());
ASSERT(attr_length_expr_);
int static_length;
if ( attr_length_expr_->ConstFold(env, &static_length) )
I have checked the generated code, everything is fine except a harmless
superflours statement near the end
of the type's ParseBuffer() method:
sourcedata_.set_end(t_begin_of_data + t_TEST_Req_plain__size);
Really I can not think of a reason why incremental types cannot have a
sourcedata field.
------------------ Original ------------------
From: "Song"<[email protected]>;
Date: Sat, Mar 9, 2019 05:39 PM
To: "[email protected]"<[email protected]>;
Cc: "ronka_mata"<[email protected]>;
Subject: Re: [Zeek-Dev] Fwd: Re: Fwd: binpac crash triggeredbyexportsourcedata
I tried to set flow type of the controlling analyzer to datagram and used
&exportsourcedata. Although
the resulted analyzer works great against my test pcap file, but after checking
the code binpac generated
I think the datagram analyzer is not suited to the TCP-based protocols. Below
are the generated codes:
2308 void TEST_Flow::NewData(const_byteptr t_begin_of_data, const_byteptr
t_end_of_data)
2309 {
2310 try
2311 {
2312 dataunit_ = new TEST_PDU(is_orig());
2313 context_ = new ContextTEST(connection(), this);
2314 int t_dataunit__size;
2315 t_dataunit__size = dataunit_->Parse(t_begin_of_data,
t_end_of_data, context_);
2316 // Evaluate 'let' and 'withinput' fields
2317 delete dataunit_;
2318 dataunit_ = 0;
2319 delete context_;
2320 context_ = 0;
2321 }
2322 catch ( binpac::Exception const &e )
2323 {
2324 delete dataunit_;
2325 dataunit_ = 0;
2326 delete context_;
2327 context_ = 0;
2328 throw;
2329 }
2330 }
Notice that in line #2312, every piece of data will be treated as a new PDU
which obviously is not good for TCP
data stream.
I think now the only option I have is to build a new bytestring from the length
and data fields and to feed it to
the RPC analyzer. This solution is bad from the performance point of view
since we have to do 2 extra memory
copy: first to generate data field, second to regenerate the original whole PDU.
------------------ Original ------------------
From: "Song"<[email protected]>;
Date: Sat, Mar 9, 2019 11:13 AM
To: "ronka_mata"<[email protected]>;
Cc: "[email protected]"<[email protected]>;
Subject: Re: [Zeek-Dev] Fwd: Re: Fwd: binpac crash triggered byexportsourcedata
Hi Ronka,
The protocol I'm trying to analyze supports multiple authentication methods,
including SASL Kerberos GSSAPI.
After authentication, according to the authentication method chosen and
security layer negotiated, the RPC
requests/responses followed could be in plain text, signed or encrypted.
In the plain text form, the PDU is like:
<4 bytes length field>
<request/response data with length indicated by the 4 bytes length field>
While in signed or encrypted form, the outmost layer of PDU is like:
<4 bytes length field>
<Kerberos 5 GSSAPI Wrap Token with length indicated by the 4 bytes length
field>
In the later case, the RPC requests/responses PDU (including the 4 bytes length
field indicating the length of the
request/response data) is encapsulated in the Wrap Tokens. It is possible that
a big RPC request/response will
be carried by multiple Wrap Token PDUs.
So I have two analyzers:
- controlling analyzer: deal with authentication and decryption, forward
decrypted RPC PDU data to RPC analyzer
- RPC analyzer: decode RPC request/response
I need the &exportsourcedata for the plain text case in which the whole
controlling analyzer PDU should be forwarded
to the RPC analyzer.
Today I will try to change the type of controlling analyzer to datagram.
Best regards,
Song
------------------ Original ------------------
From: "ronka_mata"<[email protected]>;
Date: Fri, Mar 8, 2019 10:08 PM
To: "Song"<[email protected]>;
Cc: "[email protected]"<[email protected]>;
Subject: Re: Fwd: Re: Fwd: [Zeek-Dev] binpac crash triggered by
exportsourcedata
Hi Song,
Could you explain a bit more what you are trying to achieve? Do you want to
deliver the same data into two analyzers? Or is it just part of it? Or deliver
to one if condition is met and to second one otherwise?
Do you have to wait with second analyzer after it was passed by the first
interpreter or can you call second in the deliverstream function? Make the
second one into child analyzer and then call ForwardStream function or some
similar approach?
I understand where your problem is with your current code. I was not able to
get around the len problem yet, but I will give it a go a bit later today,
unless someone else knows solution first hand.
For delivery of parts of stream data, defined as a bytestream, you can take
example from smb-pipe.pac forward_dce_rpc funct.
Hope this helps a bit.
Ronka
__________________________________________________________
> Od: "Song" <[email protected]>
> Komu: "ronka_mata" <[email protected]>
> Datum: 08.03.2019 05:01
> Předmět: Re: Fwd: Re: Fwd: [Zeek-Dev] binpac crash triggered by
> exportsourcedata
>
> CC: <[email protected]>
Thank you Ronka.It is a flowunit analyzer. I checked zeek source tree and found
that there is only 1 flowunitanalyzer (tls-handshake) uses exportsourcedata
directive. I guess that exportsourcedata onlyapply to non-incremental types.
Maybe these are true: - all types in a datagram analyzer can use
exportsourcedata directive - only non-incremental types in a flowunit analyzer
can use exportsourcedataBut I'm not sure about what is non-incremental type, I
have to check the generated code.The reason that I want sourcedata field is
that I want to feed the whole test_pdu to anotheranalyzer. Now as a workaround,
I have to do something like this: test_rpc->DeliverStream(${data}.length() + 4,
${data}.begin() - 4, is_orig);to bring back the first 4 bytes to form the
original whole PDU.Maybe I should try datagram analyzer.Song------------------
Original ------------------From: "ronka_mata"<[email protected]>;Date:
Thu, Mar 7, 2019 10:05 PMTo: "Song"<[email protected]>;Cc: "zee
k-dev"<[email protected]>; Subject: Fwd: Re: Fwd: [Zeek-Dev] binpac crash
triggered by exportsourcedataHi,What might help is checking how you defined the
the PDU in .pac file. If it is datagram, mostly used for DNS type traffic or if
it is flowunit. You can read more on it here
https://github.com/zeek/binpac/blob/master/README.rst#flowYou do not need to
define length for datagrams. Look at other protocols for example of
differences. Eg radius for datagrams and smb for flows.Ronka----------
Forwarded message ---------From: Song <[email protected]>Date: Thu, Mar 7,
2019, 13:40Subject: [Zeek-Dev] binpac crash triggered by exportsourcedataTo:
zeek-dev <[email protected]>Hi,I define a PDU like below:type test_pdu = record
{ lenAB : uint32; pduAB : test_pdu_ab(lenAB);} &length=(lenAB
+ 4), &exportsourcedata; # fail to compile without &length, &exportsourcedata
will cause binpac crashtype test_pdu_ab(len: uint32) = record { lenA
: uint
16; dataA : bytestring &length = lenA; dataB : bytestring
&length = (len - 2 - lenA);} &exportsourcedata; # &exportsourcedata here is
OKThe error message is:binpac:
/home/grid/git/zeek/aux/binpac/src/pac_type.cc:857: std::__cxx11::string
Type::EvalLengthExpr(Output*, Env*): Assertion `!incremental_input()!`
failed.Aborted (core dumped)_______________________________________________
zeek-dev mailing list
[email protected]
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev
