Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Chema Gonzalez]

And the timings:

$ time tshark -n -T fields -e frame.number -e frame.time_epoch -e
ip.proto -e ip.src -e ip.dst -e ip.len -e tcp.srcport -e tcp.dstport
-e tcp.seq -e tcp.len -e tcp.nxtseq -e tcp.flags.syn -e tcp.flags.ack
-r Traffic2*.cap > /dev/null

real  2m36.218s
user  2m33.604s
sys 0m2.464s

$ time tshark -Xlua_script:seq.lua -n -T fields -e frame.number -e
frame.time_epoch -e ip.proto -e ip.src -e ip.dst -e ip.len -e
tcp.srcport -e tcp.dstport -e tcp.seq -e tcp.seq_abs -e tcp.len -e
tcp.nxtseq -e tcp.flags.syn -e tcp.flags.ack -r Traffic2*.cap >
/dev/null

real  3m4.884s
user  3m1.468s
sys 0m3.396s

$ python -c "print (184-156) / 156."
0.179487179487

-Chema



On Thu, Mar 30, 2017 at 10:16 AM, Chema Gonzalez  wrote:
> On Thu, Mar 30, 2017 at 10:03 AM, Peter Wu  wrote:
>> On Thu, Mar 30, 2017 at 09:40:13AM -0700, Chema Gonzalez wrote:
>> [..]
>>> > Alternatively, you could use a Lua post-dissector to parse out the
>>> > buffer that backs the field ("seq.range" below):
>>> >
>>> > tshark -Xlua_script:seq.lua -Tfields -e tcp.seq -e tcp.seq_abs ...
>>> >
>>> > -- seq.lua
>>> > local myproto = Proto("dummy", "dummy description")
>>> > myproto.fields.seq = ProtoField.uint32("tcp.seq_abs", "Abs seq no")
>>> > local tcp_seq = Field.new("tcp.seq")
>>> > function myproto.dissector(tvb, pinfo, tree)
>>> > local seq = tcp_seq()
>>> > if seq then
>>> > tree:add(myproto.fields.seq, seq.range)
>>> > end
>>> > end
>>> > register_postdissector(myproto)
>>> Hmm.. When I saw this, I was worried about performance. I tried, and
>>> it only adds an extra 20% time. It solves my problem faster than
>>> writing a patch to add tcp.seqraw.
>>>
>>> Thanks a lot!
>>
>> Great, that is a good trace off I suppose :-)
>>
>> Though 20% is still a lot. Is somebody interested to do some profiling?
>> Chema, what is your environment (OS, OS version) and the approximate
>> description of your data (pcap with x frames and y% TCP).
> Linux 3.13.0-108.
>
> The trace is 1.4 GB long, 1.5M full packets, almost all http/tcp.
>
> $ tcpdump -n -nn -r Traffic2*cap|wc -l
> ...
> 1538474
> $ tcpdump -n -nn -r Traffic2*cap tcp |wc -l
> ...
> 1538464
>
> -Chema
>
>
>> --
>> Kind regards,
>> Peter Wu
>> https://lekensteyn.nl
>> ___
>> Sent via:Wireshark-dev mailing list 
>> Archives:https://www.wireshark.org/lists/wireshark-dev
>> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>>  mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Chema Gonzalez]

On Thu, Mar 30, 2017 at 10:03 AM, Peter Wu  wrote:
> On Thu, Mar 30, 2017 at 09:40:13AM -0700, Chema Gonzalez wrote:
> [..]
>> > Alternatively, you could use a Lua post-dissector to parse out the
>> > buffer that backs the field ("seq.range" below):
>> >
>> > tshark -Xlua_script:seq.lua -Tfields -e tcp.seq -e tcp.seq_abs ...
>> >
>> > -- seq.lua
>> > local myproto = Proto("dummy", "dummy description")
>> > myproto.fields.seq = ProtoField.uint32("tcp.seq_abs", "Abs seq no")
>> > local tcp_seq = Field.new("tcp.seq")
>> > function myproto.dissector(tvb, pinfo, tree)
>> > local seq = tcp_seq()
>> > if seq then
>> > tree:add(myproto.fields.seq, seq.range)
>> > end
>> > end
>> > register_postdissector(myproto)
>> Hmm.. When I saw this, I was worried about performance. I tried, and
>> it only adds an extra 20% time. It solves my problem faster than
>> writing a patch to add tcp.seqraw.
>>
>> Thanks a lot!
>
> Great, that is a good trace off I suppose :-)
>
> Though 20% is still a lot. Is somebody interested to do some profiling?
> Chema, what is your environment (OS, OS version) and the approximate
> description of your data (pcap with x frames and y% TCP).
Linux 3.13.0-108.

The trace is 1.4 GB long, 1.5M full packets, almost all http/tcp.

$ tcpdump -n -nn -r Traffic2*cap|wc -l
...
1538474
$ tcpdump -n -nn -r Traffic2*cap tcp |wc -l
...
1538464

-Chema


> --
> Kind regards,
> Peter Wu
> https://lekensteyn.nl
> ___
> Sent via:Wireshark-dev mailing list 
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Chema Gonzalez]

On Thu, Mar 30, 2017 at 9:15 AM, Peter Wu  wrote:
>> My goal was getting both at the same time. Unless I'm mistaken, I can
>> only get either the relative or the absolute seq number.
>
> If you can parse the PDML (XML) output (tshar -Tpdml), the data is
> already there:
>
>pos="34">
>   ...
>
> 
> 
>
> Here you cvan see that the relative seq number is 1 and that the
> absolute value is eaee29e7 (hexadecimal for 3941476839).
I'm trying to process very large traces efficiently. Parsing xml is
likely a bad option.

> Alternatively, you could use a Lua post-dissector to parse out the
> buffer that backs the field ("seq.range" below):
>
> tshark -Xlua_script:seq.lua -Tfields -e tcp.seq -e tcp.seq_abs ...
>
> -- seq.lua
> local myproto = Proto("dummy", "dummy description")
> myproto.fields.seq = ProtoField.uint32("tcp.seq_abs", "Abs seq no")
> local tcp_seq = Field.new("tcp.seq")
> function myproto.dissector(tvb, pinfo, tree)
> local seq = tcp_seq()
> if seq then
> tree:add(myproto.fields.seq, seq.range)
> end
> end
> register_postdissector(myproto)
Hmm.. When I saw this, I was worried about performance. I tried, and
it only adds an extra 20% time. It solves my problem faster than
writing a patch to add tcp.seqraw.

Thanks a lot!

-Chema



> --
> Kind regards,
> Peter Wu
> https://lekensteyn.nl
> ___
> Sent via:Wireshark-dev mailing list 
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Peter Wu]

On Thu, Mar 30, 2017 at 08:39:31AM -0700, Chema Gonzalez wrote:
> Hi,
> 
> On Thu, Mar 30, 2017 at 12:23 AM, Pascal Quantin wrote:
> > Hi Chema,
> >
> > 2017-03-30 1:32 GMT+02:00 Chema Gonzalez:
> >>
> >> Hi,
> >>
> >> I'm using tshark to extract some fields from packet traces. Using `-e
> >> tcp.seq`, tshark prints the relative sequence number. I'd like to
> >> print the raw (absolute) at the same time. I don't think this is
> >> possible right now (but please let me know if that's the case).

> My goal was getting both at the same time. Unless I'm mistaken, I can
> only get either the relative or the absolute seq number.

If you can parse the PDML (XML) output (tshar -Tpdml), the data is
already there:

  
  ...




Here you cvan see that the relative seq number is 1 and that the
absolute value is eaee29e7 (hexadecimal for 3941476839).

Alternatively, you could use a Lua post-dissector to parse out the
buffer that backs the field ("seq.range" below):

tshark -Xlua_script:seq.lua -Tfields -e tcp.seq -e tcp.seq_abs ...

-- seq.lua
local myproto = Proto("dummy", "dummy description")
myproto.fields.seq = ProtoField.uint32("tcp.seq_abs", "Abs seq no")
local tcp_seq = Field.new("tcp.seq")
function myproto.dissector(tvb, pinfo, tree)
local seq = tcp_seq()
if seq then
tree:add(myproto.fields.seq, seq.range)
end
end
register_postdissector(myproto)
-- 
Kind regards,
Peter Wu
https://lekensteyn.nl
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Chema Gonzalez]

Hi,

On Thu, Mar 30, 2017 at 12:23 AM, Pascal Quantin
 wrote:
> Hi Chema,
>
> 2017-03-30 1:32 GMT+02:00 Chema Gonzalez :
>>
>> Hi,
>>
>> I'm using tshark to extract some fields from packet traces. Using `-e
>> tcp.seq`, tshark prints the relative sequence number. I'd like to
>> print the raw (absolute) at the same time. I don't think this is
>> possible right now (but please let me know if that's the case).
>>
>> A quick check at the code suggests I need to set tcp_relative_seq to
>> FALSE to have absolute tcp seq numbers. I can't see how to set this
>> value using the tshark CLI.
>
>
> simply add the following to your command line:
> -o "tcp.relative_sequence_numbers: false"
>
> so your command becomes:
> tshark -r test.pcapng -T fields -e tcp.seq -o
> "tcp.relative_sequence_numbers: false"
Great! Thanks a lot.

>> Final question: Any hints on what's the best way to add a "tcp.rawseq"
>> ("tcp.seqraw"?) option?
>
>
> Given that there is already an option for this, is it really required ?
My goal was getting both at the same time. Unless I'm mistaken, I can
only get either the relative or the absolute seq number.

Thanks,
-Chema



>
>>
>>
>> Thanks,
>> -Chema
>>
>> ___
>> Sent via:Wireshark-dev mailing list 
>> Archives:https://www.wireshark.org/lists/wireshark-dev
>> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>>
>> mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
>
>
>
> ___
> Sent via:Wireshark-dev mailing list 
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] tshark: access to tcp raw seq number

[Pascal Quantin]

Hi Chema,

2017-03-30 1:32 GMT+02:00 Chema Gonzalez :

> Hi,
>
> I'm using tshark to extract some fields from packet traces. Using `-e
> tcp.seq`, tshark prints the relative sequence number. I'd like to
> print the raw (absolute) at the same time. I don't think this is
> possible right now (but please let me know if that's the case).
>
> A quick check at the code suggests I need to set tcp_relative_seq to
> FALSE to have absolute tcp seq numbers. I can't see how to set this
> value using the tshark CLI.
>

simply add the following to your command line:
-o "tcp.relative_sequence_numbers: false"

so your command becomes:
tshark -r test.pcapng -T fields -e tcp.seq -o
"tcp.relative_sequence_numbers: false"


>
> Final question: Any hints on what's the best way to add a "tcp.rawseq"
> ("tcp.seqraw"?) option?
>

Given that there is already an option for this, is it really required ?


>
> Thanks,
> -Chema
> 
> ___
> Sent via:Wireshark-dev mailing list 
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=
> unsubscribe
>
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

[Wireshark-dev] tshark: access to tcp raw seq number

[Chema Gonzalez]

Hi,

I'm using tshark to extract some fields from packet traces. Using `-e
tcp.seq`, tshark prints the relative sequence number. I'd like to
print the raw (absolute) at the same time. I don't think this is
possible right now (but please let me know if that's the case).

A quick check at the code suggests I need to set tcp_relative_seq to
FALSE to have absolute tcp seq numbers. I can't see how to set this
value using the tshark CLI.

Final question: Any hints on what's the best way to add a "tcp.rawseq"
("tcp.seqraw"?) option?

Thanks,
-Chema
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe