Re: [Wireshark-dev] Adding support for a new PCAP-NG block

[Guy Harris]

On Jan 19, 2018, at 12:24 AM, Guy Harris  wrote:

> On Jan 17, 2018, at 4:47 AM, Paul Offord  wrote:
> 
>> I want to make a start on the plan below.  Last night I took a look at the 
>> relevant code.
>> 
>> I started by adding support for TSDBs into the function pcapng_open(…) in 
>> pcapng.c
> 
> The *first* thing to do is to start by either
> 
>   1) getting an official block type value from pcap-ng-for...@winpcap.org
> 
> or
> 
>   2) getting a Private Enterprise Number from the IANA and using a custom 
> block:
> 
>   
> http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/pcapng/pcapng/master/draft-tuexen-opsawg-pcapng.xml=html/ascii=ascii#rfc.section.4.7

Or

3) using a "reserved for local use" block type:


http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/pcapng/pcapng/master/draft-tuexen-opsawg-pcapng.xml=html/ascii=ascii#rfc.section.11.1

but bear in mind that the standard Wireshark releases will never include 
built-in support for any "reserved for local use" block type, and there is no 
guarantee that somebody else won't use the same "reserved for local use" type, 
so that should be used only for types that 1) you're only going to use at your 
site or 2) that you're experimenting with prior to getting an official block 
type value or getting a Private Enterprise Number and using a custom block.
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Adding support for a new PCAP-NG block

[Guy Harris]

On Jan 17, 2018, at 4:47 AM, Paul Offord  wrote:

> I want to make a start on the plan below.  Last night I took a look at the 
> relevant code.
>  
> I started by adding support for TSDBs into the function pcapng_open(…) in 
> pcapng.c

The *first* thing to do is to start by either

1) getting an official block type value from pcap-ng-for...@winpcap.org

or

2) getting a Private Enterprise Number from the IANA and using a custom 
block:


http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/pcapng/pcapng/master/draft-tuexen-opsawg-pcapng.xml=html/ascii=ascii#rfc.section.4.7

> but I then stumbled across wtap_opttype_register_custom_block_type(…) in 
> wtap_opttypes.c which seems to be a framework to add support for new block 
> types.  I did a check on code that refers to this function and found that 
> nothing uses it at the moment.

That's correct - pcapng.c doesn't currently support custom blocks.
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Adding support for a new PCAP-NG block

[Paul Offord]

Hi Roland.

Does that mean that from within a dissector I could:


  *   Register the new block type
  *   Get access to the new block data from the dissector

Basically, can I do everything from a new dissector and not touch Wireshark 
core code?

Thanks and regards…Paul

From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] On Behalf Of 
Roland Knall
Sent: 17 January 2018 14:05
To: Developer support list for Wireshark <wireshark-dev@wireshark.org>
Subject: Re: [Wireshark-dev] Adding support for a new PCAP-NG block

Hi

Yes you should use the wtap_opttype_register_custom_block_type structure. In 
theory, the data from the block get's stored inside the necessary wtap 
structures and can later be accessible via packet_info if I remember correctly. 
Or you could also write a separate dissector, store the data from there in 
packet_info and call it from your original dissector.

And no, there is no documentation as of yet.

I would just start to write a dissector for the block only. If you have that, I 
would start a discussion about data storage from there.

regards
Roland

On Wed, Jan 17, 2018 at 1:47 PM, Paul Offord 
<paul.off...@advance7.com<mailto:paul.off...@advance7.com>> wrote:
I want to make a start on the plan below.  Last night I took a look at the 
relevant code.

I started by adding support for TSDBs into the function pcapng_open(…) in 
pcapng.c but I then stumbled across wtap_opttype_register_custom_block_type(…) 
in wtap_opttypes.c which seems to be a framework to add support for new block 
types.  I did a check on code that refers to this function and found that 
nothing uses it at the moment.

So my questions are:


  *   Should I modify pcapng_open or use 
wtap_opttype_register_custom_block_type?
  *   If the latter, is there any documentation, or even just an explanation of 
wtap_opttype_register_custom_block_type and how to use it?
  *   I will need to save interpreted data from the new TSDBs somewhere that 
will later be accessible to dissectors.  Which structure should I use?

Thanks and regards…Paul


From: Paul Offord
Sent: 03 November 2017 16:41
To: Developer support list for Wireshark 
<wireshark-dev@wireshark.org<mailto:wireshark-dev@wireshark.org>>
Subject: RE: [Wireshark-dev] Capture filename not available at plugin init time

So the plan would be:


  *   Add support to read the TSDB and create the resulting structures
  *   Add support to read Text Record Blocks (TRBs)

 *   This is mostly stuff that Guy Harris described a while back
 *   In my current code the data records are encapsulated in a dummy 
Ethernet frame

  *   Add support to mergecap to correctly handle the TSDBs

 *   Similar to adjusting IDBs when files are merged

  *   Add the dumpcap code to read text files and produce pcap-ng

I think I’ll leave this idea to circulate for a few days before I start writing 
code.  Maybe I’ll pester the developers next week at SF EU.

I will also see if there is an easy way to get plugin_if_get_ws_info to work in 
the init routine as I still believe this will be useful to dissector developers.

Thanks and regards…Paul

From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] On Behalf Of 
Roland Knall
Sent: 03 November 2017 16:24
To: Developer support list for Wireshark 
<wireshark-dev@wireshark.org<mailto:wireshark-dev@wireshark.org>>
Subject: Re: [Wireshark-dev] Capture filename not available at plugin init time

This is a different thing here. If TSDB is a common code block, I think the 
chances are really good.

But still it needs the basic read functionality in dumpcap

cheers

On Fri, Nov 3, 2017 at 5:18 PM, Paul Offord 
<paul.off...@advance7.com<mailto:paul.off...@advance7.com>> wrote:
OK – I understand.

If I write the code to read the TSDB and make it available do you think it 
would be accepted into the main project?  I’m thinking about my syncro 
experience here.

Thanks and regards…Paul

From: Wireshark-dev 
[mailto:wireshark-dev-boun...@wireshark.org<mailto:wireshark-dev-boun...@wireshark.org>]
 On Behalf Of Roland Knall
Sent: 03 November 2017 14:15

To: Developer support list for Wireshark 
<wireshark-dev@wireshark.org<mailto:wireshark-dev@wireshark.org>>
Subject: Re: [Wireshark-dev] Capture filename not available at plugin init time

Hi Paul

You should never assume, that you will be able to read the file, while WS is 
reading it. If this is working right now, it might be out of pure coincidence, 
that said, the real thing here should be to get dumpcap to use pcapng as input 
format, which would give you the tsdb block where you need it to be, during 
dissection.

The support for any pcap-ng extension block is already in Wireshark. The issue 
still is, to get the block structure through.

cheers
Roland

On Fri, Nov 3, 2017 at 3:07 PM, Paul Offord 
<paul.off...@advance7.com<mailto:paul.off...@advance7.com>> wrote:
Thanks for responding Roland.

I’v

Re: [Wireshark-dev] Adding support for a new PCAP-NG block

[Roland Knall]

Hi

Yes you should use the wtap_opttype_register_custom_block_type structure.
In theory, the data from the block get's stored inside the necessary wtap
structures and can later be accessible via packet_info if I remember
correctly. Or you could also write a separate dissector, store the data
from there in packet_info and call it from your original dissector.

And no, there is no documentation as of yet.

I would just start to write a dissector for the block only. If you have
that, I would start a discussion about data storage from there.

regards
Roland

On Wed, Jan 17, 2018 at 1:47 PM, Paul Offord 
wrote:

> I want to make a start on the plan below.  Last night I took a look at the
> relevant code.
>
>
>
> I started by adding support for TSDBs into the function pcapng_open(…) in
> pcapng.c but I then stumbled across wtap_opttype_register_custom_block_type(…)
> in wtap_opttypes.c which seems to be a framework to add support for new
> block types.  I did a check on code that refers to this function and found
> that nothing uses it at the moment.
>
>
>
> So my questions are:
>
>
>
>- Should I modify pcapng_open or use wtap_opttype_register_custom_
>block_type?
>- If the latter, is there any documentation, or even just an
>explanation of wtap_opttype_register_custom_block_type and how to use
>it?
>- I will need to save interpreted data from the new TSDBs somewhere
>that will later be accessible to dissectors.  Which structure should I use?
>
>
>
> Thanks and regards…Paul
>
>
>
>
>
> *From:* Paul Offord
> *Sent:* 03 November 2017 16:41
> *To:* Developer support list for Wireshark 
> *Subject:* RE: [Wireshark-dev] Capture filename not available at plugin
> init time
>
>
>
> So the plan would be:
>
>
>
>- Add support to read the TSDB and create the resulting structures
>- Add support to read Text Record Blocks (TRBs)
>   - This is mostly stuff that Guy Harris described a while back
>   - In my current code the data records are encapsulated in a dummy
>   Ethernet frame
>- Add support to mergecap to correctly handle the TSDBs
>   - Similar to adjusting IDBs when files are merged
>- Add the dumpcap code to read text files and produce pcap-ng
>
>
>
> I think I’ll leave this idea to circulate for a few days before I start
> writing code.  Maybe I’ll pester the developers next week at SF EU.
>
>
>
> I will also see if there is an easy way to get plugin_if_get_ws_info to
> work in the init routine as I still believe this will be useful to
> dissector developers.
>
>
>
> Thanks and regards…Paul
>
>
>
> *From:* Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org
> ] *On Behalf Of *Roland Knall
> *Sent:* 03 November 2017 16:24
> *To:* Developer support list for Wireshark 
> *Subject:* Re: [Wireshark-dev] Capture filename not available at plugin
> init time
>
>
>
> This is a different thing here. If TSDB is a common code block, I think
> the chances are really good.
>
>
>
> But still it needs the basic read functionality in dumpcap
>
>
>
> cheers
>
>
>
> On Fri, Nov 3, 2017 at 5:18 PM, Paul Offord 
> wrote:
>
> OK – I understand.
>
>
>
> If I write the code to read the TSDB and make it available do you think it
> would be accepted into the main project?  I’m thinking about my syncro
> experience here.
>
>
>
> Thanks and regards…Paul
>
>
>
> *From:* Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] *On
> Behalf Of *Roland Knall
> *Sent:* 03 November 2017 14:15
>
>
> *To:* Developer support list for Wireshark 
> *Subject:* Re: [Wireshark-dev] Capture filename not available at plugin
> init time
>
>
>
> Hi Paul
>
>
>
> You should never assume, that you will be able to read the file, while WS
> is reading it. If this is working right now, it might be out of pure
> coincidence, that said, the real thing here should be to get dumpcap to use
> pcapng as input format, which would give you the tsdb block where you need
> it to be, during dissection.
>
>
>
> The support for any pcap-ng extension block is already in Wireshark. The
> issue still is, to get the block structure through.
>
>
>
> cheers
>
> Roland
>
>
>
> On Fri, Nov 3, 2017 at 3:07 PM, Paul Offord 
> wrote:
>
> Thanks for responding Roland.
>
>
>
> I’ve written a tool that reads a log file and converts it to a PCAP-NG
> with a matching dissector.  The pcap file carries a data descriptor block
> in a new PCAP-NG block type called as TSDB.  The TSDB carries the
> information needed to register the header fields.  To add support for the
> TSDB into core Wireshark is going to be a big job (which I will submit
> later).  As a quick solution, the dissector gets the information by
> directly opening and reading it from the PCAP-NG file – hence the need for
> the filename.
>
>
>
> The above aside, and I 

[Wireshark-dev] Adding support for a new PCAP-NG block

[Paul Offord]

I want to make a start on the plan below.  Last night I took a look at the 
relevant code.

I started by adding support for TSDBs into the function pcapng_open(…) in 
pcapng.c but I then stumbled across wtap_opttype_register_custom_block_type(…) 
in wtap_opttypes.c which seems to be a framework to add support for new block 
types.  I did a check on code that refers to this function and found that 
nothing uses it at the moment.

So my questions are:


  *   Should I modify pcapng_open or use 
wtap_opttype_register_custom_block_type?
  *   If the latter, is there any documentation, or even just an explanation of 
wtap_opttype_register_custom_block_type and how to use it?
  *   I will need to save interpreted data from the new TSDBs somewhere that 
will later be accessible to dissectors.  Which structure should I use?

Thanks and regards…Paul


From: Paul Offord
Sent: 03 November 2017 16:41
To: Developer support list for Wireshark 
Subject: RE: [Wireshark-dev] Capture filename not available at plugin init time

So the plan would be:


  *   Add support to read the TSDB and create the resulting structures
  *   Add support to read Text Record Blocks (TRBs)
 *   This is mostly stuff that Guy Harris described a while back
 *   In my current code the data records are encapsulated in a dummy 
Ethernet frame
  *   Add support to mergecap to correctly handle the TSDBs
 *   Similar to adjusting IDBs when files are merged
  *   Add the dumpcap code to read text files and produce pcap-ng

I think I’ll leave this idea to circulate for a few days before I start writing 
code.  Maybe I’ll pester the developers next week at SF EU.

I will also see if there is an easy way to get plugin_if_get_ws_info to work in 
the init routine as I still believe this will be useful to dissector developers.

Thanks and regards…Paul

From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] On Behalf Of 
Roland Knall
Sent: 03 November 2017 16:24
To: Developer support list for Wireshark 
>
Subject: Re: [Wireshark-dev] Capture filename not available at plugin init time

This is a different thing here. If TSDB is a common code block, I think the 
chances are really good.

But still it needs the basic read functionality in dumpcap

cheers

On Fri, Nov 3, 2017 at 5:18 PM, Paul Offord 
> wrote:
OK – I understand.

If I write the code to read the TSDB and make it available do you think it 
would be accepted into the main project?  I’m thinking about my syncro 
experience here.

Thanks and regards…Paul

From: Wireshark-dev 
[mailto:wireshark-dev-boun...@wireshark.org]
 On Behalf Of Roland Knall
Sent: 03 November 2017 14:15

To: Developer support list for Wireshark 
>
Subject: Re: [Wireshark-dev] Capture filename not available at plugin init time

Hi Paul

You should never assume, that you will be able to read the file, while WS is 
reading it. If this is working right now, it might be out of pure coincidence, 
that said, the real thing here should be to get dumpcap to use pcapng as input 
format, which would give you the tsdb block where you need it to be, during 
dissection.

The support for any pcap-ng extension block is already in Wireshark. The issue 
still is, to get the block structure through.

cheers
Roland

On Fri, Nov 3, 2017 at 3:07 PM, Paul Offord 
> wrote:
Thanks for responding Roland.

I’ve written a tool that reads a log file and converts it to a PCAP-NG with a 
matching dissector.  The pcap file carries a data descriptor block in a new 
PCAP-NG block type called as TSDB.  The TSDB carries the information needed to 
register the header fields.  To add support for the TSDB into core Wireshark is 
going to be a big job (which I will submit later).  As a quick solution, the 
dissector gets the information by directly opening and reading it from the 
PCAP-NG file – hence the need for the filename.

The above aside, and I can guess you are thinking I’m just trying to avoid a 
bigger coding job, it’s not unreasonable to expect plugin_if_get_ws_info to get 
the filename at init time, and init is called when the filename is known.

I have a really kludgie workaround which is to read the TSDB and register the 
hf structure on the first call to the dissector, but it’s not ideal.

Best regards…Paul

From: Wireshark-dev 
[mailto:wireshark-dev-boun...@wireshark.org]
 On Behalf Of Roland Knall
Sent: 03 November 2017 13:53
To: Developer support list for Wireshark 
>
Subject: Re: [Wireshark-dev] Capture filename not available at plugin init time

Hi Paul

As far as I know, cf_open can still fail after calling