Re: [Wireshark-dev] Setting to disable all expert info

2017-08-04 Thread Dario Lombardo 
On Wed, Aug 2, 2017 at 10:21 PM, Guy Harris  wrote:

>
> Let's wait until the generation of the expert info results in a measurably
> significant CPU or memory use increase before we add a knob to turn it
> off.  There are probably other places to look at to reduce memory usage
> first.
>

That's an interesting point... any idea what they are?
I worked on the conversation code that looks one of the most memory eaters.
I was working on purging pending conversations after a while. The idea was
that if I can't find the corresponding packet of a conversation in a bunch
of seconds (let's say 5), it is very unlikely that I will find it. I am
aware that some protocols could have looser timing constraints and this
approach wouldn't work for them, but I was just playing with the idea. What
do you think?
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Sultan, Hassan via Wireshark-dev 
Thanks for the link Pascal, I wasn't aware of it. I'll look up how tshark does 
and try to replicate that.

> -Original Message-
> From: Pascal Quantin [mailto:pascal.quan...@gmail.com]
> Sent: Wednesday, August 02, 2017 1:05 PM
> To: Developer support list for Wireshark <wireshark-dev@wireshark.org>
> Cc: Sultan, Hassan <sul...@amazon.com>
> Subject: Re: [Wireshark-dev] Setting to disable all expert info
> 
> 
> 
> 2017-08-02 22:00 GMT+02:00 Sultan, Hassan via Wireshark-dev  d...@wireshark.org <mailto:wireshark-dev@wireshark.org> >:
> 
> 
>   Here's my scenario :
> 
>   I am planning on using the Wireshark parsing engine in two ways :
>   1) process massively large captures
>   2) process live traffic, hopefully in the long term in a permanent 
> manner
> once the memory growth of the engine can be controlled
> 
> 
> 
> Did you have a look at https://blog.wireshark.org/2014/07/to-infinity-and-
> beyond-capturing-forever-with-tshark/ ?
> 
> 
> 
>   In both cases, my automation does not care about any experts at all, it
> only needs the field information (length/offset/name/hierarchy/encoding...) so
> these experts are consuming memory for nothing in our case.
> 
>   As Wireshark evolves, new dissectors get added, which might add new
> experts, and not having a global setting to turn them off would force us to
> repeatedly search for new experts appearing, which is not a great experience.
> 
>   Now granted, I haven't done measurements on how much memory they
> consume, the current experts might not represent much, but it's hard to judge
> when experts can appear in any new dissector, one could appear tomorrow that
> significantly alters that.
> 
> 
> 
> Indeed they probably do not represent much compared to all the fields
> registered by dissectors. Moreover you are the first one I remember asking for
> such a feature. Like Jaap, I do not think this is a good move as of today.
> 
> 
> 
> 
>   Thanks,
> 
>   Hassan
> 
> 
>   > -Original Message-
>   > From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org
> <mailto:wireshark-dev-boun...@wireshark.org> ] On Behalf
>   > Of Jaap Keuter
>   > Sent: Wednesday, August 02, 2017 11:59 AM
>   > To: Sultan, Hassan via Wireshark-dev <wireshark-dev@wireshark.org
> <mailto:wireshark-dev@wireshark.org> >
>   > Subject: Re: [Wireshark-dev] Setting to disable all expert info
>   >
>   > Are we going to be picking off features one by one to get the memory
> footprint
>   > down? Then I see a long list of preference settings growing from this.
> Not
>   > something I look forward to.
>   >
>   >
>   > On 02-08-17 20:43, Sultan, Hassan via Wireshark-dev wrote:
>   > > Hi,
>   > >
>   > >
>   > >
>   > > Am I right in my understanding that there is no global way of
>   > > disabling insertion of expert information ?
>   > >
>   > >
>   > >
>   > > Assuming I’m correct, would anyone object to me adding that
> setting ?
>   > > That would be another way of lowering memory footprint.
>   > >
>   > >
>   > >
>   > > Thx,
>   > >
>   > >
>   > >
>   > > Hassan
>   > >
>   > >
>   >
> _
>   > __
>   > Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org
> <mailto:wireshark-dev@wireshark.org> >
>   > Archives:https://www.wireshark.org/lists/wireshark-dev
> <https://www.wireshark.org/lists/wireshark-dev>
>   > Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-
> dev <https://www.wireshark.org/mailman/options/wireshark-dev>
>   >  mailto:wireshark-dev-requ...@wireshark.org
> <mailto:wireshark-dev-requ...@wireshark.org> ?subject=unsubscribe
>   __
> _
>   Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org
> <mailto:wireshark-dev@wireshark.org> >
>   Archives:https://www.wireshark.org/lists/wireshark-dev
> <https://www.wireshark.org/lists/wireshark-dev>
>   Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-
> dev <https://www.wireshark.org/mailman/options/wireshark-dev>
>mailto:wireshark-dev-requ...@wireshark.org 
> <mailto:wireshark-
> dev-requ...@wireshark.org> ?subject=unsubscribe
> 

___
Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Guy Harris 
On Aug 2, 2017, at 1:05 PM, Pascal Quantin  wrote:

> Indeed they probably do not represent much compared to all the fields 
> registered by dissectors. Moreover you are the first one I remember asking 
> for such a feature. Like Jaap, I do not think this is a good move as of today.

+1

Let's wait until the generation of the expert info results in a measurably 
significant CPU or memory use increase before we add a knob to turn it off.  
There are probably other places to look at to reduce memory usage first.
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Pascal Quantin 
2017-08-02 22:00 GMT+02:00 Sultan, Hassan via Wireshark-dev <
wireshark-dev@wireshark.org>:

> Here's my scenario :
>
> I am planning on using the Wireshark parsing engine in two ways :
> 1) process massively large captures
> 2) process live traffic, hopefully in the long term in a permanent manner
> once the memory growth of the engine can be controlled
>

Did you have a look at
https://blog.wireshark.org/2014/07/to-infinity-and-beyond-capturing-forever-with-tshark/
?

>
> In both cases, my automation does not care about any experts at all, it
> only needs the field information (length/offset/name/hierarchy/encoding...)
> so these experts are consuming memory for nothing in our case.
>
> As Wireshark evolves, new dissectors get added, which might add new
> experts, and not having a global setting to turn them off would force us to
> repeatedly search for new experts appearing, which is not a great
> experience.
>
> Now granted, I haven't done measurements on how much memory they consume,
> the current experts might not represent much, but it's hard to judge when
> experts can appear in any new dissector, one could appear tomorrow that
> significantly alters that.
>

Indeed they probably do not represent much compared to all the fields
registered by dissectors. Moreover you are the first one I remember asking
for such a feature. Like Jaap, I do not think this is a good move as of
today.


> Thanks,
>
> Hassan
>
> > -Original Message-
> > From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] On
> Behalf
> > Of Jaap Keuter
> > Sent: Wednesday, August 02, 2017 11:59 AM
> > To: Sultan, Hassan via Wireshark-dev <wireshark-dev@wireshark.org>
> > Subject: Re: [Wireshark-dev] Setting to disable all expert info
> >
> > Are we going to be picking off features one by one to get the memory
> footprint
> > down? Then I see a long list of preference settings growing from this.
> Not
> > something I look forward to.
> >
> >
> > On 02-08-17 20:43, Sultan, Hassan via Wireshark-dev wrote:
> > > Hi,
> > >
> > >
> > >
> > > Am I right in my understanding that there is no global way of
> > > disabling insertion of expert information ?
> > >
> > >
> > >
> > > Assuming I’m correct, would anyone object to me adding that setting ?
> > > That would be another way of lowering memory footprint.
> > >
> > >
> > >
> > > Thx,
> > >
> > >
> > >
> > > Hassan
> > >
> > >
> > _
> > __
> > Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
> > Archives:https://www.wireshark.org/lists/wireshark-dev
> > Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
> >  mailto:wireshark-dev-requ...@wireshark.org?subject=
> unsubscribe
> 
> ___
> Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=
> unsubscribe
>
___
Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Sultan, Hassan via Wireshark-dev 
Here's my scenario :

I am planning on using the Wireshark parsing engine in two ways :
1) process massively large captures
2) process live traffic, hopefully in the long term in a permanent manner once 
the memory growth of the engine can be controlled

In both cases, my automation does not care about any experts at all, it only 
needs the field information (length/offset/name/hierarchy/encoding...) so these 
experts are consuming memory for nothing in our case.

As Wireshark evolves, new dissectors get added, which might add new experts, 
and not having a global setting to turn them off would force us to repeatedly 
search for new experts appearing, which is not a great experience.

Now granted, I haven't done measurements on how much memory they consume, the 
current experts might not represent much, but it's hard to judge when experts 
can appear in any new dissector, one could appear tomorrow that significantly 
alters that.

Thanks,

Hassan

> -Original Message-
> From: Wireshark-dev [mailto:wireshark-dev-boun...@wireshark.org] On Behalf
> Of Jaap Keuter
> Sent: Wednesday, August 02, 2017 11:59 AM
> To: Sultan, Hassan via Wireshark-dev <wireshark-dev@wireshark.org>
> Subject: Re: [Wireshark-dev] Setting to disable all expert info
> 
> Are we going to be picking off features one by one to get the memory footprint
> down? Then I see a long list of preference settings growing from this. Not
> something I look forward to.
> 
> 
> On 02-08-17 20:43, Sultan, Hassan via Wireshark-dev wrote:
> > Hi,
> >
> >
> >
> > Am I right in my understanding that there is no global way of
> > disabling insertion of expert information ?
> >
> >
> >
> > Assuming I’m correct, would anyone object to me adding that setting ?
> > That would be another way of lowering memory footprint.
> >
> >
> >
> > Thx,
> >
> >
> >
> > Hassan
> >
> >
> _
> __
> Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
___
Sent via:Wireshark-dev mailing list <wireshark-dev@wireshark.org>
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Pascal Quantin 
Hi Hassan,

2017-08-02 20:43 GMT+02:00 Sultan, Hassan via Wireshark-dev <
wireshark-dev@wireshark.org>:

> Hi,
>
>
>
> Am I right in my understanding that there is no global way of disabling
> insertion of expert information ?
>

You are right.


>
>
> Assuming I’m correct, would anyone object to me adding that setting ? That
> would be another way of lowering memory footprint.
>

Which memory footprint are you referring to, and for which use case would
you like to filter expert info?

Best regards,
Pascal.
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] Setting to disable all expert info

2017-08-02 Thread Jaap Keuter 
Are we going to be picking off features one by one to get the memory footprint
down? Then I see a long list of preference settings growing from this. Not
something I look forward to.


On 02-08-17 20:43, Sultan, Hassan via Wireshark-dev wrote:
> Hi,
> 
>  
> 
> Am I right in my understanding that there is no global way of disabling
> insertion of expert information ?
> 
>  
> 
> Assuming I’m correct, would anyone object to me adding that setting ? That 
> would
> be another way of lowering memory footprint.
> 
>  
> 
> Thx,
> 
>  
> 
> Hassan
> 
> 
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe