> -----Original Message----- > From: Pascal Quantin [mailto:[email protected]] > Sent: Thursday, July 13, 2017 1:16 PM > To: Developer support list for Wireshark <[email protected]> > Cc: Sultan, Hassan <[email protected]> > Subject: Re: [Wireshark-dev] hf_http_response_code in packet-http.c > > > > Le 13 juil. 2017 22:00, "Sultan, Hassan via Wireshark-dev" <wireshark- > [email protected] <mailto:[email protected]> > a écrit : > > > > > > -----Original Message----- > > From: Wireshark-dev [mailto:[email protected] > <mailto:[email protected]> ] On Behalf > > Of Erik de Jong > > > Sent: Thursday, July 13, 2017 11:58 AM > > To: Developer support list for Wireshark <wireshark- > [email protected] <mailto:[email protected]> > > > Subject: Re: [Wireshark-dev] hf_http_response_code in packet-http.c > > > > > > > > > On Thu, Jul 13, 2017 at 8:47 PM, Sultan, Hassan via Wireshark-dev > <wireshark- > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > wrote: > > > > > > > > > > > -----Original Message----- > > > From: Pascal Quantin [mailto:[email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]> > ] > > > > Sent: Thursday, July 13, 2017 11:26 AM > > > To: Sultan, Hassan <[email protected] > <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]> > > > > > Cc: Developer support list for Wireshark <wireshark- > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > > > > > Subject: RE: [Wireshark-dev] hf_http_response_code in packet- > http.c > > > > > > > > > > > > Le 13 juil. 2017 19:54, "Sultan, Hassan" <[email protected] > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > a > > > écrit : > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: Pascal Quantin [mailto:[email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]> > > > > <mailto:[email protected] > <mailto:[email protected]> > > <mailto:[email protected] > <mailto:[email protected]> > > ] > > > > Sent: Thursday, July 13, 2017 10:39 AM > > > > To: Developer support list for Wireshark <wireshark- > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > <mailto:wireshark- > <mailto:wireshark-> > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > > > > > > > > > > Cc: Sultan, Hassan <[email protected] > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > > > > > Subject: Re: [Wireshark-dev] hf_http_response_code in > packet- > > http.c > > > > > > > > > > > Hi Hassan, > > > > > > > > > > > > 2017-07-13 19:25 GMT+02:00 Sultan, Hassan via > Wireshark- > dev > > > <wireshark- > > > > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:wireshark- <mailto:wireshark-> > > <mailto:wireshark- <mailto:wireshark-> > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > <mailto:wireshark- > <mailto:wireshark-> > > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > > >: > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Wireshark-dev [mailto:wireshark-dev- > <mailto:wireshark-dev-> > > > <mailto:wireshark-dev- <mailto:wireshark-dev-> > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:wireshark- > [email protected]> <mailto:wireshark-dev- <mailto:wireshark-dev-> > > [email protected] <mailto:[email protected]> > > > > > > <mailto:[email protected] > <mailto:[email protected]> > > <mailto:[email protected] <mailto:wireshark- > [email protected]> > <mailto:wireshark- <mailto:wireshark-> > > > <mailto:wireshark- <mailto:wireshark-> > > > > [email protected] <mailto:dev- > [email protected]> <mailto:[email protected] <mailto:dev- > [email protected]> > > > > > ] On Behalf > > > > > Of Erik de Jong > > > > > Sent: Wednesday, July 12, 2017 10:12 PM > > > > > To: Developer support list for Wireshark > <wireshark- > > > > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:wireshark- <mailto:wireshark-> > > <mailto:wireshark- <mailto:wireshark-> > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > <mailto:wireshark- > <mailto:wireshark-> > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > > > > > > > > Subject: Re: [Wireshark-dev] > hf_http_response_code in > > packet- > > > http.c > > > > > > > > > > > > > > > > > > > > On Thu, Jul 13, 2017 at 1:12 AM, Sultan, > Hassan via > > Wireshark- > > > dev > > > > <wireshark- > > > > > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > > > <mailto:wireshark- <mailto:wireshark-> <mailto:wireshark- > <mailto:wireshark-> > > > > <mailto:wireshark- <mailto:wireshark-> <mailto:wireshark- > <mailto:wireshark-> > > > > > > > > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > > > <mailto:wireshark- <mailto:wireshark-> > > <mailto:wireshark- <mailto:wireshark-> > > > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]> > <mailto:wireshark- > <mailto:wireshark-> > > [email protected] <mailto:[email protected]> <mailto:wireshark- > [email protected] <mailto:[email protected]> > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > I am starting to learn the Wireshark > code base, and > one > > thing > > > > puzzles > > > > > me… > > > > > > > > > > > > > > > > > > > > Why is hf_http_response_code defined as > a > FT_UINT16 > > with > > > > BASE_DEC > > > > > rather than an FT_STRING ? > > > > > > > > > > > > > > > > > > > > It’s a text field… not an integer. > > > > > > > > > > > > > > > Presenting it as a number allows for > filtering like: > > > > > http.response.code > 200 > > > > > > > > > > Which would not be possible when presented as > a > string. > > > > > > > > Thanks for the info, but in that case would it > not be more > > > appropriate to > > > > have the normal field as an FT_STRING and add a > generated > field > > as > > > FT_UINT16 > > > > ? My understanding of generated fields is that this > is their > > purpose : > > > represent > > > > data that doesn't exactly correspond to the packet > data. > > > > We could still keep the field named as is today > (hence > > ensuring all > > > > existing filters still work), but simply make it a > generated > field, > > and add > > > an > > > > FT_STRING to represent the actual data as it is in > the packet. > > > > > > > > Thoughts ? > > > > > > > > > > > > > > > > > > > > Does having the field as it is designed today > generating you > any > > issue? > > > For fields > > > > having numerical values only, it makes much more > sense to > have > > the > > > digits > > > > directly instead of the string and that's already > what we are > doing > > in > > > numerous > > > > places. And personally I do not see any valid reason > to > change > > that. > > > Your > > > > suggestion implies that many fields would start to be > duplicated > > > without any > > > > added value. > > > > > > > > > My reason is that anyone wanting to build automation > based > on > > > Wireshark's parsing will not be able to rely on Wireshark's > reporting of > > field type > > > & format to interpret the packet data. > > > The automation would walk through parsed fields and be > told > that > > the > > > http response code is a 3-bytes UINT16 with value say 200, so > bytes 00 > > 00 c8, > > > which is wrong. Right now the way these fields are setup works > for > > human > > > interpretation, by people looking at the rendered value on a > screen, > > but it is far > > > from ideal for automation. > > > > > > > > > Thanks fact that HTTP is text based does not change the fact > that > the > > code is > > > supposed to be a number. So I'm not sure to get your point. > > > Moreover tons of people use tshark for automation and I do not > > remember > > > someone complaining about the current output. So maybe I > simply did > > not > > > understand your use case. > > > > > > > > > One could argue it's the main purpose of Wireshark, but > I > think it > > > needlessly prevent a whole class of use-cases from being > achieved > > with > > > Wireshark's technology. > > > > > > > > > > > > Could you elaborate? What use case do you have in mind > exactly? > > > > > > Imagine automation that attempts to perform modification on > packets > > using the parsed information provided by Wireshark, or somehow > goes and > > directly reads fields in the packet data based on what Wireshark's > parsed > > information says (type/offset/length/...). Wireshark reporting the > field > as > > FT_UINT16 instead of say a UTF-8 FT_STRING throws off the > automation as it > > would either attempt to write an FT_UINT16 at the offset of the field, > writing > > the wrong format, or be confused by seeing a 3 bytes FT_UINT16 and > give up. > > > > > > > > > > But in that case it's more like you're doing a part of the parsing in > another > > application, why not implement a full parser in that application? The > whole point > > of an analyzer is to make sense of the data passed into it for the > user. > And what > > about protocols that could be send with different encodings (BER, > XER, etc) > > there a field type should still be the same - eg UINT16 - but the > representation > > on the wire is totally different. > > > The beauty of this is specifically that the other app does not need to > do > any parsing at all. It fully relies on Wireshark's parsing engine to get the > tree of > fields, the offset/length/type/name/encoding/... of each field and instead of > having to write custom code for each protocol (you know, a parser), the app > simply has to implement generic handling for FT_STRING, FT_UINT16, etc... and > the various encodings. So for your BER encoded integer, having an FT_UINT16 > with an ENC_BER encoding flag for example and a size of 4 (type+length+2 bytes > for most short ints) would convey all the info needed to handle the field. > > > > That would require having an encoder able to understand Wireshark output > syntax and the structure of the protocol you are using to perform the encoding > (text based, TLV, CSN1, ASN1, etc): all the encoding rules related to the > protocol > you are playing with. Having a decoder / parser is far from giving you an > encoder.
Absolutely, but this is a lot, lot, less work than writing parsers for all the protocols. That's where Wireshark shines, it provides 70-80% of the work (all the parsing), and only the handling of encoding / shifting fields (in case user decides to write a field to be larger/smaller than originally in the packet) needs to be done. > As Eric explained, having the HTTP response code as a number simply gives you > much more advantages (filtering, comparisons, inequalities...) than having it > in > text. So this is clearly not a bug as it was done on purpose. I agree it's not a bug. For the existing purpose of Wireshark it does the job. It just makes some other use-cases a bit more difficult. > Will such a thing work for absolutely every single protocol out there ? > Maybe not, but it works for the vast majority of them. > > > I have some doubts regarding this statement, for the reason I gave above. > But as I'm eager to learn, if you can come up with a PoC I would be very > interested to look at it :) It might be possible to show something but it will take a bit of time and is a maybe. I'm still in the learning stages of how Wireshark works and there's quite a bit of work that will be needed before I can confirm if it's doable with Wireshark or what changes are needed in Wireshark to make it work (the example I just gave being one). > I did a very similar thing on top of Network Monitor in a previous life > at > Microsoft (I had the code available being internal to the company) and it was > a > huge success, helping many people in various ways. > > > > So what kind of encoding did it support? Gzip compression, asn.1 BER & DER, utf-16/8/ucs2/ascii, little/big endian, base64, custom encodings for netbios strings, and more that I forget, it also had inference for finer grained datatypes (figure out that this string field is actually a url, domain name, numerical text, email, xml, json, ... instead of text and it did so in a recursive manner so that when finding xml for example, attributes in xml could be inferred as url, numerical text, domain name, etc...). It could also infer which fields were length/count fields for other fields (so it could update them automatically upon changes) or, in certain circumstances, checksum fields. Whenever users on a protocol encountered an encoding we didn't support, we'd add the encoder for it. Much simpler and quicker task than adding a protocol parser. > > > > How about having an #ifdef for such things so that Wireshark's > use-case > > today does not have to suffer the addition of generated fields, and > such > > automation cases could still get full fidelity parsing by building > Wireshark's > > engine with the specific #define ? Today they have to build directly > against > > Wireshark's code anyway so that shouldn't be a problem. > > > > > > > __________________________________________________________ > > _________________ > > Sent via: Wireshark-dev mailing list <wireshark- > [email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:wireshark- > [email protected]> > > > > Archives: https://www.wireshark.org/lists/wireshark-dev > <https://www.wireshark.org/lists/wireshark-dev> > > <https://www.wireshark.org/lists/wireshark-dev > <https://www.wireshark.org/lists/wireshark-dev> > > > Unsubscribe: > https://www.wireshark.org/mailman/options/wireshark- > <https://www.wireshark.org/mailman/options/wireshark-> > > dev <https://www.wireshark.org/mailman/options/wireshark-dev > <https://www.wireshark.org/mailman/options/wireshark-dev> > > > mailto:[email protected] > <mailto:[email protected]> <mailto:wireshark- > <mailto:wireshark-> > > [email protected] <mailto:[email protected]> > > ?subject=unsubscribe > > > > > __________________________________________________________ > _________________ > Sent via: Wireshark-dev mailing list <[email protected] > <mailto:[email protected]> > > Archives: https://www.wireshark.org/lists/wireshark-dev > <https://www.wireshark.org/lists/wireshark-dev> > Unsubscribe: https://www.wireshark.org/mailman/options/wireshark- > dev <https://www.wireshark.org/mailman/options/wireshark-dev> > mailto:[email protected] > <mailto:wireshark- > [email protected]> ?subject=unsubscribe > ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <[email protected]> Archives: https://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev mailto:[email protected]?subject=unsubscribe
