Re: [Wireshark-dev] RFD: New language to write dissectors
On Jul 24, 2012, at 1:23 PM, Jakub Zawadzki wrote: From my perspective: I really use ~ 20 dissectors, why I need all of them? ...and if they're loaded at startup time and either interpreted or translated after being loaded, it might be easier to choose a subset of protocols (NetMon lets you do that, although I don't remember whether you can specify an arbitrary subset or if they just let you choose from subsets they've defined). ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
Hmm, I did briefly think that if we ever moved all dissectors into plug-in form (which would be unlikely, given the drive to make as many built-in as possible), we could package them according to protocol family/purpose, so that users could select only the ones that were necessary - but many have complex interdependencies; and there's always going to be a number of people who'll complain if they can't find an obscure dissector in their installation. That said, one of the appeals of Wireshark to me is that it only does everything (to steal Sony's slogan); and that the current licencing/project maintenance arrangements encourage people to contribute new dissectors/features (even if not everything gets accepted 100% of the time), and participate in their maintenance - so that everyone receives a better product in the long term. (I'm not opposed to the implementation of new mechanisms that potentially make it easier to enhance, and extend Wireshark, though). Just my 0.02p, Tyson. 2012/7/25 Guy Harris g...@alum.mit.edu On Jul 24, 2012, at 1:23 PM, Jakub Zawadzki wrote: From my perspective: I really use ~ 20 dissectors, why I need all of them? ...and if they're loaded at startup time and either interpreted or translated after being loaded, it might be easier to choose a subset of protocols (NetMon lets you do that, although I don't remember whether you can specify an arbitrary subset or if they just let you choose from subsets they've defined). ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org ?subject=unsubscribe -- Fight Internet Censorship! http://www.eff.org http://vmlemon.wordpress.com | Twitter/FriendFeed/Skype: vmlemon | 00447934365844 ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
Jakub Zawadzki wrote: From my perspective: I really use ~ 20 dissectors, why I need all of them? Admittedly 99% of the time I normally only use 10-15 dissectors, but OTOH every once in a while I'll start Wireshark to figure out why my insert service/application here is slow, not working, whatever and it's nice that my Wireshark automatically decodes service/application's traffic. Anyway, what does it matter to me if there are 20 dissectors or 1000? Wireshark's startup time is still fairly fast[1]. Could it be faster? Sure, why not. Do I care? Not as much as making dissecting and filtering faster. [1] On my *old* desktop it takes ~1.5 seconds between when I start the GUI though (manually) hitting Ctrl-Q and when the process exits. (Admittedly I probably ran it recently so most of the pages are in memory already.) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On 07/14/2012 06:31 PM, Guy Harris wrote: On Jul 14, 2012, at 8:26 AM, Jakub Zawadzki wrote: It'd be great if we have some abstract and pure (no C/assembly inline) language to write dissectors. [...] On the other hand, it might also allow protocol descriptions to be shipped either in source form or binary form with restrictions on redistribution, providing a way to get around the GPL for protocols. Some might consider that a feature (I seem to remember many years ago Cisco raised this issue about some protocols) and others might consider it a bug. If we end up with a consensus of it's a bug, we might be able to extend the protections of the GPL to dissector descriptions fed to the interpreter, so that if you make a compiled protocol description available, you must also make the source available to recipients and must give recipients the right to redistribute the source or binaries. I'd tend to think that would be a bug. This reminds me of the struggles I've had recently getting people to contribute their diameter.xml changes to Wireshark. They're able to just add the stuff they need and they don't change versions (or computers) often enough to care so they haven't felt the need to get the changes in. Having an editable XML file is great for local modifications but it does lower the urge to get the changes checked into the Wireshark trunk. Lowering the barrier to adding dissectors would likely have a similar effect. And I think part of what has made Wireshark great (from a user's perspective) is that it supports so many protocols out of the box; it supports so many protocols because it's easy to write them and get them included in Wireshark's repository. (I'm not saying this is a reason to not support such a feature, I'm just putting the thought out there.) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Tue, Jul 24, 2012 at 09:40:36AM -0400, Jeff Morriss wrote: On 07/14/2012 06:31 PM, Guy Harris wrote: On Jul 14, 2012, at 8:26 AM, Jakub Zawadzki wrote: It'd be great if we have some abstract and pure (no C/assembly inline) language to write dissectors. [...] On the other hand, it might also allow protocol descriptions to be shipped either in source form or binary form with restrictions on redistribution, providing a way to get around the GPL for protocols. Some might consider that a feature (I seem to remember many years ago Cisco raised this issue about some protocols) and others might consider it a bug. If we end up with a consensus of it's a bug, we might be able to extend the protections of the GPL to dissector descriptions fed to the interpreter, so that if you make a compiled protocol description available, you must also make the source available to recipients and must give recipients the right to redistribute the source or binaries. I'd tend to think that would be a bug. This reminds me of the struggles I've had recently getting people to contribute their diameter.xml changes to Wireshark. They're able to just add the stuff they need and they don't change versions (or computers) often enough to care so they haven't felt the need to get the changes in. Having an editable XML file is great for local modifications but it does lower the urge to get the changes checked into the Wireshark trunk. Lowering the barrier to adding dissectors would likely have a similar effect. The same situation is possible now with plugins. I think the main reason why user don't want to contribute, is because it's complicated. User need to create account, verify his mail, attach dissector sources, fix code, answer comments, fix code (again), etc, etc... But if we have some language, which is easy to write dissector, easy to check (does it parse?), and easy to maintaince. We can have 'upload your dissector' on main page. User need to upload just two files - sources and sample capture file. And I think part of what has made Wireshark great (from a user's perspective) is that it supports so many protocols out of the box; it supports so many protocols because it's easy to write them and get them included in Wireshark's repository. From my perspective: I really use ~ 20 dissectors, why I need all of them? Cheers, Jakub. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sun, Jul 15, 2012 at 08:10:42PM -0700, Richard Sharpe wrote: On Sun, Jul 15, 2012 at 2:20 AM, Jakub Zawadzki darkjames...@darkjames.pl wrote: Well, it might look stupid to you, but it represents a quick way to get simplify the process of producing dissectors, and writing a dissector is very boring. Forgot word, once again: *Manually* translating 4K lines of NPL code to C looks stupid to me (...) In addition, to produce Wireshark C-based dissectors from NPL would seem not to require LLVM. A simple recursive descent parser should suffice. LLVM is not parser generator (in wireshark we use lemon for it), and if we talk about boredom, writting recursive descent parser is boring ;-) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sun, Jul 15, 2012 at 10:53 PM, Jakub Zawadzki darkjames...@darkjames.pl wrote: On Sun, Jul 15, 2012 at 08:10:42PM -0700, Richard Sharpe wrote: On Sun, Jul 15, 2012 at 2:20 AM, Jakub Zawadzki darkjames...@darkjames.pl wrote: Well, it might look stupid to you, but it represents a quick way to get simplify the process of producing dissectors, and writing a dissector is very boring. Forgot word, once again: *Manually* translating 4K lines of NPL code to C looks stupid to me (...) In addition, to produce Wireshark C-based dissectors from NPL would seem not to require LLVM. A simple recursive descent parser should suffice. LLVM is not parser generator (in wireshark we use lemon for it), and if we talk about boredom, writting recursive descent parser is boring ;-) I should have said more :-) We can do the parsing any way we want. However, it seems that we do not need LLVM if we are just producing dissectors as C code. We would only need it if we want to compile all the way into some sort of byte code for dissectors. However, AFAIK, there is no such dissector VM today. There is a VM for filtering, but not for dissection. -- Regards, Richard Sharpe (何以解憂？唯有杜康。--曹操) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Jul 16, 2012, at 7:18 AM, Richard Sharpe wrote: We can do the parsing any way we want. However, it seems that we do not need LLVM if we are just producing dissectors as C code. We would only need it if we want to compile all the way into some sort of byte code for dissectors. $ gcc -bash: gcc: command not found $ clang -bash: clang: command not found $ cc -bash: cc: command not found ... Obviously not my machine, which *does* have gcc and clang installed, and in which cc is a link to gcc, but if somebody's not interested in writing C code but *is* interested in adding a new protocol to Wireshark, and doesn't have a C compiler installed, either translating to an interpreted bytecode or to, for example, LLVM language: http://llvm.org/docs/LangRef.html and JITting it into your machine code, could be useful. However, AFAIK, there is no such dissector VM today. There is a VM for filtering, but not for dissection. There isn't, but perhaps there should be. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sat, Jul 14, 2012 at 03:31:06PM -0700, Guy Harris wrote:
On Jul 14, 2012, at 8:26 AM, Jakub Zawadzki wrote:
It'd be great if we have some abstract and pure (no C/assembly inline)
language to write dissectors.
Or to describe protocols and the way packets for those protocols are
displayed - the languages in question wouldn't be as procedural as
C/Lua/etc, they'd be more descriptive.
We could invent yet another protocol desciption language,
...but, as you suggest, we probably shouldn't.
http://imgs.xkcd.com/comics/standards.png ;-)
I'm not sure it has to be a choice, though - we could implement both,
resources permitting, of course. (And, of course, given that there are many
already-existing languages that describe protocols - ASN.1, {OSF
IDL/MIDL/PIDL} for DCE RPC, rpcgen for ONC RPC, CORBA IDL, xcb for X11 - we
will probably never have the One True Protocol Description Language.)
I'd rather support one, and later have some compiler from language A to B.
I'm bigger fan of NPL (...) but there might exists some legal (patents for
grammar/implementation?!) issues.
That would be one concern - even having our own language, such as wsgd,
runs the risk of infringing a patent, but, well, *writing software of just
about any sort* runs the risk of infringing a patent;
however, we're dealing with a large corporation in the case of NPL, so
there's probably a greater risk that some or all of it is covered by patents.
Were Microsoft to explicitly state that there are no patents on
NPL-the-language or that they're granting a royalty-free license for all
implementations (perhaps with a mutual assured destruction clause,
so that were we to patent some feature of Wireshark and sue Microsoft for
violating that patent, our license for their patents would terminate), and
the same applied to any patents they hold on their
implementation of NPL that would block independent useful implementations,
that might help.
I'm not sure if it was covered by recent Oracle vs Google lawsuits.
Maybe Riverbed could help us with it? Gerald?
With wsgd we could reuse some existing code of plugin.
...and we also have more freedom to extend the language, e.g. to support
preferences for a protocol
or support for columns/expert info.
Were there an Open NPL Consortium of some sort where multiple implementers
of NPL could propose extensions, and perhaps a way an implementation could
offer private extensions without worrying about colliding with other
implementations or future standards, that might help.
It'd also clear legal status of NPL language.
(It also raises the question of whether interpreted execution of that
machine code or translation to C or machine language will be faster -
interpreted execution *could* result in a smaller cache footprint if the
interpreter is small enough and the code high-level enough to be fairly
dense, although it does involve difficult-at-best-to-predict branches in the
interpretive loop.)
I was thinking to use LLVM. For built-in dissectors we could compile dissectors
to object file/ assembly, for user dissectors we'll benefit from LLVM JIT.
But anyway we need compiler to C. For prototype (does it work?) and later as
fallback for people who don't have LLVM.
... Or can LLVM libraries be strong dependency?
___
Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org
Archives:http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
Hmm,
What about implementing a compiler that generates C dissector source code,
from NPLt m, or WSGD dissector code? Or would that be overkill for what
we're trying to do?
Just my 0.02p...
Tyson.
2012/7/15 Jakub Zawadzki darkjames...@darkjames.pl
On Sat, Jul 14, 2012 at 03:31:06PM -0700, Guy Harris wrote:
On Jul 14, 2012, at 8:26 AM, Jakub Zawadzki wrote:
It'd be great if we have some abstract and pure (no C/assembly inline)
language to write dissectors.
Or to describe protocols and the way packets for those protocols are
displayed - the languages in question wouldn't be as procedural as
C/Lua/etc, they'd be more descriptive.
We could invent yet another protocol desciption language,
...but, as you suggest, we probably shouldn't.
http://imgs.xkcd.com/comics/standards.png ;-)
I'm not sure it has to be a choice, though - we could implement both,
resources permitting, of course. (And, of course, given that there are
many already-existing languages that describe protocols - ASN.1, {OSF
IDL/MIDL/PIDL} for DCE RPC, rpcgen for ONC RPC, CORBA IDL, xcb for X11 - we
will probably never have the One True Protocol Description Language.)
I'd rather support one, and later have some compiler from language A to B.
I'm bigger fan of NPL (...) but there might exists some legal (patents
for grammar/implementation?!) issues.
That would be one concern - even having our own language, such as
wsgd, runs the risk of infringing a patent, but, well, *writing software of
just about any sort* runs the risk of infringing a patent;
however, we're dealing with a large corporation in the case of NPL, so
there's probably a greater risk that some or all of it is covered by
patents.
Were Microsoft to explicitly state that there are no patents on
NPL-the-language or that they're granting a royalty-free license for all
implementations (perhaps with a mutual assured destruction clause,
so that were we to patent some feature of Wireshark and sue Microsoft
for violating that patent, our license for their patents would terminate),
and the same applied to any patents they hold on their
implementation of NPL that would block independent useful
implementations, that might help.
I'm not sure if it was covered by recent Oracle vs Google lawsuits.
Maybe Riverbed could help us with it? Gerald?
With wsgd we could reuse some existing code of plugin.
...and we also have more freedom to extend the language, e.g. to support
preferences for a protocol
or support for columns/expert info.
Were there an Open NPL Consortium of some sort where multiple
implementers of NPL could propose extensions, and perhaps a way an
implementation could offer private extensions without worrying about
colliding with other implementations or future standards, that might help.
It'd also clear legal status of NPL language.
(It also raises the question of whether interpreted execution of that
machine code or translation to C or machine language will be faster -
interpreted execution *could* result in a smaller cache footprint if the
interpreter is small enough and the code high-level enough to be fairly
dense, although it does involve difficult-at-best-to-predict branches in
the interpretive loop.)
I was thinking to use LLVM. For built-in dissectors we could compile
dissectors to object file/ assembly, for user dissectors we'll benefit from
LLVM JIT.
But anyway we need compiler to C. For prototype (does it work?) and later
as fallback for people who don't have LLVM.
... Or can LLVM libraries be strong dependency?
___
Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org
Archives:http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org
?subject=unsubscribe
--
Fight Internet Censorship!
http://www.eff.org
http://vmlemon.wordpress.com | Twitter/FriendFeed/Skype: vmlemon |
00447934365844
___
Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org
Archives:http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sun, Jul 15, 2012 at 10:28:23AM +0100, Tyson Key wrote: What about implementing a compiler that generates C dissector source code, from NPLt m, or WSGD dissector code? Yes, I plan to have one: But anyway we need compiler to C. For prototype (does it work?) and later as fallback for people who don't have LLVM. Or would that be overkill for what we're trying to do? It depends what we're trying to do ;-) For me compiler to C is must-have, and even if we write only it. It'll help to fix some bugs, like yours bug #6520. Translating 4K lines of NPL code to C looks stupid to me, and it'll be great to have automatic translator. (of course if we base grammar on NPL). Later we could support interpreting (or JIT) NPL, which would make easier (and much faster) developing new dissectors. In next step we could generate dependency graph (field foo.B exists only when field foo.A = 5, etc...) and field to offset (field foo.A @ 10B, field foo.B @ 20B) or other informations which would speedup filtering. (Or generate completely new code like Guy suggested). and here we can start rewriting existsing old dissectors to new language. Anyway, Lot of possible work, which some might be seen as overkill ;-) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Jul 15, 2012, at 1:24 AM, Jakub Zawadzki wrote:
On Sat, Jul 14, 2012 at 03:31:06PM -0700, Guy Harris wrote:
I'm not sure it has to be a choice, though - we could implement both,
resources permitting, of course. (And, of course, given that there are many
already-existing languages that describe protocols - ASN.1, {OSF
IDL/MIDL/PIDL} for DCE RPC, rpcgen for ONC RPC, CORBA IDL, xcb for X11 - we
will probably never have the One True Protocol Description Language.)
I'd rather support one, and later have some compiler from language A to B.
Presumably by one you're referring to the choice between NPL and wsgd, given
that we already *have* translators to take ASN.1, PIDL, CORBA IDL, and xcb and
turn them into C dissectors.
I was thinking to use LLVM. For built-in dissectors we could compile
dissectors to object file/ assembly, for user dissectors we'll benefit from
LLVM JIT.
But anyway we need compiler to C. For prototype (does it work?) and later as
fallback for people who don't have LLVM.
...or who are using processors not supported as LLVM targets:
http://www.llvm.org/docs/CodeGenerator.html
... Or can LLVM libraries be strong dependency?
Not unless we drop support for OSes and instruction sets not currently
supported by LLVM.
___
Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org
Archives:http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sun, Jul 15, 2012 at 2:20 AM, Jakub Zawadzki darkjames...@darkjames.pl wrote: On Sun, Jul 15, 2012 at 10:28:23AM +0100, Tyson Key wrote: What about implementing a compiler that generates C dissector source code, from NPLt m, or WSGD dissector code? Yes, I plan to have one: But anyway we need compiler to C. For prototype (does it work?) and later as fallback for people who don't have LLVM. Or would that be overkill for what we're trying to do? It depends what we're trying to do ;-) For me compiler to C is must-have, and even if we write only it. It'll help to fix some bugs, like yours bug #6520. Translating 4K lines of NPL code to C looks stupid to me, and it'll be great to have automatic translator. (of course if we base grammar on NPL). Well, it might look stupid to you, but it represents a quick way to get simplify the process of producing dissectors, and writing a dissector is very boring. When I wrote the initial SMB dissector I wrote a language to describe the SMB protocol, and wrote a parser in Perl to parse the description into an AST and spit out a dissector. The same could be done with NPL, in my view, but it misses something important. Compiling an NPL-described protocol into a Wireshark dissector loses a lot of information. On the other hand, having an NPL compiler would make it easier for others to create new dissectors. Of course, it would be useful if we had access to ParserLanguage.Doc. In addition, to produce Wireshark C-based dissectors from NPL would seem not to require LLVM. A simple recursive descent parser should suffice. Later we could support interpreting (or JIT) NPL, which would make easier (and much faster) developing new dissectors. In next step we could generate dependency graph (field foo.B exists only when field foo.A = 5, etc...) and field to offset (field foo.A @ 10B, field foo.B @ 20B) or other informations which would speedup filtering. (Or generate completely new code like Guy suggested). and here we can start rewriting existsing old dissectors to new language. Anyway, Lot of possible work, which some might be seen as overkill ;-) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe -- Regards, Richard Sharpe (何以解憂？唯有杜康。--曹操) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Sat, Jul 14, 2012 at 8:26 AM, Jakub Zawadzki darkjames...@darkjames.pl wrote: Hi, It'd be great if we have some abstract and pure (no C/assembly inline) language to write dissectors. We could invent yet another protocol desciption language, but I was thinking to base grammar on netmon NPL [1] or wsgd [2]. I'm bigger fan of NPL (sorry Olivier), nmparsers project has got large collection of dissectors[3] which we could use (LLTD - bug #6071, Windows USB Port packets - bug #6520, netsh - bug #6694) but there might exists some legal (patents for grammar/implementation?!) issues. It is amusing that the document Writing a Parser from Wire to Window contains this: You can grab some HSRP sample captures from here: http://wiki.wireshark.org/SampleCaptures. If you load one of those files into Network Monitor (making sure to reload your parsers first), you should now see packets labeled as HSRP with the Description “HSRP”. The document can be found here: http://nmparsers.codeplex.com/downloads/get/53465 -- Regards, Richard Sharpe (何以解憂？唯有杜康。--曹操) ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] RFD: New language to write dissectors
On Jul 14, 2012, at 8:26 AM, Jakub Zawadzki wrote:
It'd be great if we have some abstract and pure (no C/assembly inline)
language to write dissectors.
Or to describe protocols and the way packets for those protocols are
displayed - the languages in question wouldn't be as procedural as C/Lua/etc,
they'd be more descriptive.
We could invent yet another protocol desciption language,
...but, as you suggest, we probably shouldn't.
but I was thinking to base grammar on netmon NPL [1] or wsgd [2].
Those are probably the two best choices.
I'm not sure it has to be a choice, though - we could implement both, resources
permitting, of course. (And, of course, given that there are many
already-existing languages that describe protocols - ASN.1, {OSF IDL/MIDL/PIDL}
for DCE RPC, rpcgen for ONC RPC, CORBA IDL, xcb for X11 - we will probably
never have the One True Protocol Description Language.)
I'm bigger fan of NPL (sorry Olivier), nmparsers project has got large
collection of dissectors[3]
which we could use (LLTD - bug #6071, Windows USB Port packets - bug #6520,
netsh - bug #6694)
but there might exists some legal (patents for grammar/implementation?!)
issues.
That would be one concern - even having our own language, such as wsgd, runs
the risk of infringing a patent, but, well, *writing software of just about any
sort* runs the risk of infringing a patent; however, we're dealing with a large
corporation in the case of NPL, so there's probably a greater risk that some or
all of it is covered by patents. Were Microsoft to explicitly state that there
are no patents on NPL-the-language or that they're granting a royalty-free
license for all implementations (perhaps with a mutual assured destruction
clause, so that were we to patent some feature of Wireshark and sue Microsoft
for violating that patent, our license for their patents would terminate), and
the same applied to any patents they hold on their implementation of NPL that
would block independent useful implementations, that might help.
With wsgd we could reuse some existing code of plugin.
...and we also have more freedom to extend the language, e.g. to support
preferences for a protocol - Paul Long's blog post says
A common problem: “No silly, we do HTTP traffic on port , not 80 or 8080!”
While changing port mappings for protocols could be something revealed in the
user interface, we haven’t gotten that far in Network Monitor 3.0 yet. I
expect we should address this specific problem on different fronts, i.e. a UI
for each protocol, and some way to handle dynamic port allocations. And
there are also some heuristics we can use to identify protocols as well. But
today, there is a fairly simple way to modify the NPL script for protocols on
non-standard ports.
I don't know whether, as of 3.4, they support a UI for each protocol, and some
way to handle dynamic port allocations, but we already have the infrastructure
for that.
NPL also, for strings, offers 3 encodings - to quote the help manual:
This data type extracts a specified number of characters from a sequence of
bytes. The characters can be UTF-16, UTF-8, or ASCII, depending on the
encoding specified.
There's no mention of the Extended Binary-Coded Decimal Interchange Code there,
but we have several dissectors using ENC_EBCDIC, so that would be another place
where we might want to extend NPL were we to use it.
Were there an Open NPL Consortium of some sort where multiple implementers of
NPL could propose extensions, and perhaps a way an implementation could offer
private extensions without worrying about colliding with other implementations
or future standards, that might help.
Note, by the way, that having a language of this sort could allow something
such as this.
Consider a protocol with the following description (in a C-like protocol
description language that I'm making up on the fly):
enum message_type {
Login = 0,
Logout = 1,
Request = 2,
Response = 3
};
struct login {
ascii string username[16];
ascii string password[16];
};
struct request {
uint32 bigendian requested_item;
};
struct response {
uint32 bigendian value_size;
uint8 value[value_size];
};
struct request {
protocol foo {
uint32 bigendian enum message_type type;
switch (type) {
case Login:
struct login login;
case Logout:
/* logout message has only a type */
case Request:
struct request request;
case Response:
struct response response;
}
uint32 bigendian message_id;
};
which might translate to
