Hi Sean. 2012/6/30 Sean Bartell <[email protected]>: > Hi, Vojtech, > > On Thu, Jun 28, 2012 at 06:24:44PM +0200, Vojtech Horky wrote: >> 2012/6/28 Sean Bartell <[email protected]>: >> > Now that the basics of Bithenge are in place, there are several things I >> > could start working on. I'm inclined to keep working on the language, >> > specifically by adding support for parameters and expressions, because >> Can you, please, elaborate more what you mean by parameters? As a >> matter of fact, there are some features described on the wiki (Future >> features section) that I am not sure I understand correctly. Would >> you, please, describe them in more detail? Or, please, give examples >> of file-formats/file-systems/whatever where this functionality would >> be useful. > > I've rewritten that wiki section to explain the ideas. Parameters are a > way of letting a transform depend on more than just their input data. > For instance, a Pascal string could look like this: > > .len <- uint8le; > .str <- ascii <- known_length(.len); > > Where .len is given as a parameter to known_length. At first, parameters Thanks for the example. Now it makes perfect sense.
> will be limited to literals (8) or references to previous fields (.len), > but more complicated expressions could be added. Users could also add My recommendation is to leave the complex expressions alone and implement them only if time allows. Usually, the length is directly stored inside another field anyway. I expect that once you implement the bitmap parsing, it would not be a problem if this length would be actually part of a bitfield. >> Skip some data. Either of fixed length (e.g. to skip unknown USB >> descriptors) or until some pattern is found (e.g. parts of GIF >> header). > > These could be done with a combination of parameters (for known_length > or similar), the "hidden fields" idea, and the "search" idea. Nice :-). The "hidden field" is a nice feature but definitely not a "must have" one. >> Accessing the parsed data at run-time. Currently, you are able to >> print the data but I found no way to actually work with them. For >> example, how would I read the latitude from your example once the >> structure is parsed? This is a very important feature if we want to >> use Bithenge as a basis for other tools, such as debugger. Optimal >> solution would be to parse the data into a native C struct. Question >> is how to populate it. Generate a setter of some kind from the >> grammar, use some weird macros, ... > > The primary way of accessing the data is to use the functions in tree.h > to access the output tree. As you mention, I will need to add a method > to get a specific child of an internal node. This way is more flexible > because the script can be chosen at runtime, so it would be used for a > debugger or interactive browser. Ok. > > Another way would be to parse the script file and create C source code > that declares structs and reads data. This would be an alternative to > writing the C code directly when you need to parse binary data in > HelenOS. I didn't include this in my GSoC plan because it wouldn't help > with the debugger or interactive browser. I still think that it would be a nice feature to translate the parsed data into a C struct with which you can work "natively". And I think that for simple structures with fixed format (i.e. no repetitions, conditionals..) it should be rather simply to implement it. >> What is the "Try to read 5 bytes and fail if the blob is too long." >> comment? Could that happen if the length is explicitly given by the >> other function? > > uint32le_apply needs to check whether the input node is a 4-byte blob > node. It does this by trying to read 5 bytes from the blob node. If it > succeeds and gets 5 bytes, it knows the blob node is too long for a > uint32le, so it returns EINVAL. If it gets 4 bytes, it knows the blob > node is exactly 4 bytes long. > > prefix_length_4 says that uint32les only take 4-byte inputs, but it's > only a helper. If your script is: > > transform main = uint32le; > > prefix_length_4 is never called; uint32le_apply is called directly on > the input, and it's responsible for checking whether the input has the > correct length. Thanks for the explanation. >> What are the plans for "write" support? > >> Your progress looks okay to me, but do not forget that you are already >> in the middle of GSOC (assuming you won't work during your vacation). >> If you think that you will not be able to implement everything, it is >> time to discuss what features could be dropped. Otherwise, good job! > > I'll try to work a little, but this is still a good point. In my > proposal, I planned to make these things by the GSoC deadline: > > - An interactive browser (7 days) > > - A simple DWARF implementation to browse other tasks' memory (9 days) > > - Editing support, including interactive editing (11 days) > > I think I could do these, but with very limited functionality. For > instance, the editing support might only work with simple integers and > booleans. I propose not working on editing yet, and maybe not DWARF > either; instead, I could improve the language and spend more time > working on the interactive browser. Of course, the plan could be changed > later if necessary. What is your preference? My preference depends which functionality you plan to implement into these "interactive" tools. From my point of view, I do not care about eye-candy stuff and I would be happy with command-line tools. For me, important functionality of the browser is search (e.g. file system node where .filename is hello.c) and scalability (i.e. reasonable performance when iterating over huge inputs). For the editing support, I would say that having C API is a minimum. It should allow following actions: * edit primitive values * add/delete from repetitions * add/delete optional structs (aka conditional transforms) Obviously, it must be possible to write the changes back and the changes must be checked to be valid according to the grammar. If you would built some command-line editor above this it would be nice. Regarding the DWARF support - I do not know the format well and I do not know whether it is possible to implement core functionality only first and extend later. What I would like to see in the end is a Bithenge script that describes some relatively complex and real-world data structure. It ought to prove that Bithenge works and is able to work with realistic formats. In your timeline, you mentioned FAT format but it can be something else as well. Ideally something that would demonstrate all capabilities of your implementation. If that is not possible with single format, using several simpler ones is possible as well, of course. For example, TCP frames, call stacks or USB transaction frames. Another useful example is exploring another task's memory - e.g. crawling through some linked list when pointed to its head. I expect that others would voice their opinions as well - there are more ways how to view this project and we both may have missed something obvious and important. I think you can now concentrate on the repetitions, bitfields etc. and meanwhile, the rest (browser, editor, DWARF) could be discussed in more detail. Enjoy your vacation. - Vojta > > Thanks, > Sean > > _______________________________________________ > HelenOS-devel mailing list > [email protected] > http://lists.modry.cz/cgi-bin/listinfo/helenos-devel _______________________________________________ HelenOS-devel mailing list [email protected] http://lists.modry.cz/cgi-bin/listinfo/helenos-devel
