*First off a disclaimer. This library is very new and not battle tested at all. Probably has bugs. However it has come together rapidly and is ready for people to start banging on it at least. *
Binary data structure toolkit of awesomeness: https://github.com/Benvie/reified It's for defining StructTypes, ArrayTypes, BitfieldTypes, and NumericTypes which are used to map binary data to and from buffers in either Node or Browsers. It's similar to the ES6 data binary API but goes beyond it and diverges from it. The goals of it are to allow arbitrarily complex nested structures, provide an easy to use API for defining those structures, to be both performant and as lightweight as possible. Structs and whatnot aren't a new concept but I've brought some novel ideas to the table. In all cases you can either provide an existing buffer (ArrayBuffers or node Buffers are supported). When structures are initialed without a buffer one is allocated. Since buffers are providing the backing this means multiple structures can map the same data. * var first = Uint32(10000000); /*10000000*/ // 9999972* * var second = Uint16(first); /*38528 */ // 38500* * var third = Uint8(second); /*128 */ third.write(100); // 100* Arbitrary nesting is easy because all the apis in the library are recursive and having the same call signatures regardless of type. * var RGB = new StructType('RGB', { r: 'Uint8', g: 'Uint8', b: 'Uint8' });* * var Canvas = RGB[15][15];* * * * //--> RGBx15x15(675b)[ 15 RGBx15(45b)[ 15 RGB(3b) { R: Uint8 | G: Uint8 | B: Uint8 } ] ]* Those are constructors, and here's what they make: * var canvas = new RGB[3][3] //15x15 is too big for to show here* * //-->* * RGBx3x3* * [ RGBx3* * [ RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 } ],* * RGBx3* * [ RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 } ],* * RGBx3* * [ RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 },* * RGB { R: Uint8 0 | G: Uint8 0 | B: Uint8 0 } ] ]* And finally the origin of the library name is reification, to make real, which is creating JavaScript objects and values from created structures. * canvas.reify()* * //-->* * [ [ { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 } ], * * [ { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 } ], * * [ { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 }, { R: 0, G: 0, B: 0 } ] ]* BitfieldTypes use bitwise operators to map bits in a number to a host structure * var DescriptorFlags = reified('DescriptorFlags', {* * ENUMERABLE: 1, CONFIGURABLE: 2, READONLY: 3, WRITABLE: 4,* * FROZEN: 5, HIDDEN: 6, NOTPRIVATE: 7* * }, 1);* * * * ‹DescriptorFlags›(8bit) //constructor* * 0x1 ENUMERABLE* * 0x2 CONFIGURABLE* * 0x3 READONLY* * 0x4 WRITABLE* * 0x5 FROZEN* * 0x6 HIDDEN* * 0x7 NOTPRIVATE* * * * var desc = new DescriptorFlags;* * desc.HIDDEN = true;* * { ‹DescriptorFlags›* * ENUMERABLE: false,* * CONFIGURABLE: true,* * READONLY: true,* * WRITABLE: true,* * FROZEN: true,* * HIDDEN: true,* * NOTPRIVATE: true }* * * * desc.read() // 6* There's a more succinct API as well for those who dislike the more declarative forms. The exported function `reified` can access all parts of the API based on provided parameters * reified('Uint8') // gets existing constructor named Uint8, this is why naming your structs is good* * new reified('Uint8', buffer, offset, val) // instantiate data, optional buffer, optional offset, optional init val* * reified('Uint[8]'), reified('OctetString', 'Uint[8]') //array, optionally custom named* * reified('OctetString[20][20][20][20][20]') //any depth* * reified('RGB', {R: Uint8, G: Uint8, B: Uint8 }) //constructors and their names are interchangable* * reified('Pallette', RGB, 20);* * reified('Flags', ['array','of','bit','flags','mapped','to','powersOf2'], 1) //optional bytes* * reified('Flags', { Named: 0x10, Flags: 0x14, Flip: 0x20, Bits: 0x64 }, 2) // bytes required* * new reified('RGB') //all named structs can be accessed in this way* A key feature of this library is that structures will not be instantiated into memory until needed. Using lazy prototype accessors the structures are created on demand transparently for the user. Accessing a field returns the data wrapper, not the data itself. In this way you can do something like `canvas[5][10].reify()` and have only caused two wrappers to be initialized (the top one is always there) and a single data access. Even more, you can optionally de-allocate those structures immediately after they finish reification `canvas[5][10].reify(true)` leaves no remnant structure dangling. `canvas.reify(true)` Creates the structure for a brief second to map the data and then immediately destroys it (not the memory). A provided example shows the mapping of a large chunk of the TTF file format. It contains around 30 different Types, over a hundred bitfield flags, maps a large portion of the table structure of the format, and it does this in one recursive single action (with two custom mappings, tied in via an event emitter api providing hooks are key moments). With the format mapped out before ever opening the file, it's simply a matter of initializing the top level struct over top the memory and then reifying the data in one fell swoop and immediate deallocation. Writing is similarly easy. JS objects/values are written recursively using `<data>.write(JSval)`. For writing there's no intermediate step as with reification. Setters allow the data to be written to like a normal JS object. `<data>.key = newValue` will set the value, recursively for objects -> structres or arrays -> arrayTypes. `<data>` wrappers are created as needed to write. `<data>.key = newValue; <data>.key = null`; This will write newValue and then destroy the structure for key, not write null to the buffer. The apis are also suited to cases where keeping the structures instantiated is preferable. This would be when the data needs to be measured continuously, or when one structure is reused over multiple parts of a buffer or to different buffers. For those cases there's `<data>.rebase(buffer)` and `<data>.realign(offset)`. Regardless of the desired usage, simple ways to control the behavior are part of the API and are easy to control. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
