Re: ANN: Gloss, a byte-format DSL

2011-01-12 Thread pepijn (aka fliebel)
It is not. Thanks for the information anyway. The real problem consists of a set of tags, with a type identified by a byte. One of those tags is a compound tag which can contain any number of other tags and is terminated by a \0. This means I need parser behavior, rather than a linear regex-alike b

Re: ANN: Gloss, a byte-format DSL

2011-01-10 Thread Zach Tellman
I don't know if your example codec is as simple as your real problem, but here's a codec that will work for the string you provided: (repeated (string :utf-8 :delimiters ["\n" "\n\0"]) :delimiters ["\n\0"] :strip-delimiters? false) This terminates the whole sequence only on \n\0, but doesn't

Re: ANN: Gloss, a byte-format DSL

2011-01-10 Thread pepijn (aka fliebel)
Oh, right. My code does have a start and an end. I'm using header for the start. So the only way Gloss could do this is with a fn that combines header and repeated into one? On Jan 10, 12:29 pm, Ken Wesson wrote: > On Mon, Jan 10, 2011 at 4:52 AM, pepijn (aka fliebel) > > > > > > > > > > wrote:

Re: ANN: Gloss, a byte-format DSL

2011-01-10 Thread Ken Wesson
On Mon, Jan 10, 2011 at 4:52 AM, pepijn (aka fliebel) wrote: > The later. The string in my example is \n terminated, so it should > read past the \0, and then the outer list should terminate on the last > \0. > > My point is that I need to parse recursive structures, which of course > contain the

Re: ANN: Gloss, a byte-format DSL

2011-01-10 Thread pepijn (aka fliebel)
The later. The string in my example is \n terminated, so it should read past the \0, and then the outer list should terminate on the last \0. My point is that I need to parse recursive structures, which of course contain the same terminator as the outer one. What if I wanted to parse null termina

Re: ANN: Gloss, a byte-format DSL

2011-01-06 Thread Eric Schulte
Hey Geoff, This sort of feedback and code recommendations is exactly what I was hoping for. Geoff writes: > Hey Eric > I did something similar for handling unsigned types. I think it's > simpler and probably faster, not that I've checked. Here's a gist of > it https://gist.github.com/767887 >

Re: ANN: Gloss, a byte-format DSL

2011-01-06 Thread Geoff
Hey Eric I did something similar for handling unsigned types. I think it's simpler and probably faster, not that I've checked. Here's a gist of it https://gist.github.com/767887 That code is part of a little library I wrote a while ago https://github.com/geoffsalmon/bytebuffer It's closer to your

Re: ANN: Gloss, a byte-format DSL

2011-01-05 Thread Zach Tellman
Thanks for the patch! It's too bad that Gloss wasn't directly suited to your needs, but I appreciate you taking the time to familiarize yourself with the code and add new functionality. Zach On Jan 5, 2:45 pm, "Eric Schulte" wrote: > also, here's a patch to Gloss which I've used locally but whi

Re: ANN: Gloss, a byte-format DSL

2011-01-05 Thread Zach Tellman
I'm confused by what you expect the decoded value for that string to look like. Is it: [["blabla" "hi hi" "jg"] [" g"]] or [["blabla" "hi hi" "jg\0 g"]] Zach On Jan 5, 3:36 am, "pepijn (aka fliebel)" wrote: > Nothing :( > > $ lein repl > "REPL started; server listening on localhost:3

Re: ANN: Gloss, a byte-format DSL

2011-01-05 Thread Eric Schulte
also, here's a patch to Gloss which I've used locally but which may be generally useful. Cheers -- Eric "Eric Schulte" writes: > Thanks for sharing this library, I found reading your code to be very > useful. > > I need to be able to read and write elf files from within Clojure code, > and Glo

Re: ANN: Gloss, a byte-format DSL

2011-01-05 Thread Eric Schulte
Thanks for sharing this library, I found reading your code to be very useful. I need to be able to read and write elf files from within Clojure code, and Gloss initially looked like a good option. However much of the streaming support and frame-based conception got in the way of my particular nee

Re: ANN: Gloss, a byte-format DSL

2011-01-05 Thread pepijn (aka fliebel)
Nothing :( $ lein repl "REPL started; server listening on localhost:32399." user=> (use '[gloss core io]) nil user=> (defcodec t (repeated (string :utf-8 :delimiters ["\n"]) :delimiters ["\0"])) #'user/t user=> (decode t (.getBytes "blabla\nhihi\njg\0 g\n\0")) java.lang.Exception: Cannot evenl

Re: ANN: Gloss, a byte-format DSL

2011-01-02 Thread Zach Tellman
There was a bug in repeated, which is fixed now. Pull the latest from github or clojars and let me know how it goes. Zach On Jan 2, 3:29 am, "pepijn (aka fliebel)" wrote: > Okay, clicked send to early. The header also contains 0-bytes, so the > repeated stops 'mid-sentence' and tries to balance

Re: ANN: Gloss, a byte-format DSL

2011-01-02 Thread pepijn (aka fliebel)
Okay, clicked send to early. The header also contains 0-bytes, so the repeated stops 'mid-sentence' and tries to balance things. Is there any way Gloss can handle nested structures like this? On Jan 2, 12:20 pm, "pepijn (aka fliebel)" wrote: > Hi, > > Thanks for helping out. After using codecs ra

Re: ANN: Gloss, a byte-format DSL

2011-01-02 Thread pepijn (aka fliebel)
Hi, Thanks for helping out. After using codecs rather than frames, I get even weirder errors. My code now gives me: java.lang.Exception: Cannot evenly divide bytes into sequence of frames. Hard coding the header followed by a terminating :byte works as it should: (decode (compile-frame [tag tst

Re: ANN: Gloss, a byte-format DSL

2011-01-01 Thread Zach Tellman
The header->body function in (header ...) must return a codec, so you need to call compile-frame on the vector you're generating. Since you don't want to call compile-frame every time you decode a frame, you can memoize the function. A version that does both can be found at https://gist.github.co

Re: ANN: Gloss, a byte-format DSL

2011-01-01 Thread pepijn (aka fliebel)
Hey, I am trying Gloss for reading NBT [1] files. First thing I did like is that it seems to make things real easy. First thing I did not like is the weak separation between types like :byte and extra data like :foo. I think I'm nearly done with the NBT reader [2], but I ran into a problem. What

Re: ANN: Gloss, a byte-format DSL

2010-11-28 Thread Zach Tellman
You're right, that's an omission from the frame syntax. I'll add the ability for all or part of the frame to be scoped as (little- endian ...) and (big-endian ...), with big-endian as the default. Just as a side-note, though, Calx [1] is already handling little- endian data by using encode-to-buf

Re: ANN: Gloss, a byte-format DSL

2010-11-28 Thread zoka
If Gloss is to decode incoming packet (byte array) in little-endian format it is straightforward: Wrap byte array into ByteBuffer b, invoke b.order(LITTLE_ENDIAN) and pass b to decode function that will return Clojure map of decoded values. However, when outgoing packet byte array is to be produce

Re: ANN: Gloss, a byte-format DSL

2010-11-27 Thread zoka
The most general case would be a codec that can have mixture of big and little endian fields, so legacy comms protocols or file formats can be supported. For example: (defcodec mixed (ordered-map :b :int16, :a :float32-le)) ; total 6 bytes, first 2 bytes short in big endian format (JVM default),

Re: ANN: Gloss, a byte-format DSL

2010-11-24 Thread Zach Tellman
ByteBuffers have an order() method which allows you to toggle the endianness. I haven't tested this, but since everything is built on top of Java's ByteBuffer functionality it should be fine as long as the ByteBuffers are correctly set and correctly ordered with respect to each other. Zach On No

Re: ANN: Gloss, a byte-format DSL

2010-11-24 Thread Zach Tellman
On Nov 24, 12:42 am, Michael Wood wrote: > Hi > > On 24 November 2010 04:43, Zach Tellman wrote: > > > There are a couple of different things being discussed here.  I don't > > think there's any harm in allowing maps as frames, as long as people > > understand that they're arbitrarily ordered (al

Re: ANN: Gloss, a byte-format DSL

2010-11-24 Thread zoka
JVM stores numbers in in big endian format - is there a way to process binary stream containing little endian numbers? Zoka On Nov 24, 7:24 am, Zach Tellman wrote: > Good question.  The solution didn't make the cut for my initial > release, but will be added soon.  My plan is to have an (ordered

Re: ANN: Gloss, a byte-format DSL

2010-11-24 Thread Michael Wood
Hi On 24 November 2010 04:43, Zach Tellman wrote: > There are a couple of different things being discussed here.  I don't > think there's any harm in allowing maps as frames, as long as people > understand that they're arbitrarily ordered (alphabetically by key, > but that's an implementation det

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Zach Tellman
There are a couple of different things being discussed here. I don't think there's any harm in allowing maps as frames, as long as people understand that they're arbitrarily ordered (alphabetically by key, but that's an implementation detail) and that keys specified aren't optional - if they're no

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Chris Perkins
On Nov 23, 3:24 pm, Zach Tellman wrote: > On Nov 23, 12:12 pm, Chris Perkins wrote: > > I have only taken a quick look, so maybe I'm misunderstanding the > > intent, but it's not clear to me how you would use this for sending > > and receiving structured data from, say, a C program. > > > Taking

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Daniel Kersten
Looks really good. Thanks for another great library! On 23 November 2010 21:12, Moritz Ulrich wrote: > I would go for the vector. It's easier than writing (ordered-map ...) all > the time. If you want to generate structs dynamically, you would even simply > create a vector of alternating pairs an

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Moritz Ulrich
I would go for the vector. It's easier than writing (ordered-map ...) all the time. If you want to generate structs dynamically, you would even simply create a vector of alternating pairs and do (apply ordered-map my-vec). Btw: Great library! I think I will rewrite my binary-parser from a project

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Zach Tellman
Good question. The solution didn't make the cut for my initial release, but will be added soon. My plan is to have an (ordered- map ...) frame which encodes and decodes the keys in the given order. So for C interop, the frame would be (ordered-map :a :int16, :b :float32) An alternative would be

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Chris Perkins
On Nov 23, 12:03 pm, Zach Tellman wrote: > When writing Calx [1], I discovered it was a huge pain to deal with > mixed C datatypes in Java.  When writing Aleph [2], I discovered the > problem increases by a factor of ten when dealing with streams of > bytes.  In an attempt to alleviate my own pain

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Chris Riddoch
On Tue, Nov 23, 2010 at 10:03 AM, Zach Tellman wrote: > When writing Calx [1], I discovered it was a huge pain to deal with > mixed C datatypes in Java.  When writing Aleph [2], I discovered the > problem increases by a factor of ten when dealing with streams of > bytes. You're not the only one.

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Zach Tellman
Yes. There are Eclipse license headers on all the files, I'll just update the README to reflect that. On Nov 23, 9:51 am, Stuart Halloway wrote: > Neat. License same as Clojure's? > > Stu > > > > > > > > > When writing Calx [1], I discovered it was a huge pain to deal with > > mixed C datatypes

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Dave Newton
This looks a *lot* like something I did a long time ago ("Data File Disassembler") when I spent a lot of time reverse-engineering things. I had a view/edit layer on top (On Mac, back when a Mac II was extreme hardware :) and man, was it useful... had "pluggable" modules for microcontroller disassem

Re: ANN: Gloss, a byte-format DSL

2010-11-23 Thread Stuart Halloway
Neat. License same as Clojure's? Stu > When writing Calx [1], I discovered it was a huge pain to deal with > mixed C datatypes in Java. When writing Aleph [2], I discovered the > problem increases by a factor of ten when dealing with streams of > bytes. In an attempt to alleviate my own pain, a

ANN: Gloss, a byte-format DSL

2010-11-23 Thread Zach Tellman
When writing Calx [1], I discovered it was a huge pain to deal with mixed C datatypes in Java. When writing Aleph [2], I discovered the problem increases by a factor of ten when dealing with streams of bytes. In an attempt to alleviate my own pain, and hopefully help a few other people out, I've