Re: An idea for doing pack.
On Thu, 28 Jul 2005 15:46:14 +0300, Yuval Kogman
[EMAIL PROTECTED] wrote:
[...]
I like your Pack object - that is the parsed template, but I'd also
like to be able to generate these templates with a programmatic
interface that isn't string concatenation...
Is it just a simple data structure? Or do you have anything in mind?
I was thinking of having it compile down to as close to native code as
possable. But this may be premature optimization.
Will we have more powerful nesting? I think this is in order
nowadays.
Thinking about it the declarations are very similar to (un)pack
templates. So it may be possable to build them up from them. If my
compile template macro took a name and that name was then treated as a
new declaration we could do something like this.
Pack::compile byte {
@^_ :=: { pack = chr, unpack = ord }
}
Pack::compile null {
0x00 :=: byte
}
Indeed I can see the pack module as creating all of the builtins this
way. Further more this approach makes it concivable to have a set of
methods on the Pack object itself for example.
method Pack::declaration() returns Array of Pack
is doc(The declarations that this pack consists of in the
order of there application) {...}
method Pack::packer() returns Code is doc(The code that will
be run to pack using this pack object) {...}
method Pack::unpacker() returns Code is doc(The code that will
be run to unpack using this pack object) {...}
Mm if we have something that returns previousely defined
delarions and a constructor.
method Pack::declaration(::Class $class,Str $name) returns Pack is
doc(The Pack object which corrasponds to this declaration) {...}
method Pack::new(::Class $class,@declaration of Pack) returns
Pack is doc(Create a new Pack object composed of declarations) {...}
This would allow the best of both worlds. A template based system for
those of us who are creating our Pack's at compile time, and also an
OOish programmatic interface for thouse of us who need something more
convenient then string concatination.
Actually, to be really wicked, I think that packing is just applying
rules in reverse (this is tricky and indeterminate, since there are
no alternatives or options in unpacking/packing), and unpacking is
the opposite, with two exceptions:
captures are converted by looking at the binary data and
interpreting it, not just stuffing it in a string
the syntax is different
The other diffrence is that unpacking makes use of pattens of fixed
length so it doesn't need to parse its input.
--
Please excuse my spelling as I suffer from agraphia. See
http://dformosa.zeta.org.au/~dformosa/Spelling.html to find out more.
Free the Memes.
Re: An idea for doing pack.
I have a fundamental disagreement with what pack used to be - it's too stringish... =) the printf and unpack syntaxes always bothered me because they are akin to 'eval'ing, more than they are to quasi quoting. I like your Pack object - that is the parsed template, but I'd also like to be able to generate these templates with a programmatic interface that isn't string concatenation... Is it just a simple data structure? Or do you have anything in mind? Will we have more powerful nesting? I think this is in order nowadays. Actually, to be really wicked, I think that packing is just applying rules in reverse (this is tricky and indeterminate, since there are no alternatives or options in unpacking/packing), and unpacking is the opposite, with two exceptions: captures are converted by looking at the binary data and interpreting it, not just stuffing it in a string the syntax is different Anywho, I like your proposal, but i'd like some even fresher ideas into pack and unpack, so i'm just trying to kick the thread around... -- () Yuval Kogman [EMAIL PROTECTED] 0xEBD27418 perl hacker /\ kung foo master: /me kicks %s on the nose: neeyah! pgpPzVO2omGkV.pgp Description: PGP signature
An idea for doing pack.
Last night I had an idea about a possable pack API. Most likely when
Pugs gets signifigently powerfull I will attempt to implement it.
However I would like everyones input, below is a draft of its POD.
=head1 NAME
Pack - (un)pack structures as defined by a Template
=head1 SYNOPSIS
my Pack $template = Pack::compile {uint8 rat32};
my $string = pack($template, 244, 3.14);
my @unpack = unpack($template, $string);
=head1 ABSTRACT
This perl library defines a declarative minilanuage that allows the
conversion of Perl6 variables into (and from) a string of bytes.
=head1 DESCRIPTION
=head2 MACROS and SUBS
macro Pack::compile ($template) is parsed (/\{ Pack.template \}/) {...}
This macro is used to compile a template at compile time. This may
throw a execution at compile time if there is an error in the template.
sub Pack::eval (Str $template) returns Pack {...}
And this sub will compile a template at runtime. This may return
undef and set $! if there is an error in the template.
sub Pack::pack (Pack $template,[EMAIL PROTECTED]) returns Str of byte {...}
This function takes a [EMAIL PROTECTED] of ordinary Perl values and converts
them
into a string of bytes according to the $template. If you procide
more arguments then $template requires, the extra arguments are
ignored.
sub Pack::pack (Str $template,[EMAIL PROTECTED]) returns Str of byte {...}
A convenience function, $template is compiled into a Pack object and
then run on @list;
sub Pack::unpack (Pack $template,Str of byte $expr) returns List {...}
sub Pack::unpack (Str $template,Str of byte $expr) returns List {...}
These functions do the reverse of Pack::pack
=head2 THE PACK TEMPLATE DESCRIPTION LANGUAGE (PTDL)
=head3 SYNTAX
The template description language describes the structure of the
string as a sequence of declarations. Each declaration consists of a
type, an optional argument and zero or more adjectives. Also each
declaration may be bound to a variable.
grammar {
rule template :w { [binding? declaration]* };
rule binding :w { [Perl6.variable | Perl6.literal] :=: }
rule declaration :w { [\: adjective]* Perl6.name [\( argument \)]? }
rule adjective :w { Perl6.name [\( argument \)]? }
rule argument :w { Perl6.variable | Perl6.closure | Perl6.literal }
}
For example the following are valid templates
$^0 :=: byte
@^_ :=: str(8)
@^_ :=: str($^0)
$^1 :=: rat32
=head3 SEMANTICS
When packing if a declaration is bound to a scalar variable then the
contents of that variable are packed into the return value of pack
according to the nature of that declaration.
Simmerly when unpacking each declaration will set the value of the
scalar it is bound to based on its nature and the value of $expr.
$^number has a special meaning in PTDL $^i is the ith (zero indexed)
item of @list when packing and the ith (zero indexed) item of the
return array when unpacking.
If a declaration is bound to an array, when packing items will be
shifted out of the array and then packed according to the declaration.
Simmerly when unpacking the array will have the declaration's return
values pushed into it.
@^_ has a special meaning in PTDL. It is an alias to @list when
packing and an alias to the return array when unpacking. By default a
declaration is bound to @^_ .
If a declaration is bound to a literal then when packing the value of
that literal is packed, when unpacking the unpacked value is
discarded.
The following declarations are built in.
int1
int2
int4
int8
int16
int32 (aka int on 32-bit machines)
int64 (aka int on 64-bit machines)
uint1 (aka bit) (like vec)
uint2
uint4
uint8 (aka byte)
uint16
uint32
uint64
rat32
rat64 (aka rat on most architectures)
rat128
str A string of bytes
utf8 A number encoded using the utf8 mapping (doesn't check for bad unicode)
UTF-8 A codepoint endcoded using UTF-8
noop Does nothing (though its adjectives may have side effects)
hex A hexadecimal string
uuencode A uuencoded string
base64 A base64 encoded string
base32 A base32 encoded string
base16 A base16 encoded string
berA ber compressed integer
The following adjectives are built in.
:bigend Endcode this using bigendian order
:litend Endcode this using littleendian order
:pad($n) Pad to $n bytes
:padchar($char) Pad using $char
:term($char) String is $char terminated
:mvabsMove to absolute position filling with padchar if defined.
:mvrelMove to relative position filling with padchar if defined.
For convenience the following declarations are defined.
a($n) = :pad($n) :padchar(\x00) byte
A($n) = :pad($n) :padchar( ) byte
b($n) = :bigend bit($n)
B($n) = :litend bit($n)
c = int8
C = uint8
d = rat64
f = rat32
h = :bigend hex
H = :litend hex
i = int
I = uint
l = int32
L = uint32
n = :bigend int16
N = :bigend int32
q = int64
Q = uint64
s = int16
S = uint16
u = uuencode
U = UTF-8
v = :litend int16
V = :litend int32
w = ber
x = :pad(1) :padchar(\x00) noop
X =
