Here is a useful set of utilities for working with variable length
strings/lists packed in a variable (or row) without boxes. One reason to do
such packing is that strings take much less memory than other lists , and much
less memory than boxes, together with faster general operations. The context
implies medium sized strings where most are under 255 bytes long, and those
that are more are not much more (A mailing address for example)
toVbyte =: [: (({."1 ,. 1:) #&, 255 ,. {:"1) 0 255 #: ]
fromVbyte=: 255&~: +/;.2 ]
fromSomeVbyte =: [ (;@:}. ;~ [ {. +/&>@:]) 255&~:@:] <;.2 ] NB. x is number of
Vbytes to take. Result is unpacked
NB. nums and remainder list.
toVint =: 1 : '[: (({."1 ,. 1:) #&, m ,. {:"1) (0, m) #: ]'
fromVint=: 1 : 'm&~: +/;.2 ]'
toVstring =: toVbyte@:# , a.&i.
toVstrings =: [: ; toVstring each NB. places list as length string length
string
fromVstring =: 1&unpackStrings
fromAllstrings =: [: ([: }.@:}: {."1) [: fromVstring@:( >@:{:)^:(a:) <
packStrings =: [: (([: ; {:"1) ,~ [: toVbyte@:> {."1 ) 1 fromSomeVbyte &>
toVstring each NB. places all lengths NB.at head,
then packed list after.
unpackStrings =: [: _2&}. ([ (_2&}. , (({.@:[ }. ]) (, <)~ }.@:[ ;~ {.@:[ {.
])&>/@:(_2&{.))@:]^:[ fromSomeVbyte)
There are 2 ways to pack strings depending on whether there is a fixed number
of them, or an unknown number.
toVstrings 'abc';'defg'; 257 $ 'ghj'
3 97 98 99 4 100 101 102 103 255 2 103 104 106 103...
to get originals...
a.&{ each fromAllstrings toVstrings 'abc';'defg'; 257 $ 'ghj'
The version that places all lengths at head:
packStrings 'abc';'defg'; 257 $ 'ghj'
3 4 255 2 97 98 99 100 101 102 103 103 104 106 ...
You can obtain just the lengths:
> {. 3 fromSomeVbyte packStrings 'abc';'defg'; 257 $ 'ghj'
3 4 257
fromSomeVbyte returns 2 boxes: the first x numbers followed by the remainder of
the list.
you can obtain the unpacked strings if you know how many there are.
{&a. each 2{. 3 unpackStrings packStrings 'abc';'defg'; 257 $ 'ghj'
┌───┬────┐
│abc│defg│
└───┴────┘
You can make mixed integer and string packing pretty easily: (put all non
strings/lists first)
] packing =: (toVbyte 344 622 11) , packStrings 'abc';'defg';'hj'
255 89 255 255 112 11 3 4 2 97 98 99 100 101 102 103 104 106
'
] 'ints strs' =: 3 fromSomeVbyte packing
┌──────────┬──────────────────────────────────────┐
│344 622 11│3 4 2 97 98 99 100 101 102 103 104 106│
└──────────┴──────────────────────────────────────┘
{&a. each _2}. 3 unpackStrings strs
┌───┬────┬──┐
│abc│defg│hj│
└───┴────┴──┘
----- Original Message -----
From: Raul Miller <[email protected]>
To: Programming forum <[email protected]>
Cc:
Sent: Monday, December 29, 2014 12:44 PM
Subject: Re: [Jprogramming] variable integer coding (challenge)
Here's a faster version of toVint, for at least some right arguments:
toVint2=: [: (({."1 ,. 1:) #&, 255 ,. {:"1) 0 255 #: ]
Here's an implementation of fromVint:
fromVint=: 255&~: +/;.2 ]
Thanks,
--
Raul
On Mon, Dec 29, 2014 at 12:12 PM, 'Pascal Jasmin' via Programming
<[email protected]> wrote:
> A common design decision is how large to make a field. For many numbers, we
> think that a byte will be large enough to hold practical values, but one day
> we will find out that it is not. This is essentially the Y2k problem, or
> year 2032 issue.
>
> A simple variable integer coding (tied to byte ranges)
>
> toVint =: [: ; (255&| ,~ 255 #~ <.@%&255) each
>
> toVint 566 44
> 255 255 56 44
>
> It now takes 3 bytes to store 566, and one byte to store 44.
>
> the challenge is to write fromVint such that
>
>
> 566 44 -: fromVint 255 255 56 44
>
> as an extra challenge, is there a way to write toVint or fromVint such that
> it is faster than using intermediate boxing?
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
