You may be right. I was thinking of compressing the data in individual components before writing. One of the largest items in jodsource is a character list that that is the dump of 1000+ word dictionary that I use for generating test dictionaries - (I've tested JOD dictionaries with 150,000 items) - this list is 4.5 megs that shrinks to 0.5 meg upon zipping. By writing this greatly reduced item to a component and then backing up with packd (to recover the slack) the file shrinks a lot.
On Sun, Oct 12, 2008 at 3:06 PM, Oleg Kobchenko <[EMAIL PROTECTED]> wrote: > jfiles are good for individually growing components, thus they have the > slack. > So zipping would not be very efficient. > > Whereas J locale items are stored in toto. So instead of jfiles, it would > make > sense to use zip files, where each inner file is a J item. > Folders would be locales. > Encoding is 3!:1 for nouns and 3!:1]5!:1 for verbs, advers and conjs. > File name is item name and extension is class (0,1,2,3). > Each file can have a comment. Other files can be stored as usual. > So it could be used as development storage, like JOD, as well as > deployment. > > > On Oct 11, 2008, at 5:31, "John Baker" <[EMAIL PROTECTED]> wrote: > > I will definitely take a look - this could be very useful for preparing > jfiles components. > > On Fri, Oct 10, 2008 at 11:24 PM, Oleg Kobchenko <[EMAIL PROTECTED]> > wrote: > > What J really needs is a new 5!:n call that generates a zip > style compressed byte stream for any J object that is > constructed from only visible ASCII characters. Then we > > Now the 'arc/zip/zbuffer' addon script implements in-memory > buffer compression using zlib. Combined with ascii85 it > produces plain text encoding. > > load 'arc/zip/zbuffer' > > zput85 2000$a. > GhQY8A7B.ZARjo]jPo=\jl7-g'G;/n'bcpqP`h>pQ'59G<)Qao<E%MreC)pqe^HH21bpsD2)D_G > ['I-F[Bk'rFE2PEF`[<Ho^__Gp%&\"$jmIN%1A5QN/EXPNJgS'9M/&O9hWgRbf\5Qc-%ag/1N8$ > /M"$'XK&G&XfHARChdj%D/8V(m-=$'mH^AuERNns;l<?^0ltq43HOB.?a+Lsi[%6a/RPc'e`o#- > f<*r7ERX&%>ISJ,Z#eOb4aZqj5B-,=i?LjR*C"Q7eEJc%c^hgin^?ML=0GoEEV%cK4*U/LJ+2sT > j!RWp4b*"mea#/4hnB'Zn^HYS?b_$hnakB$5C`_3B,L=AW;fBd5O/8Gs,[#+)#,0Q()tW? > ~> > > (-: # zget85 zput85) 10000$a. > 1 > (-: # zget zput ) 10000$a. > 1 > > ascii85 (used in zput85) emits multi-line LF separated output. > So it can be used together with 0 : 0 ... ). > > > > > > From: John Baker <[EMAIL PROTECTED]> > > Oleg, > > The best source of information about jod is the jod.pdf. It can be seen > at: > > http://bakerjd99.googlepages.com/jod.pdf > > You are correct about the representation used in dump files being > inefficient for character data. The dump files in jodsource > are complete dictionary dumps - essentially long JOD > scripts that can rebuild the binary jfiles dictionary. I wrote the dump > code a number of years ago and my main concern was to produce something > that was absolutely portable and editable on just about anything. > The wrapped a. encoding does this by avoiding all character > translation and and "strange character" behaviour. The wrapping > also circuments any line length limitations in J and external > editors. This brute force approach has held up well and even handled new > datatypes (like unicode) without a hiccup. > > What J really needs is a new 5!:n call that generates a zip > style compressed byte stream for any J object that is > constructed from only visible ASCII characters. Then we > could have maximum portability and low byte counts. > > On Thu, Oct 9, 2008 at 1:31 PM, Oleg Kobchenko wrote: > > JOD is an interesting project. It looks like > it manages J definitions. Is there a list of features > or datasheet with a list of things what it is capable > of to get an idea what to expect before actually fully > learning it. > > For example, is there a GUI to manage these definitions? > > Also there is the JOD source addon, which is noticeable > by its size and casually looking into it is shows some > kind of encoding. From first impression it may not be > most efficient. What is the reasoning behind it? > > For example, here JOD encoding is more than 2.5 times larger > than ascii85 encoding. > > load'general/jod' > ('zz';61) wraplinear_ajodmake_ (3 : '5!:5<''y''')a.{~ ?.~255 > zz=:'' > zz=:zz,'246 220 69 151 40 194 29 168 174 89 196 53 13 181 4 107 63 17' > zz=:zz,'5 82 249 12 193 121 187 19 186 88 242 114 231 80 96 150 147 1' > zz=:zz,'77 66 211 162 153 160 136 61 169 14 36 127 129 7 203 182 208 ' > zz=:zz,'195 133 239 243 49 117 98 100 138 157 116 5 202 142 52 200 23' > zz=:zz,'6 118 55 188 170 86 244 25 207 84 85 51 226 171 143 67 210 21' > zz=:zz,'8 234 158 120 35 173 212 183 59 101 237 190 238 180 131 206 1' > zz=:zz,'39 201 179 224 223 184 43 232 140 227 216 132 154 222 104 0 1' > zz=:zz,'5 78 122 39 8 50 251 192 109 47 229 247 245 204 23 230 165 17' > zz=:zz,'2 115 191 163 110 38 233 77 253 42 6 79 34 105 106 75 44 21 2' > zz=:zz,'0 144 95 130 56 60 111 141 3 221 164 87 112 10 94 235 228 81 ' > zz=:zz,'54 74 103 250 128 185 73 17 102 135 152 123 225 33 9 197 137 ' > zz=:zz,'46 83 18 26 97 214 159 217 134 32 71 70 31 16 64 93 248 156 9' > zz=:zz,'2 2 252 205 45 37 22 11 146 254 198 48 57 213 27 108 68 215 1' > zz=:zz,'24 65 91 145 176 58 209 113 62 219 28 119 166 76 99 240 148 1' > zz=:zz,'67 90 189 72 149 1 30 41 161 199 126 24 155 178 241 125{a. ' > zz=:912{.zz > #('zz';61) wraplinear_ajodmake_ (3 : '5!:5<''y''')a.{~ ?.~255 > 1083 > > load'convert/misc/ascii85' > 'zz=:fromascii85 0 : 0',LF,')',~toascii85 a.{~ ?.~255 > zz=:fromascii85 0 : 0 > > p=r*'.)I"'Y$&\#%CH=S5Hdu\%)I5$'=q<:Ep$8,QBSR?e"W8'Lcq^I,_(@MbI5d8L%"UfF^em= > STG%XN\niDFuR(6<)ka6X,fWrg?P8S,H]J)4([fnm^.VlMoi<`hmb3#N8rQDRbh]X%o.Eu > > #Vu:mD(k#Foun67V6"aQUM_FC:&QOG:EjLf9.UelOC*T!4E$2]h5"+k$=<&Z;)V$ZqLH+J&S?fc > Hf6+S`JH3G&[EMAIL PROTECTED] > +R35[F/T>QXiJ/L;/?P5`HT3Ro'b7+ZfP>EQ-Nd;_A)*-\8RA+5_\ > .mIl!?N\`a*nkpZLN> > ~> > ) > #'zz=:fromascii85 0 : 0',LF,')',~toascii85 a.{~ ?.~255 > 404 > > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
