Thank you for testing.
The time efficiency improvement is disappointing so that
the patch must be rejected. The genral solution will be
implementing special code in J engine to handle box array
concatenation but unsure if that is feasible and easy.
Пн, 17 мар 2014, June Kim (김창준) писал(а):
> Hi
>
> Thanks Bill. And sorry for my delayed reply. BTW, I am cross-posting this
> to the J programming mailing list, for the people who are interested in the
> follow-up.
>
> I've ran your version of json against the standard package version and the
> standard pure python json module. Your version is faster but takes larger
> memory, but the result is not exactly the same(more on this at the end).
>
> The standard version:
>
> bm=. 1!:1 <'Bookmarks'
> ts=: 6!:2,7!:2@]
> ts 'dec_json bm'
> 66.3499 5.36879e8
>
> The new version you provided:
>
> ts 'dec_json bm'
> 34.6758 1.36445e9
>
> The standard pure python version:
>
> 0.4986 75272192
>
> In J, the standard json and the new version you provided resulted in
> different output.
>
> newresult -: oldresult
> 0
>
> Yet, the enc_json'd result seems to be the same.
>
> newresult -:&enc_json standardresult
> 1
>
> However, the resulted J representation is different. I just sampled one
> difference -- I am not sure if it could help.
>
>
> 3!:3 (((<1 0) {>@(1 {>@((<1 0){ >@((<1 1) { >@((<1 1){]))))))
> standardresult
> e300000000000000
> 2000000000000000
> 0100000000000000
> 0000000000000000
> 2800000000000000
> e300000000000000
> 2000000000000000
> 0100000000000000
> 0100000000000000
> 0100000000000000
> 3000000000000000
> e300000000000000
> 2000000000000000
> 0c00000000000000
> 0200000000000000
> 0200000000000000
> 0600000000000000
> 9000000000000000
> c800000000000000
> f800000000000000
> 2801000000000000
> 7001000000000000
> a001000000000000
> d001000000000000
> 1002000000000000
> 4002000000000000
> 9002000000000000
> c002000000000000
> f002000000000000
> e300000000000000
> 0200000000000000
> 0a00000000000000
> 0100000000000000
> 0a00000000000000
> 646174655f616464
> 6564000000000000
> e300000000000000
> 0200000000000000
> 0200000000000000
> 0100000000000000
> 0200000000000000
> 6964000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 6e616d6500000000
> e300000000000000
> 0200000000000000
> 1800000000000000
> 0100000000000000
> 1800000000000000
> 73796e635f747261
> 6e73616374696f6e
> 5f76657273696f6e
> 0000000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 7479706500000000
> e300000000000000
> 0200000000000000
> 0300000000000000
> 0100000000000000
> 0300000000000000
> 75726c0000000000
> e300000000000000
> 0200000000000000
> 1100000000000000
> 0100000000000000
> 1100000000000000
> 3133303031353737
> 3430313837303032
> 3100000000000000
> e300000000000000
> 0200000000000000
> 0500000000000000
> 0100000000000000
> 0500000000000000
> 3330363133000000
> e300000000000000
> 0200000000000000
> 2200000000000000
> 0100000000000000
> 2200000000000000
> ec9584eab8b02045
> 5350202d20416e64
> 726f696420eba788
> ecbc93ec9d9820ec
> 95b1000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 3230343300000000
> e300000000000000
> 0200000000000000
> 0300000000000000
> 0100000000000000
> 0300000000000000
> 75726c0000000000
> e300000000000000
> 0200000000000000
> 7e00000000000000
> 0100000000000000
> 7e00000000000000
> 68747470733a2f2f
> 6d61726b65742e61
> 6e64726f69642e63
> 6f6d2f6465746169
> 6c733f69643d636f
> 6d2e686577697474
> 736f66742e626162
> 7926666561747572
> 653d746f702d7061
> 6964233f743d5732
> 3531624777734d53
> 77794c4449774e69
> 7769593239744c6d
> 686c64326c306448
> 4e765a6e5175596d
> 466965534a640000
>
> 3!:3 (((<1 0) {>@(1 {>@((<1 0){ >@((<1 1) { >@((<1 1){])))))) newresult
> e300000000000000
> 2000000000000000
> 0100000000000000
> 0000000000000000
> 2800000000000000
> e300000000000000
> 2000000000000000
> 0100000000000000
> 0000000000000000
> 2800000000000000
> e300000000000000
> 2000000000000000
> 0c00000000000000
> 0200000000000000
> 0200000000000000
> 0600000000000000
> 9000000000000000
> c800000000000000
> f800000000000000
> 2801000000000000
> 7001000000000000
> a001000000000000
> d001000000000000
> 1002000000000000
> 4002000000000000
> 9002000000000000
> c002000000000000
> f002000000000000
> e300000000000000
> 0200000000000000
> 0a00000000000000
> 0100000000000000
> 0a00000000000000
> 646174655f616464
> 6564000000000000
> e300000000000000
> 0200000000000000
> 0200000000000000
> 0100000000000000
> 0200000000000000
> 6964000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 6e616d6500000000
> e300000000000000
> 0200000000000000
> 1800000000000000
> 0100000000000000
> 1800000000000000
> 73796e635f747261
> 6e73616374696f6e
> 5f76657273696f6e
> 0000000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 7479706500000000
> e300000000000000
> 0200000000000000
> 0300000000000000
> 0100000000000000
> 0300000000000000
> 75726c0000000000
> e300000000000000
> 0200000000000000
> 1100000000000000
> 0100000000000000
> 1100000000000000
> 3133303031353737
> 3430313837303032
> 3100000000000000
> e300000000000000
> 0200000000000000
> 0500000000000000
> 0100000000000000
> 0500000000000000
> 3330363133000000
> e300000000000000
> 0200000000000000
> 2200000000000000
> 0100000000000000
> 2200000000000000
> ec9584eab8b02045
> 5350202d20416e64
> 726f696420eba788
> ecbc93ec9d9820ec
> 95b1000000000000
> e300000000000000
> 0200000000000000
> 0400000000000000
> 0100000000000000
> 0400000000000000
> 3230343300000000
> e300000000000000
> 0200000000000000
> 0300000000000000
> 0100000000000000
> 0300000000000000
> 75726c0000000000
> e300000000000000
> 0200000000000000
> 7e00000000000000
> 0100000000000000
> 7e00000000000000
> 68747470733a2f2f
> 6d61726b65742e61
> 6e64726f69642e63
> 6f6d2f6465746169
> 6c733f69643d636f
> 6d2e686577697474
> 736f66742e626162
> 7926666561747572
> 653d746f702d7061
> 6964233f743d5732
> 3531624777734d53
> 77794c4449774e69
> 7769593239744c6d
> 686c64326c306448
> 4e765a6e5175596d
> 466965534a640000
>
>
> On Tue, Mar 4, 2014 at 12:40 PM, bill lam <[email protected]> wrote:
>
> > June,
> >
> > I changed json decoder to avoid intermediate box contenation, the
> > trade-off might be a much larger space requirement. Please test
> > its performance, the space requirement, and will the result be
> > correct.
> >
> > Thanks.
> >
> > --
> > regards,
> > ====================================================
> > GPG key 1024D/4434BAB3 2008-08-24
> > gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
> > gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3
> >
> coclass 'json'
> enc_json=: 3 : 0
> select. t=. 3!:0 y
> case. 2;131072 do.
> if. y-:'json_true' do. 'true'
> elseif. y-:'json_false' do. 'false'
> elseif. y-:'json_null' do. 'null'
> elseif. do.
> '"', '"',~ jsonesc utf8^:(131072=t) ,y
> end.
> case. 1;4;8 do.
> ":!.17 {.,y
> case. 32 do.
> if. 2>$$y do.
> if. 0=#y do.
> '[]'
> else.
> s=. '['
> for_v. y do.
> s=. s , ',',~ enc_json >v
> end.
> ']',~ }:s
> end.
> elseif. 2=$$y do.
> assert. 0 2 e.~ {.$y
> if. 0=#y do.
> '{}'
> else.
> s=. '{'
> for_i. i.{:$y do.
> s=. s , ':',~ enc_json (<0,i){::y
> s=. s , ',',~ enc_json (<1,i){::y
> end.
> '}',~ }:s
> end.
> elseif. do. 13!:8[3
> end.
> case. do. 13!:8[3
> end.
> )
> JSONESC0=: LF, CR, TAB, FF, (8{a.), '\/"'
> JSONESC1=: 'nrtfb\/"'
> JSONASC=: JSONESC0, 32}.127{.a.
> jsonesc=: 3 : 0
> txt=. y
> msk=. txt e. JSONESC0
> if. 1 e. msk do.
> ndx=. , ((I. msk) + i. +/ msk) +/ 0 1
> new=. , '\',.JSONESC1 {~ JSONESC0 i. msk#txt
> txt=. new ndx } (1 + msk) # txt
> end.
> msk=. -. txt e. JSONASC
> if. 1 e. msk do.
> new=. 'u',"1 '0123456789abcdef' {~ 16 16 16 16 #: a. i. msk # txt
> ndx=., ((I. msk) + 5 * i. +/ msk) +/ i. 6
> txt=. (, '\',("1) new) ndx } (1 + msk * 5) # txt
> end.
> txt
> )
> classes=. 3<. '"\[],{}:' (#@[ |&>: i.) a.
> classes=. 0 (I.a.e.' ',CRLF,TAB)} (]+4*0=])classes
>
> words=: (0;(0 10#:10*".;._2]0 :0);classes)&;:
> 0.0 1.1 2.1 3.1 4.1
> 1.0 5.0 6.0 1.0 1.0
> 4.0 4.0 4.0 4.0 4.0
> 0.3 1.2 2.2 3.2 4.2
> 0.3 1.2 2.0 3.2 4.0
> 0.3 1.2 2.2 3.2 4.2
> 1.0 1.0 1.0 1.0 1.0
> )
> jsonValue=: 3 : 0
> if. 0=#y do. <y
> elseif. 'true'-:y do. 'json_true'
> elseif. 'false'-:y do. 'json_false'
> elseif. 'null'-:y do. 'json_null'
> elseif. '"'={. y do. evalbs }.}:y
> elseif. '0123456789.+-' -.@e.~ {. y do. evalbs y
> elseif. do. {. 0". 'Ee-_' charsub y
> end.
> )
> subst2=: (_2<\'\"\\\/\b\f\n\r\t')&(i.{(34 92 47 8 12 10 13 9{&.><a.),])"0
> hexchars=. 'ABCDEF0123456789abcdef'
> dfh=: 16 #. 16 | _6 + hexchars i. ]
> subst6=: (8 u:[:u:@dfh 2}.]) ::]^:(6&=@#*.('\'={.)*.'uU'e.~{.@}.)&.>
>
> splitbs=: (0;(0 10#:10*".;._2]0 :0);(a.e.hexchars)+(2*a.='\')+3*a.e.'Uu')&;:
> 1.1 1.1 2.1 1.1
> 1.0 1.0 2.2 1.0
> 3.0 3.0 3.0 4.0
> 1.2 1.2 2.2 1.2
> 1.2 5.0 2.2 1.2
> 1.2 6.0 2.2 1.2
> 1.2 7.0 2.2 1.2
> 1.2 3.0 2.2 1.2
> )
> evalbs=: [:; [:subst2 [:subst6 splitbs^:(*@#)
> 'T_LBR T_RBR T_LCB T_RCB T_COM T_COL'=: ;:' [ ] { } , :'
>
> token=: 3 : 0
> (_1+I=: I+1){::T
> )
>
> dec_json=: 3 : 0
> T=: words (LF,' ') charsub y-.CR
> I=: 0
> > ". getVal token''
> )
> getVal=: 3 : 0
> select. y
> case. T_LBR do. getList token''
> case. T_LCB do. getHash token''
> case. do. '(',')',~ '<',5!:5<'t'[t=. jsonValue y
> end.
> )
> getList=: 3 : 0
> if. T_RBR -: y do. '(<0#<'''')' return. end.
> r=. ,getVal y
> while. -. T_RBR -: y=. token'' do.
> assert T_COM -: y
> r=. r, ',', getVal token''
> end.
> '(',')',~ '<', r
> )
> getHash=: 3 : 0
> if. T_RCB -: y do. '(<0#0 0$<'''')' return. end.
> r=. getPair y
> while. -. T_RCB -: y=. token'' do.
> assert T_COM -: y
> r=. r, ',.', getPair token''
> end.
> '(',')',~ '<', r
> )
> getPair=: 3 : 0
> h=. getVal y
> assert T_COL -: token''
> '(',')',~ h, ',:', getVal token''
> )
> gethash_json=: 4 : 0
> assert. 2=$$y
> assert. 32=3!:0 y
> x=. ,&.> boxopen x
> assert. 1=$x
> if. ({.x) e. {.y do.
> ({:y){~({.y) i. {.x
> else.
> _1
> end.
> )
> emptyhash_json=: 3 : 0
> (2=$$y) *. (0={.$y) *. 32=3!:0 y
> )
> emptyarray_json=: 3 : 0
> (2>$$y) *. (0=#y) *. 32=3!:0 y
> )
> dec_json_z_=: dec_json_json_
> enc_json_z_=: enc_json_json_
> gethash_json_z_=: gethash_json_json_
> emptyhash_json_z_=: emptyhash_json_json_
> emptyarray_json_z_=: emptyarray_json_json_
--
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
