Hi
Thanks Bill. And sorry for my delayed reply. BTW, I am cross-posting this
to the J programming mailing list, for the people who are interested in the
follow-up.
I've ran your version of json against the standard package version and the
standard pure python json module. Your version is faster but takes larger
memory, but the result is not exactly the same(more on this at the end).
The standard version:
bm=. 1!:1 <'Bookmarks'
ts=: 6!:2,7!:2@]
ts 'dec_json bm'
66.3499 5.36879e8
The new version you provided:
ts 'dec_json bm'
34.6758 1.36445e9
The standard pure python version:
0.4986 75272192
In J, the standard json and the new version you provided resulted in
different output.
newresult -: oldresult
0
Yet, the enc_json'd result seems to be the same.
newresult -:&enc_json standardresult
1
However, the resulted J representation is different. I just sampled one
difference -- I am not sure if it could help.
3!:3 (((<1 0) {>@(1 {>@((<1 0){ >@((<1 1) { >@((<1 1){]))))))
standardresult
e300000000000000
2000000000000000
0100000000000000
0000000000000000
2800000000000000
e300000000000000
2000000000000000
0100000000000000
0100000000000000
0100000000000000
3000000000000000
e300000000000000
2000000000000000
0c00000000000000
0200000000000000
0200000000000000
0600000000000000
9000000000000000
c800000000000000
f800000000000000
2801000000000000
7001000000000000
a001000000000000
d001000000000000
1002000000000000
4002000000000000
9002000000000000
c002000000000000
f002000000000000
e300000000000000
0200000000000000
0a00000000000000
0100000000000000
0a00000000000000
646174655f616464
6564000000000000
e300000000000000
0200000000000000
0200000000000000
0100000000000000
0200000000000000
6964000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
6e616d6500000000
e300000000000000
0200000000000000
1800000000000000
0100000000000000
1800000000000000
73796e635f747261
6e73616374696f6e
5f76657273696f6e
0000000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
7479706500000000
e300000000000000
0200000000000000
0300000000000000
0100000000000000
0300000000000000
75726c0000000000
e300000000000000
0200000000000000
1100000000000000
0100000000000000
1100000000000000
3133303031353737
3430313837303032
3100000000000000
e300000000000000
0200000000000000
0500000000000000
0100000000000000
0500000000000000
3330363133000000
e300000000000000
0200000000000000
2200000000000000
0100000000000000
2200000000000000
ec9584eab8b02045
5350202d20416e64
726f696420eba788
ecbc93ec9d9820ec
95b1000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
3230343300000000
e300000000000000
0200000000000000
0300000000000000
0100000000000000
0300000000000000
75726c0000000000
e300000000000000
0200000000000000
7e00000000000000
0100000000000000
7e00000000000000
68747470733a2f2f
6d61726b65742e61
6e64726f69642e63
6f6d2f6465746169
6c733f69643d636f
6d2e686577697474
736f66742e626162
7926666561747572
653d746f702d7061
6964233f743d5732
3531624777734d53
77794c4449774e69
7769593239744c6d
686c64326c306448
4e765a6e5175596d
466965534a640000
3!:3 (((<1 0) {>@(1 {>@((<1 0){ >@((<1 1) { >@((<1 1){])))))) newresult
e300000000000000
2000000000000000
0100000000000000
0000000000000000
2800000000000000
e300000000000000
2000000000000000
0100000000000000
0000000000000000
2800000000000000
e300000000000000
2000000000000000
0c00000000000000
0200000000000000
0200000000000000
0600000000000000
9000000000000000
c800000000000000
f800000000000000
2801000000000000
7001000000000000
a001000000000000
d001000000000000
1002000000000000
4002000000000000
9002000000000000
c002000000000000
f002000000000000
e300000000000000
0200000000000000
0a00000000000000
0100000000000000
0a00000000000000
646174655f616464
6564000000000000
e300000000000000
0200000000000000
0200000000000000
0100000000000000
0200000000000000
6964000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
6e616d6500000000
e300000000000000
0200000000000000
1800000000000000
0100000000000000
1800000000000000
73796e635f747261
6e73616374696f6e
5f76657273696f6e
0000000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
7479706500000000
e300000000000000
0200000000000000
0300000000000000
0100000000000000
0300000000000000
75726c0000000000
e300000000000000
0200000000000000
1100000000000000
0100000000000000
1100000000000000
3133303031353737
3430313837303032
3100000000000000
e300000000000000
0200000000000000
0500000000000000
0100000000000000
0500000000000000
3330363133000000
e300000000000000
0200000000000000
2200000000000000
0100000000000000
2200000000000000
ec9584eab8b02045
5350202d20416e64
726f696420eba788
ecbc93ec9d9820ec
95b1000000000000
e300000000000000
0200000000000000
0400000000000000
0100000000000000
0400000000000000
3230343300000000
e300000000000000
0200000000000000
0300000000000000
0100000000000000
0300000000000000
75726c0000000000
e300000000000000
0200000000000000
7e00000000000000
0100000000000000
7e00000000000000
68747470733a2f2f
6d61726b65742e61
6e64726f69642e63
6f6d2f6465746169
6c733f69643d636f
6d2e686577697474
736f66742e626162
7926666561747572
653d746f702d7061
6964233f743d5732
3531624777734d53
77794c4449774e69
7769593239744c6d
686c64326c306448
4e765a6e5175596d
466965534a640000
On Tue, Mar 4, 2014 at 12:40 PM, bill lam <bbill....@gmail.com> wrote:
> June,
>
> I changed json decoder to avoid intermediate box contenation, the
> trade-off might be a much larger space requirement. Please test
> its performance, the space requirement, and will the result be
> correct.
>
> Thanks.
>
> --
> regards,
> ====================================================
> GPG key 1024D/4434BAB3 2008-08-24
> gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
> gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3
>
coclass 'json'
enc_json=: 3 : 0
select. t=. 3!:0 y
case. 2;131072 do.
if. y-:'json_true' do. 'true'
elseif. y-:'json_false' do. 'false'
elseif. y-:'json_null' do. 'null'
elseif. do.
'"', '"',~ jsonesc utf8^:(131072=t) ,y
end.
case. 1;4;8 do.
":!.17 {.,y
case. 32 do.
if. 2>$$y do.
if. 0=#y do.
'[]'
else.
s=. '['
for_v. y do.
s=. s , ',',~ enc_json >v
end.
']',~ }:s
end.
elseif. 2=$$y do.
assert. 0 2 e.~ {.$y
if. 0=#y do.
'{}'
else.
s=. '{'
for_i. i.{:$y do.
s=. s , ':',~ enc_json (<0,i){::y
s=. s , ',',~ enc_json (<1,i){::y
end.
'}',~ }:s
end.
elseif. do. 13!:8[3
end.
case. do. 13!:8[3
end.
)
JSONESC0=: LF, CR, TAB, FF, (8{a.), '\/"'
JSONESC1=: 'nrtfb\/"'
JSONASC=: JSONESC0, 32}.127{.a.
jsonesc=: 3 : 0
txt=. y
msk=. txt e. JSONESC0
if. 1 e. msk do.
ndx=. , ((I. msk) + i. +/ msk) +/ 0 1
new=. , '\',.JSONESC1 {~ JSONESC0 i. msk#txt
txt=. new ndx } (1 + msk) # txt
end.
msk=. -. txt e. JSONASC
if. 1 e. msk do.
new=. 'u',"1 '0123456789abcdef' {~ 16 16 16 16 #: a. i. msk # txt
ndx=., ((I. msk) + 5 * i. +/ msk) +/ i. 6
txt=. (, '\',("1) new) ndx } (1 + msk * 5) # txt
end.
txt
)
classes=. 3<. '"\[],{}:' (#@[ |&>: i.) a.
classes=. 0 (I.a.e.' ',CRLF,TAB)} (]+4*0=])classes
words=: (0;(0 10#:10*".;._2]0 :0);classes)&;:
0.0 1.1 2.1 3.1 4.1
1.0 5.0 6.0 1.0 1.0
4.0 4.0 4.0 4.0 4.0
0.3 1.2 2.2 3.2 4.2
0.3 1.2 2.0 3.2 4.0
0.3 1.2 2.2 3.2 4.2
1.0 1.0 1.0 1.0 1.0
)
jsonValue=: 3 : 0
if. 0=#y do. <y
elseif. 'true'-:y do. 'json_true'
elseif. 'false'-:y do. 'json_false'
elseif. 'null'-:y do. 'json_null'
elseif. '"'={. y do. evalbs }.}:y
elseif. '0123456789.+-' -.@e.~ {. y do. evalbs y
elseif. do. {. 0". 'Ee-_' charsub y
end.
)
subst2=: (_2<\'\"\\\/\b\f\n\r\t')&(i.{(34 92 47 8 12 10 13 9{&.><a.),])"0
hexchars=. 'ABCDEF0123456789abcdef'
dfh=: 16 #. 16 | _6 + hexchars i. ]
subst6=: (8 u:[:u:@dfh 2}.]) ::]^:(6&=@#*.('\'={.)*.'uU'e.~{.@}.)&.>
splitbs=: (0;(0 10#:10*".;._2]0 :0);(a.e.hexchars)+(2*a.='\')+3*a.e.'Uu')&;:
1.1 1.1 2.1 1.1
1.0 1.0 2.2 1.0
3.0 3.0 3.0 4.0
1.2 1.2 2.2 1.2
1.2 5.0 2.2 1.2
1.2 6.0 2.2 1.2
1.2 7.0 2.2 1.2
1.2 3.0 2.2 1.2
)
evalbs=: [:; [:subst2 [:subst6 splitbs^:(*@#)
'T_LBR T_RBR T_LCB T_RCB T_COM T_COL'=: ;:' [ ] { } , :'
token=: 3 : 0
(_1+I=: I+1){::T
)
dec_json=: 3 : 0
T=: words (LF,' ') charsub y-.CR
I=: 0
> ". getVal token''
)
getVal=: 3 : 0
select. y
case. T_LBR do. getList token''
case. T_LCB do. getHash token''
case. do. '(',')',~ '<',5!:5<'t'[t=. jsonValue y
end.
)
getList=: 3 : 0
if. T_RBR -: y do. '(<0#<'''')' return. end.
r=. ,getVal y
while. -. T_RBR -: y=. token'' do.
assert T_COM -: y
r=. r, ',', getVal token''
end.
'(',')',~ '<', r
)
getHash=: 3 : 0
if. T_RCB -: y do. '(<0#0 0$<'''')' return. end.
r=. getPair y
while. -. T_RCB -: y=. token'' do.
assert T_COM -: y
r=. r, ',.', getPair token''
end.
'(',')',~ '<', r
)
getPair=: 3 : 0
h=. getVal y
assert T_COL -: token''
'(',')',~ h, ',:', getVal token''
)
gethash_json=: 4 : 0
assert. 2=$$y
assert. 32=3!:0 y
x=. ,&.> boxopen x
assert. 1=$x
if. ({.x) e. {.y do.
({:y){~({.y) i. {.x
else.
_1
end.
)
emptyhash_json=: 3 : 0
(2=$$y) *. (0={.$y) *. 32=3!:0 y
)
emptyarray_json=: 3 : 0
(2>$$y) *. (0=#y) *. 32=3!:0 y
)
dec_json_z_=: dec_json_json_
enc_json_z_=: enc_json_json_
gethash_json_z_=: gethash_json_json_
emptyhash_json_z_=: emptyhash_json_json_
emptyarray_json_z_=: emptyarray_json_json_
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
