Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

And this is *exactly* why I brought that up; had a feeling I might have
done something fishy. :)

It does throw; shouldn't be too hard to modify mine to pack segment by
segment. Thanks!




On Tue, Jul 18, 2017 at 12:53 PM, Kenton Varda 
wrote:

> On Tue, Jul 18, 2017 at 9:00 AM, Julián Díaz  wrote:
>
>> Appreciate the endorsement!
>>
>> I did in fact borrow some of that test data directly, though there's an
>> interesting divergence in the TypeScript version of the packing algorithm
>> so I wound up editing segmented-packed by hand to match:
>> https://github.com/jdiaz5513/capnp-ts/pull/10.
>>
>
> Hmm. I suspect this is because the C++ implementation packs each segment
> separately, and so the two runs of zeros came before and after a segment
> boundary. Unfortunately, assuming my suspicion is correct, then I suspect
> the C++ implementation will not be able to unpack your version. This is
> because it unpacks each segment into a separate array, but for speed
> reasons the core unpacking loop targets a single array at a time and has no
> way to pass along its state to the next call. I suppose this needs to be
> documented.
>
> You can check if this is the case by using `capnp decode -p` on your
> version of the data. I suspect it will throw an error.
>
> If it doesn't throw an error, then I'm confused.
>
> -Kenton
>
>
>> Still making slow but steady progress writing tests and finding broken
>> stuff. I'll publish to npm once the serialization part isn't so... broken.
>>
>> - Julián
>>
>> On Monday, July 17, 2017 at 8:36:36 PM UTC-4, Kenton Varda wrote:
>>>
>>> So I looked at this today and I'm pretty impressed! Code looks clean and
>>> seems to be following best practices. I made some notes on the issue
>>> tracker, as you probably saw, but generally looks pretty good. I'm pretty
>>> excited to start using this -- and I really want to migrate Sandstorm to
>>> TypeScript.
>>>
>>> One question: Have you written tests using the test data in the capnp
>>> repo?
>>>
>>> https://github.com/capnproto/capnproto/tree/master/c++/src/c
>>> apnp/testdata
>>>
>>> This would help check for any misreads of the spec.
>>>
>>> -Kenton
>>>
>>> On Thu, Jun 8, 2017 at 7:01 PM, Julián Díaz  wrote:
>>>
 For those itching to get on the bleeding edge, I've got a working
 schema compiler now!

 Serialization seems to be working okay with some unimplemented edges
 here and there. Perhaps not surprisingly, I'm already seeing places where
 this can outperform JSON.parse, so that's a major win!

 I almost had compile-to-JS support working as well, but the TypeScript
 compiler is refusing to play nice with me right now. (See:
 https://github.com/jdiaz5513/capnp-ts/issues/5)

 If anyone is really interested in using this stuff *today* please
 reach out to me so I can better understand what you need and perhaps
 rearrange how I implement things. Otherwise, there's still lots to do
 before 1.0.0!

 PS: For the compiler nerds: the schema compiler actually uses the
 TypeScript compiler API directly to build an AST before printing it to a
 file.

 On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:
>
> Can't use proxies at all in TypeScript unless I set the target to ES6
> - right now I want to keep it compiling to ES5 so it's immediately useful
> for a wider range of people.
>
> It also seems like it's going to perform like crap:
> http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html
>
> I'll add it to the TODO, regardless; a separate ES6 build would be
> useful for many people. I could document it with the caveat that
> .get() will always be faster.
>
> On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>>
>> On Wed, May 10, 2017 at 11:17 PM, Mark Miller 
>> wrote:
>>
>>> https://kangax.github.io/compat-table/es6/
>>>
>>> It looks like proxies are supported everywhere.
>>>
>>
>> Unfortunately a lot of people still use old browsers.
>>
>> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>>
>> -Kenton
>>
>>
>>>
>>>
>>> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda 
>>> wrote:
>>>
 Sweet!

 Totally random comment from totally randomly opening a file and
 looking at it:

 I see lists are accessed via a method .get(n). Have you considered
 using proxies to allow array subscript [] syntax? I guess some 10-20% 
 of
 browsers still don't support proxies but that number will only go down.

 -Kenton

 On Tue, May 9, 2017 at 2:03 PM, Julián Díaz 
 wrote:

> I'm happy to report some real progress!
>
> https://github.com/jdiaz5513/capnp-ts
>
> Right now it's not very useful at all (I just barely have
> se

Re: [capnproto] Re: Reviving the JavaScript implementation

['Kenton Varda' via Cap'n Proto]

On Tue, Jul 18, 2017 at 9:00 AM, Julián Díaz  wrote:

> Appreciate the endorsement!
>
> I did in fact borrow some of that test data directly, though there's an
> interesting divergence in the TypeScript version of the packing algorithm
> so I wound up editing segmented-packed by hand to match:
> https://github.com/jdiaz5513/capnp-ts/pull/10.
>

Hmm. I suspect this is because the C++ implementation packs each segment
separately, and so the two runs of zeros came before and after a segment
boundary. Unfortunately, assuming my suspicion is correct, then I suspect
the C++ implementation will not be able to unpack your version. This is
because it unpacks each segment into a separate array, but for speed
reasons the core unpacking loop targets a single array at a time and has no
way to pass along its state to the next call. I suppose this needs to be
documented.

You can check if this is the case by using `capnp decode -p` on your
version of the data. I suspect it will throw an error.

If it doesn't throw an error, then I'm confused.

-Kenton


> Still making slow but steady progress writing tests and finding broken
> stuff. I'll publish to npm once the serialization part isn't so... broken.
>
> - Julián
>
> On Monday, July 17, 2017 at 8:36:36 PM UTC-4, Kenton Varda wrote:
>>
>> So I looked at this today and I'm pretty impressed! Code looks clean and
>> seems to be following best practices. I made some notes on the issue
>> tracker, as you probably saw, but generally looks pretty good. I'm pretty
>> excited to start using this -- and I really want to migrate Sandstorm to
>> TypeScript.
>>
>> One question: Have you written tests using the test data in the capnp
>> repo?
>>
>> https://github.com/capnproto/capnproto/tree/master/c++/src/capnp/testdata
>>
>> This would help check for any misreads of the spec.
>>
>> -Kenton
>>
>> On Thu, Jun 8, 2017 at 7:01 PM, Julián Díaz  wrote:
>>
>>> For those itching to get on the bleeding edge, I've got a working schema
>>> compiler now!
>>>
>>> Serialization seems to be working okay with some unimplemented edges
>>> here and there. Perhaps not surprisingly, I'm already seeing places where
>>> this can outperform JSON.parse, so that's a major win!
>>>
>>> I almost had compile-to-JS support working as well, but the TypeScript
>>> compiler is refusing to play nice with me right now. (See:
>>> https://github.com/jdiaz5513/capnp-ts/issues/5)
>>>
>>> If anyone is really interested in using this stuff *today* please reach
>>> out to me so I can better understand what you need and perhaps rearrange
>>> how I implement things. Otherwise, there's still lots to do before 1.0.0!
>>>
>>> PS: For the compiler nerds: the schema compiler actually uses the
>>> TypeScript compiler API directly to build an AST before printing it to a
>>> file.
>>>
>>> On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:

 Can't use proxies at all in TypeScript unless I set the target to ES6 -
 right now I want to keep it compiling to ES5 so it's immediately useful for
 a wider range of people.

 It also seems like it's going to perform like crap:
 http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html

 I'll add it to the TODO, regardless; a separate ES6 build would be
 useful for many people. I could document it with the caveat that .get()
 will always be faster.

 On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>
> On Wed, May 10, 2017 at 11:17 PM, Mark Miller 
> wrote:
>
>> https://kangax.github.io/compat-table/es6/
>>
>> It looks like proxies are supported everywhere.
>>
>
> Unfortunately a lot of people still use old browsers.
>
> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>
> -Kenton
>
>
>>
>>
>> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda 
>> wrote:
>>
>>> Sweet!
>>>
>>> Totally random comment from totally randomly opening a file and
>>> looking at it:
>>>
>>> I see lists are accessed via a method .get(n). Have you considered
>>> using proxies to allow array subscript [] syntax? I guess some 10-20% of
>>> browsers still don't support proxies but that number will only go down.
>>>
>>> -Kenton
>>>
>>> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz 
>>> wrote:
>>>
 I'm happy to report some real progress!

 https://github.com/jdiaz5513/capnp-ts

 Right now it's not very useful at all (I just barely have
 serialization working) but it's a solid starting point to wrap up the
 serialization API. The peanut gallery can start poking around to see 
 how I
 organized things – it does depart slightly from the reference
 implementation but I'm still aiming to make an external API that's very
 similar to the C++ one.

 Once the Struct/List classes are comple

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

Appreciate the endorsement!

I did in fact borrow some of that test data directly, though there's an 
interesting divergence in the TypeScript version of the packing algorithm 
so I wound up editing segmented-packed by hand to 
match: https://github.com/jdiaz5513/capnp-ts/pull/10.

Still making slow but steady progress writing tests and finding broken 
stuff. I'll publish to npm once the serialization part isn't so... broken.

- Julián

On Monday, July 17, 2017 at 8:36:36 PM UTC-4, Kenton Varda wrote:
>
> So I looked at this today and I'm pretty impressed! Code looks clean and 
> seems to be following best practices. I made some notes on the issue 
> tracker, as you probably saw, but generally looks pretty good. I'm pretty 
> excited to start using this -- and I really want to migrate Sandstorm to 
> TypeScript.
>
> One question: Have you written tests using the test data in the capnp repo?
>
> https://github.com/capnproto/capnproto/tree/master/c++/src/capnp/testdata
>
> This would help check for any misreads of the spec.
>
> -Kenton
>
> On Thu, Jun 8, 2017 at 7:01 PM, Julián Díaz  > wrote:
>
>> For those itching to get on the bleeding edge, I've got a working schema 
>> compiler now!
>>
>> Serialization seems to be working okay with some unimplemented edges here 
>> and there. Perhaps not surprisingly, I'm already seeing places where this 
>> can outperform JSON.parse, so that's a major win!
>>
>> I almost had compile-to-JS support working as well, but the TypeScript 
>> compiler is refusing to play nice with me right now. (See: 
>> https://github.com/jdiaz5513/capnp-ts/issues/5)
>>
>> If anyone is really interested in using this stuff *today* please reach 
>> out to me so I can better understand what you need and perhaps rearrange 
>> how I implement things. Otherwise, there's still lots to do before 1.0.0!
>>
>> PS: For the compiler nerds: the schema compiler actually uses the 
>> TypeScript compiler API directly to build an AST before printing it to a 
>> file.
>>
>> On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:
>>>
>>> Can't use proxies at all in TypeScript unless I set the target to ES6 - 
>>> right now I want to keep it compiling to ES5 so it's immediately useful for 
>>> a wider range of people.
>>>
>>> It also seems like it's going to perform like crap: 
>>> http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html
>>>
>>> I'll add it to the TODO, regardless; a separate ES6 build would be 
>>> useful for many people. I could document it with the caveat that .get() 
>>> will always be faster.
>>>
>>> On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:

 On Wed, May 10, 2017 at 11:17 PM, Mark Miller  wrote:

> https://kangax.github.io/compat-table/es6/
>
> It looks like proxies are supported everywhere.
>

 Unfortunately a lot of people still use old browsers.

 http://caniuse.com/#feat=proxy -- click on the "usage relative" box.

 -Kenton
  

>
>
> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda  
> wrote:
>
>> Sweet!
>>
>> Totally random comment from totally randomly opening a file and 
>> looking at it:
>>
>> I see lists are accessed via a method .get(n). Have you considered 
>> using proxies to allow array subscript [] syntax? I guess some 10-20% of 
>> browsers still don't support proxies but that number will only go down.
>>
>> -Kenton
>>
>> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  
>> wrote:
>>
>>> I'm happy to report some real progress!
>>>
>>> https://github.com/jdiaz5513/capnp-ts
>>>
>>> Right now it's not very useful at all (I just barely have 
>>> serialization working) but it's a solid starting point to wrap up the 
>>> serialization API. The peanut gallery can start poking around to see 
>>> how I 
>>> organized things – it does depart slightly from the reference 
>>> implementation but I'm still aiming to make an external API that's very 
>>> similar to the C++ one.
>>>
>>> Once the Struct/List classes are complete I'll move on to the schema 
>>> compiler, which looks like it'll be a cinch. Hoping I can keep up the 
>>> steady progress from here.
>>>
>>> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:

 Quoting Kenton Varda (2017-04-09 18:35:48) 

 >1) libcapnp and libkj together add up to some 730k of code 
 (text 
 >segment) these days. Unless emscripten builds are 
 significantly 
 >smaller, that's probably too big. 

 Hard to know without trying it, but it may well be the case that 
 wasm 
 builds will be smaller. The VM seems to be designed for small code 
 size 
 (sensibly, given its target use case). This obviously doesn't apply 
 to 
 the asm.js output. 

Re: [capnproto] Re: Reviving the JavaScript implementation

['Kenton Varda' via Cap'n Proto]

So I looked at this today and I'm pretty impressed! Code looks clean and
seems to be following best practices. I made some notes on the issue
tracker, as you probably saw, but generally looks pretty good. I'm pretty
excited to start using this -- and I really want to migrate Sandstorm to
TypeScript.

One question: Have you written tests using the test data in the capnp repo?

https://github.com/capnproto/capnproto/tree/master/c++/src/capnp/testdata

This would help check for any misreads of the spec.

-Kenton

On Thu, Jun 8, 2017 at 7:01 PM, Julián Díaz  wrote:

> For those itching to get on the bleeding edge, I've got a working schema
> compiler now!
>
> Serialization seems to be working okay with some unimplemented edges here
> and there. Perhaps not surprisingly, I'm already seeing places where this
> can outperform JSON.parse, so that's a major win!
>
> I almost had compile-to-JS support working as well, but the TypeScript
> compiler is refusing to play nice with me right now. (See:
> https://github.com/jdiaz5513/capnp-ts/issues/5)
>
> If anyone is really interested in using this stuff *today* please reach
> out to me so I can better understand what you need and perhaps rearrange
> how I implement things. Otherwise, there's still lots to do before 1.0.0!
>
> PS: For the compiler nerds: the schema compiler actually uses the
> TypeScript compiler API directly to build an AST before printing it to a
> file.
>
> On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:
>>
>> Can't use proxies at all in TypeScript unless I set the target to ES6 -
>> right now I want to keep it compiling to ES5 so it's immediately useful for
>> a wider range of people.
>>
>> It also seems like it's going to perform like crap:
>> http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html
>>
>> I'll add it to the TODO, regardless; a separate ES6 build would be useful
>> for many people. I could document it with the caveat that .get() will
>> always be faster.
>>
>> On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>>>
>>> On Wed, May 10, 2017 at 11:17 PM, Mark Miller  wrote:
>>>
 https://kangax.github.io/compat-table/es6/

 It looks like proxies are supported everywhere.

>>>
>>> Unfortunately a lot of people still use old browsers.
>>>
>>> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>>>
>>> -Kenton
>>>
>>>


 On Wed, May 10, 2017 at 8:54 PM, Kenton Varda 
 wrote:

> Sweet!
>
> Totally random comment from totally randomly opening a file and
> looking at it:
>
> I see lists are accessed via a method .get(n). Have you considered
> using proxies to allow array subscript [] syntax? I guess some 10-20% of
> browsers still don't support proxies but that number will only go down.
>
> -Kenton
>
> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:
>
>> I'm happy to report some real progress!
>>
>> https://github.com/jdiaz5513/capnp-ts
>>
>> Right now it's not very useful at all (I just barely have
>> serialization working) but it's a solid starting point to wrap up the
>> serialization API. The peanut gallery can start poking around to see how 
>> I
>> organized things – it does depart slightly from the reference
>> implementation but I'm still aiming to make an external API that's very
>> similar to the C++ one.
>>
>> Once the Struct/List classes are complete I'll move on to the schema
>> compiler, which looks like it'll be a cinch. Hoping I can keep up the
>> steady progress from here.
>>
>> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>>
>>> Quoting Kenton Varda (2017-04-09 18:35:48)
>>>
>>> >1) libcapnp and libkj together add up to some 730k of code
>>> (text
>>> >segment) these days. Unless emscripten builds are significantly
>>> >smaller, that's probably too big.
>>>
>>> Hard to know without trying it, but it may well be the case that
>>> wasm
>>> builds will be smaller. The VM seems to be designed for small code
>>> size
>>> (sensibly, given its target use case). This obviously doesn't apply
>>> to
>>> the asm.js output.
>>>
>>> That said, I agree having a pure JS implementation is preferable.
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to capnproto+...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to capnproto+...@googlegroups.com.
> Visit this group at https://groups.google.c

Re: [capnproto] Re: Reviving the JavaScript implementation

['Kenton Varda' via Cap'n Proto]

Sweet!

On Thu, Jun 8, 2017 at 7:01 PM, Julián Díaz  wrote:

> If anyone is really interested in using this stuff *today* please reach
> out to me so I can better understand what you need and perhaps rearrange
> how I implement things. Otherwise, there's still lots to do before 1.0.0!
>

I can wait until its more ready, but once it is I'll be eager to try it out
in Sandstorm.

It would be amazing if we could retire node-capnp which Sandstorm uses
currently. It leaks a lot of memory due to inability to GC through C++
objects and the V8 C++ API being hard to use correctly in general. Of
course, we'll need RPC before this can happen.

PS: For the compiler nerds: the schema compiler actually uses the
> TypeScript compiler API directly to build an AST before printing it to a
> file.
>

Nice!

-Kenton


>
> On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:
>>
>> Can't use proxies at all in TypeScript unless I set the target to ES6 -
>> right now I want to keep it compiling to ES5 so it's immediately useful for
>> a wider range of people.
>>
>> It also seems like it's going to perform like crap:
>> http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html
>>
>> I'll add it to the TODO, regardless; a separate ES6 build would be useful
>> for many people. I could document it with the caveat that .get() will
>> always be faster.
>>
>> On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>>>
>>> On Wed, May 10, 2017 at 11:17 PM, Mark Miller  wrote:
>>>
 https://kangax.github.io/compat-table/es6/

 It looks like proxies are supported everywhere.

>>>
>>> Unfortunately a lot of people still use old browsers.
>>>
>>> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>>>
>>> -Kenton
>>>
>>>


 On Wed, May 10, 2017 at 8:54 PM, Kenton Varda 
 wrote:

> Sweet!
>
> Totally random comment from totally randomly opening a file and
> looking at it:
>
> I see lists are accessed via a method .get(n). Have you considered
> using proxies to allow array subscript [] syntax? I guess some 10-20% of
> browsers still don't support proxies but that number will only go down.
>
> -Kenton
>
> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:
>
>> I'm happy to report some real progress!
>>
>> https://github.com/jdiaz5513/capnp-ts
>>
>> Right now it's not very useful at all (I just barely have
>> serialization working) but it's a solid starting point to wrap up the
>> serialization API. The peanut gallery can start poking around to see how 
>> I
>> organized things – it does depart slightly from the reference
>> implementation but I'm still aiming to make an external API that's very
>> similar to the C++ one.
>>
>> Once the Struct/List classes are complete I'll move on to the schema
>> compiler, which looks like it'll be a cinch. Hoping I can keep up the
>> steady progress from here.
>>
>> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>>
>>> Quoting Kenton Varda (2017-04-09 18:35:48)
>>>
>>> >1) libcapnp and libkj together add up to some 730k of code
>>> (text
>>> >segment) these days. Unless emscripten builds are significantly
>>> >smaller, that's probably too big.
>>>
>>> Hard to know without trying it, but it may well be the case that
>>> wasm
>>> builds will be smaller. The VM seems to be designed for small code
>>> size
>>> (sensibly, given its target use case). This obviously doesn't apply
>>> to
>>> the asm.js output.
>>>
>>> That said, I agree having a pure JS implementation is preferable.
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to capnproto+...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to capnproto+...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>



 --
   Cheers,
   --MarkM

 --
 You received this message because you are subscribed to the Google
 Groups "Cap'n Proto" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to capnproto+...@googlegroups.com.
 Visit this group at https://groups.google.com/group/capnproto.

>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegro

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

For those itching to get on the bleeding edge, I've got a working schema 
compiler now!

Serialization seems to be working okay with some unimplemented edges here 
and there. Perhaps not surprisingly, I'm already seeing places where this 
can outperform JSON.parse, so that's a major win!

I almost had compile-to-JS support working as well, but the TypeScript 
compiler is refusing to play nice with me right now. 
(See: https://github.com/jdiaz5513/capnp-ts/issues/5)

If anyone is really interested in using this stuff *today* please reach out 
to me so I can better understand what you need and perhaps rearrange how I 
implement things. Otherwise, there's still lots to do before 1.0.0!

PS: For the compiler nerds: the schema compiler actually uses the 
TypeScript compiler API directly to build an AST before printing it to a 
file.

On Thursday, May 11, 2017 at 12:58:03 PM UTC-4, Julián Díaz wrote:
>
> Can't use proxies at all in TypeScript unless I set the target to ES6 - 
> right now I want to keep it compiling to ES5 so it's immediately useful for 
> a wider range of people.
>
> It also seems like it's going to perform like crap: 
> http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html
>
> I'll add it to the TODO, regardless; a separate ES6 build would be useful 
> for many people. I could document it with the caveat that .get() will 
> always be faster.
>
> On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>>
>> On Wed, May 10, 2017 at 11:17 PM, Mark Miller  wrote:
>>
>>> https://kangax.github.io/compat-table/es6/
>>>
>>> It looks like proxies are supported everywhere.
>>>
>>
>> Unfortunately a lot of people still use old browsers.
>>
>> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>>
>> -Kenton
>>  
>>
>>>
>>>
>>> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda  
>>> wrote:
>>>
 Sweet!

 Totally random comment from totally randomly opening a file and looking 
 at it:

 I see lists are accessed via a method .get(n). Have you considered 
 using proxies to allow array subscript [] syntax? I guess some 10-20% of 
 browsers still don't support proxies but that number will only go down.

 -Kenton

 On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:

> I'm happy to report some real progress!
>
> https://github.com/jdiaz5513/capnp-ts
>
> Right now it's not very useful at all (I just barely have 
> serialization working) but it's a solid starting point to wrap up the 
> serialization API. The peanut gallery can start poking around to see how 
> I 
> organized things – it does depart slightly from the reference 
> implementation but I'm still aiming to make an external API that's very 
> similar to the C++ one.
>
> Once the Struct/List classes are complete I'll move on to the schema 
> compiler, which looks like it'll be a cinch. Hoping I can keep up the 
> steady progress from here.
>
> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>
>> Quoting Kenton Varda (2017-04-09 18:35:48) 
>>
>> >1) libcapnp and libkj together add up to some 730k of code (text 
>> >segment) these days. Unless emscripten builds are significantly 
>> >smaller, that's probably too big. 
>>
>> Hard to know without trying it, but it may well be the case that wasm 
>> builds will be smaller. The VM seems to be designed for small code 
>> size 
>> (sensibly, given its target use case). This obviously doesn't apply 
>> to 
>> the asm.js output. 
>>
>> That said, I agree having a pure JS implementation is preferable. 
>>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to capnproto+...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Cap'n Proto" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to capnproto+...@googlegroups.com.
 Visit this group at https://groups.google.com/group/capnproto.

>>>
>>>
>>>
>>> -- 
>>>   Cheers,
>>>   --MarkM
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to capnproto+...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/capnproto.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/ca

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

Can't use proxies at all in TypeScript unless I set the target to ES6 - 
right now I want to keep it compiling to ES5 so it's immediately useful for 
a wider range of people.

It also seems like it's going to perform like crap: 
http://thecodebarbarian.com/thoughts-on-es6-proxies-performance.html

I'll add it to the TODO, regardless; a separate ES6 build would be useful 
for many people. I could document it with the caveat that .get() will 
always be faster.

On Thursday, May 11, 2017 at 11:10:16 AM UTC-4, Kenton Varda wrote:
>
> On Wed, May 10, 2017 at 11:17 PM, Mark Miller  > wrote:
>
>> https://kangax.github.io/compat-table/es6/
>>
>> It looks like proxies are supported everywhere.
>>
>
> Unfortunately a lot of people still use old browsers.
>
> http://caniuse.com/#feat=proxy -- click on the "usage relative" box.
>
> -Kenton
>  
>
>>
>>
>> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda > > wrote:
>>
>>> Sweet!
>>>
>>> Totally random comment from totally randomly opening a file and looking 
>>> at it:
>>>
>>> I see lists are accessed via a method .get(n). Have you considered using 
>>> proxies to allow array subscript [] syntax? I guess some 10-20% of browsers 
>>> still don't support proxies but that number will only go down.
>>>
>>> -Kenton
>>>
>>> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz >> > wrote:
>>>
 I'm happy to report some real progress!

 https://github.com/jdiaz5513/capnp-ts

 Right now it's not very useful at all (I just barely have serialization 
 working) but it's a solid starting point to wrap up the serialization API. 
 The peanut gallery can start poking around to see how I organized things – 
 it does depart slightly from the reference implementation but I'm still 
 aiming to make an external API that's very similar to the C++ one.

 Once the Struct/List classes are complete I'll move on to the schema 
 compiler, which looks like it'll be a cinch. Hoping I can keep up the 
 steady progress from here.

 On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>
> Quoting Kenton Varda (2017-04-09 18:35:48) 
>
> >1) libcapnp and libkj together add up to some 730k of code (text 
> >segment) these days. Unless emscripten builds are significantly 
> >smaller, that's probably too big. 
>
> Hard to know without trying it, but it may well be the case that wasm 
> builds will be smaller. The VM seems to be designed for small code 
> size 
> (sensibly, given its target use case). This obviously doesn't apply to 
> the asm.js output. 
>
> That said, I agree having a pure JS implementation is preferable. 
>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "Cap'n Proto" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to capnproto+...@googlegroups.com .
 Visit this group at https://groups.google.com/group/capnproto.

>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to capnproto+...@googlegroups.com .
>>> Visit this group at https://groups.google.com/group/capnproto.
>>>
>>
>>
>>
>> -- 
>>   Cheers,
>>   --MarkM
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to capnproto+...@googlegroups.com .
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

On Wed, May 10, 2017 at 11:17 PM, Mark Miller  wrote:

> https://kangax.github.io/compat-table/es6/
>
> It looks like proxies are supported everywhere.
>

Unfortunately a lot of people still use old browsers.

http://caniuse.com/#feat=proxy -- click on the "usage relative" box.

-Kenton


>
>
> On Wed, May 10, 2017 at 8:54 PM, Kenton Varda  wrote:
>
>> Sweet!
>>
>> Totally random comment from totally randomly opening a file and looking
>> at it:
>>
>> I see lists are accessed via a method .get(n). Have you considered using
>> proxies to allow array subscript [] syntax? I guess some 10-20% of browsers
>> still don't support proxies but that number will only go down.
>>
>> -Kenton
>>
>> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:
>>
>>> I'm happy to report some real progress!
>>>
>>> https://github.com/jdiaz5513/capnp-ts
>>>
>>> Right now it's not very useful at all (I just barely have serialization
>>> working) but it's a solid starting point to wrap up the serialization API.
>>> The peanut gallery can start poking around to see how I organized things –
>>> it does depart slightly from the reference implementation but I'm still
>>> aiming to make an external API that's very similar to the C++ one.
>>>
>>> Once the Struct/List classes are complete I'll move on to the schema
>>> compiler, which looks like it'll be a cinch. Hoping I can keep up the
>>> steady progress from here.
>>>
>>> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:

 Quoting Kenton Varda (2017-04-09 18:35:48)

 >1) libcapnp and libkj together add up to some 730k of code (text
 >segment) these days. Unless emscripten builds are significantly
 >smaller, that's probably too big.

 Hard to know without trying it, but it may well be the case that wasm
 builds will be smaller. The VM seems to be designed for small code size
 (sensibly, given its target use case). This obviously doesn't apply to
 the asm.js output.

 That said, I agree having a pure JS implementation is preferable.

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to capnproto+unsubscr...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/capnproto.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+unsubscr...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>
>
> --
>   Cheers,
>   --MarkM
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Mark Miller]

https://kangax.github.io/compat-table/es6/

It looks like proxies are supported everywhere.


On Wed, May 10, 2017 at 8:54 PM, Kenton Varda  wrote:

> Sweet!
>
> Totally random comment from totally randomly opening a file and looking at
> it:
>
> I see lists are accessed via a method .get(n). Have you considered using
> proxies to allow array subscript [] syntax? I guess some 10-20% of browsers
> still don't support proxies but that number will only go down.
>
> -Kenton
>
> On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:
>
>> I'm happy to report some real progress!
>>
>> https://github.com/jdiaz5513/capnp-ts
>>
>> Right now it's not very useful at all (I just barely have serialization
>> working) but it's a solid starting point to wrap up the serialization API.
>> The peanut gallery can start poking around to see how I organized things –
>> it does depart slightly from the reference implementation but I'm still
>> aiming to make an external API that's very similar to the C++ one.
>>
>> Once the Struct/List classes are complete I'll move on to the schema
>> compiler, which looks like it'll be a cinch. Hoping I can keep up the
>> steady progress from here.
>>
>> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>>
>>> Quoting Kenton Varda (2017-04-09 18:35:48)
>>>
>>> >1) libcapnp and libkj together add up to some 730k of code (text
>>> >segment) these days. Unless emscripten builds are significantly
>>> >smaller, that's probably too big.
>>>
>>> Hard to know without trying it, but it may well be the case that wasm
>>> builds will be smaller. The VM seems to be designed for small code size
>>> (sensibly, given its target use case). This obviously doesn't apply to
>>> the asm.js output.
>>>
>>> That said, I agree having a pure JS implementation is preferable.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+unsubscr...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>



-- 
  Cheers,
  --MarkM

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

Sweet!

Totally random comment from totally randomly opening a file and looking at
it:

I see lists are accessed via a method .get(n). Have you considered using
proxies to allow array subscript [] syntax? I guess some 10-20% of browsers
still don't support proxies but that number will only go down.

-Kenton

On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:

> I'm happy to report some real progress!
>
> https://github.com/jdiaz5513/capnp-ts
>
> Right now it's not very useful at all (I just barely have serialization
> working) but it's a solid starting point to wrap up the serialization API.
> The peanut gallery can start poking around to see how I organized things –
> it does depart slightly from the reference implementation but I'm still
> aiming to make an external API that's very similar to the C++ one.
>
> Once the Struct/List classes are complete I'll move on to the schema
> compiler, which looks like it'll be a cinch. Hoping I can keep up the
> steady progress from here.
>
> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>
>> Quoting Kenton Varda (2017-04-09 18:35:48)
>>
>> >1) libcapnp and libkj together add up to some 730k of code (text
>> >segment) these days. Unless emscripten builds are significantly
>> >smaller, that's probably too big.
>>
>> Hard to know without trying it, but it may well be the case that wasm
>> builds will be smaller. The VM seems to be designed for small code size
>> (sensibly, given its target use case). This obviously doesn't apply to
>> the asm.js output.
>>
>> That said, I agree having a pure JS implementation is preferable.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Harris Hancock]

Just wanted to say this is awesome, and thanks for sharing it! I took it
for a test drive (all tests pass), and look forward to banging on it more
soon.

On Tue, May 9, 2017 at 2:03 PM, Julián Díaz  wrote:

> I'm happy to report some real progress!
>
> https://github.com/jdiaz5513/capnp-ts
>
> Right now it's not very useful at all (I just barely have serialization
> working) but it's a solid starting point to wrap up the serialization API.
> The peanut gallery can start poking around to see how I organized things –
> it does depart slightly from the reference implementation but I'm still
> aiming to make an external API that's very similar to the C++ one.
>
> Once the Struct/List classes are complete I'll move on to the schema
> compiler, which looks like it'll be a cinch. Hoping I can keep up the
> steady progress from here.
>
> On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>>
>> Quoting Kenton Varda (2017-04-09 18:35:48)
>>
>> >1) libcapnp and libkj together add up to some 730k of code (text
>> >segment) these days. Unless emscripten builds are significantly
>> >smaller, that's probably too big.
>>
>> Hard to know without trying it, but it may well be the case that wasm
>> builds will be smaller. The VM seems to be designed for small code size
>> (sensibly, given its target use case). This obviously doesn't apply to
>> the asm.js output.
>>
>> That said, I agree having a pure JS implementation is preferable.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

I'm happy to report some real progress!

https://github.com/jdiaz5513/capnp-ts

Right now it's not very useful at all (I just barely have serialization 
working) but it's a solid starting point to wrap up the serialization API. 
The peanut gallery can start poking around to see how I organized things – 
it does depart slightly from the reference implementation but I'm still 
aiming to make an external API that's very similar to the C++ one.

Once the Struct/List classes are complete I'll move on to the schema 
compiler, which looks like it'll be a cinch. Hoping I can keep up the 
steady progress from here.

On Monday, April 10, 2017 at 1:45:01 AM UTC-4, Ian Denhardt wrote:
>
> Quoting Kenton Varda (2017-04-09 18:35:48) 
>
> >1) libcapnp and libkj together add up to some 730k of code (text 
> >segment) these days. Unless emscripten builds are significantly 
> >smaller, that's probably too big. 
>
> Hard to know without trying it, but it may well be the case that wasm 
> builds will be smaller. The VM seems to be designed for small code size 
> (sensibly, given its target use case). This obviously doesn't apply to 
> the asm.js output. 
>
> That said, I agree having a pure JS implementation is preferable. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Ian Denhardt]

Quoting Kenton Varda (2017-04-09 18:35:48)

>1) libcapnp and libkj together add up to some 730k of code (text
>segment) these days. Unless emscripten builds are significantly
>smaller, that's probably too big.

Hard to know without trying it, but it may well be the case that wasm
builds will be smaller. The VM seems to be designed for small code size
(sensibly, given its target use case). This obviously doesn't apply to
the asm.js output.

That said, I agree having a pure JS implementation is preferable.

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.


signature.asc
Description: signature

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

On Sunday, April 9, 2017 at 7:22:26 PM UTC-7, Kenton Varda wrote:
>
> On Sun, Apr 9, 2017 at 7:04 PM, Wink Saville  > wrote:
>>
>> 2) The interface between JS and WASM would probably be very slow if every 
>>> accessor has to go through it. You'd probably end up wanting to translate 
>>> the whole capnp to a JSON object upfront, which of course defeats a lot of 
>>> the purpose.
>>>
>>
>> I'm hoping shared memory 
>> 
>>  
>> might be used, we could have serdes in wasm and access in JS, take with 
>> huge grain of salt :)
>>
>
> But in Cap'n Proto, there is no serdes. All of the logic happens at access 
> time.
>

Hmmm, I forgot, thinking of my old protobuf days. Anyway, one of the use 
cases is using is using messaging to communicate between wasm and js so if 
we can't share any implementation so be it.

 
>

> -Kenton
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

serial/deserializer

On Sunday, April 9, 2017 at 8:30:51 PM UTC-7, Mark Miller wrote:
>
> "serdes"?
>
>
> On Sun, Apr 9, 2017 at 10:22 PM, Kenton Varda  > wrote:
>
>> On Sun, Apr 9, 2017 at 7:04 PM, Wink Saville > > wrote:
>>>
>>> 2) The interface between JS and WASM would probably be very slow if 
 every accessor has to go through it. You'd probably end up wanting to 
 translate the whole capnp to a JSON object upfront, which of course 
 defeats 
 a lot of the purpose.

>>>
>>> I'm hoping shared memory 
>>> 
>>>  
>>> might be used, we could have serdes in wasm and access in JS, take with 
>>> huge grain of salt :)
>>>
>>
>> But in Cap'n Proto, there is no serdes. All of the logic happens at 
>> access time.
>>
>> -Kenton
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to capnproto+...@googlegroups.com .
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>
>
> -- 
>   Cheers,
>   --MarkM
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Mark Miller]

"serdes"?


On Sun, Apr 9, 2017 at 10:22 PM, Kenton Varda  wrote:

> On Sun, Apr 9, 2017 at 7:04 PM, Wink Saville  wrote:
>>
>> 2) The interface between JS and WASM would probably be very slow if every
>>> accessor has to go through it. You'd probably end up wanting to translate
>>> the whole capnp to a JSON object upfront, which of course defeats a lot of
>>> the purpose.
>>>
>>
>> I'm hoping shared memory
>> 
>> might be used, we could have serdes in wasm and access in JS, take with
>> huge grain of salt :)
>>
>
> But in Cap'n Proto, there is no serdes. All of the logic happens at access
> time.
>
> -Kenton
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>



-- 
  Cheers,
  --MarkM

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

On Sun, Apr 9, 2017 at 7:04 PM, Wink Saville  wrote:
>
> 2) The interface between JS and WASM would probably be very slow if every
>> accessor has to go through it. You'd probably end up wanting to translate
>> the whole capnp to a JSON object upfront, which of course defeats a lot of
>> the purpose.
>>
>
> I'm hoping shared memory
> 
> might be used, we could have serdes in wasm and access in JS, take with
> huge grain of salt :)
>

But in Cap'n Proto, there is no serdes. All of the logic happens at access
time.

-Kenton

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

On Sunday, April 9, 2017 at 3:36:11 PM UTC-7, Kenton Varda wrote:
>
> I don't think WASM is likely to be a good fit here, for two reasons:
>
> 1) libcapnp and libkj together add up to some 730k of code (text segment) 
> these days. Unless emscripten builds are significantly smaller, that's 
> probably too big. (If you stick to lite mode, it's still 401k, which is 
> still probably too big.)
>

For some use cases it might not matter.
 

>
> 2) The interface between JS and WASM would probably be very slow if every 
> accessor has to go through it. You'd probably end up wanting to translate 
> the whole capnp to a JSON object upfront, which of course defeats a lot of 
> the purpose.
>

I'm hoping shared memory 

 
might be used, we could have serdes in wasm and access in JS, take with 
huge grain of salt :)
 

>
> So I definitely think a pure-JS implementation is still desirable.
>

Yea, definitely wouldn't hurt.

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

I don't think WASM is likely to be a good fit here, for two reasons:

1) libcapnp and libkj together add up to some 730k of code (text segment)
these days. Unless emscripten builds are significantly smaller, that's
probably too big. (If you stick to lite mode, it's still 401k, which is
still probably too big.)

2) The interface between JS and WASM would probably be very slow if every
accessor has to go through it. You'd probably end up wanting to translate
the whole capnp to a JSON object upfront, which of course defeats a lot of
the purpose.

So I definitely think a pure-JS implementation is still desirable.

-Kenton

On Sat, Apr 8, 2017 at 1:48 PM, Julián Díaz  wrote:

> The serialization part is where I fell on my face; I was definitely
> following the C++ implementation a bit *too* closely.
>
> There's a huge amount of type polymorphism needed and that was tripping me
> up pretty hard as well; I'm going to instead actually take advantage of the
> fact that JS is a dynamic language and that should help cut down on the
> amount of code needed.
>
> WASM is definitely an appealing thought and probably the ideal way to use
> capnp in the browser, but the way I see it a pure JS implementation will
> continue to be appealing for as long as we have other non-desktop devices
> with JS but not WASM support (I'm looking at Android 4.x, especially -
> that's going to be around for a while).
>
> On Saturday, April 8, 2017 at 3:58:26 PM UTC-4, Ross Light wrote:
>>
>> I have had somebody use GopherJS to read data client side. It works, but
>> I can't vouch for how well.
>>
>> What parts are you having difficulty with representing in JS? Are you
>> still on serialization or the RPC part? I found that for Go I would look at
>> the C++ implementation to get the spirit of a feature, and then write it
>> very differently to fit in with Go.
>>
>> On Sat, Apr 8, 2017, 10:17 AM Wink Saville  wrote:
>>
>>> > Of course its very early days for wasm, but I'm following it and just
 > beginning to play around with it.

 Worth pointing out, there's also Emscripten, which has been around a
 while. So that could be used while wasm support filters into things.
>>>
>>>
>>> Yep, Emscripten is the path I was thinking, it now only output asmjs but
>>> also wasm .
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to capnproto+...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/capnproto.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

On Wed, Apr 5, 2017 at 9:01 PM,  wrote:

> Anything ever happen with this?
>

(Moderator's note: This message was actually sent four days ago, before the
recent activity on this thread, but got caught in the moderation queue.)

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[me]

Anything ever happen with this? 

On Tuesday, October 4, 2016 at 10:13:51 AM UTC-5, Julián Díaz wrote:
>
> Generated TypeScript it'll be, then! It'll be pretty trivial to 
> subsequently invoke the TypeScript compiler on the TypeScript generated 
> code to get "free" ES5 code for everyone else.
>
> I'm really digging how much of a great fit TypeScript's type system is for 
> this stuff (so far, at least). Will report back once I have something that 
> resembles a code generator.
>
> On Sunday, October 2, 2016 at 6:15:06 PM UTC-4, Kenton Varda wrote:
>>
>> For Javascript, I actually don't think runtime schema loading should be a 
>> goal. These days Javascript is essentially a compiled language anyway, with 
>> all the transpiling and minifying. And to load schemas at runtime in a 
>> browser, you'd presumably need to rewrite the parser in pure-Javascript. 
>> I'd like to avoid having multiple schema parser implementations as they'd 
>> be likely to have subtle incompatibilities that would be a pain for 
>> developers.
>>
>> So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty 
>> good plan. Then we can get auto-complete in our IDEs and all that cool 
>> stuff. :)
>>
>> -Kenton
>>
>> On Sun, Oct 2, 2016 at 10:31 AM,  
>> wrote:
>>
>>> After playing around a little and realizing how limited ES5 really is, 
>>> I'm open to writing this thing in 100% TypeScript. There's some caveats, 
>>> though - if the schema file is loaded dynamically it'll be hard 
>>> (impossible?) to add type safety without runtime type checks, and I can see 
>>> that getting expensive. TypeScript almost gets in the way in this case. 
>>> However, a schema compiler can be set up to emit either typescript or 
>>> vanilla ES5 code; *that* could work quite well! ArrayBuffers will 
>>> contain the raw data and property access/method calls will just reference 
>>> indices in the array - the only *super* annoying thing about it is 
>>> there's no native support for 64-bit words in JavaScript so everything has 
>>> to be broken up into hi/lo 32-bit words in a Uint32Array (yuck!).
>>>
>>> There are concerns about TypeScript being generally slower if you lean 
>>> heavily on the ES6 features (https://kpdecker.github.io/six-speed/), 
>>> but with proper profiling it shouldn't be hard to identify hotspots and 
>>> turn them into tightly optimized code that won't be modified much by the 
>>> transpiler. The speed problems come in when you heavily depend on things 
>>> like arrow functions and the transpiler ends up adding extra assignments 
>>> and variables you didn't expect. For what it's worth, I expect most 
>>> performance concerns won't have anything to do with using ES6.
>>>
>>> I agree that the transport layer needs to be pluggable and that's the 
>>> design I'm going to go for right from the start. Realistically I'm going to 
>>> have a hard time designing the whole thing without a sample app to 
>>> reference so *some *form of transport needs to be supported "out of the 
>>> box". WebSockets feels like an easy initial target for this. Supporting 
>>> window.postMessage should also be easy, and since you can pass ArrayBuffers 
>>> with zero-copy (I think??) it'll be crazy fast!
>>>
>>> Speaking of sample app; I plan on writing a TodoMVC type thing in 
>>> separate repos as a reference implementation (a node server and a browser 
>>> client w/ React). Any thoughts on this? Should I instead try to go for 
>>> something that better showcases the power of capability-based access 
>>> control? A todo list where you can securely share edit/view access with 
>>> others could be a fun demo. I want to keep the UI complexity low and really 
>>> focus on the RPC stuff, of course.
>>>
>>> Could also use tips on which order to implement things since this is a 
>>> bit of a *daunting* spec to implement. I'm currently having fun with 
>>> the unpacking algorithm. :)
>>>
>>> On Saturday, October 1, 2016 at 7:56:02 PM UTC-4, Nathan Hourt wrote:

 As far as the appetite front goes, I've rather wanted a pure javascript 
 implementation of capnp on several occasions now, so I'd be quite happy to 
 see this happen.

 I think websockets support is a must, but ideally the library would be 
 sufficiently portable that I would be able to use the library, unmodified, 
 on any particular transport layer I want, with nothing more than a little 
 glue code between interfaces. The library could then provide some built-in 
 adaptors for common transport layers (like Node sockets, websockets, etc) 
 which would allow those transports to be used with no customization, and 
 also documents how to write adaptors so it's easy to see how to use other 
 transports instead (exactly like what we see with the C++ implementation 
 with AsyncIoStream: I can use any transport I want simply by implementing 
 the interface).

 On Friday, September 30, 2016 at 7:08:01 PM UTC-5, Julián Díaz wrote:
>

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

The serialization part is where I fell on my face; I was definitely 
following the C++ implementation a bit *too* closely.

There's a huge amount of type polymorphism needed and that was tripping me 
up pretty hard as well; I'm going to instead actually take advantage of the 
fact that JS is a dynamic language and that should help cut down on the 
amount of code needed.

WASM is definitely an appealing thought and probably the ideal way to use 
capnp in the browser, but the way I see it a pure JS implementation will 
continue to be appealing for as long as we have other non-desktop devices 
with JS but not WASM support (I'm looking at Android 4.x, especially - 
that's going to be around for a while).

On Saturday, April 8, 2017 at 3:58:26 PM UTC-4, Ross Light wrote:
>
> I have had somebody use GopherJS to read data client side. It works, but I 
> can't vouch for how well.
>
> What parts are you having difficulty with representing in JS? Are you 
> still on serialization or the RPC part? I found that for Go I would look at 
> the C++ implementation to get the spirit of a feature, and then write it 
> very differently to fit in with Go.
>
> On Sat, Apr 8, 2017, 10:17 AM Wink Saville  > wrote:
>
>> > Of course its very early days for wasm, but I'm following it and just 
>>> > beginning to play around with it. 
>>>
>>> Worth pointing out, there's also Emscripten, which has been around a 
>>> while. So that could be used while wasm support filters into things. 
>>
>>
>> Yep, Emscripten is the path I was thinking, it now only output asmjs but
>> also wasm .
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to capnproto+...@googlegroups.com .
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Ross Light]

I have had somebody use GopherJS to read data client side. It works, but I
can't vouch for how well.

What parts are you having difficulty with representing in JS? Are you still
on serialization or the RPC part? I found that for Go I would look at the
C++ implementation to get the spirit of a feature, and then write it very
differently to fit in with Go.

On Sat, Apr 8, 2017, 10:17 AM Wink Saville  wrote:

> > Of course its very early days for wasm, but I'm following it and just
>> > beginning to play around with it.
>>
>> Worth pointing out, there's also Emscripten, which has been around a
>> while. So that could be used while wasm support filters into things.
>
>
> Yep, Emscripten is the path I was thinking, it now only output asmjs but
> also wasm .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> Visit this group at https://groups.google.com/group/capnproto.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

>
> > Of course its very early days for wasm, but I'm following it and just 
> > beginning to play around with it. 
>
> Worth pointing out, there's also Emscripten, which has been around a 
> while. So that could be used while wasm support filters into things. 


Yep, Emscripten is the path I was thinking, it now only output asmjs but
also wasm .

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

Re: [capnproto] Re: Reviving the JavaScript implementation

[Ian Denhardt]

Quoting Wink Saville (2017-04-07 23:31:43)

> Of course its very early days for wasm, but I'm following it and just
> beginning to play around with it.

Worth pointing out, there's also Emscripten, which has been around a
while. So that could be used while wasm support filters into things.

Has anyone tried running capnproto through Emscripten? Might be nice to
try plumbing the rust implementation through it as well.

Along similar lines, there's a mature Go -> Javascript compiler
(gopherjs), and a pure-go implementation of capnproto. This might "just
work," and if not I strongly suspect the fixes will be easy.

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.


signature.asc
Description: signature

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

:(

I'm new to this Javascript/Typescript/Web programming, but I find comfort 
in the quasi strict types of Typescript. I'm surprised Typescript is 
getting in the way, especially since Typescript is a "proper" superset of 
Javascript. I can certainly see why translating the C++ to Typescript would 
be alluring, but I can imagine its neigh on impossible given what you've 
said.

I wonder if there might be another approach, WebAssembly/wasm 
. I find this extremely intriguing and maybe there 
is a way to use the C++ code compiled to wasm, no idea if that is possible 
or practical, but just a thought. Of course its very early days for wasm, 
but I'm following it and just beginning to play around with it. The 
WebAssembly Minimally Viable Product (MVP) is available on the latest 
Chrome and Firefox browsers. And Node v8 will support it when its released 
around the end of the month, so you can actually play around with 
"released" code.

On Friday, April 7, 2017 at 7:43:05 PM UTC-7, Julián Díaz wrote:
>
> It is stalled, I'm afraid. Not quite in a condition where someone else 
> could pick it up, either. This is mostly due to lack of time with work 
> obligations, though that's on track to change soon.
>
> Truthfully, I may need to nuke-and-pave the current repository and start 
> over with ES5 JavaScript. Translating the C++ code more or less verbatim 
> has proven extraordinarily difficult for three reasons: pointer arithmetic 
> (I managed to create nice helpers to deal with this), C++11's robust 
> templating and type system which is used *heavily* in the capnp source 
> (TypeScript cannot come close), and pass-by-reference function arguments 
> which are *mutated* by the method itself (JavaScript cannot do this). I 
> totally understand now why the other two JS libs have been basically 
> abandoned.
>
> I'll use ES5 directly for the second pass rather than ES6/2016/etc since 
> I'll have much tighter control over the performance characteristics that 
> way. I will also take more of a top-down approach and loosely follow the 
> C++ API, only taking bits of inspiration from the original source when 
> appropriate. This is easier now that I have a deeper understanding of the 
> internals.
>
> Switching the library to ES5 isn't a death sentence for TypeScript 
> support. I can manually generate type definitions for the library and even 
> potentially have the code generator output full typescript or type 
> definition files as well.
>
> In any case, this project hasn't been forgotten!
>
> On Friday, April 7, 2017 at 10:24:42 PM UTC-4, Wink Saville wrote:
>>
>> Julian, What's the status of this?
>>
>> On Tuesday, October 4, 2016 at 8:13:51 AM UTC-7, Julián Díaz wrote:
>>>
>>> Generated TypeScript it'll be, then! It'll be pretty trivial to 
>>> subsequently invoke the TypeScript compiler on the TypeScript generated 
>>> code to get "free" ES5 code for everyone else.
>>>
>>> I'm really digging how much of a great fit TypeScript's type system is 
>>> for this stuff (so far, at least). Will report back once I have something 
>>> that resembles a code generator.
>>>
>>> On Sunday, October 2, 2016 at 6:15:06 PM UTC-4, Kenton Varda wrote:

 For Javascript, I actually don't think runtime schema loading should be 
 a goal. These days Javascript is essentially a compiled language anyway, 
 with all the transpiling and minifying. And to load schemas at runtime in 
 a 
 browser, you'd presumably need to rewrite the parser in pure-Javascript. 
 I'd like to avoid having multiple schema parser implementations as they'd 
 be likely to have subtle incompatibilities that would be a pain for 
 developers.

 So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty 
 good plan. Then we can get auto-complete in our IDEs and all that cool 
 stuff. :)

 -Kenton

 On Sun, Oct 2, 2016 at 10:31 AM,  
 wrote:

> After playing around a little and realizing how limited ES5 really is, 
> I'm open to writing this thing in 100% TypeScript. There's some caveats, 
> though - if the schema file is loaded dynamically it'll be hard 
> (impossible?) to add type safety without runtime type checks, and I can 
> see 
> that getting expensive. TypeScript almost gets in the way in this case. 
> However, a schema compiler can be set up to emit either typescript or 
> vanilla ES5 code; *that* could work quite well! ArrayBuffers will 
> contain the raw data and property access/method calls will just reference 
> indices in the array - the only *super* annoying thing about it is 
> there's no native support for 64-bit words in JavaScript so everything 
> has 
> to be broken up into hi/lo 32-bit words in a Uint32Array (yuck!).
>
> There are concerns about TypeScript being generally slower if you lean 
> heavily on the ES6 features (https://kpdecker.github.io/six-speed/), 
> but 

Re: [capnproto] Re: Reviving the JavaScript implementation

[julian . diaz]

It is stalled, I'm afraid. Not quite in a condition where someone else 
could pick it up, either. This is mostly due to lack of time with work 
obligations, though that's on track to change soon.

Truthfully, I may need to nuke-and-pave the current repository and start 
over with ES5 JavaScript. Translating the C++ code more or less verbatim 
has proven extraordinarily difficult for three reasons: pointer arithmetic 
(I managed to create nice helpers to deal with this), C++11's robust 
templating and type system which is used *heavily* in the capnp source 
(TypeScript cannot come close), and pass-by-reference function arguments 
which are *mutated* by the method itself (JavaScript cannot do this). I 
totally understand now why the other two JS libs have been basically 
abandoned.

I'll use ES5 directly for the second pass rather than ES6/2016/etc since 
I'll have much tighter control over the performance characteristics that 
way. I will also take more of a top-down approach and loosely follow the 
C++ API, only taking bits of inspiration from the original source when 
appropriate. This is easier now that I have a deeper understanding of the 
internals.

Switching the library to ES5 isn't a death sentence for TypeScript support. 
I can manually generate type definitions for the library and even 
potentially have the code generator output full typescript or type 
definition files as well.

In any case, this project hasn't been forgotten!

On Friday, April 7, 2017 at 10:24:42 PM UTC-4, Wink Saville wrote:
>
> Julian, What's the status of this?
>
> On Tuesday, October 4, 2016 at 8:13:51 AM UTC-7, Julián Díaz wrote:
>>
>> Generated TypeScript it'll be, then! It'll be pretty trivial to 
>> subsequently invoke the TypeScript compiler on the TypeScript generated 
>> code to get "free" ES5 code for everyone else.
>>
>> I'm really digging how much of a great fit TypeScript's type system is 
>> for this stuff (so far, at least). Will report back once I have something 
>> that resembles a code generator.
>>
>> On Sunday, October 2, 2016 at 6:15:06 PM UTC-4, Kenton Varda wrote:
>>>
>>> For Javascript, I actually don't think runtime schema loading should be 
>>> a goal. These days Javascript is essentially a compiled language anyway, 
>>> with all the transpiling and minifying. And to load schemas at runtime in a 
>>> browser, you'd presumably need to rewrite the parser in pure-Javascript. 
>>> I'd like to avoid having multiple schema parser implementations as they'd 
>>> be likely to have subtle incompatibilities that would be a pain for 
>>> developers.
>>>
>>> So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty 
>>> good plan. Then we can get auto-complete in our IDEs and all that cool 
>>> stuff. :)
>>>
>>> -Kenton
>>>
>>> On Sun, Oct 2, 2016 at 10:31 AM,  
>>> wrote:
>>>
 After playing around a little and realizing how limited ES5 really is, 
 I'm open to writing this thing in 100% TypeScript. There's some caveats, 
 though - if the schema file is loaded dynamically it'll be hard 
 (impossible?) to add type safety without runtime type checks, and I can 
 see 
 that getting expensive. TypeScript almost gets in the way in this case. 
 However, a schema compiler can be set up to emit either typescript or 
 vanilla ES5 code; *that* could work quite well! ArrayBuffers will 
 contain the raw data and property access/method calls will just reference 
 indices in the array - the only *super* annoying thing about it is 
 there's no native support for 64-bit words in JavaScript so everything has 
 to be broken up into hi/lo 32-bit words in a Uint32Array (yuck!).

 There are concerns about TypeScript being generally slower if you lean 
 heavily on the ES6 features (https://kpdecker.github.io/six-speed/), 
 but with proper profiling it shouldn't be hard to identify hotspots and 
 turn them into tightly optimized code that won't be modified much by the 
 transpiler. The speed problems come in when you heavily depend on things 
 like arrow functions and the transpiler ends up adding extra assignments 
 and variables you didn't expect. For what it's worth, I expect most 
 performance concerns won't have anything to do with using ES6.

 I agree that the transport layer needs to be pluggable and that's the 
 design I'm going to go for right from the start. Realistically I'm going 
 to 
 have a hard time designing the whole thing without a sample app to 
 reference so *some *form of transport needs to be supported "out of 
 the box". WebSockets feels like an easy initial target for this. 
 Supporting 
 window.postMessage should also be easy, and since you can pass 
 ArrayBuffers 
 with zero-copy (I think??) it'll be crazy fast!

 Speaking of sample app; I plan on writing a TodoMVC type thing in 
 separate repos as a reference implementation (a node server and a browser 
 

Re: [capnproto] Re: Reviving the JavaScript implementation

[Wink Saville]

Julian, What's the status of this?

On Tuesday, October 4, 2016 at 8:13:51 AM UTC-7, Julián Díaz wrote:
>
> Generated TypeScript it'll be, then! It'll be pretty trivial to 
> subsequently invoke the TypeScript compiler on the TypeScript generated 
> code to get "free" ES5 code for everyone else.
>
> I'm really digging how much of a great fit TypeScript's type system is for 
> this stuff (so far, at least). Will report back once I have something that 
> resembles a code generator.
>
> On Sunday, October 2, 2016 at 6:15:06 PM UTC-4, Kenton Varda wrote:
>>
>> For Javascript, I actually don't think runtime schema loading should be a 
>> goal. These days Javascript is essentially a compiled language anyway, with 
>> all the transpiling and minifying. And to load schemas at runtime in a 
>> browser, you'd presumably need to rewrite the parser in pure-Javascript. 
>> I'd like to avoid having multiple schema parser implementations as they'd 
>> be likely to have subtle incompatibilities that would be a pain for 
>> developers.
>>
>> So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty 
>> good plan. Then we can get auto-complete in our IDEs and all that cool 
>> stuff. :)
>>
>> -Kenton
>>
>> On Sun, Oct 2, 2016 at 10:31 AM,  
>> wrote:
>>
>>> After playing around a little and realizing how limited ES5 really is, 
>>> I'm open to writing this thing in 100% TypeScript. There's some caveats, 
>>> though - if the schema file is loaded dynamically it'll be hard 
>>> (impossible?) to add type safety without runtime type checks, and I can see 
>>> that getting expensive. TypeScript almost gets in the way in this case. 
>>> However, a schema compiler can be set up to emit either typescript or 
>>> vanilla ES5 code; *that* could work quite well! ArrayBuffers will 
>>> contain the raw data and property access/method calls will just reference 
>>> indices in the array - the only *super* annoying thing about it is 
>>> there's no native support for 64-bit words in JavaScript so everything has 
>>> to be broken up into hi/lo 32-bit words in a Uint32Array (yuck!).
>>>
>>> There are concerns about TypeScript being generally slower if you lean 
>>> heavily on the ES6 features (https://kpdecker.github.io/six-speed/), 
>>> but with proper profiling it shouldn't be hard to identify hotspots and 
>>> turn them into tightly optimized code that won't be modified much by the 
>>> transpiler. The speed problems come in when you heavily depend on things 
>>> like arrow functions and the transpiler ends up adding extra assignments 
>>> and variables you didn't expect. For what it's worth, I expect most 
>>> performance concerns won't have anything to do with using ES6.
>>>
>>> I agree that the transport layer needs to be pluggable and that's the 
>>> design I'm going to go for right from the start. Realistically I'm going to 
>>> have a hard time designing the whole thing without a sample app to 
>>> reference so *some *form of transport needs to be supported "out of the 
>>> box". WebSockets feels like an easy initial target for this. Supporting 
>>> window.postMessage should also be easy, and since you can pass ArrayBuffers 
>>> with zero-copy (I think??) it'll be crazy fast!
>>>
>>> Speaking of sample app; I plan on writing a TodoMVC type thing in 
>>> separate repos as a reference implementation (a node server and a browser 
>>> client w/ React). Any thoughts on this? Should I instead try to go for 
>>> something that better showcases the power of capability-based access 
>>> control? A todo list where you can securely share edit/view access with 
>>> others could be a fun demo. I want to keep the UI complexity low and really 
>>> focus on the RPC stuff, of course.
>>>
>>> Could also use tips on which order to implement things since this is a 
>>> bit of a *daunting* spec to implement. I'm currently having fun with 
>>> the unpacking algorithm. :)
>>>
>>> On Saturday, October 1, 2016 at 7:56:02 PM UTC-4, Nathan Hourt wrote:

 As far as the appetite front goes, I've rather wanted a pure javascript 
 implementation of capnp on several occasions now, so I'd be quite happy to 
 see this happen.

 I think websockets support is a must, but ideally the library would be 
 sufficiently portable that I would be able to use the library, unmodified, 
 on any particular transport layer I want, with nothing more than a little 
 glue code between interfaces. The library could then provide some built-in 
 adaptors for common transport layers (like Node sockets, websockets, etc) 
 which would allow those transports to be used with no customization, and 
 also documents how to write adaptors so it's easy to see how to use other 
 transports instead (exactly like what we see with the C++ implementation 
 with AsyncIoStream: I can use any transport I want simply by implementing 
 the interface).

 On Friday, September 30, 2016 at 7:08:01 PM UTC-5, Julián Díaz wrote:

Re: [capnproto] Re: Reviving the JavaScript implementation

[Julián Díaz]

Generated TypeScript it'll be, then! It'll be pretty trivial to 
subsequently invoke the TypeScript compiler on the TypeScript generated 
code to get "free" ES5 code for everyone else.

I'm really digging how much of a great fit TypeScript's type system is for 
this stuff (so far, at least). Will report back once I have something that 
resembles a code generator.

On Sunday, October 2, 2016 at 6:15:06 PM UTC-4, Kenton Varda wrote:
>
> For Javascript, I actually don't think runtime schema loading should be a 
> goal. These days Javascript is essentially a compiled language anyway, with 
> all the transpiling and minifying. And to load schemas at runtime in a 
> browser, you'd presumably need to rewrite the parser in pure-Javascript. 
> I'd like to avoid having multiple schema parser implementations as they'd 
> be likely to have subtle incompatibilities that would be a pain for 
> developers.
>
> So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty 
> good plan. Then we can get auto-complete in our IDEs and all that cool 
> stuff. :)
>
> -Kenton
>
> On Sun, Oct 2, 2016 at 10:31 AM,  > wrote:
>
>> After playing around a little and realizing how limited ES5 really is, 
>> I'm open to writing this thing in 100% TypeScript. There's some caveats, 
>> though - if the schema file is loaded dynamically it'll be hard 
>> (impossible?) to add type safety without runtime type checks, and I can see 
>> that getting expensive. TypeScript almost gets in the way in this case. 
>> However, a schema compiler can be set up to emit either typescript or 
>> vanilla ES5 code; *that* could work quite well! ArrayBuffers will 
>> contain the raw data and property access/method calls will just reference 
>> indices in the array - the only *super* annoying thing about it is 
>> there's no native support for 64-bit words in JavaScript so everything has 
>> to be broken up into hi/lo 32-bit words in a Uint32Array (yuck!).
>>
>> There are concerns about TypeScript being generally slower if you lean 
>> heavily on the ES6 features (https://kpdecker.github.io/six-speed/), but 
>> with proper profiling it shouldn't be hard to identify hotspots and turn 
>> them into tightly optimized code that won't be modified much by the 
>> transpiler. The speed problems come in when you heavily depend on things 
>> like arrow functions and the transpiler ends up adding extra assignments 
>> and variables you didn't expect. For what it's worth, I expect most 
>> performance concerns won't have anything to do with using ES6.
>>
>> I agree that the transport layer needs to be pluggable and that's the 
>> design I'm going to go for right from the start. Realistically I'm going to 
>> have a hard time designing the whole thing without a sample app to 
>> reference so *some *form of transport needs to be supported "out of the 
>> box". WebSockets feels like an easy initial target for this. Supporting 
>> window.postMessage should also be easy, and since you can pass ArrayBuffers 
>> with zero-copy (I think??) it'll be crazy fast!
>>
>> Speaking of sample app; I plan on writing a TodoMVC type thing in 
>> separate repos as a reference implementation (a node server and a browser 
>> client w/ React). Any thoughts on this? Should I instead try to go for 
>> something that better showcases the power of capability-based access 
>> control? A todo list where you can securely share edit/view access with 
>> others could be a fun demo. I want to keep the UI complexity low and really 
>> focus on the RPC stuff, of course.
>>
>> Could also use tips on which order to implement things since this is a 
>> bit of a *daunting* spec to implement. I'm currently having fun with the 
>> unpacking algorithm. :)
>>
>> On Saturday, October 1, 2016 at 7:56:02 PM UTC-4, Nathan Hourt wrote:
>>>
>>> As far as the appetite front goes, I've rather wanted a pure javascript 
>>> implementation of capnp on several occasions now, so I'd be quite happy to 
>>> see this happen.
>>>
>>> I think websockets support is a must, but ideally the library would be 
>>> sufficiently portable that I would be able to use the library, unmodified, 
>>> on any particular transport layer I want, with nothing more than a little 
>>> glue code between interfaces. The library could then provide some built-in 
>>> adaptors for common transport layers (like Node sockets, websockets, etc) 
>>> which would allow those transports to be used with no customization, and 
>>> also documents how to write adaptors so it's easy to see how to use other 
>>> transports instead (exactly like what we see with the C++ implementation 
>>> with AsyncIoStream: I can use any transport I want simply by implementing 
>>> the interface).
>>>
>>> On Friday, September 30, 2016 at 7:08:01 PM UTC-5, Julián Díaz wrote:

 For better or worse my curiosity has led me to try reviving the 
 JavaScript implementation of Cap'n Proto; at least all the way to level 1 
 RPC in native JavaScript.

 Looking 

Re: [capnproto] Re: Reviving the JavaScript implementation

[Kenton Varda]

For Javascript, I actually don't think runtime schema loading should be a
goal. These days Javascript is essentially a compiled language anyway, with
all the transpiling and minifying. And to load schemas at runtime in a
browser, you'd presumably need to rewrite the parser in pure-Javascript.
I'd like to avoid having multiple schema parser implementations as they'd
be likely to have subtle incompatibilities that would be a pain for
developers.

So, yes, capnp -> TypeScript compile-time codegen sounds like a pretty good
plan. Then we can get auto-complete in our IDEs and all that cool stuff. :)

-Kenton

On Sun, Oct 2, 2016 at 10:31 AM, 
wrote:

> After playing around a little and realizing how limited ES5 really is, I'm
> open to writing this thing in 100% TypeScript. There's some caveats, though
> - if the schema file is loaded dynamically it'll be hard (impossible?) to
> add type safety without runtime type checks, and I can see that getting
> expensive. TypeScript almost gets in the way in this case. However, a
> schema compiler can be set up to emit either typescript or vanilla ES5
> code; *that* could work quite well! ArrayBuffers will contain the raw
> data and property access/method calls will just reference indices in the
> array - the only *super* annoying thing about it is there's no native
> support for 64-bit words in JavaScript so everything has to be broken up
> into hi/lo 32-bit words in a Uint32Array (yuck!).
>
> There are concerns about TypeScript being generally slower if you lean
> heavily on the ES6 features (https://kpdecker.github.io/six-speed/), but
> with proper profiling it shouldn't be hard to identify hotspots and turn
> them into tightly optimized code that won't be modified much by the
> transpiler. The speed problems come in when you heavily depend on things
> like arrow functions and the transpiler ends up adding extra assignments
> and variables you didn't expect. For what it's worth, I expect most
> performance concerns won't have anything to do with using ES6.
>
> I agree that the transport layer needs to be pluggable and that's the
> design I'm going to go for right from the start. Realistically I'm going to
> have a hard time designing the whole thing without a sample app to
> reference so *some *form of transport needs to be supported "out of the
> box". WebSockets feels like an easy initial target for this. Supporting
> window.postMessage should also be easy, and since you can pass ArrayBuffers
> with zero-copy (I think??) it'll be crazy fast!
>
> Speaking of sample app; I plan on writing a TodoMVC type thing in separate
> repos as a reference implementation (a node server and a browser client w/
> React). Any thoughts on this? Should I instead try to go for something that
> better showcases the power of capability-based access control? A todo list
> where you can securely share edit/view access with others could be a fun
> demo. I want to keep the UI complexity low and really focus on the RPC
> stuff, of course.
>
> Could also use tips on which order to implement things since this is a bit
> of a *daunting* spec to implement. I'm currently having fun with the
> unpacking algorithm. :)
>
> On Saturday, October 1, 2016 at 7:56:02 PM UTC-4, Nathan Hourt wrote:
>>
>> As far as the appetite front goes, I've rather wanted a pure javascript
>> implementation of capnp on several occasions now, so I'd be quite happy to
>> see this happen.
>>
>> I think websockets support is a must, but ideally the library would be
>> sufficiently portable that I would be able to use the library, unmodified,
>> on any particular transport layer I want, with nothing more than a little
>> glue code between interfaces. The library could then provide some built-in
>> adaptors for common transport layers (like Node sockets, websockets, etc)
>> which would allow those transports to be used with no customization, and
>> also documents how to write adaptors so it's easy to see how to use other
>> transports instead (exactly like what we see with the C++ implementation
>> with AsyncIoStream: I can use any transport I want simply by implementing
>> the interface).
>>
>> On Friday, September 30, 2016 at 7:08:01 PM UTC-5, Julián Díaz wrote:
>>>
>>> For better or worse my curiosity has led me to try reviving the
>>> JavaScript implementation of Cap'n Proto; at least all the way to level 1
>>> RPC in native JavaScript.
>>>
>>> Looking at the current two implementations they don't really feel viable
>>> to hack on - one is using AMD for the browser (we've moved on!) and the
>>> other is clearly a mess as-is.
>>>
>>> After scratching the surface a good bit, my initial goals for the
>>> library are:
>>>
>>> - Full level-1 RPC support
>>> - Extensive unit test coverage
>>> - Full browser and nodejs compatibility
>>> - Pure ES5 implementation (for speed)
>>> - Low-to-zero library dependencies (lodash might become a must-have)
>>> - Loading schema files directly
>>>
>>> Stretch goals:
>>>
>>> - WebSocket transport