Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

Thank you!

Great help here. Amazing folks!

On Nov 17, 2017 6:21 PM, "J Decker"  wrote:

>
>
> On Fri, Nov 17, 2017 at 7:25 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> My assumption is that my test app terminates before the GC has a chance
>> to run.
>> To test this:
>>
>>- I run node with the --expose-gc option
>>- I call global.gc() at the end of my test app
>>
>> In this case, the destructor of my ObjectWrap is indeed called.
>>
>> *So it seams the GC doesn't simply "collect everything" when the app
>> terminates.*
>> *Why is that?*
>>
>> It sounds like a pretty bad things to do: some behaviors might expect the
>> destructor to be called in order to close a socket, etc...
>>
>
> because all memory is released. when the app exits.  Why do extra work of
> minor side effects?
> but ya, if tf you were returning like a file to a handle and expected the
> close in order to flush all data out to a device it's bad...
>
> You can attach to Node::AtExit( ... )
> https://nodejs.org/api/addons.html#addons_atexit_hooks
>
> which should be called so you an finalize such things.
>
> but sockets get closed by the system when a application exits also.  and
> files are closed
>
>
>
>>
>> On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux
>> wrote:
>>>
>>> Thanks for the great help!
>>> I think it makes a lot of sense now.
>>>
>>> So I've extended ObjectWrap like so:
>>>
>>> class BSONObject : public node::ObjectWrap
 {
 private:
 bson_t* _bson;
 public:
 BSONObject(bson_t* bson, Local tpl) :
 _bson(bson)
 {
 Local obj = tpl->NewInstance();
 initialize(v8::Isolate::GetCurrent(), obj, bson);
 Wrap(obj);
 }
>>>
>>>
>>>
>>>  ~BSONObject()
 {
 std::cout << "~BSONObject()" << std::endl;
 bson_destroy(_bson);
 }
>>>
>>>
>>>
>>> static
 Local
 create_template(Isolate* isolate, const bson_t* document)
 {
 Local tpl = ObjectTemplate::New(isolate);
 tpl->SetInternalFieldCount(1);
>>>
>>>
>>> And the "insert" function that I expose in JS and that instanciates the
>>> BSONObject is written like this:
>>>
>>> void
 insert(const FunctionCallbackInfo& args)
 {
 Isolate* isolate = args.GetIsolate();

 db->insert(
 json_stringify(isolate, args[0]),
 [, isolate](const bson_t* document)
 {
 if (!args[1]->IsUndefined())
 {
 Local callback =
 Local::Cast(args[1]);
 Local obj = Object::New(isolate);
 *Local argv[] = { (new
 BSONObject(bson_copy(document)))->handle(isolate) };*
 callback->Call(isolate->GetCurrentContext()->Global(),
 1, argv);
 }
 }
 );
 }
>>>
>>>
>>> It runs fine.
>>> But the destructor is never called (I never see "~BSONObject()" on
>>> stdout).
>>>
>>> Did I miss something?
>>>
>>> Thanks !
>>>
>>> 2017-11-16 22:01 GMT+01:00 J Decker :
>>>
 You don't manage the instance, the v8 engine does.  When it's no longer
 referenced in the engine, it's garbage collected which triggers the
 callback set when the perisstent object is made weak.

 You don't need to keep it anywhere, the object returned into the engine
 has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
 your native class instance.   So when the callback is called, it knows what
 to call to destruct it.

 You have to 'keep it' in the javascript scrpit if you want to keep it,
 it's not 'kept' in the native code...


 --- This is some code I've done that uses setweak directly; but it's
 fairly scattered

 https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18

 You can create an instance of your wrapped object with

 Local cons = Local::New( isolate, constructor );
 cons->NewInstance( isolate->GetCurrentContext(), argc, argv
 ).ToLocalChecked() ;

 Hmm that's not the best example source...

 https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407

 This creates a buffer, and makes it weak...
 https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438

 Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
 PARRAY_BUFFER_HOLDER holder = GetHolder();
 holder->o.Reset( isolate, arrayBuffer );
 holder->o.SetWeak( holder, releaseBuffer,
 WeakCallbackType::kParameter );
 holder->buffer = buf;

 PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
 object containing things to be released), and the persistent reference that
 has been made weak...

 struct arrayBufferHolder {
 void *buffer;
 Persistent o;
 };

Re: [v8-users] BSON to v8::Object performance issue

[J Decker]

On Fri, Nov 17, 2017 at 7:25 AM, Jean-Marc Le Roux  wrote:

> My assumption is that my test app terminates before the GC has a chance to
> run.
> To test this:
>
>- I run node with the --expose-gc option
>- I call global.gc() at the end of my test app
>
> In this case, the destructor of my ObjectWrap is indeed called.
>
> *So it seams the GC doesn't simply "collect everything" when the app
> terminates.*
> *Why is that?*
>
> It sounds like a pretty bad things to do: some behaviors might expect the
> destructor to be called in order to close a socket, etc...
>

because all memory is released. when the app exits.  Why do extra work of
minor side effects?
but ya, if tf you were returning like a file to a handle and expected the
close in order to flush all data out to a device it's bad...

You can attach to Node::AtExit( ... )
https://nodejs.org/api/addons.html#addons_atexit_hooks

which should be called so you an finalize such things.

but sockets get closed by the system when a application exits also.  and
files are closed



>
> On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux wrote:
>>
>> Thanks for the great help!
>> I think it makes a lot of sense now.
>>
>> So I've extended ObjectWrap like so:
>>
>> class BSONObject : public node::ObjectWrap
>>> {
>>> private:
>>> bson_t* _bson;
>>> public:
>>> BSONObject(bson_t* bson, Local tpl) :
>>> _bson(bson)
>>> {
>>> Local obj = tpl->NewInstance();
>>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>>> Wrap(obj);
>>> }
>>
>>
>>
>>  ~BSONObject()
>>> {
>>> std::cout << "~BSONObject()" << std::endl;
>>> bson_destroy(_bson);
>>> }
>>
>>
>>
>> static
>>> Local
>>> create_template(Isolate* isolate, const bson_t* document)
>>> {
>>> Local tpl = ObjectTemplate::New(isolate);
>>> tpl->SetInternalFieldCount(1);
>>
>>
>> And the "insert" function that I expose in JS and that instanciates the
>> BSONObject is written like this:
>>
>> void
>>> insert(const FunctionCallbackInfo& args)
>>> {
>>> Isolate* isolate = args.GetIsolate();
>>>
>>> db->insert(
>>> json_stringify(isolate, args[0]),
>>> [, isolate](const bson_t* document)
>>> {
>>> if (!args[1]->IsUndefined())
>>> {
>>> Local callback = Local::Cast(args[1])
>>> ;
>>> Local obj = Object::New(isolate);
>>> *Local argv[] = { (new
>>> BSONObject(bson_copy(document)))->handle(isolate) };*
>>> callback->Call(isolate->GetCurrentContext()->Global(),
>>> 1, argv);
>>> }
>>> }
>>> );
>>> }
>>
>>
>> It runs fine.
>> But the destructor is never called (I never see "~BSONObject()" on
>> stdout).
>>
>> Did I miss something?
>>
>> Thanks !
>>
>> 2017-11-16 22:01 GMT+01:00 J Decker :
>>
>>> You don't manage the instance, the v8 engine does.  When it's no longer
>>> referenced in the engine, it's garbage collected which triggers the
>>> callback set when the perisstent object is made weak.
>>>
>>> You don't need to keep it anywhere, the object returned into the engine
>>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
>>> your native class instance.   So when the callback is called, it knows what
>>> to call to destruct it.
>>>
>>> You have to 'keep it' in the javascript scrpit if you want to keep it,
>>> it's not 'kept' in the native code...
>>>
>>>
>>> --- This is some code I've done that uses setweak directly; but it's
>>> fairly scattered
>>>
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>>>
>>> You can create an instance of your wrapped object with
>>>
>>> Local cons = Local::New( isolate, constructor );
>>> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
>>> ).ToLocalChecked() ;
>>>
>>> Hmm that's not the best example source...
>>>
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>>>
>>> This creates a buffer, and makes it weak...
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>>>
>>> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
>>> PARRAY_BUFFER_HOLDER holder = GetHolder();
>>> holder->o.Reset( isolate, arrayBuffer );
>>> holder->o.SetWeak( holder, releaseBuffer,
>>> WeakCallbackType::kParameter );
>>> holder->buffer = buf;
>>>
>>> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
>>> object containing things to be released), and the persistent reference that
>>> has been made weak...
>>>
>>> struct arrayBufferHolder {
>>> void *buffer;
>>> Persistent o;
>>> };
>>>
>>>
>>>
>>> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
 Node offers a C++ Clss extention ObjectWrap


 Thanks !
 It is related to node. Yet I'm not sure how I'm supposed to use
 ObjectWrap.

 I understand I 

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

My assumption is that my test app terminates before the GC has a chance to 
run.
To test this:

   - I run node with the --expose-gc option
   - I call global.gc() at the end of my test app

In this case, the destructor of my ObjectWrap is indeed called.

*So it seams the GC doesn't simply "collect everything" when the app 
terminates.*
*Why is that?*

It sounds like a pretty bad things to do: some behaviors might expect the 
destructor to be called in order to close a socket, etc...

On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux wrote:
>
> Thanks for the great help!
> I think it makes a lot of sense now.
>
> So I've extended ObjectWrap like so:
>
> class BSONObject : public node::ObjectWrap
>> {
>> private:
>> bson_t* _bson;
>> public:
>> BSONObject(bson_t* bson, Local tpl) :
>> _bson(bson)
>> {
>> Local obj = tpl->NewInstance();
>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>> Wrap(obj);
>> }
>
>  
>
>  ~BSONObject()
>> {
>> std::cout << "~BSONObject()" << std::endl;
>> bson_destroy(_bson);
>> }
>
>  
>
> static
>> Local
>> create_template(Isolate* isolate, const bson_t* document)
>> {
>> Local tpl = ObjectTemplate::New(isolate);
>> tpl->SetInternalFieldCount(1); 
>
>
> And the "insert" function that I expose in JS and that instanciates the 
> BSONObject is written like this:
>
> void
>> insert(const FunctionCallbackInfo& args)
>> {
>> Isolate* isolate = args.GetIsolate();
>> 
>> db->insert(
>> json_stringify(isolate, args[0]),
>> [, isolate](const bson_t* document)
>> {
>> if (!args[1]->IsUndefined())
>> {
>> Local callback = Local::Cast(args[1]);
>> Local obj = Object::New(isolate);
>> *Local argv[] = { (new 
>> BSONObject(bson_copy(document)))->handle(isolate) };*
>> callback->Call(isolate->GetCurrentContext()->Global(), 1, 
>> argv);
>> }
>> }
>> );
>> }
>
>
> It runs fine.
> But the destructor is never called (I never see "~BSONObject()" on stdout).
>
> Did I miss something?
>
> Thanks !
>
> 2017-11-16 22:01 GMT+01:00 J Decker :
>
>> You don't manage the instance, the v8 engine does.  When it's no longer 
>> referenced in the engine, it's garbage collected which triggers the 
>> callback set when the perisstent object is made weak.
>>
>> You don't need to keep it anywhere, the object returned into the engine 
>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to 
>> your native class instance.   So when the callback is called, it knows what 
>> to call to destruct it.
>>
>> You have to 'keep it' in the javascript scrpit if you want to keep it, 
>> it's not 'kept' in the native code...
>>
>>
>> --- This is some code I've done that uses setweak directly; but it's 
>> fairly scattered
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>>
>> You can create an instance of your wrapped object with
>>
>> Local cons = Local::New( isolate, constructor );
>> cons->NewInstance( isolate->GetCurrentContext(), argc, argv  
>> ).ToLocalChecked() ;
>>
>> Hmm that's not the best example source... 
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>>
>> This creates a buffer, and makes it weak... 
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>>
>> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
>> PARRAY_BUFFER_HOLDER holder = GetHolder();
>> holder->o.Reset( isolate, arrayBuffer );
>> holder->o.SetWeak( holder, releaseBuffer, 
>> WeakCallbackType::kParameter );
>> holder->buffer = buf;
>>
>> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or 
>> object containing things to be released), and the persistent reference that 
>> has been made weak...
>>
>> struct arrayBufferHolder {
>> void *buffer;
>> Persistent o;
>> };
>>
>>
>>
>> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
>> jeanmarc.ler...@aerys.in> wrote:
>>
>>> Node offers a C++ Clss extention ObjectWrap 
>>>
>>>
>>> Thanks !
>>> It is related to node. Yet I'm not sure how I'm supposed to use 
>>> ObjectWrap.
>>>
>>> I understand I have to create my own class that extends ObjectWrap and 
>>> implement a proper destructor.
>>> Then I'll instanciate that class to wrap my Local. But how do I 
>>> manage such instance?
>>> If I don't keep it somewhere, it will be destructed.
>>> If I keep it around, then I'm keeping references to potentially 
>>> deallocated memory.
>>>
>>> It feels like I end up with the same problem...
>>>
>>> 2017-11-16 18:36 GMT+01:00 J Decker :
>>>


 On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
 jeanmarc.ler...@aerys.in> wrote:

> I assume you're storing references to the objects as Persistent?
>
>
> Nope. You've got the whole code up 

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

Thanks for the great help!
I think it makes a lot of sense now.

So I've extended ObjectWrap like so:

class BSONObject : public node::ObjectWrap
> {
> private:
> bson_t* _bson;
> public:
> BSONObject(bson_t* bson, Local tpl) :
> _bson(bson)
> {
> Local obj = tpl->NewInstance();
> initialize(v8::Isolate::GetCurrent(), obj, bson);
> Wrap(obj);
> }



 ~BSONObject()
> {
> std::cout << "~BSONObject()" << std::endl;
> bson_destroy(_bson);
> }



static
> Local
> create_template(Isolate* isolate, const bson_t* document)
> {
> Local tpl = ObjectTemplate::New(isolate);
> tpl->SetInternalFieldCount(1);


And the "insert" function that I expose in JS and that instanciates the
BSONObject is written like this:

void
> insert(const FunctionCallbackInfo& args)
> {
> Isolate* isolate = args.GetIsolate();
>
> db->insert(
> json_stringify(isolate, args[0]),
> [, isolate](const bson_t* document)
> {
> if (!args[1]->IsUndefined())
> {
> Local callback = Local::Cast(args[1]);
> Local obj = Object::New(isolate);
> *Local argv[] = { (new
> BSONObject(bson_copy(document)))->handle(isolate) };*
> callback->Call(isolate->GetCurrentContext()->Global(), 1,
> argv);
> }
> }
> );
> }


It runs fine.
But the destructor is never called (I never see "~BSONObject()" on stdout).

Did I miss something?

Thanks !

2017-11-16 22:01 GMT+01:00 J Decker :

> You don't manage the instance, the v8 engine does.  When it's no longer
> referenced in the engine, it's garbage collected which triggers the
> callback set when the perisstent object is made weak.
>
> You don't need to keep it anywhere, the object returned into the engine
> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
> your native class instance.   So when the callback is called, it knows what
> to call to destruct it.
>
> You have to 'keep it' in the javascript scrpit if you want to keep it,
> it's not 'kept' in the native code...
>
>
> --- This is some code I've done that uses setweak directly; but it's
> fairly scattered
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>
> You can create an instance of your wrapped object with
>
> Local cons = Local::New( isolate, constructor );
> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
> ).ToLocalChecked() ;
>
> Hmm that's not the best example source...
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>
> This creates a buffer, and makes it weak...
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>
> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
> PARRAY_BUFFER_HOLDER holder = GetHolder();
> holder->o.Reset( isolate, arrayBuffer );
> holder->o.SetWeak( holder, releaseBuffer,
> WeakCallbackType::kParameter );
> holder->buffer = buf;
>
> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
> object containing things to be released), and the persistent reference that
> has been made weak...
>
> struct arrayBufferHolder {
> void *buffer;
> Persistent o;
> };
>
>
>
> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> Node offers a C++ Clss extention ObjectWrap
>>
>>
>> Thanks !
>> It is related to node. Yet I'm not sure how I'm supposed to use
>> ObjectWrap.
>>
>> I understand I have to create my own class that extends ObjectWrap and
>> implement a proper destructor.
>> Then I'll instanciate that class to wrap my Local. But how do I
>> manage such instance?
>> If I don't keep it somewhere, it will be destructed.
>> If I keep it around, then I'm keeping references to potentially
>> deallocated memory.
>>
>> It feels like I end up with the same problem...
>>
>> 2017-11-16 18:36 GMT+01:00 J Decker :
>>
>>>
>>>
>>> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
 I assume you're storing references to the objects as Persistent?


 Nope. You've got the whole code up there.
 So if I understand correctly:

- the objects I need to keep track of the lifecycle should be made
Persistent ;
- all the other values can be set as Local ;
- I should set the weak callback using MakeWeak ;
- callback will be called when such object is GCed and I can free
my C/C++ memory then.

 Is that correct ?

>>>
>>> Yes.  I sthis perhaps related to Node?
>>> Node offers a C++ Clss extention ObjectWrap
>>>
>>> https://nodejs.org/api/addons.html#addons_wrapping_c_objects
>>> https://github.com/nodejs/node/blob/master/src/node_object_wrap.h
>>>
>>> which simplifies that whole process... when the object is GC'd the class
>>> destructor will get called so you can release values there.
>>>

Re: [v8-users] BSON to v8::Object performance issue

[J Decker]

You don't manage the instance, the v8 engine does.  When it's no longer
referenced in the engine, it's garbage collected which triggers the
callback set when the perisstent object is made weak.

You don't need to keep it anywhere, the object returned into the engine has
Internal Fields ( SetInternalFieldCount(1) ) which is used to point to your
native class instance.   So when the callback is called, it knows what to
call to destruct it.

You have to 'keep it' in the javascript scrpit if you want to keep it, it's
not 'kept' in the native code...


--- This is some code I've done that uses setweak directly; but it's fairly
scattered

https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18

You can create an instance of your wrapped object with

Local cons = Local::New( isolate, constructor );
cons->NewInstance( isolate->GetCurrentContext(), argc, argv
).ToLocalChecked() ;

Hmm that's not the best example source...

https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407

This creates a buffer, and makes it weak...
https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438

Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
PARRAY_BUFFER_HOLDER holder = GetHolder();
holder->o.Reset( isolate, arrayBuffer );
holder->o.SetWeak( holder, releaseBuffer,
WeakCallbackType::kParameter );
holder->buffer = buf;

PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
object containing things to be released), and the persistent reference that
has been made weak...

struct arrayBufferHolder {
void *buffer;
Persistent o;
};



On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux  wrote:

> Node offers a C++ Clss extention ObjectWrap
>
>
> Thanks !
> It is related to node. Yet I'm not sure how I'm supposed to use ObjectWrap.
>
> I understand I have to create my own class that extends ObjectWrap and
> implement a proper destructor.
> Then I'll instanciate that class to wrap my Local. But how do I
> manage such instance?
> If I don't keep it somewhere, it will be destructed.
> If I keep it around, then I'm keeping references to potentially
> deallocated memory.
>
> It feels like I end up with the same problem...
>
> 2017-11-16 18:36 GMT+01:00 J Decker :
>
>>
>>
>> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
>> jeanmarc.ler...@aerys.in> wrote:
>>
>>> I assume you're storing references to the objects as Persistent?
>>>
>>>
>>> Nope. You've got the whole code up there.
>>> So if I understand correctly:
>>>
>>>- the objects I need to keep track of the lifecycle should be made
>>>Persistent ;
>>>- all the other values can be set as Local ;
>>>- I should set the weak callback using MakeWeak ;
>>>- callback will be called when such object is GCed and I can free my
>>>C/C++ memory then.
>>>
>>> Is that correct ?
>>>
>>
>> Yes.  I sthis perhaps related to Node?
>> Node offers a C++ Clss extention ObjectWrap
>>
>> https://nodejs.org/api/addons.html#addons_wrapping_c_objects
>> https://github.com/nodejs/node/blob/master/src/node_object_wrap.h
>>
>> which simplifies that whole process... when the object is GC'd the class
>> destructor will get called so you can release values there.
>>
>>
>>
>>>
>>> 2017-11-15 19:50 GMT+01:00 J Decker :
>>>
 I assume you're storing references to the objects as Persistent?  All
 you need to do then is call SetWeak() https://v8docs.nodes
 ource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
 void MakeWeak (void *parameters, WeakReferenceCallback callback)

 the callback is called when the object is GC'd.


 On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
 jeanmarc.ler...@aerys.in> wrote:

> So I've found a solution to make things lazy: I set an accessor
> instead of decoding/setting the actual value.
>
> void
> getter(Local property, const PropertyCallbackInfo& info)
> {
> Isolate* isolate = info.GetIsolate();
>
>   const bson_t* document = reinterpret_cast(
> info.This()
> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
> ->ToInt32()->Value()
> );
>
> char key[255];
> property->ToString(isolate)->WriteUtf8(key, 255);
>
> bson_iter_t iter;
> bson_iter_init(, document);
> // FIXME: index the property so we don't have to find it
> bson_iter_find(, key);
>
> // FIXME: replace the accessor with the deserialized value
>
> info.GetReturnValue().Set(iterator_to_value(isolate, ));
> }
>
> Local
> fill_object(Isolate* isolate, Local& obj, const bson_t*
> document)
> {
> obj->Set(
> String::NewFromUtf8(isolate, "__bson_document_ptr"),
> Int32::New(isolate, reinterpret_cast(document))
> );
>
> bson_iter_t iter;
> if (bson_iter_init(, 

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

>
> Node offers a C++ Clss extention ObjectWrap


Thanks !
It is related to node. Yet I'm not sure how I'm supposed to use ObjectWrap.

I understand I have to create my own class that extends ObjectWrap and
implement a proper destructor.
Then I'll instanciate that class to wrap my Local. But how do I
manage such instance?
If I don't keep it somewhere, it will be destructed.
If I keep it around, then I'm keeping references to potentially deallocated
memory.

It feels like I end up with the same problem...

2017-11-16 18:36 GMT+01:00 J Decker :

>
>
> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> I assume you're storing references to the objects as Persistent?
>>
>>
>> Nope. You've got the whole code up there.
>> So if I understand correctly:
>>
>>- the objects I need to keep track of the lifecycle should be made
>>Persistent ;
>>- all the other values can be set as Local ;
>>- I should set the weak callback using MakeWeak ;
>>- callback will be called when such object is GCed and I can free my
>>C/C++ memory then.
>>
>> Is that correct ?
>>
>
> Yes.  I sthis perhaps related to Node?
> Node offers a C++ Clss extention ObjectWrap
>
> https://nodejs.org/api/addons.html#addons_wrapping_c_objects
> https://github.com/nodejs/node/blob/master/src/node_object_wrap.h
>
> which simplifies that whole process... when the object is GC'd the class
> destructor will get called so you can release values there.
>
>
>
>>
>> 2017-11-15 19:50 GMT+01:00 J Decker :
>>
>>> I assume you're storing references to the objects as Persistent?  All
>>> you need to do then is call SetWeak() https://v8docs.nodes
>>> ource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
>>> void MakeWeak (void *parameters, WeakReferenceCallback callback)
>>>
>>> the callback is called when the object is GC'd.
>>>
>>>
>>> On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
 So I've found a solution to make things lazy: I set an accessor instead
 of decoding/setting the actual value.

 void
 getter(Local property, const PropertyCallbackInfo& info)
 {
 Isolate* isolate = info.GetIsolate();

   const bson_t* document = reinterpret_cast(
 info.This()
 ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
 ->ToInt32()->Value()
 );

 char key[255];
 property->ToString(isolate)->WriteUtf8(key, 255);

 bson_iter_t iter;
 bson_iter_init(, document);
 // FIXME: index the property so we don't have to find it
 bson_iter_find(, key);

 // FIXME: replace the accessor with the deserialized value

 info.GetReturnValue().Set(iterator_to_value(isolate, ));
 }

 Local
 fill_object(Isolate* isolate, Local& obj, const bson_t*
 document)
 {
 obj->Set(
 String::NewFromUtf8(isolate, "__bson_document_ptr"),
 Int32::New(isolate, reinterpret_cast(document))
 );

 bson_iter_t iter;
 if (bson_iter_init(, document))
 {
 while (bson_iter_next())
 {
 const char* key = bson_iter_key();

 if (!obj->Has(String::NewFromUtf8(isolate,
 key)))
 {
 obj->SetAccessor(
 isolate->GetCurrentContext(),
 String::NewFromUtf8(isolate, key),
 
 );
 }
 }
 }
 }

 The secret sauce is to :

- keep the original data (the bson_t) allocated
- store the corresponding pointer in a v8::Object

 But now I have a memory leak because those pointers will never be freed.
 How can I know when the Object will be disposed by the GC?

 Thanks,

 --
 --
 v8-users mailing list
 v8-users@googlegroups.com
 http://groups.google.com/group/v8-users
 ---
 You received this message because you are subscribed to the Google
 Groups "v8-users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to v8-users+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-users@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "v8-users" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>> pic/v8-users/aVeevQcHJ2c/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> v8-users+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>

Re: [v8-users] BSON to v8::Object performance issue

[J Decker]

On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux  wrote:

> I assume you're storing references to the objects as Persistent?
>
>
> Nope. You've got the whole code up there.
> So if I understand correctly:
>
>- the objects I need to keep track of the lifecycle should be made
>Persistent ;
>- all the other values can be set as Local ;
>- I should set the weak callback using MakeWeak ;
>- callback will be called when such object is GCed and I can free my
>C/C++ memory then.
>
> Is that correct ?
>

Yes.  I sthis perhaps related to Node?
Node offers a C++ Clss extention ObjectWrap

https://nodejs.org/api/addons.html#addons_wrapping_c_objects
https://github.com/nodejs/node/blob/master/src/node_object_wrap.h

which simplifies that whole process... when the object is GC'd the class
destructor will get called so you can release values there.



>
> 2017-11-15 19:50 GMT+01:00 J Decker :
>
>> I assume you're storing references to the objects as Persistent?  All you
>> need to do then is call SetWeak() https://v8docs.nodes
>> ource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
>> void MakeWeak (void *parameters, WeakReferenceCallback callback)
>>
>> the callback is called when the object is GC'd.
>>
>>
>> On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
>> jeanmarc.ler...@aerys.in> wrote:
>>
>>> So I've found a solution to make things lazy: I set an accessor instead
>>> of decoding/setting the actual value.
>>>
>>> void
>>> getter(Local property, const PropertyCallbackInfo& info)
>>> {
>>> Isolate* isolate = info.GetIsolate();
>>>
>>>   const bson_t* document = reinterpret_cast(
>>> info.This()
>>> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
>>> ->ToInt32()->Value()
>>> );
>>>
>>> char key[255];
>>> property->ToString(isolate)->WriteUtf8(key, 255);
>>>
>>> bson_iter_t iter;
>>> bson_iter_init(, document);
>>> // FIXME: index the property so we don't have to find it
>>> bson_iter_find(, key);
>>>
>>> // FIXME: replace the accessor with the deserialized value
>>>
>>> info.GetReturnValue().Set(iterator_to_value(isolate, ));
>>> }
>>>
>>> Local
>>> fill_object(Isolate* isolate, Local& obj, const bson_t* document)
>>> {
>>> obj->Set(
>>> String::NewFromUtf8(isolate, "__bson_document_ptr"),
>>> Int32::New(isolate, reinterpret_cast(document))
>>> );
>>>
>>> bson_iter_t iter;
>>> if (bson_iter_init(, document))
>>> {
>>> while (bson_iter_next())
>>> {
>>> const char* key = bson_iter_key();
>>>
>>> if (!obj->Has(String::NewFromUtf8(isolate,
>>> key)))
>>> {
>>> obj->SetAccessor(
>>> isolate->GetCurrentContext(),
>>> String::NewFromUtf8(isolate, key),
>>> 
>>> );
>>> }
>>> }
>>> }
>>> }
>>>
>>> The secret sauce is to :
>>>
>>>- keep the original data (the bson_t) allocated
>>>- store the corresponding pointer in a v8::Object
>>>
>>> But now I have a memory leak because those pointers will never be freed.
>>> How can I know when the Object will be disposed by the GC?
>>>
>>> Thanks,
>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-users@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to v8-users+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "v8-users" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/v8-users/aVeevQcHJ2c/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> *Jean-Marc Le Roux*
>
>
> Founder and CEO of Aerys (http://aerys.in)
>
> Blog: http://blogs.aerys.in/jeanmarc-leroux
> Cell: (+33)6 20 56 45 78 <+33%206%2020%2056%2045%2078>
> Phone: (+33)9 72 40 17 58 <+33%209%2072%2040%2017%2058>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

>
> I assume you're storing references to the objects as Persistent?


Nope. You've got the whole code up there.
So if I understand correctly:

   - the objects I need to keep track of the lifecycle should be made
   Persistent ;
   - all the other values can be set as Local ;
   - I should set the weak callback using MakeWeak ;
   - callback will be called when such object is GCed and I can free my
   C/C++ memory then.

Is that correct ?

2017-11-15 19:50 GMT+01:00 J Decker :

> I assume you're storing references to the objects as Persistent?  All you
> need to do then is call SetWeak() https://v8docs.
> nodesource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
> void MakeWeak (void *parameters, WeakReferenceCallback callback)
>
> the callback is called when the object is GC'd.
>
>
> On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> So I've found a solution to make things lazy: I set an accessor instead
>> of decoding/setting the actual value.
>>
>> void
>> getter(Local property, const PropertyCallbackInfo& info)
>> {
>> Isolate* isolate = info.GetIsolate();
>>
>>   const bson_t* document = reinterpret_cast(
>> info.This()
>> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
>> ->ToInt32()->Value()
>> );
>>
>> char key[255];
>> property->ToString(isolate)->WriteUtf8(key, 255);
>>
>> bson_iter_t iter;
>> bson_iter_init(, document);
>> // FIXME: index the property so we don't have to find it
>> bson_iter_find(, key);
>>
>> // FIXME: replace the accessor with the deserialized value
>>
>> info.GetReturnValue().Set(iterator_to_value(isolate, ));
>> }
>>
>> Local
>> fill_object(Isolate* isolate, Local& obj, const bson_t* document)
>> {
>> obj->Set(
>> String::NewFromUtf8(isolate, "__bson_document_ptr"),
>> Int32::New(isolate, reinterpret_cast(document))
>> );
>>
>> bson_iter_t iter;
>> if (bson_iter_init(, document))
>> {
>> while (bson_iter_next())
>> {
>> const char* key = bson_iter_key();
>>
>> if (!obj->Has(String::NewFromUtf8(isolate,
>> key)))
>> {
>> obj->SetAccessor(
>> isolate->GetCurrentContext(),
>> String::NewFromUtf8(isolate, key),
>> 
>> );
>> }
>> }
>> }
>> }
>>
>> The secret sauce is to :
>>
>>- keep the original data (the bson_t) allocated
>>- store the corresponding pointer in a v8::Object
>>
>> But now I have a memory leak because those pointers will never be freed.
>> How can I know when the Object will be disposed by the GC?
>>
>> Thanks,
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "v8-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/v8-users/aVeevQcHJ2c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*Jean-Marc Le Roux*


Founder and CEO of Aerys (http://aerys.in)

Blog: http://blogs.aerys.in/jeanmarc-leroux
Cell: (+33)6 20 56 45 78
Phone: (+33)9 72 40 17 58

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] BSON to v8::Object performance issue

[J Decker]

I assume you're storing references to the objects as Persistent?  All you
need to do then is call SetWeak()
https://v8docs.nodesource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
void MakeWeak (void *parameters, WeakReferenceCallback callback)

the callback is called when the object is GC'd.


On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux  wrote:

> So I've found a solution to make things lazy: I set an accessor instead of
> decoding/setting the actual value.
>
> void
> getter(Local property, const PropertyCallbackInfo& info)
> {
> Isolate* isolate = info.GetIsolate();
>
>   const bson_t* document = reinterpret_cast(
> info.This()
> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
> ->ToInt32()->Value()
> );
>
> char key[255];
> property->ToString(isolate)->WriteUtf8(key, 255);
>
> bson_iter_t iter;
> bson_iter_init(, document);
> // FIXME: index the property so we don't have to find it
> bson_iter_find(, key);
>
> // FIXME: replace the accessor with the deserialized value
>
> info.GetReturnValue().Set(iterator_to_value(isolate, ));
> }
>
> Local
> fill_object(Isolate* isolate, Local& obj, const bson_t* document)
> {
> obj->Set(
> String::NewFromUtf8(isolate, "__bson_document_ptr"),
> Int32::New(isolate, reinterpret_cast(document))
> );
>
> bson_iter_t iter;
> if (bson_iter_init(, document))
> {
> while (bson_iter_next())
> {
> const char* key = bson_iter_key();
>
> if (!obj->Has(String::NewFromUtf8(isolate, key)))
> {
> obj->SetAccessor(
> isolate->GetCurrentContext(),
> String::NewFromUtf8(isolate, key),
> 
> );
> }
> }
> }
> }
>
> The secret sauce is to :
>
>- keep the original data (the bson_t) allocated
>- store the corresponding pointer in a v8::Object
>
> But now I have a memory leak because those pointers will never be freed.
> How can I know when the Object will be disposed by the GC?
>
> Thanks,
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

So I've found a solution to make things lazy: I set an accessor instead of 
decoding/setting the actual value.

void
getter(Local property, const PropertyCallbackInfo& info)
{
Isolate* isolate = info.GetIsolate();

  const bson_t* document = reinterpret_cast(
info.This()
->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
->ToInt32()->Value()
);

char key[255];
property->ToString(isolate)->WriteUtf8(key, 255);

bson_iter_t iter;
bson_iter_init(, document);
// FIXME: index the property so we don't have to find it
bson_iter_find(, key);

// FIXME: replace the accessor with the deserialized value

info.GetReturnValue().Set(iterator_to_value(isolate, ));
}

Local
fill_object(Isolate* isolate, Local& obj, const bson_t* document)
{
obj->Set(
String::NewFromUtf8(isolate, "__bson_document_ptr"),
Int32::New(isolate, reinterpret_cast(document))
);

bson_iter_t iter;
if (bson_iter_init(, document))
{
while (bson_iter_next())
{
const char* key = bson_iter_key();

if (!obj->Has(String::NewFromUtf8(isolate, key))) 
{
obj->SetAccessor(
isolate->GetCurrentContext(),
String::NewFromUtf8(isolate, key),

);
}
}
}
}

The secret sauce is to :

   - keep the original data (the bson_t) allocated
   - store the corresponding pointer in a v8::Object

But now I have a memory leak because those pointers will never be freed.
How can I know when the Object will be disposed by the GC?

Thanks,

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

>
> Now it takes *30ms. That's still 5x faster* than the native code.


300ms. (not 30)

On Wed, Nov 15, 2017 at 10:27 AM, Jean-Marc Le Roux <
jeanmarc.ler...@aerys.in> wrote:

> Thanks for the quick feedback!
>
> I rewrote the JS version to be closer to the native one:
>
> var array = [];
> var input = {
>   "_id": {
> "$oid": "5a00bad8f759511811e030ba"
>   },
>   "attributes": [
> {
>   "key": "smartshape.scene.node.path|default",
>   "value": "/scene/Lot444.scene",
>   "type": "imported"
> },
> {
>   "key": "max x",
>   "value": 196.5773162841797,
>   "type": "computed"
> },
> {
>   "key": "max y",
>   "value": 18.55002021789551,
>   "type": "computed"
> },
> {
>   "key": "max z",
>   "value": 22.87815856933594,
>   "type": "computed"
> },
> {
>   "key": "min x",
>   "value": 149.9346771240234,
>   "type": "computed"
> },
> {
>   "key": "min y",
>   "value": 18.54999732971191,
>   "type": "computed"
> },
> {
>   "key": "min z",
>   "value": -23.35353088378906,
>   "type": "computed"
> },
> {
>   "key": "box radius",
>   "value": 23.32131958007814,
>   "type": "computed"
> },
> {
>   "key": "center x",
>   "value": 173.25599670410156,
>   "type": "computed"
> },
> {
>   "key": "center y",
>   "value": 18.55000877380371,
>   "type": "computed"
> },
> {
>   "key": "center z",
>   "value": -0.23768615722655895,
>   "type": "computed"
> },
> {
>   "key": "width",
>   "value": 46.64263916015628,
>   "type": "computed"
> },
> {
>   "key": "height",
>   "value": 2.2888183600855427e-05,
>   "type": "computed"
> },
> {
>   "key": "depth",
>   "value": 46.231689453125,
>   "type": "computed"
> },
> {
>   "key": "box volume",
>   "value": 0.04935534689932106,
>   "type": "computed"
> },
> {
>   "key": "box surface",
>   "value": 4312.740269302394,
>   "type": "computed"
> }
>   ],
>   "name": "default1161",
>   "uuid": "70bf7d72-1fa9-5c8f-21ff-03ef209d4404",
>   "surfaces": [
> "6f5201a2-31a7-14d0-6b2a-5130007d2a51"
>   ],
>   "aabb": [
> 196.5773162841797,
> 18.55002021789551,
> 22.87815856933594,
> 149.9346771240234,
> 18.54999732971191,
> -23.35353088378906
>   ],
>   "position": null,
>   "scale": null,
>   "rotation": null,
>   "parents": [
> "18497b66-3f32-6e98-1899-2998203e6397",
> null
>   ],
>   "file": "5a00b9a4aa5d2517d32d03da",
>   "numChildren": 0,
>   "sceneTreeIndices": [
> 0
>   ]
> };
>
> function process(value) {
>   if (typeof value === "number") {
> return value;
>   } else if (typeof value === "string") {
> return value;
>   } else if (Array.isArray(value)) {
> var array = [];
>
> for (var item of value) {
>   array.push(process(item));
> }
>
> return array;
>   } else if (typeof value === "object") {
>
> if (!value) {
>   return value;
> }
>
> var obj = {};
> for (var key of Object.keys(value)) {
>   obj[key] = process(value[key]);
> }
>
> return obj;
>   } else {
> return value;
>   }
>
>   return value;
> }
>
> for (var i = 0; i < 26000; ++i) {
>   var obj = process(input);
>
>   array.push(obj);
> }
>
>
>
> Now it takes *30ms. That's still 5x faster* than the native code.
>
> I'm suspecting my Local are copied/GCed multiple time to build the
> final object.
> Isn't that the case?
> Is there a better way to do it?
>
> using a constructor function and just filling in the values should be
>> faster than assembling them one property at a time.
>
>
> The root object is expected to be the same 99% of the time.
> Each of his field too.
> So I could definitely have a map.
> How can I implement such a constructor function in v8 ?
>
> Thanks !
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "v8-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/v8-users/aVeevQcHJ2c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*Jean-Marc Le Roux*


Founder and CEO of Aerys (http://aerys.in)

Blog: http://blogs.aerys.in/jeanmarc-leroux
Phone: (+33)6 20 56 45 78

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

Thanks for the quick feedback!

I rewrote the JS version to be closer to the native one:

var array = [];
var input = {
  "_id": {
"$oid": "5a00bad8f759511811e030ba"
  },
  "attributes": [
{
  "key": "smartshape.scene.node.path|default",
  "value": "/scene/Lot444.scene",
  "type": "imported"
},
{
  "key": "max x",
  "value": 196.5773162841797,
  "type": "computed"
},
{
  "key": "max y",
  "value": 18.55002021789551,
  "type": "computed"
},
{
  "key": "max z",
  "value": 22.87815856933594,
  "type": "computed"
},
{
  "key": "min x",
  "value": 149.9346771240234,
  "type": "computed"
},
{
  "key": "min y",
  "value": 18.54999732971191,
  "type": "computed"
},
{
  "key": "min z",
  "value": -23.35353088378906,
  "type": "computed"
},
{
  "key": "box radius",
  "value": 23.32131958007814,
  "type": "computed"
},
{
  "key": "center x",
  "value": 173.25599670410156,
  "type": "computed"
},
{
  "key": "center y",
  "value": 18.55000877380371,
  "type": "computed"
},
{
  "key": "center z",
  "value": -0.23768615722655895,
  "type": "computed"
},
{
  "key": "width",
  "value": 46.64263916015628,
  "type": "computed"
},
{
  "key": "height",
  "value": 2.2888183600855427e-05,
  "type": "computed"
},
{
  "key": "depth",
  "value": 46.231689453125,
  "type": "computed"
},
{
  "key": "box volume",
  "value": 0.04935534689932106,
  "type": "computed"
},
{
  "key": "box surface",
  "value": 4312.740269302394,
  "type": "computed"
}
  ],
  "name": "default1161",
  "uuid": "70bf7d72-1fa9-5c8f-21ff-03ef209d4404",
  "surfaces": [
"6f5201a2-31a7-14d0-6b2a-5130007d2a51"
  ],
  "aabb": [
196.5773162841797,
18.55002021789551,
22.87815856933594,
149.9346771240234,
18.54999732971191,
-23.35353088378906
  ],
  "position": null,
  "scale": null,
  "rotation": null,
  "parents": [
"18497b66-3f32-6e98-1899-2998203e6397",
null
  ],
  "file": "5a00b9a4aa5d2517d32d03da",
  "numChildren": 0,
  "sceneTreeIndices": [
0
  ]
};

function process(value) {
  if (typeof value === "number") {
return value;
  } else if (typeof value === "string") {
return value;
  } else if (Array.isArray(value)) {
var array = [];

for (var item of value) {
  array.push(process(item));
}

return array;
  } else if (typeof value === "object") {

if (!value) {
  return value;
}

var obj = {};
for (var key of Object.keys(value)) {
  obj[key] = process(value[key]);
}

return obj;
  } else {
return value;
  }

  return value;
}

for (var i = 0; i < 26000; ++i) {
  var obj = process(input);

  array.push(obj);
}



Now it takes *30ms. That's still 5x faster* than the native code.

I'm suspecting my Local are copied/GCed multiple time to build the 
final object.
Isn't that the case?
Is there a better way to do it?

using a constructor function and just filling in the values should be 
> faster than assembling them one property at a time.


The root object is expected to be the same 99% of the time.
Each of his field too.
So I could definitely have a map.
How can I implement such a constructor function in v8 ?

Thanks !

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] BSON to v8::Object performance issue

[Jakob Kummerow]

There are a bunch of optimizations going into object literal handling, so
the two versions are not at all equivalent: the JS version is expected to
be much faster. To make them more comparable, you could model the same
control flow in JS, roughly:

var input = {
"_id": {
"$oid": "5a00bad8f759511811e030ba"
},
"attributes": ...
};
function process(o) {
var result = {};
for (var key of Object.keys(o)) {
var value = o[key];
if (typeof value === "Number") {
...
} else if ... {

} else if (typeof value === "object") {
result[key] = process(value);
}
return result;
}
var output;
for (var i = 0; i < 26400; i++) {
output = process(input);
}

I don't see anything obviously wrong with your code.

However, if there are any assumptions you can make, maybe you can optimize
it. For example: do all your objects have the same shape? If so, using a
constructor function and just filling in the values should be faster than
assembling them one property at a time.


On Tue, Nov 14, 2017 at 11:46 AM Jean-Marc Le Roux <
jeanmarc.ler...@gmail.com> wrote:

> Hello,
>
> I'm trying to create NodeJS bindings for libbson.
> One of the things I need is to be able to transform a bson_t document into
> a v8::Object.
> Loading, iterating on bson_t documents and matching them with
> MongoDB-style queries it very fast: 50ms to load from my HDD, iterate and
> filter 26000 bson_t documents.
> On the same machine, converting the same BSON documents into v8::Object is
> painfully slow.
> So I suspect I missed something.
>
> Here is my code:
>
> Local
> iterator_to_value(Isolate* isolate, const bson_iter_t* iter)
> {
> switch (bson_iter_type(iter))
> {
> case BSON_TYPE_INT32:
> return Number::New(isolate, bson_iter_int32(iter));
> break;
> case BSON_TYPE_INT64:
> return Number::New(isolate, bson_iter_int64(iter));
> break;
> case BSON_TYPE_DOUBLE:
> return Number::New(isolate, bson_iter_double(iter));
> break;
> case BSON_TYPE_DOCUMENT:
> {
> bson_iter_t sub_iter;
> bson_iter_recurse(iter, _iter);
>
> Local obj = Object::New(isolate);
> while (bson_iter_next(_iter))
> {
> const char* key = bson_iter_key(_iter);
>
> obj->Set(
> String::NewFromUtf8(isolate, key),
> iterator_to_value(isolate, _iter)
> );
> }
>
> return obj;
> }
> break;
> case BSON_TYPE_ARRAY:
> {
> bson_iter_t sub_iter;
> uint32_t length;
> const uint8_t* array_data;
>
> bson_iter_array(iter, , _data);
> bson_iter_recurse(iter, _iter);
>
> Local array = Array::New(isolate);
> int i = 0;
>
> while (bson_iter_next(_iter))
> {
> array->Set(i++, iterator_to_value(isolate, _iter));
> }
>
> return array;
> }
> break;
> case BSON_TYPE_OID:
> {
> const bson_oid_t* oid = bson_iter_oid(iter);
> char oid_buffer[25];
>
> bson_oid_to_string(oid, oid_buffer);
>
> return String::NewFromOneByte(isolate, (uint8_t*)oid_buffer);
> }
> break;
> case BSON_TYPE_UTF8:
> {
> uint32_t length;
> return String::NewFromUtf8(isolate, bson_iter_utf8(iter,
> ));
> }
> break;
> }
>
> return Null(isolate);
> }
>
> Local
> fill_object(Isolate* isolate, Local& obj, const bson_t* document)
> {
> bson_iter_t iter;
> if (bson_iter_init(, document))
> {
> while (bson_iter_next())
> {
> const char* key = bson_iter_key();
>
> obj->Set(
> String::NewFromUtf8(isolate, key),
> iterator_to_value(isolate, )
> );
> }
> }
>
> return obj;
> }
>
> As you can see, it's very straight forward.
> *Transforming 26400 bson_t into v_::Object takes 1.23s.*
>
> I tried doing what I believe to be the same code in pure JS with a
> representative sample Object :
>
> var array = [];
> for (var i = 0; i < 25000; ++i)
> {
> array.push({
> "_id": {
>   "$oid": "5a00bad8f759511811e030ba"
> },
> "attributes": [
>   {
> "key": "smartshape.scene.node.path|default",
> "value": "/scene/Lot444.scene",
> "type": "imported"
>   },
>   {
> "key": "max x",
> "value": 196.5773162841797,
> "type": "computed"
>   },
>   {
> "key": "max y",
> "value": 18.55002021789551,
> "type": "computed"
>   },
>   {
> "key": "max z",
> "value":