Re: Ruby, Proton Engine and Records
On Fri, Feb 06, 2015 at 08:37:07AM -0500, Rafael Schloming wrote: Is this the approach you referenced in your other email? (Same request here for a pointer to the highlighted key bits of the approach.) No, this was a third approach to the problem, since we can't directly affect the object created by Swig via Data_Wrap_Struct. You can see the specific changes to ruby.i here [1]. http://github.com/mcpierce/Proton/commit/1e66f39535e71dc376d69abde7b5741929d33a3d -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgprYbkagDgS7.pgp Description: PGP signature
Re: Ruby, Proton Engine and Records
Is this the approach you referenced in your other email? (Same request here for a pointer to the highlighted key bits of the approach.) On Fri, Jan 30, 2015 at 4:11 PM, Darryl L. Pierce dpie...@redhat.com wrote: So over the last week I've been working on a way to avoid leaking memory or causing segfaults when the underlying C library is giving a reference to a Ruby object. And after much coding and gnashing of teeth I foudn the easiest way to do this with manually wrapping the pn_record_t struct. That would allow for adding a mark function to mark any Ruby objects so that they could not be reaped during garbage collection. And this would work fine but for one problem: the instances of pn_record are created by transport, etc. as part of *their* initializations. So there is no way for us to manually wrap the pn_record_t type for Ruby use in Swig. So looking at things, I have an alternate method that's not brittle and which gives us the feature we're looking for. From the Swig code, we can wrap the call to pn_record_set, at which point we have access to the Ruby wrapper for the pn_record_t type. We can then add/modify instance variables on that Ruby wrapper. Specifically, we can add and then update one named pn_record_attrs (or any other arbitrary name) and use it to track any objects that're added or replaced for the record. And the bonus is that, when the pn_record_t is deleted, it's Ruby wrapper is deleted as well and _all_ instance variables (including our Ruby objects) will be reaped along with it. I'm going to do some more testing on this, but my initial findings support this memory management scheme as viable. I have a test application that creates 500k+ values, storing them in a single record instance. Iterating back over the records and recovering them after running garbage collect returns variables as expected. Commenting out the portion that manages the wrapper instance variable and the same tests fail with a segfault while accessing the records. The work can all be checked out here [1], with the changes to the ruby.i file as the most recent commit in the branch. I'd love to get some feedback on this work in general as well. [1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/
Ruby, Proton Engine and Records
So over the last week I've been working on a way to avoid leaking memory or causing segfaults when the underlying C library is giving a reference to a Ruby object. And after much coding and gnashing of teeth I foudn the easiest way to do this with manually wrapping the pn_record_t struct. That would allow for adding a mark function to mark any Ruby objects so that they could not be reaped during garbage collection. And this would work fine but for one problem: the instances of pn_record are created by transport, etc. as part of *their* initializations. So there is no way for us to manually wrap the pn_record_t type for Ruby use in Swig. So looking at things, I have an alternate method that's not brittle and which gives us the feature we're looking for. From the Swig code, we can wrap the call to pn_record_set, at which point we have access to the Ruby wrapper for the pn_record_t type. We can then add/modify instance variables on that Ruby wrapper. Specifically, we can add and then update one named pn_record_attrs (or any other arbitrary name) and use it to track any objects that're added or replaced for the record. And the bonus is that, when the pn_record_t is deleted, it's Ruby wrapper is deleted as well and _all_ instance variables (including our Ruby objects) will be reaped along with it. I'm going to do some more testing on this, but my initial findings support this memory management scheme as viable. I have a test application that creates 500k+ values, storing them in a single record instance. Iterating back over the records and recovering them after running garbage collect returns variables as expected. Commenting out the portion that manages the wrapper instance variable and the same tests fail with a segfault while accessing the records. The work can all be checked out here [1], with the changes to the ruby.i file as the most recent commit in the branch. I'd love to get some feedback on this work in general as well. [1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgp3CnIpetAP6.pgp Description: PGP signature
