[jira] [Commented] (PROTON-892) pn_data_t capacity does not grow above 32768 items

2015-08-21 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/PROTON-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706929#comment-14706929
 ] 

ASF subversion and git services commented on PROTON-892:


Commit b8883c4dfbc900d7db39d41155144573557b25ce in qpid-proton's branch 
refs/heads/master from [~aconway]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=b8883c4 ]

PROTON-892: Fix test bug in tests/data.c, forgotten pn_data_free().


 pn_data_t capacity does not grow above 32768 items
 --

 Key: PROTON-892
 URL: https://issues.apache.org/jira/browse/PROTON-892
 Project: Qpid Proton
  Issue Type: Bug
  Components: proton-c
Affects Versions: 0.9, 0.9.1, 0.10
Reporter: Michael
Assignee: Alan Conway
 Fix For: 0.11

 Attachments: 0003-data-capacity.patch


 pn_data_grow() function looses half of the available data capacity.
 The following happens: when data overflows, pn_data_grow is invoked. It 
 increases data capacity 2 times and reallocates nodes array. Data capacity is 
 represented as uint16_t type and so when capacity reaches 32768 items, the 
 result of multiplication by 2 becomes 0. This makes realloc return null and 
 crashes the program.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Final (maybe really) proposal for C++ binding memory management.

2015-08-21 Thread aconway
Here it is, this time with working code and with the loan and
transfer names. I find them much better for explaining this, thanks
Andrew! 

One important simplification: proton functions will take parameters as
plain foo. I was originally going to write a pointer-like class with
the exact same semantics as foo and then try to explain it. Duh.
Restricting the fancy pointer template stuff to return values makes it
easier to explain (probably because explaining why you are doing
something you really don't need to do at all is always puzzling to the
listener.)

Any more suggestions, shout quick. This will hit master soon and it'll
be too late./**
@page memory_management

The C++ proton binding offers the following options, in decreasing order of
safety and increasing order of complexity. You don't need to understand the more
complex options to use the simpler ones.

1. memory-safe use with smart pointers (std::shared_ptr, std::unique_ptr, boost::shared_ptr, boost::intrusive_ptr)
2. unsafe plain-pointer use to avoid smart pointer overhead.
3. exchange of pointers between C and C++ code.

In this discussion `foo` is a placeholder for any proton C++ class such as
`proton::reactor::link` or `proton::reactor::session`.

## Safe smart-pointer use.

Proton functions return either `pn_loan_ptrfoo` or `pn_transfer_ptrfoo`.
They are there to correctly convert to different smart pointer types.  Don't
worry about them, don't use them in your own code, just *assign* the return
value to the smart pointer of your choice: std::shared_ptr, std::unique_ptr,
boost::shared_ptr, boost::intrusive_ptr or std::auto_ptr.

If you are stuck with old C++ and no boost library you can use
`proton::pn_shared_ptr` in your code. It does not have all the features of
std::shared_ptr but does provide basic memory safety.

## Unsafe plain pointer use.

If you use plain pointers there are no extra reference counts or allocations
compared to using the C API directly, but you are responsible for following the
rules below. Use `get()` to convert a function return value to a plain pointer.

If a function returns `pn_transfer_ptrfoo`, ownership is transferred to you
and you are responsible for deleting. For example:

proton::reactor::receiver* r1 = session-create_receiver(...).get();
...
delete r1;

If a function returns `pn_loan_ptrfoo` then the pointer is on loan to you.
You may use it in some limited scope (see the doc) but you must not delete it,
and you must not use it outside that scope. For example:

class my_handler : public proton::messaging_handler {
...
void on_link_opened(proton::reactor::event e) {
proton::reactor::link* l = e.link()
// Use it but don't delete it.
...
// On return, l is invalid. Don't save it for later use. If you
// need to, use a smart pointer instead.
}
}

Note: If you assign the result of a pn_loan_ptr function to a smart pointer,
everything works automatically because of the magic described in the next
section. When you call get() the magic is gone and you can't get it back. For
example, proton::reactor::event::link(...) returns pn_loan_ptrlink, so the
pointer is on *loan*. That doesn't matter if you use shared pointers:

std::unique_ptrproton::reactor::link l = e.link();

This is safe, l is valid even if you return it outside the current scope.

However the following will crash. The pointer is on loan, so the caller must
not delete it, either directly or indirectly by making a smart pointer own it.

std::auto_ptrproton::reactor::link l(e.link().get());   // CRASH
std::unique_ptrproton::reactor::link l(e.link().get()); // CRASH
std::shared_ptrproton::reactor::link l(e.link().get()); // CRASH

## Gory details and mixed C/C++

You don't have to read this unless you are interested in how it works or in
using mixed C and C++ code.

Here are the key things to know:

1. The public `C` API contains named but undefind C structs, e.g.  `struct
::pn_foo_t;` You can `reinterpret_cast` between `foo*` and `pn_foo_t*`. They
point at the same memory location (the C struct) so you can pass between C and
C++.

2. Given `foo* p`, calling `p-something(...)` is equivalent to the C call
`::pn_foo_something(reinterpret_cast::pn_foo_t*(p, ...))`

3. As well as `pn_foo_free()` for traditional free-style memory management,
each proton C struct also has a reference counts. `pn_object_incref/decref()`
manipulate them.

4. Proton over-rides `operator delete` so `delete p` is equivalent to
`pn_object_decref(reinterpret_castpn_foo_t*(p))`

When you call `delete`, you are deleting a *proton reference*. The underlying
proton object is only deleted if it is the *last* reference. The object lives on
until the last internal or external reference is gone.

pn_loan_ptr increases the refcount when converted to a smart pointer so the
smart pointer has a new reference that it owns. pn_transfer_ptr doesn't change
the refcount so the smart