Ah, I see I can use garrow_buffer_get_mutable_data from c_glib. On Tue, Mar 27, 2018 at 9:29 AM Saul Shanabrook <s.shanabr...@gmail.com> wrote:
> I am in the process of changing over my C wrapper to work with Arrow > 0.9.0. One change is that the plasma APIs now use arrow::Buffer instead of > of raw pointers, in Create and Get. > > In my C wrapper for Create, I need to get the mutable data pointer from > the arrow::Buffer, after it is created. Here is my current implementation: > https://github.com/plures/pxnd/blob/84e9403ef3811e0893e12076455d7cbc49c39358/libplasma/plasma.cc#L56-L75 > I do something similar to the Python wrapper around plasma: > https://github.com/apache/arrow/blob/f9f8320339692d4134d1ef42c32cb7c8d547593e/python/pyarrow/plasma.pyx#L300-L307 > > > However, this fails to link, I think because the mutable_data method is > defined in the class and so is inline. So it doesn't show up in the > libarrow.dylib/libarrow.a archive. Let me know if I am missing something > obvious, this is just my hypothesis for why it isn't working. > > Is there a way for a third party library to get the pointer to the data > from a arrow::Buffer? > > On Wed, Mar 7, 2018 at 10:59 AM Wes McKinney <wesmck...@gmail.com> wrote: > >> Where are you going to make Plasma requests, in Python or in C? >> >> * If in Python, use the buffer protocol to access the memory in C >> https://docs.python.org/3/c-api/buffer.html#bufferobjects >> * If in C, then I would suggest adding extern "C" APIs directly in >> libplasma.so so you can connect to the Plasma store and make requests >> directly in C. >> >> I want to make 100% sure that you need to interact with the Plasma >> client in C before I send you down that path. Here's an example of a C >> API embedded in an otherwise C++ library >> https://github.com/wesm/feather/blob/master/cpp/src/feather/feather-c.h. >> >> - Wes >> >> On Wed, Mar 7, 2018 at 10:53 AM, Saul Shanabrook <s.shanabr...@gmail.com> >> wrote: >> > Hey Wes, >> > >> > Since all our logic for writing and reading the data is written in C, we >> > need to be get access to the memory in C as well. I think it makes sense >> > then for me to work on creating a C++ wrapper around Plasma that exports >> > what is necessary in C. Should that be in Arrow GLib? >> > >> > Best, >> > Saul >> > >> > On Tue, Mar 6, 2018 at 11:49 PM Wes McKinney <wesmck...@gmail.com> >> wrote: >> > >> >> hi Saul -- I think the easiest solution here is the buffer/memoryview >> >> protocol. You won't have to touch the Cython or C++ API from pyarrow >> >> if you do this. >> >> >> >> You can interact with a Buffer object like any other Python object >> >> implementing the buffer protocol. See numpy.frombuffer as an example >> >> of a function that interacts with such objects. I would suggest adding >> >> a method to xnd method for this. >> >> >> >> If you need to interact with Plasma from C then things will be more >> >> complicated -- Robert or Philipp should be able to advise in this >> >> case. >> >> >> >> - Wes >> >> >> >> On Tue, Mar 6, 2018 at 4:55 PM, Saul Shanabrook < >> s.shanabr...@gmail.com> >> >> wrote: >> >> > Hey Wes, >> >> > >> >> > I don't have much experience doing C + Python + Cython development, >> so I >> >> am >> >> > probably missing something obvious, but reading the Cython docs, >> >> > < >> >> >> https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#public-declarations >> >> > >> >> > it >> >> > seems like I can only access types marked as public from C code. >> When I >> >> > compile arrow locally, I do get some C++ headers for the plasma code, >> >> but I >> >> > don't think I can use them from C code either. >> >> > >> >> > Best, >> >> > Saul >> >> > >> >> > >> >> > >> >> > On Tue, Mar 6, 2018 at 3:12 PM Wes McKinney <wesmck...@gmail.com> >> wrote: >> >> > >> >> >> hi Saul, >> >> >> >> >> >> Are you able to use the buffer/memoryview protocol? Instances of >> >> >> pyarrow.Buffer, like PlasmaBuffer, support this >> >> >> >> >> >> >> >> >> https://github.com/apache/arrow/blob/master/python/pyarrow/plasma.pyx#L182 >> >> >> >> >> >> - Wes >> >> >> >> >> >> On Tue, Mar 6, 2018 at 3:09 PM, Saul Shanabrook < >> s.shanabr...@gmail.com >> >> > >> >> >> wrote: >> >> >> > I am trying to use the Plasma store to back xnd objects. Xnd ( >> >> >> > https://xnd.readthedocs.io/en/latest/xnd/index.html) is a >> container >> >> >> library >> >> >> > in C that has Python bindings. I would like to get a pointer to >> the >> >> >> > allocated memory after creating or get an object in Plasma. I see >> that >> >> >> this >> >> >> > is supported in the C++ API ( >> >> >> > >> >> >> >> >> >> https://arrow.apache.org/docs/cpp/classplasma_1_1_plasma_client.html#ac18ab9cc792c620a97a3dcb165e0ecd7 >> >> >> ) >> >> >> > but not in the python API (as far as I can tell). Is it possible >> to >> >> use >> >> >> the >> >> >> > C++ Plasma API from a C project? If not, would it make sense to >> expose >> >> >> > pointer access on the Python API using capsules >> >> >> > https://docs.python.org/3.6/c-api/capsule.html >> >> >> > <https://docs.python.org/3.6/c-api/capsule.html#capsules>? >> >> >> >> >> >> >
