Hi all! I am a developer on qri <https://qri.io/>, a data-science tool built on IPFS written in go. We're interested in integrating Arrow and especially Plasma, in order to be able to share datasets with other apps like Jupyter Notebook. Having this functionality is going to be key for how we plan to integrate with existing frameworks.
I've been investigating possible approaches for how to use Plasma in our codecase. I realize that Plasma is still a work in progress, and doesn't have stable API yet, but we're also a ways off from being ready to fully integrate it on our side. Just figured it would be good to start this conversation early in order to plan ahead for how development should proceed. So, the prototypes I've been hacking on have revealed a few choices of how to make our golang codebase call Plasma's C++, and I wanted to see what the Plasma devs think about these approaches or if they have any preference for how the go bindings should be behave. Here are the options in order of what seems to be least to most usable: 1. cgo Use go's builtin cgo facility to call the Plasma C++ implementation. cgo is relatively easy to use, however it only can call C functions. So this would require writing and maintaining a pure C language wrapper around the C++ functionality we want to expose. A lot would be lost in translation and the resulting go code would look nothing like the original library. 2. dlopen Install Plasma as a library on the user's system, then load the library at run-time, looking up function calls and data structures as needed. Removes the need for a static dependency, but still requires a lot of shim code to be written to load the shared library calls. C++'s name mangling gets in the way a lot. 3. Swig Wrap a swig interface file that exposes whatever functionality we want to golang. The standard go tool has builtin swig support, which knows when to invoke the swig generator, in order to create go bindings that resemble the C++ original. The build process is relatively uninterrupted. I noticed there doesn't seem to be any swig in use currently in the arrow codebase, which made me think there might have been a reason that it has been avoided for other languages. I'm interested to hear any thoughts, or see if there are other suggestions on how to proceed? Regards, Dustin
