+1
On Fri, Aug 20, 2021 at 11:37 PM Micah Kornfield
wrote:
> +1 (binding)
>
> On Fri, Aug 20, 2021 at 7:46 AM Keith Kraus
> wrote:
>
> > +1 (non-binding)
> >
> > On Fri, Aug 20, 2021 at 9:49 AM Rok Mihevc wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Fri, Aug 20, 2021 at 3:46 PM Jorge
That's precisely what I was suggesting and expecting. Sounds good. I'll clean
up my POC and make a proper PR sometime soon.
Thanks much for the discussion!
--Matt
-Original Message-
From: Antoine Pitrou
Sent: Monday, August 23, 2021 2:18 PM
To: dev@arrow.apache.org
Subject: Re:
I believe so.
The encoding of a string in Flatbuffers is [byte] with a null terminator not
included in the length, so old files should still be readable (they would
simply not see the terminator anymore). And conversely, continuing to write the
null terminator means new files should still be
Then we could provide a small C dataset API somewhere in the C++ source
tree (perhaps `arrow/dataset/c/api.h`?). It would be
unstable/experimental and could undergo changes or even removal without
notice.
Regards
Antoine.
Le 23/08/2021 à 20:07, Matthew Topol a écrit :
Because go is
Because go is always statically compiled for whatever platform you're on at the
time, the default behavior is for importing go libraries using `go get` from
the command line actually does a git clone of the code and compiles it on the
fly (because go's compiler is pretty darn fast) and caches
Le 23/08/2021 à 19:53, Matthew Topol a écrit :
The only thing I don't like it being a private module in the Go implementation
is distribution. For native go code, consumers can just perform `go get` and
have it work. But for this interface, it would require both consumers of the
module and
The only thing I don't like it being a private module in the Go implementation
is distribution. For native go code, consumers can just perform `go get` and
have it work. But for this interface, it would require both consumers of the
module and any consumers of those consumers to have a local
In a lucky turn of events, Phillip actually turned out to be in my neck of
the woods on Friday so we had a chance to sit down and discuss this. To
help, I actually shared something I had been working on a few months ago
independently (before this discussion started).
For reference:
Wes PR:
Le 23/08/2021 à 17:52, David Li a écrit :
Another way forward might be to relax the value type to [byte], but also
require implementations to null-terminate binary values regardless. The C++
Flatbuffers implementation does this already [1] (though not the Java one [2]).
Old implementations
We're just going into the C++ implementation now - would having the C++
client be enough here or are we looking for both the client and the server
side?
*Kyle Porter*
CEO
Bit Quill Technologies Inc.
Office: +1.778.331.3355 | Direct: +1.604.441.7318 | ky...@bitquilltech.com
I would like to point out a PR [1] that proposes a new API for recording
and aggregating execution metrics (visible via `EXPLAIN ANALYZE` and
programmatically) in case anyone would like to offer feedback for the design
[1] https://github.com/apache/arrow-datafusion/pull/908
Le 23/08/2021 à 19:16, Matthew Topol a écrit :
Unfortunately, Go currently can only integrate with C++ libraries through a C
interface. There does exist SWIG which is a generator for creating interface
code between Go and C++, but ultimately it's just automating the creation of a
C
Unfortunately, Go currently can only integrate with C++ libraries through a C
interface. There does exist SWIG which is a generator for creating interface
code between Go and C++, but ultimately it's just automating the creation of a
C interface and Go glue code. Personally I'm not a fan of the
Le 23/08/2021 à 18:22, Matthew Topol a écrit :
That's a fair point, and part of the work I've done so far is a local Go
implementation of at least consuming the C data interface. It will also
eventually involve creating the necessary implementation to produce the C-Data
interface too. But
That's a fair point, and part of the work I've done so far is a local Go
implementation of at least consuming the C data interface. It will also
eventually involve creating the necessary implementation to produce the C-Data
interface too. But specifically I'm asking for opinions on using that
If it's just the Protobuf it would at least not generate any code, but at that
point it would probably be better to just have Kyle & co. copy-paste the file
into the C++ PR until we can get it all settled.
-David
On Sun, Aug 22, 2021, at 17:53, Micah Kornfield wrote:
> In the interest of
Hi Matt,
As the name suggests, the C data interface is not a *programming*
interface. It is a data sharing convention which relies on the
existence of dedicated endpoints to produce or consume the C data
structures.
For example in Arrow C++, there is this set of APIs:
Another way forward might be to relax the value type to [byte], but also
require implementations to null-terminate binary values regardless. The C++
Flatbuffers implementation does this already [1] (though not the Java one [2]).
Old implementations validating UTF8-ness would still be unable to
Hey All,
So I've been working on a use case where I needed to be able to use the Dataset
API from Golang and instead of trying to port all of it to Golang (which would
require porting the Compute side too) I decided to create a proof of concept
using CGO to just call into the existing C++ code
Hello guys, sorry I have to to request for review here again since
progress didn't seem to be made yet :(. These PRs are important to
Dataset Java implementation as the first version of it was too basic to
serve advanced use cases. If reviewing of the big write support PR[1]
sounds to be a bit of
20 matches
Mail list logo
