The first version of the java implementation of the file format is ready:
https://github.com/apache/arrow/pull/123
It will need some follow up work but I think we should merge it and iterate
on it.
>From there it should be easy to validate java/c++ interop.
On Wed, Aug 24, 2016 at 2:55 PM, Wes McKinney <wesmck...@gmail.com> wrote:
> When the dust settles on the current metadata discussions I can scope
> the work / JIRAs for the corresponding C++ implementation -- we will
> be able to reuse much of the IPC code that Micah has been working on.
>
> As a C++ housekeeping item, perhaps we should rename the arrow/ipc
> portion of the codebase to reflect more general data transport
> (IPC/RPC/file-based), e.g. arrow/transport and arrow::transport
> namespace).
>
> On Tue, Aug 23, 2016 at 7:20 PM, Julien Le Dem <jul...@dremio.com> wrote:
> > I'm implementing the Arrow file format (based on RPC messages in a file)
> to
> > validate the metadata on my end.
> >
> > On Sun, Aug 21, 2016 at 9:28 PM, Wes McKinney <wesmck...@gmail.com>
> wrote:
> >
> >> Thank you, Julien, for taking the lead on this.
> >>
> >> It would be great to close out the remaining items with the V1
> >> metadata; I think we are close, so that we can start moving closer to
> >> integration testing between the Java and C++ implementations.
> >>
> >> If anyone else has some time the coming week or two to engage on the
> >> outstanding items (see also the other open pull requests), it would be
> >> most helpful.
> >>
> >> - Wes
> >>
> >> On Thu, Aug 11, 2016 at 9:27 AM, Julien Le Dem <jul...@dremio.com>
> wrote:
> >> > With the goal of finalizing a spec for V1, I've created JIRAs on the
> >> format
> >> > component.
> >> > Here is a list bellow. Please discuss on individual JIRAs if you have
> >> > comments.
> >> >
> >> > - [ARROW-253] restrict ints to only power of 2 #bytes (8, 16, 32, 64)
> >> > - [ARROW-254] remove Bit as we use Boolean for nullability array
> >> (validity
> >> > vector)
> >> > - [ARROW-252] add implementation guidelines
> >> > - [ARROW-255] dictionary encoding spec
> >> > - [ARROW-258] need to clarify Buffer.{page,offset} in mem sharing and
> >> > RPC/file contexts
> >> > - [ARROW-256] add format version
> >> > - [ARROW-257] add types vector to union type (to enable using type ids
> >> > instead of child offset)
> >> >
> >> > full list:
> >> > https://issues.apache.org/jira/issues/?jql=project%20%
> >> 3D%20ARROW%20AND%20component%20%3D%20Format%20AND%20resolution%20%3D%
> >> 20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority%
> >> 20DESC%2C%20created%20ASC
> >> >
> >> > --
> >> > Julien
> >>
> >
> >
> >
> > --
> > Julien
>
--
Julien
