Hello Jacques,
we could also add the Feather format [1] from Wes and Hadley Wickham to
the list. It uses the Arrow spec to provide a common columnar
interchange format for Python and R DataFrames. With speed and
interoperability it has the same core values as Arrow itself. With it we
have already got a userbase with a native (non-JVM ;)) implementation of
Arrow. Through its buzz it also spurred some interest in Arrow in the
Julia community (though I'm not clear how far they actually got with
their prototype implementation).
Uwe
[1] https://github.com/wesm/feather
On 11.07.16 07:27, Jacques Nadeau wrote:
An update incorporating your feedback:
## Description:
Arrow is a columnar in-memory analytics layer designed to accelerate big
data.
It houses a set of canonical in-memory representations of flat and
hierarchical data along with multiple language-bindings for structure
manipulation. It also provides IPC and common algorithm implementations.
## Issues:
- There are no issues requiring board attention at this time.
## Activity:
- Awareness continues to increase with the community having done
presentations
at various meetups as well as the following conferences: Pydata Paris,
Hadoop
Summit Ireland, Hadoop Summit San Jose and Berlin Buzzwords.
- The CPP work has made good progress.
- The cross-project work with Parquet has seen substantial work (both in
the
Parquet project and the Arrow project). This should be a great first
example
proof-of-concept integration showing the benefits of an in-memory
columnar layer.
- There has been substantial progress on development of for the IPC /
memory sharing.
- Java development has slowed some but appears to be picking up again.
## Health report:
- We've seen good discussion and development activity since the last
report.
- We need to get to a first release.
- Prior to doing so, the community is working on rudimentary integration
tests between
Java and C++ and more formal format specification.
- More work can be done to make the project approachable to newly
interested parties by
creating additional documentation and quickstart. A sample application
will also help.
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months.
- Last PMC addition was Abdel Hakim Deneche on Tue Jan 20 2016
## Committer base changes:
- Currently 20 committers.
- No new committers added in the last 3 months
- Last committer addition was Ippokratis Pandis at Thu Feb 18 2016
## Releases:
- No releases yet.
## JIRA activity:
- 71 JIRA tickets created in the last 3 months
- 40 JIRA tickets closed/resolved in the last 3 months
On Sun, Jul 10, 2016 at 10:07 PM, Wes McKinney <wesmck...@gmail.com> wrote:
hi Jacques,
I would mention there's been a significant synergy between C++ efforts
in Apache Parquet to build a bridge between the projects (which will
be a nice proof-of-concept of the benefits of the common in-memory
columnar layer). Uwe Korn has been really active here with 33 commits
to Parquet and 21 to Arrow.
Micah Kornfield has been contributing significant to the reification
process of the specs and IPC / memory sharing procedure. We should
prioritize assembling a more fully formed first-cut metadata spec and
getting rudimentary integration tests working between the Java and C++
implementations before we make a release.
- Wes
On Sun, Jul 10, 2016 at 10:00 PM, Jacques Nadeau <jacq...@apache.org>
wrote:
Hello All,
Can I get some feedback and additional details to add to the Arrow Board
Report. See my draft below:
thanks,
Jacques
## Description:
Arrow is a columnar in-memory analytics layer designed to accelerate big
data.
It houses a set of canonical in-memory representations of flat and
hierarchical data along with multiple language-bindings for structure
manipulation. It also provides IPC and common algorithm implementations.
## Issues:
- There are no issues requiring board attention at this time.
## Activity:
- Awareness continues to increase with the community having done
presentations
at various meetups as well as the following conferences: Pydata Paris,
Hadoop
Summit Ireland, Hadoop Summit San Jose and Berlin Buzzwords.
- The CPP and Python work has seen steady progress and development of a
Parquet <> Arrow interchange layer is improving. A small number of
people
are
actively moving that forward.
- Java development has slowed some but appears to be picking up again.
## Health report:
- Discussion and code activity has been sporadic. We saw a good initial
flurry
of activity but need to make the project more approachable for new
users.
We
can do this by doing the following:
- We need to get a first release done.
- We need to put together a quickstart and demo application for new
users.
## PMC changes:
- Currently 17 PMC members.
- No new PMC members added in the last 3 months.
- Last PMC addition was Abdel Hakim Deneche on Tue Jan 20 2016
## Committer base changes:
- Currently 20 committers.
- No new committers added in the last 3 months
- Last committer addition was Ippokratis Pandis at Thu Feb 18 2016
## Releases:
- No releases yet.
## JIRA activity:
- 71 JIRA tickets created in the last 3 months
- 40 JIRA tickets closed/resolved in the last 3 months
