Notes from Today's Sync Call:
# Attendees
- Wes McKinney (Ursa Labs)
- Jacque Nadeau - Dremio
- Ben Keitzman (Ursa Labs)
- Uwe Korn
- Praveen Kumar - Dremio
- Francois Saint-Jacques
- Hatem Helal (MathWorks)
- Nick Haddad (MathWorks)
# Agenda Items
- Qualify MATLAB code in Arrow Travis CI
- Wes: In principal its not on the list. If it is only editing the
travis.yaml
- asf-infra is the way to do the signing of the paperwork. We aren't able
to deals with this on behalf of the
- Jacques: If it is commercial Apache might not want it.
- Wes: I had inquired about getting several MATLAB licenses.
- Nick: will follow-up after about getting these
- Jacques and Uwe: and Wes Press Release for 3 year anniversary
- Jacques: I'm happy to sketch something out and send this around.,
- Want to do a very high outline of community contributions, bindings,
and integrations into other projects along with supporting quotes from people
in the community and the industry.
- Open to any feedback and ideas.
- Wes: Notable that the project started out with a small focus on
designing an open standard for structured data and 3 years later the project is
significantly broader than that. Themes:
- Open standard has been accepted and is widely adopted
- Project has grown in many directions.
- 9 code donations into the project. Notable among apache projects.
- Wes: Any download or usage numbers would be useful
- Jacques: Estimates are around a million downloads a month.
- Wes: Notable as the python library is being taken on as a dependency
on other projects. Recently saw that Apache Beam is taking this as a
dependency. Things like this are driving install numbers.
- Jacques: This to me is a continuing sign of success.
- Uwe: I can spend an hour or two. I can contribute 1 or 2 sections.
We can also add the images from the different engines we support (Spark, etc).
- Wes: Things in flight right now and talk about the 0.13 release
- About halfway through issues around 0.13.
- 0.13 Window for patches will close by mid-March, 5 weeks from now.
- If there is anything you definitely want in the release add the fix
version on Jira.
- Arrow Flight
- Work has picked up on Arrow Flight. Will be working on building this
out. Would be helpful to get feedback on potential users for flight. This is
a general purpose messaging system. Uses GRPC but could use other mechanisms
like RDMA. For anyone who is building database protocols or making data
available with any kind of a data service should give feedback.
- With Flight and Gandiva, the addition of these to the project are adding
to the wheel.
- Wes: How big are the linux wheels for these with Gandiva? 25MB?
- Uwe: The normal wheels are 30MB so with Gandiva it would be at least
10MB more.
- Wes: Will look into the size of the flight library.
- Wes: Should we produce multiple wheel for multiple uses?
- Uwe: Yes that is a good idea.
- Wes: This comes at a good time because GRPC is gaining rapid adoption.
- Jacques: You can think of this as a replacement for JDBC and ODBC
- If ReST is the way to communicate between micro services, then
Flight is the way to communicate between data micro services.
- Also gives you parallel streaming.
-----Original Message-----
From: Wes McKinney <[email protected]>
Sent: Tuesday, February 5, 2019 12:44 PM
To: [email protected]
Subject: Arrow sync call Wednesday 17:00 UTC / 12p US-Eastern
As usual we will meet at https://meet.google.com/vtm-teks-phx
If anyone would like to be added to the recurring calendar invite, please let
me know and I'll add you
