Re: What's the proper procedure to publish a docker image to dockerhub?

It is not in compliance to publish any official Apache artifacts unless the PMC has a release vote on them (or the source artifact that produces them). You are free to publish a Docker image under a non-Apache dockerhub account of course. Wes On Tue, Apr 16, 2019, 10:10 AM Micah Kornfield

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

To summarize my understanding of the thread so far, there seems to be consensus on having a new distinct type for each "large" type. There are some reservations around the "large" types being harder to support in algorithmic implementations. I'm curious Philipp, was there a concrete use-case

What's the proper procedure to publish a docker image to dockerhub?

I'm not sure the policy here but I think if this something official then the PMC would have to set it up and control it. Could someone on the PMC chime in? On Monday, April 15, 2019, Zhiyuan Zheng wrote: > Thanks Alberto! > > If we are able to create an official repository solely for Apache

Re: What's the proper procedure to publish a docker image to dockerhub?

Thanks Alberto! If we are able to create an official repository solely for Apache Arrow, it's more flexible to publish new images in future. How to create such a repository ? 16.04.2019, 01:27, "Alberto Ramón" : > Hello Zhiyuan > > I can help you if you need help with this process > The best

[jira] [Created] (ARROW-5171) [C++] Use LESS instead of LOWER in compare enum option.

Francois Saint-Jacques created ARROW-5171: - Summary: [C++] Use LESS instead of LOWER in compare enum option. Key: ARROW-5171 URL: https://issues.apache.org/jira/browse/ARROW-5171 Project:

Re: ARROW-3191: Making ArrowBuf work with arbitrary memory

I believe reader/writer indexes are typically used when we send buffers over the wire -- so may not be necessary for all users of ArrowBuf. I am okay with the idea of providing a simple wrapper to ArrowBuf to manage the reader/writer indexes with a couple of APIs. Note that some APIs like

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

I am not Jacques, but I will try to give my own point of view on this. > Thanks for making me laugh :) I think that this is unavoidable. Even with batches, taking an example of a > binary column where the mean size of the payload is 1mb, it limits to > batches of 2048 elements. This can become

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

Thanks for the clarification Antoine, very insightful. I'd also vote for keeping the existing model for consistency. On Mon, Apr 15, 2019 at 1:40 PM Antoine Pitrou wrote: > > Hi, > > I am not Jacques, but I will try to give my own point of view on this. > > The distinction between logical and

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

Hi, I am not Jacques, but I will try to give my own point of view on this. The distinction between logical and physical types can be modelled in two different ways: 1) a physical type can denote several logical types, but a logical type can only have a single physical representation. This is

Re: What's the proper procedure to publish a docker image to dockerhub?

Hello Zhiyuan I can help you if you need help with this process The best option is request a offical repository for Apache Arrow Project (se are the ones that start with '_' Redis example On Mon, 15 Apr 2019 at 15:21, Zhiyuan Zheng wrote: > Hi, > > DataFusion

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

Hello, I would like understand where do we stand on logical types and physical types. As I understand, this proposal is for the physical representation. In the context of an execution engine, the concept of logical types becomes more important as two physical representation might have the same

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

I think that this is unavoidable. Even with batches, taking an example of a binary column where the mean size of the payload is 1mb, it limits to batches of 2048 elements. This can become annoying pretty quickly. François On Fri, Apr 12, 2019 at 11:15 PM Wes McKinney wrote: > Hi Jacques, > > I

What's the proper procedure to publish a docker image to dockerhub?

Hi, DataFusion is a component which is an in-memory query engine using Apache Arrow as the memory model. I have created a Dockerfile for DataFusion (https://issues.apache.org/jira/browse/ARROW-4467) for it. In order to help user to start using DataFusion for some simple real world use

[jira] [Created] (ARROW-5170) [Rust][Datafusion] Add datafusion-cli to the docker-compose setup

Krisztian Szucs created ARROW-5170: -- Summary: [Rust][Datafusion] Add datafusion-cli to the docker-compose setup Key: ARROW-5170 URL: https://issues.apache.org/jira/browse/ARROW-5170 Project: Apache

Re: [Python/C++] Help running HDFS integration test for C++/Python

Or: `make -f Makefile.docker run-hdfs-integration` dev/container/README.md is definitely outdated On Mon, Apr 15, 2019 at 9:09 AM Krisztián Szűcs wrote: > Hey Micah, > > Try the following [1]: > > export PYTHON_VERSION=3.6 > docker-compose build cpp > docker-compose build python >

Re: [Python/C++] Help running HDFS integration test for C++/Python

Hey Micah, Try the following [1]: export PYTHON_VERSION=3.6 docker-compose build cpp docker-compose build python docker-compose build hdfs-integration docker-compose run hdfs-integration [1]: https://github.com/apache/arrow/blob/master/docker-compose.yml#L444 On Mon, Apr 15, 2019 at 8:38 AM

Re: [Python/C++] Help running HDFS integration test for C++/Python

Thanks to a hint from Wes, I found the https://github.com/apache/arrow/tree/master/dev/container/README.md which I think I need to run through first before running the steps in the dev/README, I'll try this and update the documentation if this seems to work. On Sun, Apr 14, 2019 at 11:37 PM Micah

[Python/C++] Help running HDFS integration test for C++/Python

I'm trying to verify a PR [1] works with the HDFS integration test, but I am having trouble running docker compose. I am trying to follow the README.md in dev [2] but I am running into an error "pull access denied for arrow". The full command and error are pasted below. (I think there is also a