[jira] [Created] (ARROW-2737) [Plasma] Integrate TensorFlow Op with arrow packaging scripts

2018-06-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2737: - Summary: [Plasma] Integrate TensorFlow Op with arrow packaging scripts Key: ARROW-2737 URL: https://issues.apache.org/jira/browse/ARROW-2737 Project: Apache Arrow

[jira] [Created] (ARROW-2736) arrow-jdbc module is missing required files/dirs on GitHub

2018-06-24 Thread Aravind R Yarram (JIRA)
Aravind R Yarram created ARROW-2736: --- Summary: arrow-jdbc module is missing required files/dirs on GitHub Key: ARROW-2736 URL: https://issues.apache.org/jira/browse/ARROW-2736 Project: Apache Arrow

[jira] [Created] (ARROW-2735) arrow-java-root parent-pom module isn't made available on GitHub

2018-06-24 Thread Aravind R Yarram (JIRA)
Aravind R Yarram created ARROW-2735: --- Summary: arrow-java-root parent-pom module isn't made available on GitHub Key: ARROW-2735 URL: https://issues.apache.org/jira/browse/ARROW-2735 Project: Apache

Re: Gandiva Initiative

2018-06-24 Thread Uwe L. Korn
Hello Antoine, the LLVM API is an interesting point. I've been using PyArrow and Numba for quite a bit and this would definitely clash. A quick Google search did not reveal any workaround for this issue. In the other cases where we have such clashes, boost and jemalloc, the library itself

Re: Housing longer-term Arrow development, design, and roadmap documents

2018-06-24 Thread Uwe L. Korn
Hello, I would prefer Confluence over GitHub pages because I would hope that one can integrate the ASF JIRA via widgets into the wiki pages. The vast amount of issues should all be categorizable into some topic. Once these are triaged, they should pop up in the respective wiki pages that could

Re: [DISCUSS] Developing a standard memory layout for in-memory records / "row-oriented" data

2018-06-24 Thread Antoine Pitrou
Hi Wes, Le 24/06/2018 à 08:24, Wes McKinney a écrit : > > If this sounds interesting to the community, I could help to kickstart > a design process which would likely take a significant amount of time. > The requirements could be complex (i.e. we might want to support > variable-size record

Re: Housing longer-term Arrow development, design, and roadmap documents

2018-06-24 Thread Antoine Pitrou
Hi Wes, I wonder if GitHub wiki pages would be an easier-to-approach alternative? Regards Antoine. Le 24/06/2018 à 08:42, Wes McKinney a écrit : > hi folks, > > Since the scope of Apache Arrow has grown significantly in the last > 2.5 years to encompass many programming languages and new

Re: Gandiva Initiative

2018-06-24 Thread Antoine Pitrou
Hi, I think JIT-compiling of kernels operating on Arrow data is an important development path, but just for the record, LLVM doesn't have a stable C++ API (the API changes at each feature release). Just something to keep a mind for the ensuing packaging discussions ;-) (it also raises

Re: Gandiva Initiative

2018-06-24 Thread Phillip Cloud
This is super exciting. In particular, I think for ibis ( http://docs.ibis-project.org/) building up expressions and executing them using gandiva would fit nicely as another in-memory backend alongside the pandas backend. I think it would also drive some use cases forward for more complex datatype

[jira] [Created] (ARROW-2734) Cython api example doesn't work by default on macOS

2018-06-24 Thread Jonathan Chambers (JIRA)
Jonathan Chambers created ARROW-2734: Summary: Cython api example doesn't work by default on macOS Key: ARROW-2734 URL: https://issues.apache.org/jira/browse/ARROW-2734 Project: Apache Arrow

Housing longer-term Arrow development, design, and roadmap documents

2018-06-24 Thread Wes McKinney
hi folks, Since the scope of Apache Arrow has grown significantly in the last 2.5 years to encompass many programming languages and new areas of functionality, I'd like to discuss how we could better accommodate longer-term asynchronous discussions and stay organized about the development

[DISCUSS] Developing a standard memory layout for in-memory records / "row-oriented" data

2018-06-24 Thread Wes McKinney
hi folks, Some time ago I opened ARROW-1790 based on some discussions I'd had with users on mailing list or in person about how to deal with data similar to a C array of struct types. Indeed, while we have Structs in the Arrow columnar format, our structs are "fully shredded" columnar structs.