[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-23 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714988857 Revision: d26b20cb64debc6f022fdce1cde3ea3f812981ad Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #8513: ARROW-10368 [Rust] [DataFusion] Refactor scan nodes to allow extensions

2020-10-23 Thread GitBox
github-actions[bot] commented on pull request #8513: URL: https://github.com/apache/arrow/pull/8513#issuecomment-715255711 https://issues.apache.org/jira/browse/ARROW-10368 This is an automated message from the Apache Git

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-23 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714984695 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] rdettai opened a new pull request #8513: ARROW-10368 [Rust] [DataFusion] Refactor scan nodes to allow extensions

2020-10-23 Thread GitBox
rdettai opened a new pull request #8513: URL: https://github.com/apache/arrow/pull/8513 Replace all specific XxxScan nodes with a generic SourceScan that dynamically dispatches to any source implementation. The main goal is to make implementations of custom data sources possible

[GitHub] [arrow] raphaelauv opened a new issue #8514: Pyarraw V2 no release note

2020-10-23 Thread GitBox
raphaelauv opened a new issue #8514: URL: https://github.com/apache/arrow/issues/8514 Hello , the pypi version of pyarraw is 2.0.0 since Oct 19, 2020 ( previous was 1.0.1 ) I could not find any release note explaining the changes and breaking changes. Do you know where I can

[GitHub] [arrow] rdettai commented on pull request #8513: ARROW-10368 [Rust] [DataFusion] Refactor scan nodes to allow extensions

2020-10-23 Thread GitBox
rdettai commented on pull request #8513: URL: https://github.com/apache/arrow/pull/8513#issuecomment-715257359 @jorgecarleitao I have implemented the basic structure, with a `SourceScanner` trait proposal. I have shown how the `SourceScan` logical plan is mapped to an execution by just

[GitHub] [arrow] jorgecarleitao closed pull request #8512: ARROW-10375 [Rust] Removed PrimitiveArrayOps

2020-10-23 Thread GitBox
jorgecarleitao closed pull request #8512: URL: https://github.com/apache/arrow/pull/8512 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] rdettai commented on pull request #8513: ARROW-10368 [Rust] [DataFusion] Refactor scan nodes to allow extensions

2020-10-23 Thread GitBox
rdettai commented on pull request #8513: URL: https://github.com/apache/arrow/pull/8513#issuecomment-715378341 I just found out something! It seems that datafusion is full of interesting mysteries  ! You actually already have the abstractions required to do what I want, but they are a

[GitHub] [arrow] terencehonles edited a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-23 Thread GitBox
terencehonles edited a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-715425280 > Revision: [d26b20c](https://github.com/apache/arrow/commit/d26b20cb64debc6f022fdce1cde3ea3f812981ad) > > Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-23 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-715425280 > Revision: [d26b20c](https://github.com/apache/arrow/commit/d26b20cb64debc6f022fdce1cde3ea3f812981ad) > > Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] rok commented on a change in pull request #8510: ARROW-1614: [C++] Add a Tensor logical value type with constant dimensions, implemented using ExtensionType

2020-10-23 Thread GitBox
rok commented on a change in pull request #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r510879292 ## File path: python/pyarrow/tests/test_extension_type.py ## @@ -76,6 +76,95 @@ def __reduce__(self): return MyListType, (self.storage_type,) +def

[GitHub] [arrow] naman1996 commented on pull request #8497: removing llvm@10 test branch

2020-10-23 Thread GitBox
naman1996 commented on pull request #8497: URL: https://github.com/apache/arrow/pull/8497#issuecomment-715391963 Sure closing this . The attempt was to fix the gandiva build issue. If that is taken care of in that PR This

[GitHub] [arrow] naman1996 closed pull request #8497: removing llvm@10 test branch

2020-10-23 Thread GitBox
naman1996 closed pull request #8497: URL: https://github.com/apache/arrow/pull/8497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] chrish42 commented on pull request #7110: ARROW-8952: [C++] WIP Support for textual, JSON schema representation

2020-10-23 Thread GitBox
chrish42 commented on pull request #7110: URL: https://github.com/apache/arrow/pull/7110#issuecomment-715368877 @emkornfield Ping, as requested.  Thanks! It's really not a lot of code. too. (Since Flatbuffers is doing the JSON <-> Flatbuffer part.)

[GitHub] [arrow] wesm commented on issue #8514: Pyarraw V2 no release note

2020-10-23 Thread GitBox
wesm commented on issue #8514: URL: https://github.com/apache/arrow/issues/8514#issuecomment-715400893 Keep in mind that the major version bump does not indicate that there *are* breaking API changes but that there *may be*

[GitHub] [arrow] carols10cents commented on pull request #8402: ARROW-8426: [Rust] [Parquet] - Add more support for converting Dicts

2020-10-23 Thread GitBox
carols10cents commented on pull request #8402: URL: https://github.com/apache/arrow/pull/8402#issuecomment-715412380 > @carols10cents -- one idea I had which might be less efficient at runtime but possibly be less complicated to implement, would be to use the arrow `cast` kernels here:

[GitHub] [arrow] alamb commented on pull request #8402: ARROW-8426: [Rust] [Parquet] - Add more support for converting Dicts

2020-10-23 Thread GitBox
alamb commented on pull request #8402: URL: https://github.com/apache/arrow/pull/8402#issuecomment-715419232 > Not really, because I am using the `cast` kernels in the `Converter`: [`4b59fc9`

[GitHub] [arrow] alamb commented on a change in pull request #8503: ARROW-10366: [Rust] [DataFusion] Remove collect from within threads on merge

2020-10-23 Thread GitBox
alamb commented on a change in pull request #8503: URL: https://github.com/apache/arrow/pull/8503#discussion_r510851478 ## File path: rust/datafusion/src/physical_plan/merge.rs ## @@ -103,37 +105,56 @@ impl ExecutionPlan for MergeExec {

[GitHub] [arrow] alamb commented on pull request #8402: ARROW-8426: [Rust] [Parquet] - Add more support for converting Dicts

2020-10-23 Thread GitBox
alamb commented on pull request #8402: URL: https://github.com/apache/arrow/pull/8402#issuecomment-715396915 > @vertexclique @nevi-me I'm feeling stuck on converting primitive dictionaries... > > I have [a solution that works for one key/value

[GitHub] [arrow] wesm commented on issue #8514: Pyarraw V2 no release note

2020-10-23 Thread GitBox
wesm commented on issue #8514: URL: https://github.com/apache/arrow/issues/8514#issuecomment-715400236 We'll publish a blog post today probably about the release. You can look at the changelog in the root of the git repository int he meantime.

[GitHub] [arrow] wesm closed issue #8514: Pyarraw V2 no release note

2020-10-23 Thread GitBox
wesm closed issue #8514: URL: https://github.com/apache/arrow/issues/8514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] emkornfield commented on pull request #7110: ARROW-8952: [C++] WIP Support for textual, JSON schema representation

2020-10-23 Thread GitBox
emkornfield commented on pull request #7110: URL: https://github.com/apache/arrow/pull/7110#issuecomment-715450006 thanks for the reminder I should be able to look tonight/over the weekend. This is an automated message from

[GitHub] [arrow] nealrichardson commented on pull request #8371: ARROW-4960: [R] Build r-arrow conda package in crossbow

2020-10-23 Thread GitBox
nealrichardson commented on pull request #8371: URL: https://github.com/apache/arrow/pull/8371#issuecomment-715463399 @xhochy what's the status of this? Can we get this merged soon? This is an automated message from the

[GitHub] [arrow] emkornfield opened a new pull request #8516: PARQUET-1935: [C++] Fix bug in WriteBatchSpaced

2020-10-23 Thread GitBox
emkornfield opened a new pull request #8516: URL: https://github.com/apache/arrow/pull/8516 The arrow code path doesn't call this method of not-null columns with no null parents, so this wasn't caught in arrow tests. The only time we can pass through counts is if definition level

[GitHub] [arrow] nevi-me commented on a change in pull request #8401: ARROW-10109: [Rust] Add support to the C data interface for primitive types and utf8

2020-10-23 Thread GitBox
nevi-me commented on a change in pull request #8401: URL: https://github.com/apache/arrow/pull/8401#discussion_r511263089 ## File path: rust/arrow-c-integration/Cargo.toml ## @@ -0,0 +1,38 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [arrow] github-actions[bot] commented on pull request #8516: PARQUET-1935: [C++] Fix bug in WriteBatchSpaced

2020-10-23 Thread GitBox
github-actions[bot] commented on pull request #8516: URL: https://github.com/apache/arrow/pull/8516#issuecomment-715666118 https://issues.apache.org/jira/browse/PARQUET-1935 This is an automated message from the Apache Git

[GitHub] [arrow] carols10cents commented on pull request #8405: One definition/repetition level test

2020-10-23 Thread GitBox
carols10cents commented on pull request #8405: URL: https://github.com/apache/arrow/pull/8405#issuecomment-715588688 @nevi-me So @shepmaster and I wrote some more tests, but they're failing and we're not sure if our setup is wrong or if they're expected to fail? What do you think?

[GitHub] [arrow] carols10cents opened a new pull request #8515: [Rust] [Parquet] Start porting path_internal from C++ to handle def_levels and rep_levels

2020-10-23 Thread GitBox
carols10cents opened a new pull request #8515: URL: https://github.com/apache/arrow/pull/8515 Hey @nevi-me, I was looking into helping out with the `def_levels` and `rep_levels` handling in `get_levels` in arrow_writer.rs, and the logic is... quite complex! I honestly have no idea how

[GitHub] [arrow] github-actions[bot] commented on pull request #8515: [Rust] [Parquet] Start porting path_internal from C++ to handle def_levels and rep_levels

2020-10-23 Thread GitBox
github-actions[bot] commented on pull request #8515: URL: https://github.com/apache/arrow/pull/8515#issuecomment-715598067 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] jorgecarleitao opened a new pull request #8512: ARROW-10375 [Rust] Removed PrimitiveArrayOps

2020-10-23 Thread GitBox
jorgecarleitao opened a new pull request #8512: URL: https://github.com/apache/arrow/pull/8512 A simplification to the Arrow crate, since it is no longer needed, now that we no longer have specialization. This is an

[GitHub] [arrow] github-actions[bot] commented on pull request #8512: ARROW-10375 [Rust] Removed PrimitiveArrayOps

2020-10-23 Thread GitBox
github-actions[bot] commented on pull request #8512: URL: https://github.com/apache/arrow/pull/8512#issuecomment-714935953 https://issues.apache.org/jira/browse/ARROW-10375 This is an automated message from the Apache Git