[GitHub] [arrow] kou commented on pull request #7085: ARROW-8668: [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packages

2020-05-01 Thread GitBox
kou commented on pull request #7085: URL: https://github.com/apache/arrow/pull/7085#issuecomment-622562008 @github-actions crossbow submit -g linux This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] wesm commented on issue #7082: pyarrow 0.17 atexit handler causes a segmentation fault

2020-05-01 Thread GitBox
wesm commented on issue #7082: URL: https://github.com/apache/arrow/issues/7082#issuecomment-622562649 Please also indicate which exact version of Python you're using. From searching the internet, it seems that there was a bug fix in CPython that may affect older versions of Python 3.6 or

[GitHub] [arrow] github-actions[bot] commented on pull request #7085: ARROW-8668: [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packages

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7085: URL: https://github.com/apache/arrow/pull/7085#issuecomment-622562531 Revision: dc6d0b00d74ffd92327622cd3cff8d454b934ac4 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] zgramana commented on pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
zgramana commented on pull request #7032: URL: https://github.com/apache/arrow/pull/7032#issuecomment-622566445 Thanks @eerhardt. This was a great introduction into the internals, and happy I could contribute back something useful.

[GitHub] [arrow] tustvold commented on a change in pull request #7076: ARROW-8659: [Rust] ListBuilder allocate with_capacity

2020-05-01 Thread GitBox
tustvold commented on a change in pull request #7076: URL: https://github.com/apache/arrow/pull/7076#discussion_r418774678 ## File path: rust/parquet/src/arrow/converter.rs ## @@ -128,7 +128,10 @@ pub struct Utf8ArrayConverter {} impl Converter>, StringArray> for

[GitHub] [arrow] yordan-pavlov commented on pull request #7037: ARROW-6718: [DRAFT] [Rust] Remove packed_simd

2020-05-02 Thread GitBox
yordan-pavlov commented on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-622947442 @nevi-me I have only focused on filtering so far; I will probably implement benchmarks for other operations once I have fully explored filtering.

[GitHub] [arrow] nevi-me commented on pull request #7057: ARROW-8616: [Rust] Turn explicit SIMD off by default

2020-05-02 Thread GitBox
nevi-me commented on pull request #7057: URL: https://github.com/apache/arrow/pull/7057#issuecomment-622927396 > LGTM subject to fixing CI failure We have to rewrite the `divide` kernel for tets to pass. I've done this as part of #7037 by creating a slower path if the LHS array has

[GitHub] [arrow] yordan-pavlov commented on pull request #7037: ARROW-6718: [DRAFT] [Rust] Remove packed_simd

2020-05-02 Thread GitBox
yordan-pavlov commented on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-622928732 hi @nevi-me , I would love to have arrow use stable rust. (but I also want high performance) I have been working on some benchmarks which compare performance of

[GitHub] [arrow] nevi-me commented on pull request #7018: ARROW-8536: [Rust] [Flight] Check in proto file, conditional build if file exists

2020-05-02 Thread GitBox
nevi-me commented on pull request #7018: URL: https://github.com/apache/arrow/pull/7018#issuecomment-622927093 @andygrove @nealrichardson I've added the warning. I didn't have a good experience when trying to modify multiple files in Rust (when I was attempting creating a `build.rs`

[GitHub] [arrow] nevi-me commented on pull request #7037: ARROW-6718: [DRAFT] [Rust] Remove packed_simd

2020-05-02 Thread GitBox
nevi-me commented on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-622931053 That's interesting, and how are other benchmarks affected, or are you only focusing on filter? To the extent that there's overlap with `take()`, it would be interesting to also

[GitHub] [arrow] gramirezespinoza commented on pull request #7039: ARROW-8513: [Python] Expose Take with Table input in Python

2020-05-02 Thread GitBox
gramirezespinoza commented on pull request #7039: URL: https://github.com/apache/arrow/pull/7039#issuecomment-622946644 Hey, I did open a JIRA account. My username is `gire`. I'm one of the watchers in the issue ARROW-8513.

[GitHub] [arrow] vertexclique commented on a change in pull request #7061: ARROW-8629: [Rust] Eliminate indirection of zero sized allocations

2020-05-02 Thread GitBox
vertexclique commented on a change in pull request #7061: URL: https://github.com/apache/arrow/pull/7061#discussion_r418999354 ## File path: rust/arrow/src/util/bit_util.rs ## @@ -148,11 +148,17 @@ pub fn count_set_bits_offset(data: &[u8], offset: usize, length: usize) ->

[GitHub] [arrow] vertexclique commented on a change in pull request #7061: ARROW-8629: [Rust] Eliminate indirection of zero sized allocations

2020-05-02 Thread GitBox
vertexclique commented on a change in pull request #7061: URL: https://github.com/apache/arrow/pull/7061#discussion_r419000857 ## File path: rust/arrow/src/memory.rs ## @@ -20,34 +20,56 @@ use std::alloc::Layout; use std::mem::align_of; +use std::ptr::NonNull; pub const

[GitHub] [arrow] vertexclique commented on a change in pull request #7064: ARROW-6945: [Rust] WIP: Add initial skeleton for Rust integration tests

2020-05-02 Thread GitBox
vertexclique commented on a change in pull request #7064: URL: https://github.com/apache/arrow/pull/7064#discussion_r419001113 ## File path: rust/arrow/Cargo.toml ## @@ -50,6 +50,7 @@ chrono = "0.4" flatbuffers = "0.6" hex = "0.4" arrow-flight = { path = "../arrow-flight",

[GitHub] [arrow] vertexclique commented on a change in pull request #7061: ARROW-8629: [Rust] Eliminate indirection of zero sized allocations

2020-05-02 Thread GitBox
vertexclique commented on a change in pull request #7061: URL: https://github.com/apache/arrow/pull/7061#discussion_r419000857 ## File path: rust/arrow/src/memory.rs ## @@ -20,34 +20,56 @@ use std::alloc::Layout; use std::mem::align_of; +use std::ptr::NonNull; pub const

[GitHub] [arrow] kou commented on pull request #7085: ARROW-8668: [Packaging][APT][Yum][ARM] Use Travis CI's ARM machine to build packages

2020-05-02 Thread GitBox
kou commented on pull request #7085: URL: https://github.com/apache/arrow/pull/7085#issuecomment-623008857 +1 Most builds are about 5m faster than before. This is an automated message from the Apache Git Service. To

[GitHub] [arrow] paddyhoran commented on pull request #7037: ARROW-6718: [DRAFT] [Rust] Remove packed_simd

2020-05-02 Thread GitBox
paddyhoran commented on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-622984619 > Yes, SIMD is a lot of work, but at the moment gives the best performance. I wonder if the SIMD features could be moved to another library separate from the core arrow

[GitHub] [arrow] paddyhoran commented on pull request #7004: ARROW-3827: [Rust] Implement UnionArray Updated

2020-05-02 Thread GitBox
paddyhoran commented on pull request #7004: URL: https://github.com/apache/arrow/pull/7004#issuecomment-622984848 @andygrove @nevi-me should I break this into smaller PR's to make it easier to get merged? This is an

[GitHub] [arrow] vertexclique commented on a change in pull request #6898: ARROW-8399: [Rust] Extend memory alignments to include other architectures

2020-05-02 Thread GitBox
vertexclique commented on a change in pull request #6898: URL: https://github.com/apache/arrow/pull/6898#discussion_r418994231 ## File path: rust/arrow/src/memory.rs ## @@ -21,7 +21,58 @@ use std::alloc::Layout; use std::mem::align_of; -pub const ALIGNMENT: usize = 64;

[GitHub] [arrow] github-actions[bot] commented on pull request #7091: ARROW-8680: [Rust] Fix ComplexObjectArray null value shifting

2020-05-03 Thread GitBox
github-actions[bot] commented on pull request #7091: URL: https://github.com/apache/arrow/pull/7091#issuecomment-623132413 https://issues.apache.org/jira/browse/ARROW-8680 This is an automated message from the Apache Git

[GitHub] [arrow] tustvold opened a new pull request #7091: ARROW-8680: [Rust] Fix ComplexObjectArray null value shifting

2020-05-03 Thread GitBox
tustvold opened a new pull request #7091: URL: https://github.com/apache/arrow/pull/7091 The null shifting logic within ComplexObjectArrayReader is incorrect as it doesn't take into account the num_readers offset within the def_levels buffer. This occurs when a read spans more than one

[GitHub] [arrow] kou opened a new pull request #7092: ARROW-8682: [Ruby][Parquet] Add support for column level compression

2020-05-03 Thread GitBox
kou opened a new pull request #7092: URL: https://github.com/apache/arrow/pull/7092 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] github-actions[bot] commented on pull request #7092: ARROW-8682: [Ruby][Parquet] Add support for column level compression

2020-05-03 Thread GitBox
github-actions[bot] commented on pull request #7092: URL: https://github.com/apache/arrow/pull/7092#issuecomment-623183601 https://issues.apache.org/jira/browse/ARROW-8682 This is an automated message from the Apache Git

[GitHub] [arrow] kou opened a new pull request #7087: ARROW-8669: [C++] Add IpcWriteOptions argument to GetRecordBatchSize()

2020-05-02 Thread GitBox
kou opened a new pull request #7087: URL: https://github.com/apache/arrow/pull/7087 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] wesm commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-02 Thread GitBox
wesm commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-623026632 We could also use `BUNDLED` builds for the dependencies of interest (might also be nice to turn on ARROW_MIMALLOC while we are at it). We'd need to turn off Gandiva, though

[GitHub] [arrow] wesm edited a comment on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-02 Thread GitBox
wesm edited a comment on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-623026632 We could also use `BUNDLED` builds for the dependencies of interest (might also be nice to turn on ARROW_MIMALLOC while we are at it). We'd need to turn off Gandiva (and

[GitHub] [arrow] wesm opened a new pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-02 Thread GitBox
wesm opened a new pull request #7088: URL: https://github.com/apache/arrow/pull/7088 This builds on the work from #6631 while adding unit tests and additional benchmarks. I also renamed arrow/util/parsing.h to arrow/util/value_parsing.h to make it slightly more discoverable.

[GitHub] [arrow] wesm commented on pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-02 Thread GitBox
wesm commented on pull request #7088: URL: https://github.com/apache/arrow/pull/7088#issuecomment-623028582 @prutskov if it's helpful, you might take a look at the changes I made to the benchmark code that you contributed to see idiomatic use of `Result` and some other things (such as

[GitHub] [arrow] wesm commented on pull request #6631: ARROW-8111: [C++][CSV] Support MM/DD/YYYY date format

2020-05-02 Thread GitBox
wesm commented on pull request #6631: URL: https://github.com/apache/arrow/pull/6631#issuecomment-623028612 Closing in favor of #7088 so we can have a fresh round of code review This is an automated message from the Apache

[GitHub] [arrow] wesm commented on pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-02 Thread GitBox
wesm commented on pull request #7088: URL: https://github.com/apache/arrow/pull/7088#issuecomment-623031355 Hm unfortunately it seems that Howard Hinnant's date library does not fully support gcc < 7 -- there are issues with `date::parse` ``` /usr/bin/ccache /usr/bin/g++-4.8

[GitHub] [arrow] mcassels commented on a change in pull request #6770: ARROW-7842: [Rust] [Parquet] implement array_reader for list type columns

2020-05-03 Thread GitBox
mcassels commented on a change in pull request #6770: URL: https://github.com/apache/arrow/pull/6770#discussion_r419189845 ## File path: rust/parquet/src/arrow/array_reader.rs ## @@ -468,6 +491,391 @@ where } } +/// Implementation of list array reader. +pub struct

[GitHub] [arrow] mcassels commented on a change in pull request #6770: ARROW-7842: [Rust] [Parquet] implement array_reader for list type columns

2020-05-03 Thread GitBox
mcassels commented on a change in pull request #6770: URL: https://github.com/apache/arrow/pull/6770#discussion_r419189845 ## File path: rust/parquet/src/arrow/array_reader.rs ## @@ -468,6 +491,391 @@ where } } +/// Implementation of list array reader. +pub struct

[GitHub] [arrow] emkornfield commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-03 Thread GitBox
emkornfield commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419200568 ## File path: java/flight/flight-core/src/test/java/org/apache/arrow/flight/TestDoExchange.java ## @@ -0,0 +1,395 @@ +/* + * Licensed to the Apache

[GitHub] [arrow] emkornfield commented on pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-03 Thread GitBox
emkornfield commented on pull request #7012: URL: https://github.com/apache/arrow/pull/7012#issuecomment-623240480 Thanks, I think the tests are clearer now, left a few more comments. @jacques-n do you want to review? Otherwise, I'll take one more review pass later in the week (but I

[GitHub] [arrow] emkornfield commented on pull request #7029: ARROW-8579 [C++] Add AVX512 SIMD for spaced decoding and encoding.

2020-05-03 Thread GitBox
emkornfield commented on pull request #7029: URL: https://github.com/apache/arrow/pull/7029#issuecomment-623237943 Sorry for the late reply. Might as well append it to this PR. This is an automated message from the Apache

[GitHub] [arrow] emkornfield commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-03 Thread GitBox
emkornfield commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419200128 ## File path: java/flight/flight-core/src/test/java/org/apache/arrow/flight/TestDoExchange.java ## @@ -0,0 +1,407 @@ +/* + * Licensed to the Apache

[GitHub] [arrow] emkornfield commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-03 Thread GitBox
emkornfield commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419200063 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightClient.java ## @@ -293,6 +292,76 @@ public void onCompleted() {

[GitHub] [arrow] emkornfield commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-05-03 Thread GitBox
emkornfield commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-623249922 The separate .cc files might work as, we should probably cleanup macro naming so we can distinguish between min and max SSE instruction se to use (I imagine some people won't

[GitHub] [arrow] siddharthteotia commented on pull request #6729: ARROW-8229: [Java] Move ArrowBuf into the Arrow package

2020-05-03 Thread GitBox
siddharthteotia commented on pull request #6729: URL: https://github.com/apache/arrow/pull/6729#issuecomment-623251813 It will be good to link the related issues in the PR description. This is an automated message from the

[GitHub] [arrow] siddharthteotia commented on pull request #5930: ARROW-3495: [Java] Move validityBuffer to BaseValueVector, delete the optim…

2020-05-03 Thread GitBox
siddharthteotia commented on pull request #5930: URL: https://github.com/apache/arrow/pull/5930#issuecomment-623253871 Closing based on latest comments This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] wesm commented on issue #7082: pyarrow 0.17 atexit handler causes a segmentation fault

2020-05-03 Thread GitBox
wesm commented on issue #7082: URL: https://github.com/apache/arrow/issues/7082#issuecomment-623220977 I'm guessing you're using macOS and at least Python 3.7.5. I just opened https://issues.apache.org/jira/browse/ARROW-8684 If you could confirm your OS and Python version that

[GitHub] [arrow] emkornfield commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-03 Thread GitBox
emkornfield commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419199838 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightService.java ## @@ -155,79 +152,35 @@ public void

[GitHub] [arrow] siddharthteotia commented on pull request #6433: ARROW-7495: [Java] Remove "empty" concept from ArrowBuf, replace with custom referencemanager

2020-05-03 Thread GitBox
siddharthteotia commented on pull request #6433: URL: https://github.com/apache/arrow/pull/6433#issuecomment-623253149 @tianchen92 , can you please revise thia? Looks like the work is completed This is an automated message

[GitHub] [arrow] emkornfield commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-05-03 Thread GitBox
emkornfield commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-623271321 @nealrichardson looks like a different error now? I think I probably want to use the scalar version for 32-bit builds. Is there an example in the code that provide the

[GitHub] [arrow] nevi-me commented on a change in pull request #6770: ARROW-7842: [Rust] [Parquet] implement array_reader for list type columns

2020-05-04 Thread GitBox
nevi-me commented on a change in pull request #6770: URL: https://github.com/apache/arrow/pull/6770#discussion_r419233839 ## File path: rust/parquet/src/arrow/array_reader.rs ## @@ -468,6 +491,391 @@ where } } +/// Implementation of list array reader. +pub struct

[GitHub] [arrow] hantusk commented on issue #7082: pyarrow 0.17 atexit handler causes a segmentation fault

2020-05-04 Thread GitBox
hantusk commented on issue #7082: URL: https://github.com/apache/arrow/issues/7082#issuecomment-623301347 Yes, macOS running python 3.7.5 or 3.7.7. I will try and reproduce and continue commenting in the JIRA issue. This is

[GitHub] [arrow] fsaintjacques commented on a change in pull request #7089: ARROW-8657: [C++][Python] Add separate configuration for data pages

2020-05-04 Thread GitBox
fsaintjacques commented on a change in pull request #7089: URL: https://github.com/apache/arrow/pull/7089#discussion_r419460990 ## File path: cpp/src/parquet/properties.h ## @@ -34,10 +34,14 @@ namespace parquet { +/// Control for data types in parquet. struct

[GitHub] [arrow] pitrou commented on pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
pitrou commented on pull request #7094: URL: https://github.com/apache/arrow/pull/7094#issuecomment-623488805 AppVeyor build: https://ci.appveyor.com/project/pitrou/arrow/builds/32634981 This is an automated message from the

[GitHub] [arrow] github-actions[bot] commented on pull request #7097: ARROW-8690: [Python] Clean-up dataset+parquet tests now order is determinstic

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7097: URL: https://github.com/apache/arrow/pull/7097#issuecomment-623457675 https://issues.apache.org/jira/browse/ARROW-8690 This is an automated message from the Apache Git

[GitHub] [arrow] wesm commented on a change in pull request #7089: ARROW-8657: [C++][Python] Add separate configuration for data pages

2020-05-04 Thread GitBox
wesm commented on a change in pull request #7089: URL: https://github.com/apache/arrow/pull/7089#discussion_r419458260 ## File path: cpp/src/parquet/properties.h ## @@ -34,10 +34,14 @@ namespace parquet { +/// Control for data types in parquet. struct ParquetVersion {

[GitHub] [arrow] pitrou opened a new pull request #7098: ARROW-8692: [C++] Avoid memory copies when downloading from S3

2020-05-04 Thread GitBox
pitrou opened a new pull request #7098: URL: https://github.com/apache/arrow/pull/7098 The AWS SDK creates a auto-growing StringStream by default, entailing multiple memory copies when transferring large data blocks (because of resizes). Instead, write directly into the target data area.

[GitHub] [arrow] wesm commented on a change in pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-04 Thread GitBox
wesm commented on a change in pull request #7088: URL: https://github.com/apache/arrow/pull/7088#discussion_r419459324 ## File path: cpp/src/arrow/csv/converter.cc ## @@ -381,32 +383,98 @@ class NumericConverter : public ConcreteConverter {

[GitHub] [arrow] pitrou edited a comment on pull request #7098: ARROW-8692: [C++] Avoid memory copies when downloading from S3

2020-05-04 Thread GitBox
pitrou edited a comment on pull request #7098: URL: https://github.com/apache/arrow/pull/7098#issuecomment-623507548 @lidavidm It would be nice if you could run the benchmarks and post numbers on your setup (perhaps on S3 too?).

[GitHub] [arrow] pitrou commented on pull request #7098: ARROW-8692: [C++] Avoid memory copies when downloading from S3

2020-05-04 Thread GitBox
pitrou commented on pull request #7098: URL: https://github.com/apache/arrow/pull/7098#issuecomment-623507548 @lidavidm It would be nice if you could run the benchmarks and post number on your setup (perhaps on S3 too?).

[GitHub] [arrow] github-actions[bot] removed a comment on pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
github-actions[bot] removed a comment on pull request #7021: URL: https://github.com/apache/arrow/pull/7021#issuecomment-618494095 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW

[GitHub] [arrow] jorisvandenbossche commented on pull request #7096: ARROW-8644: [Python] Restore ParquetDataset behaviour to always include partition column for dask compatibility

2020-05-04 Thread GitBox
jorisvandenbossche commented on pull request #7096: URL: https://github.com/apache/arrow/pull/7096#issuecomment-623433040 @github-actions crossbow submit test-conda-python-3.7-dask-latest test-conda-python-3.8-dask-master

[GitHub] [arrow] github-actions[bot] commented on pull request #7096: ARROW-8644: [Python] Restore ParquetDataset behaviour to always include partition column for dask compatibility

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7096: URL: https://github.com/apache/arrow/pull/7096#issuecomment-623433630 Revision: 3e480a91833c7cd401fa120c520e5a51dad2d58a Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419401740 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419413778 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] github-actions[bot] commented on pull request #7098: ARROW-8692: [C++] Avoid memory copies when downloading from S3

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7098: URL: https://github.com/apache/arrow/pull/7098#issuecomment-623499876 https://issues.apache.org/jira/browse/ARROW-8692 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419541129 ## File path: docs/source/example1.dat ## @@ -0,0 +1 @@ +some data Review comment: Nope.

[GitHub] [arrow] jorisvandenbossche opened a new pull request #7099: ARROW-8693: [Python] Insert implicit cast in Dataset.get_fragments with filter

2020-05-04 Thread GitBox
jorisvandenbossche opened a new pull request #7099: URL: https://github.com/apache/arrow/pull/7099 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] nealrichardson commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
nealrichardson commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419518977 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,197 @@ +.. raw:: html + + + +Running Docker Builds += + +Most

[GitHub] [arrow] wesm commented on a change in pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-04 Thread GitBox
wesm commented on a change in pull request #7088: URL: https://github.com/apache/arrow/pull/7088#discussion_r419459324 ## File path: cpp/src/arrow/csv/converter.cc ## @@ -381,32 +383,98 @@ class NumericConverter : public ConcreteConverter {

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419528025 ## File path: .github/workflows/archery.yml ## @@ -51,10 +53,12 @@ jobs: python-version: '3.7' - name: Install working-directory:

[GitHub] [arrow] pitrou commented on pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
pitrou commented on pull request #7094: URL: https://github.com/apache/arrow/pull/7094#issuecomment-623469326 Ok, rebasing. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow] wesm commented on pull request #7089: ARROW-8657: [C++][Python] Add separate configuration for data pages

2020-05-04 Thread GitBox
wesm commented on pull request #7089: URL: https://github.com/apache/arrow/pull/7089#issuecomment-623480713 Sorry fat-fingered the review request. I will take a look at this This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-05-04 Thread GitBox
nealrichardson commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-623520253 @emkornfield that looks like the same R Windows 32-bit failure to me. I'm not sure I understand your other question. Are you saying you want to use the old (status

[GitHub] [arrow] github-actions[bot] commented on pull request #7099: ARROW-8693: [Python] Insert implicit cast in Dataset.get_fragments with filter

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7099: URL: https://github.com/apache/arrow/pull/7099#issuecomment-623534789 https://issues.apache.org/jira/browse/ARROW-8693 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419538949 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,197 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] pitrou commented on a change in pull request #7088: ARROW-8111: [C++] User-defined timestamp parser option to CSV, new TimestampParser interface, and strptime-compatible impl

2020-05-04 Thread GitBox
pitrou commented on a change in pull request #7088: URL: https://github.com/apache/arrow/pull/7088#discussion_r419336015 ## File path: cpp/src/arrow/util/value_parsing.cc ## @@ -79,5 +86,46 @@ bool StringToFloatConverter::StringToFloat(const char* s, size_t length, double*

[GitHub] [arrow] rymurr commented on pull request #7093: ARROW-8687: [Java] Remove references to io.netty.buffer.ArrowBuf

2020-05-04 Thread GitBox
rymurr commented on pull request #7093: URL: https://github.com/apache/arrow/pull/7093#issuecomment-623426298 Thanks both! This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419339195 ## File path: .github/workflows/java.yml ## @@ -38,6 +38,8 @@ on: env: DOCKER_BUILDKIT: 0 COMPOSE_DOCKER_CLI_BUILD: 1 + ARCHERY_DOCKER_USER: ${{

[GitHub] [arrow] github-actions[bot] commented on pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7094: URL: https://github.com/apache/arrow/pull/7094#issuecomment-623422812 https://issues.apache.org/jira/browse/ARROW-8689 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #7095: ARROW-8664: [Java] Add flag to skip null check

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7095: URL: https://github.com/apache/arrow/pull/7095#issuecomment-623428933 https://issues.apache.org/jira/browse/ARROW-8664 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #7096: ARROW-8644: [Python] Restore ParquetDataset behaviour to always include partition column for dask compatibility

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7096: URL: https://github.com/apache/arrow/pull/7096#issuecomment-623428932 https://issues.apache.org/jira/browse/ARROW-8644 This is an automated message from the Apache Git

[GitHub] [arrow] pitrou commented on pull request #7081: [CI] Cache docker volumes [WIP]

2020-05-04 Thread GitBox
pitrou commented on pull request #7081: URL: https://github.com/apache/arrow/pull/7081#issuecomment-623367064 Did they increase the available cache size? Last I looked it was a fixed size for the entire repo. This is an

[GitHub] [arrow] rymurr commented on pull request #7084: ARROW-8664: [Java] Add flag to skip null check

2020-05-04 Thread GitBox
rymurr commented on pull request #7084: URL: https://github.com/apache/arrow/pull/7084#issuecomment-623343842 build is dependent on #7093 and rebase This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] mr-smidge commented on pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-04 Thread GitBox
mr-smidge commented on pull request #7032: URL: https://github.com/apache/arrow/pull/7032#issuecomment-623353035 Hi @zgramana (and @eerhardt). I was independently working on nullable array builder support (but have not been able to contribute just yet as my organisation needs to sign a

[GitHub] [arrow] pitrou commented on pull request #7029: ARROW-8579 [C++] Add AVX512 SIMD for spaced decoding and encoding.

2020-05-04 Thread GitBox
pitrou commented on pull request #7029: URL: https://github.com/apache/arrow/pull/7029#issuecomment-623370447 A general question: why is this limited to `sizeof(T) == 4` and `sizeof(T) == 8`? There are 8-bit and 16-bit types as well.

[GitHub] [arrow] jorisvandenbossche opened a new pull request #7096: ARROW-8644: [Python] Restore ParquetDataset behaviour to always include partition column for dask compatibility

2020-05-04 Thread GitBox
jorisvandenbossche opened a new pull request #7096: URL: https://github.com/apache/arrow/pull/7096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] liyafan82 commented on pull request #6729: ARROW-8229: [Java] Move ArrowBuf into the Arrow package

2020-05-04 Thread GitBox
liyafan82 commented on pull request #6729: URL: https://github.com/apache/arrow/pull/6729#issuecomment-623427960 > It will be good to link the related issues in the PR description. @siddharthteotia Thanks a lot for your effort. I have updated the description.

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419400510 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419416278 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419416691 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] fsaintjacques commented on pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
fsaintjacques commented on pull request #7094: URL: https://github.com/apache/arrow/pull/7094#issuecomment-623434263 Java compilation errors were fixed in #7093 This is an automated message from the Apache Git Service. To

[GitHub] [arrow] jorisvandenbossche commented on pull request #7097: ARROW-8690: [Python] Clean-up dataset+parquet tests now order is determinstic

2020-05-04 Thread GitBox
jorisvandenbossche commented on pull request #7097: URL: https://github.com/apache/arrow/pull/7097#issuecomment-623445265 @github-actions crossbow submit -g python This is an automated message from the Apache Git Service. To

[GitHub] [arrow] lidavidm commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-04 Thread GitBox
lidavidm commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419412768 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightService.java ## @@ -155,79 +152,35 @@ public void

[GitHub] [arrow] github-actions[bot] commented on pull request #7097: ARROW-8690: [Python] Clean-up dataset+parquet tests now order is determinstic

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7097: URL: https://github.com/apache/arrow/pull/7097#issuecomment-623445892 Revision: 065dc03fc971c34c7d008283ef399b88939f8e98 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] lidavidm commented on a change in pull request #7012: ARROW-8555: [FlightRPC][Java] implement DoExchange

2020-05-04 Thread GitBox
lidavidm commented on a change in pull request #7012: URL: https://github.com/apache/arrow/pull/7012#discussion_r419413303 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightClient.java ## @@ -293,6 +292,76 @@ public void onCompleted() {

[GitHub] [arrow] pitrou commented on pull request #6959: ARROW-5649: [Integration][C++] Create integration test for extension types

2020-05-04 Thread GitBox
pitrou commented on pull request #6959: URL: https://github.com/apache/arrow/pull/6959#issuecomment-623381960 @wesm Do you want to take a look at this? This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] pitrou opened a new pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
pitrou opened a new pull request #7094: URL: https://github.com/apache/arrow/pull/7094 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] fsaintjacques commented on a change in pull request #7089: ARROW-8657: [C++][Python] Add separate configuration for data pages

2020-05-04 Thread GitBox
fsaintjacques commented on a change in pull request #7089: URL: https://github.com/apache/arrow/pull/7089#discussion_r419392424 ## File path: cpp/src/parquet/properties.h ## @@ -34,10 +34,14 @@ namespace parquet { +/// Control for data types in parquet. struct

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419413899 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] github-actions[bot] commented on pull request #7093: ARROW-8687: [Java] Remove references to io.netty.buffer.ArrowBuf

2020-05-04 Thread GitBox
github-actions[bot] commented on pull request #7093: URL: https://github.com/apache/arrow/pull/7093#issuecomment-623337581 https://issues.apache.org/jira/browse/ARROW-8687 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419372644 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] rymurr opened a new pull request #7095: ARROW-8664: [Java] Add flag to skip null check

2020-05-04 Thread GitBox
rymurr opened a new pull request #7095: URL: https://github.com/apache/arrow/pull/7095 All Vector containers should skip null check when null check flag is enabled This is an automated message from the Apache Git Service.

[GitHub] [arrow] pitrou commented on pull request #7094: ARROW-8689: [C++] Fix linking S3FS benchmarks

2020-05-04 Thread GitBox
pitrou commented on pull request #7094: URL: https://github.com/apache/arrow/pull/7094#issuecomment-623432600 Java issues on CI look unrelated. @kszucs can you confirm? This is an automated message from the Apache Git

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419353296 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

[GitHub] [arrow] kszucs commented on a change in pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-04 Thread GitBox
kszucs commented on a change in pull request #7021: URL: https://github.com/apache/arrow/pull/7021#discussion_r419365852 ## File path: docs/source/developers/docker.rst ## @@ -0,0 +1,143 @@ +.. raw:: html + + + +Running Docker Builds += + +Most of our

<    5   6   7   8   9   10   11   12   13   14   >