[GitHub] [arrow] github-actions[bot] commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-04-30 Thread GitBox
github-actions[bot] commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-622102421 Revision: d17f2c212f28bf672a6f46d1dbe017d632707271 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] wesm commented on a change in pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-04-30 Thread GitBox
wesm commented on a change in pull request #6985: URL: https://github.com/apache/arrow/pull/6985#discussion_r418321492 ## File path: cpp/src/arrow/util/bit_util.h ## @@ -610,6 +618,101 @@ class FirstTimeBitmapWriter { } } + /// Appends number_of_bits from word to

[GitHub] [arrow] wesm commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-04-30 Thread GitBox
wesm commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-622155576 I think this is fine to merge once most of the typos in the comments are fixed. A rebase will probably fix the Rust lint error

[GitHub] [arrow] tustvold commented on a change in pull request #7076: ARROW-8659: [Rust] ListBuilder allocate with_capacity

2020-04-30 Thread GitBox
tustvold commented on a change in pull request #7076: URL: https://github.com/apache/arrow/pull/7076#discussion_r418347285 ## File path: rust/parquet/src/arrow/converter.rs ## @@ -128,7 +128,10 @@ pub struct Utf8ArrayConverter {} impl Converter>, StringArray> for

[GitHub] [arrow] wesm commented on pull request #7060: ARROW-8619: [C++] Use distinct enum values for MonthInterval, DayTimeInterval

2020-04-30 Thread GitBox
wesm commented on pull request #7060: URL: https://github.com/apache/arrow/pull/7060#issuecomment-622156273 The ursabot build failures are spurious This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] yordan-pavlov edited a comment on pull request #7037: ARROW-6718: [Rust] Remove packed_simd

2020-04-30 Thread GitBox
yordan-pavlov edited a comment on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-621983970 Hi, I thought I would do some profiling yesterday (to help make sure packed_simd is not removed prematurely) and noticed that a lot of time in `simd_compare_op`

[GitHub] [arrow] wesm commented on pull request #6707: ARROW-300: [Format] Proposal for "trivial" IPC body buffer compression using either LZ4 or ZSTD codecs

2020-04-30 Thread GitBox
wesm commented on pull request #6707: URL: https://github.com/apache/arrow/pull/6707#issuecomment-622157711 +1. I updated the C++ generated Flatbuffers files. Will merge this once the builds run as the vote has passed on the mailing list

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418382595 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418382595 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418382595 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418393347 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -135,6 +152,11 @@ public int GetValueOffset(int index) public int

[GitHub] [arrow] emkornfield commented on pull request #7066: ARROW-8634: [Java] Add Getting Started section to Java README

2020-04-30 Thread GitBox
emkornfield commented on pull request #7066: URL: https://github.com/apache/arrow/pull/7066#issuecomment-622245195 LGTM, thanks, sorry you had to learn the hard way. This is an automated message from the Apache Git Service.

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418419358 ## File path: csharp/src/Apache.Arrow/Arrays/BooleanArray.cs ## @@ -153,17 +184,25 @@ private void CheckIndex(int index) new[] {

[GitHub] [arrow] pravindra commented on a change in pull request #7070: ARROW-8646: [Java] Allow UnionListWriter to write null values

2020-04-30 Thread GitBox
pravindra commented on a change in pull request #7070: URL: https://github.com/apache/arrow/pull/7070#discussion_r418421192 ## File path: java/vector/src/test/java/org/apache/arrow/vector/complex/writer/TestComplexWriter.java ## @@ -266,6 +266,41 @@ public void

[GitHub] [arrow] tvamsikalyan commented on a change in pull request #7070: ARROW-8646: [Java] Allow UnionListWriter to write null values

2020-05-01 Thread GitBox
tvamsikalyan commented on a change in pull request #7070: URL: https://github.com/apache/arrow/pull/7070#discussion_r418432524 ## File path: java/vector/src/test/java/org/apache/arrow/vector/complex/writer/TestComplexWriter.java ## @@ -266,6 +266,41 @@ public void

[GitHub] [arrow] emkornfield commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-05-01 Thread GitBox
emkornfield commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-622266880 > One problem with introducing more SIMD code is that we won't yet have a runtime dispatching strategy. We will need to go through all of our SIMD accelerations in this

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418383314 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -162,8 +188,8 @@ public TBuilder Swap(int i, int j) public TArray

[GitHub] [arrow] github-actions[bot] commented on pull request #7077: ARROW-8660: [C++][Gandiva] Reduce usage of Boost in Gandiva codebase

2020-04-30 Thread GitBox
github-actions[bot] commented on pull request #7077: URL: https://github.com/apache/arrow/pull/7077#issuecomment-622215642 https://issues.apache.org/jira/browse/ARROW-8660 This is an automated message from the Apache Git

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418392611 ## File path: csharp/src/Apache.Arrow/Arrays/BooleanArray.cs ## @@ -153,17 +184,25 @@ private void CheckIndex(int index) new[] {

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418393025 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -69,25 +83,28 @@ public ListArray Build(MemoryAllocator allocator = default)

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418417744 ## File path: csharp/test/Apache.Arrow.Tests/BooleanArrayTests.cs ## @@ -48,13 +48,13 @@ public void AppendsExpectedBit()

[GitHub] [arrow] nevi-me commented on pull request #7018: ARROW-8536: [Rust] [Flight] Check in proto file, conditional build if file exists

2020-04-30 Thread GitBox
nevi-me commented on pull request #7018: URL: https://github.com/apache/arrow/pull/7018#issuecomment-622254639 I'll address this later today This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418422210 ## File path: csharp/src/Apache.Arrow/Arrays/BooleanArray.cs ## @@ -153,17 +184,25 @@ private void CheckIndex(int index) new[] {

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418424144 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -135,6 +152,11 @@ public int GetValueOffset(int index) public int

[GitHub] [arrow] jorisvandenbossche commented on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
jorisvandenbossche commented on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-622265852 From a user perspective, I find that also an added convenience. In Python, the `FileSystemDataset.format` attribute let you check the format of your dataset (instead

[GitHub] [arrow] wesm commented on pull request #6631: ARROW-8111: [C++][CSV] Support MM/DD/YYYY date format

2020-04-30 Thread GitBox
wesm commented on pull request #6631: URL: https://github.com/apache/arrow/pull/6631#issuecomment-622195001 Thank you. There are some code linting issues and other code style issues (we follow the Google C++ style guide), can you fix the CI builds? I'd like to kick the tires a bit on this

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418384847 ## File path: csharp/src/Apache.Arrow/Arrays/StringArray.cs ## @@ -71,6 +76,15 @@ public string GetString(int index, Encoding encoding = default)

[GitHub] [arrow] vibhatha opened a new issue #7078: Pyarrow building from source along with CPP Libraries to link to another Cython API

2020-04-30 Thread GitBox
vibhatha opened a new issue #7078: URL: https://github.com/apache/arrow/issues/7078 I am trying to integrate arrow with an application that I am developing. Here I build Arrow from the source (CPP) and use the API to develop some custom functions to do a scientific calculation after data

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418385072 ## File path: csharp/src/Apache.Arrow/Arrays/StringArray.cs ## @@ -71,6 +76,15 @@ public string GetString(int index, Encoding encoding = default)

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418388163 ## File path: csharp/src/Apache.Arrow/Arrays/BooleanArray.cs ## @@ -153,17 +182,25 @@ private void CheckIndex(int index) new[] {

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418424732 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -53,6 +60,26 @@ public sealed class ArrayData : IDisposable Offset = offset;

[GitHub] [arrow] pravindra commented on a change in pull request #7070: ARROW-8646: [Java] Allow UnionListWriter to write null values

2020-04-30 Thread GitBox
pravindra commented on a change in pull request #7070: URL: https://github.com/apache/arrow/pull/7070#discussion_r418421121 ## File path: java/vector/src/main/codegen/templates/UnionListWriter.java ## @@ -178,6 +178,10 @@ public void write(DecimalHolder holder) {

[GitHub] [arrow] wesm commented on issue #7063: client delete (of objectid) causes an exception and abort

2020-04-30 Thread GitBox
wesm commented on issue #7063: URL: https://github.com/apache/arrow/issues/7063#issuecomment-622214971 You can also send an e-mail to the dev@ mailing list. Closing this issue since we don't do dev or user discussions on GitHub

[GitHub] [arrow] wesm commented on issue #7078: Pyarrow building from source along with CPP Libraries to link to another Cython API

2020-04-30 Thread GitBox
wesm commented on issue #7078: URL: https://github.com/apache/arrow/issues/7078#issuecomment-622214615 Can you please ask on the mailing list? We don't provide user help on GitHub. This is an automated message from the

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418391298 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] emkornfield commented on a change in pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-04-30 Thread GitBox

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418423261 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -69,25 +83,28 @@ public ListArray Build(MemoryAllocator allocator = default)

[GitHub] [arrow] sunchao commented on a change in pull request #7061: ARROW-8629: [Rust] Eliminate indirection of zero sized allocations

2020-05-01 Thread GitBox
sunchao commented on a change in pull request #7061: URL: https://github.com/apache/arrow/pull/7061#discussion_r418434103 ## File path: rust/arrow/src/util/bit_util.rs ## @@ -148,11 +148,17 @@ pub fn count_set_bits_offset(data: &[u8], offset: usize, length: usize) -> usize

[GitHub] [arrow] sunchao commented on pull request #7037: ARROW-6718: [Rust] Remove packed_simd

2020-05-01 Thread GitBox
sunchao commented on pull request #7037: URL: https://github.com/apache/arrow/pull/7037#issuecomment-622272292 This definitely looks great from # of code deduction :D , but yeah it will be better if we can keep the perf loss minimum. > Also the future of packed_simd is unclear and

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418380212 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418392155 ## File path: csharp/test/Apache.Arrow.Tests/BooleanArrayTests.cs ## @@ -48,13 +48,13 @@ public void AppendsExpectedBit()

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418392155 ## File path: csharp/test/Apache.Arrow.Tests/BooleanArrayTests.cs ## @@ -48,13 +48,13 @@ public void AppendsExpectedBit()

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418394280 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -53,6 +60,26 @@ public sealed class ArrayData : IDisposable Offset = offset;

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418423816 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -22,6 +22,8 @@ namespace Apache.Arrow { public sealed class ArrayData : IDisposable

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418425841 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -53,6 +60,26 @@ public sealed class ArrayData : IDisposable Offset = offset;

[GitHub] [arrow] wesm opened a new pull request #7077: ARROW-8660: [C++][Gandiva] Reduce usage of Boost in Gandiva codebase

2020-04-30 Thread GitBox
wesm opened a new pull request #7077: URL: https://github.com/apache/arrow/pull/7077 I noticed this while reading the Gandiva codebase as part of the C++ precompiled kernels revamp project. In general we've tried to reduce our use of Boost -- if we can eliminate Boost altogether from

[GitHub] [arrow] zgramana commented on pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
zgramana commented on pull request #7032: URL: https://github.com/apache/arrow/pull/7032#issuecomment-622217498 @eerhardt I think I covered everything. I also discovered that I had omitted adding `null` support to `ListArray.Builder` so I added that too (including test coverage) in the

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418394280 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -53,6 +60,26 @@ public sealed class ArrayData : IDisposable Offset = offset;

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-30 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418394031 ## File path: csharp/src/Apache.Arrow/Arrays/ArrayData.cs ## @@ -22,6 +22,8 @@ namespace Apache.Arrow { public sealed class ArrayData : IDisposable

[GitHub] [arrow] tvamsikalyan commented on a change in pull request #7070: ARROW-8646: [Java] Allow UnionListWriter to write null values

2020-05-01 Thread GitBox
tvamsikalyan commented on a change in pull request #7070: URL: https://github.com/apache/arrow/pull/7070#discussion_r418432449 ## File path: java/vector/src/main/codegen/templates/UnionListWriter.java ## @@ -178,6 +178,10 @@ public void write(DecimalHolder holder) {

[GitHub] [arrow] andygrove commented on a change in pull request #7061: ARROW-8629: [Rust] Eliminate indirection of zero sized allocations

2020-04-29 Thread GitBox
andygrove commented on a change in pull request #7061: URL: https://github.com/apache/arrow/pull/7061#discussion_r417658203 ## File path: rust/arrow/src/memory.rs ## @@ -20,34 +20,56 @@ use std::alloc::Layout; use std::mem::align_of; +use std::ptr::NonNull; pub const

[GitHub] [arrow] andygrove commented on pull request #6898: ARROW-8399: [Rust] Extend memory alignments to include other architectures

2020-04-29 Thread GitBox
andygrove commented on pull request #6898: URL: https://github.com/apache/arrow/pull/6898#issuecomment-621513057 I don't know much about this either, but I have no objections to these changes. This is an automated message

[GitHub] [arrow] wesm commented on pull request #5038: ARROW-6131: [C++] Optimize the Arrow Non-Ascii-string-validation

2020-04-29 Thread GitBox
wesm commented on pull request #5038: URL: https://github.com/apache/arrow/pull/5038#issuecomment-621470942 It would be useful to have an ASCII validation function (versus UTF-8 / non-UTF-8), would you be able to break out the ASCII validation changes into a new PR?

[GitHub] [arrow] andygrove commented on pull request #7018: ARROW-8536: [Rust] [Flight] Check in proto file, conditional build if file exists

2020-04-29 Thread GitBox
andygrove commented on pull request #7018: URL: https://github.com/apache/arrow/pull/7018#issuecomment-621512162 @paddyhoran any opinion on this? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] andygrove commented on pull request #7035: ARROW-8590: [Rust] Use arrow crate pretty util in DataFusion

2020-04-29 Thread GitBox
andygrove commented on pull request #7035: URL: https://github.com/apache/arrow/pull/7035#issuecomment-621511978 I just merged https://github.com/apache/arrow/pull/6972 ... could you rebase when you get a chance? This is an

[GitHub] [arrow] wesm commented on pull request #7064: ARROW-6945: [Rust] WIP: Add initial skeleton for Rust integration tests

2020-04-29 Thread GitBox
wesm commented on pull request #7064: URL: https://github.com/apache/arrow/pull/7064#issuecomment-621525214 You can also merge patches into a branch -- the PR merge tool will merge patches into branches other than master if the PR is configured in the right way

[GitHub] [arrow] abemammen opened a new issue #7063: client delete (of objectid) causes an exception and abort

2020-04-29 Thread GitBox
abemammen opened a new issue #7063: URL: https://github.com/apache/arrow/issues/7063 Built from this git repo. for cpp: >> ARROW_CHECK_OK(client.Delete(vector{objectId})); get: >>> Check failed: _s.ok() Operation failed: client.Delete(vector{objectId}) Bad status: IOError:

[GitHub] [arrow] lidavidm commented on a change in pull request #6744: PARQUET-1820: [C++] pre-buffer specified columns of row group

2020-04-29 Thread GitBox
lidavidm commented on a change in pull request #6744: URL: https://github.com/apache/arrow/pull/6744#discussion_r417626444 ## File path: cpp/src/parquet/file_reader.cc ## @@ -212,6 +237,21 @@ class SerializedFile : public ParquetFileReader::Contents { file_metadata_ =

[GitHub] [arrow] github-actions[bot] commented on pull request #7064: ARROW-6945: [Rust] WIP: Add initial skeleton for Rust integration tests

2020-04-29 Thread GitBox
github-actions[bot] commented on pull request #7064: URL: https://github.com/apache/arrow/pull/7064#issuecomment-621488703 https://issues.apache.org/jira/browse/ARROW-6945 This is an automated message from the Apache Git

[GitHub] [arrow] andygrove commented on pull request #7059: ARROW-8622: [Rust] Allow the parquet crate to be compiled on aarch64 platforms

2020-04-29 Thread GitBox
andygrove commented on pull request #7059: URL: https://github.com/apache/arrow/pull/7059#issuecomment-621510012 CI is failing due to a rustfmt change. May need to rebase on master? This is an automated message from the

[GitHub] [arrow] wesm commented on pull request #7060: ARROW-8619: [C++] Use distinct enum values for MonthInterval, DayTimeInterval

2020-04-29 Thread GitBox
wesm commented on pull request #7060: URL: https://github.com/apache/arrow/pull/7060#issuecomment-621472148 Will fix the builds This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow] pauldix opened a new pull request #7064: ARROW-6945: [Rust] WIP: Add initial skeleton for Rust integration tests

2020-04-29 Thread GitBox
pauldix opened a new pull request #7064: URL: https://github.com/apache/arrow/pull/7064 This isn't even close to being mergable, but I'm hoping to get some additional eyes on this at this point. I have an initial skeleton of the integration test. I started out only implementing part of

[GitHub] [arrow] lidavidm commented on pull request #7062: ARROW-8632: [C++] Fix conversion error warning in array_union_test.cc

2020-04-29 Thread GitBox
lidavidm commented on pull request #7062: URL: https://github.com/apache/arrow/pull/7062#issuecomment-621473542 Sorry about that! ☹️ This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] andygrove commented on pull request #7064: ARROW-6945: [Rust] WIP: Add initial skeleton for Rust integration tests

2020-04-29 Thread GitBox
andygrove commented on pull request #7064: URL: https://github.com/apache/arrow/pull/7064#issuecomment-621515127 Thanks, Paul. This is an awesome start. I will jump in and help as soon as I can. Given the importance of this integration testing and the need for Rust contributors to

[GitHub] [arrow] github-actions[bot] commented on pull request #7079: [ARROW-6501][C++] Remove non_zero_length_ field from SparseIndex class

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7079: URL: https://github.com/apache/arrow/pull/7079#issuecomment-622283900 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] github-actions[bot] commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-622346215 Revision: 8852e2f5f32402ca9c85877289c7948db141cca7 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7021: URL: https://github.com/apache/arrow/pull/7021#issuecomment-622320787 Revision: f3fe79f4e89811a0e24b240d1b306315cdce95ee Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #7079: ARROW-6501: [C++] Remove non_zero_length_ field from SparseIndex class

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7079: URL: https://github.com/apache/arrow/pull/7079#issuecomment-622287657 https://issues.apache.org/jira/browse/ARROW-6501 This is an automated message from the Apache Git

[GitHub] [arrow] sunchao commented on a change in pull request #7076: ARROW-8659: [Rust] ListBuilder allocate with_capacity

2020-05-01 Thread GitBox
sunchao commented on a change in pull request #7076: URL: https://github.com/apache/arrow/pull/7076#discussion_r418460308 ## File path: rust/parquet/src/arrow/converter.rs ## @@ -128,7 +128,10 @@ pub struct Utf8ArrayConverter {} impl Converter>, StringArray> for

[GitHub] [arrow] mrkn commented on pull request #7079: ARROW-6501: [C++] Remove non_zero_length_ field from SparseIndex class

2020-05-01 Thread GitBox
mrkn commented on pull request #7079: URL: https://github.com/apache/arrow/pull/7079#issuecomment-622315057 @pitrou @rok Could you have a look at this? This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] github-actions[bot] commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-622319822 Revision: ea4f1375f4dcbc5fe7e81dc16b2c5c239bc30d30 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kszucs commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-01 Thread GitBox
kszucs commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-622319449 @github-actions crossbow submit wheel-win-cp38 This is an automated message from the Apache Git Service. To

[GitHub] [arrow] kszucs commented on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
kszucs commented on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-62291 > * Simplified FileSystemDataset to hold a FragmentVector. Each Fragment must be a FileFragment and is checked at `FileSystemDataset::Make`. Fragments are not required to use the

[GitHub] [arrow] kszucs opened a new pull request #7080: [CI] Consolidate appveyor scripts [WIP]

2020-05-01 Thread GitBox
kszucs opened a new pull request #7080: URL: https://github.com/apache/arrow/pull/7080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] sunchao commented on a change in pull request #6898: ARROW-8399: [Rust] Extend memory alignments to include other architectures

2020-05-01 Thread GitBox
sunchao commented on a change in pull request #6898: URL: https://github.com/apache/arrow/pull/6898#discussion_r418443106 ## File path: rust/arrow/src/memory.rs ## @@ -21,7 +21,58 @@ use std::alloc::Layout; use std::mem::align_of; -pub const ALIGNMENT: usize = 64;

[GitHub] [arrow] kszucs edited a comment on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
kszucs edited a comment on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-62291 > * Simplified FileSystemDataset to hold a FragmentVector. Each Fragment must be a FileFragment and is checked at `FileSystemDataset::Make`. Fragments are not required to

[GitHub] [arrow] kszucs edited a comment on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
kszucs edited a comment on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-62291 > Fragments are not required to use the same backing filesystem nor the same format. This makes me wonder, why do we need FileSystemDataset and/or UnionDataset at

[GitHub] [arrow] github-actions[bot] commented on pull request #7080: [CI] Consolidate appveyor scripts [WIP]

2020-05-01 Thread GitBox
github-actions[bot] commented on pull request #7080: URL: https://github.com/apache/arrow/pull/7080#issuecomment-622345293 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] mrkn opened a new pull request #7079: [ARROW-6501][C++] Remove non_zero_length_ field from SparseIndex class

2020-05-01 Thread GitBox
mrkn opened a new pull request #7079: URL: https://github.com/apache/arrow/pull/7079 This field is essentially needless, and may be obstacle to the future improvement of sparse tensors, such as adding value inserting feature.

[GitHub] [arrow] kszucs commented on pull request #7060: ARROW-8619: [C++] Use distinct enum values for MonthInterval, DayTimeInterval

2020-05-01 Thread GitBox
kszucs commented on pull request #7060: URL: https://github.com/apache/arrow/pull/7060#issuecomment-622312610 @ursabot build This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow] kszucs commented on pull request #7060: ARROW-8619: [C++] Use distinct enum values for MonthInterval, DayTimeInterval

2020-05-01 Thread GitBox
kszucs commented on pull request #7060: URL: https://github.com/apache/arrow/pull/7060#issuecomment-622312875 > The ursabot build failures are spurious Occasionally happens after a force push. This is an automated

[GitHub] [arrow] kszucs commented on pull request #7021: ARROW-8628: [Dev] Wrap docker-compose commands with archery

2020-05-01 Thread GitBox
kszucs commented on pull request #7021: URL: https://github.com/apache/arrow/pull/7021#issuecomment-622320430 @github-actions crossbow submit -g test This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] kszucs commented on pull request #7074: ARROW-8656: [Python] Switch to VS2017 in the windows wheel builds

2020-05-01 Thread GitBox
kszucs commented on pull request #7074: URL: https://github.com/apache/arrow/pull/7074#issuecomment-622345879 @github-actions crossbow submit wheel-win-cp38 This is an automated message from the Apache Git Service. To

[GitHub] [arrow] zgramana commented on pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-04-27 Thread GitBox
zgramana commented on pull request #7032: URL: https://github.com/apache/arrow/pull/7032#issuecomment-620266004 @eerhardt thanks for the excellent review and feedback. I believe that I have pushed fixes for everything. I also noticed I had missed validity buffer state management in

[GitHub] [arrow] bkietz commented on a change in pull request #7033: ARROW-7759: [C++][Dataset] Add CsvFileFormat

2020-04-27 Thread GitBox
bkietz commented on a change in pull request #7033: URL: https://github.com/apache/arrow/pull/7033#discussion_r416235260 ## File path: cpp/src/arrow/dataset/file_csv.cc ## @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] bkietz commented on a change in pull request #7033: ARROW-7759: [C++][Dataset] Add CsvFileFormat

2020-04-27 Thread GitBox
bkietz commented on a change in pull request #7033: URL: https://github.com/apache/arrow/pull/7033#discussion_r416253903 ## File path: cpp/src/arrow/dataset/file_csv.cc ## @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] github-actions[bot] commented on pull request #7048: ARROW-8609: [C++] fix orc jni crash

2020-04-27 Thread GitBox
github-actions[bot] commented on pull request #7048: URL: https://github.com/apache/arrow/pull/7048#issuecomment-620323735 https://issues.apache.org/jira/browse/ARROW-8609 This is an automated message from the Apache Git

[GitHub] [arrow] bkietz commented on a change in pull request #7033: ARROW-7759: [C++][Dataset] Add CsvFileFormat

2020-04-27 Thread GitBox
bkietz commented on a change in pull request #7033: URL: https://github.com/apache/arrow/pull/7033#discussion_r416239202 ## File path: cpp/src/arrow/dataset/file_csv.cc ## @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] zhouyuan opened a new pull request #7048: ARROW-8609: [C++] fix orc jni crash

2020-04-27 Thread GitBox
zhouyuan opened a new pull request #7048: URL: https://github.com/apache/arrow/pull/7048 check if arrow buffer is null before passing to the constructor Signed-off-by: Yuan Zhou This is an automated message from the

[GitHub] [arrow] zhouyuan commented on pull request #6953: [Java] Reproduce ORC JNI binding crash [WIP]

2020-04-27 Thread GitBox
zhouyuan commented on pull request #6953: URL: https://github.com/apache/arrow/pull/6953#issuecomment-620321153 @kszucs ran into similar issue on parquet side, not sure if this #7048 works This is an automated message

[GitHub] [arrow] kszucs opened a new pull request #7081: [CI] Cache docker volumes [WIP]

2020-05-01 Thread GitBox
kszucs opened a new pull request #7081: URL: https://github.com/apache/arrow/pull/7081 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418424144 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -135,6 +152,11 @@ public int GetValueOffset(int index) public int

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418393347 ## File path: csharp/src/Apache.Arrow/Arrays/ListArray.cs ## @@ -135,6 +152,11 @@ public int GetValueOffset(int index) public int

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418383314 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -162,8 +188,8 @@ public TBuilder Swap(int i, int j) public TArray

[GitHub] [arrow] zgramana commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
zgramana commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r418380212 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] eerhardt commented on a change in pull request #7032: ARROW-6603: [C#] Adds ArrayBuilder API to support writing null values + BooleanArray null support

2020-05-01 Thread GitBox
eerhardt commented on a change in pull request #7032: URL: https://github.com/apache/arrow/pull/7032#discussion_r417709163 ## File path: csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs ## @@ -99,55 +105,75 @@ public abstract class PrimitiveArrayBuilder : IArrowArrayBu

[GitHub] [arrow] bkietz commented on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
bkietz commented on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-622364855 WRT convenient single format or single file system datasets, it would be straightforward (and possibly more useful) to provide accessories for subsets,

[GitHub] [arrow] wesm commented on pull request #6985: ARROW-8413: [C++][Parquet] Refactor Generating validity bitmap for values column

2020-05-01 Thread GitBox
wesm commented on pull request #6985: URL: https://github.com/apache/arrow/pull/6985#issuecomment-622371728 What I've seen other projects do (have to dig for some examples) is to have files like ``` functionality_nosimd.cc functionality_sse42.cc functionality_avx2.cc ```

[GitHub] [arrow] cyb70289 commented on a change in pull request #6954: ARROW-8440: [C++] Refine SIMD header files

2020-05-01 Thread GitBox
cyb70289 commented on a change in pull request #6954: URL: https://github.com/apache/arrow/pull/6954#discussion_r418558120 ## File path: cpp/src/arrow/util/simd.h ## @@ -17,6 +17,24 @@ #pragma once +#ifdef _MSC_VER +// MSVC x86_64/arm64 + +#if defined(_M_AMD64) ||

[GitHub] [arrow] jorisvandenbossche commented on pull request #7073: ARROW-8318: [C++][Dataset] Construct FileSystemDataset from fragments

2020-05-01 Thread GitBox
jorisvandenbossche commented on pull request #7073: URL: https://github.com/apache/arrow/pull/7073#issuecomment-622354887 Do we need FileSystemDataset, maybe not. Is it still useful, IMO yes. As mentioned above, I personally find it convenient to know that my dataset has a single

<    3   4   5   6   7   8   9   10   11   12   >