github-actions[bot] commented on pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#issuecomment-727810644
https://issues.apache.org/jira/browse/ARROW-10609
This is an automated message from the Apache Git
stefan-lange-dataeng commented on issue #8607:
URL: https://github.com/apache/arrow/issues/8607#issuecomment-727846447
Thanks, I have created https://issues.apache.org/jira/browse/ARROW-10611.
This is an automated message
github-actions[bot] commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-727869105
https://issues.apache.org/jira/browse/ARROW-10610
This is an automated message from the Apache Git
Dandandan opened a new pull request #8673:
URL: https://github.com/apache/arrow/pull/8673
Applies the same optimization as in ARROW-10595. Difference is smaller, but
still there:
```
min string 512 time: [3.4096 us 3.4378 us 3.4683 us]
pitrou commented on a change in pull request #8632:
URL: https://github.com/apache/arrow/pull/8632#discussion_r524195648
##
File path: cpp/src/parquet/statistics.cc
##
@@ -691,9 +634,60 @@ void TypedStatisticsImpl::PlainDecode(const
std::string& src,
dst->ptr =
xhochy opened a new pull request #8675:
URL: https://github.com/apache/arrow/pull/8675
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
kiszk commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-727958817
Not yet. I will report this issue today.
This is an automated message from the Apache Git Service.
To respond to
github-actions[bot] commented on pull request #8677:
URL: https://github.com/apache/arrow/pull/8677#issuecomment-727997218
https://issues.apache.org/jira/browse/ARROW-10617
This is an automated message from the Apache Git
pitrou closed pull request #8585:
URL: https://github.com/apache/arrow/pull/8585
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
pitrou commented on a change in pull request #8632:
URL: https://github.com/apache/arrow/pull/8632#discussion_r524192977
##
File path: cpp/src/parquet/encoding.cc
##
@@ -127,6 +129,21 @@ class PlainEncoder : public EncoderImpl, virtual public
TypedEncoder {
}
pitrou commented on a change in pull request #8632:
URL: https://github.com/apache/arrow/pull/8632#discussion_r524193403
##
File path: cpp/src/parquet/encoding.cc
##
@@ -127,6 +129,21 @@ class PlainEncoder : public EncoderImpl, virtual public
TypedEncoder {
}
alamb commented on pull request #8660:
URL: https://github.com/apache/arrow/pull/8660#issuecomment-727983561
> @alamb it might be necessary to fallback to generating an array where the
scalar value is repeated, for some operations that do not have a version which
accepts a scalar argument
xhochy opened a new pull request #8676:
URL: https://github.com/apache/arrow/pull/8676
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
xhochy commented on pull request #8676:
URL: https://github.com/apache/arrow/pull/8676#issuecomment-727946765
@github-actions autotune everything
This is an automated message from the Apache Git Service.
To respond to the
xhochy closed pull request #8676:
URL: https://github.com/apache/arrow/pull/8676
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
pitrou commented on pull request #8671:
URL: https://github.com/apache/arrow/pull/8671#issuecomment-727959654
The PR looks reasonable to me.
This is an automated message from the Apache Git Service.
To respond to the
pitrou closed pull request #8653:
URL: https://github.com/apache/arrow/pull/8653
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
vertexclique commented on pull request #8665:
URL: https://github.com/apache/arrow/pull/8665#issuecomment-727912338
@andygrove @nevi-me https://issues.apache.org/jira/browse/ARROW-10612
Umbrella issue for AVX-512. Includes CI support follow up subtask. I will
create a subtask for every
github-actions[bot] commented on pull request #8675:
URL: https://github.com/apache/arrow/pull/8675#issuecomment-727947748
https://issues.apache.org/jira/browse/ARROW-10616
This is an automated message from the Apache Git
sighingnow opened a new pull request #8677:
URL: https://github.com/apache/arrow/pull/8677
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
alamb closed pull request #8658:
URL: https://github.com/apache/arrow/pull/8658
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
xhochy commented on pull request #8678:
URL: https://github.com/apache/arrow/pull/8678#issuecomment-728059430
@github-actions crossbow submit conda-win-vs2017-py37
This is an automated message from the Apache Git Service.
To
vertexclique commented on a change in pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#discussion_r524238466
##
File path: rust/arrow/benches/aggregate_kernels.rs
##
@@ -43,6 +43,25 @@ fn create_array(size: usize, with_nulls: bool) -> ArrayRef {
vertexclique commented on pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#issuecomment-727958046
Liked it, can you address the comments, that will make some improvements on
the benches indirectly.
This is
xhochy opened a new pull request #8678:
URL: https://github.com/apache/arrow/pull/8678
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
github-actions[bot] commented on pull request #8678:
URL: https://github.com/apache/arrow/pull/8678#issuecomment-728065382
Revision: 061566200fb4db37ec973539c7f6483b468d1a2e
Submitted crossbow builds: [ursa-labs/crossbow @
pitrou commented on a change in pull request #8542:
URL: https://github.com/apache/arrow/pull/8542#discussion_r524304063
##
File path: cpp/src/arrow/util/basic_decimal.cc
##
@@ -490,49 +527,60 @@ static void FixDivisionSigns(BasicDecimal128* result,
BasicDecimal128* remainder
pitrou commented on pull request #8632:
URL: https://github.com/apache/arrow/pull/8632#issuecomment-728110464
@emkornfield Any other concern?
This is an automated message from the Apache Git Service.
To respond to the
vertexclique commented on a change in pull request #8634:
URL: https://github.com/apache/arrow/pull/8634#discussion_r524313352
##
File path: rust/arrow/src/util/bit_slice_iterator.rs
##
@@ -0,0 +1,108 @@
+#[cfg(all(test, target_endian = "big"))]
Review comment:
I think
github-actions[bot] commented on pull request #8679:
URL: https://github.com/apache/arrow/pull/8679#issuecomment-728122505
https://issues.apache.org/jira/browse/ARROW-10619
This is an automated message from the Apache Git
kiszk commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728155154
Reported at https://github.com/lemire/fast_float/issues/37
This is an automated message from the Apache Git
nealrichardson commented on pull request #8643:
URL: https://github.com/apache/arrow/pull/8643#issuecomment-728154879
@bkietz ☝️
This is an automated message from the Apache Git Service.
To respond to the message, please
pitrou commented on a change in pull request #8542:
URL: https://github.com/apache/arrow/pull/8542#discussion_r524305990
##
File path: cpp/src/arrow/util/basic_decimal.cc
##
@@ -490,49 +529,64 @@ static void FixDivisionSigns(BasicDecimal128* result,
BasicDecimal128* remainder
pitrou opened a new pull request #8679:
URL: https://github.com/apache/arrow/pull/8679
Some cases of invalid IPC stream were missed following the ARROW-10619
refactor.
This is an automated message from the Apache Git
wesm commented on a change in pull request #8671:
URL: https://github.com/apache/arrow/pull/8671#discussion_r524359452
##
File path: cpp/src/arrow/util/bitmap_generate.h
##
@@ -83,17 +83,19 @@ void GenerateBitsUnrolled(uint8_t* bitmap, int64_t
start_offset, int64_t length,
pitrou commented on a change in pull request #8542:
URL: https://github.com/apache/arrow/pull/8542#discussion_r524305460
##
File path: cpp/src/arrow/util/basic_decimal.cc
##
@@ -490,49 +529,64 @@ static void FixDivisionSigns(BasicDecimal128* result,
BasicDecimal128* remainder
pitrou commented on pull request #8624:
URL: https://github.com/apache/arrow/pull/8624#issuecomment-728124091
Rebased, will merge.
This is an automated message from the Apache Git Service.
To respond to the message, please
nealrichardson commented on pull request #8643:
URL: https://github.com/apache/arrow/pull/8643#issuecomment-728153475
Are you sure it's not from the PR I merged before this?
https://github.com/apache/arrow/pull/8256 seems like a more likely culprit to
have broken something.
Btw, we
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728178811
@kiszk Thanks for the report.
@pitrou How do you test Big Endian support? I have different ARM boards, but
none of them run in Big Endian mode. I have not been able to find
github-actions[bot] commented on pull request #8680:
URL: https://github.com/apache/arrow/pull/8680#issuecomment-728205092
https://issues.apache.org/jira/browse/ARROW-10182
This is an automated message from the Apache Git
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728217863
(The hard part is to add the necessary CI testing.)
This is an automated message from the Apache Git Service.
To
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728217716
We are looking into fixing this upstream (big endian support).
This is an automated message from the Apache Git
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728226679
(I managed to add CI tests now.)
This is an automated message from the Apache Git Service.
To respond to the
Dandandan commented on a change in pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#discussion_r524387603
##
File path: rust/arrow/benches/aggregate_kernels.rs
##
@@ -43,6 +43,25 @@ fn create_array(size: usize, with_nulls: bool) -> ArrayRef {
jorgecarleitao commented on a change in pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#discussion_r524410551
##
File path: rust/arrow/src/array/builder.rs
##
@@ -1882,6 +1975,67 @@ impl FixedSizeBinaryBuilder {
}
}
+impl DecimalBuilder {
+///
bkietz opened a new pull request #8680:
URL: https://github.com/apache/arrow/pull/8680
Adds `Future::Then(OnSuccess, OnFailure)` which registers callbacks to be
executed on completion of the future and yields a future which wraps the result
of those callbacks; if a callback returns:
velvia opened a new pull request #8688:
URL: https://github.com/apache/arrow/pull/8688
This PR implements the NULLIF() SQL function in DataFusion. It is
implemented as a BuiltInScalarFunction, with a boolean kernel at the core which
creates a new array with a modified null bitmap from
velvia commented on pull request #8688:
URL: https://github.com/apache/arrow/pull/8688#issuecomment-728504277
@andygrove @nevi-me would love to hear your feedback on this this
addition has been useful to us internally.
codecov-io commented on pull request #8660:
URL: https://github.com/apache/arrow/pull/8660#issuecomment-728556384
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8660?src=pr=h1) Report
> Merging
[#8660](https://codecov.io/gh/apache/arrow/pull/8660?src=pr=desc) (e5fce7f)
into
kou commented on pull request #8386:
URL: https://github.com/apache/arrow/pull/8386#issuecomment-728321361
The current status:
* We can't use old packages that depend on `vs2015_runtime<14.16` on
conda-forge for Python 3.9 on Windows.
Because Python 3.9 for Windows on
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728582428
@wesm Thanks to @kiszk, I figured that out.
That we could run CI on IBM mainframe-like systems was a shock to me. :-) I
knew about POWER, but not that. So I learned
pitrou closed pull request #8659:
URL: https://github.com/apache/arrow/pull/8659
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
github-actions[bot] commented on pull request #8688:
URL: https://github.com/apache/arrow/pull/8688#issuecomment-728568531
https://issues.apache.org/jira/browse/ARROW-10330
This is an automated message from the Apache Git
pitrou closed pull request #8624:
URL: https://github.com/apache/arrow/pull/8624
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
alamb closed pull request #8665:
URL: https://github.com/apache/arrow/pull/8665
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#issuecomment-728349940
CI failure seems to be infrastructure related:
```
C:\windows\System32\tar.exe -cz -f
D:\a\_temp\a71516c7-d573-4e09-b21a-03fde6e0e547\cache.tgz -C
github-actions[bot] commented on pull request #8681:
URL: https://github.com/apache/arrow/pull/8681#issuecomment-728320080
https://issues.apache.org/jira/browse/ARROW-10622
This is an automated message from the Apache Git
alamb commented on pull request #8666:
URL: https://github.com/apache/arrow/pull/8666#issuecomment-728397579
@nevi-me check it out: https://github.com/apache/arrow/pull/8687 -- which I
think enables clippy lint in all parts of the project
alamb commented on pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#issuecomment-728398825
@jorgecarleitao I will put it on my queue for tomorrow. Hopefully the morning
This is an automated message from the
wesm commented on pull request #8665:
URL: https://github.com/apache/arrow/pull/8665#issuecomment-728507289
I'm looking at contributing an AVX-512-capable machine to run occasional
builds on Buildkite, I'd guess we're looking at 2-3 month time frame for that
though. Note that anyone can
bkietz commented on pull request #8680:
URL: https://github.com/apache/arrow/pull/8680#issuecomment-728333688
Added a benchmark to measure the cost of creating a Future in
Executor::Submit instead of just using Executor::Spawn. Locally:
```
codecov-io commented on pull request #8542:
URL: https://github.com/apache/arrow/pull/8542#issuecomment-728347240
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8542?src=pr=h1) Report
> Merging
[#8542](https://codecov.io/gh/apache/arrow/pull/8542?src=pr=desc) (1df4d0c)
into
yordan-pavlov commented on pull request #8660:
URL: https://github.com/apache/arrow/pull/8660#issuecomment-728337269
@alamb I have now implemented falling back to scalar arrays for operations
where scalar arguments are currently not supported; this should now work for
operators such as
alamb commented on pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#issuecomment-728362759
The TravisCI builds are not affected by Rust, so merging this one in
This is an automated message from the Apache
alamb commented on a change in pull request #8687:
URL: https://github.com/apache/arrow/pull/8687#discussion_r524703589
##
File path: rust/parquet_derive/src/parquet_field.rs
##
@@ -252,6 +252,8 @@ impl Field {
}
}
+#[allow(clippy::enum_variant_names)]
Review comment:
alamb commented on a change in pull request #8687:
URL: https://github.com/apache/arrow/pull/8687#discussion_r524702929
##
File path: ci/scripts/rust_lint.sh
##
@@ -17,10 +17,14 @@
# specific language governing permissions and limitations
# under the License.
+# Entrypoint
alamb commented on pull request #8660:
URL: https://github.com/apache/arrow/pull/8660#issuecomment-728393610
@yordan-pavlov When I ran the benchmark locally again on my laptop:
```
cargo run --release --bin tpch -- --iterations 3 --path
alamb edited a comment on pull request #8660:
URL: https://github.com/apache/arrow/pull/8660#issuecomment-728393610
@yordan-pavlov When I ran the benchmark locally again on my laptop:
```
cargo run --release --bin tpch -- --iterations 3 --path
alamb opened a new pull request #8687:
URL: https://github.com/apache/arrow/pull/8687
We do currently run clippy (aka rust linter) on all PRs. However, the way we
invoke clippy means that any clippy warnings introduced in PRs do not cause a
CI failure
I believe the intended
alamb closed pull request #8673:
URL: https://github.com/apache/arrow/pull/8673
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
wesm commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728528823
Travis CI has s390x architecture (that's how we've been testing on BE)
This is an automated message from the Apache
jorgecarleitao closed pull request #8565:
URL: https://github.com/apache/arrow/pull/8565
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
jorgecarleitao edited a comment on pull request #8630:
URL: https://github.com/apache/arrow/pull/8630#issuecomment-728736301
This code can be re-used to implement `merge-sort` and `join`. Some notes
about this here:
jorgecarleitao commented on pull request #8630:
URL: https://github.com/apache/arrow/pull/8630#issuecomment-728736301
This code will be re-used to implement `merge-sort` and `join`. Some notes
about this here:
jorgecarleitao commented on pull request #8687:
URL: https://github.com/apache/arrow/pull/8687#issuecomment-728703547
@alamb There was no issue 10636 on Jira, so I created one, assigned it to
you, and assigned this PR to it.
jorgecarleitao closed pull request #8687:
URL: https://github.com/apache/arrow/pull/8687
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
sweb commented on a change in pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#discussion_r524925542
##
File path: rust/arrow/src/array/equal/mod.rs
##
@@ -604,6 +613,76 @@ mod tests {
test_equal(_slice, _slice, true);
}
+fn
github-actions[bot] commented on pull request #8689:
URL: https://github.com/apache/arrow/pull/8689#issuecomment-728715819
https://issues.apache.org/jira/browse/ARROW-10591
This is an automated message from the Apache Git
sweb commented on a change in pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#discussion_r524922526
##
File path: rust/arrow/src/array/builder.rs
##
@@ -1882,6 +1975,67 @@ impl FixedSizeBinaryBuilder {
}
}
+impl DecimalBuilder {
+/// Creates a
sighingnow commented on pull request #8677:
URL: https://github.com/apache/arrow/pull/8677#issuecomment-728669860
I do think a test is needed, but the case has already been tested in
`test_ipc.py` (e.g., `test_stream_simple_roundtrip`) but seems it was skiped in
the Python 3.8 case (in
jorgecarleitao commented on a change in pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#discussion_r524923532
##
File path: rust/arrow/src/array/builder.rs
##
@@ -1882,6 +1975,67 @@ impl FixedSizeBinaryBuilder {
}
}
+impl DecimalBuilder {
+///
jorgecarleitao commented on a change in pull request #8688:
URL: https://github.com/apache/arrow/pull/8688#discussion_r524889515
##
File path: rust/arrow/src/compute/kernels/boolean.rs
##
@@ -149,6 +150,64 @@ pub fn is_not_null(input: ) ->
Result {
nevi-me commented on pull request #8640:
URL: https://github.com/apache/arrow/pull/8640#issuecomment-728739069
Hi @sweb, I'm providing general comments, I'll look at this in detail over
the days.
I see that you're using `i128` as the backing type. One of the reasons
that's
jhorstmann opened a new pull request #8685:
URL: https://github.com/apache/arrow/pull/8685
This refactors the simd aggregation to a reusable trait and adds
implementations for min and max.
Some tests were failing with the simd feature active because the different
order of additions
github-actions[bot] commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-728284606
https://issues.apache.org/jira/browse/ARROW-10216
This is an automated message from the Apache Git
jhorstmann opened a new pull request #8686:
URL: https://github.com/apache/arrow/pull/8686
This makes it much easier to analyze parquet files, for example by
processing the output with other command line tools like `jq`.
I'm opening this as a draft for now since I'd like some
github-actions[bot] commented on pull request #8686:
URL: https://github.com/apache/arrow/pull/8686#issuecomment-728294214
https://issues.apache.org/jira/browse/ARROW-10297
This is an automated message from the Apache Git
kou closed pull request #8675:
URL: https://github.com/apache/arrow/pull/8675
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
github-actions[bot] commented on pull request #8681:
URL: https://github.com/apache/arrow/pull/8681#issuecomment-728236716
Revision: ea40b46b231b0f250a38e28660c25463b997
Submitted crossbow builds: [ursa-labs/crossbow @
rdettai opened a new pull request #8682:
URL: https://github.com/apache/arrow/pull/8682
> Getting the range of bytes of a column chunk inside a parquet file can be
useful for external crates (for instance if they want to pre-fetch the
columns), and is not completely obvious (it is enough
github-actions[bot] commented on pull request #8682:
URL: https://github.com/apache/arrow/pull/8682#issuecomment-728241051
https://issues.apache.org/jira/browse/ARROW-10620
This is an automated message from the Apache Git
rdettai commented on pull request #8682:
URL: https://github.com/apache/arrow/pull/8682#issuecomment-728245070
I did not expect to encounter problems on this tiny PR, but it seems that
the file `alltypes_plain.parquet` that is used in datafusion tests has some
very weird metadata
bkietz opened a new pull request #8681:
URL: https://github.com/apache/arrow/pull/8681
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
bkietz commented on pull request #8681:
URL: https://github.com/apache/arrow/pull/8681#issuecomment-728235768
@github-actions crossbow submit conda-win-vs2017-py37
This is an automated message from the Apache Git Service.
To
rdettai removed a comment on pull request #8682:
URL: https://github.com/apache/arrow/pull/8682#issuecomment-728245070
I did not expect to encounter problems on this tiny PR, but it seems that
the file `alltypes_plain.parquet` that is used in datafusion tests has some
very weird metadata
lemire commented on pull request #8674:
URL: https://github.com/apache/arrow/pull/8674#issuecomment-728252123
The issue has been fixed upstream (big endian support). I am running
exhaustive tests and once they complete, I will issue a new release.
If you wish, you can wait for this
kiszk opened a new pull request #8683:
URL: https://github.com/apache/arrow/pull/8683
This PR copies the required shared libraries into the directory
(`${ARROW_HOME}/lib`) where the test will refer to thru LD_LIBRARY_PATH.
github-actions[bot] commented on pull request #8683:
URL: https://github.com/apache/arrow/pull/8683#issuecomment-728253232
https://issues.apache.org/jira/browse/ARROW-10621
This is an automated message from the Apache Git
nealrichardson closed pull request #8618:
URL: https://github.com/apache/arrow/pull/8618
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Dandandan commented on a change in pull request #8673:
URL: https://github.com/apache/arrow/pull/8673#discussion_r52451
##
File path: rust/arrow/src/compute/kernels/aggregate.rs
##
@@ -32,19 +32,20 @@ fn min_max_string bool>(
if null_count == array.len() {
1 - 100 of 101 matches
Mail list logo
