jorgecarleitao commented on a change in pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#discussion_r471899766
##
File path: rust/datafusion/src/execution/physical_plan/udf.rs
##
@@ -146,3 +154,99 @@ impl PhysicalExpr for ScalarFunctionExpr {
(fun)()
jorgecarleitao commented on a change in pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#discussion_r471899766
##
File path: rust/datafusion/src/execution/physical_plan/udf.rs
##
@@ -146,3 +154,99 @@ impl PhysicalExpr for ScalarFunctionExpr {
(fun)()
jorgecarleitao commented on a change in pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#discussion_r471894969
##
File path: rust/datafusion/src/execution/context.rs
##
@@ -509,33 +534,76 @@ impl SchemaProvider for ExecutionContextState {
}
fn
github-actions[bot] commented on pull request #7983:
URL: https://github.com/apache/arrow/pull/7983#issuecomment-675251338
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then
github-actions[bot] commented on pull request #7983:
URL: https://github.com/apache/arrow/pull/7983#issuecomment-675256101
https://issues.apache.org/jira/browse/ARROW-9733
This is an automated message from the Apache Git
andygrove commented on pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#issuecomment-675254823
I took a very quick look at Spark just now and here are some observations:
- math expressions such as sqrt always return double and don't try to
optimize to smaller
jorgecarleitao opened a new pull request #7983:
URL: https://github.com/apache/arrow/pull/7983
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
houqp commented on pull request #7880:
URL: https://github.com/apache/arrow/pull/7880#issuecomment-675259193
One thing I do like about spark's optimizer is all optimization rules share
a common plan tree traversal and mutation routine, which made individual
optimization rule easier to
jorgecarleitao commented on a change in pull request #7880:
URL: https://github.com/apache/arrow/pull/7880#discussion_r471887798
##
File path: rust/datafusion/src/optimizer/filter_push_down.rs
##
@@ -0,0 +1,631 @@
+// regarding copyright ownership. The ASF licenses this file
andygrove commented on pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#issuecomment-675245608
When faced with choices like this, it is often helpful to look at how other
projects implement this. Perhaps we could look at calcite or spark to see
what choices they
sunchao commented on a change in pull request #7917:
URL: https://github.com/apache/arrow/pull/7917#discussion_r471927282
##
File path: rust/parquet/src/arrow/schema.rs
##
@@ -83,12 +90,77 @@ where
.map(|fields| Schema::new_with_metadata(fields, metadata))
}
+///
jorgecarleitao commented on pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#issuecomment-675041274
> Currently we only do optimizations on the logical plan, but there is a
plan to have physical plan optozations as well.
Won't those be made against a physical plan?
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471707581
##
File path: rust/datafusion/src/lib.rs
##
@@ -31,6 +31,7 @@ extern crate sqlparser;
pub mod dataframe;
pub mod datasource;
+mod datatyped;
nealrichardson commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675100389
@github-actions crossbow submit homebrew-r-autobrew
This is an automated message from the Apache Git
github-actions[bot] commented on pull request #7982:
URL: https://github.com/apache/arrow/pull/7982#issuecomment-675138080
https://issues.apache.org/jira/browse/ARROW-9744
This is an automated message from the Apache Git
nealrichardson commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675065052
@github-actions crossbow submit homebrew-r-autobrew
This is an automated message from the Apache Git
kiszk commented on pull request #7973:
URL: https://github.com/apache/arrow/pull/7973#issuecomment-675087462
Is it better to create an issue in PARQUET instead of ARROW?
This is an automated message from the Apache Git
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471097047
##
File path: rust/datafusion/src/logicalplan.rs
##
@@ -1087,7 +1101,7 @@ impl LogicalPlanBuilder {
/// Apply a projection
pub fn
jorgecarleitao commented on pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#issuecomment-675042360
@andygrove , since I rebased this branch on top of master with the changes
of the lock of scalar_functions, the tests consistently halt. Do you or @alamb
have any hints on
jorgecarleitao commented on a change in pull request #7971:
URL: https://github.com/apache/arrow/pull/7971#discussion_r471706745
##
File path: rust/datafusion/src/execution/physical_plan/udf.rs
##
@@ -20,25 +20,33 @@
use std::fmt;
use arrow::array::ArrayRef;
-use
sunchao commented on pull request #7917:
URL: https://github.com/apache/arrow/pull/7917#issuecomment-675178642
@nevi-me ah OK - I'll take a look soon!
This is an automated message from the Apache Git Service.
To respond to
nealrichardson commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675107914
Sigh,
https://github.com/apache/arrow/commit/1018a4f50fb3cd9b5f622a25ad0c69b1389c9a96
just merged, and it adds additional aws-sdk-cpp components, so the new builds
are
nevi-me commented on pull request #7917:
URL: https://github.com/apache/arrow/pull/7917#issuecomment-675177564
@sunchao I forgot to mention, I made the changes, so this is ready for review
This is an automated message from
jorgecarleitao commented on a change in pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#discussion_r471688534
##
File path: rust/datafusion/tests/sql.rs
##
@@ -232,6 +326,55 @@ fn custom_sqrt(args: &[ArrayRef]) -> Result {
Ok(Arc::new(builder.finish()))
github-actions[bot] commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675101369
Revision: 3bd693e259c897dc1d95f5cf6e724ffebab4c1bf
Submitted crossbow builds: [ursa-labs/crossbow @
kou opened a new pull request #7982:
URL: https://github.com/apache/arrow/pull/7982
pyarrow just uses SetupCxxFlags.cmake. It doesn't use
DefineOptions.cmake. We need to define ARROW_ARMV8_ARCH in pyarrow
too. It's defined in DefineOptions.cmake.
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471704229
##
File path: rust/datafusion/src/execution/physical_plan/expressions.rs
##
@@ -156,7 +168,7 @@ macro_rules! sum_accumulate {
#[derive(Debug)]
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471718884
##
File path: rust/datafusion/src/sql/planner.rs
##
@@ -476,70 +478,50 @@ impl SqlToRel {
}
SQLExpr::Function(function)
github-actions[bot] commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675066462
Revision: f4afc43981c2cdae9f1758d521ec0ac0e28921fc
Submitted crossbow builds: [ursa-labs/crossbow @
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471708914
##
File path: rust/datafusion/src/logicalplan.rs
##
@@ -364,22 +364,25 @@ pub enum Expr {
/// List of expressions to feed to the functions
jorgecarleitao commented on a change in pull request #7974:
URL: https://github.com/apache/arrow/pull/7974#discussion_r471708549
##
File path: rust/datafusion/src/logicalplan.rs
##
@@ -355,7 +355,7 @@ pub enum Expr {
/// List of expressions to feed to the functions as
jorgecarleitao commented on a change in pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#discussion_r471652148
##
File path: rust/datafusion/src/sql/planner.rs
##
@@ -515,27 +515,29 @@ impl SqlToRel {
}
_ => match
bkietz commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471664878
##
File path: r/src/arrow_cpp11.h
##
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license
pitrou commented on pull request #7928:
URL: https://github.com/apache/arrow/pull/7928#issuecomment-675030202
Just retry the build? This could be a sporadic issue.
This is an automated message from the Apache Git Service.
To
pitrou closed pull request #7945:
URL: https://github.com/apache/arrow/pull/7945
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
lidavidm commented on pull request #7981:
URL: https://github.com/apache/arrow/pull/7981#issuecomment-675035275
I originally added a unit test, but removed it since I realized it's flaky
(the error happens when two blocking operations overlap; in a test, this can't
be simulated since you
andygrove commented on pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#issuecomment-675035668
Currently we only do optimizations on the logical plan, but there is a plan
to have physical plan optozations as well.
On Mon, Aug 17, 2020, 11:37 AM Jorge Leitao
tianchen92 commented on a change in pull request #7887:
URL: https://github.com/apache/arrow/pull/7887#discussion_r471248061
##
File path: cpp/src/arrow/buffer_builder.h
##
@@ -292,6 +292,11 @@ class TypedBufferBuilder {
return Status::OK();
}
+ void
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471335906
##
File path: r/src/arrow_cpp11.h
##
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471346362
##
File path: docs/source/cpp/compute.rst
##
@@ -197,6 +197,10 @@ an ``Invalid`` :class:`Status` when overflow is detected.
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471346229
##
File path: cpp/src/arrow/compute/api_scalar.h
##
@@ -129,6 +129,20 @@ Result Multiply(const Datum& left, const Datum&
right,
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471345721
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc
##
@@ -186,6 +187,42 @@ struct MultiplyChecked {
}
};
+struct Divide {
+ template
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471346102
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc
##
@@ -186,6 +187,42 @@ struct MultiplyChecked {
}
};
+struct Divide {
+ template
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471346573
##
File path: cpp/src/arrow/compute/api_scalar.h
##
@@ -129,6 +129,20 @@ Result Multiply(const Datum& left, const Datum&
right,
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471349293
##
File path: r/src/arrow_cpp11.h
##
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471348958
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -492,6 +486,81 @@ TYPED_TEST(TestBinaryArithmeticFloating, Add) {
kszucs commented on pull request #7803:
URL: https://github.com/apache/arrow/pull/7803#issuecomment-674806014
@ursabot build
This is an automated message from the Apache Git Service.
To respond to the message, please log on
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471343433
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc
##
@@ -186,6 +187,42 @@ struct MultiplyChecked {
}
};
+struct Divide {
+ template
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471347426
##
File path: r/src/arrow_cpp11.h
##
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471347282
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -139,13 +139,7 @@ class TestBinaryArithmetic : public TestBase {
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471335034
##
File path: r/src/array_from_vector.cpp
##
@@ -406,9 +403,12 @@ std::shared_ptr
MakeFactorArray(Rcpp::IntegerVector_ factor,
case
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471335186
##
File path: r/src/array_from_vector.cpp
##
@@ -1064,42 +1063,42 @@ class FixedSizeBinaryVectorConverter : public
VectorConverter {
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471348419
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -492,6 +486,81 @@ TYPED_TEST(TestBinaryArithmeticFloating, Add) {
pitrou commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471364203
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -492,6 +486,81 @@ TYPED_TEST(TestBinaryArithmeticFloating, Add) {
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471379295
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -492,6 +486,81 @@ TYPED_TEST(TestBinaryArithmeticFloating, Add) {
kszucs commented on pull request #7952:
URL: https://github.com/apache/arrow/pull/7952#issuecomment-674783758
+1, merging on green.
This is an automated message from the Apache Git Service.
To respond to the message, please
kszucs commented on pull request #7803:
URL: https://github.com/apache/arrow/pull/7803#issuecomment-674786830
Indeed, the docker images there need to be rebuilt. I'm updating them.
This is an automated message from the
kszucs closed pull request #7952:
URL: https://github.com/apache/arrow/pull/7952
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
liyafan82 commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471345890
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc
##
@@ -186,6 +187,42 @@ struct MultiplyChecked {
}
};
+struct Divide {
+ template
kszucs commented on a change in pull request #7952:
URL: https://github.com/apache/arrow/pull/7952#discussion_r471360126
##
File path: python/pyarrow/tests/test_array.py
##
@@ -975,6 +975,47 @@ def check_result(result, expected_field_names,
expected_type_codes,
arr =
romainfrancois commented on a change in pull request #7819:
URL: https://github.com/apache/arrow/pull/7819#discussion_r471349293
##
File path: r/src/arrow_cpp11.h
##
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
kszucs commented on pull request #7932:
URL: https://github.com/apache/arrow/pull/7932#issuecomment-674782381
The tests are passing locally, but the arrow repository gets fetched
differently on github actions, so need to fix that before merging.
pitrou commented on pull request #7979:
URL: https://github.com/apache/arrow/pull/7979#issuecomment-674782797
Is there a reason to introduce this and burden the API with a new setting?
If the "new" engine supports a superset of what the current engine supports,
then it seems it should
pitrou edited a comment on pull request #7803:
URL: https://github.com/apache/arrow/pull/7803#issuecomment-674781958
@corleyma Unfortunately, the "ursabot" builders use a different execution
scheme. The packages need to be updated manually there, I believe. I pinged
@kszucs who should be
pitrou commented on pull request #7803:
URL: https://github.com/apache/arrow/pull/7803#issuecomment-674781958
@corleyma Unfortunately, the "ursabot" builders use a different execution
scheme. The packages need to be updated manually there, I beliave. I pinged
@kszucs who should be able to
github-actions[bot] commented on pull request #7980:
URL: https://github.com/apache/arrow/pull/7980#issuecomment-674850748
https://issues.apache.org/jira/browse/ARROW-9768
This is an automated message from the Apache Git
alamb commented on a change in pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#discussion_r471445155
##
File path: rust/datafusion/src/optimizer/type_coercion.rs
##
@@ -345,4 +345,143 @@ mod tests {
assert_eq!(expected, format!("{:?}", expr2));
pitrou commented on pull request #7945:
URL: https://github.com/apache/arrow/pull/7945#issuecomment-674884766
Will merge if CI passes. Thank you @MingyuZhong !
This is an automated message from the Apache Git Service.
To
kszucs commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674887212
@github-actions crossbow submit -g integration -g conda -g wheel -g linux
This is an automated message from the
pitrou commented on pull request #7803:
URL: https://github.com/apache/arrow/pull/7803#issuecomment-674891087
Seems like we're fine now, will merge.
This is an automated message from the Apache Git Service.
To respond to the
pitrou commented on pull request #7789:
URL: https://github.com/apache/arrow/pull/7789#issuecomment-674893008
@patrickpai Do you have some time to make the desired changes here?
This is an automated message from the Apache
andygrove commented on a change in pull request #7980:
URL: https://github.com/apache/arrow/pull/7980#discussion_r471502392
##
File path: rust/datafusion/src/execution/physical_plan/mod.rs
##
@@ -33,7 +33,8 @@ use arrow::{
};
use udf::ScalarFunction;
-/// Physical query
github-actions[bot] commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674910699
Revision: 2c362d6fcc0e29bd5b9ab2d5c62d0d759b44404d
Submitted crossbow builds: [ursa-labs/crossbow @
alamb opened a new pull request #7980:
URL: https://github.com/apache/arrow/pull/7980
Proposed follow up to https://github.com/apache/arrow/pull/7975 -- rename
`PhysicalPlannerImpl` to `DefaultPhysicalPlanner` to better describe what it is
and that the design allows for more than one.
alamb commented on a change in pull request #7978:
URL: https://github.com/apache/arrow/pull/7978#discussion_r471440516
##
File path: rust/datafusion/src/lib.rs
##
@@ -15,17 +15,46 @@
// specific language governing permissions and limitations
// under the License.
pitrou commented on a change in pull request #7945:
URL: https://github.com/apache/arrow/pull/7945#discussion_r471472052
##
File path: cpp/src/arrow/util/decimal.cc
##
@@ -241,64 +284,43 @@ Decimal128::operator int64_t() const {
return static_cast(low_bits());
}
-static
pitrou closed pull request #7803:
URL: https://github.com/apache/arrow/pull/7803
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
kszucs commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674909175
@github-actions crossbow submit -g integration -g conda -g wheel -g linux
This is an automated message from the
alamb commented on a change in pull request #7975:
URL: https://github.com/apache/arrow/pull/7975#discussion_r471437170
##
File path: rust/datafusion/src/execution/context.rs
##
@@ -373,363 +355,12 @@ impl ExecutionContext {
pub fn create_physical_plan(
,
andygrove closed pull request #7980:
URL: https://github.com/apache/arrow/pull/7980
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
lidavidm opened a new pull request #7981:
URL: https://github.com/apache/arrow/pull/7981
We were accidentally trying to recursively hold a lock.
This is an automated message from the Apache Git Service.
To respond to the
kszucs commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674995771
The three integration build failures and the two GHA errors are unrelated,
so merging it to the upstream maintenance branch before creating the tag.
kszucs commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674996246
`Rebase and merge` is disabled, so I'm pushing it directly to the upstream
branch.
This is an automated message
kszucs merged pull request #7933:
URL: https://github.com/apache/arrow/pull/7933
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
github-actions[bot] commented on pull request #7981:
URL: https://github.com/apache/arrow/pull/7981#issuecomment-675003254
https://issues.apache.org/jira/browse/ARROW-9670
This is an automated message from the Apache Git
alamb commented on a change in pull request #7975:
URL: https://github.com/apache/arrow/pull/7975#discussion_r471433074
##
File path: rust/datafusion/src/execution/context.rs
##
@@ -1452,11 +1109,34 @@ mod tests {
Ok(())
}
+#[test]
+fn
alamb commented on a change in pull request #7972:
URL: https://github.com/apache/arrow/pull/7972#discussion_r471438457
##
File path: rust/datafusion/src/execution/context.rs
##
@@ -89,12 +91,18 @@ impl ExecutionConfig {
}
/// Execution context for registering data sources
alamb commented on a change in pull request #7967:
URL: https://github.com/apache/arrow/pull/7967#discussion_r471442333
##
File path: rust/datafusion/src/sql/planner.rs
##
@@ -523,10 +523,14 @@ impl SqlToRel {
let mut safe_args: Vec = vec![];
pitrou commented on a change in pull request #7898:
URL: https://github.com/apache/arrow/pull/7898#discussion_r471486448
##
File path: cpp/src/arrow/array/array_dict_test.cc
##
@@ -904,6 +904,67 @@ TEST(TestDecimalDictionaryBuilder, DoubleTableSize) {
github-actions[bot] commented on pull request #7933:
URL: https://github.com/apache/arrow/pull/7933#issuecomment-674889759
Revision: af3cc932f50f5052b493f48f6d3c24b8d2179a27
Submitted crossbow builds: [ursa-labs/crossbow @
pitrou commented on a change in pull request #7748:
URL: https://github.com/apache/arrow/pull/7748#discussion_r471518131
##
File path: cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc
##
@@ -492,6 +486,81 @@ TYPED_TEST(TestBinaryArithmeticFloating, Add) {
alamb commented on pull request #7880:
URL: https://github.com/apache/arrow/pull/7880#issuecomment-674923231
> > Something that can be left for future optimization: we can also go the
other direction, i.e. break `And` filters into into individual boolean
expressions so these filters can
alamb commented on a change in pull request #7880:
URL: https://github.com/apache/arrow/pull/7880#discussion_r471103102
##
File path: rust/datafusion/src/optimizer/filter_push_down.rs
##
@@ -0,0 +1,631 @@
+// regarding copyright ownership. The ASF licenses this file
+// to
pitrou closed pull request #7826:
URL: https://github.com/apache/arrow/pull/7826
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
94 matches
Mail list logo
