[arrow] branch master updated (6c289dd -> 9162954)
This is an automated email from the ASF dual-hosted git repository. kou pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/arrow.git. from 6c289dd ARROW-13043: [GLib][Ruby] Add GArrowEqualOptions add 9162954 ARROW-13065: [Packaging][RPM] Add missing required LZ4 version information No new revisions were added by this update. Summary of changes: dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-)
[arrow] branch master updated (08781a3 -> 6c289dd)
This is an automated email from the ASF dual-hosted git repository.
kou pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.
from 08781a3 ARROW-12801: [CI][Packaging][Java] Include all modules in
script that generate Arrow jars
add 6c289dd ARROW-13043: [GLib][Ruby] Add GArrowEqualOptions
No new revisions were added by this update.
Summary of changes:
c_glib/arrow-glib/basic-array.cpp | 216 -
c_glib/arrow-glib/basic-array.h| 23 ++-
c_glib/arrow-glib/basic-array.hpp | 3 +
c_glib/arrow-glib/compute.cpp | 2 +-
c_glib/test/test-equal-options.rb | 96 +
ruby/red-arrow/lib/arrow/array.rb | 12 ++
.../lib/arrow/equal-options.rb}| 11 +-
ruby/red-arrow/lib/arrow/loader.rb | 1 +
ruby/red-arrow/test/test-array.rb | 34
9 files changed, 386 insertions(+), 12 deletions(-)
create mode 100644 c_glib/test/test-equal-options.rb
copy ruby/{red-arrow-flight/lib/arrow-flight/client-options.rb =>
red-arrow/lib/arrow/equal-options.rb} (82%)
[arrow-datafusion] branch master updated: reuse datafusion physical planner in ballista building from protobuf (#532)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/master by this push:
new ad70a1e reuse datafusion physical planner in ballista building from
protobuf (#532)
ad70a1e is described below
commit ad70a1e91667174436f2110a70e3e557c7069e9a
Author: Jiayu Liu
AuthorDate: Sat Jun 12 02:50:23 2021 +0800
reuse datafusion physical planner in ballista building from protobuf (#532)
* use logical planner in ballista building
* simplify statement
* fix unit test
* fix per comment
---
.../core/src/serde/physical_plan/from_proto.rs | 142 -
datafusion/src/physical_plan/planner.rs| 116 ++---
datafusion/src/physical_plan/windows.rs| 44 +--
3 files changed, 153 insertions(+), 149 deletions(-)
diff --git a/ballista/rust/core/src/serde/physical_plan/from_proto.rs
b/ballista/rust/core/src/serde/physical_plan/from_proto.rs
index b319d5b..d49d53c 100644
--- a/ballista/rust/core/src/serde/physical_plan/from_proto.rs
+++ b/ballista/rust/core/src/serde/physical_plan/from_proto.rs
@@ -36,7 +36,7 @@ use datafusion::execution::context::{
ExecutionConfig, ExecutionContextState, ExecutionProps,
};
use datafusion::logical_plan::{DFSchema, Expr};
-use datafusion::physical_plan::aggregates::{create_aggregate_expr,
AggregateFunction};
+use datafusion::physical_plan::aggregates::AggregateFunction;
use datafusion::physical_plan::expressions::col;
use datafusion::physical_plan::hash_aggregate::{AggregateMode,
HashAggregateExec};
use datafusion::physical_plan::hash_join::PartitionMode;
@@ -45,7 +45,6 @@ use
datafusion::physical_plan::planner::DefaultPhysicalPlanner;
use datafusion::physical_plan::window_functions::{
BuiltInWindowFunction, WindowFunction,
};
-use datafusion::physical_plan::windows::create_window_expr;
use datafusion::physical_plan::windows::WindowAggExec;
use datafusion::physical_plan::{
coalesce_batches::CoalesceBatchesExec,
@@ -205,76 +204,27 @@ impl TryInto> for
&protobuf::PhysicalPlanNode {
)
})?
.clone();
-
let physical_schema: SchemaRef =
SchemaRef::new((&input_schema).try_into()?);
-
-let catalog_list =
-Arc::new(MemoryCatalogList::new()) as Arc;
-let ctx_state = ExecutionContextState {
-catalog_list,
-scalar_functions: Default::default(),
-var_provider: Default::default(),
-aggregate_functions: Default::default(),
-config: ExecutionConfig::new(),
-execution_props: ExecutionProps::new(),
-};
-
+let ctx_state = ExecutionContextState::new();
let window_agg_expr: Vec<(Expr, String)> = window_agg
.window_expr
.iter()
.zip(window_agg.window_expr_name.iter())
.map(|(expr, name)| expr.try_into().map(|expr| (expr,
name.clone(
.collect::, _>>()?;
-
-let mut physical_window_expr = vec![];
-
let df_planner = DefaultPhysicalPlanner::default();
-
-for (expr, name) in &window_agg_expr {
-match expr {
-Expr::WindowFunction {
-fun,
-args,
-partition_by,
-order_by,
-window_frame,
-..
-} => {
-let arg = df_planner
-.create_physical_expr(
-&args[0],
-&physical_schema,
-&ctx_state,
-)
-.map_err(|e| {
-BallistaError::General(format!("{:?}", e))
-})?;
-if !partition_by.is_empty() {
-return
Err(BallistaError::NotImplemented("Window function with partition by is not yet
implemented".to_owned()));
-}
-if !order_by.is_empty() {
-return
Err(BallistaError::NotImplemented("Window function with order by is not yet
implemented".to_owned()));
-}
-if window_frame.is_some() {
-return
Err(BallistaError::NotImplemented("Window function with window frame is not yet
implemented".to_owned()));
-
[arrow-datafusion] branch master updated (3ef7f34 -> 63e3045)
This is an automated email from the ASF dual-hosted git repository. agrove pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git. from 3ef7f34 use nightly nightly-2021-05-10 (#536) add 63e3045 Make BallistaContext::collect streaming (#535) No new revisions were added by this update. Summary of changes: ballista/rust/client/src/context.rs | 113 +++- 1 file changed, 72 insertions(+), 41 deletions(-)
[arrow] branch master updated (5526633 -> 08781a3)
This is an automated email from the ASF dual-hosted git repository. kszucs pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/arrow.git. from 5526633 ARROW-13030: [CI][Go] Setup Arm64 golang CI add 08781a3 ARROW-12801: [CI][Packaging][Java] Include all modules in script that generate Arrow jars No new revisions were added by this update. Summary of changes: ci/scripts/java_jni_build.sh | 35 +++ dev/tasks/java-jars/github.yml | 19 ++- dev/tasks/tasks.yml| 39 +-- 3 files changed, 58 insertions(+), 35 deletions(-)
[arrow-site] 01/01: add kiszk as a committer
This is an automated email from the ASF dual-hosted git repository. kiszk pushed a commit to branch Arrow-13047 in repository https://gitbox.apache.org/repos/asf/arrow-site.git commit f4fbc497212b4ca44e5c09482187b3ba41c1d243 Author: ishizaki AuthorDate: Fri Jun 11 07:52:43 2021 + add kiszk as a committer --- _data/committers.yml | 4 1 file changed, 4 insertions(+) diff --git a/_data/committers.yml b/_data/committers.yml index d473c60..1b63acf 100644 --- a/_data/committers.yml +++ b/_data/committers.yml @@ -184,6 +184,10 @@ role: Committer alias: jorisvandenbossche affiliation: Ursa Computing +- name: Kazuaki ishizaki + role: Committer + alias: kiszk + affiliation: IBM - name: Kenta Murata role: Committer alias: mrkn
[arrow-site] branch Arrow-13047 created (now f4fbc49)
This is an automated email from the ASF dual-hosted git repository. kiszk pushed a change to branch Arrow-13047 in repository https://gitbox.apache.org/repos/asf/arrow-site.git. at f4fbc49 add kiszk as a committer This branch includes the following new commits: new f4fbc49 add kiszk as a committer The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[arrow] branch master updated (e6c0b81 -> 5526633)
This is an automated email from the ASF dual-hosted git repository.
kou pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.
from e6c0b81 ARROW-13045: [Packaging][RPM][deb] Don't install system
utf8proc if it's old
add 5526633 ARROW-13030: [CI][Go] Setup Arm64 golang CI
No new revisions were added by this update.
Summary of changes:
.travis.yml | 12
go/arrow/internal/cpu/{cpu_s390x.go => cpu_arm64.go} | 2 +-
go/arrow/math/{float64_s390x.go => float64_arm64.go} | 0
go/arrow/math/{int64_s390x.go => int64_arm64.go} | 0
go/arrow/math/{math_s390x.go => math_arm64.go} | 0
go/arrow/math/{uint64_s390x.go => uint64_arm64.go} | 0
.../bmi/{bitmap_bmi2_s390x.go => bitmap_bmi2_arm64.go} | 0
.../utils/{bit_packing_s390x.go => bit_packing_arm64.go} | 0
.../utils/{unpack_bool_s390x.go => unpack_bool_arm64.go} | 0
9 files changed, 13 insertions(+), 1 deletion(-)
copy go/arrow/internal/cpu/{cpu_s390x.go => cpu_arm64.go} (86%)
copy go/arrow/math/{float64_s390x.go => float64_arm64.go} (100%)
copy go/arrow/math/{int64_s390x.go => int64_arm64.go} (100%)
copy go/arrow/math/{math_s390x.go => math_arm64.go} (100%)
copy go/arrow/math/{uint64_s390x.go => uint64_arm64.go} (100%)
copy go/parquet/internal/bmi/{bitmap_bmi2_s390x.go => bitmap_bmi2_arm64.go}
(100%)
copy go/parquet/internal/utils/{bit_packing_s390x.go => bit_packing_arm64.go}
(100%)
copy go/parquet/internal/utils/{unpack_bool_s390x.go => unpack_bool_arm64.go}
(100%)
