This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-ballista.git
The following commit(s) were added to refs/heads/master by this push:
new 8032e1ab Add user guide for Flight SQL (#264)
8032e1ab is described below
commit 8032e1ab1b7fd2b1549812843571bbfdabeacc9c
Author: Brent Gardner <[email protected]>
AuthorDate: Thu Sep 22 17:43:17 2022 -0700
Add user guide for Flight SQL (#264)
---
ballista/rust/scheduler/src/planner.rs | 2 +-
.../rust/scheduler/src/state/execution_graph.rs | 2 +-
benchmarks/src/bin/tpch.rs | 6 +-
dev/build-ballista-rust.sh | 24 ++++
docs/README.md | 3 +-
docs/source/user-guide/flightsql.md | 121 +++++++++++++++++++++
docs/source/user-guide/introduction.md | 4 +
7 files changed, 156 insertions(+), 6 deletions(-)
diff --git a/ballista/rust/scheduler/src/planner.rs
b/ballista/rust/scheduler/src/planner.rs
index fc82590b..474fd63d 100644
--- a/ballista/rust/scheduler/src/planner.rs
+++ b/ballista/rust/scheduler/src/planner.rs
@@ -624,7 +624,7 @@ order by
codec.physical_extension_codec(),
)?;
let runtime = ctx.runtime_env();
- let result_exec_plan: Arc<dyn ExecutionPlan> =
(&proto).try_into_physical_plan(
+ let result_exec_plan: Arc<dyn ExecutionPlan> =
(proto).try_into_physical_plan(
&ctx,
runtime.deref(),
codec.physical_extension_codec(),
diff --git a/ballista/rust/scheduler/src/state/execution_graph.rs
b/ballista/rust/scheduler/src/state/execution_graph.rs
index ca44a24c..3d49ee93 100644
--- a/ballista/rust/scheduler/src/state/execution_graph.rs
+++ b/ballista/rust/scheduler/src/state/execution_graph.rs
@@ -551,7 +551,7 @@ impl ExecutionGraph {
.iter()
.enumerate()
.filter_map(|(idx, loc)| {
- (loc.executor_meta.id ==
executor_id).then(|| idx)
+ (loc.executor_meta.id ==
executor_id).then_some(idx)
})
.collect::<Vec<_>>();
diff --git a/benchmarks/src/bin/tpch.rs b/benchmarks/src/bin/tpch.rs
index 18b1b018..260d8949 100644
--- a/benchmarks/src/bin/tpch.rs
+++ b/benchmarks/src/bin/tpch.rs
@@ -1506,7 +1506,7 @@ mod tests {
codec.logical_extension_codec(),
)
.unwrap();
- let round_trip: LogicalPlan = (&proto)
+ let round_trip: LogicalPlan = (proto)
.try_into_logical_plan(&ctx,
codec.logical_extension_codec())
.unwrap();
assert_eq!(
@@ -1523,7 +1523,7 @@ mod tests {
codec.logical_extension_codec(),
)
.unwrap();
- let round_trip: LogicalPlan = (&proto)
+ let round_trip: LogicalPlan = (proto)
.try_into_logical_plan(&ctx,
codec.logical_extension_codec())
.unwrap();
assert_eq!(
@@ -1542,7 +1542,7 @@ mod tests {
)
.unwrap();
let runtime = ctx.runtime_env();
- let round_trip: Arc<dyn ExecutionPlan> = (&proto)
+ let round_trip: Arc<dyn ExecutionPlan> = (proto)
.try_into_physical_plan(
&ctx,
runtime.deref(),
diff --git a/dev/build-ballista-rust.sh b/dev/build-ballista-rust.sh
new file mode 100755
index 00000000..d75275fc
--- /dev/null
+++ b/dev/build-ballista-rust.sh
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+
+docker build -t ballista-builder --build-arg EXT_UID="$(id -u)" -f
dev/docker/ballista-builder.Dockerfile .
+
+docker run -v $(pwd):/home/builder/workspace ballista-builder
diff --git a/docs/README.md b/docs/README.md
index bfddcbca..920ba494 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -19,7 +19,8 @@
# Developer Documentation
-Developer documentation can be found [here](developer/).
+Developer documentation can be found [here](developer/README.md).
+User documentation can be found [here](source/user-guide/introduction.md).
# User Documentation
diff --git a/docs/source/user-guide/flightsql.md
b/docs/source/user-guide/flightsql.md
new file mode 100644
index 00000000..df990d01
--- /dev/null
+++ b/docs/source/user-guide/flightsql.md
@@ -0,0 +1,121 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Using FlightSQL to Connect to Ballista
+
+One of the easiest ways to start with Ballista is to plug it into your
existing data infrastructure using support for Arrow Flight SQL JDBC.
+
+Getting started involves these main steps:
+
+1. [Installing prerequisites](#prereq)
+2. Build the [Ballista rust code](#rust)
+3. Build and run the [Ballista docker containers](#docker)
+4. Build the [Arrow Flight SQL JDBC Driver](#jdbc)
+5. [Install the driver](#tool) into your favorite JDBC tool
+6. Run a ["hello, world!"](#hello) query
+7. Register a table and run more complicated queries
+
+## <a name="prereq"/>Prerequisites
+
+### Ubuntu
+
+```shell
+sudo apt-get update
+sudo apt-get install -y docker.io docker-compose
+```
+
+### MacOS
+
+```shell
+brew install docker docker-compose
+```
+
+### Windows
+
+```shell
+choco install docker-desktop docker-compose
+```
+
+## <a name="rust"/>Building Ballista
+
+```shell
+git clone https://github.com/apache/arrow-ballista.git
+dev/build-ballista-rust.sh
+```
+
+## <a name="docker"/> Run Docker Containers
+
+```shell
+docker-compose up --build
+```
+
+## <a name="jdbc"/>Build the FlightSQL JDBC Driver
+
+**Note:** this will no longer be necessary when Arrow v10 is released
approximately 2022-10-31
+
+**Note:** A full explaination of the Arrow Java build is out-of-scope for this
document. Please refer to that project for more detailed instructions.
+
+```shell
+git clone https://github.com/apache/arrow.git
+cd java
+mvn install -DskipTests -Dcheckstyle.skip -Drat.skip=true -pl
:flight-sql-jdbc-driver -am
+find . -name "*.jar"
+...
+./flight/flight-sql-jdbc-driver/target/flight-sql-jdbc-driver-10.0.0-SNAPSHOT.jar
+...
+```
+
+## <a name="tool"/>Use the Driver in your Favorite Data Tool
+
+The important pieces of information:
+
+| Key | Value |
+| ---------------- | -------------------------------------------------- |
+| Driver file | flight-sql-jdbc-driver-10.0.0-SNAPSHOT.jar |
+| Class Name | org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver |
+| Authentication | User & Password |
+| Username | admin |
+| Password | password |
+| Advanced Options | useEncryption=false |
+| URL | jdbc:arrow-flight://127.0.0.1:50050 |
+
+## <a name="hello"/>Run a "Hello, World!" Query
+
+```sql
+select 'Hello from Arrow Ballista!' as greeting;
+```
+
+## <a name="complex"/>Run a Complex Query
+
+In order to run queries against data, tables need to be "registered" with the
current session (and re-registered upon each new connection).
+
+To register a table, find a `.csv`, `.json`, or `.parquet` file for testing,
and use the syntax below:
+
+```sql
+create external table customer stored as CSV with header row
+ location
'/home/username/arrow-datafusion/datafusion/core/tests/tpch-csv/customer.csv';
+```
+
+Once the table has been registered, all the normal SQL queries can be
performed:
+
+```sql
+select * from customer;
+```
+
+🎉 Happy querying! 🎉
diff --git a/docs/source/user-guide/introduction.md
b/docs/source/user-guide/introduction.md
index 77db6261..db7d0e13 100644
--- a/docs/source/user-guide/introduction.md
+++ b/docs/source/user-guide/introduction.md
@@ -48,3 +48,7 @@ Although Ballista is largely inspired by Apache Spark, there
are some key differ
## Status
Ballista is still in the early stages of development but is capable of
executing complex analytical queries at scale.
+
+## Usage
+
+Ballista can be used from your favorite JDBC compliant tools such as
[DataGrip](https://www.jetbrains.com/datagrip/) or
[tableau](https://help.tableau.com/current/pro/desktop/en-us/examples_otherdatabases_jdbc.htm).
For setup instructions, please see the [FlightSQL guide](flightsql.md).
