This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/arrow-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new b5874b5a3bb Add Ballista contributor's guide (#475)
b5874b5a3bb is described below
commit b5874b5a3bb111253054c604866b224d15c5a04f
Author: Andy Grove <[email protected]>
AuthorDate: Sun Feb 11 10:44:44 2024 -0700
Add Ballista contributor's guide (#475)
---
ballista/_images/ballista.drawio.png | Bin 0 -> 73216 bytes
ballista/community/communication.html | 47 +-
ballista/contributors-guide/architecture.html | 599 +++++++++++++++++++++
.../code-organization.html} | 182 +++----
.../development.html} | 175 +++---
ballista/genindex.html | 29 +-
ballista/index.html | 40 +-
ballista/objects.inv | Bin 705 -> 754 bytes
ballista/search.html | 29 +-
ballista/searchindex.js | 2 +-
ballista/user-guide/cli.html | 29 +-
ballista/user-guide/configs.html | 29 +-
ballista/user-guide/deployment/cargo-install.html | 29 +-
ballista/user-guide/deployment/docker-compose.html | 29 +-
ballista/user-guide/deployment/docker.html | 29 +-
ballista/user-guide/deployment/index.html | 29 +-
ballista/user-guide/deployment/kubernetes.html | 29 +-
ballista/user-guide/deployment/quick-start.html | 29 +-
ballista/user-guide/faq.html | 41 +-
ballista/user-guide/flightsql.html | 29 +-
ballista/user-guide/introduction.html | 29 +-
ballista/user-guide/metrics.html | 47 +-
ballista/user-guide/python.html | 29 +-
ballista/user-guide/rust.html | 29 +-
ballista/user-guide/scheduler.html | 29 +-
ballista/user-guide/tuning-guide.html | 35 +-
26 files changed, 1253 insertions(+), 350 deletions(-)
diff --git a/ballista/_images/ballista.drawio.png
b/ballista/_images/ballista.drawio.png
new file mode 100644
index 00000000000..7d924a14b16
Binary files /dev/null and b/ballista/_images/ballista.drawio.png differ
diff --git a/ballista/community/communication.html
b/ballista/community/communication.html
index d5a911f0cd7..8e76732b980 100644
--- a/ballista/community/communication.html
+++ b/ballista/community/communication.html
@@ -33,8 +33,7 @@
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Ballista Development" href="development.html" />
- <link rel="prev" title="Frequently Asked Questions"
href="../user-guide/faq.html" />
+ <link rel="prev" title="Ballista Development"
href="../contributors-guide/development.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -172,6 +171,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../user-guide/faq.html">
Frequently Asked Questions
@@ -180,13 +184,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +220,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
@@ -317,7 +331,7 @@
<h1>Communication<a class="headerlink" href="#communication" title="Link to
this heading">¶</a></h1>
<p>We welcome participation from everyone and encourage you to join us, ask
questions, and get involved.</p>
-<p>All participation in the Apache Arrow Ballista project is governed by the
+<p>All participation in the Apache Arrow DataFusion project is governed by the
Apache Software Foundation’s <a class="reference external"
href="https://www.apache.org/foundation/policies/conduct.html";>code of
conduct</a>.</p>
<section id="questions">
@@ -330,7 +344,7 @@ coorindation and design discussions
<a class="reference external"
href="mailto:dev-unsubscribe%40arrow.apache.org";>unsubscribe</a>,
<a class="reference external"
href="https://lists.apache.org/list.html?dev@arrow.apache.org";>archives</a>).</p>
<p>When emailing the dev list, please make sure to prefix the subject line
with a
-<code class="docutils literal notranslate"><span
class="pre">[Ballista]</span></code> tag, e.g. <code class="docutils literal
notranslate"><span class="pre">"[Ballista]</span> <span
class="pre">New</span> <span class="pre">API</span> <span
class="pre">for</span> <span class="pre">remote</span> <span
class="pre">data</span> <span class="pre">sources"</span></code>, so
+<code class="docutils literal notranslate"><span
class="pre">[DataFusion]</span></code> tag, e.g. <code class="docutils literal
notranslate"><span class="pre">"[DataFusion]</span> <span
class="pre">New</span> <span class="pre">API</span> <span
class="pre">for</span> <span class="pre">remote</span> <span
class="pre">data</span> <span class="pre">sources"</span></code>, so
that the appropriate people in the Apache Arrow community notice the
message.</p>
</section>
<section id="slack-and-discord">
@@ -380,20 +394,13 @@ quick and small design discussions. For major design
change proposals, we encour
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="../user-guide/faq.html"
title="previous page">
+ <a class='left-prev' id="prev-link"
href="../contributors-guide/development.html" title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Frequently Asked Questions</p>
+ <p class="prev-next-title">Ballista Development</p>
</div>
</a>
- <a class='right-next' id="next-link" href="development.html" title="next
page">
- <div class="prev-next-info">
- <p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Ballista Development</p>
- </div>
- <i class="fas fa-angle-right"></i>
- </a>
</div>
</main>
diff --git a/ballista/contributors-guide/architecture.html
b/ballista/contributors-guide/architecture.html
new file mode 100644
index 00000000000..e2aee7f72e8
--- /dev/null
+++ b/ballista/contributors-guide/architecture.html
@@ -0,0 +1,599 @@
+<!DOCTYPE html>
+
+<html lang="en" data-content_root="../">
+ <head>
+ <meta charset="utf-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0"
/><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <title>Ballista Architecture — Apache Arrow Ballista
documentation</title>
+
+ <link href="../_static/styles/theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
+<link
href="../_static/styles/pydata-sphinx-theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
+
+
+ <link rel="stylesheet"
+ href="../_static/vendor/fontawesome/5.13.0/css/all.min.css">
+ <link rel="preload" as="font" type="font/woff2" crossorigin
+ href="../_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
+ <link rel="preload" as="font" type="font/woff2" crossorigin
+ href="../_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
+
+
+
+
+
+ <link rel="stylesheet" type="text/css"
href="../_static/pygments.css?v=362ab14a" />
+ <link rel="stylesheet" type="text/css"
href="../_static/styles/pydata-sphinx-theme.css?v=1140d252" />
+ <link rel="stylesheet" type="text/css"
href="../_static/theme_overrides.css?v=ef9fea58" />
+
+ <link rel="preload" as="script"
href="../_static/scripts/pydata-sphinx-theme.js?digest=1999514e3f237ded88cf">
+
+ <script src="../_static/documentation_options.js?v=8a448e45"></script>
+ <script src="../_static/doctools.js?v=888ff710"></script>
+ <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="Ballista Code Organization"
href="code-organization.html" />
+ <link rel="prev" title="Frequently Asked Questions"
href="../user-guide/faq.html" />
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
+ <meta name="docsearch:language" content="en">
+
+
+ <!-- Google Analytics -->
+
+ </head>
+ <body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
+
+ <div class="container-fluid" id="banner"></div>
+
+
+
+
+ <div class="container-xl">
+ <div class="row">
+
+
+ <!-- Only show if we have sidebars configured, else just a small
margin -->
+ <div class="col-12 col-md-3 bd-sidebar">
+ <div class="sidebar-start-items">
+<a class="navbar-brand" href="../index.html">
+ <img src="../_static/images/ballista-logo.png" class="logo" alt="logo">
+</a>
+
+<form class="bd-search d-flex align-items-center" action="../search.html"
method="get">
+ <i class="icon fas fa-search"></i>
+ <input type="search" class="form-control" name="q" id="search-input"
placeholder="Search the docs ..." aria-label="Search the docs ..."
autocomplete="off" >
+</form>
+
+<nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
+ <div class="bd-toc-item active">
+
+ <p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ User Guide
+ </span>
+</p>
+<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/introduction.html">
+ Introduction
+ </a>
+ </li>
+</ul>
+<p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ Cluster Deployment
+ </span>
+</p>
+<ul class="nav bd-sidenav">
+ <li class="toctree-l1 has-children">
+ <a class="reference internal" href="../user-guide/deployment/index.html">
+ Deployment
+ </a>
+ <input class="toctree-checkbox" id="toctree-checkbox-1"
name="toctree-checkbox-1" type="checkbox"/>
+ <label for="toctree-checkbox-1">
+ <i class="fas fa-chevron-down">
+ </i>
+ </label>
+ <ul>
+ <li class="toctree-l2">
+ <a class="reference internal"
href="../user-guide/deployment/quick-start.html">
+ Quick Start
+ </a>
+ </li>
+ <li class="toctree-l2">
+ <a class="reference internal"
href="../user-guide/deployment/cargo-install.html">
+ Cargo Install
+ </a>
+ </li>
+ <li class="toctree-l2">
+ <a class="reference internal" href="../user-guide/deployment/docker.html">
+ Docker
+ </a>
+ </li>
+ <li class="toctree-l2">
+ <a class="reference internal"
href="../user-guide/deployment/docker-compose.html">
+ Docker Compose
+ </a>
+ </li>
+ <li class="toctree-l2">
+ <a class="reference internal"
href="../user-guide/deployment/kubernetes.html">
+ Kubernetes
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/scheduler.html">
+ Scheduler
+ </a>
+ </li>
+</ul>
+<p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ Clients
+ </span>
+</p>
+<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/python.html">
+ Python
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/rust.html">
+ Rust
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/flightsql.html">
+ Flight SQL JDBC
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/cli.html">
+ SQL CLI
+ </a>
+ </li>
+</ul>
+<p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ Reference
+ </span>
+</p>
+<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/configs.html">
+ Configuration
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/tuning-guide.html">
+ Tuning Guide
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/faq.html">
+ Frequently Asked Questions
+ </a>
+ </li>
+</ul>
+<p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ Contributors Guide
+ </span>
+</p>
+<ul class="current nav bd-sidenav">
+ <li class="toctree-l1 current active">
+ <a class="current reference internal" href="#">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="development.html">
+ Ballista Development
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
+ Source code
+ </a>
+ </li>
+</ul>
+<p aria-level="2" class="caption" role="heading">
+ <span class="caption-text">
+ Community
+ </span>
+</p>
+<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../community/communication.html">
+ Communication
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
+ Issue tracker
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/CODE_OF_CONDUCT.md";>
+ Code of conduct
+ </a>
+ </li>
+</ul>
+
+
+ </div>
+</nav>
+ </div>
+ <div class="sidebar-end-items">
+ </div>
+ </div>
+
+
+
+
+ <div class="d-none d-xl-block col-xl-2 bd-toc">
+
+
+ <div class="toc-item">
+
+<div class="tocsection onthispage pt-5 pb-3">
+ <i class="fas fa-list"></i> On this page
+</div>
+
+<nav id="bd-toc-nav">
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#overview">
+ Overview
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#design-principles">
+ Design Principles
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#arrow-native">
+ Arrow-native
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#language-agnostic">
+ Language Agnostic
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#extensible">
+ Extensible
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#deployment-architecture">
+ Deployment Architecture
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#cluster">
+ Cluster
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#scheduler">
+ Scheduler
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#executor">
+ Executor
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#clients">
+ Clients
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#distributed-query-scheduling">
+ Distributed Query Scheduling
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link"
href="#producing-a-distributed-query-plan">
+ Producing a Distributed Query Plan
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#shuffle">
+ Shuffle
+ </a>
+ </li>
+ </ul>
+ </li>
+</ul>
+
+</nav>
+ </div>
+
+ <div class="toc-item">
+
+
+<div class="tocsection editthispage">
+ <a
href="https://github.com/apache/arrow-ballista/edit/main/docs/source/contributors-guide/architecture.md";>
+ <i class="fas fa-pencil-alt"></i> Edit this page
+ </a>
+</div>
+
+ </div>
+
+
+ </div>
+
+
+
+
+
+
+ <main class="col-12 col-md-9 col-xl-7 py-md-5 pl-md-5 pr-md-4
bd-content" role="main">
+
+ <div>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<section id="ballista-architecture">
+<h1>Ballista Architecture<a class="headerlink" href="#ballista-architecture"
title="Link to this heading">¶</a></h1>
+<section id="overview">
+<h2>Overview<a class="headerlink" href="#overview" title="Link to this
heading">¶</a></h2>
+<p>Ballista’s primary purpose is to provide a distributed SQL query engine
implemented in the Rust programming
+language and using the Apache Arrow memory model.</p>
+<p>Ballista also provides a DataFrame API (both in Rust and Python), suitable
for constructing ETL pipelines and
+analytical queries. The DataFrame API is inspired by Apache Spark and is
currently better suited for ETL/SQL work
+than for data science.</p>
+</section>
+<section id="design-principles">
+<h2>Design Principles<a class="headerlink" href="#design-principles"
title="Link to this heading">¶</a></h2>
+<section id="arrow-native">
+<h3>Arrow-native<a class="headerlink" href="#arrow-native" title="Link to this
heading">¶</a></h3>
+<p>Ballista uses the Apache Arrow memory format during query execution, and
Apache Arrow IPC format on disk for
+shuffle files and for exchanging data between executors. Queries can be
submitted using the Arrow Flight SQL API
+and the Arrow Flight SQL JDBC Driver.</p>
+</section>
+<section id="language-agnostic">
+<h3>Language Agnostic<a class="headerlink" href="#language-agnostic"
title="Link to this heading">¶</a></h3>
+<p>Although most of the implementation code is written in Rust, the scheduler
and executor APIs are based on open
+standards, including protocol buffers, gRPC, Apache Arrow IPC, and Apache
Arrow Flight SQL.</p>
+<p>This language agnostic approach will allow Ballista to eventually support
UDFs in languages other than Rust,
+including Wasm.</p>
+</section>
+<section id="extensible">
+<h3>Extensible<a class="headerlink" href="#extensible" title="Link to this
heading">¶</a></h3>
+<p>Many Ballista users have their own distributed query engines that use
Ballista as a foundation, rather than
+using Ballista directly. This allows the scheduler and executor processes to
be extended with support for
+additional data formats, operators, expressions, or custom SQL dialects or
other DSLs.</p>
+<p>Ballista uses the DataFusion query engine for query execution, but it
should be possible to plug in other execution
+engines.</p>
+</section>
+</section>
+<section id="deployment-architecture">
+<h2>Deployment Architecture<a class="headerlink"
href="#deployment-architecture" title="Link to this heading">¶</a></h2>
+<section id="cluster">
+<h3>Cluster<a class="headerlink" href="#cluster" title="Link to this
heading">¶</a></h3>
+<p>A Ballista cluster consists of one or more scheduler processes and one or
more executor processes. These processes
+can be run as native binaries and are also available as Docker Images, which
can be easily deployed with
+<a class="reference external"
href="https://arrow.apache.org/ballista/user-guide/deployment/docker-compose.html";>Docker
Compose</a> or
+<a class="reference external"
href="https://arrow.apache.org/ballista/user-guide/deployment/kubernetes.html";>Kubernetes</a>.</p>
+<p>The following diagram shows the interaction between clients and the
scheduler for submitting jobs, and the interaction
+between the executor(s) and the scheduler for fetching tasks and reporting
task status.</p>
+<p><img alt="Ballista Cluster Diagram" src="../_images/ballista.drawio.png"
/></p>
+</section>
+<section id="scheduler">
+<h3>Scheduler<a class="headerlink" href="#scheduler" title="Link to this
heading">¶</a></h3>
+<p>The scheduler provides the following interfaces:</p>
+<ul class="simple">
+<li><p>gRPC service for submitting and managing jobs</p></li>
+<li><p>Flight SQL API</p></li>
+<li><p>REST API for monitoring jobs</p></li>
+<li><p>Web user interface for monitoring jobs</p></li>
+</ul>
+<p>Jobs are submitted to the scheduler’s gRPC service from a client context,
either in the form of a logical query
+plan or a SQL string. The scheduler then creates an execution graph, which
contains a physical plan broken down into
+stages (pipelines) that can be scheduled independently. This process is
explained in detail in the Distributed
+Query Scheduling section of this guide.</p>
+<p>It is possible to have multiple schedulers running with shared state in
etcd, so that jobs can continue to run
+even if a scheduler process fails.</p>
+</section>
+<section id="executor">
+<h3>Executor<a class="headerlink" href="#executor" title="Link to this
heading">¶</a></h3>
+<p>The executor processes connect to a scheduler and poll for tasks to
perform. These tasks are physical plans in
+protocol buffer format. These physical plans are typically executed against
multiple partitions of input data. Executors
+can execute multiple partitions of the same plan in parallel.</p>
+</section>
+<section id="clients">
+<h3>Clients<a class="headerlink" href="#clients" title="Link to this
heading">¶</a></h3>
+<p>There are multiple clients available for submitting jobs to a Ballista
cluster:</p>
+<ul class="simple">
+<li><p>The <a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista-cli";>Ballista
CLI</a> provides a SQL command-line
+interface.</p></li>
+<li><p>The Python bindings (<a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/python";>PyBallista</a>)
provide a session
+context with support for SQL and DataFrame operations.</p></li>
+<li><p>The <a class="reference external"
href="https://crates.io/crates/ballista";>ballista crate</a> provides a native
Rust session context with support for
+SQL and DataFrame operations.</p></li>
+<li><p>The <a class="reference external"
href="https://arrow.apache.org/docs/java/flight_sql_jdbc_driver.html";>Flight
SQL JDBC driver</a> can be used from
+popular SQL tools to execute SQL queries against a cluster.</p></li>
+</ul>
+</section>
+</section>
+<section id="distributed-query-scheduling">
+<h2>Distributed Query Scheduling<a class="headerlink"
href="#distributed-query-scheduling" title="Link to this heading">¶</a></h2>
+<p>Distributed query plans are fundamentally different to in-process query
plans because we can’t just build a
+tree of operators and start executing them. The query now requires
co-ordination across executors which means that
+we now need a scheduler.</p>
+<p>At a high level, the concept of a distributed query scheduler is not
complex. The scheduler needs to examine the
+whole query and break it down into stages that can be executed in isolation
(usually in parallel across the executors)
+and then schedule these stages for execution based on the available resources
in the cluster. Once each query
+stage completes then any subsequent dependent query stages can be scheduled.
This process repeats until all query
+stages have been executed.</p>
+<section id="producing-a-distributed-query-plan">
+<h3>Producing a Distributed Query Plan<a class="headerlink"
href="#producing-a-distributed-query-plan" title="Link to this
heading">¶</a></h3>
+<p>Some operators can run in parallel on input partitions and some operators
require data to be repartitioned. These
+changes in partitioning are key to planning a distributed query. Changes in
partitioning within a plan are sometimes
+called pipeline breakers and these changes in partitioning define the
boundaries between query stages.</p>
+<p>We will now use the following SQL query to see how this process works.</p>
+<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="k">SELECT</span><span
class="w"> </span><span class="n">customer</span><span class="p">.</span><span
class="n">id</span><span class="p">,</span><span class="w"> </span><span
class="k">sum</span><span class="p">(</span><span class="k">order</span><span
class="p">.</span><span class="n">amount</span><span class="p">)</span><span
class="w"> </span><span class="k">as</span><span class="w"> </span><sp [...]
+<span class="k">FROM</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span
class="k">JOIN</span><span class="w"> </span><span class="k">order</span><span
class="w"> </span><span class="k">ON</span><span class="w"> </span><span
class="n">customer</span><span class="p">.</span><span class="n">id</span><span
class="w"> </span><span class="o">=</span><span class="w"> </span><span
class="k">order</span><span class="p">.</span><span class="n">customer_id</span>
+<span class="k">GROUP</span><span class="w"> </span><span
class="k">BY</span><span class="w"> </span><span class="n">customer</span><span
class="p">.</span><span class="n">id</span>
+</pre></div>
+</div>
+<p>The physical (non-distributed) plan for this query would look something
like this:</p>
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">Projection</span><span
class="p">:</span> <span class="c1">#customer.id, #total_amount</span>
+ <span class="n">HashAggregate</span><span class="p">:</span> <span
class="n">groupBy</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span><span
class="p">],</span> <span class="n">aggr</span><span class="o">=</span><span
class="p">[</span><span class="n">MAX</span><span class="p">(</span><span
class="n">max_fare</span><span class="p">)</span> <span class="n">AS</span>
<span class="n">total_amount</span><sp [...]
+ <span class="n">Join</span><span class="p">:</span> <span
class="n">condition</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span>
<span class="o">=</span> <span class="n">order</span><span
class="o">.</span><span class="n">customer_id</span><span class="p">]</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">customer</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">order</span>
+</pre></div>
+</div>
+<p>Assuming that the customer and order tables are not already partitioned on
customer id, we will need to schedule
+execution of the first two query stages to repartition this data. These two
query stages can run in parallel.</p>
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">Query</span> <span
class="n">Stage</span> <span class="c1">#1: repartition=[customer.id]</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">customer</span>
+<span class="n">Query</span> <span class="n">Stage</span> <span class="c1">#2:
repartition=[order.customer_id]</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">order</span>
+</pre></div>
+</div>
+<p>Next, we can schedule the join, which will run in parallel for each
partition of the two inputs. The next operator
+after the join is the aggregate, which is split into two parts; the aggregate
that runs in parallel and then
+the final aggregate that requires a single input partition. We can perform the
parallel part of this aggregate
+in the same query stage as the join because this first aggregate does not care
how the data is partitioned. This
+gives us our third query stage, which can now be scheduled for execution. The
output of this query stage
+remains partitioned by customer id.</p>
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">Query</span> <span
class="n">Stage</span> <span class="c1">#3: repartition=[]</span>
+ <span class="n">HashAggregate</span><span class="p">:</span> <span
class="n">groupBy</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span><span
class="p">],</span> <span class="n">aggr</span><span class="o">=</span><span
class="p">[</span><span class="n">MAX</span><span class="p">(</span><span
class="n">max_fare</span><span class="p">)</span> <span class="n">AS</span>
<span class="n">total_amount</span><sp [...]
+ <span class="n">Join</span><span class="p">:</span> <span
class="n">condition</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span>
<span class="o">=</span> <span class="n">order</span><span
class="o">.</span><span class="n">customer_id</span><span class="p">]</span>
+ <span class="n">Query</span> <span class="n">Stage</span> <span
class="c1">#1</span>
+ <span class="n">Query</span> <span class="n">Stage</span> <span
class="c1">#2</span>
+</pre></div>
+</div>
+<p>The final query stage performs the aggregate of the aggregates, reading
from all of the partitions from the previous
+stage.</p>
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">Query</span> <span
class="n">Stage</span> <span class="c1">#4:</span>
+ <span class="n">Projection</span><span class="p">:</span> <span
class="c1">#customer.id, #total_amount</span>
+ <span class="n">HashAggregate</span><span class="p">:</span> <span
class="n">groupBy</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span><span
class="p">],</span> <span class="n">aggr</span><span class="o">=</span><span
class="p">[</span><span class="n">MAX</span><span class="p">(</span><span
class="n">max_fare</span><span class="p">)</span> <span class="n">AS</span>
<span class="n">total_amount</span>< [...]
+ <span class="n">QueryStage</span> <span class="c1">#3</span>
+</pre></div>
+</div>
+<p>To recap, here is the full distributed query plan showing the query stages
that are introduced when data needs to be
+repartitioned or exchanged between pipelined operations.</p>
+<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">Query</span> <span
class="n">Stage</span> <span class="c1">#4:</span>
+ <span class="n">Projection</span><span class="p">:</span> <span
class="c1">#customer.id, #total_amount</span>
+ <span class="n">HashAggregate</span><span class="p">:</span> <span
class="n">groupBy</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span><span
class="p">],</span> <span class="n">aggr</span><span class="o">=</span><span
class="p">[</span><span class="n">MAX</span><span class="p">(</span><span
class="n">max_fare</span><span class="p">)</span> <span class="n">AS</span>
<span class="n">total_amount</span>< [...]
+ <span class="n">Query</span> <span class="n">Stage</span> <span
class="c1">#3: repartition=[]</span>
+ <span class="n">HashAggregate</span><span class="p">:</span> <span
class="n">groupBy</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span><span
class="p">],</span> <span class="n">aggr</span><span class="o">=</span><span
class="p">[</span><span class="n">MAX</span><span class="p">(</span><span
class="n">max_fare</span><span class="p">)</span> <span class="n">AS</span>
<span class="n">total_amount</sp [...]
+ <span class="n">Join</span><span class="p">:</span> <span
class="n">condition</span><span class="o">=</span><span class="p">[</span><span
class="n">customer</span><span class="o">.</span><span class="n">id</span>
<span class="o">=</span> <span class="n">order</span><span
class="o">.</span><span class="n">customer_id</span><span class="p">]</span>
+ <span class="n">Query</span> <span class="n">Stage</span> <span
class="c1">#1: repartition=[customer.id]</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">customer</span>
+ <span class="n">Query</span> <span class="n">Stage</span> <span
class="c1">#2: repartition=[order.customer_id]</span>
+ <span class="n">Scan</span><span class="p">:</span> <span
class="n">order</span>
+</pre></div>
+</div>
+</section>
+<section id="shuffle">
+<h3>Shuffle<a class="headerlink" href="#shuffle" title="Link to this
heading">¶</a></h3>
+<p>Each stage of the execution graph has the same partitioning scheme for all
of the operators in the plan. However,
+the output of each stage typically needs to be repartitioned before it can be
used as the input to the next stage. An
+example of this is when a query contains multiple joins. Data needs to be
partitioned by the join keys before the join
+can be performed.</p>
+<p>Each executor will re-partition the output of the stage it is running so
that it can be consumed by the next
+stage. This mechanism is known as an Exchange or a Shuffle. The logic for this
can be found in the <a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/core/src/execution_plans/shuffle_writer.rs";>ShuffleWriterExec</a>
+and <a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/core/src/execution_plans/shuffle_reader.rs";>ShuffleReaderExec</a>
operators.</p>
+</section>
+</section>
+</section>
+
+
+ </div>
+
+
+ <!-- Previous / next buttons -->
+<div class='prev-next-area'>
+ <a class='left-prev' id="prev-link" href="../user-guide/faq.html"
title="previous page">
+ <i class="fas fa-angle-left"></i>
+ <div class="prev-next-info">
+ <p class="prev-next-subtitle">previous</p>
+ <p class="prev-next-title">Frequently Asked Questions</p>
+ </div>
+ </a>
+ <a class='right-next' id="next-link" href="code-organization.html"
title="next page">
+ <div class="prev-next-info">
+ <p class="prev-next-subtitle">next</p>
+ <p class="prev-next-title">Ballista Code Organization</p>
+ </div>
+ <i class="fas fa-angle-right"></i>
+ </a>
+</div>
+
+ </main>
+
+
+ </div>
+ </div>
+
+ <script
src="../_static/scripts/pydata-sphinx-theme.js?digest=1999514e3f237ded88cf"></script>
+
+<!-- Based on pydata_sphinx_theme/footer.html -->
+<footer class="footer mt-5 mt-md-0">
+ <div class="container">
+
+ <div class="footer-item">
+ <p class="copyright">
+ © Copyright 2019-2024, Apache Software Foundation.<br>
+</p>
+ </div>
+
+ <div class="footer-item">
+ <p class="sphinx-version">
+Created using <a href="http://sphinx-doc.org/";>Sphinx</a> 7.2.6.<br>
+</p>
+ </div>
+
+ <div class="footer-item">
+ <p>Apache Arrow Ballista, Arrow Ballista, Apache, the Apache feather
logo, and the Apache Arrow Ballista project logo</p>
+ <p>are either registered trademarks or trademarks of The Apache Software
Foundation in the United States and other countries.</p>
+ </div>
+ </div>
+</footer>
+
+
+ </body>
+</html>
\ No newline at end of file
diff --git a/ballista/community/communication.html
b/ballista/contributors-guide/code-organization.html
similarity index 63%
copy from ballista/community/communication.html
copy to ballista/contributors-guide/code-organization.html
index d5a911f0cd7..b4a751a7088 100644
--- a/ballista/community/communication.html
+++ b/ballista/contributors-guide/code-organization.html
@@ -5,7 +5,7 @@
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0"
/><meta name="viewport" content="width=device-width, initial-scale=1" />
- <title>Communication — Apache Arrow Ballista documentation</title>
+ <title>Ballista Code Organization — Apache Arrow Ballista
documentation</title>
<link href="../_static/styles/theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
<link
href="../_static/styles/pydata-sphinx-theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
@@ -34,7 +34,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Ballista Development" href="development.html" />
- <link rel="prev" title="Frequently Asked Questions"
href="../user-guide/faq.html" />
+ <link rel="prev" title="Ballista Architecture" href="architecture.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../user-guide/faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
-<ul class="nav bd-sidenav">
+<ul class="current nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1 current active">
+ <a class="current reference internal" href="#">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -195,15 +215,10 @@
Community
</span>
</p>
-<ul class="current nav bd-sidenav">
- <li class="toctree-l1 current active">
- <a class="current reference internal" href="#">
- Communication
- </a>
- </li>
+<ul class="nav bd-sidenav">
<li class="toctree-l1">
- <a class="reference internal" href="development.html">
- Ballista Development
+ <a class="reference internal" href="../community/communication.html">
+ Communication
</a>
</li>
<li class="toctree-l1">
@@ -241,30 +256,28 @@
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#questions">
- Questions?
+ <a class="reference internal nav-link" href="#ballista-core-crate">
+ ballista-core crate
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#ballista-scheduler-crate">
+ ballista-scheduler crate
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#ballista-executor-crate">
+ ballista-executor crate
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#ballista-crate">
+ ballista crate
</a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#mailing-list">
- Mailing list
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#slack-and-discord">
- Slack and Discord
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#sync-up-video-calls">
- Sync up video calls
- </a>
- </li>
- </ul>
</li>
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#contributing">
- Contributing
+ <a class="reference internal nav-link" href="#pyballista">
+ PyBallista
</a>
</li>
</ul>
@@ -276,7 +289,7 @@
<div class="tocsection editthispage">
- <a
href="https://github.com/apache/arrow-ballista/edit/main/docs/source/community/communication.md";>
+ <a
href="https://github.com/apache/arrow-ballista/edit/main/docs/source/contributors-guide/code-organization.md";>
<i class="fas fa-pencil-alt"></i> Edit this page
</a>
</div>
@@ -313,64 +326,51 @@
specific language governing permissions and limitations
under the License.
-->
-<section id="communication">
-<h1>Communication<a class="headerlink" href="#communication" title="Link to
this heading">¶</a></h1>
-<p>We welcome participation from everyone and encourage you to join us, ask
-questions, and get involved.</p>
-<p>All participation in the Apache Arrow Ballista project is governed by the
-Apache Software Foundation’s <a class="reference external"
href="https://www.apache.org/foundation/policies/conduct.html";>code of
-conduct</a>.</p>
-<section id="questions">
-<h2>Questions?<a class="headerlink" href="#questions" title="Link to this
heading">¶</a></h2>
-<section id="mailing-list">
-<h3>Mailing list<a class="headerlink" href="#mailing-list" title="Link to this
heading">¶</a></h3>
-<p>We use arrow.apache.org’s <code class="docutils literal notranslate"><span
class="pre">dev@</span></code> mailing list for project management, release
-coorindation and design discussions
-(<a class="reference external"
href="mailto:dev-subscribe%40arrow.apache.org";>subscribe</a>,
-<a class="reference external"
href="mailto:dev-unsubscribe%40arrow.apache.org";>unsubscribe</a>,
-<a class="reference external"
href="https://lists.apache.org/list.html?dev@arrow.apache.org";>archives</a>).</p>
-<p>When emailing the dev list, please make sure to prefix the subject line
with a
-<code class="docutils literal notranslate"><span
class="pre">[Ballista]</span></code> tag, e.g. <code class="docutils literal
notranslate"><span class="pre">"[Ballista]</span> <span
class="pre">New</span> <span class="pre">API</span> <span
class="pre">for</span> <span class="pre">remote</span> <span
class="pre">data</span> <span class="pre">sources"</span></code>, so
-that the appropriate people in the Apache Arrow community notice the
message.</p>
+<section id="ballista-code-organization">
+<h1>Ballista Code Organization<a class="headerlink"
href="#ballista-code-organization" title="Link to this heading">¶</a></h1>
+<p>This section provides links to the source code for major areas of
functionality.</p>
+<section id="ballista-core-crate">
+<h2>ballista-core crate<a class="headerlink" href="#ballista-core-crate"
title="Link to this heading">¶</a></h2>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/core";>Crate
Source</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/core/proto/ballista.proto";>Protocol
Buffer Definition</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/core/src/execution_plans";>Execution
Plans</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/core/src/client.rs";>Ballista
Client</a></p></li>
+</ul>
</section>
-<section id="slack-and-discord">
-<h3>Slack and Discord<a class="headerlink" href="#slack-and-discord"
title="Link to this heading">¶</a></h3>
-<p>We use the official <a class="reference external"
href="https://s.apache.org/slack-invite";>ASF</a> Slack workspace
-for informal discussions and coordination. This is a great place to meet other
-contributors and get guidance on where to contribute. Join us in the
-<code class="docutils literal notranslate"><span
class="pre">#arrow-rust</span></code> channel.</p>
-<p>We also have a backup Arrow Rust Discord
-server (<a class="reference external"
href="https://discord.gg/Qw5gKqHxUM";>invite link</a>) in case you are not able
-to join the Slack workspace. If you need an invite to the Slack workspace, you
-can also ask for one in our Discord server.</p>
+<section id="ballista-scheduler-crate">
+<h2>ballista-scheduler crate<a class="headerlink"
href="#ballista-scheduler-crate" title="Link to this heading">¶</a></h2>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/scheduler";>Crate
Source</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/scheduler/src/planner.rs";>Distributed
Query Planner</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/scheduler/src/scheduler_server/grpc.rs";>gRPC
Service</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/scheduler/src/flight_sql.rs";>Flight
SQL Service</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/scheduler/src/api";>REST
API</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/scheduler/ui";>Web
UI</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/scheduler/src/metrics/prometheus.rs";>Prometheus
Integration</a></p></li>
+</ul>
</section>
-<section id="sync-up-video-calls">
-<h3>Sync up video calls<a class="headerlink" href="#sync-up-video-calls"
title="Link to this heading">¶</a></h3>
-<p>We have biweekly sync calls every other Thursdays at both 04:00 UTC
-and 16:00 UTC (starting September 30, 2021) depending on if there are
-items on the agenda to discuss and someone being willing to host.</p>
-<p>Please see the <a class="reference external"
href="https://docs.google.com/document/d/1atCVnoff5SR4eM4Lwf2M1BBJTY6g3_HUNR6qswYJW_U/edit";>agenda</a>
-for the video call link, add topics and to see what others plan to discuss.</p>
-<p>The goals of these calls are:</p>
-<ol class="arabic simple">
-<li><p>Help “put a face to the name” of some of other contributors we are
working with</p></li>
-<li><p>Discuss / synchronize on the goals and major initiatives from different
stakeholders to identify areas where more alignment is needed</p></li>
-</ol>
-<p>No decisions are made on the call and anything of substance will be
discussed on this mailing list or in github issues / google docs.</p>
-<p>We will send a summary of all sync ups to the dev@arrow.apache.org
mailing list.</p>
+<section id="ballista-executor-crate">
+<h2>ballista-executor crate<a class="headerlink"
href="#ballista-executor-crate" title="Link to this heading">¶</a></h2>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/executor";>Crate
Source</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/executor/src/flight_service.rs";>Flight
Service</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/executor/src/executor_server.rs";>Executor
Server</a></p></li>
+</ul>
</section>
+<section id="ballista-crate">
+<h2>ballista crate<a class="headerlink" href="#ballista-crate" title="Link to
this heading">¶</a></h2>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/ballista/client";>Crate
Source</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/ballista/client/src/context.rs";>Context</a></p></li>
+</ul>
</section>
-<section id="contributing">
-<h2>Contributing<a class="headerlink" href="#contributing" title="Link to this
heading">¶</a></h2>
-<p>Our source code is hosted on
-<a class="reference external"
href="https://github.com/apache/arrow-datafusion";>GitHub</a>. More information
on contributing is in
-the <a class="reference external"
href="https://github.com/apache/arrow-datafusion/blob/master/CONTRIBUTING.md";>Contribution
Guide</a>
-, and we have curated a [good-first-issue]
-(https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
-list to help you get started. You can find datafusion’s major designs in
docs/source/specification.</p>
-<p>We use GitHub issues for maintaining a queue of development work and as the
-public record. We often use Google docs, Github issues and pull requests for
-quick and small design discussions. For major design change proposals, we
encourage you to write a rfc.</p>
+<section id="pyballista">
+<h2>PyBallista<a class="headerlink" href="#pyballista" title="Link to this
heading">¶</a></h2>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/tree/main/python";>Source</a></p></li>
+<li><p><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/python/src/context.rs";>Context</a></p></li>
+</ul>
</section>
</section>
@@ -380,11 +380,11 @@ quick and small design discussions. For major design
change proposals, we encour
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="../user-guide/faq.html"
title="previous page">
+ <a class='left-prev' id="prev-link" href="architecture.html"
title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Frequently Asked Questions</p>
+ <p class="prev-next-title">Ballista Architecture</p>
</div>
</a>
<a class='right-next' id="next-link" href="development.html" title="next
page">
diff --git a/ballista/community/communication.html
b/ballista/contributors-guide/development.html
similarity index 63%
copy from ballista/community/communication.html
copy to ballista/contributors-guide/development.html
index d5a911f0cd7..198a48112c3 100644
--- a/ballista/community/communication.html
+++ b/ballista/contributors-guide/development.html
@@ -5,7 +5,7 @@
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0"
/><meta name="viewport" content="width=device-width, initial-scale=1" />
- <title>Communication — Apache Arrow Ballista documentation</title>
+ <title>Ballista Development — Apache Arrow Ballista
documentation</title>
<link href="../_static/styles/theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
<link
href="../_static/styles/pydata-sphinx-theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
@@ -33,8 +33,8 @@
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Ballista Development" href="development.html" />
- <link rel="prev" title="Frequently Asked Questions"
href="../user-guide/faq.html" />
+ <link rel="next" title="Communication"
href="../community/communication.html" />
+ <link rel="prev" title="Ballista Code Organization"
href="code-organization.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../user-guide/faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
-<ul class="nav bd-sidenav">
+<ul class="current nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1 current active">
+ <a class="current reference internal" href="#">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -195,15 +215,10 @@
Community
</span>
</p>
-<ul class="current nav bd-sidenav">
- <li class="toctree-l1 current active">
- <a class="current reference internal" href="#">
- Communication
- </a>
- </li>
+<ul class="nav bd-sidenav">
<li class="toctree-l1">
- <a class="reference internal" href="development.html">
- Ballista Development
+ <a class="reference internal" href="../community/communication.html">
+ Communication
</a>
</li>
<li class="toctree-l1">
@@ -241,30 +256,23 @@
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#questions">
- Questions?
+ <a class="reference internal nav-link" href="#development-environment">
+ Development Environment
</a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#mailing-list">
- Mailing list
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#slack-and-discord">
- Slack and Discord
- </a>
- </li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#sync-up-video-calls">
- Sync up video calls
- </a>
- </li>
- </ul>
</li>
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#contributing">
- Contributing
+ <a class="reference internal nav-link" href="#build-the-project">
+ Build the project
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#testing-the-project">
+ Testing the project
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#running-the-examples">
+ Running the examples
</a>
</li>
</ul>
@@ -276,7 +284,7 @@
<div class="tocsection editthispage">
- <a
href="https://github.com/apache/arrow-ballista/edit/main/docs/source/community/communication.md";>
+ <a
href="https://github.com/apache/arrow-ballista/edit/main/docs/source/contributors-guide/development.md";>
<i class="fas fa-pencil-alt"></i> Edit this page
</a>
</div>
@@ -313,64 +321,49 @@
specific language governing permissions and limitations
under the License.
-->
-<section id="communication">
-<h1>Communication<a class="headerlink" href="#communication" title="Link to
this heading">¶</a></h1>
+<section id="ballista-development">
+<h1>Ballista Development<a class="headerlink" href="#ballista-development"
title="Link to this heading">¶</a></h1>
<p>We welcome participation from everyone and encourage you to join us, ask
questions, and get involved.</p>
-<p>All participation in the Apache Arrow Ballista project is governed by the
+<p>All participation in the Apache Arrow DataFusion project is governed by the
Apache Software Foundation’s <a class="reference external"
href="https://www.apache.org/foundation/policies/conduct.html";>code of
conduct</a>.</p>
-<section id="questions">
-<h2>Questions?<a class="headerlink" href="#questions" title="Link to this
heading">¶</a></h2>
-<section id="mailing-list">
-<h3>Mailing list<a class="headerlink" href="#mailing-list" title="Link to this
heading">¶</a></h3>
-<p>We use arrow.apache.org’s <code class="docutils literal notranslate"><span
class="pre">dev@</span></code> mailing list for project management, release
-coorindation and design discussions
-(<a class="reference external"
href="mailto:dev-subscribe%40arrow.apache.org";>subscribe</a>,
-<a class="reference external"
href="mailto:dev-unsubscribe%40arrow.apache.org";>unsubscribe</a>,
-<a class="reference external"
href="https://lists.apache.org/list.html?dev@arrow.apache.org";>archives</a>).</p>
-<p>When emailing the dev list, please make sure to prefix the subject line
with a
-<code class="docutils literal notranslate"><span
class="pre">[Ballista]</span></code> tag, e.g. <code class="docutils literal
notranslate"><span class="pre">"[Ballista]</span> <span
class="pre">New</span> <span class="pre">API</span> <span
class="pre">for</span> <span class="pre">remote</span> <span
class="pre">data</span> <span class="pre">sources"</span></code>, so
-that the appropriate people in the Apache Arrow community notice the
message.</p>
-</section>
-<section id="slack-and-discord">
-<h3>Slack and Discord<a class="headerlink" href="#slack-and-discord"
title="Link to this heading">¶</a></h3>
-<p>We use the official <a class="reference external"
href="https://s.apache.org/slack-invite";>ASF</a> Slack workspace
-for informal discussions and coordination. This is a great place to meet other
-contributors and get guidance on where to contribute. Join us in the
-<code class="docutils literal notranslate"><span
class="pre">#arrow-rust</span></code> channel.</p>
-<p>We also have a backup Arrow Rust Discord
-server (<a class="reference external"
href="https://discord.gg/Qw5gKqHxUM";>invite link</a>) in case you are not able
-to join the Slack workspace. If you need an invite to the Slack workspace, you
-can also ask for one in our Discord server.</p>
+<section id="development-environment">
+<h2>Development Environment<a class="headerlink"
href="#development-environment" title="Link to this heading">¶</a></h2>
+<p>The easiest way to get started if you are using VSCode or IntelliJ IDEA is
to open the provided <a class="reference external"
href="https://containers.dev/overview";>Dev Container</a>
+which will install all the required dependencies including Rust, Docker,
Node.js and Yarn. A Dev Container is a
+development environment that runs in a Docker container. It is configured with
all the required dependencies to
+build and test the project. It also includes VS Code and the Rust and Node.js
extensions. Other supporting tools
+that use Dev Containers can be seen <a class="reference external"
href="https://containers.dev/supporting";>here</a></p>
+<p>To use the Dev Container, open the project in VS Code and then click the
“Reopen in Container” button in the
+bottom right corner of the IDE.</p>
+<p>If you are not using the Dev Container or VScode, you will need to install
these dependencies yourself.</p>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://www.rust-lang.org/tools/install";>Rust</a></p></li>
+<li><p><a class="reference external"
href="https://nodejs.org/en/download/";>Node.js</a> is required to build the
project.</p></li>
+<li><p><a class="reference external"
href="https://classic.yarnpkg.com/en/docs/install";>Yarn</a> is required to
build the UI.</p></li>
+<li><p><a class="reference external"
href="https://docs.docker.com/get-docker/";>Docker</a> is required to run the
integration tests.</p></li>
+</ul>
</section>
-<section id="sync-up-video-calls">
-<h3>Sync up video calls<a class="headerlink" href="#sync-up-video-calls"
title="Link to this heading">¶</a></h3>
-<p>We have biweekly sync calls every other Thursdays at both 04:00 UTC
-and 16:00 UTC (starting September 30, 2021) depending on if there are
-items on the agenda to discuss and someone being willing to host.</p>
-<p>Please see the <a class="reference external"
href="https://docs.google.com/document/d/1atCVnoff5SR4eM4Lwf2M1BBJTY6g3_HUNR6qswYJW_U/edit";>agenda</a>
-for the video call link, add topics and to see what others plan to discuss.</p>
-<p>The goals of these calls are:</p>
-<ol class="arabic simple">
-<li><p>Help “put a face to the name” of some of other contributors we are
working with</p></li>
-<li><p>Discuss / synchronize on the goals and major initiatives from different
stakeholders to identify areas where more alignment is needed</p></li>
-</ol>
-<p>No decisions are made on the call and anything of substance will be
discussed on this mailing list or in github issues / google docs.</p>
-<p>We will send a summary of all sync ups to the dev@arrow.apache.org
mailing list.</p>
+<section id="build-the-project">
+<h2>Build the project<a class="headerlink" href="#build-the-project"
title="Link to this heading">¶</a></h2>
+<p>From the root of the project, build release binaries.</p>
+<div class="highlight-shell notranslate"><div
class="highlight"><pre><span></span>cargo<span class="w"> </span>build<span
class="w"> </span>--release
+</pre></div>
+</div>
</section>
+<section id="testing-the-project">
+<h2>Testing the project<a class="headerlink" href="#testing-the-project"
title="Link to this heading">¶</a></h2>
+<div class="highlight-shell notranslate"><div
class="highlight"><pre><span></span>cargo<span class="w"> </span><span
class="nb">test</span>
+</pre></div>
+</div>
</section>
-<section id="contributing">
-<h2>Contributing<a class="headerlink" href="#contributing" title="Link to this
heading">¶</a></h2>
-<p>Our source code is hosted on
-<a class="reference external"
href="https://github.com/apache/arrow-datafusion";>GitHub</a>. More information
on contributing is in
-the <a class="reference external"
href="https://github.com/apache/arrow-datafusion/blob/master/CONTRIBUTING.md";>Contribution
Guide</a>
-, and we have curated a [good-first-issue]
-(https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
-list to help you get started. You can find datafusion’s major designs in
docs/source/specification.</p>
-<p>We use GitHub issues for maintaining a queue of development work and as the
-public record. We often use Google docs, Github issues and pull requests for
-quick and small design discussions. For major design change proposals, we
encourage you to write a rfc.</p>
+<section id="running-the-examples">
+<h2>Running the examples<a class="headerlink" href="#running-the-examples"
title="Link to this heading">¶</a></h2>
+<div class="highlight-shell notranslate"><div
class="highlight"><pre><span></span><span class="nb">cd</span><span class="w">
</span>examples
+cargo<span class="w"> </span>run<span class="w"> </span>--example<span
class="w"> </span>standalone_sql<span class="w"> </span>--features<span
class="o">=</span>ballista/standalone
+</pre></div>
+</div>
</section>
</section>
@@ -380,17 +373,17 @@ quick and small design discussions. For major design
change proposals, we encour
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="../user-guide/faq.html"
title="previous page">
+ <a class='left-prev' id="prev-link" href="code-organization.html"
title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Frequently Asked Questions</p>
+ <p class="prev-next-title">Ballista Code Organization</p>
</div>
</a>
- <a class='right-next' id="next-link" href="development.html" title="next
page">
+ <a class='right-next' id="next-link"
href="../community/communication.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Ballista Development</p>
+ <p class="prev-next-title">Communication</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
diff --git a/ballista/genindex.html b/ballista/genindex.html
index 1c25154a33b..bd27024f160 100644
--- a/ballista/genindex.html
+++ b/ballista/genindex.html
@@ -169,6 +169,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="user-guide/faq.html">
Frequently Asked Questions
@@ -177,13 +182,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -198,11 +218,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/index.html b/ballista/index.html
index 75d582078a5..28fbb60626a 100644
--- a/ballista/index.html
+++ b/ballista/index.html
@@ -171,6 +171,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="user-guide/faq.html">
Frequently Asked Questions
@@ -179,13 +184,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -200,11 +220,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
@@ -305,20 +320,23 @@
<ul>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/configs.html">Configuration</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/tuning-guide.html">Tuning Guide</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="user-guide/metrics.html">Ballista Scheduler Metrics</a></li>
<li class="toctree-l1"><a class="reference internal"
href="user-guide/faq.html">Frequently Asked Questions</a></li>
</ul>
</div>
-<div class="toctree-wrapper compound" id="toc-source">
-<p aria-level="2" class="caption" role="heading"><span
class="caption-text">Source Code</span></p>
+<div class="toctree-wrapper compound" id="toc-contributors">
+<p aria-level="2" class="caption" role="heading"><span
class="caption-text">Contributors Guide</span></p>
<ul>
-<li class="toctree-l1"><a class="reference external"
href="https://github.com/apache/arrow-ballista/";>Ballista</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="contributors-guide/architecture.html">Ballista Architecture</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="contributors-guide/code-organization.html">Ballista Code
Organization</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="contributors-guide/development.html">Ballista Development</a></li>
+<li class="toctree-l1"><a class="reference external"
href="https://github.com/apache/arrow-ballista/";>Source code</a></li>
</ul>
</div>
<div class="toctree-wrapper compound" id="toc-community">
<p aria-level="2" class="caption" role="heading"><span
class="caption-text">Community</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal"
href="community/communication.html">Communication</a></li>
-<li class="toctree-l1"><a class="reference internal"
href="community/development.html">Ballista Development</a></li>
<li class="toctree-l1"><a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>Issue tracker</a></li>
<li class="toctree-l1"><a class="reference external"
href="https://github.com/apache/arrow-ballista/blob/main/CODE_OF_CONDUCT.md";>Code
of conduct</a></li>
</ul>
diff --git a/ballista/objects.inv b/ballista/objects.inv
index d958a5601df..70882d05e59 100644
Binary files a/ballista/objects.inv and b/ballista/objects.inv differ
diff --git a/ballista/search.html b/ballista/search.html
index 7cf5b399e5d..124e0a9210a 100644
--- a/ballista/search.html
+++ b/ballista/search.html
@@ -175,6 +175,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="user-guide/metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="user-guide/faq.html">
Frequently Asked Questions
@@ -183,13 +188,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -204,11 +224,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/searchindex.js b/ballista/searchindex.js
index 3a870922446..3c622957822 100644
--- a/ballista/searchindex.js
+++ b/ballista/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["community/communication",
"community/development", "index", "user-guide/cli", "user-guide/configs",
"user-guide/deployment/cargo-install", "user-guide/deployment/docker",
"user-guide/deployment/docker-compose", "user-guide/deployment/index",
"user-guide/deployment/kubernetes", "user-guide/deployment/quick-start",
"user-guide/faq", "user-guide/flightsql", "user-guide/introduction",
"user-guide/metrics", "user-guide/python", "user-guide/rust", "user-guide/sch
[...]
\ No newline at end of file
+Search.setIndex({"docnames": ["community/communication",
"contributors-guide/architecture", "contributors-guide/code-organization",
"contributors-guide/development", "index", "user-guide/cli",
"user-guide/configs", "user-guide/deployment/cargo-install",
"user-guide/deployment/docker", "user-guide/deployment/docker-compose",
"user-guide/deployment/index", "user-guide/deployment/kubernetes",
"user-guide/deployment/quick-start", "user-guide/faq", "user-guide/flightsql",
"user-guide/introduc [...]
\ No newline at end of file
diff --git a/ballista/user-guide/cli.html b/ballista/user-guide/cli.html
index 6e81930fa50..e40e9772cfb 100644
--- a/ballista/user-guide/cli.html
+++ b/ballista/user-guide/cli.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/configs.html b/ballista/user-guide/configs.html
index 335a0d02a58..ae980650db0 100644
--- a/ballista/user-guide/configs.html
+++ b/ballista/user-guide/configs.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/cargo-install.html
b/ballista/user-guide/deployment/cargo-install.html
index 9aea574a155..7097a21a849 100644
--- a/ballista/user-guide/deployment/cargo-install.html
+++ b/ballista/user-guide/deployment/cargo-install.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/docker-compose.html
b/ballista/user-guide/deployment/docker-compose.html
index 4eaeed4a44a..2224c251197 100644
--- a/ballista/user-guide/deployment/docker-compose.html
+++ b/ballista/user-guide/deployment/docker-compose.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/docker.html
b/ballista/user-guide/deployment/docker.html
index 29994b101c9..6f1e2cb0c53 100644
--- a/ballista/user-guide/deployment/docker.html
+++ b/ballista/user-guide/deployment/docker.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/index.html
b/ballista/user-guide/deployment/index.html
index 303b44e1110..b48ac8163ce 100644
--- a/ballista/user-guide/deployment/index.html
+++ b/ballista/user-guide/deployment/index.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/kubernetes.html
b/ballista/user-guide/deployment/kubernetes.html
index f8019d1eeb6..158952a7aa3 100644
--- a/ballista/user-guide/deployment/kubernetes.html
+++ b/ballista/user-guide/deployment/kubernetes.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/deployment/quick-start.html
b/ballista/user-guide/deployment/quick-start.html
index 80005f1d696..6738c7f8e9b 100644
--- a/ballista/user-guide/deployment/quick-start.html
+++ b/ballista/user-guide/deployment/quick-start.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/faq.html b/ballista/user-guide/faq.html
index 3b99d6a71d1..efdf9fa5842 100644
--- a/ballista/user-guide/faq.html
+++ b/ballista/user-guide/faq.html
@@ -33,8 +33,8 @@
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Communication"
href="../community/communication.html" />
- <link rel="prev" title="Tuning Guide" href="tuning-guide.html" />
+ <link rel="next" title="Ballista Architecture"
href="../contributors-guide/architecture.html" />
+ <link rel="prev" title="Ballista Scheduler Metrics" href="metrics.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1 current active">
<a class="current reference internal" href="#">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
@@ -310,17 +325,17 @@ for parallel query execution.</p>
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="tuning-guide.html"
title="previous page">
+ <a class='left-prev' id="prev-link" href="metrics.html" title="previous
page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Tuning Guide</p>
+ <p class="prev-next-title">Ballista Scheduler Metrics</p>
</div>
</a>
- <a class='right-next' id="next-link"
href="../community/communication.html" title="next page">
+ <a class='right-next' id="next-link"
href="../contributors-guide/architecture.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Communication</p>
+ <p class="prev-next-title">Ballista Architecture</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
diff --git a/ballista/user-guide/flightsql.html
b/ballista/user-guide/flightsql.html
index 22de7f9575f..f5fd8a0519b 100644
--- a/ballista/user-guide/flightsql.html
+++ b/ballista/user-guide/flightsql.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/introduction.html
b/ballista/user-guide/introduction.html
index c90bcbe4f66..f9caac51307 100644
--- a/ballista/user-guide/introduction.html
+++ b/ballista/user-guide/introduction.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/metrics.html b/ballista/user-guide/metrics.html
index 82478768a5c..1196a37c1a0 100644
--- a/ballista/user-guide/metrics.html
+++ b/ballista/user-guide/metrics.html
@@ -33,6 +33,8 @@
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="Frequently Asked Questions" href="faq.html" />
+ <link rel="prev" title="Tuning Guide" href="tuning-guide.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -159,7 +161,7 @@
Reference
</span>
</p>
-<ul class="nav bd-sidenav">
+<ul class="current nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="configs.html">
Configuration
@@ -170,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1 current active">
+ <a class="current reference internal" href="#">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -178,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -199,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
@@ -319,6 +336,20 @@ into an existing metrics system by point your chosen
prometheus exporter at that
<!-- Previous / next buttons -->
<div class='prev-next-area'>
+ <a class='left-prev' id="prev-link" href="tuning-guide.html"
title="previous page">
+ <i class="fas fa-angle-left"></i>
+ <div class="prev-next-info">
+ <p class="prev-next-subtitle">previous</p>
+ <p class="prev-next-title">Tuning Guide</p>
+ </div>
+ </a>
+ <a class='right-next' id="next-link" href="faq.html" title="next page">
+ <div class="prev-next-info">
+ <p class="prev-next-subtitle">next</p>
+ <p class="prev-next-title">Frequently Asked Questions</p>
+ </div>
+ <i class="fas fa-angle-right"></i>
+ </a>
</div>
</main>
diff --git a/ballista/user-guide/python.html b/ballista/user-guide/python.html
index 4899f17f920..c944af00ad9 100644
--- a/ballista/user-guide/python.html
+++ b/ballista/user-guide/python.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/rust.html b/ballista/user-guide/rust.html
index e6ad329e857..f2828b70f77 100644
--- a/ballista/user-guide/rust.html
+++ b/ballista/user-guide/rust.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/scheduler.html
b/ballista/user-guide/scheduler.html
index 8aa720d194b..827ebb4c2f3 100644
--- a/ballista/user-guide/scheduler.html
+++ b/ballista/user-guide/scheduler.html
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
diff --git a/ballista/user-guide/tuning-guide.html
b/ballista/user-guide/tuning-guide.html
index 4f2b2c6c15a..2ff76a2e9fc 100644
--- a/ballista/user-guide/tuning-guide.html
+++ b/ballista/user-guide/tuning-guide.html
@@ -33,7 +33,7 @@
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Frequently Asked Questions" href="faq.html" />
+ <link rel="next" title="Ballista Scheduler Metrics" href="metrics.html" />
<link rel="prev" title="Configuration" href="configs.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -172,6 +172,11 @@
Tuning Guide
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="metrics.html">
+ Ballista Scheduler Metrics
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="faq.html">
Frequently Asked Questions
@@ -180,13 +185,28 @@
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
- Source Code
+ Contributors Guide
</span>
</p>
<ul class="nav bd-sidenav">
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/architecture.html">
+ Ballista Architecture
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal"
href="../contributors-guide/code-organization.html">
+ Ballista Code Organization
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributors-guide/development.html">
+ Ballista Development
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/";>
- Ballista
+ Source code
</a>
</li>
</ul>
@@ -201,11 +221,6 @@
Communication
</a>
</li>
- <li class="toctree-l1">
- <a class="reference internal" href="../community/development.html">
- Ballista Development
- </a>
- </li>
<li class="toctree-l1">
<a class="reference external"
href="https://github.com/apache/arrow-ballista/issues";>
Issue tracker
@@ -380,10 +395,10 @@ processes. The default is <code class="docutils literal
notranslate"><span class
<p class="prev-next-title">Configuration</p>
</div>
</a>
- <a class='right-next' id="next-link" href="faq.html" title="next page">
+ <a class='right-next' id="next-link" href="metrics.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Frequently Asked Questions</p>
+ <p class="prev-next-title">Ballista Scheduler Metrics</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
