This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2c9ceb58 deploy: 690edffcb28b787df9ddc42511f0537c232941e6
2c9ceb58 is described below
commit 2c9ceb58157d8b22031e56cad9cad22ca8dfd06a
Author: aokolnychyi <[email protected]>
AuthorDate: Wed May 31 23:26:00 2023 +0000
deploy: 690edffcb28b787df9ddc42511f0537c232941e6
---
docs/latest/api/index.html | 10 +++++-----
docs/latest/aws/index.html | 10 +++++-----
docs/latest/branching/index.html | 2 +-
docs/latest/configuration/index.html | 2 +-
docs/latest/custom-catalog/index.html | 2 +-
docs/latest/dell/index.html | 2 +-
docs/latest/delta-lake-migration/index.html | 2 +-
docs/latest/evolution/index.html | 2 +-
docs/latest/flink-actions/index.html | 4 ++--
docs/latest/flink-configuration/index.html | 2 +-
docs/latest/flink-connector/index.html | 2 +-
docs/latest/flink-ddl/index.html | 2 +-
docs/latest/flink-queries/index.html | 6 +++---
docs/latest/flink-writes/index.html | 2 +-
docs/latest/flink/index.html | 10 +++++-----
docs/latest/getting-started/index.html | 8 ++++----
docs/latest/hive-migration/index.html | 2 +-
docs/latest/hive/index.html | 4 ++--
docs/latest/index.html | 2 +-
docs/latest/index.xml | 2 +-
docs/latest/java-api-quickstart/index.html | 4 ++--
docs/latest/jdbc/index.html | 4 ++--
docs/latest/maintenance/index.html | 12 ++++++------
docs/latest/nessie/index.html | 8 ++++----
docs/latest/partitioning/index.html | 2 +-
docs/latest/performance/index.html | 2 +-
docs/latest/reliability/index.html | 2 +-
docs/latest/schemas/index.html | 2 +-
docs/latest/spark-configuration/index.html | 2 +-
docs/latest/spark-ddl/index.html | 2 +-
docs/latest/spark-procedures/index.html | 10 +++++-----
docs/latest/spark-queries/index.html | 2 +-
docs/latest/spark-structured-streaming/index.html | 2 +-
docs/latest/spark-writes/index.html | 2 +-
docs/latest/table-migration/index.html | 2 +-
35 files changed, 68 insertions(+), 68 deletions(-)
diff --git a/docs/latest/api/index.html b/docs/latest/api/index.html
index f5069d9c..e7da5371 100644
--- a/docs/latest/api/index.html
+++ b/docs/latest/api/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,7 +15,7 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul
class=sub-menu><li><a href=../java-api-quickstart/>Java
Quickstart</a></li><li><a id=active href=../api/>Java API</a></li><li><a
href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a
class="chevron-toggle collapsed" data-toggle=collapse data-parent=full
href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
</span></span></code></pre></div><p>To configure a scan, call
<code>filter</code> and <code>select</code> on the <code>TableScan</code> to
get a new <code>TableScan</code> with those changes.</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span>TableScan
filteredScan <span style=color:#f92672>=</span> scan<span
style=color:#f92672>.</span><span [...]
</span></span></code></pre></div><p>Calls to configuration methods create a
new <code>TableScan</code> so that each <code>TableScan</code> is immutable and
won’t change unexpectedly if shared across threads.</p><p>When a scan is
configured, <code>planFiles</code>, <code>planTasks</code>, and
<code>schema</code> are used to return files, tasks, and the read
projection.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;
[...]
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span
style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span
style=color:#a6e22e>equal</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"id"</span><span style=color:#f92672>,</span> <span
style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
@@ -28,7 +28,7 @@
</span></span></code></pre></div><p>When a scan is configured, call method
<code>build</code> to execute scan. <code>build</code> return
<code>CloseableIterable<Record></code></p><div class=highlight><pre
tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span
style=display:flex><span>CloseableIterable<span
style=color:#f92672><</span>Record<span style=color:#f92672>></span>
result <span styl [...]
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>where</span><span
style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span
style=color:#a6e22e>lessThan</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"id"</span><span style=color:#f92672>,</span> <span
style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>build</span><span
style=color:#f92672>();</span>
-</span></span></code></pre></div><p>where <code>Record</code> is Iceberg
record for iceberg-data module
<code>org.apache.iceberg.data.Record</code>.</p><h3 id=update-operations>Update
operations</h3><p><code>Table</code> also exposes operations that update the
table. These operations use a builder pattern, <a
href=../../../javadoc/1.2.1/index.html?org/apache/iceberg/PendingUpdate.html><code>PendingUpdate</code></a>,
that commits when <code>PendingUpdate#commit</code> is called.</p><p>For [...]
+</span></span></code></pre></div><p>where <code>Record</code> is Iceberg
record for iceberg-data module
<code>org.apache.iceberg.data.Record</code>.</p><h3 id=update-operations>Update
operations</h3><p><code>Table</code> also exposes operations that update the
table. These operations use a builder pattern, <a
href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/PendingUpdate.html><code>PendingUpdate</code></a>,
that commits when <code>PendingUpdate#commit</code> is called.</p><p>For [...]
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>addColumn</span><span
style=color:#f92672>(</span><span
style=color:#e6db74>"count"</span><span style=color:#f92672>,</span>
Types<span style=color:#f92672>.</span><span
style=color:#a6e22e>LongType</span><span style=color:#f92672>.</span><span
style=color:#a6e22e>get</span><span style=color:#f92672>())</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>commit</span><span
style=color:#f92672>();</span>
</span></span></code></pre></div><p>Available operations to update a table
are:</p><ul><li><code>updateSchema</code> – update the table
schema</li><li><code>updateProperties</code> – update table
properties</li><li><code>updateLocation</code> – update the table’s
base location</li><li><code>newAppend</code> – used to append data
files</li><li><code>newFastAppend</code> – used to append data files,
will not compact metadata</li><li><code>newOverwrite</c [...]
@@ -39,7 +39,7 @@
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>//
commit all the changes to the table
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>t<span style=color:#f92672>.</span><span
style=color:#a6e22e>commitTransaction</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><h2 id=types>Types</h2><p>Iceberg data types
are located in the <a
href=../../../javadoc/1.2.1/index.html?org/apache/iceberg/types/package-summary.html><code>org.apache.iceberg.types</code>
package</a>.</p><h3 id=primitives>Primitives</h3><p>Primitive type instances
are available from static methods in each type class. Types without parameters
use <code>get</code>, and types like <code>decimal</code> use factory
methods:</p><div class=highlight><pre tabin [...]
+</span></span></code></pre></div><h2 id=types>Types</h2><p>Iceberg data types
are located in the <a
href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/types/package-summary.html><code>org.apache.iceberg.types</code>
package</a>.</p><h3 id=primitives>Primitives</h3><p>Primitive type instances
are available from static methods in each type class. Types without parameters
use <code>get</code>, and types like <code>decimal</code> use factory
methods:</p><div class=highlight><pre tabin [...]
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>Types<span style=color:#f92672>.</span><span
style=color:#a6e22e>DoubleType</span><span style=color:#f92672>.</span><span
style=color:#a6e22e>get</span><span style=color:#f92672>()</span> <span
style=color:#75715e>// double
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>Types<span style=color:#f92672>.</span><span
style=color:#a6e22e>DecimalType</span><span style=color:#f92672>.</span><span
style=color:#a6e22e>of</span><span style=color:#f92672>(</span><span
style=color:#ae81ff>9</span><span style=color:#f92672>,</span> <span
style=color:#ae81ff>2</span><span style=color:#f92672>)</span> <span
style=color:#75715e>// decimal(9, 2)
</span></span></span></code></pre></div><h3 id=nested-types>Nested
types</h3><p>Structs, maps, and lists are created using factory methods in type
classes.</p><p>Like struct fields, map keys or values and list elements are
tracked as nested fields. Nested fields track <a
href=../evolution#correctness>field IDs</a> and nullability.</p><p>Struct
fields are created using <code>NestedField.optional</code> or
<code>NestedField.required</code>. Map value and list element nullability is
set in [...]
@@ -55,7 +55,7 @@
</span></span><span style=display:flex><span> <span
style=color:#f92672>)</span>
</span></span></code></pre></div><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span><span
style=color:#75715e>// array<1 element: int>
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>ListType list <span style=color:#f92672>=</span>
ListType<span style=color:#f92672>.</span><span
style=color:#a6e22e>ofRequired</span><span style=color:#f92672>(</span><span
style=color:#ae81ff>1</span><span style=color:#f92672>,</span> IntegerType<span
style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span
style=color:#f92672>());</span>
-</span></span></code></pre></div><h2
id=expressions>Expressions</h2><p>Iceberg’s expressions are used to
configure table scans. To create expressions, use the factory methods in <a
href=../../../javadoc/1.2.1/index.html?org/apache/iceberg/expressions/Expressions.html><code>Expressions</code></a>.</p><p>Supported
predicate expressions
are:</p><ul><li><code>isNull</code></li><li><code>notNull</code></li><li><code>equal</code></li><li><code>notEqual</code></li><li><code>lessThan</code
[...]
+</span></span></code></pre></div><h2
id=expressions>Expressions</h2><p>Iceberg’s expressions are used to
configure table scans. To create expressions, use the factory methods in <a
href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/expressions/Expressions.html><code>Expressions</code></a>.</p><p>Supported
predicate expressions
are:</p><ul><li><code>isNull</code></li><li><code>notNull</code></li><li><code>equal</code></li><li><code>notEqual</code></li><li><code>lessThan</code
[...]
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span
style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span
style=color:#a6e22e>greaterThanOrEqual</span><span
style=color:#f92672>(</span><span style=color:#e6db74>"x"</span><span
style=color:#f92672>,</span> <span style=color:#ae81ff>5</span><span
style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span
style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span
style=color:#a6e22e>lessThan</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"x"</span><span style=color:#f92672>,</span> <span
style=color:#ae81ff>10</span><span style=color:#f92672>))</span>
</span></span></code></pre></div><h2 id=modules>Modules</h2><p>Iceberg table
support is organized in library modules:</p><ul><li><code>iceberg-common</code>
contains utility classes used in other modules</li><li><code>iceberg-api</code>
contains the public Iceberg API, including expressions, types, tables, and
operations</li><li><code>iceberg-arrow</code> is an implementation of the
Iceberg type system for reading and writing data stored in Iceberg tables using
Apache Arrow as the in-mem [...]
diff --git a/docs/latest/aws/index.html b/docs/latest/aws/index.html
index e23841c1..4b2df2c9 100644
--- a/docs/latest/aws/index.html
+++ b/docs/latest/aws/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -27,7 +27,7 @@ To choose a different HTTP client library such as <a
href=https://mvnrepository.
see the section <a href=#aws-client-customization>client customization</a> for
more details.</p><p>All the AWS module features can be loaded through custom
catalog properties,
you can go to the documentations of each engine to see how to load a custom
catalog.
Here are some examples.</p><h3 id=spark>Spark</h3><p>For example, to use AWS
features with Spark 3.3 (with scala 2.12) and AWS clients version 2.20.18, you
can start the Spark SQL shell with:</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sh data-lang=sh><span style=display:flex><span><span
style=color:#75715e># add Iceberg dependency</span>
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.2.1
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.3.0
</span></span><span style=display:flex><span>DEPENDENCIES<span
style=color:#f92672>=</span><span
style=color:#e6db74>"org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:</span>$ICEBERG_VERSION<span
style=color:#e6db74>"</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e># add
AWS dependency</span>
@@ -48,7 +48,7 @@ Here are some examples.</p><h3 id=spark>Spark</h3><p>For
example, to use AWS fea
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf
spark.sql.catalog.my_catalog.catalog-impl<span
style=color:#f92672>=</span>org.apache.iceberg.aws.glue.GlueCatalog <span
style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.io-impl<span
style=color:#f92672>=</span>org.apache.iceberg.aws.s3.S3FileIO
</span></span></code></pre></div><p>As you can see, In the shell command, we
use <code>--packages</code> to specify the additional AWS bundle and HTTP
client dependencies with their version as <code>2.20.18</code>.</p><h3
id=flink>Flink</h3><p>To use AWS module with Flink, you can download the
necessary dependencies and specify them when starting the Flink SQL
client:</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-si
[...]
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.2.1
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.3.0
</span></span><span style=display:flex><span>MAVEN_URL<span
style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span
style=color:#f92672>=</span>$MAVEN_URL/org/apache/iceberg
</span></span><span style=display:flex><span>wget
$ICEBERG_MAVEN_URL/iceberg-flink-runtime/$ICEBERG_VERSION/iceberg-flink-runtime-$ICEBERG_VERSION.jar
@@ -231,7 +231,7 @@ In this case, a <a
href=https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role
Iceberg provides an AWS client factory <code>AssumeRoleAwsClientFactory</code>
to support this common use case.
This also serves as an example for users who would like to implement their own
AWS client factory.</p><p>This client factory has the following configurable
catalog
properties:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>client.assume-role.arn</td><td>null,
requires user input</td><td>ARN of the role to assume, e.g.
arn:aws:iam::123456789:role/myRoleToAssume</td></tr><tr><td>client.assume-role.region</td><td>null,
requires user inp [...]
The Glue, S3 and DynamoDB clients are then initialized with the assume-role
credential and region to access resources.
-Here is an example to start Spark shell with this client factory:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-shell data-lang=shell><span style=display:flex><span>spark-sql
--packages
org.apache.iceberg:iceberg-spark-runtime:1.2.1,software.amazon.awssdk:bundle:2.20.18
<span style=color:#ae81ff>\
+Here is an example to start Spark shell with this client factory:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-shell data-lang=shell><span style=display:flex><span>spark-sql
--packages
org.apache.iceberg:iceberg-spark-runtime:1.3.0,software.amazon.awssdk:bundle:2.20.18
<span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog<span
style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span
style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf
spark.sql.catalog.my_catalog.warehouse<span
style=color:#f92672>=</span>s3://my-bucket/my/key/prefix <span
style=color:#ae81ff>\ </span>
</span></span><span style=display:flex><span> --conf
spark.sql.catalog.my_catalog.catalog-impl<span
style=color:#f92672>=</span>org.apache.iceberg.aws.glue.GlueCatalog <span
style=color:#ae81ff>\
@@ -251,7 +251,7 @@ More details could be found <a
href=https://docs.aws.amazon.com/athena/latest/ug
Please refer to the <a
href=https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-iceberg-use-cluster.html>official
documentation</a> on how to create a cluster with Iceberg installed.</p><p>For
versions before 6.5.0, you can use a <a
href=https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-bootstrap.html>bootstrap
action</a> similar to the following to pre-install all necessary
dependencies:</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#2728 [...]
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>
</span></span><span style=display:flex><span>AWS_SDK_VERSION<span
style=color:#f92672>=</span>2.20.18
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.2.1
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.3.0
</span></span><span style=display:flex><span>MAVEN_URL<span
style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span
style=color:#f92672>=</span>$MAVEN_URL/org/apache/iceberg
</span></span><span style=display:flex><span>AWS_MAVEN_URL<span
style=color:#f92672>=</span>$MAVEN_URL/software/amazon/awssdk
diff --git a/docs/latest/branching/index.html b/docs/latest/branching/index.html
index 2cb20c0f..4f233f1c 100644
--- a/docs/latest/branching/index.html
+++ b/docs/latest/branching/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a id=active href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/configuration/index.html
b/docs/latest/configuration/index.html
index d56b5e2a..669aab76 100644
--- a/docs/latest/configuration/index.html
+++ b/docs/latest/configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a id=active
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/custom-catalog/index.html
b/docs/latest/custom-catalog/index.html
index dc46e7e5..42316849 100644
--- a/docs/latest/custom-catalog/index.html
+++ b/docs/latest/custom-catalog/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/dell/index.html b/docs/latest/dell/index.html
index cad7a2c5..739ddbb3 100644
--- a/docs/latest/dell/index.html
+++ b/docs/latest/dell/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/delta-lake-migration/index.html
b/docs/latest/delta-lake-migration/index.html
index c4f41ee0..e55b9f18 100644
--- a/docs/latest/delta-lake-migration/index.html
+++ b/docs/latest/delta-lake-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/evolution/index.html b/docs/latest/evolution/index.html
index 97139d59..e63ac74a 100644
--- a/docs/latest/evolution/index.html
+++ b/docs/latest/evolution/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
id=active href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-actions/index.html
b/docs/latest/flink-actions/index.html
index 608f7ba5..52bc9dec 100644
--- a/docs/latest/flink-actions/index.html
+++ b/docs/latest/flink-actions/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -22,7 +22,7 @@
</span></span><span style=display:flex><span>RewriteDataFilesActionResult
result <span style=color:#f92672>=</span> Actions<span
style=color:#f92672>.</span><span style=color:#a6e22e>forTable</span><span
style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span
style=color:#a6e22e>rewriteDataFiles</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span
style=color:#f92672>();</span>
-</span></span></code></pre></div><p>For more details of the rewrite files
action, please refer to <a
href=../../../javadoc/1.2.1/org/apache/iceberg/flink/actions/RewriteDataFilesAction.html>RewriteDataFilesAction</a></p></div><div
id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a
href=#rewrite-files-action>Rewrite files
action.</a></li></ul></nav></div></div></div></div></section></body><script
src=https://iceberg.apache.org/docs/latest//js/jquery-1.11.0.js></script>
+</span></span></code></pre></div><p>For more details of the rewrite files
action, please refer to <a
href=../../../javadoc/1.3.0/org/apache/iceberg/flink/actions/RewriteDataFilesAction.html>RewriteDataFilesAction</a></p></div><div
id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a
href=#rewrite-files-action>Rewrite files
action.</a></li></ul></nav></div></div></div></div></section></body><script
src=https://iceberg.apache.org/docs/latest//js/jquery-1.11.0.js></script>
<script
src=https://iceberg.apache.org/docs/latest//js/jquery.easing.min.js></script>
<script type=text/javascript
src=https://iceberg.apache.org/docs/latest//js/search.js></script>
<script
src=https://iceberg.apache.org/docs/latest//js/bootstrap.min.js></script>
diff --git a/docs/latest/flink-configuration/index.html
b/docs/latest/flink-configuration/index.html
index 38a4a770..4837a9ab 100644
--- a/docs/latest/flink-configuration/index.html
+++ b/docs/latest/flink-configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-connector/index.html
b/docs/latest/flink-connector/index.html
index c2ff51de..c583ce85 100644
--- a/docs/latest/flink-connector/index.html
+++ b/docs/latest/flink-connector/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-ddl/index.html b/docs/latest/flink-ddl/index.html
index 55db2686..af853b55 100644
--- a/docs/latest/flink-ddl/index.html
+++ b/docs/latest/flink-ddl/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-queries/index.html
b/docs/latest/flink-queries/index.html
index 4549a781..ae61acaf 100644
--- a/docs/latest/flink-queries/index.html
+++ b/docs/latest/flink-queries/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -72,7 +72,7 @@ refer to <a href=../flink-configuration/#read-options>Flink
Configuration</a></p
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>//
Submit and execute this streaming read job.
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>env<span style=color:#f92672>.</span><span
style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"Test Iceberg Streaming Read"</span><span
style=color:#f92672>);</span>
-</span></span></code></pre></div><p>There are other options that can be set,
please see the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/flink/source/FlinkSource.html>FlinkSource#Builder</a>.</p><h2
id=reading-with-datastream-flip-27-source>Reading with DataStream (FLIP-27
source)</h2><p><a
href=https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface>FLIP-27
source interface</a>
+</span></span></code></pre></div><p>There are other options that can be set,
please see the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/flink/source/FlinkSource.html>FlinkSource#Builder</a>.</p><h2
id=reading-with-datastream-flip-27-source>Reading with DataStream (FLIP-27
source)</h2><p><a
href=https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface>FLIP-27
source interface</a>
was introduced in Flink 1.12. It aims to solve several shortcomings of the old
<code>SourceFunction</code>
streaming source interface. It also unifies the source interfaces for both
batch and streaming executions.
Most source connectors (like Kafka, file) in Flink repo have migrated to the
FLIP-27 interface.
@@ -120,7 +120,7 @@ CDC read is not supported yet.</p><div class=highlight><pre
tabindex=0 style=col
</span></span><span style=display:flex><span><span style=color:#75715e>//
Submit and execute this streaming read job.
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>env<span style=color:#f92672>.</span><span
style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"Test Iceberg Streaming Read"</span><span
style=color:#f92672>);</span>
</span></span></code></pre></div><p>There are other options that could be set
by Java API, please see the
-<a
href=../../../javadoc/1.2.1/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3
id=reading-branches-and-tags-with-datastream>Reading branches and tags with
DataStream</h3><p>Branches and tags can also be read via the DataStream
API</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span
style=display:flex><span>StreamExecutionEnvironment [...]
+<a
href=../../../javadoc/1.3.0/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3
id=reading-branches-and-tags-with-datastream>Reading branches and tags with
DataStream</h3><p>Branches and tags can also be read via the DataStream
API</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span
style=display:flex><span>StreamExecutionEnvironment [...]
</span></span><span style=display:flex><span>TableLoader tableLoader <span
style=color:#f92672>=</span> TableLoader<span style=color:#f92672>.</span><span
style=color:#a6e22e>fromHadoopTable</span><span
style=color:#f92672>(</span><span
style=color:#e6db74>"hdfs://nn:8020/warehouse/path"</span><span
style=color:#f92672>);</span>
</span></span><span style=display:flex><span><span style=color:#75715e>// Read
from branch
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>DataStream<span
style=color:#f92672><</span>RowData<span style=color:#f92672>></span>
batch <span style=color:#f92672>=</span> FlinkSource<span
style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span
style=color:#f92672>()</span>
diff --git a/docs/latest/flink-writes/index.html
b/docs/latest/flink-writes/index.html
index a8cbf8fe..f3a648c9 100644
--- a/docs/latest/flink-writes/index.html
+++ b/docs/latest/flink-writes/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink/index.html b/docs/latest/flink/index.html
index c0fc6b34..72122dde 100644
--- a/docs/latest/flink/index.html
+++ b/docs/latest/flink/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -31,14 +31,14 @@
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e># Start
the flink standalone cluster</span>
</span></span><span style=display:flex><span>./bin/start-cluster.sh
-</span></span></code></pre></div><p>Start the Flink SQL client. There is a
separate <code>flink-runtime</code> module in the Iceberg project to generate a
bundled jar, which could be loaded by Flink SQL client directly. To build the
<code>flink-runtime</code> bundled jar manually, build the <code>iceberg</code>
project, and it will generate the jar under
<code><iceberg-root-dir>/flink-runtime/build/libs</code>. Or download the
<code>flink-runtime</code> jar from the <a href=https://re [...]
+</span></span></code></pre></div><p>Start the Flink SQL client. There is a
separate <code>flink-runtime</code> module in the Iceberg project to generate a
bundled jar, which could be loaded by Flink SQL client directly. To build the
<code>flink-runtime</code> bundled jar manually, build the <code>iceberg</code>
project, and it will generate the jar under
<code><iceberg-root-dir>/flink-runtime/build/libs</code>. Or download the
<code>flink-runtime</code> jar from the <a href=https://re [...]
</span></span><span style=display:flex><span>export HADOOP_CLASSPATH<span
style=color:#f92672>=</span><span
style=color:#e6db74>`</span>$HADOOP_HOME/bin/hadoop classpath<span
style=color:#e6db74>`</span>
</span></span><span style=display:flex><span>
-</span></span><span style=display:flex><span>./bin/sql-client.sh embedded -j
<flink-runtime-directory>/iceberg-flink-runtime-1.16-1.2.1.jar shell
+</span></span><span style=display:flex><span>./bin/sql-client.sh embedded -j
<flink-runtime-directory>/iceberg-flink-runtime-1.16-1.3.0.jar shell
</span></span></code></pre></div><p>By default, Iceberg ships with Hadoop jars
for Hadoop catalog. To use Hive catalog, load the Hive jars when opening the
Flink SQL client. Fortunately, Flink has provided a <a
href=https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-hive-2.3.9_2.12/1.16.1/flink-sql-connector-hive-2.3.9_2.12-1.16.1.jar>bundled
hive jar</a> for the SQL client. An example on how to download the
dependencies and get started:</p><div class=highlight><pr [...]
</span></span><span style=display:flex><span>export HADOOP_CLASSPATH<span
style=color:#f92672>=</span><span
style=color:#e6db74>`</span>$HADOOP_HOME/bin/hadoop classpath<span
style=color:#e6db74>`</span>
</span></span><span style=display:flex><span>
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.2.1
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span
style=color:#f92672>=</span>1.3.0
</span></span><span style=display:flex><span>MAVEN_URL<span
style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span
style=color:#f92672>=</span><span style=color:#e6db74>${</span>MAVEN_URL<span
style=color:#e6db74>}</span>/org/apache/iceberg
</span></span><span style=display:flex><span>ICEBERG_PACKAGE<span
style=color:#f92672>=</span>iceberg-flink-runtime
@@ -58,7 +58,7 @@
</span></span><span style=display:flex><span><span
style=color:#f92672>from</span> pyflink.datastream <span
style=color:#f92672>import</span> StreamExecutionEnvironment
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env <span
style=color:#f92672>=</span> StreamExecutionEnvironment<span
style=color:#f92672>.</span>get_execution_environment()
-</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span
style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span
style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(),
<span style=color:#e6db74>"iceberg-flink-runtime-1.16-1.2.1.jar"</span>)
+</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span
style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span
style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(),
<span style=color:#e6db74>"iceberg-flink-runtime-1.16-1.3.0.jar"</span>)
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env<span
style=color:#f92672>.</span>add_jars(<span
style=color:#e6db74>"file://</span><span style=color:#e6db74>{}</span><span
style=color:#e6db74>"</span><span
style=color:#f92672>.</span>format(iceberg_flink_runtime_jar))
</span></span></code></pre></div><p>Next, create a
<code>StreamTableEnvironment</code> and execute Flink SQL statements. The below
example shows how to create a custom catalog via the Python Table API:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-python data-lang=python><span style=display:flex><span><span
style=color:#f92672>from</span> pyflink.table <span
style=color:#f92672>import</sp [...]
diff --git a/docs/latest/getting-started/index.html
b/docs/latest/getting-started/index.html
index 9b519912..9d03ddf8 100644
--- a/docs/latest/getting-started/index.html
+++ b/docs/latest/getting-started/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,10 +15,10 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul
class=sub-menu><li><a href=../java-api-quickstart/>Java
Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a
href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a
class="chevron-toggle collapsed" data-toggle=collapse data-parent=full
href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
We recommend you to get started with Spark to understand Iceberg concepts and
features with examples.
-You can also view documentations of using Iceberg with other compute engine
under the <a href=https://iceberg.apache.org/multi-engine-support>Multi-Engine
Support</a> page.</p><h2 id=using-iceberg-in-spark-3>Using Iceberg in Spark
3</h2><p>To use Iceberg in a Spark shell, use the <code>--packages</code>
option:</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sh data-lang=sh><span style=disp [...]
-</span></span></code></pre></div><div class=info>If you want to include
Iceberg in your Spark installation, add the <a
href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/1.2.1/iceberg-spark-runtime-3.2_2.12-1.2.1.jar";><code>iceberg-spark-runtime-3.2_2.12</code>
Jar</a> to Spark’s <code>jars</code> folder.</div><h3
id=adding-catalogs>Adding catalogs</h3><p>Iceberg comes with <a
href=../spark-configuration#catalogs>catalogs</a> tha [...]
+You can also view documentations of using Iceberg with other compute engine
under the <a href=https://iceberg.apache.org/multi-engine-support>Multi-Engine
Support</a> page.</p><h2 id=using-iceberg-in-spark-3>Using Iceberg in Spark
3</h2><p>To use Iceberg in a Spark shell, use the <code>--packages</code>
option:</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sh data-lang=sh><span style=disp [...]
+</span></span></code></pre></div><div class=info>If you want to include
Iceberg in your Spark installation, add the <a
href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/1.3.0/iceberg-spark-runtime-3.2_2.12-1.3.0.jar";><code>iceberg-spark-runtime-3.2_2.12</code>
Jar</a> to Spark’s <code>jars</code> folder.</div><h3
id=adding-catalogs>Adding catalogs</h3><p>Iceberg comes with <a
href=../spark-configuration#catalogs>catalogs</a> tha [...]
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.extensions<span
style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
<span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog<span
style=color:#f92672>=</span>org.apache.iceberg.spark.SparkSessionCatalog <span
style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog.type<span
style=color:#f92672>=</span>hive <span style=color:#ae81ff>\
diff --git a/docs/latest/hive-migration/index.html
b/docs/latest/hive-migration/index.html
index 151f8532..2763bc62 100644
--- a/docs/latest/hive-migration/index.html
+++ b/docs/latest/hive-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/hive/index.html b/docs/latest/hive/index.html
index 35d17100..df911d42 100644
--- a/docs/latest/hive/index.html
+++ b/docs/latest/hive/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -71,7 +71,7 @@ The data is inserted / committed when the query finishes. So
for a transient per
</span></span><span style=display:flex><span> <span
style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span
style=color:#66d9ef>FROM</span> <span style=color:#66d9ef>source</span>;
</span></span></code></pre></div><h3 id=create-table-like-table>CREATE TABLE
LIKE TABLE</h3><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>TABLE</span> target
<span style=color:#66d9ef>LIKE</span> <span style=color:#66d9ef>source</span>
STORED <span style=color:#66d9ef>BY</span> [...]
</span></span></code></pre></div><h3
id=create-external-table-overlaying-an-existing-iceberg-table>CREATE EXTERNAL
TABLE overlaying an existing Iceberg table</h3><p>The <code>CREATE EXTERNAL
TABLE</code> command is used to overlay a Hive table “on top of” an
existing Iceberg table. Iceberg
-tables are created using either a <a
href=../../../javadoc/1.2.1/index.html?org/apache/iceberg/catalog/Catalog.html><code>Catalog</code></a>,
or an implementation of the <a
href=../../../javadoc/1.2.1/index.html?org/apache/iceberg/Tables.html><code>Tables</code></a>
interface, and Hive needs to be configured accordingly to
+tables are created using either a <a
href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/catalog/Catalog.html><code>Catalog</code></a>,
or an implementation of the <a
href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/Tables.html><code>Tables</code></a>
interface, and Hive needs to be configured accordingly to
operate on these different types of table.</p><h4 id=hive-catalog-tables>Hive
catalog tables</h4><p>As described before, tables created by the
<code>HiveCatalog</code> with Hive engine feature enabled are directly visible
by the
Hive engine, so there is no need to create an overlay.</p><h4
id=custom-catalog-tables>Custom catalog tables</h4><p>For a table in a
registered catalog, specify the catalog name in the statement using table
property <code>iceberg.catalog</code>.
For example, the SQL below creates an overlay for a table in a
<code>hadoop</code> type catalog named <code>hadoop_cat</code>:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>SET</span>
diff --git a/docs/latest/index.html b/docs/latest/index.html
index 623f98be..f2345a6e 100644
--- a/docs/latest/index.html
+++ b/docs/latest/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=./branching/>Branching and Tagging</a></li><li><a
href=./configuration/>Configuration</a></li><li><a
href=./evolution/>Evolution</a></li><li><a
href=./maintenance/>Maintenance</a></li><li><a
href=./partitioning/>Partitioning</a></li><li><a
href=./performance/>Performance</a></li><li><a
href=./reliability/>Reliability</a></li><li><a
href=./schemas/>Schemas</a></li></ul></div><li><a clas [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/index.xml b/docs/latest/index.xml
index 5832b05f..9b6c22a8 100644
--- a/docs/latest/index.xml
+++ b/docs/latest/index.xml
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0"
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Introduction on Apache
Iceberg</title><link>https://iceberg.apache.org/docs/latest/</link><description>Recent
content in Introduction on Apache Iceberg</description><generator>Hugo --
gohugo.io</generator><language>en-us</language><atom:link
href="https://iceberg.apache.org/docs/latest/index.xml"; rel="self"
type="application/rss+xml"/><item><title>Getting Sta [...]
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0"
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Introduction on Apache
Iceberg</title><link>https://iceberg.apache.org/docs/latest/</link><description>Recent
content in Introduction on Apache Iceberg</description><generator>Hugo --
gohugo.io</generator><language>en-us</language><atom:link
href="https://iceberg.apache.org/docs/latest/index.xml"; rel="self"
type="application/rss+xml"/><item><title>Getting Sta [...]
Spark is currently the most feature-rich compute engine for Iceberg
operations. We recommend you to get started with Spark to understand Iceberg
concepts and features with examples. You can also view documentations of using
Iceberg with other compute engine under the Multi-Engine Support page.
Using Iceberg in Spark 3 To use Iceberg in a Spark shell, use the --packages
option:</description></item><item><title>Hive</title><link>https://iceberg.apache.org/docs/latest/hive/</link><pubDate>Mon,
01 Jan 0001 00:00:00
+0000</pubDate><guid>https://iceberg.apache.org/docs/latest/hive/</guid><description>Hive
Iceberg supports reading and writing Iceberg tables through Hive by using a
StorageHandler.
Feature support Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports
the following features:
diff --git a/docs/latest/java-api-quickstart/index.html
b/docs/latest/java-api-quickstart/index.html
index 6bcb735f..e4856aef 100644
--- a/docs/latest/java-api-quickstart/index.html
+++ b/docs/latest/java-api-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,7 +15,7 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul
class=sub-menu><li><a id=active href=../java-api-quickstart/>Java
Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a
href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a
class="chevron-toggle collapsed" data-toggle=collapse data-parent=full
href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
You can initialize a Hive catalog with a name and some properties.
(see: <a href=../configuration/#catalog-properties>Catalog
properties</a>)</p><p><strong>Note:</strong> Currently, <code>setConf</code> is
always required for hive catalogs, but this will change in the future.</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span><span
style=color:#f92672>import</span> org.apache.iceberg.hive.HiveCatalog<span sty
[...]
</span></span><span style=display:flex><span>
diff --git a/docs/latest/jdbc/index.html b/docs/latest/jdbc/index.html
index 43d4fb81..a1787671 100644
--- a/docs/latest/jdbc/index.html
+++ b/docs/latest/jdbc/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -18,7 +18,7 @@
<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
The database that JDBC connects to must support atomic transaction to allow
the JDBC catalog implementation to
properly support atomic Iceberg table commits and read serializable
isolation.</p><h3 id=configurations>Configurations</h3><p>Because each database
and database service provider might require different configurations,
-the JDBC catalog allows arbitrary configurations
through:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>uri</td><td></td><td>the
JDBC connection
string</td></tr><tr><td>jdbc.<property_key></td><td></td><td>any key value
pairs to configure the JDBC connection</td></tr></tbody></table><h3
id=examples>Examples</h3><h4 id=spark>Spark</h4><p>You can start a Spark
session with a MySQL JDBC connection using the following configurations: [...]
+the JDBC catalog allows arbitrary configurations
through:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>uri</td><td></td><td>the
JDBC connection
string</td></tr><tr><td>jdbc.<property_key></td><td></td><td>any key value
pairs to configure the JDBC connection</td></tr></tbody></table><h3
id=examples>Examples</h3><h4 id=spark>Spark</h4><p>You can start a Spark
session with a MySQL JDBC connection using the following configurations: [...]
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog<span
style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span
style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf
spark.sql.catalog.my_catalog.warehouse<span
style=color:#f92672>=</span>s3://my-bucket/my/key/prefix <span
style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span
style=color:#ae81ff></span> --conf
spark.sql.catalog.my_catalog.catalog-impl<span
style=color:#f92672>=</span>org.apache.iceberg.jdbc.JdbcCatalog <span
style=color:#ae81ff>\
diff --git a/docs/latest/maintenance/index.html
b/docs/latest/maintenance/index.html
index 65e83e9c..30091ea4 100644
--- a/docs/latest/maintenance/index.html
+++ b/docs/latest/maintenance/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a id=active
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,12 +15,12 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul
class=sub-menu><li><a href=../java-api-quickstart/>Java
Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a
href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a
class="chevron-toggle collapsed" data-toggle=collapse data-parent=full
href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
</span></span><span style=display:flex><span><span
style=color:#66d9ef>long</span> tsToExpire <span style=color:#f92672>=</span>
System<span style=color:#f92672>.</span><span
style=color:#a6e22e>currentTimeMillis</span><span style=color:#f92672>()</span>
<span style=color:#f92672>-</span> <span style=color:#f92672>(</span><span
style=color:#ae81ff>1000</span> <span style=color:#f92672>*</span> <span
style=color:#ae81ff>60</span> <span style=color:#f92672>*</span> <span
style=color:#ae81f [...]
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>table<span style=color:#f92672>.</span><span
style=color:#a6e22e>expireSnapshots</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span
style=color:#a6e22e>expireOlderThan</span><span
style=color:#f92672>(</span>tsToExpire<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>commit</span><span
style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/ExpireSnapshots.html><code>ExpireSnapshots</code>
Javadoc</a> to see more configuration options.</p><p>There is also a Spark
action that can run table expiration in parallel for large tables:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span>Table ta
[...]
+</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/ExpireSnapshots.html><code>ExpireSnapshots</code>
Javadoc</a> to see more configuration options.</p><p>There is also a Spark
action that can run table expiration in parallel for large tables:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span>Table ta
[...]
</span></span><span style=display:flex><span>SparkActions
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span
style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span
style=color:#a6e22e>expireSnapshots</span><span
style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
@@ -34,7 +34,7 @@ Configuring
<code>write.metadata.delete-after-commit.enabled=true</code> and <co
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span
style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span
style=color:#a6e22e>deleteOrphanFiles</span><span
style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span
style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/actions/DeleteOrphanFiles.html>DeleteOrphanFiles
Javadoc</a> to see more configuration options.</p><p>This action may take a
long time to finish if you have lots of files in data and metadata directories.
It is recommended to execute this periodically, but you may not need to execute
this often.</p><div class=info>It is dangerous to remove orphan files with a
retention interval shorter than the [...]
+</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/actions/DeleteOrphanFiles.html>DeleteOrphanFiles
Javadoc</a> to see more configuration options.</p><p>This action may take a
long time to finish if you have lots of files in data and metadata directories.
It is recommended to execute this periodically, but you may not need to execute
this often.</p><div class=info>It is dangerous to remove orphan files with a
retention interval shorter than the [...]
might corrupt the table if in-progress files are considered orphaned and are
deleted. The default interval is 3 days.</div><div class=info>Iceberg uses the
string representations of paths when determining which files need to be
removed. On some file systems,
the path can change over time, but it still represents the same file. For
example, if you change authorities for an HDFS cluster,
none of the old path urls used during creation will match those that appear in
a current listing. <em>This will lead to data loss when
@@ -46,13 +46,13 @@ FileSystem API to avoid unintentional deletion.</div><h2
id=optional-maintenance
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span
style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span
style=color:#a6e22e>equal</span><span style=color:#f92672>(</span><span
style=color:#e6db74>"date"</span><span style=color:#f92672>,</span>
<span style=color:#e6db74>"2020-08-18"</span><span
style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>option</span><span
style=color:#f92672>(</span><span
style=color:#e6db74>"target-file-size-bytes"</span><span
style=color:#f92672>,</span> Long<span style=color:#f92672>.</span><span
style=color:#a6e22e>toString</span><span style=color:#f92672>(</span><span
style=color:#ae81ff>500</span> <span style=color:#f92672>*</span> <span
style=color:#ae81ff>1024</span> <span style=c [...]
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span> <span style=color:#f92672>.</span><span
style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>The <code>files</code> metadata table is
useful for inspecting data file sizes and determining when to compact
partitions.</p><p>See the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/actions/RewriteDataFiles.html><code>RewriteDataFiles</code>
Javadoc</a> to see more configuration options.</p><h3
id=rewrite-manifests>Rewrite manifests</h3><p>Iceberg uses metadata in its
manifest list and manifest files speed up query planning and to prune
unnecessar [...]
+</span></span></code></pre></div><p>The <code>files</code> metadata table is
useful for inspecting data file sizes and determining when to compact
partitions.</p><p>See the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteDataFiles.html><code>RewriteDataFiles</code>
Javadoc</a> to see more configuration options.</p><h3
id=rewrite-manifests>Rewrite manifests</h3><p>Iceberg uses metadata in its
manifest list and manifest files speed up query planning and to prune
unnecessar [...]
</span></span><span style=display:flex><span>SparkActions
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span
style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span
style=color:#a6e22e>rewriteManifests</span><span
style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>rewriteIf</span><span
style=color:#f92672>(</span>file <span style=color:#f92672>-></span>
file<span style=color:#f92672>.</span><span
style=color:#a6e22e>length</span><span style=color:#f92672>()</span> <span
style=color:#f92672><</span> <span style=color:#ae81ff>10</span> <span
style=color:#f92672>*</span> <span style=color:#ae81ff>1024</span> <span
style=color:#f92672>*</ [...]
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span> <span style=color:#f92672>.</span><span
style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code>
Javadoc</a> to see more configuration options.</p></div><div id=toc
class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a
href=#recommended-maintenance>Recommended Maintenance</a><ul><li><a
href=#expire-snapshots>Expire Snapshots</a></li><li><a
href=#remove-old-metadata-files>Remove old metadata files</a></li><li><a href=#
[...]
+</span></span></code></pre></div><p>See the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code>
Javadoc</a> to see more configuration options.</p></div><div id=toc
class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a
href=#recommended-maintenance>Recommended Maintenance</a><ul><li><a
href=#expire-snapshots>Expire Snapshots</a></li><li><a
href=#remove-old-metadata-files>Remove old metadata files</a></li><li><a href=#
[...]
<script
src=https://iceberg.apache.org/docs/latest//js/jquery.easing.min.js></script>
<script type=text/javascript
src=https://iceberg.apache.org/docs/latest//js/search.js></script>
<script
src=https://iceberg.apache.org/docs/latest//js/bootstrap.min.js></script>
diff --git a/docs/latest/nessie/index.html b/docs/latest/nessie/index.html
index 4c505c33..78654d9c 100644
--- a/docs/latest/nessie/index.html
+++ b/docs/latest/nessie/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -18,9 +18,9 @@
<i class="fa fa-chevron-down"></i></a></li><div id=Migration
class=collapse><ul class=sub-menu><li><a
href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive
Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake
Migration</a></li></ul></div><li><a
href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a
target=_blank
href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div
id=content [...]
This section describes how to use Iceberg with Nessie. Nessie provides several
key features on top of Iceberg:</p><ul><li>multi-table
transactions</li><li>git-like operations (eg branches, tags,
commits)</li><li>hive-like metastore capabilities</li></ul><p>See <a
href=https://projectnessie.org>Project Nessie</a> for more information on
Nessie. Nessie requires a server to run, see
<a href=https://projectnessie.org/try/>Getting Started</a> to start a Nessie
server.</p><h2 id=enabling-nessie-catalog>Enabling Nessie Catalog</h2><p>The
<code>iceberg-nessie</code> module is bundled with Spark and Flink runtimes for
all versions from <code>0.11.0</code>. To get started
-with Nessie (with spark-3.3) and Iceberg simply add the Iceberg runtime to
your process. Eg: <code>spark-sql --packages
org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.2.1</code>.</p><h2
id=spark-sql-extensions>Spark SQL Extensions</h2><p>From Spark 3.1 and above,
Nessie SQL extensions can be used to manage the Nessie repo as shown below.
+with Nessie (with spark-3.3) and Iceberg simply add the Iceberg runtime to
your process. Eg: <code>spark-sql --packages
org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.0</code>.</p><h2
id=spark-sql-extensions>Spark SQL Extensions</h2><p>From Spark 3.1 and above,
Nessie SQL extensions can be used to manage the Nessie repo as shown below.
Example for Spark 3.3 with scala 2.12:</p><pre tabindex=0><code>bin/spark-sql
- --packages
"org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.2.1,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.3_2.12:0.50.0"
+ --packages
"org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.0,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.3_2.12:0.50.0"
--conf
spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions"
--conf <other settings>
</code></pre><p>Please refer <a
href=https://projectnessie.org/tools/sql/>Nessie SQL extension document</a> to
learn more about it.</p><h2 id=nessie-catalog>Nessie Catalog</h2><p>One major
feature introduced in release <code>0.11.0</code> is the ability to easily
interact with a <a href=../custom-catalog>Custom
@@ -41,7 +41,7 @@ and <a href=../flink#custom-catalog>Flink Configuration</a>
for instructions for
</span></span><span style=display:flex><span><span
style=color:#f92672>from</span> pyflink.table <span
style=color:#f92672>import</span> StreamTableEnvironment
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env <span
style=color:#f92672>=</span> StreamExecutionEnvironment<span
style=color:#f92672>.</span>get_execution_environment()
-</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span
style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span
style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(),
<span style=color:#e6db74>"iceberg-flink-runtime-1.2.1.jar"</span>)
+</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span
style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span
style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(),
<span style=color:#e6db74>"iceberg-flink-runtime-1.3.0.jar"</span>)
</span></span><span style=display:flex><span>env<span
style=color:#f92672>.</span>add_jars(<span
style=color:#e6db74>"file://</span><span style=color:#e6db74>{}</span><span
style=color:#e6db74>"</span><span
style=color:#f92672>.</span>format(iceberg_flink_runtime_jar))
</span></span><span style=display:flex><span>table_env <span
style=color:#f92672>=</span> StreamTableEnvironment<span
style=color:#f92672>.</span>create(env)
</span></span><span style=display:flex><span>
diff --git a/docs/latest/partitioning/index.html
b/docs/latest/partitioning/index.html
index b5508c95..c9caa7a8 100644
--- a/docs/latest/partitioning/index.html
+++ b/docs/latest/partitioning/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a id=active
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/performance/index.html
b/docs/latest/performance/index.html
index 31008273..c74d300d 100644
--- a/docs/latest/performance/index.html
+++ b/docs/latest/performance/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a id=active
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/reliability/index.html
b/docs/latest/reliability/index.html
index c2f30358..4a630b58 100644
--- a/docs/latest/reliability/index.html
+++ b/docs/latest/reliability/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a id=active
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/schemas/index.html b/docs/latest/schemas/index.html
index 90e45c0b..a434ac5f 100644
--- a/docs/latest/schemas/index.html
+++ b/docs/latest/schemas/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse
in"><ul class=sub-menu><li><a href=../branching/>Branching and
Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a id=active
href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-configuration/index.html
b/docs/latest/spark-configuration/index.html
index 1b5b5b16..17815c20 100644
--- a/docs/latest/spark-configuration/index.html
+++ b/docs/latest/spark-configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-ddl/index.html b/docs/latest/spark-ddl/index.html
index 9d4dbfe2..9288c0cb 100644
--- a/docs/latest/spark-ddl/index.html
+++ b/docs/latest/spark-ddl/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-procedures/index.html
b/docs/latest/spark-procedures/index.html
index 0e4cbf4d..85a634d5 100644
--- a/docs/latest/spark-procedures/index.html
+++ b/docs/latest/spark-procedures/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -29,8 +29,8 @@ the <code>expire_snapshots</code> procedure will never remove
files which are st
</span></span></code></pre></div><p>Remove snapshots with snapshot ID
<code>123</code> (note that this snapshot ID should not be the current
snapshot):</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> hive_prod.<span
style=color:#66d9ef>system</span>.expire_snapshots(<span
style=color:#66d9ef>table</span> [...]
</span></span></code></pre></div><h3
id=remove_orphan_files><code>remove_orphan_files</code></h3><p>Used to remove
files which are not referenced in any metadata files of an Iceberg table and
can thus be considered “orphaned”.</p><h4
id=usage-6>Usage</h4><table><thead><tr><th>Argument
Name</th><th>Required?</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>table</code></td><td>✔️</td><td>string</td><td>Name
of the table to clean</td></tr><tr><td><code>old [...]
</span></span></code></pre></div><p>Remove any files in the
<code>tablelocation/data</code> folder which are not known to the table
<code>db.sample</code>.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<span
style=color:#66d9ef>system</span>.remove_orphan_fi [...]
-</span></span></code></pre></div><h3
id=rewrite_data_files><code>rewrite_data_files</code></h3><p>Iceberg tracks
each data file in a table. More data files leads to more metadata stored in
manifest files, and small data files causes an unnecessary amount of metadata
and less efficient queries from file open costs.</p><p>Iceberg can compact data
files in parallel using Spark with the <code>rewriteDataFiles</code> action.
This will combine small files into larger files to reduce metadata o [...]
-and <a
href=../../../javadoc/1.2.1/org/apache/iceberg/actions/SortStrategy.html#field.summary><code>SortStrategy</code>
Javadoc</a>
+</span></span></code></pre></div><h3
id=rewrite_data_files><code>rewrite_data_files</code></h3><p>Iceberg tracks
each data file in a table. More data files leads to more metadata stored in
manifest files, and small data files causes an unnecessary amount of metadata
and less efficient queries from file open costs.</p><p>Iceberg can compact data
files in parallel using Spark with the <code>rewriteDataFiles</code> action.
This will combine small files into larger files to reduce metadata o [...]
+and <a
href=../../../javadoc/1.3.0/org/apache/iceberg/actions/SortStrategy.html#field.summary><code>SortStrategy</code>
Javadoc</a>
for list of all the supported options for this action.</p><h4
id=output-6>Output</h4><table><thead><tr><th>Output
Name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>rewritten_data_files_count</code></td><td>int</td><td>Number
of data which were re-written by this
command</td></tr><tr><td><code>added_data_files_count</code></td><td>int</td><td>Number
of new data files which were written by this
command</td></tr></tbody></table><h4 id=examples-3>Examples</h4><p>Rew [...]
and also split large files according to the default write size of the
table.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<span
style=color:#66d9ef>system</span>.rewrite_data_files(<span
style=color:#e6db74>'db.sample'</span>)
</span></span></code></pre></div><p>Rewrite the data files in table
<code>db.sample</code> by sorting all the data on id and name
@@ -39,10 +39,10 @@ using the same defaults as bin-pack to determine which
files to rewrite.</p><div
Using the same defaults as bin-pack to determine which files to
rewrite.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<span
style=color:#66d9ef>system</span>.rewrite_data_files(<span
style=color:#66d9ef>table</span> <span style=color:#f92672>=></span> <s [...]
</span></span></code></pre></div><p>Rewrite the data files in table
<code>db.sample</code> using bin-pack strategy in any partition where more than
2 or more files need to be rewritten.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef> [...]
</span></span></code></pre></div><p>Rewrite the data files in table
<code>db.sample</code> and select the files that may contain data matching the
filter (id = 3 and name = “foo”) to be rewritten.</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<sp [...]
-</span></span></code></pre></div><h3
id=rewrite_manifests><code>rewrite_manifests</code></h3><p>Rewrite manifests
for a table to optimize scan planning.</p><p>Data files in manifests are sorted
by fields in the partition spec. This procedure runs in parallel using a Spark
job.</p><p>See the <a
href=../../../javadoc/1.2.1/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code>
Javadoc</a>
+</span></span></code></pre></div><h3
id=rewrite_manifests><code>rewrite_manifests</code></h3><p>Rewrite manifests
for a table to optimize scan planning.</p><p>Data files in manifests are sorted
by fields in the partition spec. This procedure runs in parallel using a Spark
job.</p><p>See the <a
href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code>
Javadoc</a>
to see more configuration options.</p><div class=info>This procedure
invalidates all cached Spark plans that reference the affected table.</div><h4
id=usage-8>Usage</h4><table><thead><tr><th>Argument
Name</th><th>Required?</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>table</code></td><td>✔️</td><td>string</td><td>Name
of the table to
update</td></tr><tr><td><code>use_caching</code></td><td>️</td><td>boolean</td><td>Use
Spark caching during operation (defaults to [...]
</span></span></code></pre></div><p>Rewrite the manifests in table
<code>db.sample</code> and disable the use of Spark caching. This could be done
to avoid memory issues on executors.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span
style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef>sy [...]
-</span></span></code></pre></div><h3
id=rewrite_position_delete_files><code>rewrite_position_delete_files</code></h3><p>Iceberg
can rewrite position delete files, which serves two purposes:</p><ul><li>Minor
Compaction: Compact small position delete files into larger ones. This reduces
the size of metadata stored in manifest files and overhead of opening small
delete files.</li><li>Remove Dangling Deletes: Filter out position delete
records that refer to data files that are no longer live [...]
+</span></span></code></pre></div><h3
id=rewrite_position_delete_files><code>rewrite_position_delete_files</code></h3><p>Iceberg
can rewrite position delete files, which serves two purposes:</p><ul><li>Minor
Compaction: Compact small position delete files into larger ones. This reduces
the size of metadata stored in manifest files and overhead of opening small
delete files.</li><li>Remove Dangling Deletes: Filter out position delete
records that refer to data files that are no longer live [...]
for list of all the supported options for this procedure.</p><p>Dangling
deletes are always filtered out during rewriting.</p><h4
id=output-8>Output</h4><table><thead><tr><th>Output
Name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>rewritten_delete_files_count</code></td><td>int</td><td>Number
of delete files which were removed by this
command</td></tr><tr><td><code>added_delete_files_count</code></td><td>int</td><td>Number
of delete files which were added by th [...]
</span></span></code></pre></div><p>Rewrite all position delete files in table
<code>db.sample</code>, writing new files <code>target-file-size-bytes</code>.
Dangling deletes are removed from rewritten delete files.</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</sp [...]
</span></span></code></pre></div><p>Rewrite position delete files in table
<code>db.sample</code>. This selects position delete files in partitions where
2 or more position delete files need to be rewritten based on size criteria.
Dangling deletes are removed from rewritten delete files.</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span s [...]
diff --git a/docs/latest/spark-queries/index.html
b/docs/latest/spark-queries/index.html
index b75c62d3..c6f32a50 100644
--- a/docs/latest/spark-queries/index.html
+++ b/docs/latest/spark-queries/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-structured-streaming/index.html
b/docs/latest/spark-structured-streaming/index.html
index 271bbb00..078986dd 100644
--- a/docs/latest/spark-structured-streaming/index.html
+++ b/docs/latest/spark-structured-streaming/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-writes/index.html
b/docs/latest/spark-writes/index.html
index bce6a60e..72b49026 100644
--- a/docs/latest/spark-writes/index.html
+++ b/docs/latest/spark-writes/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/table-migration/index.html
b/docs/latest/table-migration/index.html
index 72ae1f02..3b04430a 100644
--- a/docs/latest/table-migration/index.html
+++ b/docs/latest/table-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.2.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.0</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul
class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a
href=../configuration/>Configuration</a></li><li><a
href=../evolution/>Evolution</a></li><li><a
href=../maintenance/>Maintenance</a></li><li><a
href=../partitioning/>Partitioning</a></li><li><a
href=../performance/>Performance</a></li><li><a
href=../reliability/>Reliability</a></li><li><a
href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
