This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new bedf64e0 deploy: 02496a65c9a23ec862563d4f34e6b5b7390c5bbd
bedf64e0 is described below
commit bedf64e0ff51e7b2bc3373936a3dd23fcbcaf6b5
Author: nastra <[email protected]>
AuthorDate: Fri Oct 6 06:19:33 2023 +0000
deploy: 02496a65c9a23ec862563d4f34e6b5b7390c5bbd
---
contribute/index.html | 10 +++++-----
getting-started/index.html | 20 +-------------------
index.html | 2 +-
landingpagesearch.json | 2 +-
spec/index.html | 2 +-
view-spec/index.html | 46 ++++++++++++++++++++++------------------------
6 files changed, 31 insertions(+), 51 deletions(-)
diff --git a/contribute/index.html b/contribute/index.html
index 977e5e80..f5597f64 100644
--- a/contribute/index.html
+++ b/contribute/index.html
@@ -109,13 +109,13 @@ Please refer to the <a
href=https://assertj.github.io/doc/#assertj-core-assertio
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertThat<span
style=color:#f92672>(</span>catalog<span style=color:#f92672>.</span><span
style=color:#a6e22e>listNamespaces</span><span
style=color:#f92672>()).</span><span
style=color:#a6e22e>containsAll</span><span
style=color:#f92672>(</span>expected<span style=color:#f92672>);</span>
</span></span></code></pre></div><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span><span
style=color:#75715e>// ok
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertNotNull<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>);</span>
-</span></span><span style=display:flex><span>assertEquals<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>.</span><span style=color:#a6e22e>size</span><span
style=color:#f92672>(),</span> 4<span style=color:#f92672>);</span>
+</span></span><span style=display:flex><span>assertEquals<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>.</span><span style=color:#a6e22e>size</span><span
style=color:#f92672>(),</span> <span style=color:#ae81ff>4</span><span
style=color:#f92672>);</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>//
better: will show the content of metadataFileLocations if check fails
-</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertThat<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>).</span><span style=color:#a6e22e>isNotNull</span><span
style=color:#f92672>().</span><span style=color:#a6e22e>hasSize</span><span
style=color:#f92672>(</span>4<span style=color:#f92672>);</span>
+</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertThat<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>).</span><span style=color:#a6e22e>isNotNull</span><span
style=color:#f92672>().</span><span style=color:#a6e22e>hasSize</span><span
style=color:#f92672>(</span><span style=color:#ae81ff>4</span><span
style=color:#f92672>);</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>// or
-</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertThat<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>).</span><span style=color:#a6e22e>isNotNull</span><span
style=color:#f92672>().</span><span
style=color:#a6e22e>hasSameSizeAs</span><span
style=color:#f92672>(</span>expected<span style=color:#f92672>).</span><span
style=color:#a6e22e>hasSize</span><span style=color:#f92672>(</span>4<span
style=color:#f92672>);</span>
+</span></span></span><span style=display:flex><span><span
style=color:#75715e></span>assertThat<span
style=color:#f92672>(</span>metadataFileLocations<span
style=color:#f92672>).</span><span style=color:#a6e22e>isNotNull</span><span
style=color:#f92672>().</span><span
style=color:#a6e22e>hasSameSizeAs</span><span
style=color:#f92672>(</span>expected<span style=color:#f92672>).</span><span
style=color:#a6e22e>hasSize</span><span style=color:#f92672>(</span><span
style=color:#ae81ff>4</spa [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span style=display:flex><span><span
style=color:#75715e>// bad
</span></span></span><span style=display:flex><span><span
style=color:#75715e></span><span style=color:#66d9ef>try</span> <span
style=color:#f92672>{</span>
</span></span><span style=display:flex><span> catalog<span
style=color:#f92672>.</span><span
style=color:#a6e22e>createNamespace</span><span
style=color:#f92672>(</span>deniedNamespace<span style=color:#f92672>);</span>
@@ -130,12 +130,12 @@ Please refer to the <a
href=https://assertj.github.io/doc/#assertj-core-assertio
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>isInstanceOf</span><span
style=color:#f92672>(</span>AccessDeniedException<span
style=color:#f92672>.</span><span style=color:#a6e22e>class</span><span
style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>hasMessage</span><span
style=color:#f92672>(</span><span style=color:#e6db74>"User
'testUser' has no permission to create namespace"</span><span
style=color:#f92672>);</span>
</span></span></code></pre></div><p>Checks on exceptions should always make
sure to assert that a particular exception message has occurred.</p><h3
id=awaitility>Awaitility</h3><p>Avoid using <code>Thread.sleep()</code> in
tests as it leads to long test durations and flaky behavior if a condition
takes slightly longer than expected.</p><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data- [...]
-</span></span><span style=display:flex><span>Thread<span
style=color:#f92672>.</span><span style=color:#a6e22e>sleep</span><span
style=color:#f92672>(</span>3000L<span style=color:#f92672>);</span>
+</span></span><span style=display:flex><span>Thread<span
style=color:#f92672>.</span><span style=color:#a6e22e>sleep</span><span
style=color:#f92672>(</span><span style=color:#ae81ff>3000L</span><span
style=color:#f92672>);</span>
</span></span><span style=display:flex><span>assertThat<span
style=color:#f92672>(</span>tables<span style=color:#f92672>()).</span><span
style=color:#a6e22e>isEmpty</span><span style=color:#f92672>();</span>
</span></span></code></pre></div><p>A better alternative is using <a
href=https://github.com/awaitility/awaitility>Awaitility</a> to make sure
<code>tables()</code> are eventually empty. The below example will run the check
with a default polling interval of <strong>100 millis</strong>:</p><div
class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-java data-lang=java><span
style=display:flex><span>deleteTablesAsync<span style=color:#f92672>();</span>
</span></span><span style=display:flex><span>Awaitility<span
style=color:#f92672>.</span><span style=color:#a6e22e>await</span><span
style=color:#f92672>(</span><span style=color:#e6db74>"Tables were not
deleted"</span><span style=color:#f92672>)</span>
-</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>atMost</span><span
style=color:#f92672>(</span>5<span style=color:#f92672>,</span> TimeUnit<span
style=color:#f92672>.</span><span style=color:#a6e22e>SECONDS</span><span
style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>atMost</span><span
style=color:#f92672>(</span><span style=color:#ae81ff>5</span><span
style=color:#f92672>,</span> TimeUnit<span style=color:#f92672>.</span><span
style=color:#a6e22e>SECONDS</span><span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span
style=color:#f92672>.</span><span style=color:#a6e22e>untilAsserted</span><span
style=color:#f92672>(()</span> <span style=color:#f92672>-></span>
assertThat<span style=color:#f92672>(</span>tables<span
style=color:#f92672>()).</span><span style=color:#a6e22e>isEmpty</span><span
style=color:#f92672>());</span>
</span></span></code></pre></div><p>Please refer to the <a
href=https://github.com/awaitility/awaitility/wiki/Usage>usage guide</a> of <a
href=https://github.com/awaitility/awaitility>Awaitility</a> for more usage
examples.</p><h3 id=junit4--junit5>JUnit4 / JUnit5</h3><p>Iceberg currently
uses a mix of JUnit4 (<code>org.junit</code> imports) and JUnit5
(<code>org.junit.jupiter.api</code> imports) tests. To allow an easier
migration to JUnit5 in the future, new test classes
that are being added to the codebase should be written purely in JUnit5 where
possible.</p><h2 id=running-benchmarks>Running Benchmarks</h2><p>Some
PRs/changesets might require running benchmarks to determine whether they are
affecting the baseline performance. Currently there is
diff --git a/getting-started/index.html b/getting-started/index.html
index b3c6e0d8..52cc0335 100644
--- a/getting-started/index.html
+++ b/getting-started/index.html
@@ -1,19 +1 @@
-<!--
- - Licensed to the Apache Software Foundation (ASF) under one or more
- - contributor license agreements. See the NOTICE file distributed with
- - this work for additional information regarding copyright ownership.
- - The ASF licenses this file to You under the Apache License, Version 2.0
- - (the "License"); you may not use this file except in compliance with
- - the License. You may obtain a copy of the License at
- -
- - http://www.apache.org/licenses/LICENSE-2.0
- -
- - Unless required by applicable law or agreed to in writing, software
- - distributed under the License is distributed on an "AS IS" BASIS,
- - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- - See the License for the specific language governing permissions and
- - limitations under the License.
- -->
-<head>
- <meta http-equiv="Refresh" content="0; url='/docs/latest/getting-started'" />
-</head>
+<!doctype html><html
lang=en-us><head><title>https://iceberg.apache.org/spark-quickstart/</title><link
rel=canonical href=https://iceberg.apache.org/spark-quickstart/><meta
name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh
content="0; url=https://iceberg.apache.org/spark-quickstart/";></head></html>
\ No newline at end of file
diff --git a/index.html b/index.html
index f872a558..9e5a72c5 100644
--- a/index.html
+++ b/index.html
@@ -1,4 +1,4 @@
-<!doctype html><html><head><meta name=generator content="Hugo 0.118.2"><meta
charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta
name=viewport content="width=device-width,initial-scale=1"><meta
name=description content><meta name=author content><title>Apache
Iceberg</title><link href=/css/bootstrap.css rel=stylesheet><link
href=/css/markdown.css rel=stylesheet><link href=/css/katex.min.css
rel=stylesheet><link href=/css/iceberg-theme.css rel=stylesheet><link
href=/font [...]
+<!doctype html><html><head><meta name=generator content="Hugo 0.119.0"><meta
charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta
name=viewport content="width=device-width,initial-scale=1"><meta
name=description content><meta name=author content><title>Apache
Iceberg</title><link href=/css/bootstrap.css rel=stylesheet><link
href=/css/markdown.css rel=stylesheet><link href=/css/katex.min.css
rel=stylesheet><link href=/css/iceberg-theme.css rel=stylesheet><link
href=/font [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span>
<span class=icon-bar></span>
diff --git a/landingpagesearch.json b/landingpagesearch.json
index 1c67faa3..27649031 100644
--- a/landingpagesearch.json
+++ b/landingpagesearch.json
@@ -1 +1 @@
-[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will
get you up and running with an Iceberg and Hive environment, including sample
code to highlight some powerful features. You can learn more about Iceberg’s
Hive runtime by checking out the Hive section.\nDocker Images Creating a Table
Writing Data to a Table Reading Data from a Table Next Steps Docker Images The
fastest way to get started is to use Apache Hive images which provides a
SQL-like interface to create a [...]
\ No newline at end of file
+[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will
get you up and running with an Iceberg and Hive environment, including sample
code to highlight some powerful features. You can learn more about Iceberg’s
Hive runtime by checking out the Hive section.\nDocker Images Creating a Table
Writing Data to a Table Reading Data from a Table Next Steps Docker Images The
fastest way to get started is to use Apache Hive images which provides a
SQL-like interface to create a [...]
\ No newline at end of file
diff --git a/spec/index.html b/spec/index.html
index 8be89c19..8fd1ff12 100644
--- a/spec/index.html
+++ b/spec/index.html
@@ -42,7 +42,7 @@ Hash results are not dependent on decimal scale, which is
part of the type, not
</span></span><span style=display:flex><span> { <span
style=color:#f92672>"field-id"</span>: <span
style=color:#ae81ff>4</span>, <span style=color:#f92672>"names"</span>:
[<span style=color:#e6db74>"latitude"</span>, <span
style=color:#e6db74>"lat"</span>] },
</span></span><span style=display:flex><span> { <span
style=color:#f92672>"field-id"</span>: <span
style=color:#ae81ff>5</span>, <span style=color:#f92672>"names"</span>:
[<span style=color:#e6db74>"longitude"</span>, <span
style=color:#e6db74>"long"</span>] }
</span></span><span style=display:flex><span> ] } ]
-</span></span></code></pre></div><h2
id=appendix-d-single-value-serialization>Appendix D: Single-value
serialization</h2><h3 id=binary-single-value-serialization>Binary single-value
serialization</h3><p>This serialization scheme is for storing single values as
individual binary values in the lower and upper bounds maps of manifest
files.</p><table><thead><tr><th>Type</th><th>Binary
serialization</th></tr></thead><tbody><tr><td><strong><code>boolean</code></strong></td><td><code>0x00</cod
[...]
+</span></span></code></pre></div><h3
id=content-file-data-and-delete-serialization>Content File (Data and Delete)
Serialization</h3><p>Content file (data or delete) is serialized as a JSON
object according to the following table.</p><table><thead><tr><th>Metadata
field</th><th>JSON
representation</th><th>Example</th></tr></thead><tbody><tr><td><strong><code>spec-id</code></strong></td><td><code>JSON
int</code></td><td><code>1</code></td></tr><tr><td><strong><code>content</code></strong><
[...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript
src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/view-spec/index.html b/view-spec/index.html
index 8e52b706..12ad8d12 100644
--- a/view-spec/index.html
+++ b/view-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img
class=top-navbar-logo
src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache
Iceberg</a></div><div><input type=search class=form-control id=search-input
placeholder=Search... maxlength=64 data-hotkeys=s/></div><div
class=versions-dropdown><span>1.3.1</span> <i class="fa
fa-chevron-down"></i><div class=versions-dropdown-content><ul><li
class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>OR</span> <span
style=color:#66d9ef>REPLACE</span> <span style=color:#66d9ef>VIEW</span>
event_agg (
</span></span><span style=display:flex><span> event_count <span
style=color:#66d9ef>COMMENT</span> <span style=color:#e6db74>'Count of
events'</span>,
</span></span><span style=display:flex><span> event_date) <span
style=color:#66d9ef>AS</span>
@@ -11,7 +11,7 @@
</span></span><span style=display:flex><span> <span
style=color:#66d9ef>COUNT</span>(<span style=color:#ae81ff>1</span>), <span
style=color:#66d9ef>CAST</span>(event_ts <span style=color:#66d9ef>AS</span>
DATE)
</span></span><span style=display:flex><span><span
style=color:#66d9ef>FROM</span> events
</span></span><span style=display:flex><span><span
style=color:#66d9ef>GROUP</span> <span style=color:#66d9ef>BY</span> <span
style=color:#ae81ff>2</span>
-</span></span></code></pre></div><p>This create statement would produce the
following <code>sql</code> representation
metadata:</p><table><thead><tr><th>Field
name</th><th>Value</th></tr></thead><tbody><tr><td><code>type</code></td><td><code>"sql"</code></td></tr><tr><td><code>sql</code></td><td><code>"SELECT\n
COUNT(1), CAST(event_ts AS DATE)\nFROM events\nGROUP BY
2"</code></td></tr><tr><td><code>dialect</code></td><td><code>"spark"</code></td></tr><tr><td><code>default-catalog</code><
[...]
+</span></span></code></pre></div><p>This create statement would produce the
following <code>sql</code> representation
metadata:</p><table><thead><tr><th>Field
name</th><th>Value</th></tr></thead><tbody><tr><td><code>type</code></td><td><code>"sql"</code></td></tr><tr><td><code>sql</code></td><td><code>"SELECT\n
COUNT(1), CAST(event_ts AS DATE)\nFROM events\nGROUP BY
2"</code></td></tr><tr><td><code>dialect</code></td><td><code>"spark"</code></td></tr></tbody></table><p>If
a create statem [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0
style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code
class=language-sql data-lang=sql><span style=display:flex><span><span
style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>OR</span> <span
style=color:#66d9ef>REPLACE</span> <span style=color:#66d9ef>VIEW</span>
event_agg (
</span></span><span style=display:flex><span> event_count <span
style=color:#66d9ef>COMMENT</span> <span style=color:#e6db74>'Count of
events'</span>,
</span></span><span style=display:flex><span> event_date)
@@ -23,6 +23,7 @@
</span></span><span style=display:flex><span><span
style=color:#66d9ef>GROUP</span> <span style=color:#66d9ef>BY</span> <span
style=color:#ae81ff>2</span>
</span></span></code></pre></div><p>The metadata JSON file created looks as
follows.</p><p>The path is intentionally similar to the path for Iceberg tables
and uses a <code>metadata</code> directory.</p><pre
tabindex=0><code>s3://bucket/warehouse/default.db/event_agg/metadata/00001-(uuid).metadata.json
</code></pre><pre tabindex=0><code>{
+ "view-uuid": "fa6506c3-7681-40c8-86dc-e36561f83385",
"format-version" : 1,
"location" : "s3://bucket/warehouse/default.db/event_agg",
"current-version-id" : 1,
@@ -33,6 +34,8 @@
"version-id" : 1,
"timestamp-ms" : 1573518431292,
"schema-id" : 1,
+ "default-catalog" : "prod",
+ "default-namespace" : [ "default" ],
"summary" : {
"operation" : "create",
"engine-name" : "Spark",
@@ -41,25 +44,21 @@
"representations" : [ {
"type" : "sql",
"sql" : "SELECT\n COUNT(1), CAST(event_ts AS DATE)\nFROM
events\nGROUP BY 2",
- "dialect" : "spark",
- "default-catalog" : "prod",
- "default-namespace" : [ "default" ],
- "field-aliases" : ["event_count", "event_date"],
- "field-comments" : ["Count of events", null]
+ "dialect" : "spark"
} ]
} ],
- "current-schema-id": 1,
"schemas": [ {
"schema-id": 1,
"type" : "struct",
"fields" : [ {
"id" : 1,
- "name" : "col1",
+ "name" : "event_count",
"required" : false,
- "type" : "int"
+ "type" : "int",
+ "doc" : "Count of events"
}, {
"id" : 2,
- "name" : "col2",
+ "name" : "event_date",
"required" : false,
"type" : "date"
} ]
@@ -80,6 +79,7 @@
</span></span><span style=display:flex><span><span
style=color:#66d9ef>GROUP</span> <span style=color:#66d9ef>BY</span> <span
style=color:#ae81ff>2</span>
</span></span></code></pre></div><p>Updating the view produces a new metadata
file that completely replaces the old:</p><pre
tabindex=0><code>s3://bucket/warehouse/default.db/event_agg/metadata/00002-(uuid).metadata.json
</code></pre><pre tabindex=0><code>{
+ "view-uuid": "fa6506c3-7681-40c8-86dc-e36561f83385",
"format-version" : 1,
"location" : "s3://bucket/warehouse/default.db/event_agg",
"current-version-id" : 1,
@@ -90,6 +90,8 @@
"version-id" : 1,
"timestamp-ms" : 1573518431292,
"schema-id" : 1,
+ "default-catalog" : "prod",
+ "default-namespace" : [ "default" ],
"summary" : {
"operation" : "create",
"engine-name" : "Spark",
@@ -98,15 +100,14 @@
"representations" : [ {
"type" : "sql",
"sql" : "SELECT\n COUNT(1), CAST(event_ts AS DATE)\nFROM
events\nGROUP BY 2",
- "dialect" : "spark",
- "default-catalog" : "prod",
- "default-namespace" : [ "default" ],
- "field-aliases" : ["event_count", "event_date"],
- "field-comments" : ["Count of events", null]
+ "dialect" : "spark"
} ]
}, {
"version-id" : 2,
"timestamp-ms" : 1573518981593,
+ "schema-id" : 1,
+ "default-catalog" : "prod",
+ "default-namespace" : [ "default" ],
"summary" : {
"operation" : "create",
"engine-name" : "Spark",
@@ -115,24 +116,21 @@
"representations" : [ {
"type" : "sql",
"sql" : "SELECT\n COUNT(1), CAST(event_ts AS DATE)\nFROM
prod.default.events\nGROUP BY 2",
- "dialect" : "spark",
- "default-catalog" : "prod",
- "default-namespace" : [ "default" ],
- "field-aliases" : ["event_count", "event_date"]
+ "dialect" : "spark"
} ]
} ],
- "current-schema-id": 1,
"schemas": [ {
"schema-id": 1,
"type" : "struct",
"fields" : [ {
"id" : 1,
- "name" : "col1",
+ "name" : "event_count",
"required" : false,
- "type" : "int"
+ "type" : "int",
+ "doc" : "Count of events"
}, {
"id" : 2,
- "name" : "col2",
+ "name" : "event_date",
"required" : false,
"type" : "date"
} ]
