This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new f73e6bda Publish built docs triggered by
2741ae76bf5a7e2132800523c772f105868f472c
f73e6bda is described below
commit f73e6bda5fae0e37c8efc3abc0dcd217cd2665ba
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri May 3 19:24:31 2024 +0000
Publish built docs triggered by 2741ae76bf5a7e2132800523c772f105868f472c
---
...bility.md.txt => compatibility-template.md.txt} | 18 +-
_sources/user-guide/compatibility.md.txt | 159 +++++-
_sources/user-guide/configs.md.txt | 2 +-
objects.inv | Bin 530 -> 537 bytes
searchindex.js | 2 +-
...patibility.html => compatibility-template.html} | 53 +-
user-guide/compatibility.html | 607 ++++++++++++++++++++-
user-guide/configs.html | 4 +-
8 files changed, 745 insertions(+), 100 deletions(-)
diff --git a/_sources/user-guide/compatibility.md.txt
b/_sources/user-guide/compatibility-template.md.txt
similarity index 64%
copy from _sources/user-guide/compatibility.md.txt
copy to _sources/user-guide/compatibility-template.md.txt
index b4b4c92e..deaca2d2 100644
--- a/_sources/user-guide/compatibility.md.txt
+++ b/_sources/user-guide/compatibility-template.md.txt
@@ -34,13 +34,17 @@ There is an
[epic](https://github.com/apache/datafusion-comet/issues/313) where
## Cast
-Comet currently delegates to Apache DataFusion for most cast operations, and
this means that the behavior is not
-guaranteed to be consistent with Spark.
+Cast operations in Comet fall into three levels of support:
-There is an [epic](https://github.com/apache/datafusion-comet/issues/286)
where we are tracking the work to implement Spark-compatible cast expressions.
+- **Compatible**: The results match Apache Spark
+- **Incompatible**: The results may match Apache Spark for some inputs, but
there are known issues where some inputs
+ will result in incorrect results or exceptions. The query stage will fall
back to Spark by default. Setting
+ `spark.comet.cast.allowIncompatible=true` will allow all incompatible casts
to run natively in Comet, but this is not
+ recommended for production use.
+- **Unsupported**: Comet does not provide a native version of this cast
expression and the query stage will fall back to
+ Spark.
-### Cast from String to Timestamp
+The following table shows the current cast operations supported by Comet. Any
cast that does not appear in this
+table (such as those involving complex types and timestamp_ntz, for example)
are not supported by Comet.
-Casting from String to Timestamp is disabled by default due to
incompatibilities with Spark, including timezone
-issues, and can be enabled by setting
`spark.comet.castStringToTimestamp=true`. See the
-[tracking issue](https://github.com/apache/datafusion-comet/issues/328) for
more information.
+<!--CAST_TABLE-->
diff --git a/_sources/user-guide/compatibility.md.txt
b/_sources/user-guide/compatibility.md.txt
index b4b4c92e..9a2478d3 100644
--- a/_sources/user-guide/compatibility.md.txt
+++ b/_sources/user-guide/compatibility.md.txt
@@ -1,20 +1,20 @@
<!---
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
-->
# Compatibility Guide
@@ -34,13 +34,126 @@ There is an
[epic](https://github.com/apache/datafusion-comet/issues/313) where
## Cast
-Comet currently delegates to Apache DataFusion for most cast operations, and
this means that the behavior is not
-guaranteed to be consistent with Spark.
+Cast operations in Comet fall into three levels of support:
-There is an [epic](https://github.com/apache/datafusion-comet/issues/286)
where we are tracking the work to implement Spark-compatible cast expressions.
+- **Compatible**: The results match Apache Spark
+- **Incompatible**: The results may match Apache Spark for some inputs, but
there are known issues where some inputs
+ will result in incorrect results or exceptions. The query stage will fall
back to Spark by default. Setting
+ `spark.comet.cast.allowIncompatible=true` will allow all incompatible casts
to run natively in Comet, but this is not
+ recommended for production use.
+- **Unsupported**: Comet does not provide a native version of this cast
expression and the query stage will fall back to
+ Spark.
-### Cast from String to Timestamp
+The following table shows the current cast operations supported by Comet. Any
cast that does not appear in this
+table (such as those involving complex types and timestamp_ntz, for example)
are not supported by Comet.
-Casting from String to Timestamp is disabled by default due to
incompatibilities with Spark, including timezone
-issues, and can be enabled by setting
`spark.comet.castStringToTimestamp=true`. See the
-[tracking issue](https://github.com/apache/datafusion-comet/issues/328) for
more information.
+| From Type | To Type | Compatible? | Notes |
+| --------- | --------- | ------------ | ----------------------------------- |
+| boolean | byte | Compatible | |
+| boolean | short | Compatible | |
+| boolean | integer | Compatible | |
+| boolean | long | Compatible | |
+| boolean | float | Compatible | |
+| boolean | double | Compatible | |
+| boolean | decimal | Unsupported | |
+| boolean | string | Compatible | |
+| boolean | timestamp | Unsupported | |
+| byte | boolean | Compatible | |
+| byte | short | Compatible | |
+| byte | integer | Compatible | |
+| byte | long | Compatible | |
+| byte | float | Compatible | |
+| byte | double | Compatible | |
+| byte | decimal | Compatible | |
+| byte | string | Compatible | |
+| byte | binary | Unsupported | |
+| byte | timestamp | Unsupported | |
+| short | boolean | Compatible | |
+| short | byte | Compatible | |
+| short | integer | Compatible | |
+| short | long | Compatible | |
+| short | float | Compatible | |
+| short | double | Compatible | |
+| short | decimal | Compatible | |
+| short | string | Compatible | |
+| short | binary | Unsupported | |
+| short | timestamp | Unsupported | |
+| integer | boolean | Compatible | |
+| integer | byte | Compatible | |
+| integer | short | Compatible | |
+| integer | long | Compatible | |
+| integer | float | Compatible | |
+| integer | double | Compatible | |
+| integer | decimal | Compatible | |
+| integer | string | Compatible | |
+| integer | binary | Unsupported | |
+| integer | timestamp | Unsupported | |
+| long | boolean | Compatible | |
+| long | byte | Compatible | |
+| long | short | Compatible | |
+| long | integer | Compatible | |
+| long | float | Compatible | |
+| long | double | Compatible | |
+| long | decimal | Compatible | |
+| long | string | Compatible | |
+| long | binary | Unsupported | |
+| long | timestamp | Unsupported | |
+| float | boolean | Compatible | |
+| float | byte | Unsupported | |
+| float | short | Unsupported | |
+| float | integer | Unsupported | |
+| float | long | Unsupported | |
+| float | double | Compatible | |
+| float | decimal | Unsupported | |
+| float | string | Incompatible | |
+| float | timestamp | Unsupported | |
+| double | boolean | Compatible | |
+| double | byte | Unsupported | |
+| double | short | Unsupported | |
+| double | integer | Unsupported | |
+| double | long | Unsupported | |
+| double | float | Compatible | |
+| double | decimal | Incompatible | |
+| double | string | Incompatible | |
+| double | timestamp | Unsupported | |
+| decimal | boolean | Unsupported | |
+| decimal | byte | Unsupported | |
+| decimal | short | Unsupported | |
+| decimal | integer | Unsupported | |
+| decimal | long | Unsupported | |
+| decimal | float | Compatible | |
+| decimal | double | Compatible | |
+| decimal | string | Unsupported | |
+| decimal | timestamp | Unsupported | |
+| string | boolean | Compatible | |
+| string | byte | Compatible | |
+| string | short | Compatible | |
+| string | integer | Compatible | |
+| string | long | Compatible | |
+| string | float | Unsupported | |
+| string | double | Unsupported | |
+| string | decimal | Unsupported | |
+| string | binary | Compatible | |
+| string | date | Unsupported | |
+| string | timestamp | Incompatible | Not all valid formats are supported |
+| binary | string | Incompatible | |
+| date | boolean | Unsupported | |
+| date | byte | Unsupported | |
+| date | short | Unsupported | |
+| date | integer | Unsupported | |
+| date | long | Unsupported | |
+| date | float | Unsupported | |
+| date | double | Unsupported | |
+| date | decimal | Unsupported | |
+| date | string | Compatible | |
+| date | timestamp | Unsupported | |
+| timestamp | boolean | Unsupported | |
+| timestamp | byte | Unsupported | |
+| timestamp | short | Unsupported | |
+| timestamp | integer | Unsupported | |
+| timestamp | long | Compatible | |
+| timestamp | float | Unsupported | |
+| timestamp | double | Unsupported | |
+| timestamp | decimal | Unsupported | |
+| timestamp | string | Compatible | |
+| timestamp | date | Compatible | |
diff --git a/_sources/user-guide/configs.md.txt
b/_sources/user-guide/configs.md.txt
index 3a16cd47..02ecbd69 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -25,7 +25,7 @@ Comet provides the following configuration settings.
|--------|-------------|---------------|
| spark.comet.ansi.enabled | Comet does not respect ANSI mode in most cases
and by default will not accelerate queries when ansi mode is enabled. Enable
this setting to test Comet's experimental support for ANSI mode. This should
not be used in production. | false |
| spark.comet.batchSize | The columnar batch size, i.e., the maximum number of
rows that a batch can contain. | 8192 |
-| spark.comet.cast.stringToTimestamp | Comet is not currently fully compatible
with Spark when casting from String to Timestamp. | false |
+| spark.comet.cast.allowIncompatible | Comet is not currently fully compatible
with Spark for all cast operations. Set this config to true to allow them
anyway. See compatibility guide for more information. | false |
| spark.comet.columnar.shuffle.async.enabled | Whether to enable asynchronous
shuffle for Arrow-based shuffle. By default, this config is false. | false |
| spark.comet.columnar.shuffle.async.max.thread.num | Maximum number of
threads on an executor used for Comet async columnar shuffle. By default, this
config is 100. This is the upper bound of total number of shuffle threads per
executor. In other words, if the number of cores * the number of shuffle
threads per task `spark.comet.columnar.shuffle.async.thread.num` is larger than
this config. Comet will use this config as the number of shuffle threads per
executor instead. | 100 |
| spark.comet.columnar.shuffle.async.thread.num | Number of threads used for
Comet async columnar shuffle per shuffle task. By default, this config is 3.
Note that more threads means more memory requirement to buffer shuffle data
before flushing to disk. Also, more threads may not always improve performance,
and should be set based on the number of cores available. | 3 |
diff --git a/objects.inv b/objects.inv
index 87342e48..d0092f3e 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/searchindex.js b/searchindex.js
index b911f68b..0676bfbd 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"ANSI mode": [[5, "ansi-mode"]], "ASF Links":
[[4, null]], "Additional Info": [[1, "additional-info"]], "After your debugging
is done,": [[1, "after-your-debugging-is-done"]], "Apache DataFusion Comet":
[[4, "apache-datafusion-comet"]], "Architecture": [[12, "architecture"]],
"Asking for Help": [[0, "asking-for-help"]], "Benchmark": [[2, "benchmark"]],
"Build & Test": [[2, "build-test"]], "Building From Source": [[10,
"building-from-source"]], "CLion": [[2, [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"ANSI mode": [[5, "ansi-mode"], [6,
"ansi-mode"]], "ASF Links": [[4, null]], "Additional Info": [[1,
"additional-info"]], "After your debugging is done,": [[1,
"after-your-debugging-is-done"]], "Apache DataFusion Comet": [[4,
"apache-datafusion-comet"]], "Architecture": [[13, "architecture"]], "Asking
for Help": [[0, "asking-for-help"]], "Benchmark": [[2, "benchmark"]], "Build &
Test": [[2, "build-test"]], "Building From Source": [[11, "building-from-source
[...]
\ No newline at end of file
diff --git a/user-guide/compatibility.html
b/user-guide/compatibility-template.html
similarity index 83%
copy from user-guide/compatibility.html
copy to user-guide/compatibility-template.html
index 09772510..ad0d8d6b 100644
--- a/user-guide/compatibility.html
+++ b/user-guide/compatibility-template.html
@@ -53,8 +53,6 @@ under the License.
<script async="true" defer="true"
src="https://buttons.github.io/buttons.js";></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Contributing to Apache DataFusion Comet"
href="../contributor-guide/contributing.html" />
- <link rel="prev" title="Comet Configuration Settings" href="configs.html"
/>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -107,7 +105,7 @@ under the License.
User Guide
</span>
</p>
-<ul class="current nav bd-sidenav">
+<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="overview.html">
Comet Overview
@@ -138,8 +136,8 @@ under the License.
Configuration Settings
</a>
</li>
- <li class="toctree-l1 current active">
- <a class="current reference internal" href="#">
+ <li class="toctree-l1">
+ <a class="reference internal" href="compatibility.html">
Compatibility Guide
</a>
</li>
@@ -250,13 +248,6 @@ under the License.
<a class="reference internal nav-link" href="#cast">
Cast
</a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link"
href="#cast-from-string-to-timestamp">
- Cast from String to Timestamp
- </a>
- </li>
- </ul>
</li>
</ul>
@@ -267,7 +258,7 @@ under the License.
<div class="tocsection editthispage">
- <a
href="https://github.com/apache/datafusion-comet/edit/main/docs/source/user-guide/compatibility.md";>
+ <a
href="https://github.com/apache/datafusion-comet/edit/main/docs/source/user-guide/compatibility-template.md";>
<i class="fas fa-pencil-alt"></i> Edit this page
</a>
</div>
@@ -318,15 +309,19 @@ be used in production.</p>
</section>
<section id="cast">
<h2>Cast<a class="headerlink" href="#cast" title="Link to this
heading">¶</a></h2>
-<p>Comet currently delegates to Apache DataFusion for most cast operations,
and this means that the behavior is not
-guaranteed to be consistent with Spark.</p>
-<p>There is an <a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/286";>epic</a> where we
are tracking the work to implement Spark-compatible cast expressions.</p>
-<section id="cast-from-string-to-timestamp">
-<h3>Cast from String to Timestamp<a class="headerlink"
href="#cast-from-string-to-timestamp" title="Link to this heading">¶</a></h3>
-<p>Casting from String to Timestamp is disabled by default due to
incompatibilities with Spark, including timezone
-issues, and can be enabled by setting <code class="docutils literal
notranslate"><span
class="pre">spark.comet.castStringToTimestamp=true</span></code>. See the
-<a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/328";>tracking issue</a>
for more information.</p>
-</section>
+<p>Cast operations in Comet fall into three levels of support:</p>
+<ul class="simple">
+<li><p><strong>Compatible</strong>: The results match Apache Spark</p></li>
+<li><p><strong>Incompatible</strong>: The results may match Apache Spark for
some inputs, but there are known issues where some inputs
+will result in incorrect results or exceptions. The query stage will fall back
to Spark by default. Setting
+<code class="docutils literal notranslate"><span
class="pre">spark.comet.cast.allowIncompatible=true</span></code> will allow
all incompatible casts to run natively in Comet, but this is not
+recommended for production use.</p></li>
+<li><p><strong>Unsupported</strong>: Comet does not provide a native version
of this cast expression and the query stage will fall back to
+Spark.</p></li>
+</ul>
+<p>The following table shows the current cast operations supported by Comet.
Any cast that does not appear in this
+table (such as those involving complex types and timestamp_ntz, for example)
are not supported by Comet.</p>
+<!--CAST_TABLE-->
</section>
</section>
@@ -336,20 +331,6 @@ issues, and can be enabled by setting <code
class="docutils literal notranslate"
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="configs.html" title="previous
page">
- <i class="fas fa-angle-left"></i>
- <div class="prev-next-info">
- <p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Comet Configuration Settings</p>
- </div>
- </a>
- <a class='right-next' id="next-link"
href="../contributor-guide/contributing.html" title="next page">
- <div class="prev-next-info">
- <p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Contributing to Apache DataFusion Comet</p>
- </div>
- <i class="fas fa-angle-right"></i>
- </a>
</div>
</main>
diff --git a/user-guide/compatibility.html b/user-guide/compatibility.html
index 09772510..7466a2c4 100644
--- a/user-guide/compatibility.html
+++ b/user-guide/compatibility.html
@@ -250,13 +250,6 @@ under the License.
<a class="reference internal nav-link" href="#cast">
Cast
</a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link"
href="#cast-from-string-to-timestamp">
- Cast from String to Timestamp
- </a>
- </li>
- </ul>
</li>
</ul>
@@ -287,22 +280,22 @@ under the License.
<div>
<!---
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
+http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
-->
<section id="compatibility-guide">
<h1>Compatibility Guide<a class="headerlink" href="#compatibility-guide"
title="Link to this heading">¶</a></h1>
@@ -318,15 +311,569 @@ be used in production.</p>
</section>
<section id="cast">
<h2>Cast<a class="headerlink" href="#cast" title="Link to this
heading">¶</a></h2>
-<p>Comet currently delegates to Apache DataFusion for most cast operations,
and this means that the behavior is not
-guaranteed to be consistent with Spark.</p>
-<p>There is an <a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/286";>epic</a> where we
are tracking the work to implement Spark-compatible cast expressions.</p>
-<section id="cast-from-string-to-timestamp">
-<h3>Cast from String to Timestamp<a class="headerlink"
href="#cast-from-string-to-timestamp" title="Link to this heading">¶</a></h3>
-<p>Casting from String to Timestamp is disabled by default due to
incompatibilities with Spark, including timezone
-issues, and can be enabled by setting <code class="docutils literal
notranslate"><span
class="pre">spark.comet.castStringToTimestamp=true</span></code>. See the
-<a class="reference external"
href="https://github.com/apache/datafusion-comet/issues/328";>tracking issue</a>
for more information.</p>
-</section>
+<p>Cast operations in Comet fall into three levels of support:</p>
+<ul class="simple">
+<li><p><strong>Compatible</strong>: The results match Apache Spark</p></li>
+<li><p><strong>Incompatible</strong>: The results may match Apache Spark for
some inputs, but there are known issues where some inputs
+will result in incorrect results or exceptions. The query stage will fall back
to Spark by default. Setting
+<code class="docutils literal notranslate"><span
class="pre">spark.comet.cast.allowIncompatible=true</span></code> will allow
all incompatible casts to run natively in Comet, but this is not
+recommended for production use.</p></li>
+<li><p><strong>Unsupported</strong>: Comet does not provide a native version
of this cast expression and the query stage will fall back to
+Spark.</p></li>
+</ul>
+<p>The following table shows the current cast operations supported by Comet.
Any cast that does not appear in this
+table (such as those involving complex types and timestamp_ntz, for example)
are not supported by Comet.</p>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>From Type</p></th>
+<th class="head"><p>To Type</p></th>
+<th class="head"><p>Compatible?</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p>boolean</p></td>
+<td><p>byte</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>boolean</p></td>
+<td><p>short</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>boolean</p></td>
+<td><p>integer</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>boolean</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>boolean</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>boolean</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>boolean</p></td>
+<td><p>decimal</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>boolean</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>boolean</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>byte</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>byte</p></td>
+<td><p>short</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>byte</p></td>
+<td><p>integer</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>byte</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>byte</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>byte</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>byte</p></td>
+<td><p>decimal</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>byte</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>byte</p></td>
+<td><p>binary</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>byte</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>short</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>short</p></td>
+<td><p>byte</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>short</p></td>
+<td><p>integer</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>short</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>short</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>short</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>short</p></td>
+<td><p>decimal</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>short</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>short</p></td>
+<td><p>binary</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>short</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>integer</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>integer</p></td>
+<td><p>byte</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>integer</p></td>
+<td><p>short</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>integer</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>integer</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>integer</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>integer</p></td>
+<td><p>decimal</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>integer</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>integer</p></td>
+<td><p>binary</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>integer</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>long</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>long</p></td>
+<td><p>byte</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>long</p></td>
+<td><p>short</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>long</p></td>
+<td><p>integer</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>long</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>long</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>long</p></td>
+<td><p>decimal</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>long</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>long</p></td>
+<td><p>binary</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>long</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>float</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>float</p></td>
+<td><p>byte</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>float</p></td>
+<td><p>short</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>float</p></td>
+<td><p>integer</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>float</p></td>
+<td><p>long</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>float</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>float</p></td>
+<td><p>decimal</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>float</p></td>
+<td><p>string</p></td>
+<td><p>Incompatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>float</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>double</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>double</p></td>
+<td><p>byte</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>double</p></td>
+<td><p>short</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>double</p></td>
+<td><p>integer</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>double</p></td>
+<td><p>long</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>double</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>double</p></td>
+<td><p>decimal</p></td>
+<td><p>Incompatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>double</p></td>
+<td><p>string</p></td>
+<td><p>Incompatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>double</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>decimal</p></td>
+<td><p>boolean</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>decimal</p></td>
+<td><p>byte</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>decimal</p></td>
+<td><p>short</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>decimal</p></td>
+<td><p>integer</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>decimal</p></td>
+<td><p>long</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>decimal</p></td>
+<td><p>float</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>decimal</p></td>
+<td><p>double</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>decimal</p></td>
+<td><p>string</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>decimal</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>boolean</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>string</p></td>
+<td><p>byte</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>short</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>string</p></td>
+<td><p>integer</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>string</p></td>
+<td><p>float</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>double</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>string</p></td>
+<td><p>decimal</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>binary</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>string</p></td>
+<td><p>date</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>string</p></td>
+<td><p>timestamp</p></td>
+<td><p>Incompatible</p></td>
+<td><p>Not all valid formats are supported</p></td>
+</tr>
+<tr class="row-odd"><td><p>binary</p></td>
+<td><p>string</p></td>
+<td><p>Incompatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>date</p></td>
+<td><p>boolean</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>date</p></td>
+<td><p>byte</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>date</p></td>
+<td><p>short</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>date</p></td>
+<td><p>integer</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>date</p></td>
+<td><p>long</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>date</p></td>
+<td><p>float</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>date</p></td>
+<td><p>double</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>date</p></td>
+<td><p>decimal</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>date</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>date</p></td>
+<td><p>timestamp</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>timestamp</p></td>
+<td><p>boolean</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>timestamp</p></td>
+<td><p>byte</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>timestamp</p></td>
+<td><p>short</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>timestamp</p></td>
+<td><p>integer</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>timestamp</p></td>
+<td><p>long</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>timestamp</p></td>
+<td><p>float</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>timestamp</p></td>
+<td><p>double</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>timestamp</p></td>
+<td><p>decimal</p></td>
+<td><p>Unsupported</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p>timestamp</p></td>
+<td><p>string</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p>timestamp</p></td>
+<td><p>date</p></td>
+<td><p>Compatible</p></td>
+<td><p></p></td>
+</tr>
+</tbody>
+</table>
</section>
</section>
diff --git a/user-guide/configs.html b/user-guide/configs.html
index b1c6e825..0344865a 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -301,8 +301,8 @@ under the License.
<td><p>The columnar batch size, i.e., the maximum number of rows that a batch
can contain.</p></td>
<td><p>8192</p></td>
</tr>
-<tr class="row-even"><td><p>spark.comet.cast.stringToTimestamp</p></td>
-<td><p>Comet is not currently fully compatible with Spark when casting from
String to Timestamp.</p></td>
+<tr class="row-even"><td><p>spark.comet.cast.allowIncompatible</p></td>
+<td><p>Comet is not currently fully compatible with Spark for all cast
operations. Set this config to true to allow them anyway. See compatibility
guide for more information.</p></td>
<td><p>false</p></td>
</tr>
<tr class="row-odd"><td><p>spark.comet.columnar.shuffle.async.enabled</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
