This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hive-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new b6277116 deploy: 971efd7db933161a7b05aa5624ddf2a0d94dc579
b6277116 is described below
commit b62771160972b880f12db19ce81b42c1bc357457
Author: zabetak <[email protected]>
AuthorDate: Tue Jun 9 07:36:41 2026 +0000
deploy: 971efd7db933161a7b05aa5624ddf2a0d94dc579
---
.../hive-across-multiple-data-centers/index.html | 2 +-
.../hive-metadata-caching-proposal/index.html | 21 ++++++++++++++++-
.../hivereplicationv2development/index.html | 2 +-
development/desingdocs/indexdev/index.html | 2 +-
.../desingdocs/subqueries-in-select/index.html | 2 +-
.../index.html | 26 +++++++++++++++++++++-
.../desingdocs/type-qualifiers-in-hive/index.html | 8 ++++++-
development/gettingstarted-latest/index.html | 2 +-
.../admin/adminmanual-configuration/index.html | 2 +-
.../index.html | 2 +-
.../index.html | 2 +-
.../admin/hive-on-spark-getting-started/index.html | 6 ++---
.../latest/admin/setting-up-hiveserver2/index.html | 2 +-
index.json | 2 +-
14 files changed, 65 insertions(+), 16 deletions(-)
diff --git
a/development/desingdocs/hive-across-multiple-data-centers/index.html
b/development/desingdocs/hive-across-multiple-data-centers/index.html
index 369cfc05..57d7d5f6 100644
--- a/development/desingdocs/hive-across-multiple-data-centers/index.html
+++ b/development/desingdocs/hive-across-multiple-data-centers/index.html
@@ -10,7 +10,7 @@ Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i
+ The query ‘select .. from T11 join T21 .. ’ will be processed in
C2
+ The query ‘select .. from T11 join T31 .. ’ will fail
+ ‘Insert .. T13 select .. from T11 ..’ will be processed in C1
and the T13 will be created in C1
-+ ‘Insert .. T21 select .. from T11 ..’ will be processed in C2,
and T21 will remain in C2</p><p>The same idea can be extended for partitioned
tables.</p><ul><li>The user can also decide to run in a particular
cluster.<ul><li>Use cluster</li></ul></li><li>The system will not make an
attempt to choose the cluster for the user, but only try to figure out if the
query can be run</li></ul><p>in the . If the query can run in this cluster, it
will succeed. Otherwise, it will fail.< [...]
++ ‘Insert .. T21 select .. from T11 ..’ will be processed in C2,
and T21 will remain in C2</p><p>The same idea can be extended for partitioned
tables.</p><ul><li>The user can also decide to run in a particular
cluster.<ul><li>Use cluster <code><ClusterName></code></li></ul></li><li>The
system will not make an attempt to choose the cluster for the user, but only
try to figure out if the query can be run</li></ul><p>in the
<code><clusterName></code>. If the query can run [...]
<i class="fas fa-thumbs-up"></i> Yes
</button>
<button class="btn btn-feedback btn-negative">
diff --git a/development/desingdocs/hive-metadata-caching-proposal/index.html
b/development/desingdocs/hive-metadata-caching-proposal/index.html
index 667ac990..7e912ebf 100644
--- a/development/desingdocs/hive-metadata-caching-proposal/index.html
+++ b/development/desingdocs/hive-metadata-caching-proposal/index.html
@@ -6,7 +6,26 @@
Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--hive-metadata-caching-proposal>Apache Hive : Hive Metadata
Caching Proposal</a><ul><li><a href=#why-metastore-cache>Why Metastore
Cache</a></li><li><a href=#server-side-vs-client-side-cache>Server side vs
client side cache</a></li><li><a href=#cache-consistency>Cache
Consistency</a></li><li><a href=#><img src= [...]
Closed</p><p>). This might have a minor lag due to the event propagation, but
that should be much shorter than the cache eviction.
3. Maintain a unique id for every object in SQL database (eg, modified
timestamp, version id, or md5 signature), which is different every time we
change the object in SQL database. We will check the DB if the object is
changed for every cache access. However, even check the timestamp in SQL
database might take some time if the database latency is high
-4. In addition, we might optionally add a “flush cache” statement in Hive in
case user want to enforce a cache flush. However, this should be an admin
privilege statement and will complicate our security model.</p><p>If the
requirements present, we can also work on implementing a cache consistency
protocol among multiple metastore instances. Such a protocol will need to
replicate changes to all the active metastore before finally committing the
change and responding to a client write/upd [...]
+4. In addition, we might optionally add a “flush cache” statement in Hive in
case user want to enforce a cache flush. However, this should be an admin
privilege statement and will complicate our security model.</p><p>If the
requirements present, we can also work on implementing a cache consistency
protocol among multiple metastore instances. Such a protocol will need to
replicate changes to all the active metastore before finally committing the
change and responding to a client write/upd [...]
+ DomainTranslator.ExtractionResult decomposedPredicate =
DomainTranslator.fromPredicate(
+ metadata,
+ session,
+ deterministicPredicate,
+ types);
+ public static class ExtractionResult
+ {
+ private final TupleDomain<Symbol> tupleDomain;
+ private final Expression remainingExpression;
+ }
+</code></pre><p>tupleDomain is a mapping of column -> range or exact value.
When converting to PS, any range will be converted into wildcard and only exact
value will be considered:</p><pre
tabindex=0><code>HivePartitionManager.getFilteredPartitionNames:
+ for (HiveColumnHandle partitionKey : partitionKeys) {
+ if (domain != null && domain.isNullableSingleValue()) {
+ filter.add(((Slice) value).toStringUtf8());
+ else {
+ filter.add(PARTITION_VALUE_WILDCARD);
+ }
+ }
+</code></pre><p>For example, the expression “state = CA and date between
‘201612’ and ‘201701’ will be broken down to PS (state = CA) and remainder date
between ‘201612’ and ‘201701’. Presto will retrieve the partitions with state =
CA from the PS -> partition name cache and partition object cache, and
evaluates “date between ‘201612’ and ‘201701’ for every partitions returned.
This is a good balance compare to caching partition names for every
expression.</p><h2 id=our-approach>Our Appr [...]
Resolved</p><p>).</p><p>Currently, the size of the metastore cache can be
restricted by a combination of cache whitelist and blacklist patterns
(</p><h2><a
href="https://issues.apache.org/jira/browse/HIVE-18056?src=confmacro";><img
src="https://issues.apache.org/jira/secure/viewavatar?size=xsmall&avatarId=21146&avatarType=issuetype";
alt></a></h2><p>CachedStore: Have a whitelist/blacklist config to allow
selective caching of tables/partitions and allow read while prewarming
Closed</p><p>). Before a table is cached, it is checked against these filters
to decide if it can be cached or not. Similarly, when a table is read, if it
does not pass the above filters, it is read from the database and not the
cache.</p><h3
id=quantitative-study-memory-footprint-and-prewarm-time>Quantitative study:
memory footprint and prewarm time</h3><p>The major concern in this approach is
how much memory the metastore cache will consume and how much latency at
startup time to read [...]
Resolved</p><p>). Both options are discussed already in “Cache Consistency”
section.</p><h3 id=aggregated-statistics>Aggregated Statistics</h3><p>We
already have aggregated stats module in ObjectStore (</p><h2><a
href="https://issues.apache.org/jira/browse/HIVE-10382?src=confmacro";><img
src="https://issues.apache.org/jira/secure/viewavatar?size=xsmall&avatarId=21140&avatarType=issuetype";
alt>HIVE-10382</a></h2><p>Aggregate stats cache for RDBMS based metastore
codepath
diff --git a/development/desingdocs/hivereplicationv2development/index.html
b/development/desingdocs/hivereplicationv2development/index.html
index fcc13ee7..f8cd4f4b 100644
--- a/development/desingdocs/hivereplicationv2development/index.html
+++ b/development/desingdocs/hivereplicationv2development/index.html
@@ -10,7 +10,7 @@ INSERT INTO TABLE blah [PARTITION (p="a") VALUES 15;
</code></pre><p>Now, for each operation that occurs, a monotonically
increasing state-id is provided by DbNotificationListener, so that we have an
ability to order those events by when they occurred. For the sake of
simplicity, let’s say they occurred at states 10,20,30,40 respectively,
in order.</p><p>Now, if there were another thread running “SELECT * from
blah;” from another thread, then depending on when the SELECT command
ran, it would have differing results:</p><o [...]
Event 110: ALTER TABLE tbl DROP PARTITION (p=1);
Event 120: ALTER TABLE tbl ADD PARTITION (p=1) SET LOCATION <location>;
-</code></pre><p>When loading the dump on the destination side (at a much later
point), when the event 100 is replayed, the load task on the destination will
try to pull the files from the (the _files contains the path of ), which may
contain new or different data. To replicate the exact state of the source at
the time event 100 occurred at the source, we do the following:</p><ol><li>When
Event 100 occurs at the source, in the notification event, we store the
checksum of the file(s) in th [...]
+</code></pre><p>When loading the dump on the destination side (at a much later
point), when the event 100 is replayed, the load task on the destination will
try to pull the files from the <code><location></code> (the _files contains
the path of <code><location></code>), which may contain new or different
data. To replicate the exact state of the source at the time event 100 occurred
at the source, we do the following:</p><ol><li>When Event 100 occurs at the
source, in the notificat [...]
<i class="fas fa-thumbs-up"></i> Yes
</button>
<button class="btn btn-feedback btn-negative">
diff --git a/development/desingdocs/indexdev/index.html
b/development/desingdocs/indexdev/index.html
index 9c5c443f..2b27acf1 100644
--- a/development/desingdocs/indexdev/index.html
+++ b/development/desingdocs/indexdev/index.html
@@ -142,7 +142,7 @@ public interface HiveIndexHandler extends Configurable
public abstract class AbstractIndexHandler implements HiveIndexHandler
{
}
-</code></pre><p>For CREATE INDEX, Hive first calls usesIndexTable() on the
handler to determine whether an index table will be created. If this returns
false, the statement fails immediately if the user specified any table storage
options for the index. However, if usesIndexTable() returns true, then Hive
creates a partial table definition for the index table based on the index
definition (such as the covered columns) combined with any table storage
options supplied by the user. Next, Hi [...]
+</code></pre><p>For CREATE INDEX, Hive first calls usesIndexTable() on the
handler to determine whether an index table will be created. If this returns
false, the statement fails immediately if the user specified any table storage
options for the index. However, if usesIndexTable() returns true, then Hive
creates a partial table definition for the index table based on the index
definition (such as the covered columns) combined with any table storage
options supplied by the user. Next, Hi [...]
CREATE INDEX ...
AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler';
</code></pre><p>TBD: algorithm for building the index</p><p>TBD: mechanism for
searching the index</p><p>TBD: validation on base table (can be any managed
table?)</p><p>TBD: validation on index table format (can be any managed table
format?)</p><h2 id=tbd>TBD</h2><ul><li>specs for SHOW/DESCRIBE INDEX (<a
href=http://issues.apache.org/jira/browse/HIVE-1497>HIVE-1497</a>)</li><li>ALTER
INDEX DROP PARTITION?</li><li>ALTER INDEX SET IDXPROPERTIES, change
tableformat, etc</li><li>what happens [...]
diff --git a/development/desingdocs/subqueries-in-select/index.html
b/development/desingdocs/subqueries-in-select/index.html
index 39b6723e..b2152520 100644
--- a/development/desingdocs/subqueries-in-select/index.html
+++ b/development/desingdocs/subqueries-in-select/index.html
@@ -40,7 +40,7 @@ FROM customer
WHERE customer.customer_num = orders.customer_num
) AS total_ship_chg
FROM customer
-</code></pre><ul><li>Subqueries with DISTINCT are not allowed. Since DISTINCT
will be evaluated as GROUP BY , subqueries with DISTINCT are disallowed for
now.</li></ul><h1 id=design>Design</h1><p>Given the assumptions above, the
following kind of subqueries could be used in SELECT. </p><ul><li>Scalar
subqueries, for example: </li></ul><pre tabindex=0><code>SELECT
customer.customer_num,
+</code></pre><ul><li>Subqueries with DISTINCT are not allowed. Since
<code>DISTINCT <expression></code> will be evaluated as <code>GROUP BY
<expression></code>, subqueries with <code>DISTINCT</code> are disallowed
for now.</li></ul><h1 id=design>Design</h1><p>Given the assumptions above, the
following kind of subqueries could be used in SELECT. </p><ul><li>Scalar
subqueries, for example: </li></ul><pre tabindex=0><code>SELECT
customer.customer_num,
(SELECT SUM(ship_charge)
FROM orders
WHERE customer.customer_num = orders.customer_num
diff --git
a/development/desingdocs/support-saml-2-0-authentication-mode/index.html
b/development/desingdocs/support-saml-2-0-authentication-mode/index.html
index 8d26a324..afc18a12 100644
--- a/development/desingdocs/support-saml-2-0-authentication-mode/index.html
+++ b/development/desingdocs/support-saml-2-0-authentication-mode/index.html
@@ -3,7 +3,31 @@
<span class=navbar-toggler-icon></span></button><div class="collapse
navbar-collapse" id=navbarSupportedContent><ul class="navbar-nav me-auto"><li
class=nav-item><a class=nav-link
href=https://hive.apache.org//general/downloads>Releases</a></li><li
class="nav-item dropdown"><a class="nav-link dropdown-toggle" href=/Document
id=docsDropdown role=button data-bs-toggle=dropdown
aria-expanded=false>Documentation</a><ul class=dropdown-menu
aria-labelledby=docsDropdown><li><a class=dropdown-it [...]
<button type=submit class=search-button aria-label="Submit search">
<i class="fas
fa-search"></i></button></div></form></div></div></div></nav></menu></header><div
class=content><div class=docs-container><main class="docs-main
docs-main-full"><article class=docs-content><nav
class=docs-breadcrumb><ol><li><a href=/><i class="fas fa-home"></i>
Home</a></li><li><a href=/docs/>Documentation</a></li><li class=active>Apache
Hive : HIVE-24543: Support SAML 2.0 authentication mode</li></ol></nav><header
class=docs-header><h1 class=docs-title>Apache Hive : HIVE-2 [...]
-Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--hive-24543-support-saml-20-authentication-mode>Apache Hive :
HIVE-24543: Support SAML 2.0 authentication mode</a><ul><li><a
href=#description>Description</a></li><li><a
href=#design>Design</a><ul><li></li></ul></li><li><a
href=#attachments>Attachments:</a></li></ul></li></ul></nav></div><div
class=docs-article [...]
+Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--hive-24543-support-saml-20-authentication-mode>Apache Hive :
HIVE-24543: Support SAML 2.0 authentication mode</a><ul><li><a
href=#description>Description</a></li><li><a
href=#design>Design</a><ul><li></li></ul></li><li><a
href=#attachments>Attachments:</a></li></ul></li></ul></nav></div><div
class=docs-article [...]
+ <name>hive.server2.authentication</name>
+ <value>SAML</value>
+</property>
+</code></pre><p>This configuration will be set to SAML to indicate that the
server will use SAML 2.0 protocol to authenticate the user. </p><pre
tabindex=0><code><property>
+ <name>hive.server2.saml2.idp.metadata</name>
+ <value>path_to_idp_metadata.xml</value>
+</property>
+</code></pre><p>This configuration will provide a path to the IDP metadata xml
file.</p><pre tabindex=0><code><property>
+ <name>hive.server2.saml2.sp.entity.id</name>
+ <value>test_sp_entity_id</value>
+</property>
+</code></pre><p>This configuration should be same the service provider entity
id as configured in the IDP. Some identity providers require this to be same as
the ACS URL.</p><pre tabindex=0><code><property>
+ <name>hive.server2.saml2.group.attribute.name</name>
+ <value>group_attribute_name</value>
+</property>
+</code></pre><p>This configuration will be used to map the SAML attribute in
the response to the groups of the user. This should be configured in the
identity provider as the attribute name for the group information.</p><pre
tabindex=0><code><property>
+ <name>hive.server2.saml2.group.filter</name>
+ <value>comma_separated_group_names</value>
+</property>
+</code></pre><p>This configuration will be used to configure the allowed group
names.</p><pre tabindex=0><code><property>
+ <name>hive.server2.saml2.sp.callback.url</name>
+ <value>callback_url_of_hiveserver2</value>
+</property>
+</code></pre><p>The http URL endpoint where the SAML assertion is posted back
by the IDP. Currently this must be on the same port as HiveServer2’s http
endpoint and must be TLS enabled (https) on secure setups.</p><h5
id=jdbc-driver-changes>JDBC Driver changes</h5><p>On the driver side the
following new jdbc connection parameters will be
defined/allowed.</p><ol><li>New values of browser and token which will be
allowed for the existing connection parameter auth.</li><li>[TBD] In case the
[...]
</code></pre><p>A token based URL will look like:</p><pre
tabindex=0><code>jdbc:hive2://HiveServer2-host:10001/default;transportMode=http;httpPath=cliservice;auth=token;token=<token_string>
</code></pre><p>The Jdbc connection parameters will be passed in over the
connection URL. </p><h5 id=sso-mode-url-validations>SSO mode URL
validations</h5><p>Driver makes sure that the SSO integration is only allowed
when TLS enabled between driver and the server. If the SSL is not enabled, the
driver should error out (or optionally warn the users based on a configuration
override). Currently, both these modes will only be supported in http
mode.</p><h5 id=browser-mode>Browser mode</h5>< [...]
</code></pre><h5 id=token-expiry-and-renewal>Token Expiry and
renewal</h5><p>In the initial version the token returned by the server will be
used for a one-time validation within the default period of 30 seconds (token
will be valid for only 30 seconds) which could be configurable. The token will
be used by the server to set a cookie which will be used for further requests.
However, this is a server side implementation detail which client does not need
to be aware of. When the session ex [...]
diff --git a/development/desingdocs/type-qualifiers-in-hive/index.html
b/development/desingdocs/type-qualifiers-in-hive/index.html
index d07fc88e..9b2be8cd 100644
--- a/development/desingdocs/type-qualifiers-in-hive/index.html
+++ b/development/desingdocs/type-qualifiers-in-hive/index.html
@@ -3,7 +3,13 @@
<span class=navbar-toggler-icon></span></button><div class="collapse
navbar-collapse" id=navbarSupportedContent><ul class="navbar-nav me-auto"><li
class=nav-item><a class=nav-link
href=https://hive.apache.org//general/downloads>Releases</a></li><li
class="nav-item dropdown"><a class="nav-link dropdown-toggle" href=/Document
id=docsDropdown role=button data-bs-toggle=dropdown
aria-expanded=false>Documentation</a><ul class=dropdown-menu
aria-labelledby=docsDropdown><li><a class=dropdown-it [...]
<button type=submit class=search-button aria-label="Submit search">
<i class="fas
fa-search"></i></button></div></form></div></div></div></nav></menu></header><div
class=content><div class=docs-container><main class="docs-main
docs-main-full"><article class=docs-content><nav
class=docs-breadcrumb><ol><li><a href=/><i class="fas fa-home"></i>
Home</a></li><li><a href=/docs/>Documentation</a></li><li class=active>Apache
Hive : Type Qualifiers in Hive</li></ol></nav><header class=docs-header><h1
class=docs-title>Apache Hive : Type Qualifiers in Hive</h1><di [...]
-Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a href=#apache-hive--type-qualifiers-in-hive>Apache
Hive : Type Qualifiers in Hive</a><ul><li><ul><li><a
href=#intro>Intro</a></li><li><a href=#initial-prototype-work>Initial prototype
work</a></li><li><a href=#metastore-changes>MetaStore
Changes</a></li></ul></li></ul></li></ul></nav></div><div
class=docs-article><h1 id=apache-hi [...]
+Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a href=#apache-hive--type-qualifiers-in-hive>Apache
Hive : Type Qualifiers in Hive</a><ul><li><ul><li><a
href=#intro>Intro</a></li><li><a href=#initial-prototype-work>Initial prototype
work</a></li><li><a href=#metastore-changes>MetaStore
Changes</a></li></ul></li></ul></li></ul></nav></div><div
class=docs-article><h1 id=apache-hi [...]
+| CHARACTER_OCTET_LENGTH | bigint(21) unsigned | YES | | NULL | |
+| NUMERIC_PRECISION | bigint(21) unsigned | YES | | NULL | |
+| NUMERIC_SCALE | bigint(21) unsigned | YES | | NULL | |
+| CHARACTER_SET_NAME | varchar(32) | YES | | NULL | |
+| COLLATION_NAME | varchar(32) | YES | | NULL | |
+</code></pre><p>We could add new columns to the COLUMNS_V2 table for any type
qualifiers we are trying to support (initially looks like
CHARACTER_MAXIMUM_LENGTH, NUMERIC_PRECISION, NUMERIC_SCALE). Advantages to this
would be that it is easier to query these parameters than the first approach,
though types with no parameters would still have these columns (set to
null).</p><h4 id=new-table-with-type-qualifiers-in-megastore>New table with
type qualifiers in megastore</h4><p>Rather than hav [...]
<i class="fas fa-thumbs-up"></i> Yes
</button>
<button class="btn btn-feedback btn-negative">
diff --git a/development/gettingstarted-latest/index.html
b/development/gettingstarted-latest/index.html
index 041be1c1..e61cc2d1 100644
--- a/development/gettingstarted-latest/index.html
+++ b/development/gettingstarted-latest/index.html
@@ -22,7 +22,7 @@ Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i
examples/ (sample input and query files)
hcatalog / (hcatalog installation)
scripts / (upgrade scripts for hive-metastore)
-</code></pre><p>Here, {version} refers to the current Hive version.</p><p>If
building Hive source using Maven (mvn), we will refer to the directory
“/packaging/target/apache-hive-{version}-SNAPSHOT-bin/apache-hive-{version}-SNAPSHOT-bin”
as for the rest of the page.</p><h4 id=compile-hive-on-branch-1>Compile Hive
on branch-1</h4><p>In branch-1, Hive supports both Hadoop 1.x and 2.x. You
will need to specify which version of Hadoop to build against via a Maven
profile. To bu [...]
+</code></pre><p>Here, {version} refers to the current Hive version.</p><p>If
building Hive source using Maven (mvn), we will refer to the directory
“/packaging/target/apache-hive-{version}-SNAPSHOT-bin/apache-hive-{version}-SNAPSHOT-bin”
as <code><install-dir></code> for the rest of the page.</p><h4
id=compile-hive-on-branch-1>Compile Hive on branch-1</h4><p>In branch-1, Hive
supports both Hadoop 1.x and 2.x. You will need to specify which version of
Hadoop to build again [...]
</code></pre><h4 id=compile-hive-prior-to-013-on-hadoop-020>Compile Hive Prior
to 0.13 on Hadoop 0.20</h4><p>Prior to Hive 0.13, Hive was built using <a
href=http://ant.apache.org/>Apache Ant</a>. To build an older version of Hive
on Hadoop 0.20:</p><pre tabindex=0><code> $ svn co
http://svn.apache.org/repos/asf/hive/branches/branch-{version} hive
$ cd hive
$ ant clean package
diff --git a/docs/latest/admin/adminmanual-configuration/index.html
b/docs/latest/admin/adminmanual-configuration/index.html
index c2d8564f..d9a73357 100644
--- a/docs/latest/admin/adminmanual-configuration/index.html
+++ b/docs/latest/admin/adminmanual-configuration/index.html
@@ -10,7 +10,7 @@ Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i
<value>/tmp/mydir</value>
<description>Scratch space for Hive jobs</description>
</property>
-</code></pre><ul><li>In <strong>server-specific configuration files</strong>
(supported starting <a
href=https://issues.apache.org/jira/browse/HIVE-7342>Hive 0.14</a>). You can
set metastore-specific configuration values in
<strong>hivemetastore-site.xml</strong>, and HiveServer2-specific configuration
values in <strong>hiveserver2-site.xml</strong>.<br>The server-specific
configuration file is useful in two situations:</li></ul><ol><li><ol><li>You
want a different configuration for one [...]
+</code></pre><ul><li>In <strong>server-specific configuration files</strong>
(supported starting <a
href=https://issues.apache.org/jira/browse/HIVE-7342>Hive 0.14</a>). You can
set metastore-specific configuration values in
<strong>hivemetastore-site.xml</strong>, and HiveServer2-specific configuration
values in <strong>hiveserver2-site.xml</strong>.<br>The server-specific
configuration file is useful in two situations:</li></ul><ol><li><ol><li>You
want a different configuration for one [...]
Enter password:
Enter password again:
javax.jdo.option.ConnectionPassword has been successfully created.
diff --git
a/docs/latest/admin/adminmanual-metastore-3-0-administration/index.html
b/docs/latest/admin/adminmanual-metastore-3-0-administration/index.html
index 4e780980..480d4d85 100644
--- a/docs/latest/admin/adminmanual-metastore-3-0-administration/index.html
+++ b/docs/latest/admin/adminmanual-metastore-3-0-administration/index.html
@@ -3,7 +3,7 @@
<span class=navbar-toggler-icon></span></button><div class="collapse
navbar-collapse" id=navbarSupportedContent><ul class="navbar-nav me-auto"><li
class=nav-item><a class=nav-link
href=https://hive.apache.org//general/downloads>Releases</a></li><li
class="nav-item dropdown"><a class="nav-link dropdown-toggle" href=/Document
id=docsDropdown role=button data-bs-toggle=dropdown
aria-expanded=false>Documentation</a><ul class=dropdown-menu
aria-labelledby=docsDropdown><li><a class=dropdown-it [...]
<button type=submit class=search-button aria-label="Submit search">
<i class="fas
fa-search"></i></button></div></form></div></div></div></nav></menu></header><div
class=content><div class=docs-container><main class="docs-main
docs-main-full"><article class=docs-content><nav
class=docs-breadcrumb><ol><li><a href=/><i class="fas fa-home"></i>
Home</a></li><li><a href=/docs/>Documentation</a></li><li class=active>Apache
Hive : AdminManual Metastore 3.0 Administration</li></ol></nav><header
class=docs-header><h1 class=docs-title>Apache Hive : AdminManual Me [...]
-Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--adminmanual-metastore-30-administration>Apache Hive :
AdminManual Metastore 3.0 Administration</a><ul><li><a
href=#version-note>Version Note</a></li><li><a
href=#introduction>Introduction</a><ul><li><a
href=#changes-from-hive-2-to-hive-3>Changes From Hive 2 to Hive
3</a></li></ul></li><li><a href=#general-conf [...]
+Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--adminmanual-metastore-30-administration>Apache Hive :
AdminManual Metastore 3.0 Administration</a><ul><li><a
href=#version-note>Version Note</a></li><li><a
href=#introduction>Introduction</a><ul><li><a
href=#changes-from-hive-2-to-hive-3>Changes From Hive 2 to Hive
3</a></li></ul></li><li><a href=#general-conf [...]
</code></pre><p>Security: EXECUTE_SET_UGI,
metastore.authorization.storage.checks</p><p>Setting up Caching: CACHED*,
CATALOGS_TO_CACHE & AGGREGATE_STATS_CACHE*</p><p>Transactions: MAX_OPEN_TXNS,
TXNS_*</p><p> </p><p> </p></div><footer class=docs-footer><div
class=docs-feedback><h4><i class="fas fa-comment"></i> Feedback</h4><p>Was this
page helpful? Let us know how we can improve.</p><div
class=docs-feedback-buttons><button class="btn btn-feedback btn-positive">
<i class="fas fa-thumbs-up"></i> Yes
</button>
diff --git a/docs/latest/admin/adminmanual-metastore-administration/index.html
b/docs/latest/admin/adminmanual-metastore-administration/index.html
index 5f424cc9..08a14bc1 100644
--- a/docs/latest/admin/adminmanual-metastore-administration/index.html
+++ b/docs/latest/admin/adminmanual-metastore-administration/index.html
@@ -9,7 +9,7 @@ Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i
</property>
</code></pre><h4 id=default-configuration>Default Configuration</h4><p>The
default configuration sets up an embedded metastore which is used in unit tests
and is described in the next section. More practical options are described in
the subsequent sections.</p><h3
id=localembedded-metastore-database-derby>Local/Embedded Metastore Database
(Derby)</h3><p><strong>An embedded metastore database is mainly used for unit
tests. Only one process can connect to the metastore database at a time, [...]
</code></pre><p>In versions of Hive earlier than 0.5.0, it’s instead
necessary to run the Thrift server via direct execution of Java:</p><pre
tabindex=0><code>$JAVA_HOME/bin/java -Xmx1024m
-Dlog4j.configuration=file://$HIVE_HOME/conf/hms-log4j.properties
-Djava.library.path=$HADOOP_HOME/lib/native/Linux-amd64-64/ -cp $CLASSPATH
org.apache.hadoop.hive.metastore.HiveMetaStore
-</code></pre><p>If you execute Java directly, then JAVA_HOME, HIVE_HOME,
HADOOP_HOME must be correctly set; CLASSPATH should contain Hadoop, Hive (lib
and auxlib), and Java jars.</p><p><strong>Server Configuration
Parameters</strong></p><p>The following example uses a<a
href=#remote-metastore-database>Remote Metastore
Database</a>.</p><table><thead><tr><th>Config Param</th><th>Config
Value</th><th>Comment</th></tr></thead><tbody><tr><td>javax.jdo.option.ConnectionURL</td><td><code>jdbc:m
[...]
+</code></pre><p>If you execute Java directly, then JAVA_HOME, HIVE_HOME,
HADOOP_HOME must be correctly set; CLASSPATH should contain Hadoop, Hive (lib
and auxlib), and Java jars.</p><p><strong>Server Configuration
Parameters</strong></p><p>The following example uses a<a
href=#remote-metastore-database>Remote Metastore
Database</a>.</p><table><thead><tr><th>Config Param</th><th>Config
Value</th><th>Comment</th></tr></thead><tbody><tr><td>javax.jdo.option.ConnectionURL</td><td><code>jdbc:m
[...]
</code></pre><h3 id=supported-backend-databases-for-metastore>Supported
Backend Databases for
Metastore</h3><table><thead><tr><th>Database</th><th>Minimum Supported
Version</th><th>Name for Parameter Values</th><th>See
Also</th></tr></thead><tbody><tr><td>MySQL</td><td>5.6.17</td><td><code>mysql</code></td><td></td></tr><tr><td>Postgres</td><td>9.1.13</td><td><code>postgres</code></td><td></td></tr><tr><td>Oracle</td><td>11g</td><td><code>oracle</code></td><td><a
href=#hive-metastore-orm [...]
<i class="fas fa-thumbs-up"></i> Yes
</button>
diff --git a/docs/latest/admin/hive-on-spark-getting-started/index.html
b/docs/latest/admin/hive-on-spark-getting-started/index.html
index cf13745f..387ad3ce 100644
--- a/docs/latest/admin/hive-on-spark-getting-started/index.html
+++ b/docs/latest/admin/hive-on-spark-getting-started/index.html
@@ -4,10 +4,10 @@
<button type=submit class=search-button aria-label="Submit search">
<i class="fas
fa-search"></i></button></div></form></div></div></div></nav></menu></header><div
class=content><div class=docs-container><main class="docs-main
docs-main-full"><article class=docs-content><nav
class=docs-breadcrumb><ol><li><a href=/><i class="fas fa-home"></i>
Home</a></li><li><a href=/docs/>Documentation</a></li><li class=active>Apache
Hive : Hive on Spark: Getting Started</li></ol></nav><header
class=docs-header><h1 class=docs-title>Apache Hive : Hive on Spark: Getting S
[...]
Last updated: December 12, 2024</span></div></header><div
class=docs-toc><h4><i class="fas fa-list"></i> Table of Contents</h4><nav
id=TableOfContents><ul><li><a
href=#apache-hive--hive-on-spark-getting-started>Apache Hive : Hive on Spark:
Getting Started</a><ul><li><a href=#version-compatibility>Version
Compatibility</a></li><li><a href=#spark-installation>Spark
Installation</a></li><li><a href=#configuring-yarn>Configuring
YARN</a></li><li><a href=#configuring-hive>Configuring Hive</a> [...]
-</code></pre><p>Hive on Spark was added in <a
href=https://issues.apache.org/jira/browse/HIVE-7292>HIVE-7292</a>.</p><h2
id=version-compatibility>Version Compatibility</h2><p>Hive on Spark is only
tested with a specific version of Spark, so a given version of Hive is only
guaranteed to work with a specific version of Spark. Other versions of Spark
may work with a given version of Hive, but that is not guaranteed. Below is a
list of Hive versions and their corresponding compatible Spark v [...]
+</code></pre><p>Hive on Spark was added in <a
href=https://issues.apache.org/jira/browse/HIVE-7292>HIVE-7292</a>.</p><h2
id=version-compatibility>Version Compatibility</h2><p>Hive on Spark is only
tested with a specific version of Spark, so a given version of Hive is only
guaranteed to work with a specific version of Spark. Other versions of Spark
may work with a given version of Hive, but that is not guaranteed. Below is a
list of Hive versions and their corresponding compatible Spark v [...]
</code></pre><p>Since Spark 2.0.0:</p><pre
tabindex=0><code>./dev/make-distribution.sh --name
"hadoop2-without-hive" --tgz
"-Pyarn,hadoop-provided,hadoop-2.7,parquet-provided"
</code></pre><p>Since Spark 2.3.0:</p><pre
tabindex=0><code>./dev/make-distribution.sh --name
"hadoop2-without-hive" --tgz
"-Pyarn,hadoop-provided,hadoop-2.7,parquet-provided,orc-provided"
-</code></pre></li><li><p>Start Spark cluster</p><ul><li>Keep note of the .
This can be found in Spark master WebUI.</li></ul></li></ol><h2
id=configuring-yarn>Configuring YARN</h2><p>Instead of the <a
href=https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html>capacity
scheduler</a>, the <a
href=https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html>fair
scheduler</a> is required. This fairly distributes an equal share o [...]
+</code></pre></li><li><p>Start Spark cluster</p><ul><li>Keep note of the
<code><Spark Master URL></code>. This can be found in Spark master
WebUI.</li></ul></li></ol><h2 id=configuring-yarn>Configuring
YARN</h2><p>Instead of the <a
href=https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html>capacity
scheduler</a>, the <a
href=https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html>fair
scheduler</a> is required. This f [...]
</code></pre><p>See the <a
href=#spark-section-of-hive-configuration-properties>Spark section of Hive
Configuration Properties</a> for other properties for configuring Hive and the
Remote Spark Driver.
3. Configure Spark-application configs for Hive. See: <a
href=http://spark.apache.org/docs/latest/configuration.html>http://spark.apache.org/docs/latest/configuration.html</a>.
This can be done either by adding a file “spark-defaults.conf”
with these properties to the Hive classpath, or by setting them on Hive
configuration (<code>hive-site.xml</code>). For instance:</p><pre
tabindex=0><code>set spark.master=<Spark Master URL>
set spark.eventLog.enabled=true;
@@ -28,7 +28,7 @@ set
spark.serializer=org.apache.spark.serializer.KryoSerializer;
<name>spark.yarn.jars</name>
<value>hdfs://xxxx:8020/spark-jars/*</value>
</property>
-</code></pre></li></ol><h2 id=configuring-spark>Configuring
Spark</h2><p>Setting executor memory size is more complicated than simply
setting it to be as large as possible. There are several things that need to be
taken into consideration:</p><ul><li>More executor memory means it can enable
mapjoin optimization for more queries.</li><li>More executor memory, on the
other hand, becomes unwieldy from GC perspective.</li><li>Some experiments
shows that HDFS client doesn’t handle concurrent [...]
+</code></pre></li></ol><h2 id=configuring-spark>Configuring
Spark</h2><p>Setting executor memory size is more complicated than simply
setting it to be as large as possible. There are several things that need to be
taken into consideration:</p><ul><li>More executor memory means it can enable
mapjoin optimization for more queries.</li><li>More executor memory, on the
other hand, becomes unwieldy from GC perspective.</li><li>Some experiments
shows that HDFS client doesn’t handle concurrent [...]
hive.vectorized.execution.enabled=true
hive.cbo.enable=true
diff --git a/docs/latest/admin/setting-up-hiveserver2/index.html
b/docs/latest/admin/setting-up-hiveserver2/index.html
index 55264b39..aa61b632 100644
--- a/docs/latest/admin/setting-up-hiveserver2/index.html
+++ b/docs/latest/admin/setting-up-hiveserver2/index.html
@@ -10,7 +10,7 @@ Starting HiveServer2
usage: hiveserver2
-H,--help Print help information
--hiveconf <property=value> Use value for given property
-</code></pre><h2
id=authenticationsecurity-configuration>Authentication/Security
Configuration</h2><p>HiveServer2 supports Anonymous (no authentication) with
and without SASL, Kerberos (GSSAPI), pass through LDAP, Pluggable Custom
Authentication and Pluggable Authentication Modules (PAM, supported Hive 0.13
onwards).</p><h4
id=configuration><strong>Configuration</strong></h4><p>Authentication
mode:</p><p>hive.server2.authentication – Authentication mode, default NONE.
Options are NONE (u [...]
+</code></pre><h2
id=authenticationsecurity-configuration>Authentication/Security
Configuration</h2><p>HiveServer2 supports Anonymous (no authentication) with
and without SASL, Kerberos (GSSAPI), pass through LDAP, Pluggable Custom
Authentication and Pluggable Authentication Modules (PAM, supported Hive 0.13
onwards).</p><h4
id=configuration><strong>Configuration</strong></h4><p>Authentication
mode:</p><p>hive.server2.authentication – Authentication mode, default NONE.
Options are NONE (u [...]
-r dry-run mode, which produces a list on console
-v verbose mode, which prints extra debugging information
-s if you are using non-standard scratch directory
diff --git a/index.json b/index.json
index 533adff3..7447638f 100644
--- a/index.json
+++ b/index.json
@@ -1 +1 @@
-[{"categories":null,"contents":"Apache Hive : Iceberg REST Catalog API backed
by Hive Metastore Introduction Hive Metastore offers Iceberg REST API endpoints
for clients native to Apache Iceberg. Consequently, Iceberg users can access
Iceberg tables via either Hive Metastore Thrift API (using HiveCatalog) or
Iceberg REST Catalog API.\nBasic configurations You must configure the
following parameters.\nKey Required? Default Value
metastore.catalog.servlet.port Yes -1 The port number to whi [...]
\ No newline at end of file
+[{"categories":null,"contents":"Apache Hive : Iceberg REST Catalog API backed
by Hive Metastore Introduction Hive Metastore offers Iceberg REST API endpoints
for clients native to Apache Iceberg. Consequently, Iceberg users can access
Iceberg tables via either Hive Metastore Thrift API (using HiveCatalog) or
Iceberg REST Catalog API.\nBasic configurations You must configure the
following parameters.\nKey Required? Default Value
metastore.catalog.servlet.port Yes -1 The port number to whi [...]
\ No newline at end of file
