[jira] [Updated] (HIVE-27688) hive MoveTask cannot show the correct exception message
[
https://issues.apache.org/jira/browse/HIVE-27688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27688:
--
Labels: pull-request-available (was: )
> hive MoveTask cannot show the correct exception message
> ---
>
> Key: HIVE-27688
> URL: https://issues.apache.org/jira/browse/HIVE-27688
> Project: Hive
> Issue Type: Bug
>Reporter: liang yu
>Assignee: liang yu
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2023-09-13-17-39-27-864.png,
> image-2023-09-13-17-40-02-981.png
>
>
> I am using hive.version=3.1.3; hadoop.version=3.3.4.
> Setting hive.load.data.owner to hive, and I used user ubd_by to execute sql.
> When I try to insert *overwrite* to an {*}existing table partition{*}, I get
> the exception that: Unable to move source /path/to/source to destination
> /path/to/dest, which is a very regular error message, gives me no helpful
> information.
>
> {code:java}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.Movelask.Unable to move source
> hdfs://xl/user/ubd master/ubd_b_dwa.
> db/dwa_m_user/month_id=xxx/prov_id=xx/.staging to destination
> hdfs://xl/user/ubd master/ubd_b_dwa.
> db/dwa_m_user/month_id=xxx/prov_id=xx/.staging {code}
>
> But when I try to insert *into* an {*}existing table partition{*}, I get the
> exception that: Unable to move source /path/to/source to destination
> /path/to/dest as the file is not owned by hive and load data is also not ran
> as hive. which gives me a very helpful error message that I should change the
> hive.load.data.owner to hive.
>
> {code:java}
> FAILED: Execution Error, return code l from org.
> apache.hadoop.hive.ql.metadata.HiveException: org.
> apache.hadoop.hive.gl.exec.Movelask: Load Data failed for hdfs://xl/user/ubd
> master/ubd_b_dwa. db/dwa_m_user/month_id=xxx/prov_id=xx/
> hive-staging hive 2023-09-13 17-34-31 302 5892190500368248766-1/-ext-1 as
> the file is not owned by hive and load data is also not ran as hive{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27694) Include HiveIcebergSerDe in default list of serdes using HMS
[ https://issues.apache.org/jira/browse/HIVE-27694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam resolved HIVE-27694. -- Fix Version/s: 4.0.0 Resolution: Fixed Thank you for the review [~ayushtkn] and [~dengzh]. Fix has been merged to master. Closing the jira. > Include HiveIcebergSerDe in default list of serdes using HMS > > > Key: HIVE-27694 > URL: https://issues.apache.org/jira/browse/HIVE-27694 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > > MetastoreConf has this default list of SerDes that use HMS to persist the > metadata. Iceberg tables also use have their metadata in HMS and hence can be > fetched from metastore. This SerDe needs to be added to this list as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27576) Backport of HIVE-21001: Upgrade to calcite-1.19
[ https://issues.apache.org/jira/browse/HIVE-27576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27576: -- Labels: pull-request-available (was: ) > Backport of HIVE-21001: Upgrade to calcite-1.19 > --- > > Key: HIVE-27576 > URL: https://issues.apache.org/jira/browse/HIVE-27576 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Kamal Sharma >Assignee: Kamal Sharma >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27698) Backport of HIVE-22398: Remove legacy code that can cause issue with new Yarn releases
[ https://issues.apache.org/jira/browse/HIVE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27698: -- Labels: pull-request-available (was: ) > Backport of HIVE-22398: Remove legacy code that can cause issue with new Yarn > releases > -- > > Key: HIVE-27698 > URL: https://issues.apache.org/jira/browse/HIVE-27698 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27698) Backport of HIVE-22398: Remove legacy code that can cause issue with new Yarn releases
Aman Raj created HIVE-27698: --- Summary: Backport of HIVE-22398: Remove legacy code that can cause issue with new Yarn releases Key: HIVE-27698 URL: https://issues.apache.org/jira/browse/HIVE-27698 Project: Hive Issue Type: Sub-task Reporter: Aman Raj -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27564) Add log for ZooKeeperTokenStore
[
https://issues.apache.org/jira/browse/HIVE-27564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766082#comment-17766082
]
Ayush Saxena commented on HIVE-27564:
-
Committed to master.
Thanx [~1365976...@qq.com] for the contribution!!!
> Add log for ZooKeeperTokenStore
> ---
>
> Key: HIVE-27564
> URL: https://issues.apache.org/jira/browse/HIVE-27564
> Project: Hive
> Issue Type: Improvement
>Reporter: lvyanquan
>Assignee: lvyanquan
>Priority: Major
> Labels: pull-request-available
> Attachments: picture 1.png
>
>
> when use zookeeper to store DelegationTokenIdentifier, we use
> {code:java}
> TokenStoreDelegationTokenSecretManager.encodeWritable(tokenIdentifier) {code}
> to encode message of tokens and create the path in zookeeper, like
> picture1.png
> However, when this token was removed, error message will display
> DelegationTokenIdentifier only, then we have no idea about whether this path
> is still existed and when the path in zookeeper was created and deleted.
> {code:java}
> public byte[] retrievePassword(DelegationTokenIdentifier identifier) throws
> InvalidToken {
> DelegationTokenInformation info = this.tokenStore.getToken(identifier);
> if (info == null) {
> throw new InvalidToken("token expired or does not exist: " +
> identifier);
> }
> // must reuse super as info.getPassword is not accessible
> synchronized (this) {
> try {
> super.currentTokens.put(identifier, info);
> return super.retrievePassword(identifier);
> } finally {
> super.currentTokens.remove(identifier);
> }
> }
> } {code}
> So, I try to add so log about the lifecycle of the tokenPath.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27564) Add log for ZooKeeperTokenStore
[
https://issues.apache.org/jira/browse/HIVE-27564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved HIVE-27564.
-
Fix Version/s: 4.0.0
Resolution: Fixed
> Add log for ZooKeeperTokenStore
> ---
>
> Key: HIVE-27564
> URL: https://issues.apache.org/jira/browse/HIVE-27564
> Project: Hive
> Issue Type: Improvement
>Reporter: lvyanquan
>Assignee: lvyanquan
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: picture 1.png
>
>
> when use zookeeper to store DelegationTokenIdentifier, we use
> {code:java}
> TokenStoreDelegationTokenSecretManager.encodeWritable(tokenIdentifier) {code}
> to encode message of tokens and create the path in zookeeper, like
> picture1.png
> However, when this token was removed, error message will display
> DelegationTokenIdentifier only, then we have no idea about whether this path
> is still existed and when the path in zookeeper was created and deleted.
> {code:java}
> public byte[] retrievePassword(DelegationTokenIdentifier identifier) throws
> InvalidToken {
> DelegationTokenInformation info = this.tokenStore.getToken(identifier);
> if (info == null) {
> throw new InvalidToken("token expired or does not exist: " +
> identifier);
> }
> // must reuse super as info.getPassword is not accessible
> synchronized (this) {
> try {
> super.currentTokens.put(identifier, info);
> return super.retrievePassword(identifier);
> } finally {
> super.currentTokens.remove(identifier);
> }
> }
> } {code}
> So, I try to add so log about the lifecycle of the tokenPath.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27697) Bump org.apache.commons:commons-compress from 1.23.0 to 1.24.0
Ayush Saxena created HIVE-27697: --- Summary: Bump org.apache.commons:commons-compress from 1.23.0 to 1.24.0 Key: HIVE-27697 URL: https://issues.apache.org/jira/browse/HIVE-27697 Project: Hive Issue Type: Improvement Reporter: Ayush Saxena PR from dependabot: https://github.com/apache/hive/pull/4701 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27664) AlterTableSetLocationAnalyzer threw a confusing exception "Cannot connect to namenode"
[
https://issues.apache.org/jira/browse/HIVE-27664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved HIVE-27664.
-
Fix Version/s: 4.0.0
Resolution: Fixed
> AlterTableSetLocationAnalyzer threw a confusing exception "Cannot connect to
> namenode"
> --
>
> Key: HIVE-27664
> URL: https://issues.apache.org/jira/browse/HIVE-27664
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0-beta-1
>Reporter: xiongyinke
>Assignee: xiongyinke
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> @Override
> protected void analyzeCommand(TableName tableName, Map
> partitionSpec, ASTNode command)
> throws SemanticException {
> String newLocation = unescapeSQLString(command.getChild(0).getText());
> try {
> // To make sure host/port pair is valid, the status of the location does not
> matter
> FileSystem.get(new URI(newLocation), conf).getFileStatus(new
> Path(newLocation));
> } catch (FileNotFoundException e) {
> // Only check host/port pair is valid, whether the file exist or not does not
> matter
> } catch (Exception e) {
> throw new SemanticException("Cannot connect to namenode, please check if
> host/port pair for " + newLocation +
> " is valid", e);
> }
> When the
> "FileSystem.get(new URI(newLocation), conf).getFileStatus(new
> Path(newLocation))"
> code throws a "Permission denied" exception, the Beeline client will receive
> the confusing exception "Cannot connect to namenode, please check if
> host/port pair for". In reality, the issue is not with the namenode.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27664) AlterTableSetLocationAnalyzer threw a confusing exception "Cannot connect to namenode"
[
https://issues.apache.org/jira/browse/HIVE-27664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766081#comment-17766081
]
Ayush Saxena commented on HIVE-27664:
-
Committed to master.
Thanx [~bearbeer] for the contribution!!!
> AlterTableSetLocationAnalyzer threw a confusing exception "Cannot connect to
> namenode"
> --
>
> Key: HIVE-27664
> URL: https://issues.apache.org/jira/browse/HIVE-27664
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0-beta-1
>Reporter: xiongyinke
>Assignee: xiongyinke
>Priority: Major
> Labels: pull-request-available
>
> @Override
> protected void analyzeCommand(TableName tableName, Map
> partitionSpec, ASTNode command)
> throws SemanticException {
> String newLocation = unescapeSQLString(command.getChild(0).getText());
> try {
> // To make sure host/port pair is valid, the status of the location does not
> matter
> FileSystem.get(new URI(newLocation), conf).getFileStatus(new
> Path(newLocation));
> } catch (FileNotFoundException e) {
> // Only check host/port pair is valid, whether the file exist or not does not
> matter
> } catch (Exception e) {
> throw new SemanticException("Cannot connect to namenode, please check if
> host/port pair for " + newLocation +
> " is valid", e);
> }
> When the
> "FileSystem.get(new URI(newLocation), conf).getFileStatus(new
> Path(newLocation))"
> code throws a "Permission denied" exception, the Beeline client will receive
> the confusing exception "Cannot connect to namenode, please check if
> host/port pair for". In reality, the issue is not with the namenode.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
