[jira] [Work logged] (HIVE-25365) Insufficient privileges to show partitions when partition columns are authorized
[ https://issues.apache.org/jira/browse/HIVE-25365?focusedWorklogId=643059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643059 ] ASF GitHub Bot logged work on HIVE-25365: - Author: ASF GitHub Bot Created on: 28/Aug/21 04:21 Start Date: 28/Aug/21 04:21 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #2515: URL: https://github.com/apache/hive/pull/2515#issuecomment-907565160 Hi @kgyrtkirk, cloud this changes be merged if have no problem? Thanks, Zhihua Deng -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 643059) Time Spent: 20m (was: 10m) > Insufficient privileges to show partitions when partition columns are > authorized > > > Key: HIVE-25365 > URL: https://issues.apache.org/jira/browse/HIVE-25365 > Project: Hive > Issue Type: Bug > Components: Authorization >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When the privileges of partition columns have granted to users, showing > partitions still needs select privilege on the table, though they are able to > query from partition columns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=643057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643057 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 28/Aug/21 04:03 Start Date: 28/Aug/21 04:03 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request #2344: URL: https://github.com/apache/hive/pull/2344 …properly ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 643057) Time Spent: 7h 50m (was: 7h 40m) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 7h 50m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=643056=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643056 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 28/Aug/21 04:01 Start Date: 28/Aug/21 04:01 Worklog Time Spent: 10m Work Description: dengzhhu653 closed pull request #2344: URL: https://github.com/apache/hive/pull/2344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 643056) Time Spent: 7h 40m (was: 7.5h) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24944) When the default engine of the hiveserver is MR and the tez engine is set by the client, the client TEZ progress log cannot be printed normally
[ https://issues.apache.org/jira/browse/HIVE-24944?focusedWorklogId=643046=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643046 ] ASF GitHub Bot logged work on HIVE-24944: - Author: ASF GitHub Bot Created on: 28/Aug/21 00:09 Start Date: 28/Aug/21 00:09 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #2204: URL: https://github.com/apache/hive/pull/2204#issuecomment-907533741 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 643046) Time Spent: 1h 40m (was: 1.5h) > When the default engine of the hiveserver is MR and the tez engine is set by > the client, the client TEZ progress log cannot be printed normally > --- > > Key: HIVE-24944 > URL: https://issues.apache.org/jira/browse/HIVE-24944 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 4.0.0 >Reporter: ZhangQiDong >Assignee: ZhangQiDong >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24944.001.patch, HIVE-24944.002.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > HiveServer configuration parameter execution default MR. When set > hive.execution.engine = tez, the client cannot print the TEZ log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25303) CTAS hive.create.as.external.legacy tries to place data files in managed WH path
[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=643030=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643030
]
ASF GitHub Bot logged work on HIVE-25303:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 22:20
Start Date: 27/Aug/21 22:20
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r697746815
##
File path: ql/src/test/queries/clientpositive/ctas.q
##
@@ -73,3 +73,13 @@ select k, value from acid_ctas_part;
explain formatted
select k, value from acid_ctas_part;
+
+-- CTAS with external legacy config
Review comment:
Sure. will do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 643030)
Time Spent: 2h (was: 1h 50m)
> CTAS hive.create.as.external.legacy tries to place data files in managed WH
> path
>
>
> Key: HIVE-25303
> URL: https://issues.apache.org/jira/browse/HIVE-25303
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Under legacy table creation mode (hive.create.as.external.legacy=true), when
> a database has been created in a specific LOCATION, in a session where that
> database is Used, tables are created using the following command:
> {code:java}
> CREATE TABLE AS SELECT {code}
> should inherit the HDFS path from the database's location. Instead, Hive is
> trying to write the table data into
> /warehouse/tablespace/managed/hive//
> +Design+:
> In the CTAS query, first data is written in the target directory (which
> happens in HS2) and then the table is created(This happens in HMS). So here
> two decisions are being made i) target directory location ii) how the table
> should be created (table type, sd e.t.c).
> When HS2 needs a target location that needs to be set, it'll make create
> table dry run call to HMS (where table translation happens) and i) and ii)
> decisions are made within HMS and returns table object. Then HS2 will use
> this location set by HMS for placing the data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25303) CTAS hive.create.as.external.legacy tries to place data files in managed WH path
[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=642995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642995
]
ASF GitHub Bot logged work on HIVE-25303:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 20:30
Start Date: 27/Aug/21 20:30
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r697702962
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java
##
@@ -472,6 +474,32 @@ private void setLoadFileLocation(
loc = cmv.getLocation();
}
Path location = (loc == null) ? getDefaultCtasLocation(pCtx) : new
Path(loc);
+boolean isExternal = false;
+boolean isAcid = false;
+if (pCtx.getQueryProperties().isCTAS()) {
+ isExternal = pCtx.getCreateTable().isExternal();
+ isAcid = pCtx.getCreateTable().getTblProps().getOrDefault(
+ hive_metastoreConstants.TABLE_IS_TRANSACTIONAL,
"false").equalsIgnoreCase("true") ||
+
pCtx.getCreateTable().getTblProps().containsKey(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES);
+ if ((HiveConf.getBoolVar(conf,
HiveConf.ConfVars.CREATE_TABLE_AS_EXTERNAL) || (isExternal || !isAcid))) {
+CreateTableDesc ctd = pCtx.getCreateTable();
+ctd.getTblProps().put(hive_metastoreConstants.TABLE_IS_TRANSACTIONAL,
"false"); // create as external table
+try {
+ Table table = ctd.toTable(conf);
+ table = db.getCTASQueryDryrun(table.getTTable());
+ org.apache.hadoop.hive.metastore.api.Table tTable =
table.getTTable();
+ if (tTable.getSd() != null && tTable.getSd().getLocation() != null)
{
+location = new Path(tTable.getSd().getLocation());
+ctd.setLocation(location.toString());
+ }
+
ctd.setExternal(TableType.EXTERNAL_TABLE.toString().equals(tTable.getTableType()));
+ ctd.setTblProps(tTable.getParameters());
Review comment:
Sure. Will do the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642995)
Time Spent: 1h 50m (was: 1h 40m)
> CTAS hive.create.as.external.legacy tries to place data files in managed WH
> path
>
>
> Key: HIVE-25303
> URL: https://issues.apache.org/jira/browse/HIVE-25303
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Under legacy table creation mode (hive.create.as.external.legacy=true), when
> a database has been created in a specific LOCATION, in a session where that
> database is Used, tables are created using the following command:
> {code:java}
> CREATE TABLE AS SELECT {code}
> should inherit the HDFS path from the database's location. Instead, Hive is
> trying to write the table data into
> /warehouse/tablespace/managed/hive//
> +Design+:
> In the CTAS query, first data is written in the target directory (which
> happens in HS2) and then the table is created(This happens in HMS). So here
> two decisions are being made i) target directory location ii) how the table
> should be created (table type, sd e.t.c).
> When HS2 needs a target location that needs to be set, it'll make create
> table dry run call to HMS (where table translation happens) and i) and ii)
> decisions are made within HMS and returns table object. Then HS2 will use
> this location set by HMS for placing the data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25303) CTAS hive.create.as.external.legacy tries to place data files in managed WH path
[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=642993=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642993
]
ASF GitHub Bot logged work on HIVE-25303:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 20:28
Start Date: 27/Aug/21 20:28
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r697701637
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java
##
@@ -472,6 +474,32 @@ private void setLoadFileLocation(
loc = cmv.getLocation();
}
Path location = (loc == null) ? getDefaultCtasLocation(pCtx) : new
Path(loc);
+boolean isExternal = false;
+boolean isAcid = false;
+if (pCtx.getQueryProperties().isCTAS()) {
+ isExternal = pCtx.getCreateTable().isExternal();
+ isAcid = pCtx.getCreateTable().getTblProps().getOrDefault(
+ hive_metastoreConstants.TABLE_IS_TRANSACTIONAL,
"false").equalsIgnoreCase("true") ||
+
pCtx.getCreateTable().getTblProps().containsKey(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES);
+ if ((HiveConf.getBoolVar(conf,
HiveConf.ConfVars.CREATE_TABLE_AS_EXTERNAL) || (isExternal || !isAcid))) {
Review comment:
I agree that metadata transformer does the acid-related checks, but if
know that the table is managed in HS2 itself, we can avoid a dry-run call to
HMS which essentially skips the translation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642993)
Time Spent: 1h 40m (was: 1.5h)
> CTAS hive.create.as.external.legacy tries to place data files in managed WH
> path
>
>
> Key: HIVE-25303
> URL: https://issues.apache.org/jira/browse/HIVE-25303
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Under legacy table creation mode (hive.create.as.external.legacy=true), when
> a database has been created in a specific LOCATION, in a session where that
> database is Used, tables are created using the following command:
> {code:java}
> CREATE TABLE AS SELECT {code}
> should inherit the HDFS path from the database's location. Instead, Hive is
> trying to write the table data into
> /warehouse/tablespace/managed/hive//
> +Design+:
> In the CTAS query, first data is written in the target directory (which
> happens in HS2) and then the table is created(This happens in HMS). So here
> two decisions are being made i) target directory location ii) how the table
> should be created (table type, sd e.t.c).
> When HS2 needs a target location that needs to be set, it'll make create
> table dry run call to HMS (where table translation happens) and i) and ii)
> decisions are made within HMS and returns table object. Then HS2 will use
> this location set by HMS for placing the data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-23896:
Fix Version/s: (was: 4.0.0)
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
> 2020-07-22 07:15:33: Starting HiveServer2
> SLF4J: Class path contains multiple
[jira] [Reopened] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun reopened HIVE-23896:
-
Assignee: (was: alanwake)
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
> 2020-07-22 07:15:33: Starting HiveServer2
>
[jira] [Resolved] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-23896.
-
Fix Version/s: 4.0.0
Hadoop Flags: Reviewed
Assignee: alanwake
Resolution: Fixed
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Assignee: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100]
[jira] [Commented] (HIVE-22916) Upgrade to Hadoop 3.2.2 once it's released and follow it with jetty and guava versions
[ https://issues.apache.org/jira/browse/HIVE-22916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405982#comment-17405982 ] Brahma Reddy Battula commented on HIVE-22916: - Any update on this issue..? Already hadoop-3.2.2 is released. > Upgrade to Hadoop 3.2.2 once it's released and follow it with jetty and guava > versions > -- > > Key: HIVE-22916 > URL: https://issues.apache.org/jira/browse/HIVE-22916 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405970#comment-17405970
]
Brahma Reddy Battula commented on HIVE-22561:
-
Looks duplicate of HIVE-22098?
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.1.0, 3.0.0
>
> Attachments: HIVE-22561.1.branch-3.1.patch,
> HIVE-22561.branch-3.1.patch, HIVE-22561.patch, Screenshot 2019-11-28 at
> 8.45.17 PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405967#comment-17405967
]
Brahma Reddy Battula commented on HIVE-23896:
-
[~alanwake] looks this PR is not relvant this jira..?
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
>
[jira] [Commented] (HIVE-24573) hive 3.1.2 drop table Sometimes it can't be deleted
[ https://issues.apache.org/jira/browse/HIVE-24573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405963#comment-17405963 ] Brahma Reddy Battula commented on HIVE-24573: - is this problem with ACID or Non-ACID managed tables..? > hive 3.1.2 drop table Sometimes it can't be deleted > --- > > Key: HIVE-24573 > URL: https://issues.apache.org/jira/browse/HIVE-24573 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: paul >Priority: Blocker > > Execute drop table if exists trade_ 4_ Temp448 statement, the table cannot be > deleted; hive.log The log shows > 2020-12-29T07:30:04,840 ERROR [HiveServer2-Background-Pool: Thread-6483] > metadata.Hive: Table dc_usermanage.trade_3_temp448 not found: > hive.dc_usermanage.trade_3_temp448 table not found > > Statement returns success > > I doubt that this problem will only arise under the condition of high-level > merger. We run a lot of tasks every day, one or two tasks every day, which > will happen > > metastore mysql > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24078) result rows not equal in mr and tez
[ https://issues.apache.org/jira/browse/HIVE-24078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405961#comment-17405961 ] Brahma Reddy Battula commented on HIVE-24078: - [~code_kr_dev_s],[~liguangyu] any update on this issue..? Looks it's ok with MR. > result rows not equal in mr and tez > --- > > Key: HIVE-24078 > URL: https://issues.apache.org/jira/browse/HIVE-24078 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Tez >Affects Versions: 3.1.2 >Reporter: kuqiqi >Assignee: shubhangi priya >Priority: Blocker > > select > rank_num, > province_name, > programset_id, > programset_name, > programset_type, > cv, > uv, > pt, > rank_num2, > rank_num3, > city_name, > level, > cp_code, > cp_name, > version_type, > zz.city_code, > zz.province_alias, > '20200815' dt > from > (SELECT row_number() over(partition BY > a1.province_alias,a1.city_code,a1.version_type > ORDER BY cast(a1.cv AS bigint) DESC) AS rank_num, > province_name(a1.province_alias) AS province_name, > a1.program_set_id AS programset_id, > a2.programset_name, > a2.type_name AS programset_type, > a1.cv, > a1.uv, > cast(a1.pt/360 as decimal(20,2)) pt, > row_number() over (partition by > a1.province_alias,a1.city_code,a1.version_type order by cast(a1.uv as bigint) > desc ) as rank_num2, > row_number() over (partition by > a1.province_alias,a1.city_code,a1.version_type order by cast(a1.pt as bigint) > desc ) as rank_num3, > a1.city_code, > a1.city_name, > '3' as level, > a2.cp_code, > a2.cp_name, > '20200815'as dt, > a1.province_alias, > a1.version_type > FROM temp.dmp_device_vod_valid_day_v1_20200815_hn a1 > LEFT JOIN temp.dmp_device_vod_valid_day_v2_20200815_hn a2 ON > a1.program_set_id=a2.programset_id > WHERE a2.programset_name IS NOT NULL ) zz > where rank_num<1000 or rank_num2<1000 or rank_num3<1000 > ; > > This sql gets 76742 rows in mr, but 76681 rows in tez.How to fix it? > I think the problem maybe lies in row_number. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22098) Data loss occurs when multiple tables are join with different bucket_version
[
https://issues.apache.org/jira/browse/HIVE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405959#comment-17405959
]
Brahma Reddy Battula commented on HIVE-22098:
-
Hi [~luguangming] , [~zergtant] ,[~belugabehr] ,[~jithendhir92] and
[~kgyrtkirk].
is this data loss with managed-non transactional tables..?
are you guys using non-transactional tables with hive-3.x..?
> Data loss occurs when multiple tables are join with different bucket_version
>
>
> Key: HIVE-22098
> URL: https://issues.apache.org/jira/browse/HIVE-22098
> Project: Hive
> Issue Type: Bug
> Components: Operators
>Affects Versions: 3.1.0, 3.1.2
>Reporter: GuangMing Lu
>Priority: Blocker
> Labels: data-loss, wrongresults
> Attachments: HIVE-22098.1.patch, image-2019-08-12-18-45-15-771.png,
> join_test.sql, table_a_data.orc, table_b_data.orc, table_c_data.orc
>
>
> When different bucketVersion of tables do join and no of reducers is greater
> than 2, the result is incorrect (*data loss*).
> *Scenario 1*: Three tables join. The temporary result data of table_a in the
> first table and table_b in the second table joins result is recorded as
> tmp_a_b, When it joins with the third table, the bucket_version=2 of the
> table created by default after hive-3.0.0, temporary data tmp_a_b initialized
> the bucketVerison=-1, and then ReduceSinkOperator Verketison=-1 is joined. In
> the init method, the hash algorithm of selecting join column is selected
> according to bucketVersion. If bucketVersion = 2 and is not an acid
> operation, it will acquired the new algorithm of hash. Otherwise, the old
> algorithm of hash is acquired. Because of the inconsistency of the algorithm
> of hash, the partition of data allocation caused are different. At stage of
> Reducer, Data with the same key can not be paired resulting in data loss.
> *Scenario 2*: create two test tables, create table
> table_bucketversion_1(col_1 string, col_2 string) TBLPROPERTIES
> ('bucketing_version'='1'); table_bucketversion_2(col_1 string, col_2 string)
> TBLPROPERTIES ('bucketing_version'='2');
> when use table_bucketversion_1 to join table_bucketversion_2, partial result
> data will be loss due to bucketVerison is different.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25303) CTAS hive.create.as.external.legacy tries to place data files in managed WH path
[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=642935=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642935
]
ASF GitHub Bot logged work on HIVE-25303:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 18:02
Start Date: 27/Aug/21 18:02
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r697614262
##
File path:
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##
@@ -2427,7 +2427,7 @@ service ThriftHiveMetastore extends fb303.FacebookService
throws(1:NoSuchObjectException o1, 2:MetaException o2)
void add_check_constraint(1:AddCheckConstraintRequest req)
throws(1:NoSuchObjectException o1, 2:MetaException o2)
-
+ Table ctas_query_dryrun(1:Table tbl) throws(1:AlreadyExistsException o1,
2:InvalidObjectException o2, 3:MetaException o3, 4:NoSuchObjectException o4)
Review comment:
this rpc call is there to run the metastore side `translator` - might be
better to not dedicate the method call to "ctas"
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDefaultTransformer.java
##
@@ -632,37 +631,29 @@ public Table transformCreateTable(Table table,
List processorCapabilitie
throw new MetaException("Database " + dbName + " for table " +
table.getTableName() + " could not be found");
}
-if (TableType.MANAGED_TABLE.name().equals(tableType)) {
+ if (TableType.MANAGED_TABLE.name().equals(tableType)) {
LOG.debug("Table is a MANAGED_TABLE");
txnal = params.get(TABLE_IS_TRANSACTIONAL);
txn_properties = params.get(TABLE_TRANSACTIONAL_PROPERTIES);
- boolean ctas = Boolean.valueOf(params.getOrDefault(TABLE_IS_CTAS,
"false"));
isInsertAcid = (txn_properties != null &&
txn_properties.equalsIgnoreCase("insert_only"));
if ((txnal == null || txnal.equalsIgnoreCase("FALSE")) && !isInsertAcid)
{ // non-ACID MANAGED TABLE
-if (ctas) {
- LOG.info("Not Converting CTAS table " + newTable.getTableName() + "
to EXTERNAL tableType for " + processorId);
-} else {
- LOG.info("Converting " + newTable.getTableName() + " to EXTERNAL
tableType for " + processorId);
- newTable.setTableType(TableType.EXTERNAL_TABLE.toString());
- params.remove(TABLE_IS_TRANSACTIONAL);
- params.remove(TABLE_TRANSACTIONAL_PROPERTIES);
- params.put("EXTERNAL", "TRUE");
- params.put(EXTERNAL_TABLE_PURGE, "TRUE");
- params.put("TRANSLATED_TO_EXTERNAL", "TRUE");
- newTable.setParameters(params);
- LOG.info("Modified table params are:" + params.toString());
-
- if (getLocation(table) == null) {
-try {
- Path location = getTranslatedToExternalTableDefaultLocation(db,
newTable);
- newTable.getSd().setLocation(location.toString());
-} catch (Exception e) {
- throw new MetaException("Exception determining external table
location:" + e.getMessage());
-}
- } else {
-// table with explicitly set location
-// has "translated" properties and will be removed on drop
-// should we check tbl directory existence?
+LOG.info("Converting " + newTable.getTableName() + " to EXTERNAL
tableType for " + processorId);
+newTable.setTableType(TableType.EXTERNAL_TABLE.toString());
+params.remove(TABLE_IS_TRANSACTIONAL);
+params.remove(TABLE_TRANSACTIONAL_PROPERTIES);
+params.put("EXTERNAL", "TRUE");
+params.put(EXTERNAL_TABLE_PURGE, "TRUE");
+params.put("TRANSLATED_TO_EXTERNAL", "TRUE");
+newTable.setParameters(params);
+LOG.info("Modified table params are:" + params.toString());
+
+if (!table.isSetSd() || table.getSd().getLocation() == null) {
+ try {
+Path newPath = hmsHandler.getWh().getDefaultTablePath(db,
table.getTableName(), true);
Review comment:
one of its tests are broken:
http://ci.hive.apache.org/job/hive-precommit/job/PR-2442/11/testReport/org.apache.hadoop.hive.cli/TestNegativeLlapLocalCliDriver/Testing___split_18___PostProcess___testCliDriver_translated_external_rename_/
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDefaultTransformer.java
##
@@ -632,37 +631,29 @@ public Table transformCreateTable(Table table,
List processorCapabilitie
throw new MetaException("Database " + dbName + " for table " +
table.getTableName() + " could not be found");
}
-if (TableType.MANAGED_TABLE.name().equals(tableType)) {
+ if (TableType.MANAGED_TABLE.name().equals(tableType)) {
[jira] [Updated] (HIVE-25487) Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to find class :S_4
[ https://issues.apache.org/jira/browse/HIVE-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chengxinpeng updated HIVE-25487: Priority: Blocker (was: Major) > Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to > find class :S_4 > - > > Key: HIVE-25487 > URL: https://issues.apache.org/jira/browse/HIVE-25487 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: chengxinpeng >Assignee: chengxinpeng >Priority: Blocker > Labels: patch, pull-request-available > Attachments: 微信图片_20210827223829.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > at java.lang.CLassloader.loadClass(Classloader.java:351) > at java.lang.Class.forName0(Native Hethod) > at java.lang.Class.forName(Class.Java:348) > at org > apache.hive.com.esotericsoftware.kryo.utii.DefaultClassResolver.readName(DefaultClassResolver.java:154) > > ... 63 more > 2021-08-26 09:27:57,158[INFO[App Shared Poo1 -#1 > ]|dag.RootInputinitializerManager: Failed Inputinitiatizer for Input: > _dumy_tahle on vertex vertex_162774552112_1545_1_00 []Map 1] > 2021-08-26 09:27:57,159[ERROR(Dispatcher thread (Central) impl.VertexImpl|: > Vertex Input:dumy table initializer failed, > vertexs=vertex_1627745521112_1545_1_00 [Map 1] > org.apache.tez.dag.app.dag.impl.AMUserCodeException: > java.lang.RuntimeException: failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult > RootInputinitializerManager.Tava:158) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$1(RootInputInitializerManager.java:132) > at java.util.concurrent.Executors$RunnableAdapter.cal1(Executors.java:511) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutuzeTask.java:125) > > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) > > at > com.google.common.util.concurrent.TrustedListenableFutureTaak.run(TruseedListenableFutureTank.Java:78) > > at > java.uti1.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jave:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException Failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > > at org apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:528) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork (Utilities.java:359) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat. > java:442) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputformat.getSplits(CombineHiveInputFormat.java:508) > > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.genecateOldSplits(MRInputHelpers. > java:489) > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateinputSplitsToMem(MRInputHelpers.java:338) > at > org.apache.tez.mapreduce.common.MRInputAMSp1itGenerator.initialize(MRInputAMSplitGenerator.java:121) > at > org.apache.tez.dag.app.dag.RootInputInitializerianager.lambda$runInitializer$2(RootInputInitializerManager, > java:173) > at java.security.AccessController.doPrivileged (Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:166) > at > org.apache.tez.dag.app.dag.RootinputInitializerManager.runInitializerAndProcessResult(RootInputInitializertanager.java:147) > ...8 more > Caused by: org.apache.hive.com.esotericsoftwaze.kryo.KryoException: Unable to > find class S_4 > Serialization trace: > parentOperators (org.apache.hadoop.hive.ql.exec.fileSinkOperator) > childOperators (org.apache.hadoop.hive.ql.exec.UDTFOperacor) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasTowork (org.apache.hadoop.hive.ql.plan.Mapwork) > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolver.readName(DefaultClassResolver.java:133) > > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolvet.readClass(DefaultClassResolver.Java:156) > at org.apache.hive.com.ebotericsoftware.kryo.kryo.readClass (Kryo.java:670) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25487) Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to find class :S_4
[ https://issues.apache.org/jira/browse/HIVE-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chengxinpeng updated HIVE-25487: Labels: patch pull-request-available (was: beginner) > Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to > find class :S_4 > - > > Key: HIVE-25487 > URL: https://issues.apache.org/jira/browse/HIVE-25487 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: chengxinpeng >Assignee: chengxinpeng >Priority: Major > Labels: patch, pull-request-available > Attachments: 微信图片_20210827223829.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > at java.lang.CLassloader.loadClass(Classloader.java:351) > at java.lang.Class.forName0(Native Hethod) > at java.lang.Class.forName(Class.Java:348) > at org > apache.hive.com.esotericsoftware.kryo.utii.DefaultClassResolver.readName(DefaultClassResolver.java:154) > > ... 63 more > 2021-08-26 09:27:57,158[INFO[App Shared Poo1 -#1 > ]|dag.RootInputinitializerManager: Failed Inputinitiatizer for Input: > _dumy_tahle on vertex vertex_162774552112_1545_1_00 []Map 1] > 2021-08-26 09:27:57,159[ERROR(Dispatcher thread (Central) impl.VertexImpl|: > Vertex Input:dumy table initializer failed, > vertexs=vertex_1627745521112_1545_1_00 [Map 1] > org.apache.tez.dag.app.dag.impl.AMUserCodeException: > java.lang.RuntimeException: failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult > RootInputinitializerManager.Tava:158) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$1(RootInputInitializerManager.java:132) > at java.util.concurrent.Executors$RunnableAdapter.cal1(Executors.java:511) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutuzeTask.java:125) > > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) > > at > com.google.common.util.concurrent.TrustedListenableFutureTaak.run(TruseedListenableFutureTank.Java:78) > > at > java.uti1.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jave:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException Failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > > at org apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:528) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork (Utilities.java:359) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat. > java:442) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputformat.getSplits(CombineHiveInputFormat.java:508) > > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.genecateOldSplits(MRInputHelpers. > java:489) > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateinputSplitsToMem(MRInputHelpers.java:338) > at > org.apache.tez.mapreduce.common.MRInputAMSp1itGenerator.initialize(MRInputAMSplitGenerator.java:121) > at > org.apache.tez.dag.app.dag.RootInputInitializerianager.lambda$runInitializer$2(RootInputInitializerManager, > java:173) > at java.security.AccessController.doPrivileged (Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:166) > at > org.apache.tez.dag.app.dag.RootinputInitializerManager.runInitializerAndProcessResult(RootInputInitializertanager.java:147) > ...8 more > Caused by: org.apache.hive.com.esotericsoftwaze.kryo.KryoException: Unable to > find class S_4 > Serialization trace: > parentOperators (org.apache.hadoop.hive.ql.exec.fileSinkOperator) > childOperators (org.apache.hadoop.hive.ql.exec.UDTFOperacor) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasTowork (org.apache.hadoop.hive.ql.plan.Mapwork) > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolver.readName(DefaultClassResolver.java:133) > > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolvet.readClass(DefaultClassResolver.Java:156) > at org.apache.hive.com.ebotericsoftware.kryo.kryo.readClass (Kryo.java:670) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25487) Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to find class :S_4
[ https://issues.apache.org/jira/browse/HIVE-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405910#comment-17405910 ] chengxinpeng commented on HIVE-25487: - Dear community members, when I was using the CDP version of hive tez engine, this problem was unstable when submitting SQL containing udtf query function. I hope I can get your help > Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to > find class :S_4 > - > > Key: HIVE-25487 > URL: https://issues.apache.org/jira/browse/HIVE-25487 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: chengxinpeng >Assignee: chengxinpeng >Priority: Major > Labels: beginner > Attachments: 微信图片_20210827223829.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > at java.lang.CLassloader.loadClass(Classloader.java:351) > at java.lang.Class.forName0(Native Hethod) > at java.lang.Class.forName(Class.Java:348) > at org > apache.hive.com.esotericsoftware.kryo.utii.DefaultClassResolver.readName(DefaultClassResolver.java:154) > > ... 63 more > 2021-08-26 09:27:57,158[INFO[App Shared Poo1 -#1 > ]|dag.RootInputinitializerManager: Failed Inputinitiatizer for Input: > _dumy_tahle on vertex vertex_162774552112_1545_1_00 []Map 1] > 2021-08-26 09:27:57,159[ERROR(Dispatcher thread (Central) impl.VertexImpl|: > Vertex Input:dumy table initializer failed, > vertexs=vertex_1627745521112_1545_1_00 [Map 1] > org.apache.tez.dag.app.dag.impl.AMUserCodeException: > java.lang.RuntimeException: failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult > RootInputinitializerManager.Tava:158) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$1(RootInputInitializerManager.java:132) > at java.util.concurrent.Executors$RunnableAdapter.cal1(Executors.java:511) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutuzeTask.java:125) > > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) > > at > com.google.common.util.concurrent.TrustedListenableFutureTaak.run(TruseedListenableFutureTank.Java:78) > > at > java.uti1.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jave:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException Failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > > at org apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:528) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork (Utilities.java:359) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat. > java:442) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputformat.getSplits(CombineHiveInputFormat.java:508) > > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.genecateOldSplits(MRInputHelpers. > java:489) > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateinputSplitsToMem(MRInputHelpers.java:338) > at > org.apache.tez.mapreduce.common.MRInputAMSp1itGenerator.initialize(MRInputAMSplitGenerator.java:121) > at > org.apache.tez.dag.app.dag.RootInputInitializerianager.lambda$runInitializer$2(RootInputInitializerManager, > java:173) > at java.security.AccessController.doPrivileged (Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:166) > at > org.apache.tez.dag.app.dag.RootinputInitializerManager.runInitializerAndProcessResult(RootInputInitializertanager.java:147) > ...8 more > Caused by: org.apache.hive.com.esotericsoftwaze.kryo.KryoException: Unable to > find class S_4 > Serialization trace: > parentOperators (org.apache.hadoop.hive.ql.exec.fileSinkOperator) > childOperators (org.apache.hadoop.hive.ql.exec.UDTFOperacor) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasTowork (org.apache.hadoop.hive.ql.plan.Mapwork) > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolver.readName(DefaultClassResolver.java:133) > > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolvet.readClass(DefaultClassResolver.Java:156) > at
[jira] [Assigned] (HIVE-25487) Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to find class :S_4
[ https://issues.apache.org/jira/browse/HIVE-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chengxinpeng reassigned HIVE-25487: --- Assignee: chengxinpeng > Caused by :org.apache.hive.com.esotericsoftware.kryo.KryoException:Unable to > find class :S_4 > - > > Key: HIVE-25487 > URL: https://issues.apache.org/jira/browse/HIVE-25487 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: chengxinpeng >Assignee: chengxinpeng >Priority: Major > Labels: beginner > Attachments: 微信图片_20210827223829.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > at java.lang.CLassloader.loadClass(Classloader.java:351) > at java.lang.Class.forName0(Native Hethod) > at java.lang.Class.forName(Class.Java:348) > at org > apache.hive.com.esotericsoftware.kryo.utii.DefaultClassResolver.readName(DefaultClassResolver.java:154) > > ... 63 more > 2021-08-26 09:27:57,158[INFO[App Shared Poo1 -#1 > ]|dag.RootInputinitializerManager: Failed Inputinitiatizer for Input: > _dumy_tahle on vertex vertex_162774552112_1545_1_00 []Map 1] > 2021-08-26 09:27:57,159[ERROR(Dispatcher thread (Central) impl.VertexImpl|: > Vertex Input:dumy table initializer failed, > vertexs=vertex_1627745521112_1545_1_00 [Map 1] > org.apache.tez.dag.app.dag.impl.AMUserCodeException: > java.lang.RuntimeException: failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult > RootInputinitializerManager.Tava:158) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$1(RootInputInitializerManager.java:132) > at java.util.concurrent.Executors$RunnableAdapter.cal1(Executors.java:511) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutuzeTask.java:125) > > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) > > at > com.google.common.util.concurrent.TrustedListenableFutureTaak.run(TruseedListenableFutureTank.Java:78) > > at > java.uti1.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jave:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException Failed to load plan: > hdfs://nameservicetenant/tmp/hive/hive/8fblf9db-f922-4e31-af4a-12abb4ba405/hive_2021-08-26 > > at org apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:528) > at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork (Utilities.java:359) > at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat. > java:442) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputformat.getSplits(CombineHiveInputFormat.java:508) > > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.genecateOldSplits(MRInputHelpers. > java:489) > at > org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateinputSplitsToMem(MRInputHelpers.java:338) > at > org.apache.tez.mapreduce.common.MRInputAMSp1itGenerator.initialize(MRInputAMSplitGenerator.java:121) > at > org.apache.tez.dag.app.dag.RootInputInitializerianager.lambda$runInitializer$2(RootInputInitializerManager, > java:173) > at java.security.AccessController.doPrivileged (Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:166) > at > org.apache.tez.dag.app.dag.RootinputInitializerManager.runInitializerAndProcessResult(RootInputInitializertanager.java:147) > ...8 more > Caused by: org.apache.hive.com.esotericsoftwaze.kryo.KryoException: Unable to > find class S_4 > Serialization trace: > parentOperators (org.apache.hadoop.hive.ql.exec.fileSinkOperator) > childOperators (org.apache.hadoop.hive.ql.exec.UDTFOperacor) > childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator) > childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) > aliasTowork (org.apache.hadoop.hive.ql.plan.Mapwork) > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolver.readName(DefaultClassResolver.java:133) > > at > org.apache.hive.com.esotericsoftware.kryo.uti1.DefaultClassResolvet.readClass(DefaultClassResolver.Java:156) > at org.apache.hive.com.ebotericsoftware.kryo.kryo.readClass (Kryo.java:670) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25454) Invalid request to metadata catalog for a WITH clause table
[ https://issues.apache.org/jira/browse/HIVE-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa resolved HIVE-25454. --- Resolution: Fixed Pushed to master. Thanks [~soumyakanti.das]. > Invalid request to metadata catalog for a WITH clause table > --- > > Key: HIVE-25454 > URL: https://issues.apache.org/jira/browse/HIVE-25454 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > For CTEs, there are many calls to get_table_req(), which throws a > NoSuchObjectException and return null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25454) Invalid request to metadata catalog for a WITH clause table
[ https://issues.apache.org/jira/browse/HIVE-25454?focusedWorklogId=642899=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642899 ] ASF GitHub Bot logged work on HIVE-25454: - Author: ASF GitHub Bot Created on: 27/Aug/21 16:27 Start Date: 27/Aug/21 16:27 Worklog Time Spent: 10m Work Description: kasakrisz merged pull request #2587: URL: https://github.com/apache/hive/pull/2587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642899) Time Spent: 0.5h (was: 20m) > Invalid request to metadata catalog for a WITH clause table > --- > > Key: HIVE-25454 > URL: https://issues.apache.org/jira/browse/HIVE-25454 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > For CTEs, there are many calls to get_table_req(), which throws a > NoSuchObjectException and return null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25454) Invalid request to metadata catalog for a WITH clause table
[ https://issues.apache.org/jira/browse/HIVE-25454?focusedWorklogId=642883=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642883 ] ASF GitHub Bot logged work on HIVE-25454: - Author: ASF GitHub Bot Created on: 27/Aug/21 15:42 Start Date: 27/Aug/21 15:42 Worklog Time Spent: 10m Work Description: kasakrisz commented on pull request #2587: URL: https://github.com/apache/hive/pull/2587#issuecomment-907294977 @soumyakanti3578 Could you please check if there is test coverage for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642883) Time Spent: 20m (was: 10m) > Invalid request to metadata catalog for a WITH clause table > --- > > Key: HIVE-25454 > URL: https://issues.apache.org/jira/browse/HIVE-25454 > Project: Hive > Issue Type: Bug >Reporter: Soumyakanti Das >Assignee: Soumyakanti Das >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > For CTEs, there are many calls to get_table_req(), which throws a > NoSuchObjectException and return null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=642851=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642851
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 14:41
Start Date: 27/Aug/21 14:41
Worklog Time Spent: 10m
Work Description: kgyrtkirk opened a new pull request #2608:
URL: https://github.com/apache/hive/pull/2608
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642851)
Remaining Estimate: 0h
Time Spent: 10m
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25485:
--
Labels: pull-request-available (was: )
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405848#comment-17405848 ] Marton Bod commented on HIVE-25486: --- Pushed to master. Thanks [~pvary] for the review! > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod resolved HIVE-25486. --- Resolution: Fixed > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25486: -- Labels: pull-request-available (was: ) > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?focusedWorklogId=642843=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642843 ] ASF GitHub Bot logged work on HIVE-25486: - Author: ASF GitHub Bot Created on: 27/Aug/21 14:03 Start Date: 27/Aug/21 14:03 Worklog Time Spent: 10m Work Description: marton-bod merged pull request #2603: URL: https://github.com/apache/hive/pull/2603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642843) Remaining Estimate: 0h Time Spent: 10m > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25430) compactor.Worker.markFailed should catch and log any kind of exception
[ https://issues.apache.org/jira/browse/HIVE-25430?focusedWorklogId=642817=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642817 ] ASF GitHub Bot logged work on HIVE-25430: - Author: ASF GitHub Bot Created on: 27/Aug/21 13:01 Start Date: 27/Aug/21 13:01 Worklog Time Spent: 10m Work Description: klcopp merged pull request #2562: URL: https://github.com/apache/hive/pull/2562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642817) Time Spent: 20m (was: 10m) > compactor.Worker.markFailed should catch and log any kind of exception > -- > > Key: HIVE-25430 > URL: https://issues.apache.org/jira/browse/HIVE-25430 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25430) compactor.Worker.markFailed should catch and log any kind of exception
[ https://issues.apache.org/jira/browse/HIVE-25430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage resolved HIVE-25430. -- Fix Version/s: 4.0.0 Resolution: Fixed Committed to master branch. Thanks for reviewing [~dkuzmenko]! > compactor.Worker.markFailed should catch and log any kind of exception > -- > > Key: HIVE-25430 > URL: https://issues.apache.org/jira/browse/HIVE-25430 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25476) Remove Unused Dependencies for JDBC Driver
[ https://issues.apache.org/jira/browse/HIVE-25476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor resolved HIVE-25476. --- Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master. Thank you [~mgergely] for the review! > Remove Unused Dependencies for JDBC Driver > -- > > Key: HIVE-25476 > URL: https://issues.apache.org/jira/browse/HIVE-25476 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > I am using JDBC driver in a project and was very surprised by the number of > dependencies it has. Remove some unnecessary dependencies to make it a > little easier to work with. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25476) Remove Unused Dependencies for JDBC Driver
[ https://issues.apache.org/jira/browse/HIVE-25476?focusedWorklogId=642816=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642816 ] ASF GitHub Bot logged work on HIVE-25476: - Author: ASF GitHub Bot Created on: 27/Aug/21 12:54 Start Date: 27/Aug/21 12:54 Worklog Time Spent: 10m Work Description: belugabehr merged pull request #2599: URL: https://github.com/apache/hive/pull/2599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642816) Time Spent: 0.5h (was: 20m) > Remove Unused Dependencies for JDBC Driver > -- > > Key: HIVE-25476 > URL: https://issues.apache.org/jira/browse/HIVE-25476 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > I am using JDBC driver in a project and was very surprised by the number of > dependencies it has. Remove some unnecessary dependencies to make it a > little easier to work with. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25450) Delta metrics keys should contain database name
[
https://issues.apache.org/jira/browse/HIVE-25450?focusedWorklogId=642815=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642815
]
ASF GitHub Bot logged work on HIVE-25450:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 12:52
Start Date: 27/Aug/21 12:52
Worklog Time Spent: 10m
Work Description: klcopp commented on a change in pull request #2607:
URL: https://github.com/apache/hive/pull/2607#discussion_r697415476
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/metrics/DeltaFilesMetricReporter.java
##
@@ -240,19 +245,24 @@ public static void mergeDeltaFilesStats(AcidDirectory
dir, long checkThresholdIn
String path = getRelPath(dir);
-filterAndAddToDeltaFilesStats(NUM_DELTAS, numDeltas, deltasThreshold,
deltaFilesStats, path, maxCacheSize);
+String serializedMetadata =
conf.get(JOB_CONF_DELTA_FILES_METRICS_METADATA);
+HashMap pathToMetadata = new HashMap<>();
+pathToMetadata =
SerializationUtilities.deserializeObject(serializedMetadata,
pathToMetadata.getClass());
Review comment:
deserializeObject could return null
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java
##
@@ -224,6 +228,24 @@ private void prepare(InputInitializerContext
initializerContext) throws IOExcept
}
+HashMap
deltaFilesMetadata = new HashMap();
+
+work.getPathToPartitionInfo().entrySet().forEach(e -> {
+ DeltaFilesMetricReporter.DeltaFilesMetadata metadata = new
DeltaFilesMetricReporter.DeltaFilesMetadata();
+ TableDesc tableDesc = e.getValue().getTableDesc();
+ metadata.dbName = tableDesc.getDbName();
+ metadata.tableName = tableDesc.getTableName();
+ LinkedHashMap partSpec = e.getValue().getPartSpec();
+ if (partSpec != null && !partSpec.isEmpty()) {
+metadata.partitionName = String.valueOf(partSpec);
+ }
+ deltaFilesMetadata.put(e.getKey(), metadata);
+});
+
+String serializedMetadata =
SerializationUtilities.serializeObject(deltaFilesMetadata);
Review comment:
serializeObject could return null?
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/metrics/DeltaFilesMetricReporter.java
##
@@ -240,19 +245,24 @@ public static void mergeDeltaFilesStats(AcidDirectory
dir, long checkThresholdIn
String path = getRelPath(dir);
Review comment:
path isn't used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642815)
Time Spent: 20m (was: 10m)
> Delta metrics keys should contain database name
> ---
>
> Key: HIVE-25450
> URL: https://issues.apache.org/jira/browse/HIVE-25450
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently metrics about the number of deltas in a given partition or
> unpartitioned table include information about the table name and the
> partition name (if applicable), but they should also include the database
> name, since there could be 2 tables in different databases with the same name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-25486: -- Summary: Upgrade to Iceberg 0.12.0 (was: Upgrade to I) > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25486) Upgrade to Iceberg 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-25486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod reassigned HIVE-25486: - Assignee: Marton Bod > Upgrade to Iceberg 0.12.0 > - > > Key: HIVE-25486 > URL: https://issues.apache.org/jira/browse/HIVE-25486 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25450) Delta metrics keys should contain database name
[ https://issues.apache.org/jira/browse/HIVE-25450?focusedWorklogId=642785=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642785 ] ASF GitHub Bot logged work on HIVE-25450: - Author: ASF GitHub Bot Created on: 27/Aug/21 10:47 Start Date: 27/Aug/21 10:47 Worklog Time Spent: 10m Work Description: lcspinter opened a new pull request #2607: URL: https://github.com/apache/hive/pull/2607 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642785) Remaining Estimate: 0h Time Spent: 10m > Delta metrics keys should contain database name > --- > > Key: HIVE-25450 > URL: https://issues.apache.org/jira/browse/HIVE-25450 > Project: Hive > Issue Type: Sub-task >Reporter: Karen Coppage >Assignee: László Pintér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently metrics about the number of deltas in a given partition or > unpartitioned table include information about the table name and the > partition name (if applicable), but they should also include the database > name, since there could be 2 tables in different databases with the same name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25450) Delta metrics keys should contain database name
[ https://issues.apache.org/jira/browse/HIVE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25450: -- Labels: pull-request-available (was: ) > Delta metrics keys should contain database name > --- > > Key: HIVE-25450 > URL: https://issues.apache.org/jira/browse/HIVE-25450 > Project: Hive > Issue Type: Sub-task >Reporter: Karen Coppage >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently metrics about the number of deltas in a given partition or > unpartitioned table include information about the table name and the > partition name (if applicable), but they should also include the database > name, since there could be 2 tables in different databases with the same name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade
[ https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405737#comment-17405737 ] Ranith Sardar commented on HIVE-22818: -- Hi [~lpinter], Is there any planning for jetty 9.4.26 upgrade in Hive Master branch? > Preparation for jetty 9.4.26 upgrade > > > Key: HIVE-22818 > URL: https://issues.apache.org/jira/browse/HIVE-22818 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: László Pintér >Assignee: László Pintér >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch, > HIVE-22818.03.patch > > > Make some code adjustment, before upgrading jetty to 9.4.26. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25450) Delta metrics keys should contain database name
[ https://issues.apache.org/jira/browse/HIVE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Pintér reassigned HIVE-25450: Assignee: László Pintér > Delta metrics keys should contain database name > --- > > Key: HIVE-25450 > URL: https://issues.apache.org/jira/browse/HIVE-25450 > Project: Hive > Issue Type: Sub-task >Reporter: Karen Coppage >Assignee: László Pintér >Priority: Major > > Currently metrics about the number of deltas in a given partition or > unpartitioned table include information about the table name and the > partition name (if applicable), but they should also include the database > name, since there could be 2 tables in different databases with the same name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23688) Vectorization: IndexArrayOutOfBoundsException For map type column which includes null value
[
https://issues.apache.org/jira/browse/HIVE-23688?focusedWorklogId=642753=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642753
]
ASF GitHub Bot logged work on HIVE-23688:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 09:20
Start Date: 27/Aug/21 09:20
Worklog Time Spent: 10m
Work Description: abstractdog closed pull request #1122:
URL: https://github.com/apache/hive/pull/1122
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642753)
Time Spent: 4h 10m (was: 4h)
> Vectorization: IndexArrayOutOfBoundsException For map type column which
> includes null value
> ---
>
> Key: HIVE-23688
> URL: https://issues.apache.org/jira/browse/HIVE-23688
> Project: Hive
> Issue Type: Bug
> Components: Parquet, storage-api, Vectorization
>Affects Versions: All Versions
>Reporter: 范宜臻
>Assignee: László Bodor
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.0.0, 4.0.0
>
> Attachments: HIVE-23688.patch
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> {color:#de350b}start{color} and {color:#de350b}length{color} are empty arrays
> in MapColumnVector.values(BytesColumnVector) when values in map contain
> {color:#de350b}null{color}
> reproduce in master branch:
> {code:java}
> set hive.vectorized.execution.enabled=true;
> CREATE TABLE parquet_map_type (id int,stringMap map)
> stored as parquet;
> insert overwrite table parquet_map_typeSELECT 1, MAP('k1', null, 'k2',
> 'bar');
> select id, stringMap['k1'] from parquet_map_type group by 1,2;
> {code}
> query explain:
> {code:java}
> Stage-0
> Fetch Operator
> limit:-1
> Stage-1
> Reducer 2 vectorized
> File Output Operator [FS_12]
> Group By Operator [GBY_11] (rows=5 width=2)
> Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
> <-Map 1 [SIMPLE_EDGE] vectorized
> SHUFFLE [RS_10]
> PartitionCols:_col0, _col1
> Group By Operator [GBY_9] (rows=10 width=2)
> Output:["_col0","_col1"],keys:_col0, _col1
> Select Operator [SEL_8] (rows=10 width=2)
> Output:["_col0","_col1"]
> TableScan [TS_0] (rows=10 width=2)
>
> temp@parquet_map_type_fyz,parquet_map_type_fyz,Tbl:COMPLETE,Col:NONE,Output:["id","stringmap"]
> {code}
> runtime error:
> {code:java}
> Vertex failed, vertexName=Map 1, vertexId=vertex_1592040015150_0001_3_00,
> diagnostics=[Task failed, taskId=task_1592040015150_0001_3_00_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1592040015150_0001_3_00_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
[jira] [Resolved] (HIVE-25429) Delta metrics collection may cause number of tez counters to exceed tez.counters.max limit
[ https://issues.apache.org/jira/browse/HIVE-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage resolved HIVE-25429. -- Fix Version/s: 4.0.0 Resolution: Fixed Committed to master. Thanks [~lpinter] for the +1 and [~dkuzmenko] for the feedback! > Delta metrics collection may cause number of tez counters to exceed > tez.counters.max limit > -- > > Key: HIVE-25429 > URL: https://issues.apache.org/jira/browse/HIVE-25429 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There's a limit to the number of tez counters allowed (tez.counters.max). > Delta metrics collection (i.e. DeltaFileMetricsReporter) was creating 3 > counters for each partition touched by a given query, which can result in a > huge number of counters, which is unnecessary because we're only interested > in n number of partitions with the most deltas. This change limits the number > of counters created to hive.txn.acid.metrics.max.cache.size*3. > Also when tez.counters.max is reached a LimitExceededException is thrown but > isn't caught on the Hive side and causes the query to fail. We should catch > this and skip delta metrics collection in this case. > Also make sure that metrics are only collected if > hive.metastore.acidmetrics.ext.on=true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23016) Extract JdbcConnectionParams from Utils Class
[ https://issues.apache.org/jira/browse/HIVE-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405688#comment-17405688 ] Timur Malikin commented on HIVE-23016: -- PR added - https://github.com/apache/hive/pull/2606 > Extract JdbcConnectionParams from Utils Class > - > > Key: HIVE-23016 > URL: https://issues.apache.org/jira/browse/HIVE-23016 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Priority: Minor > Labels: n00b, newbie, noob, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > And make it its own class. > https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23016) Extract JdbcConnectionParams from Utils Class
[ https://issues.apache.org/jira/browse/HIVE-23016?focusedWorklogId=642740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642740 ] ASF GitHub Bot logged work on HIVE-23016: - Author: ASF GitHub Bot Created on: 27/Aug/21 08:56 Start Date: 27/Aug/21 08:56 Worklog Time Spent: 10m Work Description: malikin opened a new pull request #2606: URL: https://github.com/apache/hive/pull/2606 ### What changes were proposed in this pull request? Extract JdbcConnectionParams from Utils Class as mentioned here https://issues.apache.org/jira/browse/HIVE-23016 ### Why are the changes needed? Close https://issues.apache.org/jira/browse/HIVE-23016 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? All tests and codestyle checks passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642740) Remaining Estimate: 0h Time Spent: 10m > Extract JdbcConnectionParams from Utils Class > - > > Key: HIVE-23016 > URL: https://issues.apache.org/jira/browse/HIVE-23016 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Priority: Minor > Labels: n00b, newbie, noob > Time Spent: 10m > Remaining Estimate: 0h > > And make it its own class. > https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23016) Extract JdbcConnectionParams from Utils Class
[ https://issues.apache.org/jira/browse/HIVE-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23016: -- Labels: n00b newbie noob pull-request-available (was: n00b newbie noob) > Extract JdbcConnectionParams from Utils Class > - > > Key: HIVE-23016 > URL: https://issues.apache.org/jira/browse/HIVE-23016 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Priority: Minor > Labels: n00b, newbie, noob, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > And make it its own class. > https://github.com/apache/hive/blob/4700e210ef7945278c4eb313c9ebd810b0224da1/jdbc/src/java/org/apache/hive/jdbc/Utils.java#L72 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25429) Delta metrics collection may cause number of tez counters to exceed tez.counters.max limit
[ https://issues.apache.org/jira/browse/HIVE-25429?focusedWorklogId=642726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642726 ] ASF GitHub Bot logged work on HIVE-25429: - Author: ASF GitHub Bot Created on: 27/Aug/21 07:59 Start Date: 27/Aug/21 07:59 Worklog Time Spent: 10m Work Description: klcopp merged pull request #2563: URL: https://github.com/apache/hive/pull/2563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642726) Time Spent: 1h 40m (was: 1.5h) > Delta metrics collection may cause number of tez counters to exceed > tez.counters.max limit > -- > > Key: HIVE-25429 > URL: https://issues.apache.org/jira/browse/HIVE-25429 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > There's a limit to the number of tez counters allowed (tez.counters.max). > Delta metrics collection (i.e. DeltaFileMetricsReporter) was creating 3 > counters for each partition touched by a given query, which can result in a > huge number of counters, which is unnecessary because we're only interested > in n number of partitions with the most deltas. This change limits the number > of counters created to hive.txn.acid.metrics.max.cache.size*3. > Also when tez.counters.max is reached a LimitExceededException is thrown but > isn't caught on the Hive side and causes the query to fail. We should catch > this and skip delta metrics collection in this case. > Also make sure that metrics are only collected if > hive.metastore.acidmetrics.ext.on=true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25461) Add a test case to ensure Truncate table advances the write ID
[ https://issues.apache.org/jira/browse/HIVE-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25461: -- Labels: pull-request-available (was: ) > Add a test case to ensure Truncate table advances the write ID > -- > > Key: HIVE-25461 > URL: https://issues.apache.org/jira/browse/HIVE-25461 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25461) Add a test case to ensure Truncate table advances the write ID
[ https://issues.apache.org/jira/browse/HIVE-25461?focusedWorklogId=642718=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642718 ] ASF GitHub Bot logged work on HIVE-25461: - Author: ASF GitHub Bot Created on: 27/Aug/21 07:19 Start Date: 27/Aug/21 07:19 Worklog Time Spent: 10m Work Description: pvary merged pull request #2594: URL: https://github.com/apache/hive/pull/2594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642718) Remaining Estimate: 0h Time Spent: 10m > Add a test case to ensure Truncate table advances the write ID > -- > > Key: HIVE-25461 > URL: https://issues.apache.org/jira/browse/HIVE-25461 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25461) Add a test case to ensure Truncate table advances the write ID
[ https://issues.apache.org/jira/browse/HIVE-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary resolved HIVE-25461. --- Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master. Thanks for the patch [~kishendas]! > Add a test case to ensure Truncate table advances the write ID > -- > > Key: HIVE-25461 > URL: https://issues.apache.org/jira/browse/HIVE-25461 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25407) Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED LOCATION, UNSET SERDEPROPERTIES)
[ https://issues.apache.org/jira/browse/HIVE-25407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25407: -- Labels: pull-request-available (was: ) > Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED > LOCATION, UNSET SERDEPROPERTIES) > > > Key: HIVE-25407 > URL: https://issues.apache.org/jira/browse/HIVE-25407 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Below DDLs should be investigated separately on why the advancing the write > ID is not working for transactional tables, even after adding the logic to > advance the write ID. > * -ALTER TABLE SET PARTITION SPEC- > * ALTER TABLE UNSET SERDEPROPERTIES > * ALTER TABLE NOT SKEWED > * -ALTER TABLE COMPACT- > * ALTER TABLE SKEWED BY > * ALTER TABLE SET SKEWED LOCATION -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25407) Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED LOCATION, UNSET SERDEPROPERTIES)
[ https://issues.apache.org/jira/browse/HIVE-25407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary resolved HIVE-25407. --- Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master. Thanks for the patch [~kishendas]! > Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED > LOCATION, UNSET SERDEPROPERTIES) > > > Key: HIVE-25407 > URL: https://issues.apache.org/jira/browse/HIVE-25407 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Below DDLs should be investigated separately on why the advancing the write > ID is not working for transactional tables, even after adding the logic to > advance the write ID. > * -ALTER TABLE SET PARTITION SPEC- > * ALTER TABLE UNSET SERDEPROPERTIES > * ALTER TABLE NOT SKEWED > * -ALTER TABLE COMPACT- > * ALTER TABLE SKEWED BY > * ALTER TABLE SET SKEWED LOCATION -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25407) Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED LOCATION, UNSET SERDEPROPERTIES)
[ https://issues.apache.org/jira/browse/HIVE-25407?focusedWorklogId=642717=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642717 ] ASF GitHub Bot logged work on HIVE-25407: - Author: ASF GitHub Bot Created on: 27/Aug/21 07:18 Start Date: 27/Aug/21 07:18 Worklog Time Spent: 10m Work Description: pvary merged pull request #2597: URL: https://github.com/apache/hive/pull/2597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 642717) Remaining Estimate: 0h Time Spent: 10m > Advance Write ID during ALTER TABLE ( NOT SKEWED, SKEWED BY, SET SKEWED > LOCATION, UNSET SERDEPROPERTIES) > > > Key: HIVE-25407 > URL: https://issues.apache.org/jira/browse/HIVE-25407 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Below DDLs should be investigated separately on why the advancing the write > ID is not working for transactional tables, even after adding the logic to > advance the write ID. > * -ALTER TABLE SET PARTITION SPEC- > * ALTER TABLE UNSET SERDEPROPERTIES > * ALTER TABLE NOT SKEWED > * -ALTER TABLE COMPACT- > * ALTER TABLE SKEWED BY > * ALTER TABLE SET SKEWED LOCATION -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25383) Make TestMarkPartitionRemote more stable
[
https://issues.apache.org/jira/browse/HIVE-25383?focusedWorklogId=642708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642708
]
ASF GitHub Bot logged work on HIVE-25383:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 06:23
Start Date: 27/Aug/21 06:23
Worklog Time Spent: 10m
Work Description: dengzhhu653 closed pull request #2528:
URL: https://github.com/apache/hive/pull/2528
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642708)
Time Spent: 20m (was: 10m)
> Make TestMarkPartitionRemote more stable
>
>
> Key: HIVE-25383
> URL: https://issues.apache.org/jira/browse/HIVE-25383
> Project: Hive
> Issue Type: Test
> Components: Standalone Metastore
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Sometimes the TestMarkPartitionRemote failed by
> {noformat}
> org.apache.hadoop.hive.metastore.api.MetaException: Exception determining
> external table location:Default location is not available for table:
> file:/path/to/table
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transformCreateTable(MetastoreDefaultTransformer.java:660)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_table_core(HMSHandler.java:2325)
> ~[classes/:?]
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_table_req(HMSHandler.java:2578)
> [classes/:?]{noformat}
> [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-2441/15/tests]
>
> [http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-2473/3/tests]
>
> The cause is that the table path is existed before the test executed,
> TableLocationStrategy with prohibit does not allow alternate locations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
