[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969375#comment-15969375 ] Anbu Cheeralan commented on HIVE-14798: --- [~premal] Sorry for the late reply. As mentioned in the comment section by [~chetna] in that ticket, this issue was not in the master branch then (Hive 2.1.1) and I verified in Hive 2.1.1 that this issue is not happening. > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926536#comment-15926536 ] Premal Shah commented on HIVE-14798: [~alunarbeach] that ticket is still unresolved at this time. > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907683#comment-15907683 ] Anbu Cheeralan commented on HIVE-14798: --- [~premal] https://issues.apache.org/jira/browse/HIVE-14772?focusedCommentId=15529759=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15529759 > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902698#comment-15902698 ] Premal Shah commented on HIVE-14798: [~alunarbeach] What's the JIRA in which this bug was fixed for Hive 2.1.1? I am looking the release notes for 2.1.1 and don't see any mention of repair table in the list https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12335838=Text=12310843 > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885889#comment-15885889 ] Anbu Cheeralan commented on HIVE-14798: --- [~trkylmz] Yes the problem is with dataproc 1.2 image which uses Hive 2.1.0. New Dataproc 1.2 Preview image uses Hive 2.1.1 and that does not have this problem. Thank you [~cwlaird3] for the information. > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882181#comment-15882181 ] Tarik Yilmaz commented on HIVE-14798: - We had the same problem too. Probably, this problem is irrelevant with Apache Hive because our solution removes Google's Hive build and install Apache Hive. {code} #!/bin/bash # Copyright 2017 Metglobal, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # #http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. HIVE_VERSION=2.1.1 ROLE=$(/usr/share/google/get_metadata_value attributes/dataproc-role) cd /tmp # Download Apache Hive build wget http://www-eu.apache.org/dist/hive/hive-${HIVE_VERSION}/apache-hive-${HIVE_VERSION}-bin.tar.gz # Extract Hive package tar -xf apache-hive-${HIVE_VERSION}-bin.tar.gz # Delete temporary Hive archive rm -rf apache-hive-${HIVE_VERSION}-bin.tar.gz rm -rf apache-hive-${HIVE_VERSION}-bin/conf rm apache-hive-${HIVE_VERSION}-bin/lib/guava-14.0.1.jar # Install Guava >= 15.0 for GoogleHadoopFileSystem wget http://central.maven.org/maven2/com/google/guava/guava/19.0/guava-19.0.jar mv guava-19.0.jar apache-hive-${HIVE_VERSION}-bin/lib/ if [[ "${ROLE}" == 'Master' ]]; then # Stop the current services service hive-metastore stop service hive-server2 stop fi # Remove current Hive rm -rf /usr/lib/hive # Install Apache Hive mv apache-hive-${HIVE_VERSION}-bin /usr/lib/hive # Create symlink for required remote files ln -s /etc/hive/conf /usr/lib/hive/conf ln -s /usr/lib/zookeeper/zookeeper.jar /usr/lib/hive/lib/zookeeper.jar ln -s /usr/share/java/mysql.jar /usr/lib/hive/lib/mysql.jar # Create symlink for Hive libraries ln -s /usr/lib/hive/lib/hive-accumulo-handler-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-accumulo-handler.jar ln -s /usr/lib/hive/lib/hive-ant-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-ant.jar ln -s /usr/lib/hive/lib/hive-beeline-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-beeline.jar ln -s /usr/lib/hive/lib/hive-cli-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-cli.jar ln -s /usr/lib/hive/lib/hive-common-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-common.jar ln -s /usr/lib/hive/lib/hive-contrib-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-contrib.jar ln -s /usr/lib/hive/lib/hive-exec-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-exec.jar ln -s /usr/lib/hive/lib/hive-hbase-handler-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-hbase-handler.jar ln -s /usr/lib/hive/lib/hive-hplsql-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-hplsql.jar ln -s /usr/lib/hive/lib/hive-hwi-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-hwi.jar ln -s /usr/lib/hive/lib/hive-jdbc-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-jdbc.jar ln -s /usr/lib/hive/lib/hive-llap-client-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-llap-client.jar ln -s /usr/lib/hive/lib/hive-llap-common-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-llap-common.jar ln -s /usr/lib/hive/lib/hive-llap-ext-client-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-llap-ext-client.jar ln -s /usr/lib/hive/lib/hive-llap-server-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-llap-server.jar ln -s /usr/lib/hive/lib/hive-llap-tez-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-llap-tez.jar ln -s /usr/lib/hive/lib/hive-metastore-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-metastore.jar ln -s /usr/lib/hive/lib/hive-orc-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-orc.jar ln -s /usr/lib/hive/lib/hive-serde-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-serde.jar ln -s /usr/lib/hive/lib/hive-service-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-service.jar ln -s /usr/lib/hive/lib/hive-service-rpc-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-service-rpc.jar ln -s /usr/lib/hive/lib/hive-shims-0.23-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-shims-0.23.jar ln -s /usr/lib/hive/lib/hive-shims-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-shims.jar ln -s /usr/lib/hive/lib/hive-shims-common-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-shims-common.jar ln -s /usr/lib/hive/lib/hive-shims-scheduler-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-shims-scheduler.jar ln -s /usr/lib/hive/lib/hive-storage-api-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-storage-api.jar ln -s /usr/lib/hive/lib/hive-testutils-${HIVE_VERSION}.jar /usr/lib/hive/lib/hive-testutils.jar if [[ "${ROLE}" == 'Master' ]]; then # Start Hive services again service hive-metastore start service hive-server2 start fi {code} > MSCK REPAIR TABLE throws null pointer exception >
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880356#comment-15880356 ] Kenneth MacArthur commented on HIVE-14798: -- We certainly saw this in Dataproc. > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878253#comment-15878253 ] Tarik Yilmaz commented on HIVE-14798: - [~alunarbeach] We've faced with this issue too on Dataproc. Did this error appear on Dataproc? > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14798) MSCK REPAIR TABLE throws null pointer exception
[ https://issues.apache.org/jira/browse/HIVE-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736022#comment-15736022 ] Carl Laird commented on HIVE-14798: --- This appears to be fixed in 2.1.1. > MSCK REPAIR TABLE throws null pointer exception > --- > > Key: HIVE-14798 > URL: https://issues.apache.org/jira/browse/HIVE-14798 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Anbu Cheeralan > > MSCK REPAIR TABLE statement throws null pointer exception in Hive 2.1 > I have tested the same against external/internal tables created both in HDFS > and in Google Cloud. > The error shown in beeline/sql client > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) > Hive Logs: > 2016-09-20T17:28:00,717 ERROR [HiveServer2-Background-Pool: Thread-92]: > metadata.HiveMetaStoreChecker (:()) - java.lang.NullPointerException > 2016-09-20T17:28:00,717 WARN [HiveServer2-Background-Pool: Thread-92]: > exec.DDLTask (:()) - Failed to run metacheck: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:444) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.getAllLeafDirs(HiveMetaStoreChecker.java:388) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.findUnknownPartitions(HiveMetaStoreChecker.java:309) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:285) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkTable(HiveMetaStoreChecker.java:230) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker.checkMetastore(HiveMetaStoreChecker.java:109) > at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1814) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:403) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > at > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:432) > at > org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker$1.call(HiveMetaStoreChecker.java:418) > ... 4 more > Here are the steps to recreate this issue: > use default; > DROP TABLE IF EXISTS repairtable; > CREATE TABLE repairtable(col STRING) PARTITIONED BY (p1 STRING, p2 STRING); > MSCK REPAIR TABLE default.repairtable; -- This message was sent by Atlassian JIRA (v6.3.4#6332)