[jira] [Commented] (HIVE-8915) Log file explosion due to non-existence of COMPACTION_QUEUE table
[ https://issues.apache.org/jira/browse/HIVE-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14528428#comment-14528428 ] Hive QA commented on HIVE-8915: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12729641/HIVE-8915.patch {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 8895 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.TestSQLStdHiveAccessControllerHS2.testConfigProcessing org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.TestSQLStdHiveAccessControllerHS2.testConfigProcessingCustomSetWhitelistAppend org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3733/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3733/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3733/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12729641 - PreCommit-HIVE-TRUNK-Build Log file explosion due to non-existence of COMPACTION_QUEUE table - Key: HIVE-8915 URL: https://issues.apache.org/jira/browse/HIVE-8915 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 0.14.0, 0.15.0, 0.14.1 Reporter: Sushanth Sowmyan Assignee: Alan Gates Attachments: HIVE-8915.patch I hit an issue with a fresh set up of hive in a vm, where I did not have db tables as specified by hive-txn-schema-0.14.0.mysql.sql created. On metastore startup, I got an endless loop of errors being populated to the log file, which caused the log file to grow to 1.7GB in 5 minutes, with 950k copies of the same error stack trace in it before I realized what was happening and killed it. We should either have a delay of sorts to make sure we don't endlessly respin on that error so quickly, or we should error out and fail if we're not able to start. The stack trace in question is as follows: {noformat} 2014-11-19 01:44:57,654 ERROR compactor.Cleaner (Cleaner.java:run(143)) - Caught an exception in the main loop of compactor cleaner, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'hive.COMPACTION_QUEUE' doesn't exist at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:266) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) ) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:291) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8915) Log file explosion due to non-existence of COMPACTION_QUEUE table
[ https://issues.apache.org/jira/browse/HIVE-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522644#comment-14522644 ] Eugene Koifman commented on HIVE-8915: -- +1 Log file explosion due to non-existence of COMPACTION_QUEUE table - Key: HIVE-8915 URL: https://issues.apache.org/jira/browse/HIVE-8915 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 0.14.0, 0.15.0, 0.14.1 Reporter: Sushanth Sowmyan Assignee: Alan Gates Attachments: HIVE-8915.patch I hit an issue with a fresh set up of hive in a vm, where I did not have db tables as specified by hive-txn-schema-0.14.0.mysql.sql created. On metastore startup, I got an endless loop of errors being populated to the log file, which caused the log file to grow to 1.7GB in 5 minutes, with 950k copies of the same error stack trace in it before I realized what was happening and killed it. We should either have a delay of sorts to make sure we don't endlessly respin on that error so quickly, or we should error out and fail if we're not able to start. The stack trace in question is as follows: {noformat} 2014-11-19 01:44:57,654 ERROR compactor.Cleaner (Cleaner.java:run(143)) - Caught an exception in the main loop of compactor cleaner, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'hive.COMPACTION_QUEUE' doesn't exist at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:266) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) ) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:291) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8915) Log file explosion due to non-existence of COMPACTION_QUEUE table
[ https://issues.apache.org/jira/browse/HIVE-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522397#comment-14522397 ] Sushanth Sowmyan commented on HIVE-8915: Approved for 1.2. Adding it to https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status Log file explosion due to non-existence of COMPACTION_QUEUE table - Key: HIVE-8915 URL: https://issues.apache.org/jira/browse/HIVE-8915 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 0.14.0, 0.15.0, 0.14.1 Reporter: Sushanth Sowmyan Assignee: Alan Gates Attachments: HIVE-8915.patch I hit an issue with a fresh set up of hive in a vm, where I did not have db tables as specified by hive-txn-schema-0.14.0.mysql.sql created. On metastore startup, I got an endless loop of errors being populated to the log file, which caused the log file to grow to 1.7GB in 5 minutes, with 950k copies of the same error stack trace in it before I realized what was happening and killed it. We should either have a delay of sorts to make sure we don't endlessly respin on that error so quickly, or we should error out and fail if we're not able to start. The stack trace in question is as follows: {noformat} 2014-11-19 01:44:57,654 ERROR compactor.Cleaner (Cleaner.java:run(143)) - Caught an exception in the main loop of compactor cleaner, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'hive.COMPACTION_QUEUE' doesn't exist at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:266) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) ) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:291) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8915) Log file explosion due to non-existence of COMPACTION_QUEUE table
[ https://issues.apache.org/jira/browse/HIVE-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390009#comment-14390009 ] Owen O'Malley commented on HIVE-8915: - Yeah, I hit this too. What is the fix, [~alangates]? Log file explosion due to non-existence of COMPACTION_QUEUE table - Key: HIVE-8915 URL: https://issues.apache.org/jira/browse/HIVE-8915 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 0.14.0, 0.15.0, 0.14.1 Reporter: Sushanth Sowmyan Assignee: Alan Gates I hit an issue with a fresh set up of hive in a vm, where I did not have db tables as specified by hive-txn-schema-0.14.0.mysql.sql created. On metastore startup, I got an endless loop of errors being populated to the log file, which caused the log file to grow to 1.7GB in 5 minutes, with 950k copies of the same error stack trace in it before I realized what was happening and killed it. We should either have a delay of sorts to make sure we don't endlessly respin on that error so quickly, or we should error out and fail if we're not able to start. The stack trace in question is as follows: {noformat} 2014-11-19 01:44:57,654 ERROR compactor.Cleaner (Cleaner.java:run(143)) - Caught an exception in the main loop of compactor cleaner, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'hive.COMPACTION_QUEUE' doesn't exist at sun.reflect.GeneratedConstructorAccessor20.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:266) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) ) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:291) at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:86) {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)