[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739405#comment-14739405 ] Sushanth Sowmyan commented on HIVE-11510: - +1, committing to branch-1 and master. > Metatool updateLocation warning on views > > > Key: HIVE-11510 > URL: https://issues.apache.org/jira/browse/HIVE-11510 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 0.14.0 >Reporter: Eric Czech >Assignee: Wei Zheng > Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch, > HIVE-11510.3.patch > > > If views are present in a hive database, issuing a 'hive metatool > -updateLocation' command will result in an error like this: > ... > Warning: Found records with bad LOCATION in SDS table.. > bad location URI: null > bad location URI: null > bad location URI: null > > Based on the source code for Metatool, it looks like there would then be a > "bad location URI: null" message for every view and it also appears this is > happening simply because the 'sds' table in the hive schema has a column > called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14729678#comment-14729678 ] Wei Zheng commented on HIVE-11510: -- TestCompactionTxnHandler runs clean locally. testTableSchemaPropagation is not related. [~sushanth] Could you commit the patch? > Metatool updateLocation warning on views > > > Key: HIVE-11510 > URL: https://issues.apache.org/jira/browse/HIVE-11510 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 0.14.0 >Reporter: Eric Czech >Assignee: Wei Zheng > Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch, > HIVE-11510.3.patch > > > If views are present in a hive database, issuing a 'hive metatool > -updateLocation' command will result in an error like this: > ... > Warning: Found records with bad LOCATION in SDS table.. > bad location URI: null > bad location URI: null > bad location URI: null > > Based on the source code for Metatool, it looks like there would then be a > "bad location URI: null" message for every view and it also appears this is > happening simply because the 'sds' table in the hive schema has a column > called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[
https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724731#comment-14724731
]
Hive QA commented on HIVE-11510:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12753344/HIVE-11510.3.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9383 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5127/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5127/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5127/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12753344 - PreCommit-HIVE-TRUNK-Build
> Metatool updateLocation warning on views
>
>
> Key: HIVE-11510
> URL: https://issues.apache.org/jira/browse/HIVE-11510
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 0.14.0
>Reporter: Eric Czech
>Assignee: Wei Zheng
> Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch,
> HIVE-11510.3.patch
>
>
> If views are present in a hive database, issuing a 'hive metatool
> -updateLocation' command will result in an error like this:
> ...
> Warning: Found records with bad LOCATION in SDS table..
> bad location URI: null
> bad location URI: null
> bad location URI: null
>
> Based on the source code for Metatool, it looks like there would then be a
> "bad location URI: null" message for every view and it also appears this is
> happening simply because the 'sds' table in the hive schema has a column
> called location that is NULL only for views.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723808#comment-14723808 ] Sushanth Sowmyan commented on HIVE-11510: - Thank you Wei, this looks good. +1. > Metatool updateLocation warning on views > > > Key: HIVE-11510 > URL: https://issues.apache.org/jira/browse/HIVE-11510 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 0.14.0 >Reporter: Eric Czech >Assignee: Wei Zheng > Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch > > > If views are present in a hive database, issuing a 'hive metatool > -updateLocation' command will result in an error like this: > ... > Warning: Found records with bad LOCATION in SDS table.. > bad location URI: null > bad location URI: null > bad location URI: null > > Based on the source code for Metatool, it looks like there would then be a > "bad location URI: null" message for every view and it also appears this is > happening simply because the 'sds' table in the hive schema has a column > called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720974#comment-14720974 ] Wei Zheng commented on HIVE-11510: -- Test failure not related. Metatool updateLocation warning on views Key: HIVE-11510 URL: https://issues.apache.org/jira/browse/HIVE-11510 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Eric Czech Assignee: Wei Zheng Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch If views are present in a hive database, issuing a 'hive metatool -updateLocation' command will result in an error like this: ... Warning: Found records with bad LOCATION in SDS table.. bad location URI: null bad location URI: null bad location URI: null Based on the source code for Metatool, it looks like there would then be a bad location URI: null message for every view and it also appears this is happening simply because the 'sds' table in the hive schema has a column called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[
https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720936#comment-14720936
]
Hive QA commented on HIVE-11510:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12753071/HIVE-11510.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9380 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5107/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5107/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5107/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12753071 - PreCommit-HIVE-TRUNK-Build
Metatool updateLocation warning on views
Key: HIVE-11510
URL: https://issues.apache.org/jira/browse/HIVE-11510
Project: Hive
Issue Type: Bug
Components: Database/Schema
Affects Versions: 0.14.0
Reporter: Eric Czech
Assignee: Wei Zheng
Attachments: HIVE-11510.1.patch, HIVE-11510.2.patch
If views are present in a hive database, issuing a 'hive metatool
-updateLocation' command will result in an error like this:
...
Warning: Found records with bad LOCATION in SDS table..
bad location URI: null
bad location URI: null
bad location URI: null
Based on the source code for Metatool, it looks like there would then be a
bad location URI: null message for every view and it also appears this is
happening simply because the 'sds' table in the hive schema has a column
called location that is NULL only for views.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720352#comment-14720352 ] Wei Zheng commented on HIVE-11510: -- [~sushanth] Can you review? Metatool updateLocation warning on views Key: HIVE-11510 URL: https://issues.apache.org/jira/browse/HIVE-11510 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Eric Czech Assignee: Wei Zheng Attachments: HIVE-11510.1.patch If views are present in a hive database, issuing a 'hive metatool -updateLocation' command will result in an error like this: ... Warning: Found records with bad LOCATION in SDS table.. bad location URI: null bad location URI: null bad location URI: null Based on the source code for Metatool, it looks like there would then be a bad location URI: null message for every view and it also appears this is happening simply because the 'sds' table in the hive schema has a column called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11510) Metatool updateLocation warning on views
[ https://issues.apache.org/jira/browse/HIVE-11510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720434#comment-14720434 ] Sushanth Sowmyan commented on HIVE-11510: - With the current patch, the metastore will do a LOG.debug for every single null record, which can be a lot, and will also slow down that process a lot. Would it be possible to simply update the UpdateMStorageDescriptorTblURIRetVal class with a int numNullRecords initialized to zero and incremented each time you get a null? Also, in that case, I would imagine that we shouldn't add that location to badRecords, since that would bloat up the size of badRecords unnecessarily. After we do that, we can then do a singular log in HiveMetaTool.printTblURIUpdateSummary along with the other statistics, mentioning how many null records we found, and that that is okay if the user has that many indexes/views. Metatool updateLocation warning on views Key: HIVE-11510 URL: https://issues.apache.org/jira/browse/HIVE-11510 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Eric Czech Assignee: Wei Zheng Attachments: HIVE-11510.1.patch If views are present in a hive database, issuing a 'hive metatool -updateLocation' command will result in an error like this: ... Warning: Found records with bad LOCATION in SDS table.. bad location URI: null bad location URI: null bad location URI: null Based on the source code for Metatool, it looks like there would then be a bad location URI: null message for every view and it also appears this is happening simply because the 'sds' table in the hive schema has a column called location that is NULL only for views. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
