[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) patch 4 committed to master thanks Alan for the review > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 3.0.0 > > Attachments: HIVE-16722.01.patch, HIVE-16722.02.patch, > HIVE-16722.03.patch, HIVE-16722.04.patch, HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Attachment: HIVE-16722.04.patch patch 4 incorporates Alan's suggestion > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.01.patch, HIVE-16722.02.patch, > HIVE-16722.03.patch, HIVE-16722.04.patch, HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Attachment: HIVE-16722.03.patch > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.01.patch, HIVE-16722.02.patch, > HIVE-16722.03.patch, HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Attachment: HIVE-16722.02.patch > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.01.patch, HIVE-16722.02.patch, > HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Status: Patch Available (was: Open) > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.01.patch, HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Attachment: HIVE-16722.01.patch > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.01.patch, HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Attachment: HIVE-16722.WIP.patch > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-16722.WIP.patch > > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16722) Converting bucketed non-acid table to acid should perform validation
[ https://issues.apache.org/jira/browse/HIVE-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-16722: -- Summary: Converting bucketed non-acid table to acid should perform validation (was: Converting non-acid table to acid should perform validation) > Converting bucketed non-acid table to acid should perform validation > > > Key: HIVE-16722 > URL: https://issues.apache.org/jira/browse/HIVE-16722 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman > > Converting a non acid table to acid only performs metadata validation (in > _TransactionalValidationListener_). > The data read code path only understands certain directory layouts and file > names and ignores (generally) files that don't match the expected format. > In Hive, directory layout and bucket file naming (especially older releases) > is poorly enforced. > Need to add a validation step on > {noformat} > alter table T SET TBLPROPERTIES ('transactional'='true') > {noformat} > to > scan the file system and report any possible data loss scenarios. > Currently Acid understands bucket files name like "0_0" and (with > HIVE-16177) 0_0_copy1" etc at the root of the partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)