[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117892#comment-16117892 ] Lefty Leverenz commented on HIVE-16294: --- Should this be documented in the wiki? > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Fix For: 3.0.0 > > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch, > HIVE-16294.03.patch, HIVE-16294.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116281#comment-16116281 ] Barna Zsombor Klara commented on HIVE-16294: Thanks for the review. The followup Jira is linked. > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch, > HIVE-16294.03.patch, HIVE-16294.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116272#comment-16116272 ] Peter Vary commented on HIVE-16294: --- Pushed to master. Thanks [~zsombor.klara]! Could you please file the follow-up jira, to move to the new API when Hadoop will provide one? Thanks, Peter > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch, > HIVE-16294.03.patch, HIVE-16294.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[
https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114706#comment-16114706
]
Hive QA commented on HIVE-16294:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12880372/HIVE-16294.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11144 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed]
(batchId=56)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=236)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=179)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6261/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6261/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6261/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12880372 - PreCommit-HIVE-Build
> Support snapshot for truncate table
> ---
>
> Key: HIVE-16294
> URL: https://issues.apache.org/jira/browse/HIVE-16294
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Vihang Karajgaonkar
>Assignee: Barna Zsombor Klara
> Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch,
> HIVE-16294.03.patch, HIVE-16294.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114161#comment-16114161 ] Peter Vary commented on HIVE-16294: --- +1 pending tests > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch, > HIVE-16294.03.patch, HIVE-16294.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[
https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113518#comment-16113518
]
Hive QA commented on HIVE-16294:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12880218/HIVE-16294.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11143 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal]
(batchId=9)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=236)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=236)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=179)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6251/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6251/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6251/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12880218 - PreCommit-HIVE-Build
> Support snapshot for truncate table
> ---
>
> Key: HIVE-16294
> URL: https://issues.apache.org/jira/browse/HIVE-16294
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Vihang Karajgaonkar
>Assignee: Barna Zsombor Klara
> Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch,
> HIVE-16294.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113423#comment-16113423 ] Andrew Wang commented on HIVE-16294: Thanks for the ping. Yes, looks like an oversight in the HDFS API. We expose this listing API on the CLI but not via a programmatic interface. We should add it to HdfsAdmin alongside the other snapshot-related APIs. Filed HDFS-12257 for tracking, let's try and find someone to pick it up. > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch, > HIVE-16294.03.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112497#comment-16112497 ] Peter Vary commented on HIVE-16294: --- There is a non public API to find out, if a directory is snapshottable or not: - DFSClient.getSnapshottableDirListing(), or - DistributedFileSystem.getSnapshottableDirListing()) IMO, here is how we should handle this: - Create a Hadoop jira for requesting a public API. - Until the API is available it is enough to check if the fs is an instance of DistributedFileSystem, and the filename is ".snapshot" - Add a comment to remove this code, when the API is available. But I would like to know how the more experienced guys think about this. What do you think about this:[~aihuaxu], [~andrew.wang]? Thanks, Peter > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111273#comment-16111273 ] Vihang Karajgaonkar commented on HIVE-16294: Hi [~zsombor.klara] I am not actively working on this. Thanks for taking this up. Sorry I wanted to respond back on this JIRA to you after doing some research but couldn't get it it :( > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Barna Zsombor Klara > Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[
https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111269#comment-16111269
]
Hive QA commented on HIVE-16294:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12880044/HIVE-16294.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=179)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6233/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6233/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6233/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12880044 - PreCommit-HIVE-Build
> Support snapshot for truncate table
> ---
>
> Key: HIVE-16294
> URL: https://issues.apache.org/jira/browse/HIVE-16294
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Vihang Karajgaonkar
>Assignee: Barna Zsombor Klara
> Attachments: HIVE-16294.01.patch, HIVE-16294.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16294) Support snapshot for truncate table
[ https://issues.apache.org/jira/browse/HIVE-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100225#comment-16100225 ] Barna Zsombor Klara commented on HIVE-16294: [~vihangk1] are you currently working on this Jira? I was wondering how clear we are about the expected behaviour. The snapshots in Hdfs were introduced precisely to serve as backups and to guard against user errors. Enabling a user initiated feature to silently drop backups and circumvent safeguards seems counterintuitive. > Support snapshot for truncate table > --- > > Key: HIVE-16294 > URL: https://issues.apache.org/jira/browse/HIVE-16294 > Project: Hive > Issue Type: Sub-task > Components: Query Processor >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
