[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Attachment: HIVE-22499.6.patch Status: Patch Available (was: Open) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.6.patch, HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Status: Open (was: Patch Available) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.6.patch, HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[
https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986679#comment-16986679
]
Hive QA commented on HIVE-21213:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987341/HIVE-21213.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17758 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19714/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19714/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19714/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987341 - PreCommit-HIVE-Build
> Acid table bootstrap replication needs to handle directory created by
> compaction with txn id
>
>
> Key: HIVE-21213
> URL: https://issues.apache.org/jira/browse/HIVE-21213
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, repl
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch,
> HIVE-21213.03.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of compaction uses the txn id in the directory
> name. This is used to isolate the queries from reading the directory until
> compaction has finished and to avoid the compactor marking used earlier. In
> case of replication, during bootstrap , directory is copied as it is with the
> same name from source to destination cluster. But the directory created by
> compaction with txn id can not be copied as the txn list at target may be
> different from source. The txn id which is valid at source may be an aborted
> txn at target. So conversion logic is required to create a new directory with
> valid txn at target and dump the data to the newly created directory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[
https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986671#comment-16986671
]
Hive QA commented on HIVE-21213:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
15s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 85
unchanged - 0 fixed = 88 total (was 85) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19714/dev-support/hive-personality.sh
|
| git revision | master / c8b7648 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19714/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore/metastore-server ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19714/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Acid table bootstrap replication needs to handle directory created by
> compaction with txn id
>
>
> Key: HIVE-21213
> URL: https://issues.apache.org/jira/browse/HIVE-21213
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, repl
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch,
> HIVE-21213.03.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of compaction uses the txn id in the directory
> name. This is used to isolate the queries
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Patch Available (was: Open) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Attachment: HIVE-22508.1.patch > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.20.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.19.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Attachment: HIVE-21213.03.patch > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch, > HIVE-21213.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Status: Patch Available (was: Open) > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch, > HIVE-21213.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Status: Open (was: Patch Available) > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-21174) hive.stats.ndv.error parameter documentation issue
[ https://issues.apache.org/jira/browse/HIVE-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Junge reassigned HIVE-21174: -- Assignee: Pablo Junge (was: Nita Dembla) > hive.stats.ndv.error parameter documentation issue > -- > > Key: HIVE-21174 > URL: https://issues.apache.org/jira/browse/HIVE-21174 > Project: Hive > Issue Type: Improvement > Components: Documentation >Affects Versions: 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.3.0, > 3.0.0, 2.4.0, 2.2.1, 2.3.1, 2.3.2, 2.3.3, 3.1.0, 3.0.1, 3.10, 3.2.0, 3.1.1, > 2.3.4, 3.1.2 >Reporter: Pablo Junge >Assignee: Pablo Junge >Priority: Major > Fix For: 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.3.0, > 3.0.0, 2.4.0, 2.2.1, 2.3.1, 2.3.2, 2.3.3, 3.1.0, 3.0.1, 3.10, 3.2.0, 3.1.1, > 2.3.4 > > > Hive documentation for hive.stats.ndv.error does not specify that > hive.stats.ndv.error will only affect FM Sketch and not HLL. > > https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[
https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986559#comment-16986559
]
Hive QA commented on HIVE-22539:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987333/HIVE-22539.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17755 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19713/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19713/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19713/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987333 - PreCommit-HIVE-Build
> HiveServer2 SPNEGO authentication should skip if authorization header is empty
> --
>
> Key: HIVE-22539
> URL: https://issues.apache.org/jira/browse/HIVE-22539
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Minor
> Attachments: HIVE-22539.1.patch
>
>
> Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos
> before checking header. This can be checked up front to avoid doing any
> Kerberos related work if the header is empty. This is helpful in a lot of
> cases since typically the first request is empty with the client waiting for
> a 401 before returning the Authorization header.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22544: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks [~kkasa]! > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch, HIVE-22544.4.patch, > HIVE-22544.4.patch, HIVE-22544.5.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[
https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986502#comment-16986502
]
Hive QA commented on HIVE-22539:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} service: The patch generated 7 new + 26 unchanged - 5
fixed = 33 total (was 31) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} service generated 0 new + 48 unchanged - 1 fixed =
48 total (was 49) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19713/dev-support/hive-personality.sh
|
| git revision | master / f95a645 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19713/yetus/diff-checkstyle-service.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19713/yetus/whitespace-tabs.txt
|
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19713/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveServer2 SPNEGO authentication should skip if authorization header is empty
> --
>
> Key: HIVE-22539
> URL: https://issues.apache.org/jira/browse/HIVE-22539
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Minor
> Attachments: HIVE-22539.1.patch
>
>
> Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos
> before checking header. This can be checked up front to avoid doing any
> Kerberos related work if the header is empty. This is helpful in a lot of
> cases since typically the first request is empty with the client waiting for
> a 401 before returning the Authorization header.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Release Note: HiveServer2 will check if the Authorization header is present and not empty before checking SPNEGO authentication. > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Minor > Attachments: HIVE-22539.1.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Priority: Minor (was: Major) > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Minor > Attachments: HIVE-22539.1.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Attachment: HIVE-22539.1.patch > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: HIVE-22539.1.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Status: Patch Available (was: In Progress) > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: HIVE-22539.1.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Component/s: HiveServer2 > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: HIVE-22539.1.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[
https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986487#comment-16986487
]
Hive QA commented on HIVE-22499:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987329/HIVE-22499.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17755 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19712/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19712/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19712/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987329 - PreCommit-HIVE-Build
> LLAP: Add an EncodedReaderOptions to extend ORC impl for options
>
>
> Key: HIVE-22499
> URL: https://issues.apache.org/jira/browse/HIVE-22499
> Project: Hive
> Issue Type: Bug
> Components: llap, ORC
>Reporter: Gopal Vijayaraghavan
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch,
> HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch,
> HIVE-22499.WIP.patch, HIVE-22499.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ORC-570 is an ABI change to the way getFileSystem() by adding an another
> exception to the implementation.
> To accept and use that change requires waiting for an ORC release, while this
> patch serves the same purpose though falls back for a retry for
> FileSystem.get() in case the supplier fails at runtime.
> Also as a side-note, the FS.get() call is always used in the cases where the
> file is not being read from a cache such as EncodedOrcFile (so the upstream
> API change might be overkill).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[
https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986469#comment-16986469
]
Hive QA commented on HIVE-22499:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
33s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 1 new + 106 unchanged - 2
fixed = 107 total (was 108) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19712/dev-support/hive-personality.sh
|
| git revision | master / f95a645 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19712/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19712/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP: Add an EncodedReaderOptions to extend ORC impl for options
>
>
> Key: HIVE-22499
> URL: https://issues.apache.org/jira/browse/HIVE-22499
> Project: Hive
> Issue Type: Bug
> Components: llap, ORC
>Reporter: Gopal Vijayaraghavan
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch,
> HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch,
> HIVE-22499.WIP.patch, HIVE-22499.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ORC-570 is an ABI change to the way getFileSystem() by adding an another
> exception to the implementation.
> To accept and use that change requires waiting for an ORC release, while this
> patch serves the same purpose though falls back for a retry for
> FileSystem.get() in case the supplier fails at runtime.
> Also as a side-note, the FS.get() call
[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Attachment: HIVE-22499.5.patch Status: Patch Available (was: Open) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Status: Open (was: Patch Available) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986401#comment-16986401
]
Hive QA commented on HIVE-20150:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987313/HIVE-20150.18.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17761 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_partition_cluster]
(batchId=185)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19711/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19711/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19711/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987313 - PreCommit-HIVE-Build
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Teddy Choi
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch,
> HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch,
> HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch,
> HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch,
> HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch,
> HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch
>
>
> TopNKey operator is implemented in HIVE-17896, but it needs more work in
> pushdown implementation. So this issue covers TopNKey pushdown implementation
> with proper tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986364#comment-16986364
]
Hive QA commented on HIVE-20150:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} ql: The patch generated 0 new + 37 unchanged - 1
fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19711/dev-support/hive-personality.sh
|
| git revision | master / f95a645 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19711/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Teddy Choi
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch,
> HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch,
> HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch,
> HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch,
> HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch,
> HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch
>
>
> TopNKey operator is implemented in HIVE-17896, but it needs more work in
> pushdown implementation. So this issue covers TopNKey pushdown implementation
> with proper tests.
--
This message was sent by
[jira] [Updated] (HIVE-22566) Drop table involved in materialized view leaves the table in inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-22566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pablo Junge updated HIVE-22566:
---
Description:
If you try dropping a table which is part of the definition of a created
materialized view, the table is not dropped, which is the desired state as it
is part of the materialized view.
However, there was a "drop" call to the table, so it tried to drop it but did
not succeed, leaving it in an inconsistent state.
Repro:
---
1) Create tables:
{code:java}
CREATE TABLE emps ( empid INT, deptno INT, name VARCHAR(256), salary FLOAT,
hire_date TIMESTAMP)STORED AS ORCTBLPROPERTIES ('transactional'='true');
CREATE TABLE depts ( deptno INT, deptname VARCHAR(256), locationid
INT)STORED AS ORCTBLPROPERTIES ('transactional'='true');
{code}
2) Create the VM:
{code:java}
CREATE MATERIALIZED VIEW mv1ASSELECT empid, deptname, hire_dateFROM emps JOIN
depts ON (emps.deptno = depts.deptno)WHERE hire_date >= '2016-01-01';
{code}
3) Following is in backend database at this point:
{code:java}
mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where DB_ID=16;
++---+---+--+---+
| TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
++---+---+--+---+
| 81 | 16 | 81 | emps | MANAGED_TABLE |
| 83 | 16 | 83 | depts | MANAGED_TABLE |
| 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
++---+---+--+---+
3 rows in set (0.00 sec)
{code}
4) Let's drop the 'emps' table:
{code:java}
0: jdbc:hive2://c1122-node2.squadron.support.> drop table emps;
INFO : Compiling
command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b): drop
table emps
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b); Time
taken: 0.05 seconds
INFO : Executing
command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b): drop
table emps
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b); Time
taken: 10.281 seconds
INFO : OK
No rows affected (16.949 seconds)
{code}
No issue displayed
5) List tables:
{code:java}
0: jdbc:hive2://c1122-node2.squadron.support.> show tables;
INFO : Compiling
command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413): show
tables
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name,
type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling
command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413); Time
taken: 0.041 seconds
INFO : Executing
command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413): show
tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413); Time
taken: 0.016 seconds
INFO : OK
+---+
| tab_name |
+---+
| depts |
| emps |
+---+
2 rows selected (0.08 seconds)
{code}
6) Now, from the backend-db point of view:
{code:java}
mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where DB_ID=16;
++---+---+--+---+
| TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
++---+---+--+---+
| 81 | 16 | NULL | emps | MANAGED_TABLE |
| 83 | 16 | 83 | depts | MANAGED_TABLE |
| 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
++---+---+--+---+
3 rows in set (0.00 sec)
{code}
The table is left with NULL in SD_ID, making it not available.
7) From Metastore.log
{code:java}
2019-12-02T20:00:25,545 INFO [pool-6-thread-195]: metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(907)) - 196: source:172.25.34.150 drop_table :
tbl=hive.mvs.emps
2019-12-02T20:00:25,545 INFO [pool-6-thread-195]: HiveMetaStore.audit
(HiveMetaStore.java:logAuditEvent(349)) - ugi=hive ip=172.25.34.150
cmd=source:172.25.34.150 drop_table : tbl=hive.mvs.emps
2019-12-02T20:00:25,580 INFO [pool-6-thread-195]:
metastore.ObjectStore$RetryingExecutor (ObjectStore.java:run(9966)) -
Attempting to acquire the DB log notification lock: 0 out of 10 retries
javax.jdo.JDODataStoreException: Error executing SQL query "select
"NEXT_EVENT_ID" from "NOTIFICATION_SEQUENCE" for update".
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
~[datanucleus-api-jdo-4.2.4.jar:?]
at
[jira] [Commented] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[
https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986336#comment-16986336
]
Hive QA commented on HIVE-21213:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987304/HIVE-21213.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17755 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.server.TestHS2HttpServerPam.org.apache.hive.service.server.TestHS2HttpServerPam
(batchId=241)
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamCorrectConfiguration
(batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19710/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19710/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19710/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987304 - PreCommit-HIVE-Build
> Acid table bootstrap replication needs to handle directory created by
> compaction with txn id
>
>
> Key: HIVE-21213
> URL: https://issues.apache.org/jira/browse/HIVE-21213
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, repl
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of compaction uses the txn id in the directory
> name. This is used to isolate the queries from reading the directory until
> compaction has finished and to avoid the compactor marking used earlier. In
> case of replication, during bootstrap , directory is copied as it is with the
> same name from source to destination cluster. But the directory created by
> compaction with txn id can not be copied as the txn list at target may be
> different from source. The txn id which is valid at source may be an aborted
> txn at target. So conversion logic is required to create a new directory with
> valid txn at target and dump the data to the newly created directory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[
https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986297#comment-16986297
]
Hive QA commented on HIVE-21213:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
23s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} itests/hive-unit: The patch generated 6 new + 85
unchanged - 0 fixed = 91 total (was 85) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19710/dev-support/hive-personality.sh
|
| git revision | master / f95a645 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19710/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore/metastore-server ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19710/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Acid table bootstrap replication needs to handle directory created by
> compaction with txn id
>
>
> Key: HIVE-21213
> URL: https://issues.apache.org/jira/browse/HIVE-21213
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, repl
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of compaction uses the txn id in the directory
> name. This is used to isolate the queries from reading the directory
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.18.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22547) Review txn compactor Package
[
https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986260#comment-16986260
]
Hive QA commented on HIVE-22547:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987302/HIVE-22547.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17754 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
org.apache.hadoop.hive.ql.metadata.TestHiveRemote.testDropPartitionsWithPurge
(batchId=336)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19709/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19709/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19709/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987302 - PreCommit-HIVE-Build
> Review txn compactor Package
>
>
> Key: HIVE-22547
> URL: https://issues.apache.org/jira/browse/HIVE-22547
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch
>
>
> * Remove log-and-throw anti-pattern
> * Use parameterized logging
> * Add a CompactionException class to improve debug-ability
> * Introduce Java Optional utility
> * Other clean up
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22317) Beeline-site parser does not handle the variable substitution correctly
[
https://issues.apache.org/jira/browse/HIVE-22317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-22317:
--
Fix Version/s: 4.0.0
Resolution: Resolved
Status: Resolved (was: Patch Available)
> Beeline-site parser does not handle the variable substitution correctly
> ---
>
> Key: HIVE-22317
> URL: https://issues.apache.org/jira/browse/HIVE-22317
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
> Environment: Hive-4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22317.01.patch, HIVE-22317.02.patch,
> HIVE-22317.patch
>
>
> beeline-site.xml
> {code:java}
> http://www.w3.org/2001/XInclude;>
>
>
> beeline.hs2.jdbc.url.container
>
> jdbc:hive2://c3220-node2.host.com:2181,c3220-node3.host.com:2181,c3220-node4.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
>
>
>
> beeline.hs2.jdbc.url.default
> test
>
>
> beeline.hs2.jdbc.url.test
> ${beeline.hs2.jdbc.url.container}?tez.queue.name=myqueue
>
>
> beeline.hs2.jdbc.url.llap
>
> jdbc:hive2://c3220-node2.host.com:2181,c3220-node3.host.com:2181,c3220-node4.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-interactive
>
>
> {code}
> beeline fail to connect because it does not parse the substituted value
> correctly
> {code:java}
> beeline
> Error in parsing jdbc url:
> ${beeline.hs2.jdbc.url.container}?tez.queue.name=myqueue from beeline-site.xml
> beeline> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22317) Beeline-site parser does not handle the variable substitution correctly
[
https://issues.apache.org/jira/browse/HIVE-22317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986251#comment-16986251
]
Rajkumar Singh commented on HIVE-22317:
---
Thanks [~maheshk114] for review, pushed the changes to master.
> Beeline-site parser does not handle the variable substitution correctly
> ---
>
> Key: HIVE-22317
> URL: https://issues.apache.org/jira/browse/HIVE-22317
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 4.0.0
> Environment: Hive-4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22317.01.patch, HIVE-22317.02.patch,
> HIVE-22317.patch
>
>
> beeline-site.xml
> {code:java}
> http://www.w3.org/2001/XInclude;>
>
>
> beeline.hs2.jdbc.url.container
>
> jdbc:hive2://c3220-node2.host.com:2181,c3220-node3.host.com:2181,c3220-node4.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
>
>
>
> beeline.hs2.jdbc.url.default
> test
>
>
> beeline.hs2.jdbc.url.test
> ${beeline.hs2.jdbc.url.container}?tez.queue.name=myqueue
>
>
> beeline.hs2.jdbc.url.llap
>
> jdbc:hive2://c3220-node2.host.com:2181,c3220-node3.host.com:2181,c3220-node4.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-interactive
>
>
> {code}
> beeline fail to connect because it does not parse the substituted value
> correctly
> {code:java}
> beeline
> Error in parsing jdbc url:
> ${beeline.hs2.jdbc.url.container}?tez.queue.name=myqueue from beeline-site.xml
> beeline> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22547) Review txn compactor Package
[
https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986243#comment-16986243
]
Hive QA commented on HIVE-22547:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m
45s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
9s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 3 new + 70 unchanged - 10
fixed = 73 total (was 80) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
39s{color} | {color:red} ql generated 3 new + 1533 unchanged - 1 fixed = 1536
total (was 1534) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unread public/protected field:At CompactorThread.java:[line 66] |
| | Unused public or protected
field:org.apache.hadoop.hive.ql.txn.compactor.CompactorThread.rs In
CompactorThread.java |
| | Null passed for non-null parameter of
RemoteCompactorThread.resolveTable(CompactionInfo) in
org.apache.hadoop.hive.ql.txn.compactor.Worker.run() Method invoked at
Worker.java:of RemoteCompactorThread.resolveTable(CompactionInfo) in
org.apache.hadoop.hive.ql.txn.compactor.Worker.run() Method invoked at
Worker.java:[line 115] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19709/dev-support/hive-personality.sh
|
| git revision | master / d2e6b50 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19709/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19709/yetus/new-findbugs-ql.html
|
| modules | C: ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19709/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review txn compactor Package
>
>
> Key: HIVE-22547
> URL: https://issues.apache.org/jira/browse/HIVE-22547
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments:
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986234#comment-16986234
]
Hive QA commented on HIVE-22066:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 20m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
33s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
10s{color} | {color:red} contrib in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
14s{color} | {color:red} hplsql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
13s{color} | {color:red} llap-ext-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 2m
55s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 32m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/dev-support/hive-personality.sh
|
| git revision | master / d2e6b50 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-ql.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-llap-server.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-contrib.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-hplsql.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-llap-ext-client.txt
|
| modules | C: storage-api shims/common shims/0.23 shims/scheduler
standalone-metastore standalone-metastore/metastore-common common service-rpc
serde metastore vector-code-gen llap-common llap-client llap-tez spark-client
ql llap-server service accumulo-handler jdbc beeline cli contrib druid-handler
hbase-handler jdbc-handler hcatalog hcatalog/core hcatalog/hcatalog-pig-adapter
hcatalog/server-extensions hcatalog/webhcat/java-client hcatalog/webhcat/svr
hcatalog/streaming hplsql streaming llap-ext-client kudu-handler kafka-handler
standalone-metastore/metastore-tools
standalone-metastore/metastore-tools/metastore-benchmarks upgrade-acid
upgrade-acid/pre-upgrade . itests/hive-jmh itests/qtest-druid U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19708/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade Apache parent POM to version 21
>
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986205#comment-16986205
]
Hive QA commented on HIVE-22066:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987303/HIVE-22066.9.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17754 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19708/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19708/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19708/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987303 - PreCommit-HIVE-Build
> Upgrade Apache parent POM to version 21
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch,
> HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Attachment: HIVE-21213.02.patch > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Status: Patch Available (was: Open) > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch, HIVE-21213.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Attachment: (was: HIVE-21213.02.patch) > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21213) Acid table bootstrap replication needs to handle directory created by compaction with txn id
[ https://issues.apache.org/jira/browse/HIVE-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-21213: --- Status: Open (was: Patch Available) > Acid table bootstrap replication needs to handle directory created by > compaction with txn id > > > Key: HIVE-21213 > URL: https://issues.apache.org/jira/browse/HIVE-21213 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2, repl >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21213.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The current implementation of compaction uses the txn id in the directory > name. This is used to isolate the queries from reading the directory until > compaction has finished and to avoid the compactor marking used earlier. In > case of replication, during bootstrap , directory is copied as it is with the > same name from source to destination cluster. But the directory created by > compaction with txn id can not be copied as the txn list at target may be > different from source. The txn id which is valid at source may be an aborted > txn at target. So conversion logic is required to create a new directory with > valid txn at target and dump the data to the newly created directory. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Patch Available (was: Open) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Attachment: HIVE-22066.9.patch > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Open (was: Patch Available) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22547) Review txn compactor Package
[ https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22547: -- Attachment: HIVE-22547.2.patch > Review txn compactor Package > > > Key: HIVE-22547 > URL: https://issues.apache.org/jira/browse/HIVE-22547 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch > > > * Remove log-and-throw anti-pattern > * Use parameterized logging > * Add a CompactionException class to improve debug-ability > * Introduce Java Optional utility > * Other clean up -- This message was sent by Atlassian Jira (v8.3.4#803005)
