[jira] [Commented] (HIVE-18781) Create/Replicate Abort Txn event
[
https://issues.apache.org/jira/browse/HIVE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379919#comment-16379919
]
Hive QA commented on HIVE-18781:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
33s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 631
unchanged - 0 fixed = 634 total (was 631) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 20 new + 243 unchanged - 0
fixed = 263 total (was 243) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
30s{color} | {color:red} standalone-metastore: The patch generated 44 new +
1262 unchanged - 0 fixed = 1306 total (was 1262) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 16 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} standalone-metastore generated 1 new + 61 unchanged -
0 fixed = 62 total (was 61) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9399/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/whitespace-eol.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9399/yetus/diff-javadoc-javadoc-standalone-metastore.txt
|
| asflicense
[jira] [Updated] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kryvenko Igor updated HIVE-18746:
-
Attachment: HIVE-18746.5.patch
> add_months should validate the date first
> -
>
> Key: HIVE-18746
> URL: https://issues.apache.org/jira/browse/HIVE-18746
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Subhasis Gorai
>Assignee: Kryvenko Igor
>Priority: Minor
> Attachments: HIVE-18746.1.patch, HIVE-18746.3.patch,
> HIVE-18746.4.patch, HIVE-18746.5.patch, HIVE-18746.patch
>
>
> hive (sbg_hvc_ods)> select add_months('2017-02-28', 1);
> OK
> _c0
> 2017-03-31
> Time taken: 0.107 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)> select add_months('2017-02-29', 1);
> OK
> _c0
> 2017-04-01
> Time taken: 0.084 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)>
>
> '2017-02-29' is an invalid date.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18822) INSERT VALUES - HoS + Steaming File Format
[ https://issues.apache.org/jira/browse/HIVE-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18822: --- Summary: INSERT VALUES - HoS + Steaming File Format (was: INSERT VALUES - HoS + Avro) > INSERT VALUES - HoS + Steaming File Format > -- > > Key: HIVE-18822 > URL: https://issues.apache.org/jira/browse/HIVE-18822 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Priority: Minor > > Please optimize the INSERT VALUES function. When HoS is being used, and a > streaming format such as TEXT or AVRO are being used, INSERT VALUES > statements should be quick. The HiveServer2 should pass the vales to the > Executor and the Executor should simply append the data to an existing HDFS > file instead of creating a new one. This will reduce the number of small > files that exist in the file system... or perhaps the HiveServer2 performs > the append without having to first sent the data to the processing engine at > all. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[
https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379900#comment-16379900
]
Matt McCline commented on HIVE-18819:
-
Also, it looks like they way we custom instantiate the
IfExprColumn{Null|CondExpr}, IfExprCondExprCondExpr,
IfExprCondExpr{Null|Column} we don't reuse the scratch columns... You can see
this by the higher column numbers in the EXPLAIN VECTORIZATION DETAIL display.
> Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
> ---
>
> Key: HIVE-18819
> URL: https://issues.apache.org/jira/browse/HIVE-18819
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18819.01.patch
>
>
> Currently, all the rows of a batch are evaluated for the THEN and ELSE
> expressions even though only a value from one of them is needed for any
> particular row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379895#comment-16379895 ] Matt McCline commented on HIVE-18819: - Because the new code is noticing non-column and non-scalar expressions -- in other words: expressions that require computation -- those are considered candidates for conditional IF statement vector expressions. So, for example, some usages of IfExprStringScalarStringGroupColumn become IfExprColumnCondExpr. Is it worth generating more specialized versions of IfExprColumnCondExpr that do outputColVector.setVal instead of outputColVector.setElement? Not clear it is worth it for this round. > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18819.01.patch > > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[
https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379888#comment-16379888
]
Hive QA commented on HIVE-18819:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912373/HIVE-18819.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 13413 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[
https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379864#comment-16379864
]
Hive QA commented on HIVE-18819:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
52s{color} | {color:red} root: The patch generated 8 new + 779 unchanged - 1
fixed = 787 total (was 780) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 8 new + 354 unchanged - 1
fixed = 362 total (was 355) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9398/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9398/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9398/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9398/yetus/patch-asflicense-problems.txt
|
| modules | C: common . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9398/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
> ---
>
> Key: HIVE-18819
> URL: https://issues.apache.org/jira/browse/HIVE-18819
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18819.01.patch
>
>
> Currently, all the rows of a batch are evaluated for the THEN and ELSE
> expressions even though only a value from one of them is needed for any
> particular row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17580) Remove dependency of get_fields_with_environment_context API to serde
[
https://issues.apache.org/jira/browse/HIVE-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379860#comment-16379860
]
Vihang Karajgaonkar commented on HIVE-17580:
latest patch restores {{Category}} enum back into {{ObjectInspector}} since
apparently projects outside Hive also use it directly and moving it out of
ObjectInspector will break stuff. This patch moves the {{ObjectInspector}}
interface to storage-api instead of just moving Category out of it. This way
compatibility is maintained and metastore can still access it without depending
on hive-serde.
> Remove dependency of get_fields_with_environment_context API to serde
> -
>
> Key: HIVE-17580
> URL: https://issues.apache.org/jira/browse/HIVE-17580
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-17580.003-standalone-metastore.patch,
> HIVE-17580.04-standalone-metastore.patch,
> HIVE-17580.05-standalone-metastore.patch,
> HIVE-17580.06-standalone-metastore.patch,
> HIVE-17580.07-standalone-metastore.patch,
> HIVE-17580.08-standalone-metastore.patch,
> HIVE-17580.09-standalone-metastore.patch,
> HIVE-17580.091-standalone-metastore.patch
>
>
> {{get_fields_with_environment_context}} metastore API uses {{Deserializer}}
> class to access the fields metadata for the cases where it is stored along
> with the data files (avro tables). The problem is Deserializer classes is
> defined in hive-serde module and in order to make metastore independent of
> Hive we will have to remove this dependency (atleast we should change it to
> runtime dependency instead of compile time).
> The other option is investigate if we can use SearchArgument to provide this
> functionality.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17580) Remove dependency of get_fields_with_environment_context API to serde
[
https://issues.apache.org/jira/browse/HIVE-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-17580:
---
Attachment: HIVE-17580.091-standalone-metastore.patch
> Remove dependency of get_fields_with_environment_context API to serde
> -
>
> Key: HIVE-17580
> URL: https://issues.apache.org/jira/browse/HIVE-17580
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-17580.003-standalone-metastore.patch,
> HIVE-17580.04-standalone-metastore.patch,
> HIVE-17580.05-standalone-metastore.patch,
> HIVE-17580.06-standalone-metastore.patch,
> HIVE-17580.07-standalone-metastore.patch,
> HIVE-17580.08-standalone-metastore.patch,
> HIVE-17580.09-standalone-metastore.patch,
> HIVE-17580.091-standalone-metastore.patch
>
>
> {{get_fields_with_environment_context}} metastore API uses {{Deserializer}}
> class to access the fields metadata for the cases where it is stored along
> with the data files (avro tables). The problem is Deserializer classes is
> defined in hive-serde module and in order to make metastore independent of
> Hive we will have to remove this dependency (atleast we should change it to
> runtime dependency instead of compile time).
> The other option is investigate if we can use SearchArgument to provide this
> functionality.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18777) Add Authorization interface to support information_schema integration with external authorization
[ https://issues.apache.org/jira/browse/HIVE-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379854#comment-16379854 ] Daniel Dai commented on HIVE-18777: --- +1 > Add Authorization interface to support information_schema integration with > external authorization > - > > Key: HIVE-18777 > URL: https://issues.apache.org/jira/browse/HIVE-18777 > Project: Hive > Issue Type: Bug >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-18777.1.patch, HIVE-18777.2.patch > > > HIVE-1010 added support for information_schema. However, the authorization > information is not integrated when another project such as Ranger is used to > do the authorization. > We need to add API which Ranger/Sentry can implement, so that it is possible > to retrieve authorization policy information from them. > The existing API only supports checking if user has a permission on an object > and can't be used to retrieve policy details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379853#comment-16379853 ] Zoltan Haindrich commented on HIVE-17937: - oh my...it was all along an order by null problem; and the java change in the first patch is not even belongs to this ticket :D +1 > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Blocker > Attachments: HIVE-17937.05.patch, HIVE-17937.2.patch, > HIVE-17937.3.patch, HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18796) fix TestSSL
[ https://issues.apache.org/jira/browse/HIVE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379837#comment-16379837 ] Zoltan Haindrich commented on HIVE-18796: - So the problem was not even related to what the test intended to test? :) +1 > fix TestSSL > --- > > Key: HIVE-18796 > URL: https://issues.apache.org/jira/browse/HIVE-18796 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Zoltan Haindrich >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18796.patch > > > broken by HIVE-18203 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18695) fix TestAccumuloCliDriver.testCliDriver[accumulo_queries]
[ https://issues.apache.org/jira/browse/HIVE-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379835#comment-16379835 ] Thejas M Nair commented on HIVE-18695: -- +1 > fix TestAccumuloCliDriver.testCliDriver[accumulo_queries] > - > > Key: HIVE-18695 > URL: https://issues.apache.org/jira/browse/HIVE-18695 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18695.01.patch > > > seems to be broken by HIVE-15680 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18777) Add Authorization interface to support information_schema integration with external authorization
[ https://issues.apache.org/jira/browse/HIVE-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-18777: - Attachment: HIVE-18777.2.patch > Add Authorization interface to support information_schema integration with > external authorization > - > > Key: HIVE-18777 > URL: https://issues.apache.org/jira/browse/HIVE-18777 > Project: Hive > Issue Type: Bug >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-18777.1.patch, HIVE-18777.2.patch > > > HIVE-1010 added support for information_schema. However, the authorization > information is not integrated when another project such as Ranger is used to > do the authorization. > We need to add API which Ranger/Sentry can implement, so that it is possible > to retrieve authorization policy information from them. > The existing API only supports checking if user has a permission on an object > and can't be used to retrieve policy details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18777) Add Authorization interface to support information_schema integration with external authorization
[ https://issues.apache.org/jira/browse/HIVE-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379834#comment-16379834 ] Thejas M Nair commented on HIVE-18777: -- [~daijy] Thanks. fixed comment in 2.patch > Add Authorization interface to support information_schema integration with > external authorization > - > > Key: HIVE-18777 > URL: https://issues.apache.org/jira/browse/HIVE-18777 > Project: Hive > Issue Type: Bug >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Major > Labels: pull-request-available > Attachments: HIVE-18777.1.patch, HIVE-18777.2.patch > > > HIVE-1010 added support for information_schema. However, the authorization > information is not integrated when another project such as Ranger is used to > do the authorization. > We need to add API which Ranger/Sentry can implement, so that it is possible > to retrieve authorization policy information from them. > The existing API only supports checking if user has a permission on an object > and can't be used to retrieve policy details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false
[
https://issues.apache.org/jira/browse/HIVE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18818:
---
Attachment: HIVE-18818.1.patch
> Alter table add constraint unique fails with direct sql set to false
>
>
> Key: HIVE-18818
> URL: https://issues.apache.org/jira/browse/HIVE-18818
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18818.1.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.metastore.try.direct.sql=false;
> CREATE TABLE numericDataType(a TINYINT, b SMALLINT NOT NULL ENABLE, c INT,
> d BIGINT , e DOUBLE , f DECIMAL(9,2));
> INSERT INTO numericDataType values(2,45,5667,67890,5.6,678.5);
> ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;
> DESC FORMATTED numericDataType;
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> {code}
> *Error Stack*
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
>
[jira] [Updated] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false
[
https://issues.apache.org/jira/browse/HIVE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18818:
---
Status: Patch Available (was: Open)
> Alter table add constraint unique fails with direct sql set to false
>
>
> Key: HIVE-18818
> URL: https://issues.apache.org/jira/browse/HIVE-18818
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18818.1.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.metastore.try.direct.sql=false;
> CREATE TABLE numericDataType(a TINYINT, b SMALLINT NOT NULL ENABLE, c INT,
> d BIGINT , e DOUBLE , f DECIMAL(9,2));
> INSERT INTO numericDataType values(2,45,5667,67890,5.6,678.5);
> ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;
> DESC FORMATTED numericDataType;
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> {code}
> *Error Stack*
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
>
[jira] [Updated] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false
[
https://issues.apache.org/jira/browse/HIVE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18818:
---
Description:
*Reproducer*
{code:sql}
set hive.metastore.try.direct.sql=false;
CREATE TABLE numericDataType(a TINYINT, b SMALLINT NOT NULL ENABLE, c INT,
d BIGINT , e DOUBLE , f DECIMAL(9,2));
INSERT INTO numericDataType values(2,45,5667,67890,5.6,678.5);
ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;
DESC FORMATTED numericDataType;
{code}
*Error*
{code:sql}
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
{code}
*Error Stack*
{noformat}
java.lang.NullPointerException: null
at
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_101]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_101]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_101]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_101]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_101]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_101]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4564)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:3626)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:455)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2314)
[jira] [Commented] (HIVE-18693) Snapshot Isolation does not work for Micromanaged table when a insert transaction is aborted
[
https://issues.apache.org/jira/browse/HIVE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379821#comment-16379821
]
Hive QA commented on HIVE-18693:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912366/HIVE-18693.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 13417 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Comment Edited] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379801#comment-16379801
]
Eugene Koifman edited comment on HIVE-18817 at 2/28/18 5:24 AM:
patch1 fixes the issue
The test case generates a file with 2 stripes. w/o the fix to
OrcRecorUpdater.KeyIndexBuilder, the hive.acid.key.index in the file has 1
entry - with the fix it has 2 and no AIOOBException happens.
todo: file an ORC ticket to fix this in ORC.
[~jdere], [~prasanth_j] please review
was (Author: ekoifman):
patch1 fixes the issue
todo: file an ORC ticket to fix this in ORC.
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18817.01.patch, repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18817:
--
Status: Patch Available (was: Open)
patch1 fixes the issue
todo: file an ORC ticket to fix this in ORC.
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18817.01.patch, repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18817:
--
Attachment: HIVE-18817.01.patch
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18817.01.patch, repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18693) Snapshot Isolation does not work for Micromanaged table when a insert transaction is aborted
[
https://issues.apache.org/jira/browse/HIVE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379798#comment-16379798
]
Hive QA commented on HIVE-18693:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} streaming in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
48s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 4 new + 436 unchanged - 3
fixed = 440 total (was 439) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore: The patch generated 5 new + 628
unchanged - 1 fixed = 633 total (was 629) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9397/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/patch-mvninstall-hcatalog_streaming.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus/patch-asflicense-problems.txt
|
| modules | C: hcatalog/streaming ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9397/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Snapshot Isolation does not work for Micromanaged table when a insert
> transaction is aborted
>
>
> Key: HIVE-18693
> URL: https://issues.apache.org/jira/browse/HIVE-18693
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Steve Yeom
>
[jira] [Updated] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18797:
---
Status: Patch Available (was: Open)
Rebased patch to kickstart ptests
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18797:
---
Attachment: HIVE-18797.4.patch
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18797:
---
Status: Open (was: Patch Available)
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[
https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379773#comment-16379773
]
Hive QA commented on HIVE-18820:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912365/HIVE-18820.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 13413 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Attachment: HIVE-18819.01.patch > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18819.01.patch > > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Attachment: (was: HIVE-18819.01.patch) > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18464) Fix TestSSL test failures
[ https://issues.apache.org/jira/browse/HIVE-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379759#comment-16379759 ] Bharathkrishna Guruvayoor Murali commented on HIVE-18464: - I did not work on this ticket yet. Can close this issue as a patch is already available for the duplicate Jira HIVE-18796 > Fix TestSSL test failures > - > > Key: HIVE-18464 > URL: https://issues.apache.org/jira/browse/HIVE-18464 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > > TestSSL test has been failing for a while now. Following three tests fail > everytime precommit job runs. > org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232) > org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232) > org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[
https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379740#comment-16379740
]
Hive QA commented on HIVE-18820:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9396/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9396/yetus/patch-asflicense-problems.txt
|
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9396/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Operation doesn't always clean up log4j for operation log
> -
>
> Key: HIVE-18820
> URL: https://issues.apache.org/jira/browse/HIVE-18820
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18820.patch
>
>
> Operation log may only be enabled for some operations, however the routing
> appender still creates appenders for the queries where it's not enabled as
> far as I see, that don't get cleaned up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[
https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379732#comment-16379732
]
Hive QA commented on HIVE-17937:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912333/HIVE-17937.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 96 failed/errored test(s), 13017 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Attachment: HIVE-18819.01.patch > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18819.01.patch > > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Attachment: (was: HIVE-18819.01.patch) > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18821) dfs -ls not working when executed from beeline
[ https://issues.apache.org/jira/browse/HIVE-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ranjan Choudhury updated HIVE-18821: Priority: Critical (was: Major) > dfs -ls not working when executed from beeline > -- > > Key: HIVE-18821 > URL: https://issues.apache.org/jira/browse/HIVE-18821 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 1.2.1 > Environment: Hive 1.2.1000.2.5.3.0-37 > Hadoop 2.7.3.2.5.3.0-37 >Reporter: Ranjan Choudhury >Priority: Critical > > The command dfs -ls is not producing any output when executed from beeline > even though there are files in hdfs. > Steps to reproduce: > 1. Run beeline > 2. Connect to hive using command !connect jdbc:hive2://localhost:1 > 3. Provide username/password > 4. Execute dfs -ls /user > There is not output of the command dfs -ls /user, even though there are files > in HDFS. When i execute the command hive CLI, it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Status: Patch Available (was: Open) Work-In-Progress. Needs a many more queries that use IF statements. Also, since 2 "case" CASE WHEN statements are converted to IF statements, we need more of those queries, too. > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18819.01.patch > > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18819: Attachment: HIVE-18819.01.patch > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18819.01.patch > > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18781) Create/Replicate Abort Txn event
[ https://issues.apache.org/jira/browse/HIVE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-18781: --- Attachment: HIVE-18781.01.patch > Create/Replicate Abort Txn event > > > Key: HIVE-18781 > URL: https://issues.apache.org/jira/browse/HIVE-18781 > Project: Hive > Issue Type: Task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18781.01.patch > > > *EVENT_ABORT_TXN* > Source Warehouse: > - Create new event type EVENT_ABORT_TXN with related message format etc. > - Capture this event when abort the txn. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Validate if source txn ID from the event is there in the source-target txn > ID map. If not there, just noop the event. > - If valid, then Abort the corresponding target txn and remove the entry > from source-target txn ID map. > All these new events should be idempotent such that if it is applied twice, > then second time it should be noop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18781) Create/Replicate Abort Txn event
[ https://issues.apache.org/jira/browse/HIVE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-18781: --- Status: Patch Available (was: Open) contains changes related to open and commit txn also > Create/Replicate Abort Txn event > > > Key: HIVE-18781 > URL: https://issues.apache.org/jira/browse/HIVE-18781 > Project: Hive > Issue Type: Task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18781.01.patch > > > *EVENT_ABORT_TXN* > Source Warehouse: > - Create new event type EVENT_ABORT_TXN with related message format etc. > - Capture this event when abort the txn. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Validate if source txn ID from the event is there in the source-target txn > ID map. If not there, just noop the event. > - If valid, then Abort the corresponding target txn and remove the entry > from source-target txn ID map. > All these new events should be idempotent such that if it is applied twice, > then second time it should be noop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18810) Parquet Or ORC
[ https://issues.apache.org/jira/browse/HIVE-18810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379702#comment-16379702 ] Owen O'Malley commented on HIVE-18810: -- It isn't clear that a jira is the best way of documenting this. We should probably add a page to either the Hive wiki or the ORC website. That said, you can see my presentation on the file format benchmarks: https://www.slideshare.net/oom65/file-format-benchmarks-avro-json-orc-parquet There has also been a lot of work recently to improve the performance of ORC from Spark : https://community.hortonworks.com/articles/148917/orc-improvements-for-apache-spark-22.html Other comparisons: ORC predicate pushdown happens at: file, stripe, and 10,000 rows. Parquet predicate pushdown happens at: file and stripe ORC has optional bloom filters, Parquet doesn't. This is related to the previous point, because bloom filters only make sense at levels below the stripe level. ORC's type system is much closer to Hive's than Parquet's. > Parquet Or ORC > -- > > Key: HIVE-18810 > URL: https://issues.apache.org/jira/browse/HIVE-18810 > Project: Hive > Issue Type: Test > Components: Hive >Affects Versions: 1.1.0 > Environment: Hadoop 1.2.1 > Hive 1.1 >Reporter: Suddhasatwa Bhaumik >Priority: Major > > Hello Experts, > I would like to know for which data types (based on size and complexity of > data) should one be using Parquet or ORC tables in Hive. E.g., On Hadoop > 0.20.0 with hive 0.13, the performance of ORC tables in Hive is very good > when accessed even by 3rd party BI systems like SAP Business Objects or > Tableau; performing the same tests on Hadoop 1.2.1 with Hive 1.1 does not > yield such reliability in queries, although ETL or insert/update of tables > are taking nominal time the read performance is not within acceptable limits. > In case of any queries, kindly advise. > Thanks > [~suddhasatwa_bhaumik] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[ https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379688#comment-16379688 ] Gunther Hagleitner commented on HIVE-18820: --- +1 > Operation doesn't always clean up log4j for operation log > - > > Key: HIVE-18820 > URL: https://issues.apache.org/jira/browse/HIVE-18820 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18820.patch > > > Operation log may only be enabled for some operations, however the routing > appender still creates appenders for the queries where it's not enabled as > far as I see, that don't get cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18693) Snapshot Isolation does not work for Micromanaged table when a insert transaction is aborted
[ https://issues.apache.org/jira/browse/HIVE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-18693: -- Status: Patch Available (was: Open) > Snapshot Isolation does not work for Micromanaged table when a insert > transaction is aborted > > > Key: HIVE-18693 > URL: https://issues.apache.org/jira/browse/HIVE-18693 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-18693.01.patch, HIVE-18693.02.patch > > > TestTxnCommands2#writeBetweenWorkerAndCleaner with minor > changes (changing delete command to insert command) fails on MM table. > Specifically the last SELECT commands returns wrong results. > But this test works fine with full ACID table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18693) Snapshot Isolation does not work for Micromanaged table when a insert transaction is aborted
[ https://issues.apache.org/jira/browse/HIVE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-18693: -- Attachment: HIVE-18693.02.patch > Snapshot Isolation does not work for Micromanaged table when a insert > transaction is aborted > > > Key: HIVE-18693 > URL: https://issues.apache.org/jira/browse/HIVE-18693 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-18693.01.patch, HIVE-18693.02.patch > > > TestTxnCommands2#writeBetweenWorkerAndCleaner with minor > changes (changing delete command to insert command) fails on MM table. > Specifically the last SELECT commands returns wrong results. > But this test works fine with full ACID table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18693) Snapshot Isolation does not work for Micromanaged table when a insert transaction is aborted
[ https://issues.apache.org/jira/browse/HIVE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366552#comment-16366552 ] Steve Yeom edited comment on HIVE-18693 at 2/28/18 2:29 AM: Talked with Eugene. The patch has the following requirements: 1. Allow Snapshot Isolation to work. Esp., we should keep TXNS row for aborted transaction until the related delta directories are removed (which can contain many components). For this we will add rows for components (partitions/table) intoTXN_COMPONENTS. 2. Remove aborted transaction delta/base directories for MM tables. 3. Still the states of a compaction for MM table is INITIATED, IN-PROGRESS, COMPACTED, READY-FOR-CLEANING, and CLEANED until parallelism is introduced for Worker. 4. Clean obsolete directories, and Metastore metadata rows for transaction tables. was (Author: steveyeom2017): Talked with Eugene. The patch has the following requirements: 1. Allow Snapshot Isolation to work. Esp., we should keep TXNS row for aborted transaction until the related delta directories are removed (which can contain many components). For this we will add rows for components (partitions/table) intoTXN_COMPONENTS. 2. Remove aborted transaction delta/base directories for MM tables. 3. Do not initiate compaction for MM tables. So a compaction request for MM table may have INITIATING/INITIATED, CLEANING, FINISHED state without COMPACTING/COMPACTED state. 3. Clean obsolete directories, and Metastore metadata rows for transaction tables. > Snapshot Isolation does not work for Micromanaged table when a insert > transaction is aborted > > > Key: HIVE-18693 > URL: https://issues.apache.org/jira/browse/HIVE-18693 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-18693.01.patch > > > TestTxnCommands2#writeBetweenWorkerAndCleaner with minor > changes (changing delete command to insert command) fails on MM table. > Specifically the last SELECT commands returns wrong results. > But this test works fine with full ACID table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[ https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379666#comment-16379666 ] Sergey Shelukhin commented on HIVE-18820: - [~thejas] [~aihuaxu] can you take a look? Before this patch and adding a log line to the cleanup code, I see tons of appenders being created in memory dump, but only a few cleanup calls. > Operation doesn't always clean up log4j for operation log > - > > Key: HIVE-18820 > URL: https://issues.apache.org/jira/browse/HIVE-18820 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18820.patch > > > Operation log may only be enabled for some operations, however the routing > appender still creates appenders for the queries where it's not enabled as > far as I see, that don't get cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[ https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18820: Status: Patch Available (was: Open) > Operation doesn't always clean up log4j for operation log > - > > Key: HIVE-18820 > URL: https://issues.apache.org/jira/browse/HIVE-18820 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18820.patch > > > Operation log may only be enabled for some operations, however the routing > appender still creates appenders for the queries where it's not enabled as > far as I see, that don't get cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[
https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379664#comment-16379664
]
Hive QA commented on HIVE-17937:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
57s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9395/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9395/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9395/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> llap_acid_fast test is flaky
>
>
> Key: HIVE-17937
> URL: https://issues.apache.org/jira/browse/HIVE-17937
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Teddy Choi
>Priority: Blocker
> Attachments: HIVE-17937.05.patch, HIVE-17937.2.patch,
> HIVE-17937.3.patch, HIVE-17937.4.patch, HIVE-17993.patch
>
>
> See for example
> https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/
> (the history link is the same from any build number with a test run, just
> replace 7521 if this one expires).
> Looks like results change, which may not be good.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[ https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18820: --- > Operation doesn't always clean up log4j for operation log > - > > Key: HIVE-18820 > URL: https://issues.apache.org/jira/browse/HIVE-18820 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18820.patch > > > Operation log may only be enabled for some operations, however the routing > appender still creates appenders for the queries where it's not enabled as > far as I see, that don't get cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18820) Operation doesn't always clean up log4j for operation log
[ https://issues.apache.org/jira/browse/HIVE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18820: Attachment: HIVE-18820.patch > Operation doesn't always clean up log4j for operation log > - > > Key: HIVE-18820 > URL: https://issues.apache.org/jira/browse/HIVE-18820 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18820.patch > > > Operation log may only be enabled for some operations, however the routing > appender still creates appenders for the queries where it's not enabled as > far as I see, that don't get cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379661#comment-16379661 ] Matt McCline commented on HIVE-18819: - An approach that creates (native) vector expressions instead of pushing code into VectorUDFAdaptor like HIVE-17139 was doing. > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18815) Remove unused feature in HPL/SQL
[
https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379659#comment-16379659
]
Hive QA commented on HIVE-18815:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912330/HIVE-18815.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 13017 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Assigned] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE
[ https://issues.apache.org/jira/browse/HIVE-18819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-18819: --- > Vectorization: Optimize IF statement expression evaluation of THEN/ELSE > --- > > Key: HIVE-18819 > URL: https://issues.apache.org/jira/browse/HIVE-18819 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > Currently, all the rows of a batch are evaluated for the THEN and ELSE > expressions even though only a value from one of them is needed for any > particular row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18695) fix TestAccumuloCliDriver.testCliDriver[accumulo_queries]
[ https://issues.apache.org/jira/browse/HIVE-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379649#comment-16379649 ] Anthony Hsu commented on HIVE-18695: +1 (non-binding – I am not a Hive committer) on your patch, [~kgyrtkirk] > fix TestAccumuloCliDriver.testCliDriver[accumulo_queries] > - > > Key: HIVE-18695 > URL: https://issues.apache.org/jira/browse/HIVE-18695 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18695.01.patch > > > seems to be broken by HIVE-15680 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18695) fix TestAccumuloCliDriver.testCliDriver[accumulo_queries]
[
https://issues.apache.org/jira/browse/HIVE-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379647#comment-16379647
]
Anthony Hsu commented on HIVE-18695:
[~elserj]
{quote}bq.[~erwaman], I'm confused on the phrasing: does HIVE-18802 apply both
before and after HIVE-15680? I think you're saying that it was an existing
problem, but it would also fix the test failures that HIVE-15680 caused?
{quote}
Yes, the bug reported in HIVE-18802 happens with or without the changes made in
HIVE-15680. I take back what I said about fixing HIVE-18802 also fixing
HIVE-15680. I think they are related but slightly different bugs. With
HIVE-18802, I think there's something funky going on with the way
AccumuloStorageHandler is handling predicate pushdown.
> fix TestAccumuloCliDriver.testCliDriver[accumulo_queries]
> -
>
> Key: HIVE-18695
> URL: https://issues.apache.org/jira/browse/HIVE-18695
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18695.01.patch
>
>
> seems to be broken by HIVE-15680
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18788) Clean up inputs in JDBC PreparedStatement
[ https://issues.apache.org/jira/browse/HIVE-18788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379631#comment-16379631 ] Thejas M Nair commented on HIVE-18788: -- +1 pending test verification > Clean up inputs in JDBC PreparedStatement > - > > Key: HIVE-18788 > URL: https://issues.apache.org/jira/browse/HIVE-18788 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18788.1.patch, HIVE-18788.2.patch, > HIVE-18788.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18815) Remove unused feature in HPL/SQL
[ https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379629#comment-16379629 ] Thejas M Nair commented on HIVE-18815: -- +1 > Remove unused feature in HPL/SQL > > > Key: HIVE-18815 > URL: https://issues.apache.org/jira/browse/HIVE-18815 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18815.1.patch > > > Remove FTP feature in HPL/SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-18810) Parquet Or ORC
[ https://issues.apache.org/jira/browse/HIVE-18810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair resolved HIVE-18810. -- Resolution: Invalid > Parquet Or ORC > -- > > Key: HIVE-18810 > URL: https://issues.apache.org/jira/browse/HIVE-18810 > Project: Hive > Issue Type: Test > Components: Hive >Affects Versions: 1.1.0 > Environment: Hadoop 1.2.1 > Hive 1.1 >Reporter: Suddhasatwa Bhaumik >Priority: Major > > Hello Experts, > I would like to know for which data types (based on size and complexity of > data) should one be using Parquet or ORC tables in Hive. E.g., On Hadoop > 0.20.0 with hive 0.13, the performance of ORC tables in Hive is very good > when accessed even by 3rd party BI systems like SAP Business Objects or > Tableau; performing the same tests on Hadoop 1.2.1 with Hive 1.1 does not > yield such reliability in queries, although ETL or insert/update of tables > are taking nominal time the read performance is not within acceptable limits. > In case of any queries, kindly advise. > Thanks > [~suddhasatwa_bhaumik] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18810) Parquet Or ORC
[ https://issues.apache.org/jira/browse/HIVE-18810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379624#comment-16379624 ] Thejas M Nair commented on HIVE-18810: -- Please use the mailing list for questions. You can find some data about ORC , Parquet performance in these slides - https://www.slideshare.net/oom65/file-format-benchmarks-avro-json-orc-parquet In general, reads would be where you see performance improvement. And you would see lot more performance improvement when ORC is combined with hive vectorization. Use Tez execution engine for more performance, and for even more performance improvement use LLAP for execution. > Parquet Or ORC > -- > > Key: HIVE-18810 > URL: https://issues.apache.org/jira/browse/HIVE-18810 > Project: Hive > Issue Type: Test > Components: Hive >Affects Versions: 1.1.0 > Environment: Hadoop 1.2.1 > Hive 1.1 >Reporter: Suddhasatwa Bhaumik >Priority: Major > > Hello Experts, > I would like to know for which data types (based on size and complexity of > data) should one be using Parquet or ORC tables in Hive. E.g., On Hadoop > 0.20.0 with hive 0.13, the performance of ORC tables in Hive is very good > when accessed even by 3rd party BI systems like SAP Business Objects or > Tableau; performing the same tests on Hadoop 1.2.1 with Hive 1.1 does not > yield such reliability in queries, although ETL or insert/update of tables > are taking nominal time the read performance is not within acceptable limits. > In case of any queries, kindly advise. > Thanks > [~suddhasatwa_bhaumik] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false
[
https://issues.apache.org/jira/browse/HIVE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379602#comment-16379602
]
Vineet Garg commented on HIVE-18818:
I have a fix, will upload the patch.
> Alter table add constraint unique fails with direct sql set to false
>
>
> Key: HIVE-18818
> URL: https://issues.apache.org/jira/browse/HIVE-18818
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> *Reproducer*
> {code:sql}
> set hive.metastore.try.direct.sql=false;
> CREATE TABLE numericDataType(a TINYINT CONSTRAINT tinyint_constraint DEFAULT
> 127Y ENABLE, b SMALLINT DEFAULT 32767S, c INT DEFAULT 2147483647,
> d BIGINT DEFAULT 9223372036854775807L, e DOUBLE DEFAULT 3.4E38, f
> DECIMAL(9,2) DEFAULT 1234567.89);
> ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;
> DESC FORMATTED numericDataType;
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> {code}
> *Error Stack*
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582)
>
[jira] [Assigned] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false
[
https://issues.apache.org/jira/browse/HIVE-18818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18818:
--
> Alter table add constraint unique fails with direct sql set to false
>
>
> Key: HIVE-18818
> URL: https://issues.apache.org/jira/browse/HIVE-18818
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> *Reproducer*
> {code:sql}
> set hive.metastore.try.direct.sql=false;
> CREATE TABLE numericDataType(a TINYINT CONSTRAINT tinyint_constraint DEFAULT
> 127Y ENABLE, b SMALLINT DEFAULT 32767S, c INT DEFAULT 2147483647,
> d BIGINT DEFAULT 9223372036854775807L, e DOUBLE DEFAULT 3.4E38, f
> DECIMAL(9,2) DEFAULT 1234567.89);
> ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;
> DESC FORMATTED numericDataType;
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> {code}
> *Error Stack*
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
>
[jira] [Commented] (HIVE-18815) Remove unused feature in HPL/SQL
[
https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379600#comment-16379600
]
Hive QA commented on HIVE-18815:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hplsql: The patch generated 0 new + 281 unchanged -
72 fixed = 281 total (was 353) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9394/dev-support/hive-personality.sh
|
| git revision | master / a0b2628 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9394/yetus/patch-asflicense-problems.txt
|
| modules | C: hplsql U: hplsql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9394/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unused feature in HPL/SQL
>
>
> Key: HIVE-18815
> URL: https://issues.apache.org/jira/browse/HIVE-18815
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18815.1.patch
>
>
> Remove FTP feature in HPL/SQL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16494) udaf percentile_approx() may fail on CBO
[ https://issues.apache.org/jira/browse/HIVE-16494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-16494: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) I couldn't repro failures in TestNegativeCliDriver for truncate_column_list_bucketing.q,truncate_nonexistant_column.q,truncate_partition_column.q,union22.q,update_notnull_constraint.q Pushed to master. Thanks, Miklos! > udaf percentile_approx() may fail on CBO > > > Key: HIVE-16494 > URL: https://issues.apache.org/jira/browse/HIVE-16494 > Project: Hive > Issue Type: Bug > Components: CBO, Logical Optimizer, UDF >Reporter: Ashutosh Chauhan >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-16494.patch > > > select percentile_approx(key, array(0.50, 0.70, 0.90, 0.95, 0.99)) from t; > fails with error : The second argument must be a constant. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379576#comment-16379576
]
Hive QA commented on HIVE-18746:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912328/HIVE-18746.4.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 13025 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18816) CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
[
https://issues.apache.org/jira/browse/HIVE-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18816:
--
Component/s: Transactions
> CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
> --
>
> Key: HIVE-18816
> URL: https://issues.apache.org/jira/browse/HIVE-18816
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> *Reproducer*
> {code:sql}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> CREATE TABLE table_acid(d int, tz timestamp with local time zone)
> clustered by (d) into 2 buckets stored as orc TBLPROPERTIES
> ('transactional'='true');
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException:
> Unknown primitive type TIMESTAMPLOCALTZ
> {code}
> *Error stack*
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Unknown primitive type TIMESTAMPLOCALTZ
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:906)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4788)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:389)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2314)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1985)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1687)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1438)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1427)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1345)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1319)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.11.jar:?]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.11.jar:?]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> [junit-4.11.jar:?]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> [junit-4.11.jar:?]
> at
[jira] [Commented] (HIVE-18608) ORC should allow selectively disabling dictionary-encoding on specified columns
[
https://issues.apache.org/jira/browse/HIVE-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379551#comment-16379551
]
Owen O'Malley commented on HIVE-18608:
--
I've just opened a jira and a pull request that is useful to this and other
changes that need to specify column names.
https://issues.apache.org/jira/browse/ORC-308
it allows you to specify subfields by name such as your example:
myInfoArray._elem_.emailBody.
> ORC should allow selectively disabling dictionary-encoding on specified
> columns
> ---
>
> Key: HIVE-18608
> URL: https://issues.apache.org/jira/browse/HIVE-18608
> Project: Hive
> Issue Type: New Feature
> Components: ORC
>Affects Versions: 3.0.0, 2.4.0, 2.2.1
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
>Priority: Major
> Attachments: HIVE-18608.1-branch-2.2.patch
>
>
> Just as ORC allows the choice of columns to enable bloom-filters on, it would
> be nice to have a way to specify which columns {{DICTIONARY_V2}} encoding
> should be disabled on.
> Currently, the choice of dictionary-encoding depends on the results of
> sampling the first row-stride within a stripe. If the user knows that a
> column's cardinality is bound to prevent an effective dictionary, she might
> choose to simply disable it on just that column, and avoid the cost of
> sampling in the first row-stride.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18608) ORC should allow selectively disabling dictionary-encoding on specified columns
[
https://issues.apache.org/jira/browse/HIVE-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379551#comment-16379551
]
Owen O'Malley edited comment on HIVE-18608 at 2/28/18 12:26 AM:
I've just opened a jira and a pull request that is useful to this and other
changes that need to specify column names.
https://issues.apache.org/jira/browse/ORC-308
it allows you to specify subfields by name such as your example:
myInfoArray.\_elem.emailBody.
was (Author: owen.omalley):
I've just opened a jira and a pull request that is useful to this and other
changes that need to specify column names.
https://issues.apache.org/jira/browse/ORC-308
it allows you to specify subfields by name such as your example:
myInfoArray._elem_.emailBody.
> ORC should allow selectively disabling dictionary-encoding on specified
> columns
> ---
>
> Key: HIVE-18608
> URL: https://issues.apache.org/jira/browse/HIVE-18608
> Project: Hive
> Issue Type: New Feature
> Components: ORC
>Affects Versions: 3.0.0, 2.4.0, 2.2.1
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
>Priority: Major
> Attachments: HIVE-18608.1-branch-2.2.patch
>
>
> Just as ORC allows the choice of columns to enable bloom-filters on, it would
> be nice to have a way to specify which columns {{DICTIONARY_V2}} encoding
> should be disabled on.
> Currently, the choice of dictionary-encoding depends on the results of
> sampling the first row-stride within a stripe. If the user knows that a
> column's cardinality is bound to prevent an effective dictionary, she might
> choose to simply disable it on just that column, and avoid the cost of
> sampling in the first row-stride.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379531#comment-16379531
]
Hive QA commented on HIVE-18746:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
3s{color} | {color:red} root: The patch generated 9 new + 60 unchanged - 1
fixed = 69 total (was 61) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 9 new + 53 unchanged - 1 fixed
= 62 total (was 54) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9393/dev-support/hive-personality.sh
|
| git revision | master / 388945f |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9393/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9393/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9393/yetus/patch-asflicense-problems.txt
|
| modules | C: common . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9393/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add_months should validate the date first
> -
>
> Key: HIVE-18746
> URL: https://issues.apache.org/jira/browse/HIVE-18746
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Subhasis Gorai
>Assignee: Kryvenko Igor
>Priority: Minor
> Attachments: HIVE-18746.1.patch, HIVE-18746.3.patch,
> HIVE-18746.4.patch, HIVE-18746.patch
>
>
> hive (sbg_hvc_ods)> select add_months('2017-02-28', 1);
> OK
> _c0
> 2017-03-31
> Time taken: 0.107 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)> select add_months('2017-02-29', 1);
> OK
> _c0
> 2017-04-01
> Time taken: 0.084 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)>
>
> '2017-02-29' is an invalid date.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID
[ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379515#comment-16379515 ] Sergey Shelukhin commented on HIVE-18192: - [~sankarh] thanks! Is there a ticket #? > Introduce WriteID per table rather than using global transaction ID > --- > > Key: HIVE-18192 > URL: https://issues.apache.org/jira/browse/HIVE-18192 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, DR, pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch, > HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch, > HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch, > HIVE-18192.09.patch, HIVE-18192.10.patch, HIVE-18192.11.patch, > HIVE-18192.12.patch, HIVE-18192.13.patch, HIVE-18192.14.patch, > HIVE-18192.15.patch, HIVE-18192.16.patch, HIVE-18192.17.patch > > > To support ACID replication, we will be introducing a per table write Id > which will replace the transaction id in the primary key for each row in a > ACID table. > The current primary key is determined via > > which will move to > > For each table modified by the given transaction will have a table level > write ID allocated and a persisted map of global txn id -> to table -> write > id for that table has to be maintained to allow Snapshot isolation. > Readers should use the combination of ValidTxnList and > ValidWriteIdList(Table) for snapshot isolation. > > [Hive Replication - ACID > Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf] > has a section "Per Table Sequences (Write-Id)" with more detials -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18796) fix TestSSL
[ https://issues.apache.org/jira/browse/HIVE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379512#comment-16379512 ] Sergey Shelukhin commented on HIVE-18796: - This fixes TestSSL. [~kgyrtkirk] can you review? > fix TestSSL > --- > > Key: HIVE-18796 > URL: https://issues.apache.org/jira/browse/HIVE-18796 > Project: Hive > Issue Type: Bug > Components: Test >Reporter: Zoltan Haindrich >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18796.patch > > > broken by HIVE-18203 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379511#comment-16379511 ] Jesus Camacho Rodriguez commented on HIVE-18776: I have pushed this fix to master, as it will fix the race condition that [~alangates] identified. TestNegativeCliDriver does not seem to be related as it has been timing out consistently for other patches too. > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18776: --- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere reassigned HIVE-18817:
-
Assignee: Eugene Koifman
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-18817:
--
Attachment: repro.patch
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Priority: Major
> Attachments: repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379491#comment-16379491 ] Jesus Camacho Rodriguez commented on HIVE-18776: I take it back, not sure if it is related, as it is failing for other patches too. > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379486#comment-16379486 ] Jesus Camacho Rodriguez commented on HIVE-18776: [~alangates], though I cannot repro locally, there seems to be some race condition with TestNegativeCliDriver still, as it timed out in all runs with the patch. I need to explore further before pushing it. > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18816) CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
[
https://issues.apache.org/jira/browse/HIVE-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18816:
--
Assignee: Jesus Camacho Rodriguez
> CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
> --
>
> Key: HIVE-18816
> URL: https://issues.apache.org/jira/browse/HIVE-18816
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> *Reproducer*
> {code:sql}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> CREATE TABLE table_acid(d int, tz timestamp with local time zone)
> clustered by (d) into 2 buckets stored as orc TBLPROPERTIES
> ('transactional'='true');
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException:
> Unknown primitive type TIMESTAMPLOCALTZ
> {code}
> *Error stack*
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Unknown primitive type TIMESTAMPLOCALTZ
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:906)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4788)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:389)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2314)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1985)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1687)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1438)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1427)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1345)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1319)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.11.jar:?]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.11.jar:?]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> [junit-4.11.jar:?]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> [junit-4.11.jar:?]
> at
>
[jira] [Commented] (HIVE-18816) CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
[
https://issues.apache.org/jira/browse/HIVE-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379478#comment-16379478
]
Vineet Garg commented on HIVE-18816:
cc [~jcamachorodriguez]
> CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
> --
>
> Key: HIVE-18816
> URL: https://issues.apache.org/jira/browse/HIVE-18816
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Vineet Garg
>Priority: Major
>
> *Reproducer*
> {code:sql}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> CREATE TABLE table_acid(d int, tz timestamp with local time zone)
> clustered by (d) into 2 buckets stored as orc TBLPROPERTIES
> ('transactional'='true');
> {code}
> *Error*
> {code:sql}
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException:
> Unknown primitive type TIMESTAMPLOCALTZ
> {code}
> *Error stack*
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Unknown primitive type TIMESTAMPLOCALTZ
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:906)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4788)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:389)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2314)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1985)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1687)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1438)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1427)
> [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> [hive-cli-3.0.0-SNAPSHOT.jar:?]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1345)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1319)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.11.jar:?]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> [junit-4.11.jar:?]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.11.jar:?]
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> [junit-4.11.jar:?]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> [junit-4.11.jar:?]
> at
>
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[
https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379471#comment-16379471
]
Hive QA commented on HIVE-18776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912314/HIVE-18776.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 13016 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[
https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379391#comment-16379391
]
Hive QA commented on HIVE-18776:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} standalone-metastore: The patch generated 0 new +
546 unchanged - 1 fixed = 546 total (was 547) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9392/dev-support/hive-personality.sh
|
| git revision | master / 388945f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9392/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9392/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> MaterializationsInvalidationCache loading causes race condition in the
> metastore
>
>
> Key: HIVE-18776
> URL: https://issues.apache.org/jira/browse/HIVE-18776
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch,
> HIVE-18776.patch
>
>
> I am seeing occasional failures running metastore tests where operations are
> failing saying that there is no open transaction. I have traced this to a
> race condition in loading the materialized view invalidation cache. When it
> is initialized (either in HiveMetaStoreClient in embedded mode or in
> HiveMetaStore in remote mode) it grabs a copy of the current RawStore
> instance and then loads the cache in a separate thread. But ObjectStore
> keeps state regarding JDO transactions with the underlying RDBMS. So with
> the loader thread and the initial thread both doing operations against the
> RawStore they sometimes mess up each others transaction stack. In a quick
> test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work.
> A reference to the TxnHandler is also called. I suspect this will run into a
> similar issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18659) add acid version marker to acid files/directories
[ https://issues.apache.org/jira/browse/HIVE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18659: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) committed to master thanks Prasanth for the review > add acid version marker to acid files/directories > - > > Key: HIVE-18659 > URL: https://issues.apache.org/jira/browse/HIVE-18659 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18659.01.patch, HIVE-18659.04.patch, > HIVE-18659.05.patch, HIVE-18659.06.patch, HIVE-18659.07.patch, > HIVE-18659.09.patch, HIVE-18659.09.patch, HIVE-18659.10.patch, > HIVE-18659.11.patch, HIVE-18659.12.patch, HIVE-18659.13.patch, > HIVE-18659.14.patch > > > add acid version marker to acid files so that we know which version of acid > wrote the file -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17626) Query reoptimization using cached runtime statistics
[
https://issues.apache.org/jira/browse/HIVE-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379361#comment-16379361
]
Hive QA commented on HIVE-17626:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912311/HIVE-17626.05.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 13424 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17937: Attachment: HIVE-17937.05.patch > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Blocker > Attachments: HIVE-17937.05.patch, HIVE-17937.2.patch, > HIVE-17937.3.patch, HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379351#comment-16379351 ] Sergey Shelukhin commented on HIVE-17937: - I've rerun the tests locally and updated the patch. Query results are consistent between the two drivers in both tests. Will commit on a good test run (of those two tests) > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi >Priority: Blocker > Attachments: HIVE-17937.05.patch, HIVE-17937.2.patch, > HIVE-17937.3.patch, HIVE-17937.4.patch, HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17626) Query reoptimization using cached runtime statistics
[
https://issues.apache.org/jira/browse/HIVE-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379333#comment-16379333
]
Hive QA commented on HIVE-17626:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
16s{color} | {color:red} root: The patch generated 60 new + 1809 unchanged -
157 fixed = 1869 total (was 1966) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} druid-handler: The patch generated 0 new + 1
unchanged - 1 fixed = 1 total (was 2) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} itests/util: The patch generated 0 new + 103
unchanged - 1 fixed = 103 total (was 104) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 59 new + 1277 unchanged - 153
fixed = 1336 total (was 1430) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore: The patch generated 1 new + 3
unchanged - 2 fixed = 4 total (was 5) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
9s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9391/dev-support/hive-personality.sh
|
| git revision | master / 5b2ab39 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9391/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9391/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9391/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9391/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9391/yetus/patch-asflicense-problems.txt
|
| modules | C: common . druid-handler
[jira] [Updated] (HIVE-18815) Remove unused feature in HPL/SQL
[ https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-18815: -- Status: Patch Available (was: Open) > Remove unused feature in HPL/SQL > > > Key: HIVE-18815 > URL: https://issues.apache.org/jira/browse/HIVE-18815 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18815.1.patch > > > Remove FTP feature in HPL/SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18815) Remove unused feature in HPL/SQL
[ https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-18815: -- Attachment: HIVE-18815.1.patch > Remove unused feature in HPL/SQL > > > Key: HIVE-18815 > URL: https://issues.apache.org/jira/browse/HIVE-18815 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18815.1.patch > > > Remove FTP feature in HPL/SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18815) Remove unused feature in HPL/SQL
[ https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned HIVE-18815: - > Remove unused feature in HPL/SQL > > > Key: HIVE-18815 > URL: https://issues.apache.org/jira/browse/HIVE-18815 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > > Remove FTP feature in HPL/SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-18692) Remove HPL/SQL From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved HIVE-18692. --- Resolution: Won't Fix As discussed in mailing list, we will give HPL some time to mature. Close it as we don't plan to remove it in near future. > Remove HPL/SQL From Apache Hive > --- > > Key: HIVE-18692 > URL: https://issues.apache.org/jira/browse/HIVE-18692 > Project: Hive > Issue Type: Improvement > Components: hpl/sql >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Priority: Major > > I don't know anyone using HPL/SQL [HIVE-11055]. Perhaps we can just remove > it from Hive 3.0? It perhaps should be in another project since it's stated > that it can be used for any JDBC connection. Maybe something like > commons-dbutils ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kryvenko Igor updated HIVE-18746:
-
Attachment: HIVE-18746.4.patch
> add_months should validate the date first
> -
>
> Key: HIVE-18746
> URL: https://issues.apache.org/jira/browse/HIVE-18746
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Subhasis Gorai
>Assignee: Kryvenko Igor
>Priority: Minor
> Attachments: HIVE-18746.1.patch, HIVE-18746.3.patch,
> HIVE-18746.4.patch, HIVE-18746.patch
>
>
> hive (sbg_hvc_ods)> select add_months('2017-02-28', 1);
> OK
> _c0
> 2017-03-31
> Time taken: 0.107 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)> select add_months('2017-02-29', 1);
> OK
> _c0
> 2017-04-01
> Time taken: 0.084 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)>
>
> '2017-02-29' is an invalid date.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17735) ObjectStore.addNotificationEvent is leaking queries
[
https://issues.apache.org/jira/browse/HIVE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379270#comment-16379270
]
Thejas M Nair commented on HIVE-17735:
--
[~aihuaxu]
Looks like this needs to be marked as resolved with a fix version ?
> ObjectStore.addNotificationEvent is leaking queries
> ---
>
> Key: HIVE-17735
> URL: https://issues.apache.org/jira/browse/HIVE-17735
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-17735.1.patch, HIVE-17735.2.patch
>
>
> In ObjectStore.addNotificationEvent():
> {code}
> Query objectQuery = pm.newQuery(MNotificationNextId.class);
> Collection ids = (Collection)
> objectQuery.execute();
> {code}
> The query is never closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18659) add acid version marker to acid files/directories
[
https://issues.apache.org/jira/browse/HIVE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379246#comment-16379246
]
Hive QA commented on HIVE-18659:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912308/HIVE-18659.14.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 13409 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Assigned] (HIVE-18814) Support Add Partition For Acid tables
[
https://issues.apache.org/jira/browse/HIVE-18814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman reassigned HIVE-18814:
-
> Support Add Partition For Acid tables
> -
>
> Key: HIVE-18814
> URL: https://issues.apache.org/jira/browse/HIVE-18814
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual%2BDDL#LanguageManualDDL-AddPartitions]
> Add Partition command creates a \{{Partition}} metadata object and set the
> location to the directory containing data files.
> In current master (Hive 3.0), Add partition on an acid table doesn't fail and
> at read time the data is decorated with row__id but the original transaction
> is 0. I suspect in earlier Hive versions this will throw or return no data.
>
> One option is follow Load Data approach and create a new delta_x_x/ and
> move/copy the data there.
>
> Another is to allocate a new writeid and save it in Partition metadata. This
> could then be used to decorate data with ROW__IDs. This avoids move/copy but
> retains data "outside" of the table tree which make it more likely that this
> data will be modified in some way which can really break things if done after
> and SQL update/delete on this data have happened.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence Gby
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379190#comment-16379190
]
Julian Hyde commented on HIVE-16924:
I added a new patch, HIVE-16924.06.patch, based upon
https://github.com/julianhyde/hive/tree/16924-distinct-group-by-squashed. It
has the same content as HIVE-16924.05.patch, except that I have rebased to
latest master, squashed, and cleaned up (e.g. removed commented out code). Most
tests are succeeding, but the 7 failures listed above probably still remain.
> Support distinct in presence Gby
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Julian Hyde
>Priority: Major
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch
>
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence Gby
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated HIVE-16924:
---
Attachment: HIVE-16924.06.patch
> Support distinct in presence Gby
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Julian Hyde
>Priority: Major
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch
>
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence Gby
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated HIVE-16924:
---
Status: Open (was: Patch Available)
> Support distinct in presence Gby
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Julian Hyde
>Priority: Major
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch
>
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18659) add acid version marker to acid files/directories
[
https://issues.apache.org/jira/browse/HIVE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379185#comment-16379185
]
Hive QA commented on HIVE-18659:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 2 new + 1010 unchanged - 22
fixed = 1012 total (was 1032) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9390/dev-support/hive-personality.sh
|
| git revision | master / 5b2ab39 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9390/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9390/yetus/patch-asflicense-problems.txt
|
| modules | C: hcatalog/streaming ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9390/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add acid version marker to acid files/directories
> -
>
> Key: HIVE-18659
> URL: https://issues.apache.org/jira/browse/HIVE-18659
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18659.01.patch, HIVE-18659.04.patch,
> HIVE-18659.05.patch, HIVE-18659.06.patch, HIVE-18659.07.patch,
> HIVE-18659.09.patch, HIVE-18659.09.patch, HIVE-18659.10.patch,
> HIVE-18659.11.patch, HIVE-18659.12.patch, HIVE-18659.13.patch,
> HIVE-18659.14.patch
>
>
> add acid version marker to acid files so that we know which version of acid
> wrote the file
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18750) Exchange partition should be disabled on ACID/Insert-only tables with per table write ID.
[
https://issues.apache.org/jira/browse/HIVE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379150#comment-16379150
]
Hive QA commented on HIVE-18750:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912299/HIVE-18750.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 13017 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18776: --- Attachment: HIVE-18776.02.patch > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
[ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379147#comment-16379147 ] Jesus Camacho Rodriguez commented on HIVE-18776: Thanks [~alangates], I am running ptests again. > MaterializationsInvalidationCache loading causes race condition in the > metastore > > > Key: HIVE-18776 > URL: https://issues.apache.org/jira/browse/HIVE-18776 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18776.01.patch, HIVE-18776.02.patch, > HIVE-18776.patch > > > I am seeing occasional failures running metastore tests where operations are > failing saying that there is no open transaction. I have traced this to a > race condition in loading the materialized view invalidation cache. When it > is initialized (either in HiveMetaStoreClient in embedded mode or in > HiveMetaStore in remote mode) it grabs a copy of the current RawStore > instance and then loads the cache in a separate thread. But ObjectStore > keeps state regarding JDO transactions with the underlying RDBMS. So with > the loader thread and the initial thread both doing operations against the > RawStore they sometimes mess up each others transaction stack. In a quick > test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. > A reference to the TxnHandler is also called. I suspect this will run into a > similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17626) Query reoptimization using cached runtime statistics
[ https://issues.apache.org/jira/browse/HIVE-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17626: Attachment: HIVE-17626.05.patch > Query reoptimization using cached runtime statistics > > > Key: HIVE-17626 > URL: https://issues.apache.org/jira/browse/HIVE-17626 > Project: Hive > Issue Type: New Feature > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-17626.01.patch, HIVE-17626.01wip01.patch, > HIVE-17626.02.patch, HIVE-17626.03.patch, HIVE-17626.04.patch, > HIVE-17626.05.patch, runtimestats.patch > > > Something similar to "EXPLAIN ANALYZE" where we annotate explain plan with > actual and estimated statistics. The runtime stats can be cached at query > level and subsequent execution of the same query can make use of the cached > statistics from the previous run for better optimization. > Some use cases, > 1) re-planning join query (mapjoin failures can be converted to shuffle joins) > 2) better statistics for table scan operator if dynamic partition pruning is > involved > 3) Better estimates for bloom filter initialization (setting expected entries > during merge) > This can extended to support wider queries by caching fragments of operator > plans scanning same table(s) or matching some operator sequences. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
