[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418463#comment-16418463
]
Rui Li commented on HIVE-18831:
---
[~stakiar], I think the right way to register the serializer is to call
{{Kryo::register(Class type, Serializer serializer)}}.
> Differentiate errors that are thrown by Spark tasks
> ---
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch,
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch,
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we
> don't differentiate between errors from HS2 / RSC vs. errors thrown by
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its
> difficult to know what part of the excution threw the exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18268) Hive Prepared Statement when split with double quoted in query fails
[
https://issues.apache.org/jira/browse/HIVE-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418446#comment-16418446
]
Hive QA commented on HIVE-18268:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} jdbc: The patch generated 11 new + 105 unchanged - 4
fixed = 116 total (was 109) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 10 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9897/dev-support/hive-personality.sh
|
| git revision | master / c018869 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9897/yetus/diff-checkstyle-jdbc.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9897/yetus/whitespace-tabs.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9897/yetus/patch-asflicense-problems.txt
|
| modules | C: jdbc U: jdbc |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9897/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive Prepared Statement when split with double quoted in query fails
>
>
> Key: HIVE-18268
> URL: https://issues.apache.org/jira/browse/HIVE-18268
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 2.3.2
>Reporter: Choi JaeHwan
>Assignee: Choi JaeHwan
>Priority: Major
> Attachments: HIVE-18268.1.patch, HIVE-18268.2.patch,
> HIVE-18268.3.patch, HIVE-18268.4.patch, HIVE-18268.patch
>
>
> HIVE-13625, Change sql statement split when odd number of escape characters,
> and add parameter counter validation, above
> {code:java}
> // prev code
> StringBuilder newSql = new StringBuilder(parts.get(0));
> for(int i=1;i if(!parameters.containsKey(i)){
> throw new SQLException("Parameter #"+i+" is unset");
> }
> newSql.append(parameters.get(i));
> newSql.append(parts.get(i));
> }
> // change from HIVE-13625
> int paramLoc = 1;
> while (getCharIndexFromSqlByParamLocation(sql, '?', paramLoc) > 0) {
> // check the user has set the needs parameters
> if (parameters.containsKey(paramLoc)) {
> int tt = getCharIndexFromSqlByParamLocation(newSql.toString(), '?',
> 1);
> newSql.deleteCharAt(tt);
> newSql.insert(tt, parameters.get(paramLoc));
> }
> paramLoc++;
> }
> {code}
> If the number of split SQL and the number
[jira] [Commented] (HIVE-19061) WM needs to output an event for allocation update
[
https://issues.apache.org/jira/browse/HIVE-19061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418433#comment-16418433
]
Hive QA commented on HIVE-19061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916461/HIVE-19061.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 13047 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-19048) Initscript errors are ignored
[
https://issues.apache.org/jira/browse/HIVE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418432#comment-16418432
]
Bharathkrishna Guruvayoor Murali commented on HIVE-19048:
-
I performed the following command and noting the results below:
{code:java}
$ cat init.sql
invalid;
select 1;
{code}
{code:java}
$ ./beeline -u jdbc:hive2://localhost:1/ -n hive -i init.sql --force=true
0: jdbc:hive2://localhost:1/> invalid;
Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot
recognize input near 'invalid' '' '' (state=42000,code=4)
0: jdbc:hive2://localhost:1/> select 1;
[...]
+--+
| _c0 |
+--+
| 1 |
+--+
1 row selected (0.32 seconds)
{code}
When I gave the --force option as true, both statements in the init.sql file
were executed.
If the --force option is not used(which defaults to false), only the first
statement in init.sql is executed and it does not proceed after the error in
the first statement.
Can you please share the exact command you used with --force option where it
did not execute all the lines after the error occurred?
> Initscript errors are ignored
> -
>
> Key: HIVE-19048
> URL: https://issues.apache.org/jira/browse/HIVE-19048
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
>
> I've been running some queries for a while when I've noticed that my
> initscript has an error; and beeline stops interpreting the initscript after
> encountering the first error.
> {code}
> echo 'invalid;' > init.sql
> echo 'select 1;' > s1.sql
> beeline -u jdbc:hive2://localhost:1/ -n hive -i init.sql -f s1.sql
> [...]
> Running init script init.sql
> 0: jdbc:hive2://localhost:1/> invalid;
> Error: Error while compiling statement: FAILED: ParseException line 1:0
> cannot recognize input near 'invalid' '' '' (state=42000,code=4)
> 0: jdbc:hive2://localhost:1/> select 1;
> [...]
> $ echo $?
> 0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19073) StatsOptimizer may mangle constant columns
[
https://issues.apache.org/jira/browse/HIVE-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-19073:
---
Attachment: HIVE-19073.patch
> StatsOptimizer may mangle constant columns
> --
>
> Key: HIVE-19073
> URL: https://issues.apache.org/jira/browse/HIVE-19073
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.2.2
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19073.patch
>
>
> Following query is giving wrong result:
> {code:sql}
> SELECT DATE_SUB(CURRENT_DATE,0) as GROUP_BY_FIELD, count (*) as src_cnt from
> mytable WHERE 1=1 group by DATE_SUB(CURRENT_DATE,0);
> +-+--+--+
> | group_by_field | src_cnt |
> +-+--+--+
> | 239 | NULL |
> +-+--+--+
> 1 row selected (5.175 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-19073) StatsOptimizer may mangle constant columns
[
https://issues.apache.org/jira/browse/HIVE-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-19073 started by Jesus Camacho Rodriguez.
--
> StatsOptimizer may mangle constant columns
> --
>
> Key: HIVE-19073
> URL: https://issues.apache.org/jira/browse/HIVE-19073
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.2.2
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Following query is giving wrong result:
> {code:sql}
> SELECT DATE_SUB(CURRENT_DATE,0) as GROUP_BY_FIELD, count (*) as src_cnt from
> mytable WHERE 1=1 group by DATE_SUB(CURRENT_DATE,0);
> +-+--+--+
> | group_by_field | src_cnt |
> +-+--+--+
> | 239 | NULL |
> +-+--+--+
> 1 row selected (5.175 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19073) StatsOptimizer may mangle constant columns
[
https://issues.apache.org/jira/browse/HIVE-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-19073:
---
Status: Patch Available (was: In Progress)
> StatsOptimizer may mangle constant columns
> --
>
> Key: HIVE-19073
> URL: https://issues.apache.org/jira/browse/HIVE-19073
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.2.2
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Following query is giving wrong result:
> {code:sql}
> SELECT DATE_SUB(CURRENT_DATE,0) as GROUP_BY_FIELD, count (*) as src_cnt from
> mytable WHERE 1=1 group by DATE_SUB(CURRENT_DATE,0);
> +-+--+--+
> | group_by_field | src_cnt |
> +-+--+--+
> | 239 | NULL |
> +-+--+--+
> 1 row selected (5.175 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19073) StatsOptimizer may mangle constant columns
[
https://issues.apache.org/jira/browse/HIVE-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-19073:
--
> StatsOptimizer may mangle constant columns
> --
>
> Key: HIVE-19073
> URL: https://issues.apache.org/jira/browse/HIVE-19073
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.2.2
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Following query is giving wrong result:
> {code:sql}
> SELECT DATE_SUB(CURRENT_DATE,0) as GROUP_BY_FIELD, count (*) as src_cnt from
> mytable WHERE 1=1 group by DATE_SUB(CURRENT_DATE,0);
> +-+--+--+
> | group_by_field | src_cnt |
> +-+--+--+
> | 239 | NULL |
> +-+--+--+
> 1 row selected (5.175 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19047) Only the first init file is interpreted
[
https://issues.apache.org/jira/browse/HIVE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418418#comment-16418418
]
Bharathkrishna Guruvayoor Murali commented on HIVE-19047:
-
Fixing the checkstyle errors.
[~kgyrtkirk] Thanks for taking a look at the patch.
I added the variable executionResult in such a way that it would return
ERRNO_OK only if all initFiles are executed without any errors.
Can you let me know why the variable is not required here?
> Only the first init file is interpreted
> ---
>
> Key: HIVE-19047
> URL: https://issues.apache.org/jira/browse/HIVE-19047
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19047.1.patch, HIVE-19047.2.patch,
> HIVE-19047.3.patch
>
>
> I've passed multiple {{-i}} options to beeline; and I've expected to load
> both of them...but unfortunately it only parsed the first file (and ignored
> the second entirely)
> I think it would be better to:
> * either reject the command if it has multiple "-i" given
> * or load *all* "-i" scripts
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19047) Only the first init file is interpreted
[
https://issues.apache.org/jira/browse/HIVE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali updated HIVE-19047:
Attachment: HIVE-19047.3.patch
> Only the first init file is interpreted
> ---
>
> Key: HIVE-19047
> URL: https://issues.apache.org/jira/browse/HIVE-19047
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19047.1.patch, HIVE-19047.2.patch,
> HIVE-19047.3.patch
>
>
> I've passed multiple {{-i}} options to beeline; and I've expected to load
> both of them...but unfortunately it only parsed the first file (and ignored
> the second entirely)
> I think it would be better to:
> * either reject the command if it has multiple "-i" given
> * or load *all* "-i" scripts
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19061) WM needs to output an event for allocation update
[
https://issues.apache.org/jira/browse/HIVE-19061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418391#comment-16418391
]
Hive QA commented on HIVE-19061:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-9896/patches/PreCommit-HIVE-Build-9896.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9896/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> WM needs to output an event for allocation update
> -
>
> Key: HIVE-19061
> URL: https://issues.apache.org/jira/browse/HIVE-19061
> Project: Hive
> Issue Type: Bug
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19061.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18115) Fix schema version info for Hive-2.3.2
[
https://issues.apache.org/jira/browse/HIVE-18115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418377#comment-16418377
]
Hive QA commented on HIVE-18115:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898821/HIVE-18115.02-branch-2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10667 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=227)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=142)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9895/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9895/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9895/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898821 - PreCommit-HIVE-Build
> Fix schema version info for Hive-2.3.2
> --
>
> Key: HIVE-18115
> URL: https://issues.apache.org/jira/browse/HIVE-18115
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.1, 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Minor
> Fix For: 2.4.0
>
> Attachments: HIVE-18115.02-branch-2.patch, HIVE-18115.1.patch
>
>
> Error while starting HiveMeta
> {code}
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema
> version 2.3.2 does not match metastore's schema version 2.3.0 Metastore is
> not upgraded or corrupt
> at
> org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7600)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7563)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at com.sun.proxy.$Proxy23.verifySchema(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:591)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:584)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:651)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:427)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
>
[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418367#comment-16418367
]
Sahil Takiar commented on HIVE-18831:
-
[~lirui] thats a good idea. I spent a while trying to get that to work, but am
having some trouble with Kryo. I attached what I have so far, any errors stick
out to you?
The custom serializer is called {{JobResultSerializer}}. I wrote a few tests
for it in {{TestJobResultSerializer}}, right now
{{testSerializeNonSerializableObjectWriteObject}} succeeds, but
{{testSerializeNonSerializableObjectWriteClassAndObject}} fails. And I added an
integration test in {{TestSparkClient}} called {{testErrorJobNotSerializable}}
which fails with the same error.
Error:
{code}
com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID:
13994
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
at
org.apache.hive.spark.client.TestJobResultSerializer.testSerializeNonSerializableObjectWriteClassAndObject(TestJobResultSerializer.java:79)
{code}
Any ideas how to fix this? I tried registering the classes explicitly, but that
didn't help. What confuses me is why
{{testSerializeNonSerializableObjectWriteObject}} works, but
{{testSerializeNonSerializableObjectWriteClassAndObject}} fails.
> Differentiate errors that are thrown by Spark tasks
> ---
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch,
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch,
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we
> don't differentiate between errors from HS2 / RSC vs. errors thrown by
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its
> difficult to know what part of the excution threw the exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.8.WIP.patch > Differentiate errors that are thrown by Spark tasks > --- > > Key: HIVE-18831 > URL: https://issues.apache.org/jira/browse/HIVE-18831 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, > HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, > HIVE-18831.7.patch, HIVE-18831.8.WIP.patch > > > We propagate exceptions from Spark task failures to the client well, but we > don't differentiate between errors from HS2 / RSC vs. errors thrown by > individual tasks. > Main motivation is that when the client sees a propagated Spark exception its > difficult to know what part of the excution threw the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18770) Additional tests and fixes for materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418338#comment-16418338
]
Hive QA commented on HIVE-18770:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916648/HIVE-18770.02.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 162 failed/errored test(s), 13453 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Updated] (HIVE-19072) incorrect token handling for LLAP plugin endpoint
[ https://issues.apache.org/jira/browse/HIVE-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19072: Status: Patch Available (was: Open) [~jdere] another tiny patch :) > incorrect token handling for LLAP plugin endpoint > - > > Key: HIVE-19072 > URL: https://issues.apache.org/jira/browse/HIVE-19072 > Project: Hive > Issue Type: Bug >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19072.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19072) incorrect token handling for LLAP plugin endpoint
[ https://issues.apache.org/jira/browse/HIVE-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19072: Attachment: HIVE-19072.patch > incorrect token handling for LLAP plugin endpoint > - > > Key: HIVE-19072 > URL: https://issues.apache.org/jira/browse/HIVE-19072 > Project: Hive > Issue Type: Bug >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19072.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19072) incorrect token handling for LLAP plugin endpoint
[ https://issues.apache.org/jira/browse/HIVE-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19072: --- > incorrect token handling for LLAP plugin endpoint > - > > Key: HIVE-19072 > URL: https://issues.apache.org/jira/browse/HIVE-19072 > Project: Hive > Issue Type: Bug >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18770) Additional tests and fixes for materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418316#comment-16418316
]
Hive QA commented on HIVE-18770:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 24 new + 409 unchanged - 13
fixed = 433 total (was 422) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9894/dev-support/hive-personality.sh
|
| git revision | master / c018869 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9894/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9894/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9894/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9894/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Additional tests and fixes for materialized view rewriting
> --
>
> Key: HIVE-18770
> URL: https://issues.apache.org/jira/browse/HIVE-18770
> Project: Hive
> Issue Type: Improvement
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18770.02.patch
>
>
> This patch requires upgrading to Calcite 1.16.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19049) Add support for Alter table add columns for Druid
[
https://issues.apache.org/jira/browse/HIVE-19049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418299#comment-16418299
]
Hive QA commented on HIVE-19049:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916558/HIVE-19049.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 78 failed/errored test(s), 13445 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-19024) Vectorization: Disable complex type constants for VectorUDFAdaptor
[ https://issues.apache.org/jira/browse/HIVE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19024: Attachment: HIVE-19024.02.patch > Vectorization: Disable complex type constants for VectorUDFAdaptor > -- > > Key: HIVE-19024 > URL: https://issues.apache.org/jira/browse/HIVE-19024 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19024.01.patch, HIVE-19024.02.patch > > > Currently, complex type constants are not detected and cause execution > failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19024) Vectorization: Disable complex type constants for VectorUDFAdaptor
[ https://issues.apache.org/jira/browse/HIVE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19024: Status: Patch Available (was: In Progress) > Vectorization: Disable complex type constants for VectorUDFAdaptor > -- > > Key: HIVE-19024 > URL: https://issues.apache.org/jira/browse/HIVE-19024 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19024.01.patch, HIVE-19024.02.patch > > > Currently, complex type constants are not detected and cause execution > failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19024) Vectorization: Disable complex type constants for VectorUDFAdaptor
[ https://issues.apache.org/jira/browse/HIVE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19024: Status: In Progress (was: Patch Available) > Vectorization: Disable complex type constants for VectorUDFAdaptor > -- > > Key: HIVE-19024 > URL: https://issues.apache.org/jira/browse/HIVE-19024 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19024.01.patch, HIVE-19024.02.patch > > > Currently, complex type constants are not detected and cause execution > failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19032) Vectorization: Disable GROUP BY aggregations with DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19032: Status: Patch Available (was: In Progress) > Vectorization: Disable GROUP BY aggregations with DISTINCT > -- > > Key: HIVE-19032 > URL: https://issues.apache.org/jira/browse/HIVE-19032 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19032.01.patch, HIVE-19032.02.patch > > > Vectorized GROUP BY does not support DISTINCT aggregation functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19032) Vectorization: Disable GROUP BY aggregations with DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19032: Attachment: HIVE-19032.02.patch > Vectorization: Disable GROUP BY aggregations with DISTINCT > -- > > Key: HIVE-19032 > URL: https://issues.apache.org/jira/browse/HIVE-19032 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19032.01.patch, HIVE-19032.02.patch > > > Vectorized GROUP BY does not support DISTINCT aggregation functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19032) Vectorization: Disable GROUP BY aggregations with DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19032: Status: In Progress (was: Patch Available) > Vectorization: Disable GROUP BY aggregations with DISTINCT > -- > > Key: HIVE-19032 > URL: https://issues.apache.org/jira/browse/HIVE-19032 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19032.01.patch, HIVE-19032.02.patch > > > Vectorized GROUP BY does not support DISTINCT aggregation functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19012) Support builds for ARM and PPC arch
[ https://issues.apache.org/jira/browse/HIVE-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19012: - Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Thanks for the patch [~vilamon]! Committed to master. > Support builds for ARM and PPC arch > --- > > Key: HIVE-19012 > URL: https://issues.apache.org/jira/browse/HIVE-19012 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Vi On >Assignee: Vi On >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19012.patch, HIVE-19012.patch > > > Hive standalone metastore uses protoc-jar-maven-plugin 3.5.1.1 which supports > downloading from maven repo. Artifact download should be supported for ARM > and PPC architecture since some protobuf versions do not exist in ARM/PPC. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Attachment: HIVE-19059.2.patch
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Status: Patch Available (was: Open)
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Status: Open (was: Patch Available)
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Attachment: (was: HIVE-19059.2.patch)
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418267#comment-16418267
]
Vineet Garg commented on HIVE-19059:
[~jcamachorodriguez] Can you take a look? Review is at
https://reviews.apache.org/r/66349/
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19049) Add support for Alter table add columns for Druid
[
https://issues.apache.org/jira/browse/HIVE-19049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418266#comment-16418266
]
Hive QA commented on HIVE-19049:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
51s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 167 unchanged - 1
fixed = 168 total (was 168) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore: The patch generated 4 new + 207
unchanged - 0 fixed = 211 total (was 207) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9893/dev-support/hive-personality.sh
|
| git revision | master / cf6b63e |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/patch-mvninstall-druid-handler.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/patch-mvninstall-ql.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/patch-compile-druid-handler.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/patch-compile-druid-handler.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/diff-checkstyle-standalone-metastore.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus/patch-asflicense-problems.txt
|
| modules | C: druid-handler ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9893/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add support for Alter table add columns for Druid
> -
>
> Key: HIVE-19049
> URL: https://issues.apache.org/jira/browse/HIVE-19049
> Project: Hive
> Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
>
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Status: Open (was: Patch Available)
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Status: Patch Available (was: Open)
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19059) Support DEFAULT keyword with INSERT and UPDATE
[
https://issues.apache.org/jira/browse/HIVE-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19059:
---
Attachment: HIVE-19059.2.patch
> Support DEFAULT keyword with INSERT and UPDATE
> --
>
> Key: HIVE-19059
> URL: https://issues.apache.org/jira/browse/HIVE-19059
> Project: Hive
> Issue Type: New Feature
> Components: SQL
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19059.1.patch, HIVE-19059.2.patch
>
>
> Support DEFAULT keyword in INSERT e.g.
> {code:sql}
> INSERT INTO TABLE t values (DEFAULT, DEFAULT)
> {code}
> or with UPDATE
> {code:sql}
> UPDATE TABLE t SET col1=DEFAULT WHERE col2 > 4
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19032) Vectorization: Disable GROUP BY aggregations with DISTINCT
[ https://issues.apache.org/jira/browse/HIVE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418263#comment-16418263 ] Gopal V commented on HIVE-19032: Change looks +1 The hive count distinct rewrite in CBO should be removing most of these cases, so it shouldn't be a common scenario. However the q.out for vectorized_distinct_gby might need a golden update. > Vectorization: Disable GROUP BY aggregations with DISTINCT > -- > > Key: HIVE-19032 > URL: https://issues.apache.org/jira/browse/HIVE-19032 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19032.01.patch > > > Vectorized GROUP BY does not support DISTINCT aggregation functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19024) Vectorization: Disable complex type constants for VectorUDFAdaptor
[ https://issues.apache.org/jira/browse/HIVE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418259#comment-16418259 ] Gopal V commented on HIVE-19024: Patch LGTM - +1 Can you run vectorized_dynamic_semijoin_reduction locally and confirm if it was just test flakiness? > Vectorization: Disable complex type constants for VectorUDFAdaptor > -- > > Key: HIVE-19024 > URL: https://issues.apache.org/jira/browse/HIVE-19024 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19024.01.patch > > > Currently, complex type constants are not detected and cause execution > failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19012) Support builds for ARM and PPC arch
[
https://issues.apache.org/jira/browse/HIVE-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418245#comment-16418245
]
Hive QA commented on HIVE-19012:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916424/HIVE-19012.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 13047 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-18115) Fix schema version info for Hive-2.3.2
[
https://issues.apache.org/jira/browse/HIVE-18115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18115:
--
Fix Version/s: (was: 2.3.3)
2.4.0
Thanks [~vihangk1], move it to 2.4.0 for now.
> Fix schema version info for Hive-2.3.2
> --
>
> Key: HIVE-18115
> URL: https://issues.apache.org/jira/browse/HIVE-18115
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.1, 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Minor
> Fix For: 2.4.0
>
> Attachments: HIVE-18115.02-branch-2.patch, HIVE-18115.1.patch
>
>
> Error while starting HiveMeta
> {code}
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema
> version 2.3.2 does not match metastore's schema version 2.3.0 Metastore is
> not upgraded or corrupt
> at
> org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7600)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7563)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at com.sun.proxy.$Proxy23.verifySchema(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:591)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:584)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:651)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:427)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79)
> ~[hive-exec-2.3.2.jar:2.3.2]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18702) INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
[
https://issues.apache.org/jira/browse/HIVE-18702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18702:
--
Fix Version/s: (was: 2.3.3)
2.4.0
Move it to 2.4.0
> INSERT OVERWRITE TABLE doesn't clean the table directory before overwriting
> ---
>
> Key: HIVE-18702
> URL: https://issues.apache.org/jira/browse/HIVE-18702
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: HIVE-18702.1.patch, HIVE-18702.2.patch
>
>
> Enable Hive on TEZ. (MR works fine).
> *STEP 1. Create test data*
> {code}
> nano /home/test/users.txt
> {code}
> Add to file:
> {code}
> Peter,34
> John,25
> Mary,28
> {code}
> {code}
> hadoop fs -mkdir /bug
> hadoop fs -copyFromLocal /home/test/users.txt /bug
> hadoop fs -ls /bug
> {code}
> *EXPECTED RESULT:*
> {code}
> Found 2 items
>
> -rwxr-xr-x 3 root root 25 2015-10-15 16:11 /bug/users.txt
> {code}
> *STEP 2. Upload data to hive*
> {code}
> create external table bug(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug';
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Peter 34
> John25
> Mary28
> {code}
> {code}
> create external table bug1(name string, age int) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/bug1';
> insert overwrite table bug select * from bug1;
> select * from bug;
> {code}
> *EXPECTED RESULT:*
> {code}
> OK
> Time taken: 0.097 seconds
> {code}
> *ACTUAL RESULT:*
> {code}
> hive> select * from bug;
> OK
> Peter 34
> John 25
> Mary 28
> Time taken: 0.198 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17924) Restore SerDe by reverting HIVE-15167 to unbreak API compatibility
[ https://issues.apache.org/jira/browse/HIVE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-17924: -- Target Version/s: 2.4.0 (was: 2.3.3) Move it to 2.4.0. Let me know if any thoughts. > Restore SerDe by reverting HIVE-15167 to unbreak API compatibility > -- > > Key: HIVE-17924 > URL: https://issues.apache.org/jira/browse/HIVE-17924 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > > HIVE-15167 broke compatibility badly for very little gain and caused a lot of > pain for our users. We should revert it and restore the SerDe interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418214#comment-16418214 ] Thejas M Nair commented on HIVE-19033: -- Security aspect looks good to me. Ranger implements the HiveAuthorizer class and it would need additional change to handle the two new operation types. > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19012) Support builds for ARM and PPC arch
[
https://issues.apache.org/jira/browse/HIVE-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418206#comment-16418206
]
Hive QA commented on HIVE-19012:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9891/dev-support/hive-personality.sh
|
| git revision | master / cf6b63e |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9891/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9891/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support builds for ARM and PPC arch
> ---
>
> Key: HIVE-19012
> URL: https://issues.apache.org/jira/browse/HIVE-19012
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Vi On
>Assignee: Vi On
>Priority: Major
> Attachments: HIVE-19012.patch, HIVE-19012.patch
>
>
> Hive standalone metastore uses protoc-jar-maven-plugin 3.5.1.1 which supports
> downloading from maven repo. Artifact download should be supported for ARM
> and PPC architecture since some protobuf versions do not exist in ARM/PPC.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18952) Tez session disconnect and reconnect on HS2 HA failover
[
https://issues.apache.org/jira/browse/HIVE-18952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418202#comment-16418202
]
Eric Wohlstadter commented on HIVE-18952:
-
minor:
{code:java}
stop.(isStop)
{code}
seems a bit awkward.
lgtm
> Tez session disconnect and reconnect on HS2 HA failover
> ---
>
> Key: HIVE-18952
> URL: https://issues.apache.org/jira/browse/HIVE-18952
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18952.01.patch, HIVE-18952.02.patch,
> HIVE-18952.patch
>
>
> Now that TEZ-3892 is committed, HIVE-18281 can make use of tez session
> disconnect and reconnect on HA failover.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18115) Fix schema version info for Hive-2.3.2
[
https://issues.apache.org/jira/browse/HIVE-18115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418199#comment-16418199
]
Vihang Karajgaonkar commented on HIVE-18115:
I took a relook at this and I don't think this is a problem. Like I explained
in my comment above {{MetaStoreSchemaInfo.getHiveSchemaVersion()}} will return
2.3.0 and the database VERSION table will store 2.3.0 assuming the schema is
initialized/upgraded using the right scripts. I tried to reproduce this issue
on the current branch-2.3 (pointing to 2.3.3) and also the released tag 2.3.2
(git checkout tags/rel/release-2.3.2). I set
{{hive.metastore.schema.verification}} to true and in both the cases I didn't
hit the issue. The line number in the ObjectStore.java in the stack above
{{ObjectStore.checkSchema(ObjectStore.java:7600)}} also doesn't match with
2.3.2 with this exception so I think the reporter is not using release build of
Hive 2.3.2
I think it is okay to ignore this issue for 2.3.3 and we should close this JIRA
as not reproducible unless someone can advise what I am doing wrong above.
> Fix schema version info for Hive-2.3.2
> --
>
> Key: HIVE-18115
> URL: https://issues.apache.org/jira/browse/HIVE-18115
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.1, 2.3.2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Minor
> Fix For: 2.3.3
>
> Attachments: HIVE-18115.02-branch-2.patch, HIVE-18115.1.patch
>
>
> Error while starting HiveMeta
> {code}
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema
> version 2.3.2 does not match metastore's schema version 2.3.0 Metastore is
> not upgraded or corrupt
> at
> org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7600)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7563)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at com.sun.proxy.$Proxy23.verifySchema(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:591)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:584)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:651)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:427)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_141]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_141]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79)
> ~[hive-exec-2.3.2.jar:2.3.2]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19029) Load Data should prevent loading acid files
[
https://issues.apache.org/jira/browse/HIVE-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418192#comment-16418192
]
Hive QA commented on HIVE-19029:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916416/HIVE-19029.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 80 failed/errored test(s), 13445 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418189#comment-16418189 ] Sergey Shelukhin commented on HIVE-19033: - +1 for existing patch. I think we can handle Ranger in a follow-up. I am not sure if this actually works for Ranger... I didn't think ranger goes thru authorizer, it goes thru some separate plugin. Also IIRC for kill query we had to make changes to ranger to add operation type. [~thejas] can you comment? > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418183#comment-16418183 ] Prasanth Jayachandran commented on HIVE-19033: -- [~sershe] can you please take another look? > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18971) add HS2 WM metrics for use in Grafana and such
[ https://issues.apache.org/jira/browse/HIVE-18971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18971: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Looks like HiveQA is effectively stuck... Based on the previous test pass and some more local runs, committed to master. > add HS2 WM metrics for use in Grafana and such > -- > > Key: HIVE-18971 > URL: https://issues.apache.org/jira/browse/HIVE-18971 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18971.01.patch, HIVE-18971.02.patch, > HIVE-18971.03.patch, HIVE-18971.patch > > > HS2 should have metrics added per pool, tagged accordingly. Not clear if HS2 > even sets up metrics right now... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418164#comment-16418164 ] Prasanth Jayachandran commented on HIVE-19071: -- i see.. makes sense now.. +1 on the change. > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19065) Metastore client compatibility check should include syncMetaStoreClient
[ https://issues.apache.org/jira/browse/HIVE-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418165#comment-16418165 ] Thejas M Nair commented on HIVE-19065: -- +1 > Metastore client compatibility check should include syncMetaStoreClient > --- > > Key: HIVE-19065 > URL: https://issues.apache.org/jira/browse/HIVE-19065 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19065.1.patch > > > I saw a case Hive.get(HiveConf c) reuse syncMetaStoreClient with different > config (in my case, hive.metastore.uris is different), which makes > syncMetaStoreClient connect to wrong metastore server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418155#comment-16418155 ] Sergey Shelukhin edited comment on HIVE-19071 at 3/28/18 9:35 PM: -- [~prasanth_j] _ is a valid hive identifier character while - isn't. The issue is that if you try to do stuff with the old plan with a dash it doesn't work unless quoted idenitifiers are enabled (actually with current .g file syntax it might not work even then, we'd need to change to account for quoted identifiers) was (Author: sershe): [~prasanth_j] _ is a valid hive identifier character while - isn't > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418155#comment-16418155 ] Sergey Shelukhin commented on HIVE-19071: - [~prasanth_j] _ is a valid hive identifier character while - isn't > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418147#comment-16418147 ] Prasanth Jayachandran commented on HIVE-19071: -- what is the primary issue? I don't understand how replacing "-" with "_" solves the issue? > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19071: Status: Patch Available (was: Open) > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418139#comment-16418139 ] Sergey Shelukhin commented on HIVE-19071: - [~prasanth_j] another trivial patch :) > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19071: Attachment: HIVE-19071.patch > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19071.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18905) HS2: SASL auth loads HiveConf for every JDBC call
[
https://issues.apache.org/jira/browse/HIVE-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18905:
-
Attachment: HIVE-18905.01.patch
> HS2: SASL auth loads HiveConf for every JDBC call
> -
>
> Key: HIVE-18905
> URL: https://issues.apache.org/jira/browse/HIVE-18905
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Igor Kryvenko
>Priority: Minor
> Attachments: HIVE-18905.01.patch, HIVE-18905.patch
>
>
> SASL authentication filter does a new HiveConf() for no good reason.
> {code}
> public static PasswdAuthenticationProvider
> getAuthenticationProvider(AuthMethods authMethod)
> throws AuthenticationException {
> return getAuthenticationProvider(authMethod, new HiveConf());
> }
> {code}
> The session HiveConf is not needed to do this operation & it can't be changed
> after the HS2 starts up (today).
> {code}
> org.apache.hadoop.hive.conf.HiveConf.() HiveConf.java:4404
> org.apache.hive.service.auth.AuthenticationProviderFactory.getAuthenticationProvider(AuthenticationProviderFactory$AuthMethods)
> AuthenticationProviderFactory.java:61
> org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(Callback[])
> PlainSaslHelper.java:106
> org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(byte[])
> PlainSaslServer.java:103
> org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(byte[])
> TSaslTransport.java:539
> org.apache.thrift.transport.TSaslTransport.open() TSaslTransport.java:283
> org.apache.thrift.transport.TSaslServerTransport.open()
> TSaslServerTransport.java:41
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TTransport)
> TSaslServerTransport.java:216
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run()
> TThreadPoolServer.java:269
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
> ThreadPoolExecutor.java:1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run()
> ThreadPoolExecutor.java:617
> java.lang.Thread.run() Thread.java:745
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18983) Add support for table properties inheritance in Create table like
[
https://issues.apache.org/jira/browse/HIVE-18983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18983:
-
Attachment: HIVE-18983.07.patch
> Add support for table properties inheritance in Create table like
> -
>
> Key: HIVE-18983
> URL: https://issues.apache.org/jira/browse/HIVE-18983
> Project: Hive
> Issue Type: Improvement
>Reporter: Igor Kryvenko
>Assignee: Igor Kryvenko
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-18983.01.patch, HIVE-18983.02.patch,
> HIVE-18983.03.patch, HIVE-18983.04.patch, HIVE-18983.05.patch,
> HIVE-18983.06.patch, HIVE-18983.07.patch
>
>
> Currently, Create table like support table properties.
> But it doesn't inherit table properties from original table.
> {code}
> create table T1(a int, b int) clustered by (a) into 2 buckets stored as orc
> TBLPROPERTIES ('comment'='comm');
> create table T like T1;
> show create table T;
> {code}
> *Output:*
> {code}
> CREATE TABLE `T`(
> `a` int,
> `b` int)
> CLUSTERED BY (
> a)
> INTO 2 BUCKETS
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
> 'maprfs:/user/hive/warehouse/t'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\"}',
> 'numFiles'='0',
> 'numRows'='0',
> 'rawDataSize'='0',
> 'totalSize'='0',
> 'transient_lastDdlTime'='1521230300')
> {code}
> It uses just default table properties and doesn't inherit properties from
> original table.
> It would be great if create table like will inherit origin table properties
> and will override them if they are specified in query .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-18990: - Attachment: HIVE-18990.02.patch > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19071) WM: backup resource plans cannot be used without quoted idenitifiers
[ https://issues.apache.org/jira/browse/HIVE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19071: --- > WM: backup resource plans cannot be used without quoted idenitifiers > > > Key: HIVE-19071 > URL: https://issues.apache.org/jira/browse/HIVE-19071 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19029) Load Data should prevent loading acid files
[
https://issues.apache.org/jira/browse/HIVE-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418100#comment-16418100
]
Hive QA commented on HIVE-19029:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 3 new + 140 unchanged - 1
fixed = 143 total (was 141) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9890/dev-support/hive-personality.sh
|
| git revision | master / ad7652e |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9890/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9890/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9890/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Load Data should prevent loading acid files
> ---
>
> Key: HIVE-19029
> URL: https://issues.apache.org/jira/browse/HIVE-19029
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19029.01.patch, HIVE-19029.02.patch,
> HIVE-19029.04.patch
>
>
> {{Load Data into T}} where T is acid should check to make sure input files
> were not copied from another Acid table, i.e. make sure the files don't have
> Acid metadata columns.
> AcidUtils.MetaData.isRawFormat()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18952) Tez session disconnect and reconnect on HS2 HA failover
[ https://issues.apache.org/jira/browse/HIVE-18952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18952: Attachment: HIVE-18952.02.patch > Tez session disconnect and reconnect on HS2 HA failover > --- > > Key: HIVE-18952 > URL: https://issues.apache.org/jira/browse/HIVE-18952 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18952.01.patch, HIVE-18952.02.patch, > HIVE-18952.patch > > > Now that TEZ-3892 is committed, HIVE-18281 can make use of tez session > disconnect and reconnect on HA failover. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418082#comment-16418082 ] Prasanth Jayachandran commented on HIVE-19033: -- Added service name priv changes for Ranger. Will need admin privilege for llap cache command object and service name. > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19033: - Attachment: HIVE-19033.4.patch > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19033) Provide an option to purge LLAP IO cache
[
https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416312#comment-16416312
]
Prasanth Jayachandran edited comment on HIVE-19033 at 3/28/18 8:42 PM:
---
Based on offline feedback from [~gopalv] added llap specific commands under
HiveCommandOperation to get Hive authorization. Also updated the patch to get
use LLAP management API for security.
2 commands are add
1) LLAP cluster commands (only info is implemented in this patch)
2) LLAP cache commands (only purge is implemented in this patch)
"llap cluster -info;" has no authorization, any user can read this information.
"llap cache -purge;" requires users to have admin role.
Some examples:
{code:title=user in non-admin role trying to purge the cache}
0: jdbc:hive2://localhost:1> set hive.security.authorization.enabled;
+---+
|set|
+---+
| hive.security.authorization.enabled=true |
+---+
1 row selected (0.165 seconds)
0: jdbc:hive2://localhost:1> llap cache -purge;
Error while processing statement: Permission denied: Principal [name=user1,
type=USER] does not have following privileges for operation LLAP_CACHE [[ADMIN
PRIVILEGE] on Object [type=COMMAND_PARAMS, name=[-purge]], [ADMIN PRIVILEGE] on
Object [type=SERVICE_NAME, name=localhost]]
{code}
{code:title=user in admin role trying to purge the cache}
0: jdbc:hive2://localhost:1> set role admin;
No rows affected (1.019 seconds)
0: jdbc:hive2://localhost:1> llap cache -purge;
+++
| hostName | purgedMemoryBytes |
+++
| localhost | 50429952 |
+++
{code}
{code:title=when authZ is disabled, user can read cluster info}
0: jdbc:hive2://localhost:1> set hive.security.authorization.enabled;
++
|set |
++
| hive.security.authorization.enabled=false |
++
1 row selected (0.159 seconds)
0: jdbc:hive2://localhost:1> llap cluster -info;
++---++--+-+-+
| applicationId |workerIdentity | hostname | rpcPort
| memory| vcores |
++---++--+-+-+
| null | 873b7438-01b1-4974-90e2-1c5631602db9 | localhost | 15001
| 3145728000 | 3 |
++---++--+-+-+
{code}
ignore "null" applicationId as this is my local setup which is not deployed via
slider or yarn services.
was (Author: prasanth_j):
Based on offline feedback from [~gopalv] added llap specific commands under
HiveCommandOperation to get Hive authorization. Also updated the patch to get
use LLAP management API for security.
2 commands are add
1) LLAP cluster commands (only info is implemented in this patch)
2) LLAP cache commands (only purge is implemented in this patch)
"llap cluster -info;" has no authorization, any user can read this information.
"llap cache -purge;" requires users to have admin role.
Some examples:
{code:title=user in non-admin role trying to purge the cache}
0: jdbc:hive2://localhost:1> set hive.security.authorization.enabled;
+---+
|set|
+---+
| hive.security.authorization.enabled=true |
+---+
1 row selected (0.165 seconds)
0: jdbc:hive2://localhost:1> llap cache -purge;
Error: Error while processing statement: Permission denied: Principal
[name=pjayachandran, type=USER] does not have following privileges for
operation LLAP_CACHE [[ADMIN PRIVILEGE] on Object [type=COMMAND_PARAMS,
name=[-purge]]] (state=,code=1)
{code}
{code:title=user in admin role trying to purge the cache}
0: jdbc:hive2://localhost:1> set role admin;
No rows affected (1.019 seconds)
0: jdbc:hive2://localhost:1> llap cache -purge;
+++
| hostName | purgedMemoryBytes |
+++
| localhost | 50429952 |
+++
{code}
{code:title=when authZ is disabled, user can read cluster info}
0: jdbc:hive2://localhost:1> set hive.security.authorization.enabled;
++
|set |
++
| hive.security.authorization.enabled=false |
[jira] [Assigned] (HIVE-18609) Results cache invalidation based on table updates
[ https://issues.apache.org/jira/browse/HIVE-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-18609: - Assignee: Jason Dere > Results cache invalidation based on table updates > - > > Key: HIVE-18609 > URL: https://issues.apache.org/jira/browse/HIVE-18609 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Look into using the materialized view invalidation mechanisms to > automatically invalidate queries in the results cache if the underlying > tables used in the cached queries have been modified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18609) Results cache invalidation based on table updates
[ https://issues.apache.org/jira/browse/HIVE-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418075#comment-16418075 ] Jason Dere commented on HIVE-18609: --- Proposed changes: - At query time, the validity of the cached result will be checked by comparing the transactional WriteIDs of the tables used in the cached query to the WriteIDs of the current query, similar to HIVE-19027. This approach only guarantees freshness if all of the tables used in the query are transactional tables. As a result, the results cache will be disabled by default for queries involving non-transactional tables. - The above approach will only invalidate entries in the cache if that query looked up again, which could potentially leave a lot of undetected invalid entries in the cache taking up space which could cause other entries to be evicted. To remove these entries in a more timely fashion, have a background thread to periodically check the notification events for updates to the tables used in the results cache. > Results cache invalidation based on table updates > - > > Key: HIVE-18609 > URL: https://issues.apache.org/jira/browse/HIVE-18609 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Priority: Major > > Look into using the materialized view invalidation mechanisms to > automatically invalidate queries in the results cache if the underlying > tables used in the cached queries have been modified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19060) Fix the TestAppendPartitions.testAppendPartitionNullPartValues
[
https://issues.apache.org/jira/browse/HIVE-19060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418071#comment-16418071
]
Hive QA commented on HIVE-19060:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916411/HIVE-19060.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 13047 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-18963) JDBC: Provide an option to simplify beeline usage by supporting default and named URL for beeline
[
https://issues.apache.org/jira/browse/HIVE-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-18963:
Attachment: HIVE-18963.2.patch
> JDBC: Provide an option to simplify beeline usage by supporting default and
> named URL for beeline
> -
>
> Key: HIVE-18963
> URL: https://issues.apache.org/jira/browse/HIVE-18963
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-18963.1.patch, HIVE-18963.2.patch
>
>
> Currently, after opening Beeline CLI, the user needs to supply a connection
> string to use the HS2 instance and set up the jdbc driver. Since we plan to
> replace Hive CLI with Beeline in future (HIVE-10511), it will help the
> usability if the user can simply type {{beeline}} and get start the hive
> session. The jdbc url can be specified in a beeline-site.xml (which can
> contain other named jdbc urls as well, and they can be accessed by something
> like: {{beeline -c namedUrl}}. The use of beeline-site.xml can also be
> potentially expanded later if needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19040) get_partitions_by_expr() implementation in HiveMetaStore causes backward incompatibility easily
[
https://issues.apache.org/jira/browse/HIVE-19040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417987#comment-16417987
]
Sergey Shelukhin edited comment on HIVE-19040 at 3/28/18 7:30 PM:
--
[~vihangk1] as per above... HMS version doesn't matter, only Hive jars and HS2.
The reason this API exists is actually also described above although it's not
very clear I guess.
What happens is we send actual Hive expression to MS.
On the main path, where it can be pushed down into SQL store, it gets
deserialized, converted to string and pushed, so that part is indeed redundant
and can be replaced.
However the reason we send it as bytes (and the reason the whole API was added
on top of the old string filter API), is to allow HMS to evaluate it when
pushdown is not available (which is actually in most cases - Filter.g and SQL
pushdown supports only basic stuff). This way MS can get (or potentially
stream) all the partitions and actually evaluate a full Hive expression on them
with any standard UDFs.
The alternative without this API is to send all partitions back to client
(which unlike the SQL db can be far away) and evaluate Hive expression on
client, which can be very expensive if there are many.
That's why bytes and proxy class into ql jar is used (if long ago we had
metastore client and metastore server module separation, it would have used QL
classes directly). Also Hive stuff is already serializable into bytes (due to
needing to serialize the plan).
So with just filter.g this functionality can be lost.
If we want to keep the ability of MS to evaluate expressions locally, it's
possible to beef up this API (and the proxy class config). They can refer to a
named "expression evaluator" that will be configured to refer to a particular
class, and also supplied in the request. Version can also be included to handle
compat...
It's also possible to create support for native expressions in metastore that
would handle most Hive cases, i.e. basically replace Filter.g with Hive
expression parsing and include common UDFs like IN, etc..
was (Author: sershe):
[~vihangk1] as per above... HMS version doesn't matter, only Hive jars and HS2.
The reason this API exists is actually also described above although it's not
very clear I guess.
See the impl.
What happens is we send actual Hive expression to MS.
On the main path, where it can be pushed down, it gets deserialized, converted
to string and pushed, so that part is indeed redundant and can be replaced.
However the reason we send it as bytes (and the reason the whole API was added
on top of the old string filter API), is to allow HMS to evaluate it when
pushdown is not available (which is actually in most cases - Filter.g and SQL
pushdown supports only basic stuff). This way MS can get (or potentially
stream) all the partitions and actually evaluate a full Hive expression on them
with any standard UDFs.
The alternative without this API is to send all partitions back to client
(which unlike the SQL db can be far away) and evaluate Hive expression on
client, which can be very expensive if there are many.
That's why bytes and proxy class into ql jar is used (if long ago we had
metastore client and metastore server module separation, it would have used QL
classes directly). Also Hive stuff is already serializable into bytes (due to
needing to serialize the plan).
So with just filter.g this functionality can be lost.
If we want to keep the ability of MS to evaluate expressions locally, it's
possible to beef up this API (and the proxy class config). They can refer to a
named "expression evaluator" that will be configured to refer to a particular
class, and also supplied in the request. Version can also be included to handle
compat...
It's also possible to create support for native expressions in metastore that
would handle most Hive cases, i.e. basically replace Filter.g with Hive
expression parsing and include common UDFs like IN, etc..
> get_partitions_by_expr() implementation in HiveMetaStore causes backward
> incompatibility easily
>
>
> Key: HIVE-19040
> URL: https://issues.apache.org/jira/browse/HIVE-19040
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Priority: Major
>
> In the HiveMetaStore implementation of {{public PartitionsByExprResult
> get_partitions_by_expr(PartitionsByExprRequest req) throws TException}} , an
> expression is serialized into byte array from the client side and passed
> through PartitionsByExprRequest. Then HMS will deserialize back into the
> expression and filter the partitions by it.
> Such partition filtering expression can contain
[jira] [Commented] (HIVE-19040) get_partitions_by_expr() implementation in HiveMetaStore causes backward incompatibility easily
[
https://issues.apache.org/jira/browse/HIVE-19040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417987#comment-16417987
]
Sergey Shelukhin commented on HIVE-19040:
-
[~vihangk1] as per above... HMS version doesn't matter, only Hive jars and HS2.
The reason this API exists is actually also described above although it's not
very clear I guess.
See the impl.
What happens is we send actual Hive expression to MS.
On the main path, where it can be pushed down, it gets deserialized, converted
to string and pushed, so that part is indeed redundant and can be replaced.
However the reason we send it as bytes (and the reason the whole API was added
on top of the old string filter API), is to allow HMS to evaluate it when
pushdown is not available (which is actually in most cases - Filter.g and SQL
pushdown supports only basic stuff). This way MS can get (or potentially
stream) all the partitions and actually evaluate a full Hive expression on them
with any standard UDFs.
The alternative without this API is to send all partitions back to client
(which unlike the SQL db can be far away) and evaluate Hive expression on
client, which can be very expensive if there are many.
That's why bytes and proxy class into ql jar is used (if long ago we had
metastore client and metastore server module separation, it would have used QL
classes directly). Also Hive stuff is already serializable into bytes (due to
needing to serialize the plan).
So with just filter.g this functionality can be lost.
If we want to keep the ability of MS to evaluate expressions locally, it's
possible to beef up this API (and the proxy class config). They can refer to a
named "expression evaluator" that will be configured to refer to a particular
class, and also supplied in the request. Version can also be included to handle
compat...
It's also possible to create support for native expressions in metastore that
would handle most Hive cases, i.e. basically replace Filter.g with Hive
expression parsing and include common UDFs like IN, etc..
> get_partitions_by_expr() implementation in HiveMetaStore causes backward
> incompatibility easily
>
>
> Key: HIVE-19040
> URL: https://issues.apache.org/jira/browse/HIVE-19040
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Priority: Major
>
> In the HiveMetaStore implementation of {{public PartitionsByExprResult
> get_partitions_by_expr(PartitionsByExprRequest req) throws TException}} , an
> expression is serialized into byte array from the client side and passed
> through PartitionsByExprRequest. Then HMS will deserialize back into the
> expression and filter the partitions by it.
> Such partition filtering expression can contain various UDFs. If there are
> some changes to one of the UDFs between different Hive versions, HS2 on the
> older version will serialize the expression in old format which won't be able
> to be deserialized by HMS on the newer version. One example of that is,
> GenericUDFIn class adds {{transient}} to the field constantInSet which will
> cause such incompatibility.
> One approach I'm thinking of is, instead of converting the expression object
> to byte array, we can pass the expression string directly.
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19060) Fix the TestAppendPartitions.testAppendPartitionNullPartValues
[
https://issues.apache.org/jira/browse/HIVE-19060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417986#comment-16417986
]
Hive QA commented on HIVE-19060:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} standalone-metastore: The patch generated 0 new +
372 unchanged - 1 fixed = 372 total (was 373) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9889/dev-support/hive-personality.sh
|
| git revision | master / ad7652e |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9889/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9889/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix the TestAppendPartitions.testAppendPartitionNullPartValues
> --
>
> Key: HIVE-19060
> URL: https://issues.apache.org/jira/browse/HIVE-19060
> Project: Hive
> Issue Type: Test
> Components: Test
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Minor
> Attachments: HIVE-19060.1.patch
>
>
> I saw in the pre-commit jobs that the
> TestAppendPartitions.testAppendPartitionNullPartValues test is failing at
> least since 17th March.
> It can be reproduced locally and the test is failing because it expected a
> NPE to be thrown, but actually a NPE wrapped to a MetaException is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18696) The partition folders might not get cleaned up properly in the HiveMetaStore.add_partitions_core method if an exception occurs
[
https://issues.apache.org/jira/browse/HIVE-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417931#comment-16417931
]
Hive QA commented on HIVE-18696:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916399/HIVE-18696.5.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 13063 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-19070) Add More Test To Druid Mini Cluster queries.
[ https://issues.apache.org/jira/browse/HIVE-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19070: Resolution: Invalid Status: Resolved (was: Patch Available) > Add More Test To Druid Mini Cluster queries. > - > > Key: HIVE-19070 > URL: https://issues.apache.org/jira/browse/HIVE-19070 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.0.0 > > > In This patch am adding 200 new tableau query that runs over a new Data set > called calcs. > The data set is very small. > I also have consolidated 3 different tests to run as one test this will help > with keeping execution time low. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19070) Add More Test To Druid Mini Cluster queries.
[ https://issues.apache.org/jira/browse/HIVE-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19070: Attachment: (was: HIVE-19070.patch) > Add More Test To Druid Mini Cluster queries. > - > > Key: HIVE-19070 > URL: https://issues.apache.org/jira/browse/HIVE-19070 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.0.0 > > > In This patch am adding 200 new tableau query that runs over a new Data set > called calcs. > The data set is very small. > I also have consolidated 3 different tests to run as one test this will help > with keeping execution time low. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18781) Create/Replicate Open, Commit (without writes) and Abort Txn events
[ https://issues.apache.org/jira/browse/HIVE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417912#comment-16417912 ] Sankar Hariappan commented on HIVE-18781: - +1 Pending the ptest results on 11.patch. > Create/Replicate Open, Commit (without writes) and Abort Txn events > --- > > Key: HIVE-18781 > URL: https://issues.apache.org/jira/browse/HIVE-18781 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18781.01.patch, HIVE-18781.02.patch, > HIVE-18781.03.patch, HIVE-18781.04.patch, HIVE-18781.05.patch, > HIVE-18781.06.patch, HIVE-18781.07.patch, HIVE-18781.08.patch, > HIVE-18781.09.patch, HIVE-18781.10.patch, HIVE-18781.11.patch > > > *EVENT_OPEN_TXN:* > *Source Warehouse:* > - Create new event type EVENT_OPEN_TXN with related message format etc. > - When any transaction is opened either by auto-commit mode or > multi-statement mode, need to capture this event. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Open a txn in target warehouse. > - Create a map of source txn ID against target txn ID and persist the same > in metastore. There should be one map per replication policy (DBName.* incase > of DB level replication, DBName.TableName incase of table level replication) > > *EVENT_COMMIT_TXN (Without writes)* > Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions > modified within the txn. > *Source warehouse:* > - Create EVENT_COMMIT_TXN event type with corresponding message format etc. > *Target warehouse:* > - Repl load should read this event from the dump. > - Validate the source txn ID from the event using the Source-Target Txn ID > map maintained in target metastore. Also, need to check if corresponding > target txn ID is valid. > - If valid, then apply the event and commit the corresponding target > transaction. > - This new event should be idempotent such that if it is applied twice, then > second time it should be loop. > > *EVENT_ABORT_TXN* > Source Warehouse: > - Create new event type EVENT_ABORT_TXN with related message format etc. > - Capture this event when abort the txn. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Validate if source txn ID from the event is there in the source-target txn > ID map. If not there, just noop the event. > - If valid, then Abort the corresponding target txn and remove the entry > from source-target txn ID map. > All these new events should be idempotent such that if it is applied twice, > then second time it should be noop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19019) Vectorization: When vectorized, orc_merge_incompat_schema.q throws HiveException "Not implemented yet" from VectorExpressionWriterMap
[
https://issues.apache.org/jira/browse/HIVE-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19019:
Status: Patch Available (was: In Progress)
> Vectorization: When vectorized, orc_merge_incompat_schema.q throws
> HiveException "Not implemented yet" from VectorExpressionWriterMap
> -
>
> Key: HIVE-19019
> URL: https://issues.apache.org/jira/browse/HIVE-19019
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19019.01.patch, HIVE-19019.02.patch,
> HIVE-19019.03.patch, HIVE-19019.04.patch
>
>
> Adding "SET hive.vectorized.execution.enabled=true;" to
> orc_merge_incompat_schema.q triggers this call stack:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Not implemented
> yet
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$19.writeValue(VectorExpressionWriterFactory.java:1496)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:199)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:151)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.flushDeserializerBatch(VectorMapOperator.java:630)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setupPartitionContextVars(VectorMapOperator.java:698)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.cleanUpInputFileChangedOp(VectorMapOperator.java:607)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1210)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:829)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:154)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> ~[hadoop-mapreduce-client-core-3.0.0-beta1.jar:?]
> {noformat}
> The complex types in VectorExpressionWriterFactory are not fully implemented.
> Also, null_cast.q, nullMap.q, and nested_column_pruning.q
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19019) Vectorization: When vectorized, orc_merge_incompat_schema.q throws HiveException "Not implemented yet" from VectorExpressionWriterMap
[
https://issues.apache.org/jira/browse/HIVE-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19019:
Status: In Progress (was: Patch Available)
> Vectorization: When vectorized, orc_merge_incompat_schema.q throws
> HiveException "Not implemented yet" from VectorExpressionWriterMap
> -
>
> Key: HIVE-19019
> URL: https://issues.apache.org/jira/browse/HIVE-19019
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19019.01.patch, HIVE-19019.02.patch,
> HIVE-19019.03.patch, HIVE-19019.04.patch
>
>
> Adding "SET hive.vectorized.execution.enabled=true;" to
> orc_merge_incompat_schema.q triggers this call stack:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Not implemented
> yet
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$19.writeValue(VectorExpressionWriterFactory.java:1496)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:199)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:151)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.flushDeserializerBatch(VectorMapOperator.java:630)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setupPartitionContextVars(VectorMapOperator.java:698)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.cleanUpInputFileChangedOp(VectorMapOperator.java:607)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1210)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:829)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:154)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> ~[hadoop-mapreduce-client-core-3.0.0-beta1.jar:?]
> {noformat}
> The complex types in VectorExpressionWriterFactory are not fully implemented.
> Also, null_cast.q, nullMap.q, and nested_column_pruning.q
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19019) Vectorization: When vectorized, orc_merge_incompat_schema.q throws HiveException "Not implemented yet" from VectorExpressionWriterMap
[
https://issues.apache.org/jira/browse/HIVE-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19019:
Attachment: HIVE-19019.04.patch
> Vectorization: When vectorized, orc_merge_incompat_schema.q throws
> HiveException "Not implemented yet" from VectorExpressionWriterMap
> -
>
> Key: HIVE-19019
> URL: https://issues.apache.org/jira/browse/HIVE-19019
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19019.01.patch, HIVE-19019.02.patch,
> HIVE-19019.03.patch, HIVE-19019.04.patch
>
>
> Adding "SET hive.vectorized.execution.enabled=true;" to
> orc_merge_incompat_schema.q triggers this call stack:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Not implemented
> yet
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$19.writeValue(VectorExpressionWriterFactory.java:1496)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:199)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:151)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.flushDeserializerBatch(VectorMapOperator.java:630)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setupPartitionContextVars(VectorMapOperator.java:698)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.cleanUpInputFileChangedOp(VectorMapOperator.java:607)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1210)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:829)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:154)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> ~[hadoop-mapreduce-client-core-3.0.0-beta1.jar:?]
> {noformat}
> The complex types in VectorExpressionWriterFactory are not fully implemented.
> Also, null_cast.q, nullMap.q, and nested_column_pruning.q
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19019) Vectorization: When vectorized, orc_merge_incompat_schema.q throws HiveException "Not implemented yet" from VectorExpressionWriterMap
[
https://issues.apache.org/jira/browse/HIVE-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19019:
Attachment: (was: HIVE-19019.04.patch)
> Vectorization: When vectorized, orc_merge_incompat_schema.q throws
> HiveException "Not implemented yet" from VectorExpressionWriterMap
> -
>
> Key: HIVE-19019
> URL: https://issues.apache.org/jira/browse/HIVE-19019
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19019.01.patch, HIVE-19019.02.patch,
> HIVE-19019.03.patch
>
>
> Adding "SET hive.vectorized.execution.enabled=true;" to
> orc_merge_incompat_schema.q triggers this call stack:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Not implemented
> yet
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$19.writeValue(VectorExpressionWriterFactory.java:1496)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:199)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:151)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.flushDeserializerBatch(VectorMapOperator.java:630)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setupPartitionContextVars(VectorMapOperator.java:698)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.cleanUpInputFileChangedOp(VectorMapOperator.java:607)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1210)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:829)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:154)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> ~[hadoop-mapreduce-client-core-3.0.0-beta1.jar:?]
> {noformat}
> The complex types in VectorExpressionWriterFactory are not fully implemented.
> Also, null_cast.q, nullMap.q, and nested_column_pruning.q
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19019) Vectorization: When vectorized, orc_merge_incompat_schema.q throws HiveException "Not implemented yet" from VectorExpressionWriterMap
[
https://issues.apache.org/jira/browse/HIVE-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19019:
Attachment: HIVE-19019.04.patch
> Vectorization: When vectorized, orc_merge_incompat_schema.q throws
> HiveException "Not implemented yet" from VectorExpressionWriterMap
> -
>
> Key: HIVE-19019
> URL: https://issues.apache.org/jira/browse/HIVE-19019
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19019.01.patch, HIVE-19019.02.patch,
> HIVE-19019.03.patch, HIVE-19019.04.patch
>
>
> Adding "SET hive.vectorized.execution.enabled=true;" to
> orc_merge_incompat_schema.q triggers this call stack:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Not implemented
> yet
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$19.writeValue(VectorExpressionWriterFactory.java:1496)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFArgDesc.getDeferredJavaObject(VectorUDFArgDesc.java:123)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setResult(VectorUDFAdaptor.java:199)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.evaluate(VectorUDFAdaptor.java:151)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.flushDeserializerBatch(VectorMapOperator.java:630)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setupPartitionContextVars(VectorMapOperator.java:698)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.cleanUpInputFileChangedOp(VectorMapOperator.java:607)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1210)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:829)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:154)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> ~[hadoop-mapreduce-client-core-3.0.0-beta1.jar:?]
> {noformat}
> The complex types in VectorExpressionWriterFactory are not fully implemented.
> Also, null_cast.q, nullMap.q, and nested_column_pruning.q
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19043) Vectorization: LazySimpleDeserializeRead fewer fields handling is broken for Complex Types
[ https://issues.apache.org/jira/browse/HIVE-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19043: Status: Patch Available (was: Open) > Vectorization: LazySimpleDeserializeRead fewer fields handling is broken for > Complex Types > -- > > Key: HIVE-19043 > URL: https://issues.apache.org/jira/browse/HIVE-19043 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19043.01.patch, HIVE-19043.02.patch > > > Issues were revealed by vectorizing create_struct_table.q -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19037) Vectorization: Miscellaneous cleanup
[
https://issues.apache.org/jira/browse/HIVE-19037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19037:
Status: Patch Available (was: Open)
> Vectorization: Miscellaneous cleanup
>
>
> Key: HIVE-19037
> URL: https://issues.apache.org/jira/browse/HIVE-19037
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19037.01.patch, HIVE-19037.02.patch,
> HIVE-19037.03.patch
>
>
> # Extraneous INFO logging in VectorReduceSinkCommonOperator
> # NPE in EXPLAIN for some SelectColumnIsTrue vector expressions
> # Don't throw instantiation exception when UDF has NULL SCALAR that cannot
> match vectorization expression description. Catch it early and don't try to
> create vectorization expression for UDF. (try VectorUDFAdaptor instead).
> # Support vectorizing SELECT NULL.
> # Support constant casting from \{STRING|CHAR|VARCHAR} to
> \{LONG|DOUBLE|DECIMAL} case.
> # Suppress fatal Vectorizer exceptions. Put single line exception stack
> trace innotVectorizedReason. Add test only variable
> hive.test.vectorizer.suppress.fatal.exceptions to turn that off so Vectorizer
> execution failure can be found more easily.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19037) Vectorization: Miscellaneous cleanup
[
https://issues.apache.org/jira/browse/HIVE-19037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-19037:
Attachment: HIVE-19037.03.patch
> Vectorization: Miscellaneous cleanup
>
>
> Key: HIVE-19037
> URL: https://issues.apache.org/jira/browse/HIVE-19037
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19037.01.patch, HIVE-19037.02.patch,
> HIVE-19037.03.patch
>
>
> # Extraneous INFO logging in VectorReduceSinkCommonOperator
> # NPE in EXPLAIN for some SelectColumnIsTrue vector expressions
> # Don't throw instantiation exception when UDF has NULL SCALAR that cannot
> match vectorization expression description. Catch it early and don't try to
> create vectorization expression for UDF. (try VectorUDFAdaptor instead).
> # Support vectorizing SELECT NULL.
> # Support constant casting from \{STRING|CHAR|VARCHAR} to
> \{LONG|DOUBLE|DECIMAL} case.
> # Suppress fatal Vectorizer exceptions. Put single line exception stack
> trace innotVectorizedReason. Add test only variable
> hive.test.vectorizer.suppress.fatal.exceptions to turn that off so Vectorizer
> execution failure can be found more easily.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18909) Metrics for results cache
[ https://issues.apache.org/jira/browse/HIVE-18909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417863#comment-16417863 ] Jason Dere commented on HIVE-18909: --- Hey [~gopalv], does the updated patch look ok? > Metrics for results cache > - > > Key: HIVE-18909 > URL: https://issues.apache.org/jira/browse/HIVE-18909 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Labels: Metrics > Attachments: HIVE-18909.1.patch, HIVE-18909.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19057) Query result caching cannot be disabled by client
[
https://issues.apache.org/jira/browse/HIVE-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-19057:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Committed to master, thanks for the patch.
> Query result caching cannot be disabled by client
> -
>
> Key: HIVE-19057
> URL: https://issues.apache.org/jira/browse/HIVE-19057
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19057.1.patch
>
>
> HIVE-18513 introduced query results caching along with some toggles to
> control enabling/disabling it. We should whiltelist the following configs so
> that the end user can dynamically control it in their session.
> {noformat}
> hive.query.results.cache.enabled
> hive.query.results.cache.wait.for.pending.results
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18696) The partition folders might not get cleaned up properly in the HiveMetaStore.add_partitions_core method if an exception occurs
[
https://issues.apache.org/jira/browse/HIVE-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417838#comment-16417838
]
Hive QA commented on HIVE-18696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore: The patch generated 2 new + 394
unchanged - 3 fixed = 396 total (was 397) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9888/dev-support/hive-personality.sh
|
| git revision | master / 6edff8a |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9888/yetus/diff-checkstyle-standalone-metastore.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9888/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9888/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> The partition folders might not get cleaned up properly in the
> HiveMetaStore.add_partitions_core method if an exception occurs
> --
>
> Key: HIVE-18696
> URL: https://issues.apache.org/jira/browse/HIVE-18696
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-18696.1.patch, HIVE-18696.2.patch,
> HIVE-18696.3.patch, HIVE-18696.4.patch, HIVE-18696.5.patch
>
>
> When trying to add multiple partitions, but one of them cannot be created
> successfully, none of the partitions are created, but the folders might not
> be cleaned up properly. See the test case "testAddPartitionsOneInvalid" in
> the TestAddPartitions test.
> This is the problematic code in the HiveMetaStore.add_partitions_core method:
> {code:java}
> for (final Partition part : parts) {
> if (!part.getTableName().equals(tblName) ||
> !part.getDbName().equals(dbName)) {
> throw new MetaException("Partition does not belong to target
> table "
> + dbName + "." + tblName + ": " + part);
> }
> boolean shouldAdd = startAddPartition(ms, part, ifNotExists);
> if (!shouldAdd) {
> existingParts.add(part);
> LOG.info("Not adding partition " + part + " as it already
>
[jira] [Commented] (HIVE-18862) qfiles: prepare .q files for using datasets
[
https://issues.apache.org/jira/browse/HIVE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417810#comment-16417810
]
Hive QA commented on HIVE-18862:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12916356/HIVE-18862.06.patch
{color:green}SUCCESS:{color} +1 due to 2093 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 165 failed/errored test(s), 13839 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=92)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_subq_exists]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_dependency2]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_dependency]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fp_literal_arithmetic]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby3] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby7] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_complex_types]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_map_ppr]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_map_ppr_multi_distinct]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join25] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_7]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample5] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_bucketmapjoin1]
(batchId=25)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[authorization_view_8]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez_empty]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_exists]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby3]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dynpart_hashjoin_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[update_access_time_non_current_db]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_div0]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=104)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_notnull_constraint_violation]
(batchId=95)
[jira] [Commented] (HIVE-18786) NPE in Hive windowing functions
[ https://issues.apache.org/jira/browse/HIVE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417796#comment-16417796 ] Michael Bieniosek commented on HIVE-18786: -- I can work around it by removing windowing functions, or by downgrading to 2.2.0 > NPE in Hive windowing functions > --- > > Key: HIVE-18786 > URL: https://issues.apache.org/jira/browse/HIVE-18786 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.2 >Reporter: Michael Bieniosek >Priority: Major > > When I run a Hive query with windowing functions, if there's enough data I > get an NPE. > For example something like this query might break: > select id, created_date, max(created_date) over (partition by id) > latest_created_any from ... > The only workaround I've found is to remove the windowing functions entirely. > The stacktrace looks suspiciously similar to +HIVE-15278+, but I'm in > hive-2.3.2 which appears to have the bugfix applied. > > Caused by: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row (tag=0) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:297) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317) > at > org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185) > ... 14 more > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row (tag=0) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:287) > ... 16 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115) > at > org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114) > at > org.apache.hadoop.hive.ql.udf.ptf.BasePartitionEvaluator.getPartitionAgg(BasePartitionEvaluator.java:200) > at > org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.evaluateFunctionOnPartition(WindowingTableFunction.java:155) > at > org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.iterator(WindowingTableFunction.java:538) > at > org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:349) > at > org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:123) > at > org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18862) qfiles: prepare .q files for using datasets
[
https://issues.apache.org/jira/browse/HIVE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417793#comment-16417793
]
Hive QA commented on HIVE-18862:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
35s{color} | {color:red} root: The patch generated 3 new + 2151 unchanged - 2
fixed = 2154 total (was 2153) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/util: The patch generated 3 new + 189 unchanged
- 2 fixed = 192 total (was 191) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 11 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9887/dev-support/hive-personality.sh
|
| git revision | master / 9b84ed4 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9887/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9887/yetus/diff-checkstyle-itests_util.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9887/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9887/yetus/patch-asflicense-problems.txt
|
| modules | C: . accumulo-handler contrib hbase-handler itests/util ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9887/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> qfiles: prepare .q files for using datasets
> ---
>
> Key: HIVE-18862
> URL: https://issues.apache.org/jira/browse/HIVE-18862
> Project: Hive
> Issue Type: Sub-task
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18862.01.patch, HIVE-18862.02.patch,
> HIVE-18862.03.patch, HIVE-18862.04.patch, HIVE-18862.05.patch,
> HIVE-18862.06.patch
>
>
> # Parse .q files for source table usage
> # Add needed dataset annotations
> # Remove setInitScript("q_test_init.sql");
[jira] [Updated] (HIVE-18781) Create/Replicate Open, Commit (without writes) and Abort Txn events
[ https://issues.apache.org/jira/browse/HIVE-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-18781: --- Attachment: HIVE-18781.11.patch > Create/Replicate Open, Commit (without writes) and Abort Txn events > --- > > Key: HIVE-18781 > URL: https://issues.apache.org/jira/browse/HIVE-18781 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 3.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18781.01.patch, HIVE-18781.02.patch, > HIVE-18781.03.patch, HIVE-18781.04.patch, HIVE-18781.05.patch, > HIVE-18781.06.patch, HIVE-18781.07.patch, HIVE-18781.08.patch, > HIVE-18781.09.patch, HIVE-18781.10.patch, HIVE-18781.11.patch > > > *EVENT_OPEN_TXN:* > *Source Warehouse:* > - Create new event type EVENT_OPEN_TXN with related message format etc. > - When any transaction is opened either by auto-commit mode or > multi-statement mode, need to capture this event. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Open a txn in target warehouse. > - Create a map of source txn ID against target txn ID and persist the same > in metastore. There should be one map per replication policy (DBName.* incase > of DB level replication, DBName.TableName incase of table level replication) > > *EVENT_COMMIT_TXN (Without writes)* > Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions > modified within the txn. > *Source warehouse:* > - Create EVENT_COMMIT_TXN event type with corresponding message format etc. > *Target warehouse:* > - Repl load should read this event from the dump. > - Validate the source txn ID from the event using the Source-Target Txn ID > map maintained in target metastore. Also, need to check if corresponding > target txn ID is valid. > - If valid, then apply the event and commit the corresponding target > transaction. > - This new event should be idempotent such that if it is applied twice, then > second time it should be loop. > > *EVENT_ABORT_TXN* > Source Warehouse: > - Create new event type EVENT_ABORT_TXN with related message format etc. > - Capture this event when abort the txn. > - Repl dump should read this event from EventNotificationTable and dump the > message. > *Target Warehouse:* > - Repl load should read the event from the dump and get the message. > - Validate if source txn ID from the event is there in the source-target txn > ID map. If not there, just noop the event. > - If valid, then Abort the corresponding target txn and remove the entry > from source-target txn ID map. > All these new events should be idempotent such that if it is applied twice, > then second time it should be noop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19007) Support REPL LOAD from primary using replica connection configurations received through WITH clause.
[ https://issues.apache.org/jira/browse/HIVE-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417758#comment-16417758 ] ASF GitHub Bot commented on HIVE-19007: --- Github user sankarh closed the pull request at: https://github.com/apache/hive/pull/324 > Support REPL LOAD from primary using replica connection configurations > received through WITH clause. > > > Key: HIVE-19007 > URL: https://issues.apache.org/jira/browse/HIVE-19007 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.0.0 > > Attachments: HIVE-19007.01.patch, HIVE-19007.02.patch > > > Need to support running REPL LOAD command from primary for different > use-cases such as Cloud replication (for efficient use of cloud resources) or > workload management. > To achieve this, WITH clause of REPL LOAD lets user to pass Hive configs such > as hive.metastore.warehouse.dir, hive.metastore.uris, > hive.repl.replica.functions.root.dir etc, which can be used to establish > connection with replica warehouse. > The configs received from WITH clause of REPL LOAD are not set properly (due > to changes by HIVE-18716) to the tasks created. It is also required to re-get > the Hive db object if the configs are changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19007) Support REPL LOAD from primary using replica connection configurations received through WITH clause.
[ https://issues.apache.org/jira/browse/HIVE-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19007: Resolution: Fixed Status: Resolved (was: Patch Available) > Support REPL LOAD from primary using replica connection configurations > received through WITH clause. > > > Key: HIVE-19007 > URL: https://issues.apache.org/jira/browse/HIVE-19007 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.0.0 > > Attachments: HIVE-19007.01.patch, HIVE-19007.02.patch > > > Need to support running REPL LOAD command from primary for different > use-cases such as Cloud replication (for efficient use of cloud resources) or > workload management. > To achieve this, WITH clause of REPL LOAD lets user to pass Hive configs such > as hive.metastore.warehouse.dir, hive.metastore.uris, > hive.repl.replica.functions.root.dir etc, which can be used to establish > connection with replica warehouse. > The configs received from WITH clause of REPL LOAD are not set properly (due > to changes by HIVE-18716) to the tasks created. It is also required to re-get > the Hive db object if the configs are changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19007) Support REPL LOAD from primary using replica connection configurations received through WITH clause.
[ https://issues.apache.org/jira/browse/HIVE-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417733#comment-16417733 ] Sankar Hariappan commented on HIVE-19007: - Test failures are irrelevant to the patch and so it is committed to master. Thanks [~thejas] for the review! > Support REPL LOAD from primary using replica connection configurations > received through WITH clause. > > > Key: HIVE-19007 > URL: https://issues.apache.org/jira/browse/HIVE-19007 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.0.0 > > Attachments: HIVE-19007.01.patch, HIVE-19007.02.patch > > > Need to support running REPL LOAD command from primary for different > use-cases such as Cloud replication (for efficient use of cloud resources) or > workload management. > To achieve this, WITH clause of REPL LOAD lets user to pass Hive configs such > as hive.metastore.warehouse.dir, hive.metastore.uris, > hive.repl.replica.functions.root.dir etc, which can be used to establish > connection with replica warehouse. > The configs received from WITH clause of REPL LOAD are not set properly (due > to changes by HIVE-18716) to the tasks created. It is also required to re-get > the Hive db object if the configs are changed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19070) Add More Test To Druid Mini Cluster queries.
[ https://issues.apache.org/jira/browse/HIVE-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19070: Summary: Add More Test To Druid Mini Cluster queries. (was: Add More Test To Druid Mini Cluster 200 Tableau kind queries.) > Add More Test To Druid Mini Cluster queries. > - > > Key: HIVE-19070 > URL: https://issues.apache.org/jira/browse/HIVE-19070 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19070.patch > > > In This patch am adding 200 new tableau query that runs over a new Data set > called calcs. > The data set is very small. > I also have consolidated 3 different tests to run as one test this will help > with keeping execution time low. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18052) Run p-tests on mm tables
[ https://issues.apache.org/jira/browse/HIVE-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417716#comment-16417716 ] Steve Yeom commented on HIVE-18052: --- The major fixes and changes from the january patch of this jira are for my Hive 18237 and 18618, for whose problems I think [~sershe] have already checked in patches for Hive 18536 and 18571. But still I have been working on to find any changes (especially 18618's BasicStatsTask and etc) that can be applicable to the current master (and to fix any issues). > Run p-tests on mm tables > > > Key: HIVE-18052 > URL: https://issues.apache.org/jira/browse/HIVE-18052 > Project: Hive > Issue Type: Task >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-18052.1.patch, HIVE-18052.10.patch, > HIVE-18052.11.patch, HIVE-18052.12.patch, HIVE-18052.13.patch, > HIVE-18052.14.patch, HIVE-18052.15.patch, HIVE-18052.16.patch, > HIVE-18052.17.patch, HIVE-18052.18.patch, HIVE-18052.2.patch, > HIVE-18052.3.patch, HIVE-18052.4.patch, HIVE-18052.5.patch, > HIVE-18052.6.patch, HIVE-18052.7.patch, HIVE-18052.8.patch, HIVE-18052.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19070) Add More Test To Druid Mini Cluster 200 Tableau kind queries.
[ https://issues.apache.org/jira/browse/HIVE-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19070: -- Attachment: HIVE-19070.patch > Add More Test To Druid Mini Cluster 200 Tableau kind queries. > - > > Key: HIVE-19070 > URL: https://issues.apache.org/jira/browse/HIVE-19070 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19070.patch > > > In This patch am adding 200 new tableau query that runs over a new Data set > called calcs. > The data set is very small. > I also have consolidated 3 different tests to run as one test this will help > with keeping execution time low. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19070) Add More Test To Druid Mini Cluster 200 Tableau kind queries.
[ https://issues.apache.org/jira/browse/HIVE-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19070: -- Status: Patch Available (was: Open) > Add More Test To Druid Mini Cluster 200 Tableau kind queries. > - > > Key: HIVE-19070 > URL: https://issues.apache.org/jira/browse/HIVE-19070 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19070.patch > > > In This patch am adding 200 new tableau query that runs over a new Data set > called calcs. > The data set is very small. > I also have consolidated 3 different tests to run as one test this will help > with keeping execution time low. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
