[jira] [Updated] (HIVE-14426) Extensive logging on info level in WebHCat
[ https://issues.apache.org/jira/browse/HIVE-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-14426: -- Attachment: HIVE-14426.4.patch Updated configuration description > Extensive logging on info level in WebHCat > -- > > Key: HIVE-14426 > URL: https://issues.apache.org/jira/browse/HIVE-14426 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14426.2.patch, HIVE-14426.3.patch, > HIVE-14426.4.patch, HIVE-14426.patch > > > There is an extensive logging in WebHCat at info level, and even some > sensitive information could be logged -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12656) Turn hive.compute.query.using.stats on by default
[
https://issues.apache.org/jira/browse/HIVE-12656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420143#comment-15420143
]
Hive QA commented on HIVE-12656:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823596/HIVE-12656.05.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 58 failed/errored test(s), 10471 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[escape1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[escape2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_13]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_1]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_2]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_3]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_4]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_5]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_6]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_7]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_8]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_9]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_coltype_literals]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_all]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_partition_pruning]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[tez_union]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[vector_complex_all]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge1]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[alter_merge_2_orc]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[alter_merge_orc]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[cbo_udf_udaf]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[dynamic_partition_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[dynpart_sort_optimization]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_merge10]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_merge1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_merge_diff_fs]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_ppd_basic]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_union]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_complex_all]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vectorization_short_regress]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[bucketizedhiveinputformat]
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[orc_merge_diff_fs]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into6]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[lockneg_query_tbl_in_locked_db]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[alter_merge_orc]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[cbo_udf_udaf]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[list_bucket_dml_2]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_18]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_19]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_20]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats3]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats_noscan_2]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_view]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_short_regress]
org.apache.hive.beeline.TestBeeLineWithArgs.testEmbeddedBeelineOutputs
[jira] [Commented] (HIVE-14525) beeline still writing log data to stdout as of version 2.1.0
[
https://issues.apache.org/jira/browse/HIVE-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420119#comment-15420119
]
stephen sprague commented on HIVE-14525:
ok. lemme back up for a sec.
would people agree with me that stdout is reserved for *results* and stderr is
where everything else goes? like info, debug, errors and warnings and anything
else that comes up? This is especially relevant for csv and tsv style output
(machine readable.) for pretty output with ascii art around the results (human
readable) it isn't quite as critical.
i'm curious if there is disagreement here.
for context i'd like to create a tsv file and i'd like to use beeline. :)i
suppose the de-facto method would be to use insert overwrite [local] directory
and rsync the thing back to the cient - but i was hoping i could do it all
through beeline.
> beeline still writing log data to stdout as of version 2.1.0
>
>
> Key: HIVE-14525
> URL: https://issues.apache.org/jira/browse/HIVE-14525
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.1.0
>Reporter: stephen sprague
>
> simple test. note that i'm looking to get a tsv file back.
> {code}
> $ beeline -u dwrdevnn1 --showHeader=false --outputformat=tsv2 2>stderr
> > select count(*)
> > from default.dual;
> > SQL
> {code}
> instead i get this in stdout:
> {code}
> $ cat stdout
> 0: jdbc:hive2://dwrdevnn1.sv2.trulia.com:1000> select count(*)
> . . . . . . . . . . . . . . . . . . . . . . .> from default.dual;
> 0
> 0: jdbc:hive2://dwrdevnn1.sv2.trulia.com:1000>
> {code}
> i should only get one row which is the *result* of the query (which is 0) -
> not the ovthe loggy kind of lines you see above. that stuff goes to stderr my
> friends.
> also i refer to this ticket b/c the last comment suggested so - its close but
> not exactly the same.
> https://issues.apache.org/jira/browse/HIVE-14183
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12656) Turn hive.compute.query.using.stats on by default
[ https://issues.apache.org/jira/browse/HIVE-12656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12656: --- Status: Patch Available (was: Open) > Turn hive.compute.query.using.stats on by default > - > > Key: HIVE-12656 > URL: https://issues.apache.org/jira/browse/HIVE-12656 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12656.01.patch, HIVE-12656.02.patch, > HIVE-12656.03.patch, HIVE-12656.04.patch, HIVE-12656.05.patch > > > We now have hive.compute.query.using.stats=false by default. We plan to turn > it on by default so that we can have better performance. We can also set it > to false in some test cases to maintain the original purpose of those tests.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12656) Turn hive.compute.query.using.stats on by default
[ https://issues.apache.org/jira/browse/HIVE-12656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12656: --- Attachment: HIVE-12656.05.patch > Turn hive.compute.query.using.stats on by default > - > > Key: HIVE-12656 > URL: https://issues.apache.org/jira/browse/HIVE-12656 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12656.01.patch, HIVE-12656.02.patch, > HIVE-12656.03.patch, HIVE-12656.04.patch, HIVE-12656.05.patch > > > We now have hive.compute.query.using.stats=false by default. We plan to turn > it on by default so that we can have better performance. We can also set it > to false in some test cases to maintain the original purpose of those tests.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12656) Turn hive.compute.query.using.stats on by default
[ https://issues.apache.org/jira/browse/HIVE-12656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-12656: --- Status: Open (was: Patch Available) > Turn hive.compute.query.using.stats on by default > - > > Key: HIVE-12656 > URL: https://issues.apache.org/jira/browse/HIVE-12656 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-12656.01.patch, HIVE-12656.02.patch, > HIVE-12656.03.patch, HIVE-12656.04.patch, HIVE-12656.05.patch > > > We now have hive.compute.query.using.stats=false by default. We plan to turn > it on by default so that we can have better performance. We can also set it > to false in some test cases to maintain the original purpose of those tests.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14373) Add integration tests for hive on S3
[ https://issues.apache.org/jira/browse/HIVE-14373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420103#comment-15420103 ] Zoltan Haindrich commented on HIVE-14373: - [~yalovyyi] [~ayousufi] I've noticed your rb on hive-dev; and i've looked into it...it contains some new 'vm' files - which might cause some troubles now, that HIVE-1 is in...because we are currently moving away from using vm files to create integration tests: I was going to propose an updated version of the patch to save you the effort to migrate these tests into the new one...but I don't know which one to update(or maybe both?) ;) > Add integration tests for hive on S3 > > > Key: HIVE-14373 > URL: https://issues.apache.org/jira/browse/HIVE-14373 > Project: Hive > Issue Type: Sub-task >Reporter: Sergio Peña >Assignee: Abdullah Yousufi > Attachments: HIVE-14373.02.patch, HIVE-14373.patch > > > With Hive doing improvements to run on S3, it would be ideal to have better > integration testing on S3. > These S3 tests won't be able to be executed by HiveQA because it will need > Amazon credentials. We need to write suite based on ideas from the Hadoop > project where: > - an xml file is provided with S3 credentials > - a committer must run these tests manually to verify it works > - the xml file should not be part of the commit, and hiveqa should not run > these tests. > https://wiki.apache.org/hadoop/HowToContribute#Submitting_patches_against_object_stores_such_as_Amazon_S3.2C_OpenStack_Swift_and_Microsoft_Azure -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13930) upgrade Hive to latest Hadoop version
[ https://issues.apache.org/jira/browse/HIVE-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420060#comment-15420060 ] Ashutosh Chauhan commented on HIVE-13930: - [~stakiar] Any update on this one? > upgrade Hive to latest Hadoop version > - > > Key: HIVE-13930 > URL: https://issues.apache.org/jira/browse/HIVE-13930 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13930.01.patch, HIVE-13930.02.patch, > HIVE-13930.03.patch, HIVE-13930.04.patch, HIVE-13930.05.patch, > HIVE-13930.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14290) Refactor HIVE-14054 to use Collections#newSetFromMap
[ https://issues.apache.org/jira/browse/HIVE-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14290: Status: Patch Available (was: Open) > Refactor HIVE-14054 to use Collections#newSetFromMap > > > Key: HIVE-14290 > URL: https://issues.apache.org/jira/browse/HIVE-14290 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Peter Slawski >Assignee: Peter Slawski >Priority: Trivial > Attachments: HIVE-14290.1.patch, HIVE-14290.1.patch > > > There is a minor refactor that can be made to HiveMetaStoreChecker so that it > cleanly creates and uses a set that is backed by a Map implementation. In > this case, the underlying Map implementation is ConcurrentHashMap. This > refactor will help prevent issues such as the one reported in HIVE-14054. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14290) Refactor HIVE-14054 to use Collections#newSetFromMap
[ https://issues.apache.org/jira/browse/HIVE-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14290: Status: Open (was: Patch Available) > Refactor HIVE-14054 to use Collections#newSetFromMap > > > Key: HIVE-14290 > URL: https://issues.apache.org/jira/browse/HIVE-14290 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Peter Slawski >Assignee: Peter Slawski >Priority: Trivial > Attachments: HIVE-14290.1.patch, HIVE-14290.1.patch > > > There is a minor refactor that can be made to HiveMetaStoreChecker so that it > cleanly creates and uses a set that is backed by a Map implementation. In > this case, the underlying Map implementation is ConcurrentHashMap. This > refactor will help prevent issues such as the one reported in HIVE-14054. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14290) Refactor HIVE-14054 to use Collections#newSetFromMap
[ https://issues.apache.org/jira/browse/HIVE-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420056#comment-15420056 ] Ashutosh Chauhan commented on HIVE-14290: - [~petersla] You may want to reupload your patch to trigger Hive QA. > Refactor HIVE-14054 to use Collections#newSetFromMap > > > Key: HIVE-14290 > URL: https://issues.apache.org/jira/browse/HIVE-14290 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 2.1.0 >Reporter: Peter Slawski >Assignee: Peter Slawski >Priority: Trivial > Attachments: HIVE-14290.1.patch, HIVE-14290.1.patch > > > There is a minor refactor that can be made to HiveMetaStoreChecker so that it > cleanly creates and uses a set that is backed by a Map implementation. In > this case, the underlying Map implementation is ConcurrentHashMap. This > refactor will help prevent issues such as the one reported in HIVE-14054. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14345) Beeline result table has erroneous characters
[
https://issues.apache.org/jira/browse/HIVE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420051#comment-15420051
]
Ashutosh Chauhan commented on HIVE-14345:
-
+1
> Beeline result table has erroneous characters
> --
>
> Key: HIVE-14345
> URL: https://issues.apache.org/jira/browse/HIVE-14345
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Jeremy Beard
>Assignee: Miklos Csanady
>Priority: Minor
> Attachments: HIVE-14345.3.patch, HIVE-14345.4.patch,
> HIVE-14345.5.patch, HIVE-14345.patch
>
>
> Beeline returns query results with erroneous characters. For example:
> {code}
> 0: jdbc:hive2://:1/def> select 10;
> +--+--+
> | _c0 |
> +--+--+
> | 10 |
> +--+--+
> 1 row selected (3.207 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14519) Multi insert query bug
[
https://issues.apache.org/jira/browse/HIVE-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-14519:
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Yongzhi!
> Multi insert query bug
> --
>
> Key: HIVE-14519
> URL: https://issues.apache.org/jira/browse/HIVE-14519
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Fix For: 2.2.0
>
> Attachments: HIVE-14519.1.patch
>
>
> When running multi-insert queries, when one of the query is not returning
> results, the other query is not returning the right result.
> For example:
> After following query, there is no value in /tmp/emp/dir3/00_0
> {noformat}
> From (select * from src) a
> insert overwrite directory '/tmp/emp/dir1/'
> select key, value
> insert overwrite directory '/tmp/emp/dir2/'
> select 'header'
> where 1=2
> insert overwrite directory '/tmp/emp/dir3/'
> select key, value
> where key = 100;
> {noformat}
> where clause in the second insert should not affect the third insert.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14460) AccumuloCliDriver migration to junit4
[ https://issues.apache.org/jira/browse/HIVE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14460: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Zoltan! > AccumuloCliDriver migration to junit4 > - > > Key: HIVE-14460 > URL: https://issues.apache.org/jira/browse/HIVE-14460 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Fix For: 2.2.0 > > Attachments: HIVE-14460.1.patch > > > This test have been left behind in HIVE-1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14351) Minor improvement in genUnionPlan method
[
https://issues.apache.org/jira/browse/HIVE-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-14351:
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Ratandeep!
> Minor improvement in genUnionPlan method
>
>
> Key: HIVE-14351
> URL: https://issues.apache.org/jira/browse/HIVE-14351
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Ratandeep Ratti
>Assignee: Ratandeep Ratti
> Fix For: 2.2.0
>
> Attachments: HIVE-14351.patch
>
>
> {{org.apache.hadoop.hive.ql.parse.SemanticAnalyzer#genUnionPlan}} method can
> trip new users reading the code.
> Specifically on line 8979
> {code}
> HashMap leftmap = leftRR.getFieldMap(leftalias);
> HashMap rightmap = rightRR.getFieldMap(rightalias);
> {code}
> These column maps are actually LinkedHashMaps and the code relies on this
> fact when iterating the two union branches in order.
> This was not clear immediately and left me wondering how is it that
> traversal order is consistent.
> I've updated the code with this simple fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14361) Empty method in TestClientCommandHookFactory
[ https://issues.apache.org/jira/browse/HIVE-14361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14361: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Peter! > Empty method in TestClientCommandHookFactory > > > Key: HIVE-14361 > URL: https://issues.apache.org/jira/browse/HIVE-14361 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Trivial > Fix For: 2.2.0 > > Attachments: HIVE-14361.patch > > > Remove the empty method left in TestClientCommandHookFactory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14361) Empty method in TestClientCommandHookFactory
[ https://issues.apache.org/jira/browse/HIVE-14361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420016#comment-15420016 ] Ashutosh Chauhan commented on HIVE-14361: - +1 > Empty method in TestClientCommandHookFactory > > > Key: HIVE-14361 > URL: https://issues.apache.org/jira/browse/HIVE-14361 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Trivial > Attachments: HIVE-14361.patch > > > Remove the empty method left in TestClientCommandHookFactory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14519) Multi insert query bug
[
https://issues.apache.org/jira/browse/HIVE-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420015#comment-15420015
]
Yongzhi Chen commented on HIVE-14519:
-
The failures are not related.
Thanks [~ashutoshc] for reviewing the code.
> Multi insert query bug
> --
>
> Key: HIVE-14519
> URL: https://issues.apache.org/jira/browse/HIVE-14519
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Attachments: HIVE-14519.1.patch
>
>
> When running multi-insert queries, when one of the query is not returning
> results, the other query is not returning the right result.
> For example:
> After following query, there is no value in /tmp/emp/dir3/00_0
> {noformat}
> From (select * from src) a
> insert overwrite directory '/tmp/emp/dir1/'
> select key, value
> insert overwrite directory '/tmp/emp/dir2/'
> select 'header'
> where 1=2
> insert overwrite directory '/tmp/emp/dir3/'
> select key, value
> where key = 100;
> {noformat}
> where clause in the second insert should not affect the third insert.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14460) AccumuloCliDriver migration to junit4
[ https://issues.apache.org/jira/browse/HIVE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420013#comment-15420013 ] Ashutosh Chauhan commented on HIVE-14460: - +1 > AccumuloCliDriver migration to junit4 > - > > Key: HIVE-14460 > URL: https://issues.apache.org/jira/browse/HIVE-14460 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-14460.1.patch > > > This test have been left behind in HIVE-1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14519) Multi insert query bug
[
https://issues.apache.org/jira/browse/HIVE-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420008#comment-15420008
]
Ashutosh Chauhan commented on HIVE-14519:
-
+1
> Multi insert query bug
> --
>
> Key: HIVE-14519
> URL: https://issues.apache.org/jira/browse/HIVE-14519
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Attachments: HIVE-14519.1.patch
>
>
> When running multi-insert queries, when one of the query is not returning
> results, the other query is not returning the right result.
> For example:
> After following query, there is no value in /tmp/emp/dir3/00_0
> {noformat}
> From (select * from src) a
> insert overwrite directory '/tmp/emp/dir1/'
> select key, value
> insert overwrite directory '/tmp/emp/dir2/'
> select 'header'
> where 1=2
> insert overwrite directory '/tmp/emp/dir3/'
> select key, value
> where key = 100;
> {noformat}
> where clause in the second insert should not affect the third insert.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420007#comment-15420007 ] Saket Saurabh commented on HIVE-14035: -- Thanks [~leftylev]. Sure I will add it as a wiki page too. > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: Design.Document.Improving ACID performance in > Hive.01.docx, Design.Document.Improving ACID performance in Hive.02.docx, > HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, > HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, > HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch, > HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, > HIVE-14035.14.patch, HIVE-14035.15.patch, HIVE-14035.16.patch, > HIVE-14035.17.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14532) Enable qtests from IDE
[
https://issues.apache.org/jira/browse/HIVE-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419989#comment-15419989
]
Hive QA commented on HIVE-14532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823544/HIVE-14532.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[schemeAuthority]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/880/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/880/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-880/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823544 - PreCommit-HIVE-MASTER-Build
> Enable qtests from IDE
> --
>
> Key: HIVE-14532
> URL: https://issues.apache.org/jira/browse/HIVE-14532
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14532.1.patch
>
>
> with HIVE-1 applied; i've played around with executing qtest-s from
> eclipse...after the patch seemed ok; i've checked it with:
> {code}
> git clean -dfx
> mvn package install eclipse:eclipse -Pitests -DskipTests
> mvn -q test -Pitests -Dtest=TestNegativeCliDriver -Dqfile=combine2.q
> {code}
> the last step I think is not required...but I bootstrapped and checked my
> project integrity this way.
> After this I was able to execute {{TestCliDriver}} from eclipse using
> {{-Dqfile=combine.q}}, other qfiles may or may not work...but will have at
> least some chances to be usable.
> For my biggest surprise {{alter_concatenate_indexed_table.q}} also
> passed...which contains relative file references - and I suspected that it
> will have issues with that..
> note: I've the datanucleus plugin installed...and i use it when I need to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14525) beeline still writing log data to stdout as of version 2.1.0
[
https://issues.apache.org/jira/browse/HIVE-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419956#comment-15419956
]
Miklos Csanady commented on HIVE-14525:
---
The "loggy kind of lines" contains the query, which is the subject of the other
issue.
> beeline still writing log data to stdout as of version 2.1.0
>
>
> Key: HIVE-14525
> URL: https://issues.apache.org/jira/browse/HIVE-14525
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.1.0
>Reporter: stephen sprague
>
> simple test. note that i'm looking to get a tsv file back.
> {code}
> $ beeline -u dwrdevnn1 --showHeader=false --outputformat=tsv2 2>stderr
> > select count(*)
> > from default.dual;
> > SQL
> {code}
> instead i get this in stdout:
> {code}
> $ cat stdout
> 0: jdbc:hive2://dwrdevnn1.sv2.trulia.com:1000> select count(*)
> . . . . . . . . . . . . . . . . . . . . . . .> from default.dual;
> 0
> 0: jdbc:hive2://dwrdevnn1.sv2.trulia.com:1000>
> {code}
> i should only get one row which is the *result* of the query (which is 0) -
> not the ovthe loggy kind of lines you see above. that stuff goes to stderr my
> friends.
> also i refer to this ticket b/c the last comment suggested so - its close but
> not exactly the same.
> https://issues.apache.org/jira/browse/HIVE-14183
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14460) AccumuloCliDriver migration to junit4
[
https://issues.apache.org/jira/browse/HIVE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419952#comment-15419952
]
Hive QA commented on HIVE-14460:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822893/HIVE-14460.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/879/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/879/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-879/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822893 - PreCommit-HIVE-MASTER-Build
> AccumuloCliDriver migration to junit4
> -
>
> Key: HIVE-14460
> URL: https://issues.apache.org/jira/browse/HIVE-14460
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14460.1.patch
>
>
> This test have been left behind in HIVE-1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14345) Beeline result table has erroneous characters
[
https://issues.apache.org/jira/browse/HIVE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419922#comment-15419922
]
Miklos Csanady commented on HIVE-14345:
---
None of the failures involve the beeline component, so they are not relevant to
this patch.
> Beeline result table has erroneous characters
> --
>
> Key: HIVE-14345
> URL: https://issues.apache.org/jira/browse/HIVE-14345
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Jeremy Beard
>Assignee: Miklos Csanady
>Priority: Minor
> Attachments: HIVE-14345.3.patch, HIVE-14345.4.patch,
> HIVE-14345.5.patch, HIVE-14345.patch
>
>
> Beeline returns query results with erroneous characters. For example:
> {code}
> 0: jdbc:hive2://:1/def> select 10;
> +--+--+
> | _c0 |
> +--+--+
> | 10 |
> +--+--+
> 1 row selected (3.207 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14413) Extend HivePreFilteringRule to traverse inside elements of DNF/CNF and extract more deterministic pieces out
[
https://issues.apache.org/jira/browse/HIVE-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419913#comment-15419913
]
Hive QA commented on HIVE-14413:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823577/HIVE-14413.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[schemeAuthority]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/878/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/878/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-878/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823577 - PreCommit-HIVE-MASTER-Build
> Extend HivePreFilteringRule to traverse inside elements of DNF/CNF and
> extract more deterministic pieces out
>
>
> Key: HIVE-14413
> URL: https://issues.apache.org/jira/browse/HIVE-14413
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14413.01.patch, HIVE-14413.02.patch,
> HIVE-14413.03.patch, HIVE-14413.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14533) improve performance of enforceMaxLength in HiveCharWritable/HiveVarcharWritable
[
https://issues.apache.org/jira/browse/HIVE-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419882#comment-15419882
]
Hive QA commented on HIVE-14533:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823557/HIVE-14533.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10470 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/877/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/877/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-877/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823557 - PreCommit-HIVE-MASTER-Build
> improve performance of enforceMaxLength in
> HiveCharWritable/HiveVarcharWritable
> ---
>
> Key: HIVE-14533
> URL: https://issues.apache.org/jira/browse/HIVE-14533
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Affects Versions: 1.2.1, 2.1.0
>Reporter: Thomas Friedrich
>Assignee: Thomas Friedrich
>Priority: Minor
> Labels: performance
> Attachments: HIVE-14533.patch
>
>
> The enforceMaxLength method in HiveVarcharWritable calls
> set(getHiveVarchar(), maxLength); and in HiveCharWritable set(getHiveChar(),
> maxLength); no matter how long the string is. The calls to getHiveVarchar()
> and getHiveChar() decode the string every time the method is called
> (Text.toString() calls Text.decode). This can be very expensive and is
> unnecessary if the string is shorter than maxLength for HiveVarcharWritable
> or different than maxLength for HiveCharWritable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14527) Schema evolution tests are not running in TestCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419858#comment-15419858
]
Hive QA commented on HIVE-14527:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823560/HIVE-14527.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/876/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/876/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-876/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-MASTER-Build-876/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 0b46f4e HIVE-14448: Queries with predicate fail when ETL split
strategy is chosen for ACID tables (Matt McCline, reviewed by Sergey Shelukhin)
+ git clean -f -d
Removing itests/src/test/resources/testconfiguration.properties.orig
Removing ql/src/test/queries/clientpositive/acid_bucket_pruning.q
Removing ql/src/test/results/clientpositive/tez/acid_bucket_pruning.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 0b46f4e HIVE-14448: Queries with predicate fail when ETL split
strategy is chosen for ACID tables (Matt McCline, reviewed by Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823560 - PreCommit-HIVE-MASTER-Build
> Schema evolution tests are not running in TestCliDriver
> ---
>
> Key: HIVE-14527
> URL: https://issues.apache.org/jira/browse/HIVE-14527
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 2.2.0
>Reporter: Matt McCline
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14527.1.patch
>
>
> HIVE-14376 broke something that makes schema evolution tests being excluded
> from TestCliDriver test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14199) Enable Bucket Pruning for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-14199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419857#comment-15419857
]
Hive QA commented on HIVE-14199:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823552/HIVE-14199.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10471 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/875/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/875/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-875/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823552 - PreCommit-HIVE-MASTER-Build
> Enable Bucket Pruning for ACID tables
> -
>
> Key: HIVE-14199
> URL: https://issues.apache.org/jira/browse/HIVE-14199
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Saket Saurabh
>Assignee: Saket Saurabh
> Attachments: HIVE-14199.01.patch, HIVE-14199.02.patch,
> HIVE-14199.03.patch
>
>
> Currently, ACID tables do not benefit from the bucket pruning feature
> introduced in HIVE-11525. The reason for this has been the fact that bucket
> pruning happens at split generation level and for ACID, traditionally the
> delta files were never split. The parallelism for ACID was then restricted to
> the number of buckets. There would be as many splits as the number of buckets
> and each worker processing one split would inevitably read all the delta
> files for that bucket, even when the query may have originally required only
> one of the buckets to be read.
> However, HIVE-14035 now enables even the delta files to be also split. What
> this means is that now we have enough information at the split generation
> level to determine appropriate buckets to process for the delta files. This
> can efficiently allow us to prune unnecessary buckets for delta files and
> will lead to good performance gain for a large number of selective queries on
> ACID tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14513) Enhance custom query feature in LDAP atn to support resultset of ldap groups
[
https://issues.apache.org/jira/browse/HIVE-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419849#comment-15419849
]
Lefty Leverenz commented on HIVE-14513:
---
Should this be documented in the wiki?
> Enhance custom query feature in LDAP atn to support resultset of ldap groups
>
>
> Key: HIVE-14513
> URL: https://issues.apache.org/jira/browse/HIVE-14513
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14513.patch
>
>
> LDAP Authenticator can be configured to use a result set from a LDAP query to
> authenticate. However, is it expected that this LDAP query would only result
> a set of users (aka full DNs for the users in LDAP).
> However, its not always straightforward to be able to author queries that
> return users. For example, say you would like to allow "all users from group1
> and group2" to be authenticated. The LDAP query has to return a union of all
> members of the group1 and group2.
> For example, one common configuration is that groups contain a list of its
> users
> "dn: uid=group1,ou=Groups,dc=example,dc=com",
> "distinguishedName: uid=group1,ou=Groups,dc=example,dc=com",
> "objectClass: top",
> "objectClass: groupOfNames",
> "objectClass: ExtensibleObject",
> "cn: group1",
> "ou: Groups",
> "sn: group1",
> "member: uid=user1,ou=People,dc=example,dc=com",
> The query
> {{(&(objectClass=groupOfNames)(|(cn=group1)(cn=group2)))}}
> will return the entries
> uid=group1,ou=Groups,dc=example,dc=com
> uid=group2,ou=Groups,dc=example,dc=com
> but there is no means to form a query that would return just the values of
> "member" attributes. (ldap client tools are able to do by filtering out the
> attributes on these entries.
> So it will be useful to have such support to be able to specify queries that
> return groups.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14532) Enable qtests from IDE
[
https://issues.apache.org/jira/browse/HIVE-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419844#comment-15419844
]
Zoltan Haindrich commented on HIVE-14532:
-
if someone could check the usability of this patch with idea: that would be
invaluable...
and there is one more thing i'm intrested in: after creating the idea project
files and importing it, are there any "problems" with the project? eclipse has
missing project references to hive-metastore in 3 modules: contrib, llap-server
and some other - i guess that this is caused by the 'shading' plugin...
> Enable qtests from IDE
> --
>
> Key: HIVE-14532
> URL: https://issues.apache.org/jira/browse/HIVE-14532
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14532.1.patch
>
>
> with HIVE-1 applied; i've played around with executing qtest-s from
> eclipse...after the patch seemed ok; i've checked it with:
> {code}
> git clean -dfx
> mvn package install eclipse:eclipse -Pitests -DskipTests
> mvn -q test -Pitests -Dtest=TestNegativeCliDriver -Dqfile=combine2.q
> {code}
> the last step I think is not required...but I bootstrapped and checked my
> project integrity this way.
> After this I was able to execute {{TestCliDriver}} from eclipse using
> {{-Dqfile=combine.q}}, other qfiles may or may not work...but will have at
> least some chances to be usable.
> For my biggest surprise {{alter_concatenate_indexed_table.q}} also
> passed...which contains relative file references - and I suspected that it
> will have issues with that..
> note: I've the datanucleus plugin installed...and i use it when I need to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14532) Enable qtests from IDE
[
https://issues.apache.org/jira/browse/HIVE-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-14532:
Status: Patch Available (was: Open)
> Enable qtests from IDE
> --
>
> Key: HIVE-14532
> URL: https://issues.apache.org/jira/browse/HIVE-14532
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14532.1.patch
>
>
> with HIVE-1 applied; i've played around with executing qtest-s from
> eclipse...after the patch seemed ok; i've checked it with:
> {code}
> git clean -dfx
> mvn package install eclipse:eclipse -Pitests -DskipTests
> mvn -q test -Pitests -Dtest=TestNegativeCliDriver -Dqfile=combine2.q
> {code}
> the last step I think is not required...but I bootstrapped and checked my
> project integrity this way.
> After this I was able to execute {{TestCliDriver}} from eclipse using
> {{-Dqfile=combine.q}}, other qfiles may or may not work...but will have at
> least some chances to be usable.
> For my biggest surprise {{alter_concatenate_indexed_table.q}} also
> passed...which contains relative file references - and I suspected that it
> will have issues with that..
> note: I've the datanucleus plugin installed...and i use it when I need to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14460) AccumuloCliDriver migration to junit4
[ https://issues.apache.org/jira/browse/HIVE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-14460: Status: Patch Available (was: Open) > AccumuloCliDriver migration to junit4 > - > > Key: HIVE-14460 > URL: https://issues.apache.org/jira/browse/HIVE-14460 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-14460.1.patch > > > This test have been left behind in HIVE-1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14345) Beeline result table has erroneous characters
[
https://issues.apache.org/jira/browse/HIVE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419841#comment-15419841
]
Hive QA commented on HIVE-14345:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823583/HIVE-14345.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10467 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/874/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/874/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-874/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823583 - PreCommit-HIVE-MASTER-Build
> Beeline result table has erroneous characters
> --
>
> Key: HIVE-14345
> URL: https://issues.apache.org/jira/browse/HIVE-14345
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Jeremy Beard
>Assignee: Miklos Csanady
>Priority: Minor
> Attachments: HIVE-14345.3.patch, HIVE-14345.4.patch,
> HIVE-14345.5.patch, HIVE-14345.patch
>
>
> Beeline returns query results with erroneous characters. For example:
> {code}
> 0: jdbc:hive2://:1/def> select 10;
> +--+--+
> | _c0 |
> +--+--+
> | 10 |
> +--+--+
> 1 row selected (3.207 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419834#comment-15419834 ] Lefty Leverenz commented on HIVE-14035: --- Doc note: Besides the design document, which should be added to the wiki, there is a new configuration parameter (*hive.txn.operational.properties*) that will need to be documented in the wiki for release 2.2.0. * [Configuration Properties -- Transactions and Compactor | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-TransactionsandCompactor] Added a TODOC2.2 label. > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: Design.Document.Improving ACID performance in > Hive.01.docx, Design.Document.Improving ACID performance in Hive.02.docx, > HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, > HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, > HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch, > HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, > HIVE-14035.14.patch, HIVE-14035.15.patch, HIVE-14035.16.patch, > HIVE-14035.17.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-14035: -- Labels: TODOC2.2 (was: ) > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: Design.Document.Improving ACID performance in > Hive.01.docx, Design.Document.Improving ACID performance in Hive.02.docx, > HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, > HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, > HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch, > HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, > HIVE-14035.14.patch, HIVE-14035.15.patch, HIVE-14035.16.patch, > HIVE-14035.17.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419832#comment-15419832 ] Lefty Leverenz commented on HIVE-14035: --- [~saketj], I added a link to your design document (version 2) in the wiki's Design Docs. Perhaps the link will become out-of-date if you revise the doc again, but eventually the doc should be reformatted as a wiki page. * [Design Docs -- In Progress | https://cwiki.apache.org/confluence/display/Hive/DesignDocs#DesignDocs-InProgress] > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Fix For: 2.2.0 > > Attachments: Design.Document.Improving ACID performance in > Hive.01.docx, Design.Document.Improving ACID performance in Hive.02.docx, > HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, > HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, > HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch, > HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, > HIVE-14035.14.patch, HIVE-14035.15.patch, HIVE-14035.16.patch, > HIVE-14035.17.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14448) Queries with predicate fail when ETL split strategy is chosen for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419826#comment-15419826
]
Matt McCline commented on HIVE-14448:
-
Committed to master and branch-2.1
> Queries with predicate fail when ETL split strategy is chosen for ACID tables
> -
>
> Key: HIVE-14448
> URL: https://issues.apache.org/jira/browse/HIVE-14448
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Saket Saurabh
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14448.01.patch, HIVE-14448.02.patch,
> HIVE-14448.03.patch, HIVE-14448.04.patch, HIVE-14448.patch
>
>
> When ETL split strategy is applied to ACID tables with predicate pushdown
> (SARG enabled), split generation fails for ACID. This bug will be usually
> exposed when working with data at scale, because in most otherwise cases only
> BI split strategy is chosen. My guess is that this is happening because the
> correct readerSchema is not being picked up when we try to extract SARG
> column names.
> Quickest way to reproduce is to add the following unit test to
> ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java
> {code:title=ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java|borderStyle=solid}
> @Test
> public void testETLSplitStrategyForACID() throws Exception {
> hiveConf.setVar(HiveConf.ConfVars.HIVE_ORC_SPLIT_STRATEGY, "ETL");
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVEOPTINDEXFILTER, true);
> runStatementOnDriver("insert into " + Table.ACIDTBL + " values(1,2)");
> runStatementOnDriver("alter table " + Table.ACIDTBL + " compact 'MAJOR'");
> runWorker(hiveConf);
> List rs = runStatementOnDriver("select * from " + Table.ACIDTBL
> + " where a = 1");
> int[][] resultData = new int[][] {{1,2}};
> Assert.assertEquals(stringifyValues(resultData), rs);
> }
> {code}
> Back-trace for this failed test is as follows:
> {code}
> exec.Task: Job Submission failed with exception
> 'java.lang.RuntimeException(ORC split generation failed with exception:
> java.lang.NegativeArraySizeException)'
> java.lang.RuntimeException: ORC split generation failed with exception:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1570)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1656)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:370)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:488)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:329)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:321)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:197)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1297)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1294)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1294)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:417)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:141)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1962)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1389)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1131)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1119)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.runStatementOnDriver(TestTxnCommands2.java:1292)
> at
>
[jira] [Updated] (HIVE-14448) Queries with predicate fail when ETL split strategy is chosen for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-14448:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Queries with predicate fail when ETL split strategy is chosen for ACID tables
> -
>
> Key: HIVE-14448
> URL: https://issues.apache.org/jira/browse/HIVE-14448
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Saket Saurabh
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14448.01.patch, HIVE-14448.02.patch,
> HIVE-14448.03.patch, HIVE-14448.04.patch, HIVE-14448.patch
>
>
> When ETL split strategy is applied to ACID tables with predicate pushdown
> (SARG enabled), split generation fails for ACID. This bug will be usually
> exposed when working with data at scale, because in most otherwise cases only
> BI split strategy is chosen. My guess is that this is happening because the
> correct readerSchema is not being picked up when we try to extract SARG
> column names.
> Quickest way to reproduce is to add the following unit test to
> ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java
> {code:title=ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java|borderStyle=solid}
> @Test
> public void testETLSplitStrategyForACID() throws Exception {
> hiveConf.setVar(HiveConf.ConfVars.HIVE_ORC_SPLIT_STRATEGY, "ETL");
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVEOPTINDEXFILTER, true);
> runStatementOnDriver("insert into " + Table.ACIDTBL + " values(1,2)");
> runStatementOnDriver("alter table " + Table.ACIDTBL + " compact 'MAJOR'");
> runWorker(hiveConf);
> List rs = runStatementOnDriver("select * from " + Table.ACIDTBL
> + " where a = 1");
> int[][] resultData = new int[][] {{1,2}};
> Assert.assertEquals(stringifyValues(resultData), rs);
> }
> {code}
> Back-trace for this failed test is as follows:
> {code}
> exec.Task: Job Submission failed with exception
> 'java.lang.RuntimeException(ORC split generation failed with exception:
> java.lang.NegativeArraySizeException)'
> java.lang.RuntimeException: ORC split generation failed with exception:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1570)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1656)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:370)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:488)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:329)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:321)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:197)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1297)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1294)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1294)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:417)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:141)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1962)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1389)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1131)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1119)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.runStatementOnDriver(TestTxnCommands2.java:1292)
> at
>
