date:20180114

[jira] [Updated] (HIVE-18341) Add repl load support for adding "raw" namespace for TDE with same encryption keys

2018-01-14 Thread anishek (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-18341:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to master. Thanks for review [~thejas]!

> Add repl load support for adding "raw" namespace for TDE with same encryption 
> keys
> --
>
> Key: HIVE-18341
> URL: https://issues.apache.org/jira/browse/HIVE-18341
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18341.0.patch, HIVE-18341.1.patch, 
> HIVE-18341.2.patch, HIVE-18341.3.patch
>
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Running_as_the_superuser
> "a new virtual path prefix, /.reserved/raw/, that gives superusers direct 
> access to the underlying block data in the filesystem. This allows superusers 
> to distcp data without needing having access to encryption keys, and also 
> avoids the overhead of decrypting and re-encrypting data."
> We need to introduce a new option in "Repl Load" command that will change the 
> files being copied in distcp to have this "/.reserved/raw/" namespace before 
> the file paths.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18341) Add repl load support for adding "raw" namespace for TDE with same encryption keys

2018-01-14 Thread anishek (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325894#comment-16325894
 ] 

anishek commented on HIVE-18341:


*  
[org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample_islocalmode_hook]
 : works on local machine
* 
[org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]:
 fails on local machine, but code path for this test case was not changed, also 
across two runs for the same patch it failed in one of them, so seems like a 
sability issue. 
*  [org.apache.hive.jdbc.TestSchedulerQueue.testQueueMappingCheckDisabled: 
works on local machine.
*  
[org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]:
 works on local machine.

* 
[org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]:
 works on local machine.

 others are from older builds. 

 

> Add repl load support for adding "raw" namespace for TDE with same encryption 
> keys
> --
>
> Key: HIVE-18341
> URL: https://issues.apache.org/jira/browse/HIVE-18341
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18341.0.patch, HIVE-18341.1.patch, 
> HIVE-18341.2.patch, HIVE-18341.3.patch
>
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Running_as_the_superuser
> "a new virtual path prefix, /.reserved/raw/, that gives superusers direct 
> access to the underlying block data in the filesystem. This allows superusers 
> to distcp data without needing having access to encryption keys, and also 
> avoids the overhead of decrypting and re-encrypting data."
> We need to introduce a new option in "Repl Load" command that will change the 
> files being copied in distcp to have this "/.reserved/raw/" namespace before 
> the file paths.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID

2018-01-14 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-18192:
--
Labels: ACID DR pull-request-available  (was: ACID DR)

> Introduce WriteID per table rather than using global transaction ID
> ---
>
> Key: HIVE-18192
> URL: https://issues.apache.org/jira/browse/HIVE-18192
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: Sankar Hariappan
>  Labels: ACID, DR, pull-request-available
> Fix For: 3.0.0
>
>
> To support ACID replication, we will be introducing a per table write Id 
> which will replace the transaction id in the primary key for each row in a 
> ACID table. 
> The current primary key is determined via 
> 
> which will move to 
> 
> a persistable map of global txn id -> to table -> write id for that table has 
> to be maintained to now allow Snapshot isolation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID

2018-01-14 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325734#comment-16325734
 ] 

ASF GitHub Bot commented on HIVE-18192:
---

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/290

HIVE-18192: Introduce WriteID per table rather than using global 
transaction ID



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-18192

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/290.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #290


commit ced6d749a7e65c42ed31a8af9e38faaf5941251a
Author: Sankar Hariappan 
Date:   2018-01-03T05:47:38Z

HIVE-18192: Introduce WriteID per table rather than using global 
transaction ID




> Introduce WriteID per table rather than using global transaction ID
> ---
>
> Key: HIVE-18192
> URL: https://issues.apache.org/jira/browse/HIVE-18192
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: Sankar Hariappan
>  Labels: ACID, DR, pull-request-available
> Fix For: 3.0.0
>
>
> To support ACID replication, we will be introducing a per table write Id 
> which will replace the transaction id in the primary key for each row in a 
> ACID table. 
> The current primary key is determined via 
> 
> which will move to 
> 
> a persistable map of global txn id -> to table -> write id for that table has 
> to be maintained to now allow Snapshot isolation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths

2018-01-14 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325597#comment-16325597
 ] 

Hive QA commented on HIVE-18061:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906024/HIVE-18061.09.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 11559 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook] 
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] 
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] 
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
 (batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[leftsemijoin_mr]
 (batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] 
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
 (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] 
(batchId=103)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
 (batchId=94)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
 (batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_16]
 (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket4] 
(batchId=140)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin7] 
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[disable_merge_for_bucketing]
 (batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[leftsemijoin_mr] 
(batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] 
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[scriptfile1] 
(batchId=143)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
 (batchId=214)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=254)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill 
(batchId=236)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8616/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8616/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8616/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12906024 - PreCommit-HIVE-Build

> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch, 
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch, 
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch, 
> HIVE-18061.09.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths

2018-01-14 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325559#comment-16325559
 ] 

Hive QA commented on HIVE-18061:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} itests/util: The patch generated 1 new + 188 unchanged 
- 6 fixed = 189 total (was 194) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 21s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 970d8c9 |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8616/yetus/diff-checkstyle-itests_util.txt
 |
| modules | C: ql itests/util U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8616/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch, 
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch, 
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch, 
> HIVE-18061.09.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-18061) q.outs: be more selective with masking hdfs paths

2018-01-14 Thread Laszlo Bodor (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Bodor updated HIVE-18061:

Attachment: HIVE-18061.09.patch

> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch, 
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch, 
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch, 
> HIVE-18061.09.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-18445) qtests: auto_join25.q fails permanently

2018-01-14 Thread Laszlo Bodor (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325504#comment-16325504
 ] 

Laszlo Bodor commented on HIVE-18445:
-

Or we can completely turn of auto join for statements out of scope, the test 
case will test conditonal task and exhaustion anyway. The current problem is, 
sometimes there are repeated errors messages in q.out. Further investigation is 
in progress.

q file
{code}
set hive.auto.convert.join = true;
FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src src3 ON (src1.key 
+ src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value;
set hive.auto.convert.join = false;
{code}

q.out file
{code}
PREHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output: default@dest_j2
*Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
POSTHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j2
{code}

> qtests: auto_join25.q fails permanently
> ---
>
> Key: HIVE-18445
> URL: https://issues.apache.org/jira/browse/HIVE-18445
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] 
> (batchId=72)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HIVE-18445) qtests: auto_join25.q fails permanently

2018-01-14 Thread Laszlo Bodor (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325504#comment-16325504
 ] 

Laszlo Bodor edited comment on HIVE-18445 at 1/14/18 10:46 AM:
---

Or we can completely turn off auto join for statements out of scope, the test 
case will test conditonal task and exhaustion anyway. The current problem is, 
sometimes there are repeated errors messages in q.out. Further investigation is 
in progress.

q file
{code}
set hive.auto.convert.join = true;
FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src src3 ON (src1.key 
+ src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value;
set hive.auto.convert.join = false;
{code}

q.out file
{code}
PREHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output: default@dest_j2
Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
POSTHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j2
{code}


was (Author: abstractdog):
Or we can completely turn of auto join for statements out of scope, the test 
case will test conditonal task and exhaustion anyway. The current problem is, 
sometimes there are repeated errors messages in q.out. Further investigation is 
in progress.

q file
{code}
set hive.auto.convert.join = true;
FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src src3 ON (src1.key 
+ src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value;
set hive.auto.convert.join = false;
{code}

q.out file
{code}
PREHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output: default@dest_j2
*Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Hive Runtime Error: Map local work exhausted memory
FAILED: Execution Error, return code 3 from 
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
POSTHOOK: query: FROM src src1 JOIN src src2 ON (src1.key = src2.key) JOIN src 
src3 ON (src1.key + src2.key = src3.key)
INSERT OVERWRITE TABLE dest_j2 SELECT src1.key, src3.value
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j2
{code}

> qtests: auto_join25.q fails permanently
> ---
>
> Key: HIVE-18445
> URL: https://issues.apache.org/jira/browse/HIVE-18445
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] 
> (batchId=72)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HIVE-18341) Add repl load support for adding "raw" namespace for TDE with same encryption keys

[jira] [Commented] (HIVE-18341) Add repl load support for adding "raw" namespace for TDE with same encryption keys

[jira] [Updated] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID

[jira] [Commented] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID

[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths

[jira] [Commented] (HIVE-18061) q.outs: be more selective with masking hdfs paths

[jira] [Updated] (HIVE-18061) q.outs: be more selective with masking hdfs paths

[jira] [Commented] (HIVE-18445) qtests: auto_join25.q fails permanently

[jira] [Comment Edited] (HIVE-18445) qtests: auto_join25.q fails permanently

9 matches

Site Navigation

Mail list logo

Footer information