[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793280#comment-13793280 ] jeff little commented on HIVE-5245: --- Hi, Yin Huai. What is going on next step Can you try the trunk? I deem that the middle join results of join operator may not be saved and not be written to the temp HDFS. In other words, it may be failure, like 'Stage-7 is filtered out by condition resolver'. The other problem that we encountered recently is like below: hive (test) select a.* from test_01 a join (select b.id from test_02 b join test_03 c on (b.id =c.id)) d on (a.id=d.id); Total MapReduce jobs = 4 setting HADOOP_USER_NAMEhadoop Execution log at: /tmp/hadoop/.log 2013-10-12 02:36:42 Starting to launch local task to process map join; maximum memory = 932118528 2013-10-12 02:36:43 Processing rows:4 Hashtable size: 4 Memory usage: 110930744 rate: 0.119 2013-10-12 02:36:43 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-10-12_14-36-40_657_1301190087196742169/-local-10011/HashTable-Stage-9/MapJoin-mapfile41--.hashtable 2013-10-12 02:36:43 Upload 1 File to: file:/tmp/hadoop/hive_2013-10-12_14-36-40_657_1301190087196742169/-local-10011/HashTable-Stage-9/MapJoin-mapfile41--.hashtable File size: 444 2013-10-12 02:36:43 End of local task; Time Taken: 0.413 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 4 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201308241420_4028, Tracking URL = http://namenode:50030/jobdetails.jsp?jobid=job_201308241420_4028 Kill Command = /home/hadoop/package/hadoop-1.0.4/libexec/../bin/hadoop job -kill job_201308241420_4028 Hadoop job information for Stage-9: number of mappers: 2; number of reducers: 0 2013-10-12 14:36:58,185 Stage-9 map = 0%, reduce = 0% 2013-10-12 14:37:04,207 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:05,213 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:06,218 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:07,223 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:08,228 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:09,232 Stage-9 map = 100%, reduce = 0%, Cumulative CPU 2.66 sec 2013-10-12 14:37:10,237 Stage-9 map = 100%, reduce = 100%, Cumulative CPU 2.66 sec MapReduce Total cumulative CPU time: 2 seconds 660 msec Ended Job = job_201308241420_4028 Stage-12 is filtered out by condition resolver. MapReduce Jobs Launched: Job 0: Map: 2 Cumulative CPU: 2.66 sec HDFS Read: 822 HDFS Write: 2190 SUCCESS Total MapReduce CPU Time Spent: 2 seconds 660 msec OK Time taken: 29.662 seconds hive (test) Note： the table of test_01, test_02 and test_03 have data, and have the same values of id, but we can't get results. Inversely, it returns nothing. The problem may also be caused by “Stage-12 is filtered out by condition resolver”. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793439#comment-13793439 ] Yin Huai commented on HIVE-5245: I meant you can try hive trunk and see if the error also exist. If the error also exist, we need to find a way to reproduce it. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_09; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792648#comment-13792648 ] Yin Huai commented on HIVE-5245: I think that error log is fine. Because you have a CTAS query and test_10 did not exist in your db, so when we first tried to get the table for test_10, we would not be able to get the corresponding object. Seems stages for move operator and create table operator were not executed. Can you try the trunk? hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_09; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791333#comment-13791333 ] jeff little commented on HIVE-5245: --- hive (test) create table test_10 as select a.* from test_01 a join test_02 b on (a.id=b.id); Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop Execution log at: /tmp/hadoop/.log 2013-10-10 04:53:51 Starting to launch local task to process map join; maximum memory = 932118528 2013-10-10 04:53:51 Processing rows:6 Hashtable size: 6 Memory usage: 110948712 rate: 0.119 2013-10-10 04:53:51 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-10-10_16-53-48_503_1486914121771638453/-local-10005/HashTable-Stage-6/MapJoin-mapfile10--.hashtable 2013-10-10 04:53:51 Upload 1 File to: file:/tmp/hadoop/hive_2013-10-10_16-53-48_503_1486914121771638453/-local-10005/HashTable-Stage-6/MapJoin-mapfile10--.hashtable File size: 692 2013-10-10 04:53:51 End of local task; Time Taken: 0.413 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201308241420_3453, Tracking URL = http://namenode:50030/jobdetails.jsp?jobid=job_201308241420_3453 Kill Command = /home/hadoop/package/hadoop-1.0.4/libexec/../bin/hadoop job -kill job_201308241420_3453 Hadoop job information for Stage-6: number of mappers: 2; number of reducers: 0 2013-10-10 16:54:03,981 Stage-6 map = 0%, reduce = 0% 2013-10-10 16:54:10,015 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:11,022 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:12,029 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:13,035 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:14,041 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:15,047 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 2.82 sec 2013-10-10 16:54:16,053 Stage-6 map = 100%, reduce = 100%, Cumulative CPU 2.82 sec MapReduce Total cumulative CPU time: 2 seconds 820 msec Ended Job = job_201308241420_3453 Stage-7 is filtered out by condition resolver. 24 Rows loaded to hdfs://namenode:9000/tmp/hive-hadoop/hive_2013-10-10_16-53-48_503_1486914121771638453/-ext-1 MapReduce Jobs Launched: Job 0: Map: 2 Cumulative CPU: 2.82 sec HDFS Read: 822 HDFS Write: 452 SUCCESS Total MapReduce CPU Time Spent: 2 seconds 820 msec OK Time taken: 27.598 seconds hive (test) select * from test_10; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_10' hive (test) why??? hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[
https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791546#comment-13791546
]
Yin Huai commented on HIVE-5245:
[~jeff_little] I tried both trunk and 0.11 with tables used in our unit tests
(src and src1) with the query
{code:sql}
create table test_10 as
select a.* from src a join src1 b on (a.key=b.key);
{code}
I did not see the error in your post.
Since Stage-7 is filtered out by condition resolver. appears in your log,
seems hive.auto.convert.join.noconditionaltask was false in your test. Can you
post the results of EXPLAIN with hive.auto.convert.join.noconditionaltask=true
and hive.auto.convert.join.noconditionaltask=false?
Because it is a CTAS query, in the query plan, we should see two extra stages
besides stages for the select query, one for Move Operator and another for
Create Table Operator.
hive create table as select(CTAS) can not work(not support) with join on
operator
-
Key: HIVE-5245
URL: https://issues.apache.org/jira/browse/HIVE-5245
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0
Reporter: jeff little
Labels: CTAS, hive
Original Estimate: 96h
Remaining Estimate: 96h
hello everyone, recently i came across one hive problem as below:
hive (test) create table test_09 as
select a.* from test_01 a
join test_02 b
on (a.id=b.id);
Automatically selecting local only mode for query
Total MapReduce jobs = 2
setting HADOOP_USER_NAMEhadoop
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
2013-09-09 05:22:36 Starting to launch local task to process map join;
maximum memory = 932118528
2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4
Memory usage: 113068056 rate: 0.121
2013-09-09 05:22:37 Dump the hashtable into file:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
2013-09-09 05:22:37 Upload 1 File to:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
File size: 788
2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec.
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Mapred Local Task Succeeded . Convert the Join into MapJoin
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2013-09-09 17:22:41,807 null map = 0%, reduce = 0%
2013-09-09 17:22:44,814 null map = 100%, reduce = 0%
Ended Job = job_local_0001
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Stage-7 is filtered out by condition resolver.
OK
Time taken: 13.138 seconds
hive (test) select * from test_09;
FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09'
hive (test)
Problem:
I can't get the created table, namely this CTAS is nonavailable, and this
table is not created by this hql sentence at all.who can explain for
me.Thanks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[
https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791563#comment-13791563
]
jeff little commented on HIVE-5245:
---
hive (test) set hive.auto.convert.join.noconditionaltask=true;
hive (test) explain
create table test_10 as
select a.* from test_01 a
join test_02 b
on (a.id=b.id);
OK
ABSTRACT SYNTAX TREE:
(TOK_CREATETABLE (TOK_TABNAME test_10) TOK_LIKETABLE (TOK_QUERY (TOK_FROM
(TOK_JOIN (TOK_TABREF (TOK_TABNAME test_01) a) (TOK_TABREF (TOK_TABNAME
test_02) b) (= (. (TOK_TABLE_OR_COL a) id) (. (TOK_TABLE_OR_COL b) id
(TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR
(TOK_ALLCOLREF (TOK_TABNAME a)))
STAGE DEPENDENCIES:
Stage-8 is a root stage
Stage-6 depends on stages: Stage-8
Stage-5 depends on stages: Stage-6 , consists of Stage-7
Stage-7
Stage-4 depends on stages: Stage-7
Stage-0 depends on stages: Stage-1, Stage-4
Stage-9 depends on stages: Stage-0
Stage-2 depends on stages: Stage-9
STAGE PLANS:
Stage: Stage-8
Map Reduce Local Work
Alias - Map Local Tables:
a
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
a
TableScan
alias: a
HashTable Sink Operator
condition expressions:
0 {id} {name} {sex} {record_day}
1
handleSkewJoin: false
keys:
0 [Column[id]]
1 [Column[id]]
Position of Big Table: 1
Stage: Stage-6
Map Reduce
Alias - Map Operator Tree:
b
TableScan
alias: b
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {id} {name} {sex} {record_day}
1
handleSkewJoin: false
keys:
0 [Column[id]]
1 [Column[id]]
outputColumnNames: _col0, _col1, _col2, _col3
Position of Big Table: 1
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: string
expr: _col2
type: string
expr: _col3
type: string
outputColumnNames: _col0, _col1, _col2, _col3
File Output Operator
compressed: false
GlobalTableId: 1
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
name: test.test_10
Local Work:
Map Reduce Local Work
Stage: Stage-5
Conditional Operator
Stage: Stage-7
Map Reduce Local Work
Alias - Map Local Tables:
1
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
1
HashTable Sink Operator
condition expressions:
0 {0_VALUE_0} {0_VALUE_1} {0_VALUE_2} {0_VALUE_3}
1
handleSkewJoin: false
keys:
0 [Column[joinkey0]]
1 [Column[joinkey0]]
Position of Big Table: 0
Stage: Stage-4
Map Reduce
Alias - Map Operator Tree:
0
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {0_VALUE_0} {0_VALUE_1} {0_VALUE_2} {0_VALUE_3}
1
handleSkewJoin: false
keys:
0 [Column[joinkey0]]
1 [Column[joinkey0]]
outputColumnNames: _col0, _col1, _col2, _col3
Position of Big Table: 0
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: string
expr: _col2
type: string
expr: _col3
type: string
outputColumnNames: _col0, _col1, _col2, _col3
File Output Operator
compressed: false
GlobalTableId: 1
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
name: test.test_10
Local Work:
Map Reduce Local Work
Stage: Stage-0
Move Operator
files:
hdfs directory: true
destination:
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791577#comment-13791577 ] jeff little commented on HIVE-5245: --- And, what the mean is filtered out by condition resolver. I think that is filtered out by condition resolver causing built table failure. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_09; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[
https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791578#comment-13791578
]
Yin Huai commented on HIVE-5245:
Can you check your log and see you can find something like (after the line of
INFO exec.ExecDriver: Execution completed successfully) ...
{code}
13/10/10 11:10:23 INFO exec.Task: Moving data to: ...
13/10/10 11:10:23 INFO exec.DDLTask: Default to LazySimpleSerDe for table
test_10
13/10/10 11:10:23 INFO metastore.HiveMetaStore: 0: create_table:
Table(tableName:test_10, ...
{code}
You can direct the log to you console by using
{code}
bin/hive -hiveconf hive.root.logger=INFO,console
{code}
What version of hive are you using? 0.11?
hive create table as select(CTAS) can not work(not support) with join on
operator
-
Key: HIVE-5245
URL: https://issues.apache.org/jira/browse/HIVE-5245
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0
Reporter: jeff little
Labels: CTAS, hive
Original Estimate: 96h
Remaining Estimate: 96h
hello everyone, recently i came across one hive problem as below:
hive (test) create table test_09 as
select a.* from test_01 a
join test_02 b
on (a.id=b.id);
Automatically selecting local only mode for query
Total MapReduce jobs = 2
setting HADOOP_USER_NAMEhadoop
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
2013-09-09 05:22:36 Starting to launch local task to process map join;
maximum memory = 932118528
2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4
Memory usage: 113068056 rate: 0.121
2013-09-09 05:22:37 Dump the hashtable into file:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
2013-09-09 05:22:37 Upload 1 File to:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
File size: 788
2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec.
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Mapred Local Task Succeeded . Convert the Join into MapJoin
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2013-09-09 17:22:41,807 null map = 0%, reduce = 0%
2013-09-09 17:22:44,814 null map = 100%, reduce = 0%
Ended Job = job_local_0001
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Stage-7 is filtered out by condition resolver.
OK
Time taken: 13.138 seconds
hive (test) select * from test_09;
FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09'
hive (test)
Problem:
I can't get the created table, namely this CTAS is nonavailable, and this
table is not created by this hql sentence at all.who can explain for
me.Thanks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791589#comment-13791589 ] jeff little commented on HIVE-5245: --- Task Logs: 'attempt_201308241420_3665_m_00_0' syslog logs 2013-10-10 23:25:14,771 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2013-10-10 23:25:14,841 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/distcache/-6415303628198910415_1127677245_586502988/namenode/tmp/hive-hadoop/hive_2013-10-10_23-21-25_380_3054038040192061682/-mr-10010/319df19a-bb7f-44d8-8bc5-e2cc1b034bb2 - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/HIVE_PLAN319df19a-bb7f-44d8-8bc5-e2cc1b034bb2 2013-10-10 23:25:14,846 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/org - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/org 2013-10-10 23:25:14,847 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/javolution - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/javolution 2013-10-10 23:25:14,847 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/META-INF - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/META-INF 2013-10-10 23:25:14,848 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/.job.jar.crc - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/.job.jar.crc 2013-10-10 23:25:14,849 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/javaewah - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/javaewah 2013-10-10 23:25:14,850 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/com - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/com 2013-10-10 23:25:14,850 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/hive-exec-log4j.properties - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/hive-exec-log4j.properties 2013-10-10 23:25:14,851 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/javax - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/javax 2013-10-10 23:25:14,852 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/jars/job.jar - /home/hadoop/hdfs/mapred/local/taskTracker/hadoop/jobcache/job_201308241420_3665/attempt_201308241420_3665_m_00_0/work/job.jar 2013-10-10 23:25:15,089 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-10-10 23:25:15,091 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@28bb494b 2013-10-10 23:25:15,493 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library 2013-10-10 23:25:15,495 INFO com.hadoop.compression.lzo.LzoCodec: Successfully loaded initialized native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8] 2013-10-10 23:25:15,503 INFO org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader: Processing file hdfs://namenode:9000/user/hive/warehouse/test.db/test_02/record_day=20130813/test_02.txt 2013-10-10 23:25:15,503 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-10-10 23:25:15,511 INFO ExecMapper: maximum memory = 200998912 2013-10-10 23:25:15,511 INFO ExecMapper: conf classpath = [file:/home/hadoop/package/hadoop-1.0.4/conf.cluster/,
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[
https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791604#comment-13791604
]
Yin Huai commented on HIVE-5245:
Seems those logs are at runtime from Map tasks. I meant the log from the Hive
driver.
Can you use
{code}
bin/hive -hiveconf hive.root.logger=INFO,console
{code}
hive create table as select(CTAS) can not work(not support) with join on
operator
-
Key: HIVE-5245
URL: https://issues.apache.org/jira/browse/HIVE-5245
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.11.0
Reporter: jeff little
Labels: CTAS, hive
Original Estimate: 96h
Remaining Estimate: 96h
hello everyone, recently i came across one hive problem as below:
hive (test) create table test_09 as
select a.* from test_01 a
join test_02 b
on (a.id=b.id);
Automatically selecting local only mode for query
Total MapReduce jobs = 2
setting HADOOP_USER_NAMEhadoop
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:36 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
2013-09-09 05:22:36 Starting to launch local task to process map join;
maximum memory = 932118528
2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4
Memory usage: 113068056 rate: 0.121
2013-09-09 05:22:37 Dump the hashtable into file:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
2013-09-09 05:22:37 Upload 1 File to:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
File size: 788
2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec.
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Mapred Local Task Succeeded . Convert the Join into MapJoin
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.system.dir; Ignoring.
13/09/09 17:22:38 WARN conf.Configuration:
file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
Execution log at: /tmp/hadoop/.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2013-09-09 17:22:41,807 null map = 0%, reduce = 0%
2013-09-09 17:22:44,814 null map = 100%, reduce = 0%
Ended Job = job_local_0001
Execution completed successfully
Mapred Local Task Succeeded . Convert the Join into MapJoin
Stage-7 is filtered out by condition resolver.
OK
Time taken: 13.138 seconds
hive (test) select * from test_09;
FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09'
hive (test)
Problem:
I can't get the created table, namely this CTAS is nonavailable, and this
table is not created by this hql sentence at all.who can explain for
me.Thanks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792214#comment-13792214 ] jeff little commented on HIVE-5245: --- hive (test) create table test_10 as select a.* from test_01 a join test_02 b on (a.id=b.id); 13/10/11 09:17:16 INFO ql.Driver: PERFLOG method=Driver.run 13/10/11 09:17:16 INFO ql.Driver: PERFLOG method=TimeToSubmit 13/10/11 09:17:16 INFO ql.Driver: PERFLOG method=compile 13/10/11 09:17:17 INFO parse.ParseDriver: Parsing command: create table test_10 as select a.* from test_01 a join test_02 b on (a.id=b.id) 13/10/11 09:17:17 INFO parse.ParseDriver: Parse Completed 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Starting Semantic Analysis 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Creating table test_10 position=13 13/10/11 09:17:17 INFO metastore.HiveMetaStore: 0: get_database: test 13/10/11 09:17:17 INFO HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_database: test 13/10/11 09:17:17 INFO metastore.HiveMetaStore: 0: get_table : db=test tbl=test_10 13/10/11 09:17:17 INFO HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table : db=test tbl=test_10 13/10/11 09:17:17 ERROR metastore.RetryingHMSHandler: NoSuchObjectException(message:test.test_10 table not found) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1369) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102) at $Proxy10.get_table(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:838) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74) at $Proxy11.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:948) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:9385) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8647) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic Analysis 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Get metadata for source tables 13/10/11 09:17:17 INFO metastore.HiveMetaStore: 0: get_table : db=test tbl=test_02 13/10/11 09:17:17 INFO HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table : db=test tbl=test_02 13/10/11 09:17:17 INFO metastore.HiveMetaStore: 0: get_table : db=test tbl=test_01 13/10/11 09:17:17 INFO HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table : db=test tbl=test_01 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Get metadata for subqueries 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Get metadata for destination tables 13/10/11 09:17:17 INFO metastore.HiveMetaStore: 0: get_database: test 13/10/11 09:17:17 INFO HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_database: test 13/10/11 09:17:17 INFO parse.SemanticAnalyzer: Completed getting MetaData in Semantic Analysis 13/10/11 09:17:17 WARN parse.TypeCheckProcFactory: Invalid type entry
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13792252#comment-13792252 ] jeff little commented on HIVE-5245: --- ERROR metastore.RetryingHMSHandler: NoSuchObjectException(message:test.test_10 table not found), that may be the key factor of this problem. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_09; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13789227#comment-13789227 ] Yin Huai commented on HIVE-5245: [~jeff_little] In the description of this jira, your log shows that you created a table called test_09 and then you queried a table test_08. Since test_08 had not been created, you got FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_08'. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_08; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_08' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762819#comment-13762819 ] Navis commented on HIVE-5245: - Cannot reproduce in trunk. hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Fix For: 0.11.0 Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_08; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_08' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762821#comment-13762821 ] jeff little commented on HIVE-5245: --- what the mean Cannot reproduce in trunk.? Could you explain detailedly. I suppose that Hive.0.11.0 can not support the join on operator for CTAS. But it is available for left outer join right outer join! hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Fix For: 0.11.0 Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_08; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_08' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762823#comment-13762823 ] jeff little commented on HIVE-5245: --- hive (test) create table test_09 as select a.* from test_01 a left outer join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 1 setting HADOOP_USER_NAMEhadoop 13/09/10 15:44:32 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10005/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/10 15:44:32 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10005/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-10 03:44:32 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-10 03:44:33 Processing rows:4 Hashtable size: 4 Memory usage: 110986304 rate: 0.119 2013-09-10 03:44:33 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10002/HashTable-Stage-4/MapJoin-mapfile01--.hashtable 2013-09-10 03:44:33 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10002/HashTable-Stage-4/MapJoin-mapfile01--.hashtable File size: 444 2013-09-10 03:44:33 End of local task; Time Taken: 0.562 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/10 15:44:35 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10006/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/10 15:44:35 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-10_15-44-30_620_3609535307275149310/-local-10006/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-10 15:44:38,793 null map = 0%, reduce = 0% 2013-09-10 15:44:41,800 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Moving data to: hdfs://namenode:9000/user/hive/warehouse/test.db/test_09 Table test.test_09 stats: [num_partitions: 0, num_files: 2, num_rows: 26, total_size: 491, raw_data_size: 465] OK Time taken: 14.86 seconds hive (test) select * from test_09; OK 11 jefff 20130812 11 jefff 20130812 11 jefff 20130812 11 jefff 20130812 12 smith f 20130812 12 smith f 20130812 12 smith f 20130812 12 smith f 20130812 13 alexf 20130812 13 alexf 20130812 13 alexf 20130812 13 alexf 20130812 14 tom f 20130812 14 tom f 20130812 14 tom f 20130812 14 tom f 20130812 11 jefff 20130813 11 jefff 20130813 11 jefff 20130813 11 jefff 20130813 17 smith f 20130813 18 alexf 20130813 14 tom f 20130813 14 tom f 20130813 14 tom f 20130813 14 tom f 20130813 Time taken: 0.071 seconds, Fetched: 26 row(s) Problem: why? the left outer join operator for CTAS, otherwise join not! hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Fix For: 0.11.0 Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration:
[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator
[ https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762832#comment-13762832 ] jeff little commented on HIVE-5245: --- that is why? hive create table as select(CTAS) can not work(not support) with join on operator - Key: HIVE-5245 URL: https://issues.apache.org/jira/browse/HIVE-5245 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.11.0 Reporter: jeff little Labels: CTAS, hive Fix For: 0.11.0 Original Estimate: 96h Remaining Estimate: 96h hello everyone, recently i came across one hive problem as below: hive (test) create table test_09 as select a.* from test_01 a join test_02 b on (a.id=b.id); Automatically selecting local only mode for query Total MapReduce jobs = 2 setting HADOOP_USER_NAMEhadoop 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:36 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log 2013-09-09 05:22:36 Starting to launch local task to process map join; maximum memory = 932118528 2013-09-09 05:22:37 Processing rows:4 Hashtable size: 4 Memory usage: 113068056 rate: 0.121 2013-09-09 05:22:37 Dump the hashtable into file: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable 2013-09-09 05:22:37 Upload 1 File to: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable File size: 788 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec. Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Mapred Local Task Succeeded . Convert the Join into MapJoin Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.system.dir; Ignoring. 13/09/09 17:22:38 WARN conf.Configuration: file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. Execution log at: /tmp/hadoop/.log Job running in-process (local Hadoop) Hadoop job information for null: number of mappers: 0; number of reducers: 0 2013-09-09 17:22:41,807 null map = 0%, reduce = 0% 2013-09-09 17:22:44,814 null map = 100%, reduce = 0% Ended Job = job_local_0001 Execution completed successfully Mapred Local Task Succeeded . Convert the Join into MapJoin Stage-7 is filtered out by condition resolver. OK Time taken: 13.138 seconds hive (test) select * from test_08; FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_08' hive (test) Problem: I can't get the created table, namely this CTAS is nonavailable, and this table is not created by this hql sentence at all.who can explain for me.Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
