Hive jdbc client - result is NULL when I run a query to select a large of data
with runing mapreduce
----------------------------------------------------------------------------------------------------
Key: HIVE-943
URL: https://issues.apache.org/jira/browse/HIVE-943
Project: Hadoop Hive
Issue Type: Bug
Components: Clients
Affects Versions: 0.4.0
Reporter: Vu Hoang
- some main output messages i got from console:
Total MapReduce jobs = 1
09/11/18 15:56:03 INFO ql.Driver: Total MapReduce jobs = 1
09/11/18 15:56:03 INFO exec.ExecDriver: BytesPerReducer=1000000000
maxReducers=999 totalInputFileSize=1289288953
Number of reduce tasks not specified. Estimated from input data size: 2
09/11/18 15:56:03 INFO exec.ExecDriver: Number of reduce tasks not specified.
Estimated from input data size: 2
In order to change the average load for a reducer (in bytes):
09/11/18 15:56:03 INFO exec.ExecDriver: In order to change the average load for
a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
09/11/18 15:56:03 INFO exec.ExecDriver: set
hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
09/11/18 15:56:03 INFO exec.ExecDriver: In order to limit the maximum number of
reducers:
set hive.exec.reducers.max=<number>
09/11/18 15:56:03 INFO exec.ExecDriver: set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
09/11/18 15:56:03 INFO exec.ExecDriver: In order to set a constant number of
reducers:
set mapred.reduce.tasks=<number>
09/11/18 15:56:03 INFO exec.ExecDriver: set mapred.reduce.tasks=<number>
09/11/18 15:56:03 INFO exec.ExecDriver: Using
org.apache.hadoop.hive.ql.io.HiveInputFormat
Starting Job = job_200911122011_0639, Tracking URL =
http://tmp-harvester-1:50030/jobdetails.jsp?jobid=job_200911122011_0639
09/11/18 15:56:04 INFO exec.ExecDriver: Starting Job = job_200911122011_0639,
Tracking URL =
http://tmp-harvester-1:50030/jobdetails.jsp?jobid=job_200911122011_0639
Kill Command = /data/hadoop-hive/bin/../bin/hadoop job
-Dmapred.job.tracker=tmp-harvester-1:51031 -kill job_200911122011_0639
09/11/18 15:56:04 INFO exec.ExecDriver: Kill Command =
/data/hadoop-hive/bin/../bin/hadoop job
-Dmapred.job.tracker=tmp-harvester-1:51031 -kill job_200911122011_0639
2009-11-18 03:56:05,701 map = 0%, reduce = 0%
09/11/18 15:56:05 INFO exec.ExecDriver: 2009-11-18 03:56:05,701 map = 0%,
reduce = 0%
2009-11-18 03:56:21,798 map = 4%, reduce = 0%
09/11/18 15:56:21 INFO exec.ExecDriver: 2009-11-18 03:56:21,798 map = 4%,
reduce = 0%
2009-11-18 03:56:22,818 map = 8%, reduce = 0%
09/11/18 15:56:22 INFO exec.ExecDriver: 2009-11-18 03:56:22,818 map = 8%,
reduce = 0%
2009-11-18 03:56:23,832 map = 13%, reduce = 0%
09/11/18 15:56:23 INFO exec.ExecDriver: 2009-11-18 03:56:23,832 map = 13%,
reduce = 0%
2009-11-18 03:56:24,854 map = 17%, reduce = 0%
09/11/18 15:56:24 INFO exec.ExecDriver: 2009-11-18 03:56:24,854 map = 17%,
reduce = 0%
2009-11-18 03:56:25,864 map = 21%, reduce = 0%
09/11/18 15:56:25 INFO exec.ExecDriver: 2009-11-18 03:56:25,864 map = 21%,
reduce = 0%
2009-11-18 03:56:29,890 map = 25%, reduce = 0%
09/11/18 15:56:29 INFO exec.ExecDriver: 2009-11-18 03:56:29,890 map = 25%,
reduce = 0%
2009-11-18 03:56:30,900 map = 29%, reduce = 0%
09/11/18 15:56:30 INFO exec.ExecDriver: 2009-11-18 03:56:30,900 map = 29%,
reduce = 0%
2009-11-18 03:56:31,909 map = 33%, reduce = 0%
09/11/18 15:56:31 INFO exec.ExecDriver: 2009-11-18 03:56:31,909 map = 33%,
reduce = 0%
2009-11-18 03:56:33,933 map = 37%, reduce = 0%
09/11/18 15:56:33 INFO exec.ExecDriver: 2009-11-18 03:56:33,933 map = 37%,
reduce = 0%
2009-11-18 03:56:35,946 map = 50%, reduce = 0%
09/11/18 15:56:35 INFO exec.ExecDriver: 2009-11-18 03:56:35,946 map = 50%,
reduce = 0%
2009-11-18 03:56:36,956 map = 54%, reduce = 0%
09/11/18 15:56:36 INFO exec.ExecDriver: 2009-11-18 03:56:36,956 map = 54%,
reduce = 0%
2009-11-18 03:56:37,965 map = 58%, reduce = 0%
09/11/18 15:56:37 INFO exec.ExecDriver: 2009-11-18 03:56:37,965 map = 58%,
reduce = 0%
2009-11-18 03:56:38,978 map = 79%, reduce = 0%
09/11/18 15:56:38 INFO exec.ExecDriver: 2009-11-18 03:56:38,978 map = 79%,
reduce = 0%
2009-11-18 03:56:39,988 map = 83%, reduce = 0%
09/11/18 15:56:39 INFO exec.ExecDriver: 2009-11-18 03:56:39,988 map = 83%,
reduce = 0%
2009-11-18 03:56:40,998 map = 96%, reduce = 0%
09/11/18 15:56:41 INFO exec.ExecDriver: 2009-11-18 03:56:40,998 map = 96%,
reduce = 0%
2009-11-18 03:56:42,006 map = 100%, reduce = 0%
09/11/18 15:56:42 INFO exec.ExecDriver: 2009-11-18 03:56:42,006 map = 100%,
reduce = 0%
2009-11-18 03:56:46,031 map = 100%, reduce = 13%
09/11/18 15:56:46 INFO exec.ExecDriver: 2009-11-18 03:56:46,031 map = 100%,
reduce = 13%
2009-11-18 03:56:51,060 map = 100%, reduce = 25%
09/11/18 15:56:51 INFO exec.ExecDriver: 2009-11-18 03:56:51,060 map = 100%,
reduce = 25%
2009-11-18 03:56:56,091 map = 100%, reduce = 67%
09/11/18 15:56:56 INFO exec.ExecDriver: 2009-11-18 03:56:56,091 map = 100%,
reduce = 67%
2009-11-18 03:56:57,102 map = 100%, reduce = 100%
09/11/18 15:56:57 INFO exec.ExecDriver: 2009-11-18 03:56:57,102 map = 100%,
reduce = 100%
Ended Job = job_200911122011_0639
09/11/18 15:56:59 INFO exec.ExecDriver: Ended Job = job_200911122011_0639
09/11/18 15:56:59 INFO exec.FileSinkOperator: Moving tmp dir:
hdfs://tmp-harvester-1:51000/tmp/hive-asadm/1751400872/_tmp.10001 to:
hdfs://tmp-harvester-1:51000/tmp/hive-asadm/1751400872/_tmp.10001.intermediate
09/11/18 15:56:59 INFO exec.FileSinkOperator: Moving tmp dir:
hdfs://tmp-harvester-1:51000/tmp/hive-asadm/1751400872/_tmp.10001.intermediate
to: hdfs://tmp-harvester-1:51000/tmp/hive-asadm/1751400872/10001
OK
09/11/18 15:56:59 INFO ql.Driver: OK
09/11/18 15:56:59 INFO ql.Driver: Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:_col0, type:string, comment:from
deserializer)], properties:null)
09/11/18 15:56:59 INFO ql.Driver: Returning Thrift schema:
Schema(fieldSchemas:[FieldSchema(name:_col0, type:string, comment:from
deserializer)], properties:null)
09/11/18 15:56:59 INFO service.HiveServer: Returning schema:
Schema(fieldSchemas:[FieldSchema(name:_col0, type:string, comment:from
deserializer)], properties:null)
09/11/18 15:56:59 INFO mapred.FileInputFormat: Total input paths to process : 2
+-------+
| _col0 |
+-------+
| NULL |
+-------+
- this problem DOES NOT appear when mapreduce was not running ?
- i made something wrong at configuration from Hive jdbc api ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
