[
https://issues.apache.org/jira/browse/HIVE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516983#comment-14516983
]
Nemon Lou commented on HIVE-5672:
---------------------------------
The query plan for local directory is simple:
{quote}
explain insert overwrite local directory '/tmp/xxx' select * from src;
+-----------------------------------------------------------------------------------------------------------------------+
| Explain
|
+-----------------------------------------------------------------------------------------------------------------------+
| STAGE DEPENDENCIES:
|
| Stage-1 is a root stage
|
| Stage-0 depends on stages: Stage-1
|
|
|
| STAGE PLANS:
|
| Stage: Stage-1
|
| Map Reduce
|
| Map Operator Tree:
|
| TableScan
|
| alias: src
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats: COMPLETE
Column stats: NONE |
| Select Operator
|
| expressions: id (type: string), starttime (type: bigint),
callerno (type: string), note (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats: COMPLETE
Column stats: NONE |
| File Output Operator
|
| compressed: false
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats:
COMPLETE Column stats: NONE |
| table:
|
| input format: org.apache.hadoop.mapred.TextInputFormat
|
| output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
|
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
|
|
|
| Stage: Stage-0
|
| Move Operator
|
| files:
|
| hdfs directory: false
|
| destination: /tmp/xxx
|
|
|
+-----------------------------------------------------------------------------------------------------------------------+
29 rows selected (5.957 seconds)
{quote}
While the query plan for DFS directory is complicated：
{quote}
explain insert overwrite directory '/tmp/xxx' select * from src;
+-------------------------------------------------------------------------------------------------------------------------+
| Explain
|
+-------------------------------------------------------------------------------------------------------------------------+
| STAGE DEPENDENCIES:
|
| Stage-1 is a root stage
|
| Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2, Stage-4
|
| Stage-3
|
| Stage-0 depends on stages: Stage-3, Stage-2, Stage-5
|
| Stage-2
|
| Stage-4
|
| Stage-5 depends on stages: Stage-4
|
|
|
| STAGE PLANS:
|
| Stage: Stage-1
|
| Map Reduce
|
| Map Operator Tree:
|
| TableScan
|
| alias: src
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats: COMPLETE
Column stats: NONE |
| Select Operator
|
| expressions: id (type: string), starttime (type: bigint),
callerno (type: string), note (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats: COMPLETE
Column stats: NONE |
| File Output Operator
|
| compressed: false
|
| Statistics: Num rows: 38 Data size: 11900 Basic stats:
COMPLETE Column stats: NONE |
| table:
|
| input format: org.apache.hadoop.mapred.TextInputFormat
|
| output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
|
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
|
|
|
| Stage: Stage-6
|
| Conditional Operator
|
|
|
| Stage: Stage-3
|
| Move Operator
|
| files:
|
| hdfs directory: true
|
| destination:
hdfs://hacluster/tmp/hive-scratch/hive_2015-04-28_20-49-24_582_7448801537099184702-2/-ext-10000
|
|
|
| Stage: Stage-0
|
| Move Operator
|
| files:
|
| hdfs directory: true
|
| destination: /tmp/xxx
|
|
|
| Stage: Stage-2
|
| Map Reduce
|
| Map Operator Tree:
|
| TableScan
|
| File Output Operator
|
| compressed: false
|
| table:
|
| input format: org.apache.hadoop.mapred.TextInputFormat
|
| output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
|
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
|
|
|
| Stage: Stage-4
|
| Map Reduce
|
| Map Operator Tree:
|
| TableScan
|
| File Output Operator
|
| compressed: false
|
| table:
|
| input format: org.apache.hadoop.mapred.TextInputFormat
|
| output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
|
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
|
|
|
| Stage: Stage-5
|
| Move Operator
|
| files:
|
| hdfs directory: true
|
| destination:
hdfs://hacluster/tmp/hive-scratch/hive_2015-04-28_20-49-24_582_7448801537099184702-2/-ext-10000
|
|
|
+-------------------------------------------------------------------------------------------------------------------------+
71 rows selected (3.911 seconds)
{quote}
> Insert with custom separator not supported for non-local directory
> ------------------------------------------------------------------
>
> Key: HIVE-5672
> URL: https://issues.apache.org/jira/browse/HIVE-5672
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.12.0, 1.0.0
> Reporter: Romain Rigaux
> Assignee: Nemon Lou
> Attachments: HIVE-5672.1.patch, HIVE-5672.2.patch, HIVE-5672.3.patch,
> HIVE-5672.4.patch, HIVE-5672.5.patch, HIVE-5672.5.patch.tar.gz,
> HIVE-5672.6.patch, HIVE-5672.6.patch.tar.gz, HIVE-5672.7.patch.tar.gz
>
>
> https://issues.apache.org/jira/browse/HIVE-3682 is great but non local
> directory don't seem to be supported:
> {code}
> insert overwrite directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select description FROM sample_07
> {code}
> {code}
> Error while compiling statement: FAILED: ParseException line 2:0 cannot
> recognize input near 'row' 'format' 'delimited' in select clause
> {code}
> This works (with 'local'):
> {code}
> insert overwrite local directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select code, description FROM sample_07
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
