[jira] [Commented] (HIVE-5672) Insert with custom separator not supported for non-local directory

[Nemon Lou (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HIVE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512316#comment-14512316
 ] 

Nemon Lou commented on HIVE-5672:
---------------------------------

I have tried both Sushanth's and  Xuefu 's  solution of simplified grammar 
,they both treated local directory wrongly as DFS directory.
Some analysis from SemanticAnalyzer.java：
The genFileSinkPlan method use dest_type from QBMetaData to determine the 
destination of final output.
{code}
    QBMetaData qbm = qb.getMetaData();
    Integer dest_type = qbm.getDestTypeForAlias(dest);
    switch (dest_type.intValue()) {
    case QBMetaData.DEST_TABLE: {
    ...  
    }
    case QBMetaData.DEST_PARTITION:{
   ... 
   }
case QBMetaData.DEST_LOCAL_FILE:
      isLocal = true;
      // fall through
    case QBMetaData.DEST_DFS_FILE: {
      ...
      if (isLocal) {
        ...
      } else {
        ...
    }
 default:
      throw new SemanticException("Unknown destination type: " + dest_type);
{code}

The dest_type from QBMetaData is determined by this method :
{code}
  qb.getMetaData().setDestForAlias(name, fname,
              (ast.getToken().getType() == HiveParser.TOK_DIR));
{code}

The code for setDestForAlias is this :
{code}
   public void setDestForAlias(String alias, String fname, boolean isDfsFile) {
    nameToDestType.put(alias, isDfsFile ? Integer.valueOf(DEST_DFS_FILE)
        : Integer.valueOf(DEST_LOCAL_FILE));
    nameToDestFile.put(alias, fname);
  }
{code}
The simplified grammar proposed by  Sushanth or Xuefu can not tell which kind 
of directory is used (DEST_DFS_FILE or DEST_LOCAL_FILE).
Maybe i understood something wrong,please correct me. :)

> Insert with custom separator not supported for non-local directory
> ------------------------------------------------------------------
>
>                 Key: HIVE-5672
>                 URL: https://issues.apache.org/jira/browse/HIVE-5672
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.12.0, 1.0.0
>            Reporter: Romain Rigaux
>            Assignee: Nemon Lou
>         Attachments: HIVE-5672.1.patch, HIVE-5672.2.patch, HIVE-5672.3.patch, 
> HIVE-5672.4.patch, HIVE-5672.5.patch, HIVE-5672.5.patch.tar.gz, 
> HIVE-5672.6.patch, HIVE-5672.6.patch.tar.gz
>
>
> https://issues.apache.org/jira/browse/HIVE-3682 is great but non local 
> directory don't seem to be supported:
> {code}
> insert overwrite directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select description FROM sample_07
> {code}
> {code}
> Error while compiling statement: FAILED: ParseException line 2:0 cannot 
> recognize input near 'row' 'format' 'delimited' in select clause
> {code}
> This works (with 'local'):
> {code}
> insert overwrite local directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select code, description FROM sample_07
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)