[
https://issues.apache.org/jira/browse/HIVE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512520#comment-14512520
]
Xuefu Zhang commented on HIVE-5672:
-----------------------------------
[~nemon], I think you need to change semantic analyzer in addition to the
grammar in order to make it work. Since the grammar changes, the AST generated
will also change. Thus, semantic analyzer that processes the AST will need to
change accordingly in order to get the correct information such as hdfs vs
local.
Specifically, you need to change the following code block:
{code}
qb.getMetaData().setDestForAlias(name, fname,
(ast.getToken().getType() == HiveParser.TOK_DIR));
CreateTableDesc localDirectoryDesc = new CreateTableDesc();
boolean localDirectoryDescIsSet = false;
int numCh = ast.getChildCount();
for (int num = 1; num < numCh ; num++){
ASTNode child = (ASTNode) ast.getChild(num);
if (child != null) {
if (storageFormat.fillStorageFormat(child)) {
localDirectoryDesc.setOutputFormat(storageFormat.getOutputFormat());
localDirectoryDesc.setSerName(storageFormat.getSerde());
localDirectoryDescIsSet = true;
continue;
}
switch (child.getToken().getType()) {
case HiveParser.TOK_TABLEROWFORMAT:
rowFormatParams.analyzeRowFormat(child);
localDirectoryDesc.setFieldDelim(rowFormatParams.fieldDelim);
localDirectoryDesc.setLineDelim(rowFormatParams.lineDelim);
localDirectoryDesc.setCollItemDelim(rowFormatParams.collItemDelim);
localDirectoryDesc.setMapKeyDelim(rowFormatParams.mapKeyDelim);
localDirectoryDesc.setFieldEscape(rowFormatParams.fieldEscape);
localDirectoryDesc.setNullFormat(rowFormatParams.nullFormat);
localDirectoryDescIsSet=true;
break;
case HiveParser.TOK_TABLESERIALIZER:
ASTNode serdeChild = (ASTNode) child.getChild(0);
storageFormat.setSerde(unescapeSQLString(serdeChild.getChild(0).getText()));
localDirectoryDesc.setSerName(storageFormat.getSerde());
localDirectoryDescIsSet=true;
break;
}
}
}
{code}
You need to remove this line:
{code}
qb.getMetaData().setDestForAlias(name, fname,
(ast.getToken().getType() == HiveParser.TOK_DIR));
{code}
and you need to change the for loop after that because child count is
different. The child at index 1 tells whether directory is local, and children
at index 2 and 3 tells you the output format and storage serde.
> Insert with custom separator not supported for non-local directory
> ------------------------------------------------------------------
>
> Key: HIVE-5672
> URL: https://issues.apache.org/jira/browse/HIVE-5672
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.12.0, 1.0.0
> Reporter: Romain Rigaux
> Assignee: Nemon Lou
> Attachments: HIVE-5672.1.patch, HIVE-5672.2.patch, HIVE-5672.3.patch,
> HIVE-5672.4.patch, HIVE-5672.5.patch, HIVE-5672.5.patch.tar.gz,
> HIVE-5672.6.patch, HIVE-5672.6.patch.tar.gz
>
>
> https://issues.apache.org/jira/browse/HIVE-3682 is great but non local
> directory don't seem to be supported:
> {code}
> insert overwrite directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select description FROM sample_07
> {code}
> {code}
> Error while compiling statement: FAILED: ParseException line 2:0 cannot
> recognize input near 'row' 'format' 'delimited' in select clause
> {code}
> This works (with 'local'):
> {code}
> insert overwrite local directory '/tmp/test-02'
> row format delimited
> FIELDS TERMINATED BY ':'
> select code, description FROM sample_07
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
