[
https://issues.apache.org/jira/browse/KYLIN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281003#comment-16281003
]
Vsevolod Ostapenko commented on KYLIN-3070:
-------------------------------------------
I made a fix and tested it on my copy of the master branch.
My version of the fix introduces two new parameters in the kylin.properties:
* kylin.source.hive.flat-table-storage-format, which defaults to SEQUENCEFILE
* kylin.source.hive.flat-table-field-delimiter, which defaults to \u001F (Unit
separator, the same default field separator that Hive uses)
I tested my changes internally and confirmed that they are working as expected.
Btw, while making the change I found a problem with existing handling of the
TEXTFILE field separators - namely, the value was always fetched from
kylin.source.jdbc.field-delimiter (apparently a kludge), which technically has
no direct relations to flat table, so introduction of the
kylin.source.hive.flat-table-field-delimiter seems warranted.
If you don't have changes ready, please reassign this JIRA ticket to me.
> Add a config property for flat table storage format
> ---------------------------------------------------
>
> Key: KYLIN-3070
> URL: https://issues.apache.org/jira/browse/KYLIN-3070
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: v2.2.0
> Environment: HDP 2.5.6, Kylin 2.2.0
> Reporter: Vsevolod Ostapenko
> Assignee: Rong H
> Priority: Minor
> Labels: newbie
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Flat table storage format is currently hard-coded as SEQUENCEFILE in the
> core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
> That prevents using Impala as a SQL engine while using beeline CLI (via
> custom JDBC URL), as Impala cannot write sequence files.
> Adding a parameter to kylin.properties to override the default setting would
> address the issue.
> Removing a hard-coded value for storage format might be good idea in and on
> itself.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
