tianli created CARBONDATA-833:
---------------------------------
Summary: load data from dataframe,generater data row may be error
when delimiterLevel1 or delimiterLevel2 is special character
Key: CARBONDATA-833
URL: https://issues.apache.org/jira/browse/CARBONDATA-833
Project: CarbonData
Issue Type: Bug
Components: spark-integration
Affects Versions: 1.0.0-incubating, 1.1.0-incubating
Reporter: tianli
Assignee: tianli
Fix For: 1.1.0-incubating, 1.0.0-incubating
load data from dataframe,generater data row may be error by delimiterLevel1 or
delimiterLevel2 is special character
because delimiterLevel1 and delimiterLevel2 when carbonLoadModel is create by
CarbonUtil.delimiterConverter(), CarbonScalaUtil.getString direct use
carbonLoadModel.getComplexDelimiterLevel1 and
carbonLoadModel.getComplexDelimiterLevel2
val delimiter = if (level == 1) {
delimiterLevel1
} else {
delimiterLevel2
}
val builder = new StringBuilder()
s.foreach { x =>
builder.append(getString(x, serializationNullFormat,
delimiterLevel1,
delimiterLevel2, timeStampFormat, dateFormat, level +
1)).append(delimiter)
}
make primitive data added a more char \ when datatype is complex
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
