[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15754865#comment-15754865 ]
Navya Krishnappa edited comment on SPARK-18877 at 3/30/17 12:44 PM: -------------------------------------------------------------------- I'm using SparkContext.read() to read the content. Refer the given code using to read the csv file. Dataset dataset = getSqlContext().read() .option(HEADER, "true") .option(PARSER_LIB, "commons") .option(INFER_SCHEMA, "true") .option(DELIMITER, ",") .option(QUOTE, "\"") .option(ESCAPE, "\\") .option(MODE, Mode.PERMISSIVE) .csv(sourceFile); if we collect the dataset (dataset.collect()). i will get java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20 exception. was (Author: navya krishnappa): I'm using SparkContext.read() to read the content. Refer the given code using to read the csv file. Dataset dataset = getSqlContext().read() .option(DAWBConstant.HEADER, "true") .option(DAWBConstant.PARSER_LIB, "commons") .option(DAWBConstant.INFER_SCHEMA, "true") .option(DAWBConstant.DELIMITER, ",") .option(DAWBConstant.QUOTE, "\"") .option(DAWBConstant.ESCAPE, "\\") .option(DAWBConstant.MODE, Mode.PERMISSIVE) .csv(sourceFile); if we collect the dataset (dataset.collect()). i will get java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20 exception. > Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: > requirement failed: Decimal precision 28 exceeds max precision 20 > ---------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-18877 > URL: https://issues.apache.org/jira/browse/SPARK-18877 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.2 > Reporter: Navya Krishnappa > Assignee: Dongjoon Hyun > Fix For: 2.0.3, 2.1.1, 2.2.0 > > > When reading below mentioned csv data, even though the maximum decimal > precision is 38, following exception is thrown > java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 > exceeds max precision 20 > Decimal > 2323366225312000000000000000 > 24335739714000000 > 23233662253000 > 232336622530000 -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org