[ https://issues.apache.org/jira/browse/SPARK-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581357#comment-15581357 ]
Jianfei Wang commented on SPARK-17969: -------------------------------------- I can do this mini job. thank you! > I think it's user unfriendly to process standard json file with DataFrame > -------------------------------------------------------------------------- > > Key: SPARK-17969 > URL: https://issues.apache.org/jira/browse/SPARK-17969 > Project: Spark > Issue Type: New Feature > Components: SQL > Affects Versions: 2.0.1 > Reporter: Jianfei Wang > Priority: Minor > > Currently, with DataFrame API, we can't load standard json file directly, > maybe we can provide an override method to process this, the logic is as > below: > ``` > val df = spark.sparkContext.wholeTextFiles("data/test.json") > val json_rdd = df.map( x => x.toString.replaceAll("\\s+","")).map{ x => > val index = x.indexOf(',') > x.substring(index + 1, x.length - 1) > } > val json_df = spark.read.json(json_rdd) > ``` -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org