[ https://issues.apache.org/jira/browse/SPARK-25931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673980#comment-16673980 ]
Apache Spark commented on SPARK-25931: -------------------------------------- User 'MaxGekk' has created a pull request for this issue: https://github.com/apache/spark/pull/22920 > Benchmarking creation of Jackson parser > --------------------------------------- > > Key: SPARK-25931 > URL: https://issues.apache.org/jira/browse/SPARK-25931 > Project: Spark > Issue Type: Test > Components: SQL > Affects Versions: 2.4.0 > Reporter: Maxim Gekk > Priority: Minor > > Existing JSON benchmarks perlineParsing and perlineParsingOfWideColumn don't > invoke Jackson parser at all due to an optimization for empty schema > introduced SPARK-24959. Need to add new benchmark which should forcibly > create Jackson parser for short and wide columns. For example: > {code:scala} > spark.read > .schema(schema) > .json(path) > .filter((_: Row) => true) > .count() > {code} > The *.filter((_: Row) => true)* prevents projection pushdown to JSON > datasource and forces fully parsing of JSON content. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org