[ https://issues.apache.org/jira/browse/SPARK-25931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun reassigned SPARK-25931: ------------------------------------- Assignee: Maxim Gekk > Benchmarking creation of Jackson parser > --------------------------------------- > > Key: SPARK-25931 > URL: https://issues.apache.org/jira/browse/SPARK-25931 > Project: Spark > Issue Type: Test > Components: SQL > Affects Versions: 3.0.0 > Reporter: Maxim Gekk > Assignee: Maxim Gekk > Priority: Minor > Fix For: 3.0.0 > > > Existing JSON benchmarks perlineParsing and perlineParsingOfWideColumn don't > invoke Jackson parser at all due to an optimization for empty schema > introduced SPARK-24959. Need to add new benchmark which should forcibly > create Jackson parser for short and wide columns. For example: > {code:scala} > spark.read > .schema(schema) > .json(path) > .filter((_: Row) => true) > .count() > {code} > The *.filter((_: Row) => true)* prevents projection pushdown to JSON > datasource and forces fully parsing of JSON content. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org