Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21657
> Do you mean we remove the option for column pruning in csv?
I mean reverting back the index mapping - `tokenIndexArr`. In this case,
your changes in `buildReader` are not needed.
If `spark.sql.csv.parser.columnPruning.enabled`is set to `false`, the
`tokenIndexArr` will map data index to required index. In the default case
(`spark.sql.csv.parser.columnPruning.enabled` is `true`), the array is actually
not needed, and it can be replaced by the identity function.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
