[ https://issues.apache.org/jira/browse/SPARK-30128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicholas Chammas updated SPARK-30128: ------------------------------------- Description: Following on to SPARK-29903 and similar issues (linked), there are options available to the DataFrameReader for certain source formats, but which are not exposed properly in the relevant APIs. These options include `timeZone` and `pathGlobFilter`. Instead of being noted under [the option() method|https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrameReader.option], they should be implemented directly into load APIs that support them. was: Following on to SPARK-27990 and similar issues (linked), there are options available to the DataFrameReader for certain source formats, but which are not exposed properly in the relevant APIs. These options include `timeZone` and `pathGlobFilter`. > Promote remaining "hidden" PySpark DataFrameReader options to load APIs > ----------------------------------------------------------------------- > > Key: SPARK-30128 > URL: https://issues.apache.org/jira/browse/SPARK-30128 > Project: Spark > Issue Type: Improvement > Components: PySpark, SQL > Affects Versions: 2.4.4, 3.0.0 > Reporter: Nicholas Chammas > Priority: Minor > > Following on to SPARK-29903 and similar issues (linked), there are options > available to the DataFrameReader for certain source formats, but which are > not exposed properly in the relevant APIs. > These options include `timeZone` and `pathGlobFilter`. Instead of being noted > under [the option() > method|https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrameReader.option], > they should be implemented directly into load APIs that support them. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org