[
https://issues.apache.org/jira/browse/SPARK-60589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Pavlak updated SPARK-60589:
--------------------------------
Affects Version/s: 5.0.0
(was: 4.0.6)
> Add native fixed-width file (FWF) data source
> ---------------------------------------------
>
> Key: SPARK-60589
> URL: https://issues.apache.org/jira/browse/SPARK-60589
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Sean Pavlak
> Priority: Major
>
> Add a built-in fixed-width file data source (format "fwf") with
> spark.read.fwf and DataFrame.write.fwf, matching the existing CSV, JSON, and
> XML shortcuts.
> Supports explicit colspecs or widths, optional inference of column positions,
> header, comment, skipRows, nullValue, and encoding, plus PERMISSIVE,
> DROPMALFORMED, and FAILFAST parse modes. Scala, Java, Python, and Spark
> Connect APIs, docs, and examples are included.
> This is separate from SPARK-36392, which is pandas API on Spark read_fwf.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
