Sean Pavlak created SPARK-60589:
-----------------------------------
Summary: Add native fixed-width file (FWF) data source
Key: SPARK-60589
URL: https://issues.apache.org/jira/browse/SPARK-60589
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 4.0.6
Reporter: Sean Pavlak
Add a built-in fixed-width file data source (format "fwf") with spark.read.fwf
and DataFrame.write.fwf, matching the existing CSV, JSON, and XML shortcuts.
Supports explicit colspecs or widths, optional inference of column positions,
header, comment, skipRows, nullValue, and encoding, plus PERMISSIVE,
DROPMALFORMED, and FAILFAST parse modes. Scala, Java, Python, and Spark Connect
APIs, docs, and examples are included.
This is separate from SPARK-36392, which is pandas API on Spark read_fwf.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
