Patrick Young created SPARK-23612:

             Summary: Specify formats for individual DateType and TimestampType 
columns in schemas
                 Key: SPARK-23612
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, SQL
    Affects Versions: 2.3.0
            Reporter: Patrick Young


It would be very helpful if it were possible to specify the format for 
individual columns in a schema when reading csv files, rather than one format:


# Currently can only do something like:"**dateFormat", "yyyyMMdd").csv(...) 

# Would like to be able to do something like:

schema = StructType([

    StructField("date1", DateType(format="MM/dd/yyyy"), True),

    StructField("date2", DateType(format="yyyyMMdd"), True)




This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to