[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Cutler updated SPARK-21187: --------------------------------- Description: This is to track adding the remaining type support in Arrow Converters. Currently, only primitive data types are supported. ' Remaining types: * *Date* * *Timestamp* * *Complex*: Struct, Array, Map * *Decimal* Some things to do before closing this out: * Look to upgrading to Arrow 0.7 for better Decimal support (can now write values as BigDecimal) * Need to add some user docs * Make sure Python tests are thorough * Check into complex type support mentioned in comments by [~leif], should we support mulit-indexing? was: This is to track adding the remaining type support in Arrow Converters. Currently, only primitive data types are supported. ' Remaining types: * *Date* * *Timestamp* * *Complex*: Struct, Array, Map * *Decimal* > Complete support for remaining Spark data types in Arrow Converters > ------------------------------------------------------------------- > > Key: SPARK-21187 > URL: https://issues.apache.org/jira/browse/SPARK-21187 > Project: Spark > Issue Type: Umbrella > Components: PySpark, SQL > Affects Versions: 2.3.0 > Reporter: Bryan Cutler > > This is to track adding the remaining type support in Arrow Converters. > Currently, only primitive data types are supported. ' > Remaining types: > * *Date* > * *Timestamp* > * *Complex*: Struct, Array, Map > * *Decimal* > Some things to do before closing this out: > * Look to upgrading to Arrow 0.7 for better Decimal support (can now write > values as BigDecimal) > * Need to add some user docs > * Make sure Python tests are thorough > * Check into complex type support mentioned in comments by [~leif], should we > support mulit-indexing? -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org