[jira] [Updated] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
[ https://issues.apache.org/jira/browse/SPARK-33920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdul Rafay Abdul Rafay updated SPARK-33920: Attachment: (was: Screenshot 2020-12-28 at 2.23.13 PM.png) > We cannot pass schema to a createDataFrame function in scala, however we can > do this in python. > --- > > Key: SPARK-33920 > URL: https://issues.apache.org/jira/browse/SPARK-33920 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.1 >Reporter: Abdul Rafay Abdul Rafay >Priority: Major > Attachments: Screenshot 2020-12-28 at 2.23.13 PM.png > > Original Estimate: 168h > Remaining Estimate: 168h > > ~spark.createDataFrame(data, schema)~ > ~I am able to pass schema as a parameter to a function createDataFrame in > python but cannot pass this in scala for static data.~ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
[ https://issues.apache.org/jira/browse/SPARK-33920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdul Rafay Abdul Rafay updated SPARK-33920: Attachment: Screenshot 2020-12-28 at 2.23.13 PM.png > We cannot pass schema to a createDataFrame function in scala, however we can > do this in python. > --- > > Key: SPARK-33920 > URL: https://issues.apache.org/jira/browse/SPARK-33920 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.1 >Reporter: Abdul Rafay Abdul Rafay >Priority: Major > Attachments: Screenshot 2020-12-28 at 2.23.13 PM.png, Screenshot > 2020-12-28 at 2.23.13 PM.png > > Original Estimate: 168h > Remaining Estimate: 168h > > ~spark.createDataFrame(data, schema)~ > ~I am able to pass schema as a parameter to a function createDataFrame in > python but cannot pass this in scala for static data.~ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
[ https://issues.apache.org/jira/browse/SPARK-33920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdul Rafay Abdul Rafay updated SPARK-33920: Attachment: Screenshot 2020-12-28 at 2.23.13 PM.png > We cannot pass schema to a createDataFrame function in scala, however we can > do this in python. > --- > > Key: SPARK-33920 > URL: https://issues.apache.org/jira/browse/SPARK-33920 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.1 >Reporter: Abdul Rafay Abdul Rafay >Priority: Major > Attachments: Screenshot 2020-12-28 at 2.23.13 PM.png, Screenshot > 2020-12-28 at 2.23.13 PM.png > > Original Estimate: 168h > Remaining Estimate: 168h > > ~spark.createDataFrame(data, schema)~ > ~I am able to pass schema as a parameter to a function createDataFrame in > python but cannot pass this in scala for static data.~ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
[ https://issues.apache.org/jira/browse/SPARK-33920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255481#comment-17255481 ] Abdul Rafay Abdul Rafay commented on SPARK-33920: - [~viirya] you're right and I know Scala API uses Scala reflection to infer the schema of the given Product. But where I want to assign a datatype explicitly like DecimalType or FloatType to a dataframe created from a static sequence of rows, it creates a problem there, however, in the case of pyspark it does not > We cannot pass schema to a createDataFrame function in scala, however we can > do this in python. > --- > > Key: SPARK-33920 > URL: https://issues.apache.org/jira/browse/SPARK-33920 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.1 >Reporter: Abdul Rafay Abdul Rafay >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > ~spark.createDataFrame(data, schema)~ > ~I am able to pass schema as a parameter to a function createDataFrame in > python but cannot pass this in scala for static data.~ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
Abdul Rafay Abdul Rafay created SPARK-33920: --- Summary: We cannot pass schema to a createDataFrame function in scala, however we can do this in python. Key: SPARK-33920 URL: https://issues.apache.org/jira/browse/SPARK-33920 Project: Spark Issue Type: Improvement Components: Build, SQL Affects Versions: 3.0.1 Reporter: Abdul Rafay Abdul Rafay ~spark.createDataFrame(data, schema)~ ~I am able to pass schema as a parameter to a function createDataFrame in python but cannot pass this in scala for static data.~ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
