[jira] [Commented] (SPARK-24064) [Spark SQL] Create table using csv does not support binary column Type
[ https://issues.apache.org/jira/browse/SPARK-24064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1822#comment-1822 ] ABHISHEK KUMAR GUPTA commented on SPARK-24064: -- Ok will raise Jira for supported type which can be capture in Product Document. > [Spark SQL] Create table using csv does not support binary column Type > --- > > Key: SPARK-24064 > URL: https://issues.apache.org/jira/browse/SPARK-24064 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.3.0 > Environment: OS Type: Suse 11 > Spark Version: 2.3.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > Labels: test > > # Launch spark-sql --master yarn > # create table csvTable (time timestamp, name string, isright boolean, > datetoday date, num binary, height double, score float, decimaler > decimal(10,0), id tinyint, age int, license bigint, length smallint) using > CSV options (path "/user/datatmo/customer1.csv"); > Result: Table creation is successful > 3. Select * from csvTable; > Throws below Exception > ERROR SparkSQLDriver:91 - Failed in [select * from csvtable] > java.lang.UnsupportedOperationException: *CSV data source does not support > binary data type*. > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$.org$apache$spark$sql$execution$datasources$csv$CSVUtils$$verifyType$1(CSVUtils.scala:127) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at org.apache.spark.sql.types.StructType.foreach(StructType.scala:99) > > But Normal table supports binary Data Type. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24064) [Spark SQL] Create table using csv does not support binary column Type
[ https://issues.apache.org/jira/browse/SPARK-24064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1765#comment-1765 ] Hyukjin Kwon commented on SPARK-24064: -- That's trivial. It explicitly throws an exception for now. Please file another JIRA that targets to list up supported types. I am resolving this. > [Spark SQL] Create table using csv does not support binary column Type > --- > > Key: SPARK-24064 > URL: https://issues.apache.org/jira/browse/SPARK-24064 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.3.0 > Environment: OS Type: Suse 11 > Spark Version: 2.3.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > Labels: test > > # Launch spark-sql --master yarn > # create table csvTable (time timestamp, name string, isright boolean, > datetoday date, num binary, height double, score float, decimaler > decimal(10,0), id tinyint, age int, license bigint, length smallint) using > CSV options (path "/user/datatmo/customer1.csv"); > Result: Table creation is successful > 3. Select * from csvTable; > Throws below Exception > ERROR SparkSQLDriver:91 - Failed in [select * from csvtable] > java.lang.UnsupportedOperationException: *CSV data source does not support > binary data type*. > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$.org$apache$spark$sql$execution$datasources$csv$CSVUtils$$verifyType$1(CSVUtils.scala:127) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at org.apache.spark.sql.types.StructType.foreach(StructType.scala:99) > > But Normal table supports binary Data Type. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24064) [Spark SQL] Create table using csv does not support binary column Type
[ https://issues.apache.org/jira/browse/SPARK-24064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627376#comment-16627376 ] Gururaj Shetty commented on SPARK-24064: If Binary is not supported only for CSV, then the same can be captured in the document. Useful for the user while using CSV > [Spark SQL] Create table using csv does not support binary column Type > --- > > Key: SPARK-24064 > URL: https://issues.apache.org/jira/browse/SPARK-24064 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.3.0 > Environment: OS Type: Suse 11 > Spark Version: 2.3.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > Labels: test > > # Launch spark-sql --master yarn > # create table csvTable (time timestamp, name string, isright boolean, > datetoday date, num binary, height double, score float, decimaler > decimal(10,0), id tinyint, age int, license bigint, length smallint) using > CSV options (path "/user/datatmo/customer1.csv"); > Result: Table creation is successful > 3. Select * from csvTable; > Throws below Exception > ERROR SparkSQLDriver:91 - Failed in [select * from csvtable] > java.lang.UnsupportedOperationException: *CSV data source does not support > binary data type*. > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$.org$apache$spark$sql$execution$datasources$csv$CSVUtils$$verifyType$1(CSVUtils.scala:127) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at org.apache.spark.sql.types.StructType.foreach(StructType.scala:99) > > But Normal table supports binary Data Type. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24064) [Spark SQL] Create table using csv does not support binary column Type
[ https://issues.apache.org/jira/browse/SPARK-24064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468817#comment-16468817 ] ABHISHEK KUMAR GUPTA commented on SPARK-24064: -- I am from QA side. Development group can answer it better. > [Spark SQL] Create table using csv does not support binary column Type > --- > > Key: SPARK-24064 > URL: https://issues.apache.org/jira/browse/SPARK-24064 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.3.0 > Environment: OS Type: Suse 11 > Spark Version: 2.3.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > Labels: test > > # Launch spark-sql --master yarn > # create table csvTable (time timestamp, name string, isright boolean, > datetoday date, num binary, height double, score float, decimaler > decimal(10,0), id tinyint, age int, license bigint, length smallint) using > CSV options (path "/user/datatmo/customer1.csv"); > Result: Table creation is successful > 3. Select * from csvTable; > Throws below Exception > ERROR SparkSQLDriver:91 - Failed in [select * from csvtable] > java.lang.UnsupportedOperationException: *CSV data source does not support > binary data type*. > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$.org$apache$spark$sql$execution$datasources$csv$CSVUtils$$verifyType$1(CSVUtils.scala:127) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at > org.apache.spark.sql.execution.datasources.csv.CSVUtils$$anonfun$verifySchema$1.apply(CSVUtils.scala:131) > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at org.apache.spark.sql.types.StructType.foreach(StructType.scala:99) > > But Normal table supports binary Data Type. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
