[jira] [Commented] (SPARK-11148) Unable to create views
[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261734#comment-15261734 ] Lunen commented on SPARK-11148: --- Thanks Yin. Appreciate your help with this. Do you have more or less an idea when Spark 2.0 will be released? > Unable to create views > -- > > Key: SPARK-11148 > URL: https://issues.apache.org/jira/browse/SPARK-11148 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: Ubuntu 14.04 > Spark-1.5.1-bin-hadoop2.6 > (I don't have Hadoop or Hive installed) > Start spark-all.sh and thriftserver with mysql jar driver >Reporter: Lunen >Priority: Critical > Fix For: 2.0.0 > > > I am unable to create views within spark SQL. > Creating tables without specifying the column names work. eg. > CREATE TABLE trade2 > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Ceating tables with datatypes gives an error: > CREATE TABLE trade2( > COL1 timestamp, > COL2 STRING, > COL3 STRING) > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Error: org.apache.spark.sql.AnalysisException: > org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not allow > user-specified schemas.; SQLState: null ErrorCode: 0 > Trying to create a VIEW from the table that was created.(The select statement > below returns data) > CREATE VIEW viewtrade as Select Col1 from trade2; > Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: > SemanticException [Error 10004]: Line 1:30 Invalid table alias or column > reference 'Col1': (possible column names are: col) > SQLState: null > ErrorCode: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11148) Unable to create views
[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259868#comment-15259868 ] Lunen commented on SPARK-11148: --- Hi Guys, Has there been any progress on this topic? I can see all but one item has been completed on Spark-11012(Canonicalize view definitions) Can we expect to see the new improvements within the next release? Thanks > Unable to create views > -- > > Key: SPARK-11148 > URL: https://issues.apache.org/jira/browse/SPARK-11148 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: Ubuntu 14.04 > Spark-1.5.1-bin-hadoop2.6 > (I don't have Hadoop or Hive installed) > Start spark-all.sh and thriftserver with mysql jar driver >Reporter: Lunen >Priority: Critical > > I am unable to create views within spark SQL. > Creating tables without specifying the column names work. eg. > CREATE TABLE trade2 > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Ceating tables with datatypes gives an error: > CREATE TABLE trade2( > COL1 timestamp, > COL2 STRING, > COL3 STRING) > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Error: org.apache.spark.sql.AnalysisException: > org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not allow > user-specified schemas.; SQLState: null ErrorCode: 0 > Trying to create a VIEW from the table that was created.(The select statement > below returns data) > CREATE VIEW viewtrade as Select Col1 from trade2; > Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: > SemanticException [Error 10004]: Line 1:30 Invalid table alias or column > reference 'Col1': (possible column names are: col) > SQLState: null > ErrorCode: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121080#comment-15121080 ] Lunen commented on SPARK-12403: --- Microsoft just updated their driver and it's working. (Just search for Microsoft Spark ODBC Driver) Thanks for your time guys! > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Closed] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lunen closed SPARK-12403. - Microsoft released an updated driver. Tested and working. > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lunen updated SPARK-12403: -- Comment: was deleted (was: Microsoft released an updated driver. Tested and working.) > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095178#comment-15095178 ] Lunen commented on SPARK-12403: --- Hi Guys. Thanks for replies. I tried adding the database name. It works on all persisted tables but not on the temporary tables. I have reported this to Simba. Simba's latest driver(1.0.9) works perfectly, but Databricks is not updating their driver. Apparently Simba provides the driver as OEM to Databricks. I am using the latest driver Databricks is providing - v1.0.8.1008 (64 bit). Hive – I’m using the 1.5.2 package as is. No other packages or integrations made. So I guess embedded ..? > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082687#comment-15082687 ] Lunen commented on SPARK-12403: --- I've managed to get in contact with the people who develops the Spark ODBC drivers. They told me that they OEM the driver to Databricks and that they don't understand why they would not make the latest driver available. I've also tested a trail version of the developer's latest driver and it works perfectly fine. Asked on Databricks' forumn and sent emails to their sales and info department explaining the situation. Hopefully someone can help. > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lunen updated SPARK-12403: -- Affects Version/s: 1.5.0 Fix Version/s: 1.4.1 > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.0, 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1, 1.4.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
Lunen created SPARK-12403: - Summary: "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore Key: SPARK-12403 URL: https://issues.apache.org/jira/browse/SPARK-12403 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.2, 1.5.1 Environment: ODBC connector query Reporter: Lunen Fix For: 1.3.1 We are unable to query the SPARK tables using the ODBC driver from Simba Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show databases and show tables, but not any queries. eg. Working: Select * from openquery(SPARK,'SHOW DATABASES') Select * from openquery(SPARK,'SHOW TABLES') Not working: Select * from openquery(SPARK,'Select * from default.lunentest') -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062062#comment-15062062 ] Lunen commented on SPARK-12403: --- Something must have changed on the Thriftserver side because test DB and tables are created in the exact same way. Comparing WORKING: 1.3.1 and 1.4.1 NOT WORKING 1.5.x As I said earlier, I can do a "SHOW DATABASES" and "SHOW TABLES" without any issues, but not any SELECT. > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062104#comment-15062104 ] Lunen commented on SPARK-12403: --- I've tried all available driver versions on 1.5.2 I don't know much on the product side in terms of code as I'm a infrastructure guy. If I had to guess... Maybe it's the table object structure that has changed. > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062114#comment-15062114 ] Lunen commented on SPARK-12403: --- Sorry, which driver are you referring to? ODBC? If it's the Databricks ODBC driver, they have only released one driver mid last year. It's been working on all previous thriftserver versions. I've also tried other paid for ODBC drivers in the market without any luck.(Progress and Microsoft) > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11148) Unable to create views
[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061909#comment-15061909 ] Lunen commented on SPARK-11148: --- Hi Yin, I have noticed that the Databricks Windows ODBC driver is not able to query tables. Please can you guys look into this as well. Thanks again for everything. Kind Regards, Lunen > Unable to create views > -- > > Key: SPARK-11148 > URL: https://issues.apache.org/jira/browse/SPARK-11148 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: Ubuntu 14.04 > Spark-1.5.1-bin-hadoop2.6 > (I don't have Hadoop or Hive installed) > Start spark-all.sh and thriftserver with mysql jar driver >Reporter: Lunen >Priority: Critical > > I am unable to create views within spark SQL. > Creating tables without specifying the column names work. eg. > CREATE TABLE trade2 > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Ceating tables with datatypes gives an error: > CREATE TABLE trade2( > COL1 timestamp, > COL2 STRING, > COL3 STRING) > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Error: org.apache.spark.sql.AnalysisException: > org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not allow > user-specified schemas.; SQLState: null ErrorCode: 0 > Trying to create a VIEW from the table that was created.(The select statement > below returns data) > CREATE VIEW viewtrade as Select Col1 from trade2; > Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: > SemanticException [Error 10004]: Line 1:30 Invalid table alias or column > reference 'Col1': (possible column names are: col) > SQLState: null > ErrorCode: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lunen updated SPARK-12403: -- Description: We are unable to query the SPARK tables using the ODBC driver from Simba Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show databases and show tables, but not any queries. eg. Working: Select * from openquery(SPARK,'SHOW DATABASES') Select * from openquery(SPARK,'SHOW TABLES') Not working: Select * from openquery(SPARK,'Select * from lunentest') The error I get is: OLE DB provider "MSDASQL" for linked server "SPARK" returned message "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". Msg 7321, Level 16, State 2, Line 2 An error occurred while preparing the query "Select * from lunentest" for execution against OLE DB provider "MSDASQL" for linked server "SPARK" was: We are unable to query the SPARK tables using the ODBC driver from Simba Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show databases and show tables, but not any queries. eg. Working: Select * from openquery(SPARK,'SHOW DATABASES') Select * from openquery(SPARK,'SHOW TABLES') Not working: Select * from openquery(SPARK,'Select * from default.lunentest') > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lunen updated SPARK-12403: -- Description: We are unable to query the SPARK tables using the ODBC driver from Simba Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show databases and show tables, but not any queries. eg. Working: Select * from openquery(SPARK,'SHOW DATABASES') Select * from openquery(SPARK,'SHOW TABLES') Not working: Select * from openquery(SPARK,'Select * from lunentest') The error I get is: OLE DB provider "MSDASQL" for linked server "SPARK" returned message "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". Msg 7321, Level 16, State 2, Line 2 An error occurred while preparing the query "Select * from lunentest" for execution against OLE DB provider "MSDASQL" for linked server "SPARK" was: We are unable to query the SPARK tables using the ODBC driver from Simba Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show databases and show tables, but not any queries. eg. Working: Select * from openquery(SPARK,'SHOW DATABASES') Select * from openquery(SPARK,'SHOW TABLES') Not working: Select * from openquery(SPARK,'Select * from lunentest') The error I get is: OLE DB provider "MSDASQL" for linked server "SPARK" returned message "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". Msg 7321, Level 16, State 2, Line 2 An error occurred while preparing the query "Select * from lunentest" for execution against OLE DB provider "MSDASQL" for linked server "SPARK" > "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore > > > Key: SPARK-12403 > URL: https://issues.apache.org/jira/browse/SPARK-12403 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1, 1.5.2 > Environment: ODBC connector query >Reporter: Lunen > Fix For: 1.3.1 > > > We are unable to query the SPARK tables using the ODBC driver from Simba > Spark(Databricks - "Simba Spark ODBC Driver 1.0") We are able to do a show > databases and show tables, but not any queries. eg. > Working: > Select * from openquery(SPARK,'SHOW DATABASES') > Select * from openquery(SPARK,'SHOW TABLES') > Not working: > Select * from openquery(SPARK,'Select * from lunentest') > The error I get is: > OLE DB provider "MSDASQL" for linked server "SPARK" returned message > "[Simba][SQLEngine] (31740) Table or view not found: spark..lunentest". > Msg 7321, Level 16, State 2, Line 2 > An error occurred while preparing the query "Select * from lunentest" for > execution against OLE DB provider "MSDASQL" for linked server "SPARK" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11148) Unable to create views
[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029732#comment-15029732 ] Lunen commented on SPARK-11148: --- Hi Maciej, Spark implementation at our client site will be in January 2016. Kind regards > Unable to create views > -- > > Key: SPARK-11148 > URL: https://issues.apache.org/jira/browse/SPARK-11148 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: Ubuntu 14.04 > Spark-1.5.1-bin-hadoop2.6 > (I don't have Hadoop or Hive installed) > Start spark-all.sh and thriftserver with mysql jar driver >Reporter: Lunen >Priority: Critical > > I am unable to create views within spark SQL. > Creating tables without specifying the column names work. eg. > CREATE TABLE trade2 > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Ceating tables with datatypes gives an error: > CREATE TABLE trade2( > COL1 timestamp, > COL2 STRING, > COL3 STRING) > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:mysql://192.168.30.191:3318/?user=root", > dbtable "database.trade", > driver "com.mysql.jdbc.Driver" > ); > Error: org.apache.spark.sql.AnalysisException: > org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not allow > user-specified schemas.; SQLState: null ErrorCode: 0 > Trying to create a VIEW from the table that was created.(The select statement > below returns data) > CREATE VIEW viewtrade as Select Col1 from trade2; > Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: > SemanticException [Error 10004]: Line 1:30 Invalid table alias or column > reference 'Col1': (possible column names are: col) > SQLState: null > ErrorCode: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-11148) Unable to create views
Lunen created SPARK-11148: - Summary: Unable to create views Key: SPARK-11148 URL: https://issues.apache.org/jira/browse/SPARK-11148 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Environment: Ubuntu 14.04 Spark-1.5.1-bin-hadoop2.6 (I don't have Hadoop or Hive installed) Start spark-all.sh and thriftserver with mysql jar driver Reporter: Lunen Priority: Critical I am unable to create views within spark SQL. Creating tables without specifying the column names work. eg. CREATE TABLE trade2 USING org.apache.spark.sql.jdbc OPTIONS ( url "jdbc:mysql://192.168.30.191:3318/?user=root", dbtable "database.trade", driver "com.mysql.jdbc.Driver" ); Ceating tables with datatypes gives an error: CREATE TABLE trade2( COL1 timestamp, COL2 STRING, COL3 STRING) USING org.apache.spark.sql.jdbc OPTIONS ( url "jdbc:mysql://192.168.30.191:3318/?user=root", dbtable "database.trade", driver "com.mysql.jdbc.Driver" ); Error: org.apache.spark.sql.AnalysisException: org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not allow user-specified schemas.; SQLState: null ErrorCode: 0 Trying to create a VIEW from the table that was created.(The select statement below returns data) CREATE VIEW viewtrade as Select Col1 from trade2; Error: org.apache.spark.sql.execution.QueryExecutionException: FAILED: SemanticException [Error 10004]: Line 1:30 Invalid table alias or column reference 'Col1': (possible column names are: col) SQLState: null ErrorCode: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-10633) Persisting Spark stream to MySQL - Spark tries to create the table for every stream even if it exist already.
Lunen created SPARK-10633:
-
Summary: Persisting Spark stream to MySQL - Spark tries to create
the table for every stream even if it exist already.
Key: SPARK-10633
URL: https://issues.apache.org/jira/browse/SPARK-10633
Project: Spark
Issue Type: Bug
Components: SQL, Streaming
Affects Versions: 1.5.0, 1.4.0
Environment: Ubuntu 14.04
IntelliJ IDEA 14.1.4
sbt
mysql-connector-java 5.1.35 (Tested and working with Spark 1.3.1)
Reporter: Lunen
Priority: Blocker
Persisting Spark Kafka stream to MySQL
Spark 1.4 + tries to create a table automatically every time the stream gets
sent to a specified table.
Please note, Spark 1.3.1 works.
Code sample:
val url = "jdbc:mysql://host:port/db?user=user=password
val crp = RowSetProvider.newFactory()
val crsSql: CachedRowSet = crp.createCachedRowSet()
val crsTrg: CachedRowSet = crp.createCachedRowSet()
crsSql.beforeFirst()
crsTrg.beforeFirst()
//Read Stream from Kafka
//Produce SQL INSERT STRING
streamT.foreachRDD { rdd =>
if (rdd.toLocalIterator.nonEmpty) {
sqlContext.read.json(rdd).registerTempTable(serverEvents + "_events")
while (crsSql.next) {
sqlContext.sql("SQL INSERT STRING").write.jdbc(url, "SCHEMA_NAME",
new Properties)
println("Persisted Data: " + 'SQL INSERT STRING')
}
crsSql.beforeFirst()
}
stmt.close()
conn.close()
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
