[jira] [Assigned] (SPARK-7758) Failed to start thrift server when metastore is postgre sql
[ https://issues.apache.org/jira/browse/SPARK-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7758: --- Assignee: Apache Spark Failed to start thrift server when metastore is postgre sql --- Key: SPARK-7758 URL: https://issues.apache.org/jira/browse/SPARK-7758 Project: Spark Issue Type: Bug Components: SQL Reporter: Tao Wang Assignee: Apache Spark Priority: Critical Attachments: hive-site.xml, with error.log, with no error.log I am using today's master branch to start thrift server with setting metastore to postgre sql, and it shows error like: `15/05/20 20:43:57 DEBUG Schema: DROP TABLE DELETEME1432125837197 CASCADE 15/05/20 20:43:57 ERROR Datastore: Error thrown executing DROP TABLE DELETEME1432125837197 CASCADE : Syntax error: Encountered CASCADE at line 1, column 34. java.sql.SQLSyntaxErrorException: Syntax error: Encountered CASCADE at line 1, column 34. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264)` But it works well with earlier master branch (on 7th, April). After printing their debug level log, I found current branch tries to connect with derby but didn't know why, maybe the big reconstructure in sql module cause this issue. The Datastore shows in current branch: 15/05/20 20:43:57 DEBUG Datastore: === Datastore = 15/05/20 20:43:57 DEBUG Datastore: StoreManager : rdbms (org.datanucleus.store.rdbms.RDBMSStoreManager) 15/05/20 20:43:57 DEBUG Datastore: Datastore : read-write 15/05/20 20:43:57 DEBUG Datastore: Schema Control : AutoCreate(None), Validate(None) 15/05/20 20:43:57 DEBUG Datastore: Query Languages : [JDOQL, JPQL, SQL, STOREDPROC] 15/05/20 20:43:57 DEBUG Datastore: Queries : Timeout=0 15/05/20 20:43:57 DEBUG Datastore: === 15/05/20 20:43:57 DEBUG Datastore: Datastore Adapter : org.datanucleus.store.rdbms.adapter.PostgreSQLAdapter 15/05/20 20:43:57 DEBUG Datastore: Datastore : name=Apache Derby version=10.10.1.1 - (1458268) 15/05/20 20:43:57 DEBUG Datastore: Datastore Driver : name=Apache Derby Embedded JDBC Driver version=10.10.1.1 - (1458268) 15/05/20 20:43:57 DEBUG Datastore: Primary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true] 15/05/20 20:43:57 DEBUG Datastore: Secondary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true] 15/05/20 20:43:57 DEBUG Datastore: Datastore Identifiers : factory=datanucleus1 case=UPPERCASE catalog= schema=SPARK 15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Cases : MixedCase UPPERCASE MixedCase-Sensitive 15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Lengths (max) : Table=128 Column=128 Constraint=128 Index=128 Delimiter= 15/05/20 20:43:57 DEBUG Datastore: Support for Identifiers in DDL : catalog=false schema=true 15/05/20 20:43:57 DEBUG Datastore: Datastore : checkTableViewExistence, rdbmsConstraintCreateMode=DataNucleus, initialiseColumnInfo=ALL 15/05/20 20:43:57 DEBUG Datastore: Support Statement Batching : yes (max-batch-size=50) 15/05/20 20:43:57 DEBUG Datastore: Queries : Results direction=forward, type=forward-only, concurrency=read-only 15/05/20 20:43:57 DEBUG Datastore: Java-Types : string-default-length=255 15/05/20 20:43:57 DEBUG Datastore: JDBC-Types : [id=2009], BLOB, CLOB, TIME, DATE, BOOLEAN, VARCHAR, DECIMAL, NUMERIC, CHAR, BINARY, FLOAT, LONGVARBINARY, VARBINARY, JAVA_OBJECT 15/05/20 20:43:57 DEBUG Datastore: === The Datastore in earlier master branch: 15/05/20 20:18:10 DEBUG Datastore: === Datastore = 15/05/20 20:18:10 DEBUG Datastore: StoreManager : rdbms
[jira] [Assigned] (SPARK-7758) Failed to start thrift server when metastore is postgre sql
[ https://issues.apache.org/jira/browse/SPARK-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7758: --- Assignee: (was: Apache Spark) Failed to start thrift server when metastore is postgre sql --- Key: SPARK-7758 URL: https://issues.apache.org/jira/browse/SPARK-7758 Project: Spark Issue Type: Bug Components: SQL Reporter: Tao Wang Priority: Critical Attachments: hive-site.xml, with error.log, with no error.log I am using today's master branch to start thrift server with setting metastore to postgre sql, and it shows error like: `15/05/20 20:43:57 DEBUG Schema: DROP TABLE DELETEME1432125837197 CASCADE 15/05/20 20:43:57 ERROR Datastore: Error thrown executing DROP TABLE DELETEME1432125837197 CASCADE : Syntax error: Encountered CASCADE at line 1, column 34. java.sql.SQLSyntaxErrorException: Syntax error: Encountered CASCADE at line 1, column 34. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264)` But it works well with earlier master branch (on 7th, April). After printing their debug level log, I found current branch tries to connect with derby but didn't know why, maybe the big reconstructure in sql module cause this issue. The Datastore shows in current branch: 15/05/20 20:43:57 DEBUG Datastore: === Datastore = 15/05/20 20:43:57 DEBUG Datastore: StoreManager : rdbms (org.datanucleus.store.rdbms.RDBMSStoreManager) 15/05/20 20:43:57 DEBUG Datastore: Datastore : read-write 15/05/20 20:43:57 DEBUG Datastore: Schema Control : AutoCreate(None), Validate(None) 15/05/20 20:43:57 DEBUG Datastore: Query Languages : [JDOQL, JPQL, SQL, STOREDPROC] 15/05/20 20:43:57 DEBUG Datastore: Queries : Timeout=0 15/05/20 20:43:57 DEBUG Datastore: === 15/05/20 20:43:57 DEBUG Datastore: Datastore Adapter : org.datanucleus.store.rdbms.adapter.PostgreSQLAdapter 15/05/20 20:43:57 DEBUG Datastore: Datastore : name=Apache Derby version=10.10.1.1 - (1458268) 15/05/20 20:43:57 DEBUG Datastore: Datastore Driver : name=Apache Derby Embedded JDBC Driver version=10.10.1.1 - (1458268) 15/05/20 20:43:57 DEBUG Datastore: Primary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true] 15/05/20 20:43:57 DEBUG Datastore: Secondary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true] 15/05/20 20:43:57 DEBUG Datastore: Datastore Identifiers : factory=datanucleus1 case=UPPERCASE catalog= schema=SPARK 15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Cases : MixedCase UPPERCASE MixedCase-Sensitive 15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Lengths (max) : Table=128 Column=128 Constraint=128 Index=128 Delimiter= 15/05/20 20:43:57 DEBUG Datastore: Support for Identifiers in DDL : catalog=false schema=true 15/05/20 20:43:57 DEBUG Datastore: Datastore : checkTableViewExistence, rdbmsConstraintCreateMode=DataNucleus, initialiseColumnInfo=ALL 15/05/20 20:43:57 DEBUG Datastore: Support Statement Batching : yes (max-batch-size=50) 15/05/20 20:43:57 DEBUG Datastore: Queries : Results direction=forward, type=forward-only, concurrency=read-only 15/05/20 20:43:57 DEBUG Datastore: Java-Types : string-default-length=255 15/05/20 20:43:57 DEBUG Datastore: JDBC-Types : [id=2009], BLOB, CLOB, TIME, DATE, BOOLEAN, VARCHAR, DECIMAL, NUMERIC, CHAR, BINARY, FLOAT, LONGVARBINARY, VARBINARY, JAVA_OBJECT 15/05/20 20:43:57 DEBUG Datastore: === The Datastore in earlier master branch: 15/05/20 20:18:10 DEBUG Datastore: === Datastore = 15/05/20 20:18:10 DEBUG Datastore: StoreManager : rdbms
