Re: creating database issue
And, I can't start spark-sql shell, the error as below. Does this mean I need to install Hive on local machine? Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27, see the next exception for details. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source) at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source) at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189) at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361) at com.jolbox.bonecp.BoneCP.(BoneCP.java:416) ... 89 more Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27, see the next exception for details. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 105 more Thanks. On 2021/12/8 9:28, bitfox wrote: Hello This is just a standalone deployment for testing purpose. The version: Spark 3.2.0 (git revision 5d45a415f3) built for Hadoop 3.3.1 Build flags: -B -Pmesos -Pyarn -Pkubernetes -Psparkr -Pscala-2.12 -Phadoop-3.2 -Phive -Phive-thriftserver I just started one master and one worker for the test. Thanks On 2021/12/8 9:15, Qian Sun wrote: It seems to be a hms question. Would u like to provide the information about spark version, hive version and spark application configuration? - To unsubscribe e-mail: user-unsubscr...@spark.apache.org - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: creating database issue
Hello This is just a standalone deployment for testing purpose. The version: Spark 3.2.0 (git revision 5d45a415f3) built for Hadoop 3.3.1 Build flags: -B -Pmesos -Pyarn -Pkubernetes -Psparkr -Pscala-2.12 -Phadoop-3.2 -Phive -Phive-thriftserver I just started one master and one worker for the test. Thanks On 2021/12/8 9:15, Qian Sun wrote: It seems to be a hms question. Would u like to provide the information about spark version, hive version and spark application configuration? - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: creating database issue
Hi, It seems to be a hms question. Would u like to provide the information about spark version, hive version and spark application configuration? Best > 2021年12月8日 上午9:04,bitfox 写道: > > sorry I am newbie to spark. > > When I created a database in pyspark shell following the book content of > learning spark 2.0, it gets: > > >>> spark.sql("CREATE DATABASE learn_spark_db") > 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout > does not exist > 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.retries.wait > does not exist > 21/12/08 09:01:39 WARN ObjectStore: Version information not found in > metastore. hive.metastore.schema.verification is not enabled so recording the > schema version 2.3.0 > 21/12/08 09:01:39 WARN ObjectStore: setMetaStoreSchemaVersion called but > recording version is disabled: version = 2.3.0, comment = Set by MetaStore > pyh@185.213.174.249 > 21/12/08 09:01:40 WARN ObjectStore: Failed to get database default, returning > NoSuchObjectException > 21/12/08 09:01:40 WARN ObjectStore: Failed to get database global_temp, > returning NoSuchObjectException > 21/12/08 09:01:40 WARN ObjectStore: Failed to get database learn_spark_db, > returning NoSuchObjectException > > Can you point to me where is wrong? > > Thanks. > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
creating database issue
sorry I am newbie to spark. When I created a database in pyspark shell following the book content of learning spark 2.0, it gets: >>> spark.sql("CREATE DATABASE learn_spark_db") 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist 21/12/08 09:01:34 WARN HiveConf: HiveConf of name hive.stats.retries.wait does not exist 21/12/08 09:01:39 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0 21/12/08 09:01:39 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore pyh@185.213.174.249 21/12/08 09:01:40 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 21/12/08 09:01:40 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException 21/12/08 09:01:40 WARN ObjectStore: Failed to get database learn_spark_db, returning NoSuchObjectException Can you point to me where is wrong? Thanks. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org