Re: How to use SKIP_SCHEMA_INIT=TRUE from command line

[Sanjay Gupta]

Thanks, it solves issue. Much appreciated.


Thanks
Sanjay Gupta
________________________________
From: Akshat m <akshatats...@gmail.com>
Sent: Friday, December 22, 2023 5:55:38 AM
To: user@hive.apache.org <user@hive.apache.org>
Subject: Re: How to use SKIP_SCHEMA_INIT=TRUE from command line

Hi Sanjay,

Instead of using  --env SKIP_SCHEMA_INIT=TRUE,

Please use --env IS_RESUME="true" while running,

docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
--add-host=host.docker.internal:host-gateway \
     --env IS_RESUME="true" \
     --env DB_DRIVER=mysql \
     --env 
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
 
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=****" \
     --mount source=warehouse,target=/opt/hive/data/warehouse \
     --name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash

This should skip the schema initOrUpgrade process: 
https://github.com/apache/hive/blob/5022b85b5f50615f85da07bce42aebd414deb9b0/packaging/src/docker/entrypoint.sh#L24
from the 2nd time you run the container.

Regards,
Akshat


On Fri, Dec 22, 2023 at 11:53 AM Sanjay Gupta 
<sanja...@gmail.com<mailto:sanja...@gmail.com>> wrote:
Hi All,

If my metastore schema already exists with correct version so what I
need to do so it doesn't do init or upgrade when starting metastore
container

I have tried following command line


On MAC environment variables
export HIVE_VERSION=3.1.3
and even
SKIP_SCHEMA_INIT=TRUE

docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore
--add-host=host.docker.internal:host-gateway \
     --env SKIP_SCHEMA_INIT=TRUE \
     --env DB_DRIVER=mysql \
     --env 
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
 
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=****" \
     --mount source=warehouse,target=/opt/hive/data/warehouse \
     --name metastore-standalone apache/hive:${HIVE_VERSION} /bin/bash



-----------------------

Docker Logs , it still tries to initSchema

docker logs 1c
+ : mysql
+ SKIP_SCHEMA_INIT=false
+ export HIVE_CONF_DIR=/opt/hive/conf
+ HIVE_CONF_DIR=/opt/hive/conf
+ '[' -d '' ']'
+ export 'HADOOP_CLIENT_OPTS= -Xmx1G
-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=hive'
+ HADOOP_CLIENT_OPTS=' -Xmx1G
-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=hive'
+ [[ false == \f\a\l\s\e ]]
+ initialize_hive
+ COMMAND=-initOrUpgradeSchema
++ cut -d . -f1
++ echo 3.1.3
+ '[' 3 -lt 4 ']'
+ COMMAND=-initSchema
+ /opt/hive/bin/schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:
jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.cj.jdbc.Driver
Metastore connection User: hive
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql


Error: Table 'ctlgs' already exists (state=42S01,code=1050)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema
initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
[WARN] Failed to create directory: /home/hive/.beeline
No such file or directory
+ '[' 1 -eq 0 ']'
+ echo 'Schema initialization failed!'
Schema initialization failed!
+ exit 1
-----------------



Docker entrypoint.sh have following code

SKIP_SCHEMA_INIT="${IS_RESUME:-false}"

function initialize_hive {
  COMMAND="-initOrUpgradeSchema"
  if [ "$(echo "$HIVE_VER" | cut -d '.' -f1)" -lt "4" ]; then
     COMMAND="-${SCHEMA_COMMAND:-initSchema}"
  fi
  $HIVE_HOME/bin/schematool -dbType $DB_DRIVER $COMMAND
  if [ $? -eq 0 ]; then
    echo "Initialized schema successfully.."
  else
    echo "Schema initialization failed!"
    exit 1
  fi
}

export HIVE_CONF_DIR=$HIVE_HOME/conf
if [ -d "${HIVE_CUSTOM_CONF_DIR:-}" ]; then
  find "${HIVE_CUSTOM_CONF_DIR}" -type f -exec \
    ln -sfn {} "${HIVE_CONF_DIR}"/ \;
  export HADOOP_CONF_DIR=$HIVE_CONF_DIR
  export TEZ_CONF_DIR=$HIVE_CONF_DIR
fi

export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Xmx1G $SERVICE_OPTS"
if [[ "${SKIP_SCHEMA_INIT}" == "false" ]]; then
  # handles schema initialization
  initialize_hive
fi

\