This is an automated email from the ASF dual-hosted git repository.
dengzh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 06ef7c82315 HIVE-27955: Missing Postgres driver when start services
from Docker compose (#4948)
06ef7c82315 is described below
commit 06ef7c8231501c0314875d640f081adc35a70db6
Author: dengzh <[email protected]>
AuthorDate: Tue Jan 16 09:17:29 2024 +0800
HIVE-27955: Missing Postgres driver when start services from Docker compose
(#4948)
---
packaging/src/docker/README.md | 36 ++++++++++++++++++++++++---------
packaging/src/docker/docker-compose.yml | 3 +++
2 files changed, 30 insertions(+), 9 deletions(-)
diff --git a/packaging/src/docker/README.md b/packaging/src/docker/README.md
index ebcbfcaf617..e452013aac5 100644
--- a/packaging/src/docker/README.md
+++ b/packaging/src/docker/README.md
@@ -57,18 +57,19 @@ For a quick start, launch the Metastore with Derby,
Everything would be lost when the service is down. In order to save the Hive
table's schema and data, start the container with an external Postgres and
Volume to keep them,
```shell
- docker run -d -p 9083:9083 --env SERVICE_NAME=metastore \
- --env DB_DRIVER=postgres \
+ docker run -d -p 9083:9083 --env SERVICE_NAME=metastore --env
DB_DRIVER=postgres \
--env
SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=org.postgresql.Driver
-Djavax.jdo.option.ConnectionURL=jdbc:postgresql://postgres:5432/metastore_db
-Djavax.jdo.option.ConnectionUserName=hive
-Djavax.jdo.option.ConnectionPassword=password" \
--mount source=warehouse,target=/opt/hive/data/warehouse \
+ --mount type=bind,source=`mvn help:evaluate
-Dexpression=settings.localRepository -q
-DforceStdout`/org/postgresql/postgresql/42.5.1/postgresql-42.5.1.jar,target=/opt/hive/lib/postgres.jar
\
--name metastore-standalone apache/hive:${HIVE_VERSION}
```
If you want to use your own `hdfs-site.xml` or `yarn-site.xml` for the
service, you can provide the environment variable `HIVE_CUSTOM_CONF_DIR` for
the command. For instance, put the custom configuration file under the
directory `/opt/hive/conf`, then run,
```shell
- docker run -d -p 9083:9083 --env SERVICE_NAME=metastore \
- --env DB_DRIVER=postgres -v /opt/hive/conf:/hive_custom_conf --env
HIVE_CUSTOM_CONF_DIR=/hive_custom_conf \
+ docker run -d -p 9083:9083 --env SERVICE_NAME=metastore --env
DB_DRIVER=postgres \
+ -v /opt/hive/conf:/hive_custom_conf --env
HIVE_CUSTOM_CONF_DIR=/hive_custom_conf \
+ --mount type=bind,source=`mvn help:evaluate
-Dexpression=settings.localRepository -q
-DforceStdout`/org/postgresql/postgresql/42.5.1/postgresql-42.5.1.jar,target=/opt/hive/lib/postgres.jar
\
--name metastore apache/hive:${HIVE_VERSION}
```
@@ -81,7 +82,7 @@ then add "--env SCHEMA_COMMAND=upgradeSchema" to the command.
Launch the HiveServer2 with an embedded Metastore,
```shell
- docker run -d -p 10000:10000 -p 10002:10002 --env SERVICE_NAME=hiveserver2
--name hiveserver2-standalone apache/hive:${HIVE_VERSION}
+ docker run -d -p 10000:10000 -p 10002:10002 --env SERVICE_NAME=hiveserver2
--name hiveserver2-standalone apache/hive:${HIVE_VERSION}
```
or specify a remote Metastore if it's available,
```shell
@@ -109,12 +110,24 @@ To skip schematool initialisation or upgrade for
metastore use `IS_RESUME="true"
- HiveServer2, Metastore
-To get a quick overview of both HiveServer2 and Metastore, you can run:
+To get a quick overview of both HiveServer2 and Metastore, there is a
`docker-compose.yml` placed under `packaging/src/docker` for this purpose,
+specify the `POSTGRES_LOCAL_PATH` first:
+```shell
+export POSTGRES_LOCAL_PATH=your_local_path_to_postgres_driver
+```
+Example:
+```shell
+mvn dependency:copy -Dartifact="org.postgresql:postgresql:42.5.1" && \
+export POSTGRES_LOCAL_PATH=`mvn help:evaluate
-Dexpression=settings.localRepository -q
-DforceStdout`/org/postgresql/postgresql/42.5.1/postgresql-42.5.1.jar
+```
+If you don't install maven or have problem in resolving the postgres driver,
you can always download this jar yourself,
+change the `POSTGRES_LOCAL_PATH` to the path of the downloaded jar.
+Then,
```shell
- cd packaging/src/docker
- docker compose up -d
- ```
+docker compose up -d
+```
+HiveServer2, Metastore and Postgres services will be started as a consequence.
Volumes are used to persist data generated by Hive inside Postgres and
HiveServer2 containers,
- hive_db
@@ -123,6 +136,11 @@ Volumes are used to persist data generated by Hive inside
Postgres and HiveServe
The volume stores tables' files inside HiveServer2 container.
+To stop/remove them all,
+```shell
+docker compose down
+```
+
#### Usage
- HiveServer2 web
diff --git a/packaging/src/docker/docker-compose.yml
b/packaging/src/docker/docker-compose.yml
index 6f9560b8952..d0602f08f5c 100644
--- a/packaging/src/docker/docker-compose.yml
+++ b/packaging/src/docker/docker-compose.yml
@@ -32,6 +32,9 @@ services:
- '9083:9083'
volumes:
- warehouse:/opt/hive/data/warehouse
+ - type: bind
+ source: ${POSTGRES_LOCAL_PATH}
+ target: /opt/hive/lib/postgres.jar
hiveserver2:
image: apache/hive:${HIVE_VERSION}
