Zoltan Borok-Nagy has uploaded this change for review. (
http://gerrit.cloudera.org:8080/23141
Change subject: IMPALA-14018: Adding utility scripts to run Lakekeeper in
Impala dev envinroment
......................................................................
IMPALA-14018: Adding utility scripts to run Lakekeeper in Impala dev envinroment
This patch adds utility scripts to run Lakekeeper (an open source
Iceberg REST Catalog) in Impala's dev environment. Lakekeeper's HDFS
support is in preview phase, so we are using a preview docker image
for now.
IcebergRESTCatalog's config setup is also refactored, and now we don't
always set "credentials" in the SessionContext, only if they are
provided.
Usage
To start Lakekeeper:
testdata/bin/run-lakekeeper.sh
To stop Lakekeeper:
testdata/bin/stop-lakekeeper.sh
Now you can create schemas and tables via Trino (need to rebuild the
Trino image for this, TODO: use docker compose for this):
docker stop impala-minicluster-trino
docker rm impala-minicluster-trino
./testdata/bin/build-trino-docker-image.sh
./testdata/bin/run-trino.sh
Then via Trino CLI:
testdata/bin/trino-cli.sh
show catalogs;
create schema iceberg_lakekeeper.trino_db;
create table trino_t (i int);
insert into trino_t values (35);
After this, you should be able to query the table via Impala:
mkdir /tmp/iceberg_lakekeeper
cp testdata/bin/minicluster_trino/iceberg_lakekeeper.properties
/tmp/iceberg_lakekeeper
bin/start-impala-cluster.py --no_catalogd \
--impalad_args="--catalogd_deployed=false --use_local_catalog=true \
--catalog_config_dir=/tmp/iceberg_lakekeeper/"
bin/impala-shell.sh
Change-Id: I610f5859f92b2ff82e310f46356e3f118e986b2c
---
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergRESTCatalog.java
A testdata/bin/minicluster_lakekeeper/create-default-warehouse.json
A testdata/bin/minicluster_lakekeeper/docker-compose.yaml
M testdata/bin/minicluster_trino/Dockerfile
A testdata/bin/minicluster_trino/iceberg_lakekeeper.properties
A testdata/bin/run-lakekeeper.sh
A testdata/bin/stop-lakekeeper.sh
7 files changed, 266 insertions(+), 31 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/23141/1
--
To view, visit http://gerrit.cloudera.org:8080/23141
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I610f5859f92b2ff82e310f46356e3f118e986b2c
Gerrit-Change-Number: 23141
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
