Mate Szalay-Beko created HBASE-25318:
----------------------------------------
Summary: Config option for IntegrationTestImportTsv where to
generate HFiles to bulkload
Key: HBASE-25318
URL: https://issues.apache.org/jira/browse/HBASE-25318
Project: HBase
Issue Type: Improvement
Components: integration tests
Reporter: Mate Szalay-Beko
Assignee: Mate Szalay-Beko
When IntegrationTestImportTsv is executed against a real distributed cluster,
it is generating HFiles under the working directory of the current hdfs user
executing the tool, before bulkloading it into HBase.
Assuming you encrypt the HBase root directory within HDFS (using HDFS
Transparent Encryption), you can bulkload HFiles only if they sit in the same
encryption zone in HDFS as the HBase root directory itself. If the working
directory of the current user (e.g. /user/hbase) is not in the same encryption
zone as the HBase root directory (e.g. /hbase/data) then you will get an
exception when executing the IntegrationTestImportTsv tool:
{code:java}
2020-11-21 01:06:28,963 ERROR org.apache.hadoop.hbase.regionserver.HRegion:
There was a partial failure due to IO when attempting to load d :
hdfs://mycluster/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc
org.apache.hadoop.ipc.RemoteException(java.io.IOException):
/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc
can't be moved into an encryption zone.
{code}
We should make it configurable where the IntegrationTestImportTsv generates the
HFiles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
