[
https://issues.apache.org/jira/browse/HBASE-27877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725524#comment-17725524
]
Wei-Chiu Chuang edited comment on HBASE-27877 at 5/23/23 6:07 PM:
------------------------------------------------------------------
Try specify -Dfs.defaultFS=ofs://ozone1/ and see if that addresses the issue.
Maybe the solution is to update hbase reference guide stating that the
workaround is to add the parameter whenever the error message is seen.
was (Author: jojochuang):
Try specify -Dfs.defaultFS=ofs://ozone1/ and see if that addresses the issue.
> Hbase ImportTsv doesn't take ofs:// as a FS
> -------------------------------------------
>
> Key: HBASE-27877
> URL: https://issues.apache.org/jira/browse/HBASE-27877
> Project: HBase
> Issue Type: Bug
> Reporter: Pratyush Bhatt
> Priority: Major
> Labels: ozone
>
> While running the bulkLoad command:
> {noformat}
> hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dhbase.fs.tmp.dir=ofs://ozone1/vol1/bucket1/hbase/bulkload
> -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> -Dimporttsv.bulk.output=ofs://ozone1/vol1/bucket1/hbase/test_VerifyHBaseNoWriteBulkloadHDFSQuota/hfiles
> table_dau3f3374e
> ofs://ozone1/vol1/bucket1/hbase/test_VerifyHBaseNoWriteBulkloadHDFSQuota/data.tsv{noformat}
> Getting:
> {noformat}
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|Exception in thread "main"
> java.lang.IllegalArgumentException: Wrong FS:
> ofs://ozone1/vol1/bucket1/hbase/bulkload/partitions_72cbb1f1-d9b6-46a4-be39-e27a427c5842,
> expected: hdfs://ns1{noformat}
> Complete trace:
> {noformat}
> server-resourcemanager-3.1.1.7.1.8.3-339.jar:/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p3.40935426/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-registry-3.1.1.7.1.8.3-339.jar
> 2023-05-22 17:01:19,925|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client
> environment:java.library.path=/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p3.40935426/bin/../lib/hadoop/lib/native
> 2023-05-22 17:01:19,925|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
> 2023-05-22 17:01:19,925|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
> 2023-05-22 17:01:19,925|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.name=Linux
> 2023-05-22 17:01:19,925|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.arch=amd64
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.version=5.4.0-135-generic
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:user.name=hrt_qa
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:user.home=/home/hrt_qa
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:user.dir=/hwqe/hadoopqe
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.memory.free=108MB
> 2023-05-22 17:01:19,926|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.memory.max=228MB
> 2023-05-22 17:01:19,927|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Client environment:os.memory.total=145MB
> 2023-05-22 17:01:19,930|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=ozn-lease16-1.ozn-lease16.root.hwx.site:2181,ozn-lease16-2.ozn-lease16.root.hwx.site:2181,ozn-lease16-3.ozn-lease16.root.hwx.site:2181
> sessionTimeout=30000
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$16/169226355@454e763
> 2023-05-22 17:01:19,942|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> common.X509Util: Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true
> to disable client-initiated TLS renegotiation
> 2023-05-22 17:01:19,952|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes
> 2023-05-22 17:01:19,963|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.ClientCnxn: zookeeper.request.timeout value is 0. feature enabled=
> 2023-05-22 17:01:19,983|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:19 INFO
> zookeeper.Login: Client successfully logged in.
> 2023-05-22 17:01:20,003|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.Login: TGT refresh thread started.
> 2023-05-22 17:01:20,009|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.Login: TGT valid starting at: Mon May 22 13:07:24 UTC 2023
> 2023-05-22 17:01:20,010|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.Login: TGT expires: Tue May 23 13:07:24 UTC 2023
> 2023-05-22 17:01:20,010|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.Login: TGT refresh sleeping until: Tue May 23 08:54:46 UTC 2023
> 2023-05-22 17:01:20,010|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
> 2023-05-22 17:01:20,033|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.ClientCnxn: Opening socket connection to server
> ozn-lease16-1.ozn-lease16.root.hwx.site/172.27.16.139:2181. Will attempt to
> SASL-authenticate using Login Context section 'Client'
> 2023-05-22 17:01:20,039|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.ClientCnxn: Socket connection established, initiating session,
> client: /10.104.25.105:47830, server:
> ozn-lease16-1.ozn-lease16.root.hwx.site/172.27.16.139:2181
> 2023-05-22 17:01:20,045|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:20 INFO
> zookeeper.ClientCnxn: Session establishment complete on server
> ozn-lease16-1.ozn-lease16.root.hwx.site/172.27.16.139:2181, sessionid =
> 0x10b4fe100361851, negotiated timeout = 30000
> 2023-05-22 17:01:22,655|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:22 INFO
> mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
> 2023-05-22 17:01:22,655|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:22 INFO
> mapreduce.HFileOutputFormat2: Looking up current regions for table
> table_dau3f3374e
> 2023-05-22 17:01:22,733|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:22 INFO
> mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
> current region count for all tables
> 2023-05-22 17:01:23,261|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|23/05/22 17:01:23 INFO
> client.ConnectionImplementation: Closing master protocol: MasterService
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|Exception in thread "main"
> java.lang.IllegalArgumentException: Wrong FS:
> ofs://ozone1/vol1/bucket1/hbase/bulkload/partitions_72cbb1f1-d9b6-46a4-be39-e27a427c5842,
> expected: hdfs://ns1
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:788)
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:647)
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:864)
> 2023-05-22 17:01:23,263|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:661)
> 2023-05-22 17:01:23,264|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:598)
> 2023-05-22 17:01:23,264|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:546)
> 2023-05-22 17:01:23,264|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:754)
> 2023-05-22 17:01:23,264|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> 2023-05-22 17:01:23,264|INFO|MainThread|machine.py:203 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|at
> org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:767)
> 2023-05-22 17:01:23,635|INFO|MainThread|machine.py:232 -
> run()||GUID=427ca366-5f0f-426c-a2e5-0c4e12bdae2d|Exit Code: 1
> 2023-05-22 17:01:23,635|INFO|MainThread|machine.py:238 - run()|Command
> /opt/cloudera/parcels/CDH/bin/hbase
> org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dhbase.fs.tmp.dir=ofs://ozone1/vol1/bucket1/hbase/bulkload
> -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> -Dimporttsv.bulk.output=ofs://ozone1/vol1/bucket1/hbase/test_VerifyHBaseNoWriteBulkloadHDFSQuota/hfiles
> table_dau3f3374e
> ofs://ozone1/vol1/bucket1/hbase/test_VerifyHBaseNoWriteBulkloadHDFSQuota/data.tsv
> failed after 0 retries {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
