Thank you very much.
From: "Radar Lei" <r...@pivotal.io>
To: "user" <u...@hawq.incubator.apache.org>
Cc: "dev" <dev@hawq.incubator.apache.org>
Sent: Thursday, May 31, 2018 11:46:09 AM
Subject: Re: how hawq use HA hdfs
Seems like you set the 'dfs.nameservices' as skydata, but not 'dx' which you
defined in hawq-site.xml.
Regards,
Radar
On Thu, May 31, 2018 at 11:22 AM, < xiang....@sky-data.cn > wrote:
I had, here is related configs in hdfs-client.xml:
<property>
<name>dfs.nameservices</name>
<value>skydata</value>
</property>
<property>
<name>dfs.ha.namenodes.skydata</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.skydata.nn1</name>
<value> 192.168.60.24:8020 </value>
</property>
<property>
<name>dfs.namenode.rpc-address.skydata.nn2</name>
<value> 192.168.60.32:8020 </value>
</property>
<property>
<name>dfs.namenode.http-address.skydata.nn1</name>
<value> 192.168.60.24:50070 </value>
</property>
<property>
<name>dfs.namenode.http-address.skydata.nn2</name>
<value> 192.168.60.32:50070 </value>
</property>
From: "Radar Lei" < r...@pivotal.io >
To: "user" < u...@hawq.incubator.apache.org >
Cc: "dev" < dev@hawq.incubator.apache.org >
Sent: Thursday, May 31, 2018 10:27:21 AM
Subject: Re: how hawq use HA hdfs
Have you made changes to HAWQ configuration file ' hdfs-client.xml'?
Regards,
Radar
On Thu, May 31, 2018 at 10:07 AM, < xiang....@sky-data.cn > wrote:
BQ_BEGIN
Change the following parameter in the $GPHOME/etc/hawq-site.xml file:
<property> <name> hawq_dfs_url </name> <value> hdpcluster/hawq_default </value>
<description> URL for accessing HDFS. </description> </property>
In the listing above:
* Replace hdpcluster with the actual service ID that is configured in HDFS.
* Replace /hawq_default with the directory you want to use for storing data
on HDFS. Make sure this directory exists and is writable.
In my case, i think that hdpcluster is "dx" which defined in core-site.xml.
So i use "dx/hawq_default" but failed.
Could anyone help me about this?
Thanks very much.
From: "Radar Lei" < r...@pivotal.io >
To: "user" < u...@hawq.incubator.apache.org >, "dev" <
dev@hawq.incubator.apache.org >
Sent: Thursday, May 31, 2018 9:50:39 AM
Subject: Re: how hawq use HA hdfs
If you are installing a new HAWQ, then file space move is not required.
I think HAWQ will treat the host string as an url unless you configured HAWQ
hdfs HA correctly. So please verify if you missed any other steps.
Regards,
Radar
On Thu, May 31, 2018 at 9:40 AM, < xiang....@sky-data.cn > wrote:
BQ_BEGIN
I think that move filespace is the action that hawq has installed before HA
hdfs,
but in my case, i installed HA hdfs then installed hawq.
In this way, i think there is no need to move filespace.
Besides, my question is that why URI can not work as expected?
Or which URI i should use?
From: "Radar Lei" < r...@pivotal.io >
To: "user" < u...@hawq.incubator.apache.org >
Sent: Wednesday, May 30, 2018 8:30:01 PM
Subject: Re: how hawq use HA hdfs
As the steps in the document you mentioned, you need to finish all the steps to
convert your HAWQ cluster to HA mode.
E.g. Do filespace move, make changes in hdfs-client.xml etc.
Regards,
Radar
On Wed, May 30, 2018 at 7:44 PM, < xiang....@sky-data.cn > wrote:
BQ_BEGIN
Hi!
I has setting HA hdfs using ZKFC with below config:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://dx</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value> 192.168.60.24 : 2181,192.168.60.32 : 2181,192.168.60.37 :2181</value>
</property>
</configuration>
Then i install hawq in another node(not any node installed hdfs).
I follow
http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-incubating/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html
to set `hawq_dfs_url` as "dx/hawq_default", but init failed when check if hdfs
path is available:
[WARNING]:-ERROR: Can not connect to 'hdfs://dx:0'
Then i change `hawq_dfs_url` to "dx:8020/hawq_default", still failed:
ERROR Failed to setup RPC connection to "dx:8020" caused by:
TcpSocket.cpp: 171: HdfsNetworkConnectException: Failed to resolve address
"dx:8020" Name or service not known
It consider "dx" as hostname, which is not expected.
I also test "hdfs://dx", " file://dx ", all failed.
I know that " 192.168.60.24 :8020" can work, but it does not use HA hdfs.
How can hawq use HA hdfs?
Thanks
--
戴翔
南京天数信息科技有限公司
电话: +86 1 3382776490
公司官网: www.sky-data.cn
免费使用天数润科智能计算平台 SkyDiscovery
BQ_END
--
戴翔
南京天数信息科技有限公司
电话: +86 1 3382776490
公司官网: www.sky-data.cn
免费使用天数润科智能计算平台 SkyDiscovery
BQ_END
--
戴翔
南京天数信息科技有限公司
电话: +86 1 3382776490
公司官网: www.sky-data.cn
免费使用天数润科智能计算平台 SkyDiscovery
BQ_END
--
戴翔
南京天数信息科技有限公司
电话: +86 1 3382776490
公司官网: www.sky-data.cn
免费使用天数润科智能计算平台 SkyDiscovery
