regarding HDFS-RAID
-------------------
Key: HDFS-2600
URL: https://issues.apache.org/jira/browse/HDFS-2600
Project: Hadoop HDFS
Issue Type: Improvement
Components: contrib/raid
Affects Versions: 0.21.0
Reporter: kiupbyun
Priority: Critical
Fix For: 0.21.0
I've tried to do HDFS-RAID over hadoop-0.21.0 ...
But, It's failed to raid when I've tried to start raidnode (start-raidnode.sh)
What do I have to correct it?
This is the information. Would you tell me how to correct it?
Namenode : xxx.23.129.40:9006
Datanode: xxx.23.129.39, 38
Raidnode : xxx.23.129.40 ( I want to seperate raidnoe from namenode, how can I
do?)
* hdfs-site.xml
<name>hdfs.raid.locations</name>
<value>hdfs://xxx.23.129.40:9006/raid</value>
=> (What's the meaning of this, which address do I have to write down?)
<description>The location for parity files. If this is
is not defined, then defaults to /raid.
</description>
</property>
* raid.xml
<configuration>
<srcPath prefix="hdfs://xxx.23.129.40:9006/TEST">
=> (What's the meaning of this, which address do I have to write down?)
<policy name = "TEST"> => (What's this? )
<property>
<name>srcReplication</name>
<value>3</value>
<description> pick files for RAID only if their replication factor is
greater than or equal to this value.
</description>
</property>
<property>
<name>targetReplication</name>
<value>2</value>
<description> after RAIDing, decrease the replication factor of a
file to
this value.
</description>
</property>
<property>
<name>metaReplication</name>
<value>2</value>
<description> the replication factor of the RAID meta file
</description>
</property>
<property>
<name>modTimePeriod</name>
<value>100</value>
<description> time (milliseconds) after a file is modified to make it
a
candidate for RAIDing
</description>
</property>
*log: when start-raidnode.sh
/************************************************************
SHUTDOWN_MSG: Shutting down RaidNode at test40/xxx.23.129.40
************************************************************/
2011-11-25 23:59:13,417 INFO org.apache.hadoop.raid.RaidNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting RaidNode
STARTUP_MSG: host = test40/xxx.23.129.40
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.21.0
************************************************************/
2011-11-25 23:59:13,518 ERROR org.apache.hadoop.raid.ConfigManager: Reloading
config
file /KTBHOME/hadoop/hadoop/conf/raid.xml
2011-11-25 23:59:13,611 INFO org.apache.hadoop.security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
2011-11-25 23:59:13,647 WARN org.apache.hadoop.conf.Configuration:
mapred.task.id is
deprecated. Instead, use mapreduce.task.attempt.id
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager:
TEST.srcReplication = 3
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager:
TEST.targetReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager:
TEST.metaReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager:
TEST.modTimePeriod = 100
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.targetReplication = 1
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.metaReplication = 2
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.modTimePeriod = 100
2011-11-25 23:59:13,728 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2011-11-25 23:59:13,751 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing
RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,810 INFO org.apache.hadoop.ipc.metrics.RpcDetailedMetrics:
Initializing RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,812 INFO org.apache.hadoop.raid.RaidNode: RaidNode up at:
/127.0.0.1:60000
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder:
starting
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on
60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0
on
60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1
on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2
on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3
on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4
on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5
on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6
on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7
on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8
on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9
on
60000: starting
2011-11-25 23:59:13,818 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy
Filter TEST hdfs://xxx.23.129.40:9006/TEST
2011-11-25 23:59:13,819 INFO org.apache.hadoop.raid.RaidNode: Started archive
scan
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: No filtered paths
for
policy TEST
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy
Filter k_table1 hdfs://xxx.23.129.40:9006
2011-11-25 23:59:13,829 INFO org.apache.hadoop.raid.RaidNode: No filtered paths
for
policy k_table1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
