[ https://issues.apache.org/jira/browse/HADOOP-16770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee resolved HADOOP-16770. --------------------------------- Resolution: Invalid You will probably get more suggestions by asking at the user mailing list. > Compare two directories in HDFS filesystem for every 5 mins interval for same > cluster. (smiliar like diff command in linux) > --------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-16770 > URL: https://issues.apache.org/jira/browse/HADOOP-16770 > Project: Hadoop Common > Issue Type: Task > Components: hdfs-client > Affects Versions: 2.10.0 > Reporter: GanGSTR > Priority: Major > > Hi team, > Created two hadoop clusters, one cluster is storing files in new directories > based on TIME based directories are created in Hadoop FileSystem say > /a/b/time/a.txt b.txt.. > For every 5 mins, compare this cluster 1 filesytem for two different > directories whether any new directories with list of files are updated or not > , if its updated in dir 1, then update those files only to be moved to dir 2. > Later those new directories files copied to HDFS cluster 2 file system. > Currently HDFS not supported hdfs dfs -diff command, Any solution for this? > Have tried -copyFromLocal and copyToLocal command, it uses lot of diskspace > while copying local to hdfs & hdfs to local. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org