Samarth Gupta created FALCON-169:
------------------------------------
Summary: multiple "/" in target for replication for multi target
feed
Key: FALCON-169
URL: https://issues.apache.org/jira/browse/FALCON-169
Project: Falcon
Issue Type: Bug
Components: replication
Environment: QA
Reporter: Samarth Gupta
Assignee: Venkatesh Seetharam
multiple "/" are getting appended to target dir, before concatenating partition
exp postfix.
For example while running single source multi target test, following is the
value being passed to distCp which can be viewed in tasktracker logs:
** for patch from FALCON-163
{quote}
-Dfalcon.include.path=hdfs://gs1001.grid.corp.inmobi.com:54310/localDC/rc/billing/2012/10/01/12/10//ua3
at the bottom of logs is can be seen:
2013-11-05 06:33:20,219 INFO - Inclusion pattern =
hdfs://gs1001.grid.corp.inmobi.com:54310/localDC/rc/billing/2012/10/01/12/10//ua3
(FilteredCopyListing:59)
2013-11-05 06:33:20,219 INFO - Regex pattern =
(hdfs://gs1001\.grid\.corp\.inmobi\.com:54310/localDC/rc/billing/2012/10/01/12/10//ua3/)|(hdfs://gs1001\.grid\.corp\.inmobi\.com:54310/localDC/rc/billing/2012/10/01/12/10//ua3$)
(FilteredCopyListing:60)
2013-11-05 06:33:20,460 INFO - Number of paths considered for copy: 0
(CustomReplicator:57)
2013-11-05 06:33:20,461 INFO - Number of bytes considered for copy: 0 (Actual
number of bytes copied depends on whether any files are skipped or
overwritten.) (CustomReplicator:58)
2013-11-05 06:33:21,212 INFO - DistCp job-id: job_201310290719_0445
(DistCp:146)
2013-11-05 06:33:21,213 INFO - DistCp job may be tracked at:
http://ivoryqa-1.corp.inmobi.com:50030/jobdetails.jsp?jobid=job_201310290719_0445
(DistCp:147)
2013-11-05 06:33:21,213 INFO - To cancel, run the following command: hadoop
job -kill job_201310290719_0445 (DistCp:148)
2013-11-05 06:33:21,213 INFO - Running job: job_201310290719_0445
(JobClient:1315)
2013-11-05 06:33:22,216 INFO - map 0% reduce 0% (JobClient:1328)
2013-11-05 06:33:33,244 INFO - Job complete: job_201310290719_0445
(JobClient:1383)
2013-11-05 06:33:33,252 INFO - Counters: 4 (JobClient:589)
2013-11-05 06:33:33,252 INFO - Job Counters (JobClient:591)
2013-11-05 06:33:33,253 INFO - SLOTS_MILLIS_MAPS=5822 (JobClient:593)
2013-11-05 06:33:33,253 INFO - Total time spent by all reduces waiting
after reserving slots (ms)=0 (JobClient:593)
2013-11-05 06:33:33,254 INFO - Total time spent by all maps waiting after
reserving slots (ms)=0 (JobClient:593)
2013-11-05 06:33:33,255 INFO - SLOTS_MILLIS_REDUCES=0 (JobClient:593)
2013-11-05 06:33:33,307 INFO - No files present in path:
hdfs://ivoryqa-1.corp.inmobi.com:8020/localDC/rc/billing/ua2/2012/10/01/12/10/ua3
(FeedReplicator:146)
2013-11-05 06:33:33,308 INFO - Completed DistCp (FeedReplicator:77)
{quote}
where as if same is run on the current code from trunk, following are the
values in task tracker:
{quote}
-Dfalcon.include.path=hdfs://gs1001.grid.corp.inmobi.com:54310/localDC/rc/billing/2012/10/01/12/10/ua3
{quote}
and replication is successful .....
--
This message was sent by Atlassian JIRA
(v6.1#6144)
