Steve Loughran created HADOOP-15384: ---------------------------------------
Summary: distcp numListstatusThreads option doesn't get to -delete scan Key: HADOOP-15384 URL: https://issues.apache.org/jira/browse/HADOOP-15384 Project: Hadoop Common Issue Type: Sub-task Components: tools/distcp Affects Versions: 3.1.0 Reporter: Steve Loughran Assignee: Steve Loughran The distcp {{numListstatusThreads}} option isn't used when configuring the GlobbedCopyListing used in {{CopyComitter.deleteMissing()}} This means that for large scans of object stores, performance is significantly worse. Fix: pass the option down from the task conf -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org