Ryota Egashira created OOZIE-1682:
-------------------------------------
Summary: remove fs.glob.max for FS action
Key: OOZIE-1682
URL: https://issues.apache.org/jira/browse/OOZIE-1682
Project: Oozie
Issue Type: Bug
Reporter: Ryota Egashira
OOZIE-1471 introduces glob support for FS action. glob max value (default to
1000) limits # of files to do operation at one time (if exceed, fail workflow)
in order to protect namenode. but we see cases where user handle more files in
FS action, which requires admin to bump up max value and restart oozie-server
every time. also it's difficult to determine what is a right value.
NN becomes slower in such a case where thousands of mapper do -lsr at the same
time, but it is not the case for oozie FS action.
since NN becomes matured and improved in stability (for example, HDFS-985),
how about removing safeguard on oozie-side to make operation easier?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
