Github user mattf commented on the pull request:
https://github.com/apache/spark/pull/2471#issuecomment-57676552
> Really, this is not an expensive process that will bring down the HDFS
server
i'm not concerned about bringing down HDFS. the operation run from spark or
a log archiving tool should be of equal load on HDFS.
> Finally, Spark theoretically supports Windows
i'm concered that spark becomes a repository of tools that are duplicated
by one platform or another. i'm not much of a windows person, but i'd expect
there are tools that fill the same function as cron and
something-like-logrotate that are native to windows. those tools are likely
familiar to windows admins just as cron&logrotate are for a linux admin.
> AFAIK, logrotate doesn't work on HDFS.
i don't think logrotate works on HDFS either
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
