Hi, > It happened once, now I'm seeing happening again: some web crawler
... are a PITA. :| > stumbled across our MoinMoin wiki (1.9.4), and the auto-generated > pages keep sprouting, It's not really (valid) pages, right? But just directories with an empty edit-log I guess? > eventually filling up a directory (32k files): Use "moin ... maint cleanpage" to periodically clean up until you have installed the fix. After seeing that problem getting worse, I recently found and fixed the issue: http://hg.moinmo.in/moin/1.9/rev/6489ec33874d It should at least help for some cases, esp. if there are no write permissions for the visitors creating such pagedirs. It can't help for cases when people (bots) really start editing, but never cancel or save. > According to apache log, the crawler keeps doing > http GET requests like these: > > GET .../HelpOnMacros/MonthCalendar?calparms=HelpOnMacros/ > MonthCalendar,2001,12,2,-51,,,MonthCalendarTemplate HTTP/1.1 Yeah, spidering gone wild. If the bot is too much of a pain, you can try adding it to robots.txt and if that doesn't help, hosts_deny and/or ua_spiders. > Other page creation is restricted by only allowing > a POST request from internal networks, and page creation > is restricted to logged-in (Known) users. What you see is not creation of a real page, it was just a bug in the backend code leading to (more or less) empty pagedirs. > Since the MonthCalendar macro is not actually needed, > perhaps it should just be disabled entirely. What is > the proper way of doing that? As it is a builtin macro, you would either have to remove it from MoinMoin/macro/... or put a macro of same name (but not doing same thing) into data/plugin/macro/... But first try the patch, I guess it should help. BTW, check if your moin has all the security patches needed, see: https://moinmo.in/SecurityFixes Cheers, Thomas ------------------------------------------------------------------------------ Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html _______________________________________________ Moin-user mailing list Moin-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/moin-user
