You can generate robots.txt as part of your package build, and/or have a scheduled task t oregenerate it. then you can stick it on a fast non-ASPNET application, or lighttpd, or CDN
On Wed, May 20, 2009 at 11:36 PM, Jimmy Shimizu <[email protected]>wrote: > I would agree on first point, seems like unecessary overhead. I just want > the possibility to add certain explicit filetypes without having to fool > around with the config (like site-validation etc). I'm gonna write a > controller-action for robots.txt anyway because I want that automated. > > so basically its *.html for google, aswell as *.ico for favicon (for the > retarded browsers that doesn't support icon-filename in markup) > > > On 20 maj 2009, at 20:37, Ken Egozi wrote: > > 1. isn't it better to have static files not reach MRHHF at all? > 2. I really don't like the "if there's a real file, do not process MR > logic". > on many occasions, when server is not secured correctly, a loophole allows > an attacker to upload files (not change or delete, but add) to the site. > > now if you have a url that triggers an MR action under > "someurl.anextension", and you use the "if there's a file bypass MR", all > an attacker needs to do is to upload a malicious markup file named > someurl.anextension, and you're in really big trouble. > > This kind of attack have happened to me and to people I know on several > shared hosting scenarios. Usually the attacker adds the common default > documents (index.htm) for that, and I solved that problem by removing all > default documents from the IIS. > however the said scenario will open up a whole new bunch of attack > surfaces. > > > > On Wed, May 20, 2009 at 9:06 PM, Henrik Feldt <[email protected]> wrote: > >> >> Yes, it's a part of it, but if monorail also has a hit on the url which >> has >> a static file, you can override the static file with the routing. I moved >> a >> lot of knowledge from the http handler to the routing module. >> >> Currently, routingmoduleex: >> if (File.Exists(request.PhysicalPath)) >> { >> return; // Possibly requesting a static >> file, so we skip routing altogether >> } >> >> But path="*" on mrhttphfac means it will try to find a controller for it >> anyway. Instead of doing this in routingmoduleex I'm returning a route >> with >> a property isstaticfile which then the mrhandler delegates to a static >> file >> handler. Because the handler fac originally was written under the >> assumption >> it won't get static requests, it tries to find a controller/action for >> this, >> which obviously fails causing this annoying exception you mailed about >> before. >> >> I'm still in the testing stages for this, because what I'm after is the >> dynamic content compression in iis7 and I'm not sure how returning a >> custom >> handler for static files affects this. >> >> I extended the classes with a base class and different types of routing >> matches. >> >> Regards, >> Henrik >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Jimmy Shimizu >> Sent: den 20 maj 2009 14:54 >> To: [email protected] >> Subject: Re: robots.txt and sitemaps with routing >> >> >> What do you do when matching a static file? Do you explicitly check the >> filesystem for existance? >> >> I think I'd still keep my static-folder for content that always will be >> static (to avoid unnessecary processing of files through MR), but for >> certain root-files this approach isn't applicable. >> >> Henrik Feldt wrote: >> > Hello Jimmy, >> > >> > What I did, was to modify the routing in the routing module and also >> > add a property "IsStaticFile" on the RouteMatch, so that the routing, >> > which is responsible for rewriting the url, checks for static files, >> > rather than letting MRHttpHandlerFactory do it. What MRHHF does is to >> > check if the route is for static. >> > >> > I also made the routing aware of the controller tree, because in my >> > opinion there's no use in routing to mr handlers if there's no >> > controller/action for the route anyway; then we might fail with >> > ASP.Net's error handling (+ being able to set 404 pages through web >> > admin API in IIS7, or error handlers in web.config in iis6) like >> > default and save ourselves the trouble of mapping every single file >> extension manually in Web.config. >> > >> > Regards, >> > Henrik >> > >> > -----Original Message----- >> > From: [email protected] >> > [mailto:[email protected]] On Behalf Of Jimmy >> > Shimizu >> > Sent: den 20 maj 2009 12:22 >> > To: [email protected] >> > Subject: Re: robots.txt and sitemaps with routing >> > >> > >> > I'm using RoutingModuleEx, basic MonoRail routing that is mapped like >> this: >> > >> > <add name="MonoRail" path="*" verb="*" >> > type="Castle.MonoRail.Framework.MonoRailHttpHandlerFactory, >> > Castle.MonoRail.Framework" preCondition="integratedMode"/> >> > >> > I was under the impression that with this approach, static files are >> > never handled correctly. If that is not the case, I'd love to hear >> > about the solution :) >> > >> > I get Url smaller than 2 tokens if I try to request a static file. >> > >> > Jokin Cuadrado wrote: >> > >> >> What routing are you using? >> >> >> >> the one i use check if a file exist before handling the dynamic url, >> >> so just put a static robot.txt in the root and it will work. >> >> >> >> >> >> On Mon, May 18, 2009 at 12:01 PM, Jimmy Shimizu >> >> <[email protected]> >> >> >> > wrote: >> > >> >> >> >> >> >>> How do you guys solve the issue with files that are supposed to be >> >>> found directly under the siteroot when using advanced routing >> >>> (meaning, catching * with Monorail)? >> >>> >> >>> I was planning on using a specific controller that servers >> >>> robots.txt and sitemaps dynamically, but when someone needs to >> >>> verify for example domain-ownership with analytics or webmastertools >> >>> or such, how do you handle that? >> >>> >> >>> >> >>> >> >> >> >> >> >> >> > >> > >> > >> > > >> > >> >> >> >> >> > > > -- > Ken Egozi. > http://www.kenegozi.com/blog > http://www.delver.com > http://www.musicglue.com > http://www.castleproject.org > http://www.gotfriends.co.il > > > > > > > > -- Ken Egozi. http://www.kenegozi.com/blog http://www.delver.com http://www.musicglue.com http://www.castleproject.org http://www.gotfriends.co.il --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Castle Project Users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/castle-project-users?hl=en -~----------~----~----~----~------~----~------~--~---
