Hi Bruno there are several issues: 1) users should not land on /Application.aspx at all. You have an issue with url rewriting in implementation of
gadgets.views.requestNavigateTo(canvas_view, params); this call land on /Application.aspx rather than on /Application this should be fixed, I think it influences the crawler. 2) I'm not sure either /Application.aspx or /Application are behind the login. Should they be behind the login? It's on the part of the app to check the viewer, and if it is null it should display a link to install, I think that is the flow explained to us by orkut team over the years, (i.e. there should not be automatic redirect to install page). 3) I'm getting also a lot of adsense crawler errors that the content is behind the login on urls of the type: http://682hjovlc0q8jnnr2ueel60s3q6v6sjc.a.orkut.gmodules.com/gadgets/ifr?url=http ... http://k0bd715biegqe7vcf80atgebi24ng48e.a.orkut.gmodules.com/gadgets* /ifr?url=http*... http://o4c1teg8q13go818l55a4f0shnhr6r12.a.orkut.gmodules.com/gadgets* /ifr?url=http*... http://1kkb0un3av50ln5f3963qgkj20jmneil.a.orkut.gmodules.com/gadgets* /ifr?url=http*<http://1kkb0un3av50ln5f3963qgkj20jmneil.a.orkut.gmodules.com/gadgets/ifr?url=http> **... etc. Adsense crawler can crawl over authenticated pages if it can land on them, i.e. if there is no redirect, etc. I get thousands of crawler errors from adsense to fix, while you should get them and not orkut developers since it is entirely out of developer's control. Since adsense and orkut is the same company, I think it depends on your teams cooperation to resolve it and improve adsense quality. At the end of the day google would earn more if there are more relevant adsense ads. thanks On Thursday, March 15, 2012 10:37:56 PM UTC+2, Bruno Oliveira (Google) wrote: > Hello, > > So, I've been discussing this with the engineers here. Adsense crawlers > don't crawl over authenticated pages (right?). So, even if > /Application.aspx were not blocked, it would not be able to crawl it. So, > isn't this warning spurious? That is, the fact that Application.aspx is > blocked shouldn't alter your ads quality at all... > > Or am I mistaken? > > Thanks! > > - > Bruno Oliveira (Google) > > > On Monday, March 12, 2012 6:42:35 PM UTC-3, Bruno Oliveira (Google) wrote: >> >> Our team is looking into the issue with the additional information you >> provided! I'll update the thread as soon as I hear news. >> >> Thanks! >> >> - >> Bruno Oliveira (Google) >> >> >> On Thursday, March 8, 2012 2:39:22 PM UTC-3, anatoly wrote: >>> >>> Hi Bruno >>> >>> >>> thanks for the update >>> >>> Please update this thread if you make any decisions or changes about >>> this. >>> I think it crawls the page where the ad markup appears plus sometimes >>> the parent page if it the former is iframe. >>> Adsense cralwer errors are reported to the publisher, so I can see where >>> it fails. If you need any info let mwe know. >>> >>> >>> thanks >>> >>> >>> >>> On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) >>> wrote: >>> >>>> Hello, >>>> >>>> Thanks for the explanation, it clarified the finer points of how the >>>> adsense crawler works which I wasn't aware of. I assumed it crawled the >>>> landing page only, not the page on which the ad appears. We'll look into >>>> it. Thanks! >>>> >>>> - >>>> Bruno Oliveira (Google) >>>> >>>> >>>> On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: >>>>> >>>>> Any update on the issue? >>>>> The minimum orkut team should do is fix the bug of requestNavigateTo >>>>> implementation >>>>> >>>>> Currently requestNavigateTo redirects to /Application.aspx instead of >>>>> /Application and it blocks adsense crawler. >>>>> I'd suggest to work closely with adsense team, you're the same >>>>> company, I can't coordinate your internal cooperation. >>>>> Just ask Adsense team how much crawler errors do they get on >>>>> google.com.br, I bet there are billions. >>>>> >>>>> >>>>> Please fix and update, otherwise it makes counterproductive to use >>>>> adsense on apps. >>>>> >>>>> >>>>> thanks >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) >>>>> wrote: >>>>> >>>>>> We do not plan to modify robots.txt at the moment... but as far as I >>>>>> understand, the crawler would not need to crawl over /Application, >>>>>> because >>>>>> /PublicAppInfo does *not* automatically redirect to /Application >>>>>> unless the user is logged in and has the app installed. >>>>>> >>>>>> Since the crawler is not an Orkut user, this redirection would not >>>>>> happen and it would never see /Application or any other page. At least >>>>>> that's how it *should* happen :-) I've tested /PublicAppInfo myself >>>>>> while not logged in (which mimics the way the adsense crawler would see >>>>>> it) >>>>>> and indeed it does not perform any redirections that I could tell... >>>>>> >>>>>> Thanks! >>>>>> >>>>>> - >>>>>> Bruno Oliveira (Google) >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "orkut Developer Forum" group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/kFVjZUdheDQJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
