[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello, We are looking into the issue but we don't have a position to share on how/when this issue will be addressed yet. The Orkut team has debated it a lot with the AdSense team since the beginning of this thread and we'll keep you updated on the progress. Thanks for your input and continued support! - Bruno Oliveira (Google) On Wednesday, July 11, 2012 4:12:58 PM UTC-3, anatoly wrote: +1 On Wednesday, July 11, 2012 9:59:35 PM UTC+3, davew wrote: Hi Bruno, Will the Orkut team work on resolving the Adsense crawler problems? In order for app developers to support Orkut we need to make money on the platform. To make money we need targeted ads. If Google's Adsense crawler can't crawl pages then those ads have very low rpm. At one point I know the Google Adsense for Games team was using Orkut user profile data to target ads on application pages. I know because we use Adsense for Games on BuddyPoke.. But I'm guessing, based on the drop of rpm that Orkut is now completely blocking Adsense's crawlers, including Adsense for Games? Why can't Google Adsense servers access Google Orkut pages? Please help? Thanks Dave @ BuddyPoke On Thursday, April 12, 2012 12:54:30 PM UTC-7, anatoly wrote: Hi Bruno Following this thread discussion let me clarify the main issue: *.orkut.gmodules.com/gadgets urls are behind a login so adsense crawler can't access these pages while trying to crawl adsense ads on app pages. If it is unable to crawl using adsense ads is not effective on app pages. Is there any workaround? It is impossible to define a crawler login for google hosted urls in adsense configuration. thanks On Thursday, March 15, 2012 10:37:56 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, So, I've been discussing this with the engineers here. Adsense crawlers don't crawl over authenticated pages (right?). So, even if /Application.aspx were not blocked, it would not be able to crawl it. So, isn't this warning spurious? That is, the fact that Application.aspx is blocked shouldn't alter your ads quality at all... Or am I mistaken? Thanks! - Bruno Oliveira (Google) On Monday, March 12, 2012 6:42:35 PM UTC-3, Bruno Oliveira (Google) wrote: Our team is looking into the issue with the additional information you provided! I'll update the thread as soon as I hear news. Thanks! - Bruno Oliveira (Google) On Thursday, March 8, 2012 2:39:22 PM UTC-3, anatoly wrote: Hi Bruno thanks for the update Please update this thread if you make any decisions or changes about this. I think it crawls the page where the ad markup appears plus sometimes the parent page if it the former is iframe. Adsense cralwer errors are reported to the publisher, so I can see where it fails. If you need any info let mwe know. thanks On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, Thanks for the explanation, it clarified the finer points of how the adsense crawler works which I wasn't aware of. I assumed it crawled the landing page only, not the page on which the ad appears. We'll look into it. Thanks! - Bruno Oliveira (Google) On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/gFtjHHLr5B4J. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hi Bruno, Will the Orkut team work on resolving the Adsense crawler problems? In order for app developers to support Orkut we need to make money on the platform. To make money we need targeted ads. If Google's Adsense crawler can't crawl pages then those ads have very low rpm. At one point I know the Google Adsense for Games team was using Orkut user profile data to target ads on application pages. I know because we use Adsense for Games on BuddyPoke.. But I'm guessing, based on the drop of rpm that Orkut is now completely blocking Adsense's crawlers, including Adsense for Games? Why can't Google Adsense servers access Google Orkut pages? Please help? Thanks Dave @ BuddyPoke On Thursday, April 12, 2012 12:54:30 PM UTC-7, anatoly wrote: Hi Bruno Following this thread discussion let me clarify the main issue: *.orkut.gmodules.com/gadgets urls are behind a login so adsense crawler can't access these pages while trying to crawl adsense ads on app pages. If it is unable to crawl using adsense ads is not effective on app pages. Is there any workaround? It is impossible to define a crawler login for google hosted urls in adsense configuration. thanks On Thursday, March 15, 2012 10:37:56 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, So, I've been discussing this with the engineers here. Adsense crawlers don't crawl over authenticated pages (right?). So, even if /Application.aspx were not blocked, it would not be able to crawl it. So, isn't this warning spurious? That is, the fact that Application.aspx is blocked shouldn't alter your ads quality at all... Or am I mistaken? Thanks! - Bruno Oliveira (Google) On Monday, March 12, 2012 6:42:35 PM UTC-3, Bruno Oliveira (Google) wrote: Our team is looking into the issue with the additional information you provided! I'll update the thread as soon as I hear news. Thanks! - Bruno Oliveira (Google) On Thursday, March 8, 2012 2:39:22 PM UTC-3, anatoly wrote: Hi Bruno thanks for the update Please update this thread if you make any decisions or changes about this. I think it crawls the page where the ad markup appears plus sometimes the parent page if it the former is iframe. Adsense cralwer errors are reported to the publisher, so I can see where it fails. If you need any info let mwe know. thanks On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, Thanks for the explanation, it clarified the finer points of how the adsense crawler works which I wasn't aware of. I assumed it crawled the landing page only, not the page on which the ad appears. We'll look into it. Thanks! - Bruno Oliveira (Google) On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/5yg9D8cbC9sJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
+1 On Wednesday, July 11, 2012 9:59:35 PM UTC+3, davew wrote: Hi Bruno, Will the Orkut team work on resolving the Adsense crawler problems? In order for app developers to support Orkut we need to make money on the platform. To make money we need targeted ads. If Google's Adsense crawler can't crawl pages then those ads have very low rpm. At one point I know the Google Adsense for Games team was using Orkut user profile data to target ads on application pages. I know because we use Adsense for Games on BuddyPoke.. But I'm guessing, based on the drop of rpm that Orkut is now completely blocking Adsense's crawlers, including Adsense for Games? Why can't Google Adsense servers access Google Orkut pages? Please help? Thanks Dave @ BuddyPoke On Thursday, April 12, 2012 12:54:30 PM UTC-7, anatoly wrote: Hi Bruno Following this thread discussion let me clarify the main issue: *.orkut.gmodules.com/gadgets urls are behind a login so adsense crawler can't access these pages while trying to crawl adsense ads on app pages. If it is unable to crawl using adsense ads is not effective on app pages. Is there any workaround? It is impossible to define a crawler login for google hosted urls in adsense configuration. thanks On Thursday, March 15, 2012 10:37:56 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, So, I've been discussing this with the engineers here. Adsense crawlers don't crawl over authenticated pages (right?). So, even if /Application.aspx were not blocked, it would not be able to crawl it. So, isn't this warning spurious? That is, the fact that Application.aspx is blocked shouldn't alter your ads quality at all... Or am I mistaken? Thanks! - Bruno Oliveira (Google) On Monday, March 12, 2012 6:42:35 PM UTC-3, Bruno Oliveira (Google) wrote: Our team is looking into the issue with the additional information you provided! I'll update the thread as soon as I hear news. Thanks! - Bruno Oliveira (Google) On Thursday, March 8, 2012 2:39:22 PM UTC-3, anatoly wrote: Hi Bruno thanks for the update Please update this thread if you make any decisions or changes about this. I think it crawls the page where the ad markup appears plus sometimes the parent page if it the former is iframe. Adsense cralwer errors are reported to the publisher, so I can see where it fails. If you need any info let mwe know. thanks On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, Thanks for the explanation, it clarified the finer points of how the adsense crawler works which I wasn't aware of. I assumed it crawled the landing page only, not the page on which the ad appears. We'll look into it. Thanks! - Bruno Oliveira (Google) On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/U4lQxpS83ggJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hi Bruno there are several issues: 1) users should not land on /Application.aspx at all. You have an issue with url rewriting in implementation of gadgets.views.requestNavigateTo(canvas_view, params); this call land on /Application.aspx rather than on /Application this should be fixed, I think it influences the crawler. 2) I'm not sure either /Application.aspx or /Application are behind the login. Should they be behind the login? It's on the part of the app to check the viewer, and if it is null it should display a link to install, I think that is the flow explained to us by orkut team over the years, (i.e. there should not be automatic redirect to install page). 3) I'm getting also a lot of adsense crawler errors that the content is behind the login on urls of the type: http://682hjovlc0q8jnnr2ueel60s3q6v6sjc.a.orkut.gmodules.com/gadgets/ifr?url=http ... http://k0bd715biegqe7vcf80atgebi24ng48e.a.orkut.gmodules.com/gadgets* /ifr?url=http*... http://o4c1teg8q13go818l55a4f0shnhr6r12.a.orkut.gmodules.com/gadgets* /ifr?url=http*... http://1kkb0un3av50ln5f3963qgkj20jmneil.a.orkut.gmodules.com/gadgets* /ifr?url=http*http://1kkb0un3av50ln5f3963qgkj20jmneil.a.orkut.gmodules.com/gadgets/ifr?url=http **... etc. Adsense crawler can crawl over authenticated pages if it can land on them, i.e. if there is no redirect, etc. I get thousands of crawler errors from adsense to fix, while you should get them and not orkut developers since it is entirely out of developer's control. Since adsense and orkut is the same company, I think it depends on your teams cooperation to resolve it and improve adsense quality. At the end of the day google would earn more if there are more relevant adsense ads. thanks On Thursday, March 15, 2012 10:37:56 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, So, I've been discussing this with the engineers here. Adsense crawlers don't crawl over authenticated pages (right?). So, even if /Application.aspx were not blocked, it would not be able to crawl it. So, isn't this warning spurious? That is, the fact that Application.aspx is blocked shouldn't alter your ads quality at all... Or am I mistaken? Thanks! - Bruno Oliveira (Google) On Monday, March 12, 2012 6:42:35 PM UTC-3, Bruno Oliveira (Google) wrote: Our team is looking into the issue with the additional information you provided! I'll update the thread as soon as I hear news. Thanks! - Bruno Oliveira (Google) On Thursday, March 8, 2012 2:39:22 PM UTC-3, anatoly wrote: Hi Bruno thanks for the update Please update this thread if you make any decisions or changes about this. I think it crawls the page where the ad markup appears plus sometimes the parent page if it the former is iframe. Adsense cralwer errors are reported to the publisher, so I can see where it fails. If you need any info let mwe know. thanks On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, Thanks for the explanation, it clarified the finer points of how the adsense crawler works which I wasn't aware of. I assumed it crawled the landing page only, not the page on which the ad appears. We'll look into it. Thanks! - Bruno Oliveira (Google) On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/kFVjZUdheDQJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hi Bruno thanks for the update Please update this thread if you make any decisions or changes about this. I think it crawls the page where the ad markup appears plus sometimes the parent page if it the former is iframe. Adsense cralwer errors are reported to the publisher, so I can see where it fails. If you need any info let mwe know. thanks On Thursday, March 8, 2012 7:04:08 PM UTC+2, Bruno Oliveira (Google) wrote: Hello, Thanks for the explanation, it clarified the finer points of how the adsense crawler works which I wasn't aware of. I assumed it crawled the landing page only, not the page on which the ad appears. We'll look into it. Thanks! - Bruno Oliveira (Google) On Wednesday, March 7, 2012 6:32:44 PM UTC-3, anatoly wrote: Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/x3LkHTcMKgUJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Any update on the issue? The minimum orkut team should do is fix the bug of requestNavigateTo implementation Currently requestNavigateTo redirects to /Application.aspx instead of /Application and it blocks adsense crawler. I'd suggest to work closely with adsense team, you're the same company, I can't coordinate your internal cooperation. Just ask Adsense team how much crawler errors do they get on google.com.br, I bet there are billions. Please fix and update, otherwise it makes counterproductive to use adsense on apps. thanks On Tuesday, January 3, 2012 10:08:35 PM UTC+2, Bruno Oliveira (Google) wrote: We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/JVjz_IxsEEsJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
But why would your call to requestNavigateTo() affect the crawler? Sorry if I'm asking something obvious... I don't really know how the Adsense crawler works, as that's not my area :-) - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/XOEomPs4nMQJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
I'm not sure it is related solely to requestNavigateTo there are several app urls 1) orkut links (for example from profile page) http://www.orkut.com/Main#Application?uid=1234appId=1234 2) PublicAppInfo http://www.orkut.com/Main#PublicAppInfo?appId=1234 which redirects to http://www.orkut.com/Main#Application?appId=1234 (which is similar to #1) 3) orkut implementation of OpenSocial method requestNavigateTo goes to *http://www.orkut.com/Application.aspx?appId=1234appParams=bla*http://www.orkut.com/Application.aspx?appId=1234appParams=bla which briefly redirects to http://www.orkut.com/Main#Application.aspx?appId=1234appParams=bla adsense crawler reports errors on http://www.orkut.com/Application.aspx and *http://www.orkut.com/Application* since /Application.aspx and /Application are blocked by robots.txt These are real urls before url rewrite so somehow it might be related to url rewrite on orkut. In any case you can just solve it by adding User-agent: Mediapartners-Google Disallow: to robots txt (or just allow /Application.aspx and /Application for Mediapartners-Google) which will not effect other crawlers It will be in any case for Google's benefit (and for developers benefit). -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/3jd1C-u04yIJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
In any case unrelated to crawlers error you can unify */Main#Application.aspx?appId=1234appParams=bla* ** with */Main#Application?appId=1234appParams=bla* ** as */Main#Application* ** is a standard scheme on orkut and only requestNavigateTo uses Application.aspx -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/8Co_T4warAcJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
We do not plan to modify robots.txt at the moment... but as far as I understand, the crawler would not need to crawl over /Application, because /PublicAppInfo does *not* automatically redirect to /Application unless the user is logged in and has the app installed. Since the crawler is not an Orkut user, this redirection would not happen and it would never see /Application or any other page. At least that's how it *should* happen :-) I've tested /PublicAppInfo myself while not logged in (which mimics the way the adsense crawler would see it) and indeed it does not perform any redirections that I could tell... Thanks! - Bruno Oliveira (Google) -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/-k5lJwdhQnwJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello Bruno Google Adsense crawler crawls the page where the ad tag is placed - it is on the application page which is an iframe on orkut, when the crawler sees that the page is the iframe it attempts to crawl it's parent page (in our case it is /Application, /Application.aspx) which is blocked by robots.txt. Somehow it detects that the page is /Application and not /Main#Application. I suspect the issue might be with url rewriting from /Application to /Main#Application I suggest to at least change requestNavigateTo implementation to redirect to /Main#Application instead of /Main#Application.aspx to make it standard. Otherwise .aspx looks like a redundant orkut url format for pages. -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/Ex2sBNN16W4J. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello, We're still looking into this! Sorry for the delay. Thank you! -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/VjeSoXFiPBYJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello, Just found a possible reason. Are you using PublicAppInfo or AppInfo as a landing page for your ad? The correct is to use PublicAppInfo. If you are using AppInfo, then it's normal to see this error, because AppInfo is not open to the crawler, but PublicAppInfo is. Can you check what your landing page is please? Thanks! -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/NkMjbDggXC4J. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello Bruno I'm using PublicAppInfo everywhere, however I'm also using requestNavigateTo and this method navigates to Application.aspx (which is closed to crawlers) var canvas_view = new gadgets.views.View(canvas); gadgets.views.requestNavigateTo(canvas_view, params); Can you check this please? Based on your info sounds like the bug is in the implementation of requestNavigateTo. Does it make sense? -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/ak2EynZzQzoJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
any updates? thanks -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/ITXQ-2Tvs5QJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Hello, Sorry for our delay. We're still looking into the issue. Thank you! -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/I0uvS6wQ-WMJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
any updates? lately it seems like this forum is abandoned by orkut :( hopefully devs at orkut are just busy... -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/Y4-Bg_SV5JEJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
any updates? thanks -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/pqJxVOtA8WoJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
We're looking into it. Thanks for reporting! -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/YuauDf9gZ2IJ. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Bruno thanks a lot for reply it is related also to *orkut.com/robots.txt*http://orkut.com.br/robots.txt as well as *orkut.com.br/robots.txt* http://orkut.com.br/robots.txt specific lines that block adsense crawler from app pages are: Disallow: /Application Disallow: /Application.aspx Per adsense team suggestion adding User-agent: Mediapartners-Google Disallow: should unblock adsense crawler only and shouldn't have any effect on google crawler or other search bots. -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/CLnUn2ZMK_4J. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
[orkut-developer] Re: robots.txt issue on orkut.com.br - blocking adsense crawler
Anyone from orkut team here who is able to respond? there is a line in *orkut.com.br/robots.txt*http://orkut.com.br/robots.txt which blocks adsense access: Disallow: /Application.aspx thanks -- You received this message because you are subscribed to the Google Groups orkut Developer Forum group. To view this discussion on the web visit https://groups.google.com/d/msg/opensocial-orkut/-/-jhBTjQhqh0J. To post to this group, send email to opensocial-orkut@googlegroups.com. To unsubscribe from this group, send email to opensocial-orkut+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/opensocial-orkut?hl=en.
