On Mon, Feb 17, 2014 at 11:52 PM, kişisel Pazar <[email protected]>wrote:
> robots.txt What to write?
>
Wont help.
the best you can do is affect a file at
proxy.appspot.com/{yourdomain}/robots.txt which no engine will look for.
You would need to affect the file at proxy.appspot.com/robots.txt which you
can't do.
It would of been nice whoever created this proxy projects if they included
a generic disallow in robots.txt, but alas they didnt.
> + I wonder if I moved the extension of the site whether it will be
> effective? http://www.evdenevenakliyatfirmalari.pro/tr &
> http://www.istanbulevdenevetasimacilik.net/tr ??
>
Effective to do what? These proxy apps will proxy just about any domain, it
doesnt matter what the domain is.
Moving domain might make it a bit harder to find you - ie a crawler has to
find your site via the proxy, but it doesnt stop it happening.
plus these sites often keep the cache for a long time, so even if you move
site, the content will available via the proxy for a while.
>
>
> 2014-02-17 18:59 GMT+02:00 Barry Hunter <[email protected]>:
>
>> Yes, there are literally thousands of these proxy apps on AppEngine.
>> Debatable if epidemic levels :)
>>
>> https://www.google.com/search?q=inurl:www+site:appspot.com&start=10
>> for a small selection that Google has indexed. Probably not all of them
>> are, but a fair number of those are proxies.
>>
>> ... as noted you can take some steps to make sure your content is not
>> indexed this way
>>
>> the robots meta tag should work quite well. If no html pages are
>> crawled, it shouldnt find images that it can index.
>>
>> The canonical tag, would be ideal, but most proxies will just blindly
>> rewrite that too :(
>>
>>
>> But you can also block your content (by configuring your server) so that
>> no content is even accessible from appengine. Wont help with content
>> already cached by these apps, but will stop new stuff appearing.
>>
>>
>>
>>
>>
>> On Mon, Feb 17, 2014 at 4:46 PM, kişisel Pazar <[email protected]>wrote:
>>
>>> site:absentis64.appspot.com < search pls..
>>>
>>>
>>> https://www.google.com.tr/?gws_rd=cr&ei=CPPsUo7hH4GWtAbw9oDoCQ#q=site:absentis64.appspot.com
>>>
>>> Approximately 63,800 results found..........
>>>
>>>
>>> 63,800 web page google deleted ! ..........
>>>
>>>
>>>
>>>
>>> 17 Şubat 2014 Pazartesi 18:43:12 UTC+2 tarihinde kişisel Pazar yazdı:
>>>
>>>> *The content on this site is my site attracts
>>>> complete.................!!!!*
>>>>
>>>> *https://absentis64.appspot.com/www.istanbulevdenevetasimacilik.net/images/telefon-numarasi.jpg
>>>> <https://absentis64.appspot.com/www.istanbulevdenevetasimacilik.net/images/telefon-numarasi.jpg>*
>>>>
>>>>
>>>> *https://absentis64.appspot.com/www.evdenevenakliyatfirmalari.pro/images/ist-nakliyat.jpg
>>>> <https://absentis64.appspot.com/www.evdenevenakliyatfirmalari.pro/images/ist-nakliyat.jpg>
>>>> *
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 17 Şubat 2014 Pazartesi 01:44:13 UTC+2 tarihinde barryhunter yazdı:
>>>>>
>>>>> As noted its a proxy site.
>>>>>
>>>>> You can easily detect requests coming from this app. The User-Agent
>>>>> header in the request, will make mention of both AppEngine, and the
>>>>> specific app id (absentis64)
>>>>>
>>>>> If you get that, just return a forbidden message.
>>>>>
>>>>> Or can get more tricky and just add
>>>>>
>>>>> <meta name="robots" content="noindex, nofollow">
>>>>>
>>>>> to the responce. Then even if crawlers find their site, they should
>>>>> never index it. But people will still be able to browse.
>>>>>
>>>>>
>>>>> Test your code to make sure not returning that header for legitimate
>>>>> requests!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Feb 14, 2014 at 12:26 PM, kişisel Pazar
>>>>> <[email protected]>wrote:
>>>>>
>>>>>> Hi www.evdenevenakliyatfirmalari.pro domains that are assigned to me
>>>>>> the following web sites have taken away my content of this page I want
>>>>>> closure. If you are interested I would appreciate good work
>>>>>>
>>>>>>
>>>>>>
>>>>>> https://absentis64.appspot.com/https://absentis64.appspot.com/www.
>>>>>> *evdenevenakliyatfirmalari*.*pro*/
>>>>>> <https://www.google.com.tr/search?q=evdenevenakliyatfirmalari.pro&oq=evdenevenakliyatfirmal&aqs=chrome.1.69i57j69i59j0l4.8205j0j4&sourceid=chrome&espv=210&es_sm=122&ie=UTF-8#>
>>>>>>
>>>>>> -
>>>>>>
>>>>>> <http://webcache.googleusercontent.com/search?q=cache:BdtAaEsc3YgJ:https://absentis64.appspot.com/www.evdenevenakliyatfirmalari.pro/+&cd=1&hl=tr&ct=clnk&gl=tr>
>>>>>> -
>>>>>>
>>>>>> *evdenevenakliyatfirmalari*.*pro*/
>>>>>> <https://www.google.com.tr/search?q=evdenevenakliyatfirmalari.pro&oq=evdenevenakliyatfirmal&aqs=chrome.1.69i57j69i59j0l4.8205j0j4&sourceid=chrome&espv=210&es_sm=122&ie=UTF-8#>
>>>>>>
>>>>>> -
>>>>>>
>>>>>> <http://webcache.googleusercontent.com/search?q=cache:BdtAaEsc3YgJ:https://absentis64.appspot.com/www.evdenevenakliyatfirmalari.pro/+&cd=1&hl=tr&ct=clnk&gl=tr>
>>>>>> -
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Google App Engine" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To post to this group, send email to [email protected].
>>>>>> Visit this group at http://groups.google.com/group/google-appengine.
>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>
>>>>>
>>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>>
>>> To post to this group, send email to [email protected].
>>> Visit this group at http://groups.google.com/group/google-appengine.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Google App Engine" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/google-appengine/cGlDEQfd964/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>>
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/google-appengine.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/google-appengine.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.
