mikemccand commented on issue #1:
URL: 
https://github.com/apache/lucene-jira-archive/issues/1#issuecomment-1173701233

   > I found at least one test issue in the test repo 
https://github.com/mocobeta/sandbox-lucene-10557/issues appears in google's top 
search result. I didn't think it happens so quickly, but I might have to make 
the repo private. If there is anyone who is interested in reviewing/debugging 
this issue, please let me know. I'll give you access to the repo.
   
   Hmm ... we could maybe rename the repository so that it [falls under one of 
the `robots.txt` rules at github.com](https://github.com/robots.txt)?  Reading 
at least one answer on stackoverflow suggested this approach.  Of course, it is 
brittle: if the `robots.txt` changes, the web crawlers will see the content 
again, but maybe for our short-term purposes it is acceptable?
   
   Here's the current `robots.txt` content:
   
   ```
   # If you would like to crawl GitHub contact us via 
https://support.github.com?tags=dotcom-robots
   # We also provide an extensive API: https://docs.github.com
   User-agent: baidu
   crawl-delay: 1
   
   
   User-agent: *
   
   Disallow: /*/pulse
   Disallow: /*/tree/
   Disallow: /gist/
   Disallow: /*/forks
   Disallow: /*/stars
   Disallow: /*/download
   Disallow: /*/revisions
   Disallow: /*/issues/new
   Disallow: /*/issues/search
   Disallow: /*/commits/
   Disallow: /*/commits/*?author
   Disallow: /*/commits/*?path
   Disallow: /*/branches
   Disallow: /*/tags
   Disallow: /*/contributors
   Disallow: /*/comments
   Disallow: /*/stargazers
   Disallow: /*/archive/
   Disallow: /*/blame/
   Disallow: /*/watchers
   Disallow: /*/network
   Disallow: /*/graphs
   Disallow: /*/raw/
   Disallow: /*/compare/
   Disallow: /*/cache/
   Disallow: /.git/
   Disallow: */.git/
   Disallow: /*.git$
   Disallow: /search/advanced
   Disallow: /search
   Disallow: */search
   Disallow: /*q=
   Disallow: /*.atom
   
   Disallow: /ekansa/Open-Context-Data
   Disallow: /ekansa/opencontext-*
   Disallow: */tarball/
   Disallow: */zipball/
   
   Disallow: /*source=*
   Disallow: /*ref_cta=*
   Disallow: /*plan=*
   Disallow: /*return_to=*
   Disallow: /*ref_loc=*
   Disallow: /*setup_organization=*
   Disallow: /*source_repo=*
   Disallow: /*ref_page=*
   Disallow: /*source=*
   Disallow: /*referrer=*
   Disallow: /*report=*
   Disallow: /*author=*
   Disallow: /*since=*
   Disallow: /*until=*
   Disallow: /*commits?author=*
   Disallow: /*report-abuse?report=*
   Disallow: /*tab=*
   Allow: /*?tab=achievements&achievement=*
   
   Disallow: /account-login
   Disallow: /Explodingstuff/
   ```
   
   So maybe if named/renamed this test repo with a prefix of `forks-` or 
`stars-`?  Of course, GitHub might disallow this, but it's worth a shot?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to