Hi Kirill, I can’t answer your question about general Google, but I wonder if the Google Scholar inclusion instructions will be useful?
https://scholar.google.com/intl/en/scholar/inclusion.html There have also been several presentations on Google Scholar indexing over the years that you can find on the DSpace Lyrasis Wiki https://wiki.lyrasis.org/display/DSPACE/ - if you search Google Scholar in the upper right corner you’ll start to see the list. Hope this helps. Kimberly Kimberly Chapman Campus Repository Services University of Arizona Libraries [email protected]<mailto:[email protected]> 520-626-1910 From: [email protected] <[email protected]> On Behalf Of Kirill Batyuk Sent: Friday, December 1, 2023 6:55 AM To: [email protected] Subject: [EXT][dspace-community] Robots.txt for google indexing External Email Hello, Could someone please share their robots.txt that only allows for Google to index their DSpace? We are on DSpace 7.2 and just noticed that none of our content is indexed by Google. We do not want to be overloaded by a large number of bots, but it is important to us to have our content discoverable in Google searches. Thank you, Kirill Batyuk [A button for name playback in email signature] <https://www.name-coach.com/kirill-batyuk> Systems Librarian MBLWHOI Library Data Library and Archives Woods Hole Oceanographic Institution 508-289-2850 [email protected]<mailto:[email protected]> mblwhoilibrary.org<https://urldefense.com/v3/__http:/mblwhoilibrary.org/__;!!NkSChLMW7qAT!qzl-Bw_htCSDBNQJszDywSpH7dOtkXO9Vs5MFihCtlGhhw559rfEgq-mCQ8O4cqor2tQ47n6Alhw$> -- whoi.edu<http://whoi.edu/> -- All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx --- You received this message because you are subscribed to the Google Groups "DSpace Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/DM6PR16MB35452DA6E3C3C426B25921E6C281A%40DM6PR16MB3545.namprd16.prod.outlook.com<https://groups.google.com/d/msgid/dspace-community/DM6PR16MB35452DA6E3C3C426B25921E6C281A%40DM6PR16MB3545.namprd16.prod.outlook.com?utm_medium=email&utm_source=footer>. -- All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx --- You received this message because you are subscribed to the Google Groups "DSpace Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/BY5PR19MB356913108BC2F64A65597A12B781A%40BY5PR19MB3569.namprd19.prod.outlook.com.
