[ https://issues.apache.org/jira/browse/FLINK-9541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689936#comment-16689936 ]
Ken Krugler commented on FLINK-9541: ------------------------------------ I'd asked [on the bui...@apache.org|mailto:on%c2%a0the%c2%a0bui...@apache.org] about setting this up, but didn't hear back. Turns out Gavin McDonald had responded... {quote}Ok Ken and anyone else interested. I have updated the robots.txt [1] file to point to a sitemap-index.xml [2] file. So, all you now need to do is ensure you have a flink.xml.gz sitemap in ci.apache.org/projects/flink <http://ci.apache.org/projects/flink> and create a PR against our sitemap-index.xml file, and done, hopefully. {quote} I can create the sitemap file and build the pull request, but it would be good to get some input on what to put in the sitemap. For example, as a first cut it would be easiest to just have [https://ci.apache.org/projects/flink/flink-docs-stable/] as the only docs, as (I assume) that's what we'd want most people to find if they were doing a search without a version number in the query, yes? Maybe [~fhueske] can weigh in here. > Add robots.txt and sitemap.xml to Flink website > ----------------------------------------------- > > Key: FLINK-9541 > URL: https://issues.apache.org/jira/browse/FLINK-9541 > Project: Flink > Issue Type: Improvement > Components: Project Website > Reporter: Fabian Hueske > Priority: Major > > From the [dev mailing > list|https://lists.apache.org/thread.html/71ce1bfbed1cf5f0069b27a46df1cd4dccbe8abefa75ac85601b088b@%3Cdev.flink.apache.org%3E]: > {quote} > It would help to add a sitemap (and the robots.txt required to reference it) > for flink.apache.org and ci.apache.org (for /projects/flink) > You can see what Tomcat did along these lines - > http://tomcat.apache.org/robots.txt references > http://tomcat.apache.org/sitemap.xml, which is a sitemap index file pointing > to http://tomcat.apache.org/sitemap-main.xml > By doing this, you can emphasize more recent versions of docs. There are > other benefits, but reducing poor Google search results (to me) is the > biggest win. > E.g. https://www.google.com/search?q=flink+reducingstate > <https://www.google.com/search?q=flink+reducingstate> (search on flink > reducing state) shows the 1.3 Javadocs (hit #1), master (1.6-SNAPSHOT) > Javadocs (hit #2), and then many pages of other results. > Whereas the Javadocs for 1.5 > <https://ci.apache.org/projects/flink/flink-docs-release-1.5/api/java/org/apache/flink/api/common/state/ReducingState.html> > and 1.4 > <https://ci.apache.org/projects/flink/flink-docs-release-1.4/api/java/org/apache/flink/api/common/state/ReducingState.html> > are nowhere to be found. > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005)