Re: no mailing list hits in google
On Wed, Aug 28, 2019 at 10:31 PM Alvaro Herrera wrote: > On 2019-Aug-28, Thomas Kellerer wrote: > > > Merlin Moncure schrieb am 28.08.2019 um 18:22: > > > My test case here is the query: pgsql-hackers > > > > That search term is the first hit on DuckDuckGo: > > https://duckduckgo.com/?q=pgsql-hackers+ExecHashJoinNewBatch=h_=web > > Yes, but that's an old post, not the one from this year. > > It does show another interesting point though -- it *also* includes hits from third party list archiving sites, which are *also* gone from Google at this point. And those are definitely not gone from Google because we have a robots.txt blocking /list/ -- it must be something else. //Magnus
Re: no mailing list hits in google
On 2019-Aug-28, Thomas Kellerer wrote: > Merlin Moncure schrieb am 28.08.2019 um 18:22: > > My test case here is the query: pgsql-hackers > > That search term is the first hit on DuckDuckGo: > https://duckduckgo.com/?q=pgsql-hackers+ExecHashJoinNewBatch=h_=web Yes, but that's an old post, not the one from this year. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: no mailing list hits in google
Merlin Moncure schrieb am 28.08.2019 um 18:22: My test case here is the query: pgsql-hackers That search term is the first hit on DuckDuckGo: https://duckduckgo.com/?q=pgsql-hackers+ExecHashJoinNewBatch=h_=web Searching for "postgres ExecHashJoinNewBatch" returns that ot position 4 https://duckduckgo.com/?q=postgres+ExecHashJoinNewBatch=h_=web
Re: no mailing list hits in google
Hi, On 2019-08-28 10:26:35 -0700, Andres Freund wrote: > On August 28, 2019 9:22:44 AM PDT, Merlin Moncure wrote: > >Hackers, > >[apologies if this is the incorrect list or is already discussed > >material] > > Probably should be on the -www list. Redirecting. Please trim in future > replies. > > >I've noticed that mailing list discussions in -hackers and other > >mailing lists appear to not be indexed by google -- at all. > > I noticed that there's fewer and fewer hits too. Pretty annoying. I have an > online archive I can search, but that's not something everyone should have to > do. > > I think it's because robots.txt tells search engines to ignore the lists. > Quite hard to understand how that's a good idea. > > https://www.postgresql.org/robots.txt > > User-agent: * > Disallow: /admin/ > Disallow: /account/ > Disallow: /docs/devel/ > Disallow: /list/ > Disallow: /search/ > Disallow: /message-id/raw/ > Disallow: /message-id/flat/ > > Sitemap: https://www.postgresql.org/sitemap.xml > > Without /list, there's no links to the individual messages. So there needs to > be another external reference for a search engine to arrive at individual > messages. > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. For reasons that I do not understand, the previous mail had a broken html part, making the above message invisible for people viewing the html part. Greetings, Andres Freund
Re: no mailing list hits in google
Hi, On August 28, 2019 9:22:44 AM PDT, Merlin Moncure wrote: >Hackers, >[apologies if this is the incorrect list or is already discussed >material] Probably should be on the -www list. Redirecting. Please trim in future replies. >I've noticed that mailing list discussions in -hackers and other >mailing lists appear to not be indexed by google -- at all. I noticed that there's fewer and fewer hits too. Pretty annoying. I have an online archive I can search, but that's not something everyone should have to do. I think it's because robots.txt tells search engines to ignore the lists. Quite hard to understand how that's a good idea. https://www.postgresql.org/robots.txt User-agent: * Disallow: /admin/ Disallow: /account/ Disallow: /docs/devel/ Disallow: /list/ Disallow: /search/ Disallow: /message-id/raw/ Disallow: /message-id/flat/ Sitemap: https://www.postgresql.org/sitemap.xml Without /list, there's no links to the individual messages. So there needs to be another external reference for a search engine to arrive at individual messages. Andres -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
no mailing list hits in google
Hackers, [apologies if this is the incorrect list or is already discussed material] I've noticed that mailing list discussions in -hackers and other mailing lists appear to not be indexed by google -- at all. We are also not being tracked by any mailing list aggregators -- in contrast to a decade ago where we had nabble and other systems to collect and organize results (tbh, often better than we do) we are now at an extreme disadvantage; mailing list activity was formerly and absolutely fantastic research via google to find solutions to obscure technical problems in the database. Limited access to this information will directly lead to increased bug reports, lack of solution confidence, etc. My test case here is the query: pgsql-hackers ExecHashJoinNewBatch I was searching out a link to recent bug report for copy/paste into corporate email. In the old days this would fire right up but now returns no hits even though the discussion is available in the archives (which I had to find by looking up the specific day the thread was active). Just a heads up. merlin
