[Gluster-infra] [Bug 1367588] Improve the redirection for specific URL for RTD coming from old website
https://bugzilla.redhat.com/show_bug.cgi?id=1367588 Nigel Babu changed: What|Removed |Added Status|NEW |CLOSED Resolution|--- |WONTFIX Last Closed||2018-08-13 00:19:17 --- Comment #4 from Nigel Babu --- I'd like to close this bug as WONT FIX. We should identify gaps in our current docs and file issues to fix them against glusterdocs. -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=mkParRj2lT&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1367588] Improve the redirection for specific URL for RTD coming from old website
https://bugzilla.redhat.com/show_bug.cgi?id=1367588 --- Comment #3 from M. Scherer --- So I did a quick verification on the whole set of logs, and we have since the 26 July around 22 000 hits. # grep /community/documentation/index.php www.gluster.org-access_log* |wc -l 22708 Around 90% of the traffic is bots: # grep /community/documentation/index.php www.gluster.org-access_log* |grep -v g2reader-bot/ | grep -v Slurp\; |grep -vi bingbot |grep -vi googlebot |grep -v Baiduspider/ |grep -v AhrefsBot/ |grep -v MJ12bot/ | grep -v 'Sogou web' |grep -v SeznamBot/ |grep -v electricmonk/ | grep -v 'HaosouSpider;' |grep -v archive.org_bot |grep -v Feedly/1.0 |grep -v SputnikBot/ | grep -v yoozBot |wc -l 2598 I suspect on top of that that there is lots of refresh and duplicate ips # grep /community/documentation/index.php www.gluster.org-access_log* |grep -v g2reader-bot/ | grep -v Slurp\; |grep -vi bingbot |grep -vi googlebot |grep -v Baiduspider/ |grep -v AhrefsBot/ |grep -v MJ12bot/ | grep -v 'Sogou web' |grep -v SeznamBot/ |grep -v electricmonk/ | grep -v 'HaosouSpider;' |grep -v archive.org_bot |grep -v Feedly/1.0 |grep -v SputnikBot/ | grep -v yoozBot |awk '{print $1}' |awk -F: '{print $2}' |sort -u |wc -l 649 Then trying to group by network just show around 600 hits. That's roughly 2 to 3 visitors per day on the wiki. After removing the various hacking attempt (aimed at joomla), the hit on the redirect page itself, the tentative to login for spam, and favicon, we are down to 1500 hits (without deduplication): # grep /community/documentation/index.php www.gluster.org-access_log* |grep -v g2reader-bot/ | grep -v Slurp\; |grep -vi bingbot |grep -vi googlebot |grep -v Baiduspider/ |grep -v AhrefsBot/ |grep -v MJ12bot/ | grep -v 'Sogou web' |grep -v SeznamBot/ |grep -v electricmonk/ | grep -v 'HaosouSpider;' |grep -v archive.org_bot |grep -v Feedly/1.0 |grep -v SputnikBot/ | grep -v yoozBot |grep -v docs-redirect |awk '{print $7}' |grep -v 'Special:UserLogin' |grep -v '&action=history' |grep -v '%22%20h=/' |grep -v /favicon.ico |wc -l 1524 Then the 30 most popular URLs are: [root@supercolony httpd]# grep /community/documentation/index.php www.gluster.org-access_log* |grep -v g2reader-bot/ | grep -v Slurp\; |grep -vi bingbot |grep -vi googlebot |grep -v Baiduspider/ |grep -v AhrefsBot/ |grep -v MJ12bot/ | grep -v 'Sogou web' |grep -v SeznamBot/ |grep -v electricmonk/ | grep -v 'HaosouSpider;' |grep -v archive.org_bot |grep -v Feedly/1.0 |grep -v SputnikBot/ | grep -v yoozBot |grep -v docs-redirect |awk '{print $7}' |grep -v 'Special:UserLogin' |grep -v '&action=history' |grep -v '%22%20h=/' |grep -v /favicon.ico |sort |uniq -c |sort -rn | head -n 30 206 /community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes 143 /community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options 87 /community/documentation/index.php/QuickStart 69 /community/documentation/index.php/Gluster_3.2:_Starting_Gluster_Geo-replication 52 /community/documentation/index.php/Gluster_3.2:_gluster_Command 43 /community/documentation/index.php/Main_Page 37 /community/documentation/index.php/Translators/storage/bdb 37 /community/documentation/index.php/Gluster_3.2:_Monitoring_your_GlusterFS_Workload 36 /community/documentation/index.php/Gluster_3.2:_Terminology 35 /community/documentation/index.php/Gluster_3.2:_Displaying_Volume_Information 29 /community/documentation/index.php/Gluster_3.2:_Expanding_Volumes 24 /community/documentation/index.php/Gluster_3.2:_Manually_Mounting_Volumes 22 /community/documentation/index.php/GlusterFS_Concepts 21 /community/documentation/index.php/Gluster_3.2:_Configuring_Distributed_Striped_Volumes 16 /community/documentation/index.php/User_Guide 16 /community/documentation/index.php/Gluster_3.2:_Tuning_Volume_Options 16 /community/documentation/index.php/Getting_started_overview 15 /community/documentation/index.php/Gluster_3.4:_Brick_Restoration_-_Replace_Crashed_Server 14 /community/documentation/index.php/Gluster_3.1:_Understanding_the_GlusterFS_License 12 /community/documentation/index.php/Translators/performance 12 /community/documentation/index.php/Gluster_Translators 12 /community/documentation/index.php/GlusterHPC_FAQ 12 /community/documentation/index.php/Gluster_3.2:_Manually_Mounting_Volumes_Using_NFS 12 /community/documentation/index.php/Getting_started_test_it_out 10 /community/documentation/index.php/About_GlusterFS_3.3 9 /community/documentation/index.php/Gluster_3.2:_Installing_GlusterFS_on_Red_Hat_Package_Manager_(RPM)_Distributions 9 /community/documentation/index.php/Gluster_3.2:_GlusterFS_Geo-replication_Deployment_Overview 9 /community/documentation/index.php/Documenting_the_undocumented 8 /community/documentation/index.php/MediaWiki:Userlogin 8 /community/documentation/index.php/Gluster_3.2
[Gluster-infra] [Bug 1367588] Improve the redirection for specific URL for RTD coming from old website
https://bugzilla.redhat.com/show_bug.cgi?id=1367588 --- Comment #2 from M. Scherer --- So, looking in more details, for the url Gluster_3.2:_Configuring_Distributed_Striped_Volumes , there is more bots I didn't filtered, and the same ip downloading the page 10 times. The same goes for Gluster_3.1:_Manually_Mounting_Volumes, 27 hits from the same ip in Island, and bots. And ip from the same country ( 2 times ), and 2 indians hits. I suspect that we would need more data to see what should be mapped, and/or make a editorial choice based on existing stuff. Alternatively, someone can decide to revert the complete change and redirection for the time being, but that trading one set of issue for another one. -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=yR7cor2ChD&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://www.gluster.org/mailman/listinfo/gluster-infra
[Gluster-infra] [Bug 1367588] Improve the redirection for specific URL for RTD coming from old website
https://bugzilla.redhat.com/show_bug.cgi?id=1367588 M. Scherer changed: What|Removed |Added Summary|Setup redirection from |Improve the redirection for |gluster.org/community/docum |specific URL for RTD coming |entation to the new RTD |from old website |site| -- You are receiving this mail because: You are on the CC list for the bug. Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=8XLT5uCLZ8&a=cc_unsubscribe ___ Gluster-infra mailing list Gluster-infra@gluster.org http://www.gluster.org/mailman/listinfo/gluster-infra