Re: [OSM-talk] HTTPS all the Things (Automated Edit)
On 2/22/2019 3:48 PM, Mike N wrote: On 2/22/2019 3:36 PM, Jmapb wrote: IMO the value of an automated edit when there's already a redirect in place is minimal enough that I don't think it justifies bumping the version and modification date. Just my opinion. The value of the automated edit is that there is a small improvement in security. Assuming that someone ever clicks on a link in our data, it is more secure to go directly to the HTTPS site rather than start with the HTTP site. True, that's exactly why I update these when I find them. I don't think it warrants an automated edit, but there's room on the map for those who do ;) J ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] HTTPS all the Things (Automated Edit)
On 2/22/2019 3:36 PM, Jmapb wrote: IMO the value of an automated edit when there's already a redirect in place is minimal enough that I don't think it justifies bumping the version and modification date. Just my opinion. The value of the automated edit is that there is a small improvement in security. Assuming that someone ever clicks on a link in our data, it is more secure to go directly to the HTTPS site rather than start with the HTTP site. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] HTTPS all the Things (Automated Edit)
On 2/22/2019 2:02 AM, Bryce Jasmer wrote: I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..."; or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value. This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-) Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests. Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding. The wiki page is https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above. The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743 The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775 I welcome your input. Hi Bryce -- I've been observing these automated changes around NYC. I'd like to humbly request you run these sorts of large projects by the Talk-US mailing list before implementation, since there are many mappers (dozens of us!) who don't choose to spend time on Slack. (Apologies if you did post and I missed it -- that's bound to happen sometimes.) Personally I've been updating these tags to https manually as I come across them (sometimes prompted by Keepright), IF I can verify that the business (etc) in question is still a going concern and still located in the same place. IMO the value of an automated edit when there's already a redirect in place is minimal enough that I don't think it justifies bumping the version and modification date. Just my opinion. (Also, why are you adding a trailing slash to everything?) Thanks for posting your code -- I'm contemplating an automated import of my own and I've been meaning to browse some modern bot code to get me started. Jason ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] HTTPS all the Things (Automated Edit)
Good point Stephan about protocol-less urls being left to the "browser" using the same protocol as it is currently using. But I think my approach is pretty sound in that I'll only update the value if there is a redirect from http to https. I did a sample of a dozen websites that don't redirect and tried out the https version of their site. 100% of them were broken. So I can't assume https, but trying http and looking for a published redirect seems pretty sensible to me. Thanks for the feedback. On Fri, Feb 22, 2019 at 12:55 AM Stephan Knauss wrote: > Hi, > > Please be aware that protocol independent URLs do not mean that http is > used. The client will simply continue using the protocol it used before. > > Real need for that is quite limited. So in most cases they are better > written as https. > > But it then needs to be changed where the URL is used and not on the > provider end. > > Stephan > > > On February 22, 2019 8:02:20 AM GMT+01:00, Bryce Jasmer > wrote: >> >> I have written a script that will search for OSM objects that have a >> website tag that explicitly states "http://..."; or implicitly uses http >> by leaving of the protocol specification. The script will then loop through >> all that it discovers and asks the http site if it will redirect me to the >> secure version of the website over the https protocol. If it does, I will >> update the database with the new value. >> >> This has a couple of advantages. From now through the end of time, any >> user clicking on one of those links will be spared the time it takes to >> establish the connection, ask if there is a secure version of the site, and >> tear down the connection. It's on the order of 10-200 ms to do, but over >> the life of the link and the number of objects that are clicked and the >> population, this could save centuries of time :-) >> >> Another advantage is that it will make https more pervasive and hopefully >> people will start thinking https and forgetting all about http. A more >> secure internet is in all of our best interests. >> >> Anyway, I'd like to (slowly) run this across the planet. I've discussed >> this on the US Slack channel and have performed the actions on the United >> States already. I've addressed many questions and have heard no strong >> objections. I'm seeking feedback from the larger community now before >> proceeding. >> >> The wiki page is >> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz >> >> The Slack conversation is available, but has died down and the transcript >> is available at the wiki page mentioned above. >> >> The diary entry with some more conversation is at the bot's page: >> https://www.openstreetmap.org/user/b-jazz-bot/diary/47743 >> >> The source code is available on GitLab for review: >> https://gitlab.com/b-jazz/https_all_the_things >> >> Example changeset for a run over the "9yfd" geohash: >> https://www.openstreetmap.org/changeset/67454775 >> >> I welcome your input. >> >> >> ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] HTTPS all the Things (Automated Edit)
s basically you copied this? https://www.eff.org/https-everywhere On Fri., Feb. 22, 2019, 2:05 a.m. Bryce Jasmer, wrote: > I have written a script that will search for OSM objects that have a > website tag that explicitly states "http://..."; or implicitly uses http > by leaving of the protocol specification. The script will then loop through > all that it discovers and asks the http site if it will redirect me to the > secure version of the website over the https protocol. If it does, I will > update the database with the new value. > > This has a couple of advantages. From now through the end of time, any > user clicking on one of those links will be spared the time it takes to > establish the connection, ask if there is a secure version of the site, and > tear down the connection. It's on the order of 10-200 ms to do, but over > the life of the link and the number of objects that are clicked and the > population, this could save centuries of time :-) > > Another advantage is that it will make https more pervasive and hopefully > people will start thinking https and forgetting all about http. A more > secure internet is in all of our best interests. > > Anyway, I'd like to (slowly) run this across the planet. I've discussed > this on the US Slack channel and have performed the actions on the United > States already. I've addressed many questions and have heard no strong > objections. I'm seeking feedback from the larger community now before > proceeding. > > The wiki page is > https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz > > The Slack conversation is available, but has died down and the transcript > is available at the wiki page mentioned above. > > The diary entry with some more conversation is at the bot's page: > https://www.openstreetmap.org/user/b-jazz-bot/diary/47743 > > The source code is available on GitLab for review: > https://gitlab.com/b-jazz/https_all_the_things > > Example changeset for a run over the "9yfd" geohash: > https://www.openstreetmap.org/changeset/67454775 > > I welcome your input. > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] HTTPS all the Things (Automated Edit)
Hi, Please be aware that protocol independent URLs do not mean that http is used. The client will simply continue using the protocol it used before. Real need for that is quite limited. So in most cases they are better written as https. But it then needs to be changed where the URL is used and not on the provider end. Stephan On February 22, 2019 8:02:20 AM GMT+01:00, Bryce Jasmer wrote: >I have written a script that will search for OSM objects that have a >website tag that explicitly states "http://..."; or implicitly uses http >by >leaving of the protocol specification. The script will then loop >through >all that it discovers and asks the http site if it will redirect me to >the >secure version of the website over the https protocol. If it does, I >will >update the database with the new value. > >This has a couple of advantages. From now through the end of time, any >user >clicking on one of those links will be spared the time it takes to >establish the connection, ask if there is a secure version of the site, >and >tear down the connection. It's on the order of 10-200 ms to do, but >over >the life of the link and the number of objects that are clicked and the >population, this could save centuries of time :-) > >Another advantage is that it will make https more pervasive and >hopefully >people will start thinking https and forgetting all about http. A more >secure internet is in all of our best interests. > >Anyway, I'd like to (slowly) run this across the planet. I've discussed >this on the US Slack channel and have performed the actions on the >United >States already. I've addressed many questions and have heard no strong >objections. I'm seeking feedback from the larger community now before >proceeding. > >The wiki page is >https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz > >The Slack conversation is available, but has died down and the >transcript >is available at the wiki page mentioned above. > >The diary entry with some more conversation is at the bot's page: >https://www.openstreetmap.org/user/b-jazz-bot/diary/47743 > >The source code is available on GitLab for review: >https://gitlab.com/b-jazz/https_all_the_things > >Example changeset for a run over the "9yfd" geohash: >https://www.openstreetmap.org/changeset/67454775 > >I welcome your input. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
