[Review] bsdconfig pkgng integration
Subject says it all. I need a review on the attached patch. Come on, you know you want to... -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. Index: http.subr === --- http.subr (revision 256550) +++ http.subr (working copy) @@ -79,6 +79,7 @@ f_dialog_menu_media_http() local menu_list= '$msg_main_site' 'ftp.freebsd.org' 'URL' '$msg_specify_some_other_http_site' + 'pkg $msg_main_site' 'pkg.freebsd.org' # END-QUOTE local hline=$msg_select_a_site_thats_close signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On 2013-11-06 13:32, Teske, Devin wrote: Subject says it all. I need a review on the attached patch. Come on, you know you want to... I think you missed something, you attached a 1 line diff -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 3:02 PM, Allan Jude wrote: On 2013-11-06 13:32, Teske, Devin wrote: Subject says it all. I need a review on the attached patch. Come on, you know you want to... I think you missed something, you attached a 1 line diff Oh I see... too much to review. Hey, don't all review it at once now... Seriously... come on... I'm asking for some feedback on the most important part of the patchset... The user-facing element. The least everyone can do is chime in with their favorite mirrors. So... please... Tell me if the mirror list is incomplete (I *know* there are more mirrors than that, and I *know* we want pkg.eu and pkg.us-* ... but I don't actually know what people want to see ... so please... feedback!) -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On 2013-11-06 18:15, Teske, Devin wrote: On Nov 6, 2013, at 3:02 PM, Allan Jude wrote: On 2013-11-06 13:32, Teske, Devin wrote: Subject says it all. I need a review on the attached patch. Come on, you know you want to... I think you missed something, you attached a 1 line diff Oh I see... too much to review. Hey, don't all review it at once now... Seriously... come on... I'm asking for some feedback on the most important part of the patchset... The user-facing element. The least everyone can do is chime in with their favorite mirrors. So... please... Tell me if the mirror list is incomplete (I *know* there are more mirrors than that, and I *know* we want pkg.eu and pkg.us-* ... but I don't actually know what people want to see ... so please... feedback!) pkg.freebsd.org is a SVR record that lists the mirrors, and pkg automatically picks the correct one, so it is best to list only that 1 mirror. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 3:19 PM, Allan Jude wrote: On 2013-11-06 18:15, Teske, Devin wrote: On Nov 6, 2013, at 3:02 PM, Allan Jude wrote: On 2013-11-06 13:32, Teske, Devin wrote: Subject says it all. I need a review on the attached patch. Come on, you know you want to... I think you missed something, you attached a 1 line diff Oh I see... too much to review. Hey, don't all review it at once now... Seriously... come on... I'm asking for some feedback on the most important part of the patchset... The user-facing element. The least everyone can do is chime in with their favorite mirrors. So... please... Tell me if the mirror list is incomplete (I *know* there are more mirrors than that, and I *know* we want pkg.eu and pkg.us-* ... but I don't actually know what people want to see ... so please... feedback!) pkg.freebsd.org is a SVR record This matters not... see SVN r257755 that lists the mirrors, We do not ever want to display the contents of the SVR record in the list. There are multiple pkg*.freebsd.org SVR zones. and pkg automatically picks the correct one, pkg does no such thing. It picks the first one that works. The decisioning for which order and what content gets returned from the SVR query is on the back-end, not on the pkg-end; pkg just takes the first one that works. For example, host -t SVR _http._tcp.pkg.freebsd.org (for me at least) returns 3 entries. pkg goes after the first one. I'd be shocked if you told me that pkg was doing geo-mapping with the 3 names it returns and determining which was was closest. Rather, pkg just does a DNS query and goes through the list of hosts. so it is best to list only that 1 mirror. No. You're thinking that I'm talking about the contents of the SVR query. I'm not. I'm instead talking about... *actual* and *real* multiple names that you can query SVR data from. pkg.freebsd.org is only one There are multiples, and I believe that we should add them. -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Wed, Nov 06, 2013 at 11:28:25PM +, Teske, Devin wrote: We do not ever want to display the contents of the SVR record in the list. There are multiple pkg*.freebsd.org SVR zones. You do not need to show any servers except pkg.FreeBSD.org. In fact, you can use that by default, and not show any servers at all. Do not add more complexity to this than there needs to be. By adding explicit listing of servers here, you are putting a dependency on clusteradm that is not necessary at all. Glen pgpov93NkmbuZ.pgp Description: PGP signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 3:32 PM, Glen Barber wrote: On Wed, Nov 06, 2013 at 11:28:25PM +, Teske, Devin wrote: We do not ever want to display the contents of the SVR record in the list. There are multiple pkg*.freebsd.org SVR zones. You do not need to show any servers except pkg.FreeBSD.org. Bapt *just* got finished telling me in IRC that we should also list: pkg.eu pkg.us-* Which I assume to be the same make/model of pkg.f.o (does not return an A or but only SVR). If that is the case (that pkg.eu.f.o is just like pkg.f.o -- and it just happens to be a name that returns SVR names for servers local to Europe), then I don't see why we would want to omit this. I'm not the only one you have to convince... bapt was the one that brought up the existence of the other names. In fact, you can use that by default, and not show any servers at all. Do not add more complexity to this than there needs to be. By adding explicit listing of servers here, you are putting a dependency on clusteradm that is not necessary at all. I want to get some confusion out of the way... I do not want to add even one single name that resolves to an A or . I want *only* names that resolve to SVR. I've just heard from bapt that pkg.f.o is not the only such name, that there are other such names that similarly do not return A or . I don't see the increase in complexity or maintenance. I'm asking for locale-specific CDN names. The user ought to be able to choose the master CDN or a locale-specific CDN. This is not rocket science. -- Devin Glen _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Wed, Nov 06, 2013 at 11:37:43PM +, Teske, Devin wrote: You do not need to show any servers except pkg.FreeBSD.org. Bapt *just* got finished telling me in IRC that we should also list: pkg.eu pkg.us-* Why? Which I assume to be the same make/model of pkg.f.o (does not return an A or but only SVR). And if we need to change something with the infrastructure, we now have to tell people that this was done. You are putting hard-coded limitations on us. Don't. With a single entry (pkg.FreeBSD.org), we do not have to worry about what is broken if (when) we need to change something. Glen pgp78e6rrcJzB.pgp Description: PGP signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 3:44 PM, Glen Barber wrote: On Wed, Nov 06, 2013 at 11:37:43PM +, Teske, Devin wrote: You do not need to show any servers except pkg.FreeBSD.org. Bapt *just* got finished telling me in IRC that we should also list: pkg.eu pkg.us-* Why? Which I assume to be the same make/model of pkg.f.o (does not return an A or but only SVR). And if we need to change something with the infrastructure, we now have to tell people that this was done. You are putting hard-coded limitations on us. Don't. I disagree. You're making the argument for what if a CDN goes away. Well... people stop using that entry because it's broken and in the next release you remove it. What sounds like a maintenance issue will actually save you support events. And for whatever hours that this proposed maintenance actually cost you, you'll more than make up that in prevented support events. I'm talking about saving time (because we're all volunteers, right?) Let me be very explicit... With a single entry (pkg.FreeBSD.org), we do not have to worry about what is broken if (when) we need to change something. Let's say that the one entry doesn't do what you want (which requires trusting the CDN maintenance on the back-end)... You're telling me that it's more efficient to immediately defer every single user to the support forums the moment there's a single hiccup in the ONE and ONLY CDN name that we're providing? Ok... but let's look at one alternative... If we listed more than one CDN, and one breaks... the wise and tenacious user simply ... tries another? But if there are no other choices... then you'll be headed strait to the forums or mailing lists and generating what I call a support event. I'm sorry, but I don't agree that listing only one single entry is going to be beneficial to the end-user; only to the person that thinks having a single CDN to maintain is going to actually save them work (which may only be true if they are not the ones that have to manage support events). -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Wed, Nov 06, 2013 at 11:53:28PM +, Teske, Devin wrote: And if we need to change something with the infrastructure, we now have to tell people that this was done. You are putting hard-coded limitations on us. Don't. I disagree. You're making the argument for what if a CDN goes away. No, you are thinking about it backwards. If a node goes down (there is no CDN), we take it out of DNS. A single entry, removed from the pkg.FreeBSD.org SRV record. We do not have to redirect anything, we do not have to update various configuration files in various places in the tree. Glen pgpukwIArTYKc.pgp Description: PGP signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 3:53 PM, Teske, Devin wrote: On Nov 6, 2013, at 3:44 PM, Glen Barber wrote: On Wed, Nov 06, 2013 at 11:37:43PM +, Teske, Devin wrote: You do not need to show any servers except pkg.FreeBSD.org. Bapt *just* got finished telling me in IRC that we should also list: pkg.eu pkg.us-* Why? Which I assume to be the same make/model of pkg.f.o (does not return an A or but only SVR). And if we need to change something with the infrastructure, we now have to tell people that this was done. You are putting hard-coded limitations on us. Don't. I disagree. You're making the argument for what if a CDN goes away. Well... people stop using that entry because it's broken and in the next release you remove it. What sounds like a maintenance issue will actually save you support events. And for whatever hours that this proposed maintenance actually cost you, you'll more than make up that in prevented support events. I'm talking about saving time (because we're all volunteers, right?) Let me be very explicit... With a single entry (pkg.FreeBSD.org), we do not have to worry about what is broken if (when) we need to change something. Let's say that the one entry doesn't do what you want (which requires trusting the CDN maintenance on the back-end)... You're telling me that it's more efficient to immediately defer every single user to the support forums the moment there's a single hiccup in the ONE and ONLY CDN name that we're providing? Ok... but let's look at one alternative... If we listed more than one CDN, and one breaks... the wise and tenacious user simply ... tries another? But if there are no other choices... then you'll be headed strait to the forums or mailing lists and generating what I call a support event. I'm sorry, but I don't agree that listing only one single entry is going to be beneficial to the end-user; only to the person that thinks having a single CDN to maintain is going to actually save them work (which may only be true if they are not the ones that have to manage support events). I think there's a lot of misunderstanding going on because of the way that clusteradm has set up our CDN. It is true that right now, pkg.f.o returns SRV records for all known mirrors. It is also true that previously mentioned pkg.eu.f.o returns the one and only SRV record for the one mirror in Europe. However, I am planning for a day when our CDN is administered in a more traditional fashion. That is... I envisage pkg.f.o not returning all known mirrors (because there will be too many as the CDN grows), but instead pkg.f.o would return only the 3 closest mirrors. Meanwhile, pkg.eu.f.o would grow to return records for the top-3 closest servers in Europe. Just because the amount of machines that we have available for our CDN is small, doesn't mean that it will be small forever. Eventually (as is the plan), the list of servers should far exceed the number of SRV records you will want to return. And when we get to that point, I highly recommend that we allow the user to choose the CDN specific to their country (at the very least). -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 4:01 PM, Glen Barber wrote: On Wed, Nov 06, 2013 at 11:53:28PM +, Teske, Devin wrote: And if we need to change something with the infrastructure, we now have to tell people that this was done. You are putting hard-coded limitations on us. Don't. I disagree. You're making the argument for what if a CDN goes away. No, you are thinking about it backwards. If a node goes down (there is no CDN), we take it out of DNS. A single entry, removed from the pkg.FreeBSD.org SRV record. We do not have to redirect anything, we do not have to update various configuration files in various places in the tree. Please help me to understand why on Earth you think: [I am] putting hard-coded limitations on [you] The facts: 1. The DNS names I am talking about: 1.a. pkg.f.o 1.b. pkg.eu.f.o 2. Do not resolve to an A or record So what hard-coding are you talking about? You talk about how if a node goes down we take it out of DNS but that has absolutely nothing to do with me because I'm not putting A/-resolving names in the menu. You do realize don't you that pkg.eu.f.o is a locale-specific name that will eventually hold potentially many-more European server names, right? You do realize that the actual European server is *NOT* pkg.eu, right? You do realize that while the name pkg.f.o may wholly encompass all the mirrors, that this will not always be true, right? I have no idea what you're talking about with the updating of config files. The only reason you would ever have to update a config file is if pkg.eu.f.o went away, but ... Now I will repeat... pkg.eu.f.o is NOT A REAL SERVER It is a name, that resolves to the SRV record of the European server. That being said... that name should not go away unless we no longer have even one single server in Europe. Oh, and by the way... The name pkg.f.o does *NOT* resolve to pkg.eu... pkg.eu is a sibling name that is disassociated -- it was actually *designed* to be used for this. -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Thu, Nov 07, 2013 at 12:13:49AM +, Teske, Devin wrote: So what hard-coding are you talking about? You are trying to hard-code hostnames for a service in the FreeBSD src/ tree, when it is *absolutely* unnecessary. You talk about how if a node goes down we take it out of DNS but that has absolutely nothing to do with me because I'm not putting A/-resolving names in the menu. I did not once say anything about A or records. You do realize don't you that pkg.eu.f.o is a locale-specific name that will eventually hold potentially many-more European server names, right? So? You do realize that the actual European server is *NOT* pkg.eu, right? *Sigh*... You do realize that while the name pkg.f.o may wholly encompass all the mirrors, that this will not always be true, right? No, you are wrong. I have no idea what you're talking about with the updating of config files. Clearly. If you hard-code anything other than pkg.FreeBSD.org in bsdconfig, now clusteradm has to become aware of it, and make sure that record *always* exists, no matter what the endpoint is. This nonsense happened with sysinstall, and anything else that used the FTP mirror list. And when a node disappears, for whatever reason, it is an absolute nightmare to sort out. For the last time, you do not need to have *any* host entries other than 'pkg.FreeBSD.org'. Period. That being said... that name should not go away unless we no longer have even one single server in Europe. This has nothing to do with bsdconfig. Try to see my larger point. Oh, and by the way... The name pkg.f.o does *NOT* resolve to pkg.eu... pkg.eu is a sibling name that is disassociated -- it was actually *designed* to be used for this. So was pkg.FreeBSD.org. It is why I do *NOT* want you to hard-code anything other than that. Glen pgpIrm4IqoZND.pgp Description: PGP signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 4:22 PM, Glen Barber wrote: On Thu, Nov 07, 2013 at 12:13:49AM +, Teske, Devin wrote: So what hard-coding are you talking about? You are trying to hard-code hostnames for a service in the FreeBSD src/ tree, when it is *absolutely* unnecessary. Not all hostnames are equal. As you know, pkg.freebsd.org is one that you would never deny. So just as pkg.freebsd.org is different from a normal hostname, pkg.eu.freebsd.org and pkg.us-east.freebsd.org and pkg.us-west.freebsd.org are similarly different (in the same way). I'm not trying to hard-code any ol' hostnames... I'm trying to hardcode the hostnames which can be administered to point to one or more grouped servers. You talk about how pkg.freebsd.org is unique because we can simply change where it points... I'm talking about doing the same thing for the other 3 names we have which were designed for the same thing. I think that you're thinking that: pkg.eu.freebsd.org pkg.us-east.freebsd.org pkg.us-west.freebsd.org Will somehow change? The only way they would change is in the same way that you would change pkg.freebsd.org -- I don't know how much clearer I can make this. You talk about how if a node goes down we take it out of DNS but that has absolutely nothing to do with me because I'm not putting A/-resolving names in the menu. I did not once say anything about A or records. You talked about a node going down. A node has a A or quad-A record. The name for that node is served via the SRV record. Read the above paragraph again carefully. I said A/-resolving names That is equivalent to node name and you were talking about what if a node goes down. And my answer to that was... I don't care if a node goes down, because I'm not using node names in the menu. You do realize don't you that pkg.eu.f.o is a locale-specific name that will eventually hold potentially many-more European server names, right? So? So, the name pkg.eu is never going to change. It, like pkg.freebsd.org will just have modified SRV records to track the European nodes. You do realize that the actual European server is *NOT* pkg.eu, right? *Sigh*... Your sign leads me to be concerned that there is a false assumption that the DNS name that serves the SRV records should be a one-to-one mapping. There's absolutely no reason whatsoever to dedicate an entire locale specific domain to one SRV record. That's silly. As we grow, this domain -- which is decidedly locale-specific by way of its actual name -- should be used for what it was intended... a locale abstraction of many-to-one. You do realize that while the name pkg.f.o may wholly encompass all the mirrors, that this will not always be true, right? No, you are wrong. I surely hope I am not. You're saying that: + We'll never have more than one server in Europe. + We'll never have more than one server in the West Coast of the USA. + We'll never have more than one server in the East Coast of the USA. That makes me very sad. Very sad indeed. I envisaged a growth that would have had dozens upon dozens of servers all throughout, and the locale-specific domains would then return geographically based SRV records (of which we administer on the back-end). I have no idea what you're talking about with the updating of config files. Clearly. If you hard-code anything other than pkg.FreeBSD.org in bsdconfig, now clusteradm has to become aware of it, and make sure that record *always* exists, no matter what the endpoint is. http://lists.freebsd.org/pipermail/freebsd-pkg/2013-October/000107.html According to that mail, clusteradm *should* already be away of the three I mentioned... pkg.eu, pkg.us-east, and pkg.us-west This nonsense happened with sysinstall, and anything else that used the FTP mirror list. And when a node disappears, for whatever reason, it is an absolute nightmare to sort out. I have to continue to beat the drum... You had to do that in sysinstall, because the names that were used there were A/quad-A resolving names (node names). Just as pkg.freebsd.org is not one of those names... pkg.eu and pkg.us-* are not like the names used in sysinstall. So please... tell me again, how or why you would *ever* have to modify pkg.eu or pkg.us-* ??? For the last time, you do not need to have *any* host entries other than 'pkg.FreeBSD.org'. Period. Until you acknowledge that pkg.eu and pkg.us-* are of the same pedigree as pkg.f.o, I assume that you are still confusing this list of names with a traditional list of names like sysinstall had. That being said... that name should not go away unless we no longer have even one single server in Europe. This has nothing to do with bsdconfig. Try to see my larger point. Oh, I agree... I'm looking forward to the big picture when pkg.eu returns more than one SRV record. I'm looking forward to the day that pkg.us-east and pkg.us-west return more than one
Re: [Review] bsdconfig pkgng integration
On Thu, Nov 07, 2013 at 12:48:02AM +, Teske, Devin wrote: I think that you're thinking that: pkg.eu.freebsd.org pkg.us-east.freebsd.org pkg.us-west.freebsd.org Will somehow change? I think you're thinking they *won't* change. Don't assume. Glen pgpYvjMEXknes.pgp Description: PGP signature
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 5:02 PM, Glen Barber wrote: On Thu, Nov 07, 2013 at 12:48:02AM +, Teske, Devin wrote: I think that you're thinking that: pkg.eu.freebsd.org pkg.us-east.freebsd.org pkg.us-west.freebsd.org Will somehow change? I think you're thinking they *won't* change. Don't assume. Well, what *kind* of change? Changes to what they serve should be no problem... but what I was assuming that wouldn't change was their actual names. (e.g. pkg.eu stays pkg.eu). Couldn't care about any other change, because that would percolate at runtime off the DNS servers. -- Devin _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Review] bsdconfig pkgng integration
On Nov 6, 2013, at 5:13 PM, Teske, Devin wrote:
On Nov 6, 2013, at 5:02 PM, Glen Barber wrote:
On Thu, Nov 07, 2013 at 12:48:02AM +, Teske, Devin wrote:
I think that you're thinking that:
pkg.eu.freebsd.org
pkg.us-east.freebsd.org
pkg.us-west.freebsd.org
Will somehow change?
I think you're thinking they *won't* change. Don't assume.
Well, what *kind* of change?
Changes to what they serve should be no problem...
but what I was assuming that wouldn't change was
their actual names. (e.g. pkg.eu stays pkg.eu).
Couldn't care about any other change, because that
would percolate at runtime off the DNS servers.
We concluded that despite the fact that pkg.* are officially
supported names, a POLA issue has been brought to the
top of the discussion.
That is to say, that the concern is that someone may choose
pkg.eu or pkg.us-* and then become astonished if/when
that name mirror is down (precisely because we do not yet
have geodns and because the pkg.{eu,us-*} names are
currently one-to-one mappings).
So in that spirit, we move forward with only the pkg.f.o entry.
--
Devin
_
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
signature.asc
Description: Message signed with OpenPGP using GPGMail
