> -1, I would like to see a proposal on list where it will be permanently available > in the archives instead of being directed to a Google doc which can be edited at any point in time. > This has been an incredibly difficult proposal to keep track of.
This entire SIP process has been incredibly difficult to coordinate. The idea of doing this on a mail thread was a terrible one. There's no archive available publicly for this list that doesn't break the threading functionality (I guess someone among us is using some funky mail client that's messing with Pony Mail). JIRA for discussion would've been much better. I suppose JIRA used to have threaded discussions once upon a time, that would've been very handy here. On Tue, Nov 23, 2021 at 9:07 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Okay, sure, I'll rescind my assertion about having reached a lazy > consensus. I'll work through all the recent comments and feedback. > I think at the last moment, there was an edit by Noble to the Google Doc > that hasn't made it into the SIP document, and I'll review that change, the > recent comments and get back here tomorrow. Other than that last moment > change, the SIP document is up to date. > > On Tue, Nov 23, 2021 at 8:24 PM Gus Heck <gus.h...@gmail.com> wrote: > >> Also, I don't know that it's defined anywhere, but I feel that declaring >> lazy consensus on a SIP in less than a week is rushing things. Some folks >> won't have the flexibility to address things daily and I've been >> uncomfortably pulled away from paid work trying to keep up with this as is. >> Lazy consensus should include at least one weekend (IMHO) >> >> On Tue, Nov 23, 2021 at 9:49 AM Gus Heck <gus.h...@gmail.com> wrote: >> >>> IMHO two things must happen before lazy consensus wait period can begin: >>> 1) The wiki must be updated to reflect the results of discussion in >>> the google doc. >>> 2) The fact that the wiki has been updated, and that you think the >>> discussion is at an end must be posted here, preferably with a summary on >>> the list. >>> >>> If it didn't happen on the list it didn't happen. >>> >>> Also the google doc has not been notifying me of changes/responses. So a >>> note here when you've responded and some time to respond before the above >>> steps are taken would be appreciated. >>> >>> -Gus >>> >>> On Tue, Nov 23, 2021 at 9:40 AM Mike Drob <md...@mdrob.com> wrote: >>> >>>> -1, I would like to see a proposal on list where it will be permanently >>>> available in the archives instead of being directed to a Google doc which >>>> can be edited at any point in time. >>>> >>>> This has been an incredibly difficult proposal to keep track of. >>>> >>>> Mike >>>> >>>> On Tue, Nov 23, 2021 at 6:09 AM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>>>> This proposal has passed with lazy consensus. We can proceed to the >>>>> implementation phase. >>>>> Thanks to everyone for feedback, esp. Gus for the patience. >>>>> >>>>> On Sun, Nov 21, 2021 at 2:24 PM Gus Heck <gus.h...@gmail.com> wrote: >>>>> >>>>>> Thanks for adding that :) and Thanks to Nobel for translating my >>>>>> ravings :). Sorry about the acronym. Amusingly I had to look up gish >>>>>> gallop >>>>>> to understand what you meant :) I can assure you that I will never gish >>>>>> gallop you or anyone on the list, and I find such techniques repugnant. >>>>>> If >>>>>> I'm not making sense, I'm likely covering ground too fast, skipping >>>>>> things >>>>>> that are in my head but not yours, expressing too many tangents (when I >>>>>> start nesting parentheses this is (usually) a sign I'm wandering too >>>>>> much), >>>>>> or I am simply mistaken about something. Always feel free to ask me to >>>>>> explain! >>>>>> >>>>>> -Gus >>>>>> >>>>>> On Sun, Nov 21, 2021 at 12:17 AM Ishan Chattopadhyaya < >>>>>> ichattopadhy...@gmail.com> wrote: >>>>>> >>>>>>> Added both your suggestions to Google Docs for inline commenting and >>>>>>> the SIP document at confluence. Thanks for the feedback. >>>>>>> >>>>>>> On Sun, Nov 21, 2021 at 10:28 AM Ishan Chattopadhyaya < >>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>> >>>>>>>> I'll add the lifecycle details to the document as well. >>>>>>>> >>>>>>>> On Sun, 21 Nov, 2021, 9:08 am Ishan Chattopadhyaya, < >>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Gus, >>>>>>>>> Thanks for bringing it up again. Initially, I wasn't clear what >>>>>>>>> you meant and mistook them for gish gallop. >>>>>>>>> >>>>>>>>> These were the things I am/was confused about (Noble explained it >>>>>>>>> to me what you mean): >>>>>>>>> - adherence to DRY via zk >>>>>>>>> - runtime config info should come from zk >>>>>>>>> - add dynamic features >>>>>>>>> - same interaction patterns for consulting config values. >>>>>>>>> >>>>>>>>> Now, after Noble's help, I think I understand your motivations. >>>>>>>>> - DRY = Don't Repeat Yourself ;-) >>>>>>>>> - You're suggesting that we have a standard way to have role >>>>>>>>> specific configurations (in ZK), so that a new role added later can >>>>>>>>> access >>>>>>>>> the configurations in a standard way >>>>>>>>> >>>>>>>>> This is a very good suggestion, and makes complete sense now. This >>>>>>>>> was definitely a missing piece! >>>>>>>>> >>>>>>>>> Here's a proposal to address that (adding it to Google Docs: >>>>>>>>> https://docs.google.com/document/d/1hijvM1WX9u2TOUdLEkFYVCofLeJlv2MRZqe-ncobJVw/edit >>>>>>>>> ): >>>>>>>>> - Per Role ZNode >>>>>>>>> - Every role's ZNode to hold configuration data (this is the key >>>>>>>>> addition that you're asking for, IIUC) >>>>>>>>> - Children of role ZNodes to be list of ephemeral solr nodes >>>>>>>>> >>>>>>>>> We can iterate over this on the Google Docs (internal >>>>>>>>> representation ZK section) with inline commenting if we need to. >>>>>>>>> >>>>>>>>> Please let us know how that sounds. >>>>>>>>> Regards, >>>>>>>>> Ishan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Nov 21, 2021 at 4:49 AM Gus Heck <gus.h...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> It is difficult to track everything in this thread for sure. >>>>>>>>>> Specifically, I don't feel my email of Wed, Nov 17, 7:43 PM (EST) >>>>>>>>>> has been >>>>>>>>>> addressed or responded to except for one difference of opinion with >>>>>>>>>> Jan on >>>>>>>>>> whether or not we should prohibit the notion of any role ever >>>>>>>>>> changing at >>>>>>>>>> runtime. >>>>>>>>>> >>>>>>>>>> Note that I made a specific proposal for an addition to the sip, >>>>>>>>>> regarding start up lifecycle and adherence to DRY via zk, in the >>>>>>>>>> (probably >>>>>>>>>> not clearly expressed) hope that we can move to a general overall >>>>>>>>>> application principle that at runtime config info should come from >>>>>>>>>> zk. If >>>>>>>>>> we follow such a principle, we will always be in a position to add >>>>>>>>>> dynamic >>>>>>>>>> features without significant rework, and all code could hopefully >>>>>>>>>> reuse the >>>>>>>>>> same interaction patterns for consulting config values. >>>>>>>>>> >>>>>>>>>> Neither item is addressed in the SIP currently. >>>>>>>>>> >>>>>>>>>> Also I had put that out there as a single item so it could be >>>>>>>>>> focused on (simplifying the discussion I hope), and depending on how >>>>>>>>>> discussion proceeds, I may have other follow on items. >>>>>>>>>> >>>>>>>>>> -Gus >>>>>>>>>> >>>>>>>>>> On Sat, Nov 20, 2021 at 8:45 AM Ishan Chattopadhyaya < >>>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, 19 Nov, 2021, 7:03 pm Ilan Ginzburg, <ilans...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Is the request here for everybody to express again the concerns >>>>>>>>>>>> already expressed in this email thread and not addressed? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think the expectation now (after we've expressed our intention >>>>>>>>>>> to reach a lazy consensus) is that if someone wants to block this >>>>>>>>>>> SIP from >>>>>>>>>>> adoption, then to put forward the objections. Sort of like a veto. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I suggest instead the authors review the thread, match expressed >>>>>>>>>>>> concerns with how the concern was addressed (or not addressed) >>>>>>>>>>>> and >>>>>>>>>>>> provide an exhaustive list. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I've summarized most of the discussion in the SIP document. If >>>>>>>>>>> you feel I could do a better job with it, please help me with areas >>>>>>>>>>> of >>>>>>>>>>> improvement. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> This proposal in its current form (data and overseer roles) >>>>>>>>>>>> doesn't >>>>>>>>>>>> offer much that can't be reasonably achieved by other means. >>>>>>>>>>>> I'd find >>>>>>>>>>>> much more value in making sure what is done now is a solid >>>>>>>>>>>> foundation >>>>>>>>>>>> for the future. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Fair enough, I understand your perspective. Thanks for your >>>>>>>>>>> feedback. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Ilan >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 18, 2021 at 11:24 PM Noble Paul < >>>>>>>>>>>> noble.p...@gmail.com> wrote: >>>>>>>>>>>> > >>>>>>>>>>>> > After so many back and forth mails, I just can't say who has >>>>>>>>>>>> an outstanding concern and if they are already addressed or not. I >>>>>>>>>>>> think >>>>>>>>>>>> the Google doc would help us get clarity on that. Please take a >>>>>>>>>>>> moment to >>>>>>>>>>>> give your inputs >>>>>>>>>>>> > >>>>>>>>>>>> > On Fri, Nov 19, 2021, 9:18 AM Ishan Chattopadhyaya < >>>>>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>>>>> >> >>>>>>>>>>>> >> Apologies, the vote hasn't passed formally and I was under >>>>>>>>>>>> some confusion on the process. >>>>>>>>>>>> >> >>>>>>>>>>>> >> I'd like to proceed with a lazy consensus and proceed to the >>>>>>>>>>>> implementation phase now. >>>>>>>>>>>> >> >>>>>>>>>>>> >> However, I would appreciate it if someone wants to bring out >>>>>>>>>>>> any outstanding concerns about the SIP document. >>>>>>>>>>>> >> >>>>>>>>>>>> >> To facilitate in-line comments, here's a temporary Google >>>>>>>>>>>> Docs version of this document. >>>>>>>>>>>> >> >>>>>>>>>>>> https://docs.google.com/document/d/1hijvM1WX9u2TOUdLEkFYVCofLeJlv2MRZqe-ncobJVw/edit?usp=sharing >>>>>>>>>>>> >> (I shall copy changes back to confluence eventually) >>>>>>>>>>>> >> >>>>>>>>>>>> >> Thanks and apologies again regarding the confusion with the >>>>>>>>>>>> voting, >>>>>>>>>>>> >> Regards, >>>>>>>>>>>> >> Ishan >>>>>>>>>>>> >> >>>>>>>>>>>> >> On Thu, Nov 18, 2021 at 9:50 PM Ishan Chattopadhyaya < >>>>>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> The SIP passed the voting phase. Thanks for all for the >>>>>>>>>>>> feedback and insights. >>>>>>>>>>>> >>> Looking forward to your collaboration and reviews as we >>>>>>>>>>>> implement this. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> On Thu, Nov 18, 2021 at 9:42 PM Ishan Chattopadhyaya < >>>>>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> > It's fine if we don't provide any ability for runtime >>>>>>>>>>>> modification of roles at this time but I'm leery of precluding it >>>>>>>>>>>> in the >>>>>>>>>>>> future. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> In future, the necessity for such a facility can dictate >>>>>>>>>>>> our course of action. We cannot lay down rules cast in stone for >>>>>>>>>>>> functionality that we can't foresee yet. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> On Thu, Nov 18, 2021 at 9:40 PM Ishan Chattopadhyaya < >>>>>>>>>>>> ichattopadhy...@gmail.com> wrote: >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> Thanks Jan, I added both those points to the SIP document >>>>>>>>>>>> in the Notes section. >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> On Thu, Nov 18, 2021 at 7:18 PM Jan Høydahl < >>>>>>>>>>>> jan....@cominvent.com> wrote: >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> 18. nov. 2021 kl. 01:43 skrev Gus Heck < >>>>>>>>>>>> gus.h...@gmail.com>: >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> 2) Roles will not be checked by loading config from disk >>>>>>>>>>>> or caching disk config in memory. (zk ONLY source of truth) >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> It sounds a bit backward for a local node to first parse >>>>>>>>>>>> solr.node.roles, determine its local set of roles, then publish >>>>>>>>>>>> them to >>>>>>>>>>>> Zookeeper, and then read back its own roles from ZK. >>>>>>>>>>>> >>>>>> Code that only needs to determine "Do I have the XXX >>>>>>>>>>>> role?" or find out "What roles do I have" should be able to fetch >>>>>>>>>>>> the >>>>>>>>>>>> (static) roles from some roles utility class without consulting ZK. >>>>>>>>>>>> >>>>>> Code that needs to check what nodes have a certain role >>>>>>>>>>>> (such as placement) would obviously need ot consult ZK. >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> Perhaps the SIP should also state some Non-goals or >>>>>>>>>>>> assertions such as >>>>>>>>>>>> >>>>>> * Roles are static and immutable (also in zk) for the >>>>>>>>>>>> entire life cycle of a node >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> I also think we should state that the bar for adding new >>>>>>>>>>>> roles should be high so it is not abused as any other tag or label >>>>>>>>>>>> for any >>>>>>>>>>>> tiny feature. It should be reserved for functionality that may >>>>>>>>>>>> benefit from >>>>>>>>>>>> a dedicated set of nodes. That may be clear already, but you never >>>>>>>>>>>> know... >>>>>>>>>>>> >>>>>> >>>>>>>>>>>> >>>>>> Jan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org >>>>>>>>>>>> For additional commands, e-mail: dev-h...@solr.apache.org >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> http://www.needhamsoftware.com (work) >>>>>>>>>> http://www.the111shift.com (play) >>>>>>>>>> >>>>>>>>> >>>>>> >>>>>> -- >>>>>> http://www.needhamsoftware.com (work) >>>>>> http://www.the111shift.com (play) >>>>>> >>>>> >>> >>> -- >>> http://www.needhamsoftware.com (work) >>> http://www.the111shift.com (play) >>> >> >> >> -- >> http://www.needhamsoftware.com (work) >> http://www.the111shift.com (play) >> >
