I'll add the lifecycle details to the document as well. On Sun, 21 Nov, 2021, 9:08 am Ishan Chattopadhyaya, < ichattopadhy...@gmail.com> wrote:
> Hi Gus, > Thanks for bringing it up again. Initially, I wasn't clear what you meant > and mistook them for gish gallop. > > These were the things I am/was confused about (Noble explained it to me > what you mean): > - adherence to DRY via zk > - runtime config info should come from zk > - add dynamic features > - same interaction patterns for consulting config values. > > Now, after Noble's help, I think I understand your motivations. > - DRY = Don't Repeat Yourself ;-) > - You're suggesting that we have a standard way to have role specific > configurations (in ZK), so that a new role added later can access the > configurations in a standard way > > This is a very good suggestion, and makes complete sense now. This was > definitely a missing piece! > > Here's a proposal to address that (adding it to Google Docs: > https://docs.google.com/document/d/1hijvM1WX9u2TOUdLEkFYVCofLeJlv2MRZqe-ncobJVw/edit > ): > - Per Role ZNode > - Every role's ZNode to hold configuration data (this is the key addition > that you're asking for, IIUC) > - Children of role ZNodes to be list of ephemeral solr nodes > > We can iterate over this on the Google Docs (internal representation ZK > section) with inline commenting if we need to. > > Please let us know how that sounds. > Regards, > Ishan > > > > On Sun, Nov 21, 2021 at 4:49 AM Gus Heck <gus.h...@gmail.com> wrote: > >> It is difficult to track everything in this thread for sure. >> Specifically, I don't feel my email of Wed, Nov 17, 7:43 PM (EST) has been >> addressed or responded to except for one difference of opinion with Jan on >> whether or not we should prohibit the notion of any role ever changing at >> runtime. >> >> Note that I made a specific proposal for an addition to the sip, >> regarding start up lifecycle and adherence to DRY via zk, in the (probably >> not clearly expressed) hope that we can move to a general overall >> application principle that at runtime config info should come from zk. If >> we follow such a principle, we will always be in a position to add dynamic >> features without significant rework, and all code could hopefully reuse the >> same interaction patterns for consulting config values. >> >> Neither item is addressed in the SIP currently. >> >> Also I had put that out there as a single item so it could be focused on >> (simplifying the discussion I hope), and depending on how discussion >> proceeds, I may have other follow on items. >> >> -Gus >> >> On Sat, Nov 20, 2021 at 8:45 AM Ishan Chattopadhyaya < >> ichattopadhy...@gmail.com> wrote: >> >>> >>> On Fri, 19 Nov, 2021, 7:03 pm Ilan Ginzburg, <ilans...@gmail.com> wrote: >>> >>>> Is the request here for everybody to express again the concerns >>>> already expressed in this email thread and not addressed? >>>> >>> >>> I think the expectation now (after we've expressed our intention to >>> reach a lazy consensus) is that if someone wants to block this SIP from >>> adoption, then to put forward the objections. Sort of like a veto. >>> >>> >>>> >>>> I suggest instead the authors review the thread, match expressed >>>> concerns with how the concern was addressed (or not addressed) and >>>> provide an exhaustive list. >>>> >>> >>> I've summarized most of the discussion in the SIP document. If you feel >>> I could do a better job with it, please help me with areas of improvement. >>> >>> >>>> >>>> This proposal in its current form (data and overseer roles) doesn't >>>> offer much that can't be reasonably achieved by other means. I'd find >>>> much more value in making sure what is done now is a solid foundation >>>> for the future. >>>> >>> >>> Fair enough, I understand your perspective. Thanks for your feedback. >>> >>> >>>> >>>> Ilan >>>> >>>> On Thu, Nov 18, 2021 at 11:24 PM Noble Paul <noble.p...@gmail.com> >>>> wrote: >>>> > >>>> > After so many back and forth mails, I just can't say who has an >>>> outstanding concern and if they are already addressed or not. I think the >>>> Google doc would help us get clarity on that. Please take a moment to give >>>> your inputs >>>> > >>>> > On Fri, Nov 19, 2021, 9:18 AM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >> >>>> >> Apologies, the vote hasn't passed formally and I was under some >>>> confusion on the process. >>>> >> >>>> >> I'd like to proceed with a lazy consensus and proceed to the >>>> implementation phase now. >>>> >> >>>> >> However, I would appreciate it if someone wants to bring out any >>>> outstanding concerns about the SIP document. >>>> >> >>>> >> To facilitate in-line comments, here's a temporary Google Docs >>>> version of this document. >>>> >> >>>> https://docs.google.com/document/d/1hijvM1WX9u2TOUdLEkFYVCofLeJlv2MRZqe-ncobJVw/edit?usp=sharing >>>> >> (I shall copy changes back to confluence eventually) >>>> >> >>>> >> Thanks and apologies again regarding the confusion with the voting, >>>> >> Regards, >>>> >> Ishan >>>> >> >>>> >> On Thu, Nov 18, 2021 at 9:50 PM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>> >>>> >>> The SIP passed the voting phase. Thanks for all for the feedback >>>> and insights. >>>> >>> Looking forward to your collaboration and reviews as we implement >>>> this. >>>> >>> >>>> >>> On Thu, Nov 18, 2021 at 9:42 PM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>>> >>>> >>>> > It's fine if we don't provide any ability for runtime >>>> modification of roles at this time but I'm leery of precluding it in the >>>> future. >>>> >>>> >>>> >>>> In future, the necessity for such a facility can dictate our >>>> course of action. We cannot lay down rules cast in stone for functionality >>>> that we can't foresee yet. >>>> >>>> >>>> >>>> On Thu, Nov 18, 2021 at 9:40 PM Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>>>> >>>> >>>>> Thanks Jan, I added both those points to the SIP document in the >>>> Notes section. >>>> >>>>> >>>> >>>>> On Thu, Nov 18, 2021 at 7:18 PM Jan Høydahl < >>>> jan....@cominvent.com> wrote: >>>> >>>>>> >>>> >>>>>> 18. nov. 2021 kl. 01:43 skrev Gus Heck <gus.h...@gmail.com>: >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> 2) Roles will not be checked by loading config from disk or >>>> caching disk config in memory. (zk ONLY source of truth) >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> It sounds a bit backward for a local node to first parse >>>> solr.node.roles, determine its local set of roles, then publish them to >>>> Zookeeper, and then read back its own roles from ZK. >>>> >>>>>> Code that only needs to determine "Do I have the XXX role?" or >>>> find out "What roles do I have" should be able to fetch the (static) roles >>>> from some roles utility class without consulting ZK. >>>> >>>>>> Code that needs to check what nodes have a certain role (such as >>>> placement) would obviously need ot consult ZK. >>>> >>>>>> >>>> >>>>>> Perhaps the SIP should also state some Non-goals or assertions >>>> such as >>>> >>>>>> * Roles are static and immutable (also in zk) for the entire >>>> life cycle of a node >>>> >>>>>> >>>> >>>>>> I also think we should state that the bar for adding new roles >>>> should be high so it is not abused as any other tag or label for any tiny >>>> feature. It should be reserved for functionality that may benefit from a >>>> dedicated set of nodes. That may be clear already, but you never know... >>>> >>>>>> >>>> >>>>>> Jan >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org >>>> For additional commands, e-mail: dev-h...@solr.apache.org >>>> >>>> >> >> -- >> http://www.needhamsoftware.com (work) >> http://www.the111shift.com (play) >> >
