[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-687329334 I withdraw all outstanding concerns. Verbosity, clunkiness of configuration etc are all my "perceptions" that I don't want to come in the way of the completion of this effort. > This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686808345 > Wow. I'm not even sure that would be polite to say to a junior engineer (that may be presumed to not know concurrency) than a very senior one here. I think you know full well that Ilan doesn't need your help writing concurrent code. Yet why would you offer help? I think this is a false offer (you know Ilan will not take you up on this); -- it's a pejorative offer that belittles who you offer it to as being beneath the abilities of your own. I wonder why you would say that? Maybe you don't believe it's worth it to make it easier for plugin writers at the expense of Solr having such concurrent code (i.e. where does the concurrency/complexity go) -- fare, but then just say that without belittling anyone's intelligence. Ilan is a reasonably new committer, so I am not fully aware of his background or his "seniority". I just offered help in writing concurrent code that I consider clean. But, now you have made couple of assertions that I will keep in mind: "Ilan doesn't need your help writing concurrent code" "Ilan will not take you up on this" > "pejorative offer that belittles who you offer it" > "as being beneath the abilities of your own" > "belittling [..] intelligence" Even though I won't stop you representing/misrepresenting Ilan as you have done above (by claiming he is not open to help in this area), at least please don't put misrepresent me with the motives behind my words. Just to clarify: I thought Ilan was inclined towards having tons of classes (and thereby "polluting solr-core") because he wanted to avoid having the plugins make concurrent requests, since I thought he perceived doing so as complicated. I am of course not doubting anyone's ability to write concurrent code (and of course not "belittling anyone's intelligence"), but I wanted to offer suggestions on doing so *cleanly* (without complications). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686380223 I broadly agree with those principals, and I'm sure everyone does. >1. placement plugin writing is easy, Agree, but not at the cost of complicated server side code that resides in solr-core. > 2. implementation for this placement API is efficient, and +1 > 3. plugins do not break when Solr is refactored (plugins might be client private so can’t be refactored when Solr code changes). Mostly agree, but not at the cost of thousands of classes. There are simpler ways, as I think we now are clear on, to achieve the same goal. I would also like you to consider another goal (which, IMHO, is *most important*), which is to keeping the solr-core as lean, clean and the footprint of doing so as minimal as possible. Everything that can be done outside solr-core should be done. Autoscaling is not a first class citizen in a search engine, but replica assignment/placement can arguably be so. > Can we make this assumption? Plugin requests a set of properties from a set of nodes, and this is done efficiently. If so I'll gladly simplify my proposal +1. I'm glad we're all on the same page (at least on the verbosity aspect). Thanks for your work, @murblanc. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686357674 > Please also remember that we are on a deadline - we need to have some kind of replacement for the autoscaling in 9.0. +1, this is the cause for maximum urgency at the moment. We need to: 1. design the interfaces, 2. design the proper user interaction layer (DSL?), 3. build efficient implementations. We cannot afford to be stuck at step 1. However, building a frankenstein framework as part of 1 is not conducive for overall goals of making/keeping Solr lean and clean. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684:
URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686355702
> @noblepaul & @chatman I find the tone of your latest comments offensive -
that's no way to build a consensus. Please think twice before posting and calm
down - if you have a different opinion about technical merits of this PR then
I'm sure you can express it without personal attacks.
I don't see how Noble's comments can construed as offensive. I may be biased
in favour of my own comments, but I apologise if they were perceived as such.
In any case, there is no personal attack anywhere.
> By all means, if you disagree so strongly with the approach presented here
then please do so - just be sure that you actually will do it instead of just
complaining.
I find choice of such words (" instead of just complaining") as
unprofessional. This is a proposal, and comments are added to critique the
design, not complain.
On the other hand, Ilan wrote this on Slack:
> If there’s consensus for Noble’s approach (or for that matter no consensus
that goals 1-3 above are good guiding principles), I will stop work on
SOLR-14613 and move on to other unrelated topics.
Such threats of "stop work" unless one's design is agreed upon should cease,
and constructive ways to collaborate should be explored.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686320915 > Yeah, the solution to API surface area problem is to make them inner classes/interfaces. This is such a elegant & simple proposition. We should try the same in other places in Solr where there is a surface area problem. Totally disagree, @noblepaul. I hope you're not serious here. Having getter methods like getFreeDiskSpace() etc. are better than having a FreeDiskProperty class, no matter it is inner or outer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686316250 > If the plugin requests data node by node, it's either sequential or forces the plugin to implement the concurrency mechanism itself, making it more complicated. Plugin making concurrent requests is "complicated"? Java is hard, but I can help you write a plugin that can make concurrent requests cleanly. Please don't pollute solr-core because of your perceived inability to do so cleanly from a plugin. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] chatman commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
chatman commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-686308531 > You need to make a realistic proposal here IMO @noblepaul. > And we need something that already exists. If placement plugins depend on some > future rehaul of configuration in SolrCloud, we're pushing this way out because > from past discussion there doesn't seem to be a consensus on how/if to > remove solr.xml and how to support its features differently. Noble has made an instructive example of how to deal with configuration properly (using clusterprops.json, not solr.xml): https://github.com/apache/lucene-solr/pull/1813 How much more hand holding do you need in order to do the right thing? If you want solr.xml to go away completely before using clusterprops, then let us wait until that is done. Using solr.xml (per node config) for a cluster wide replica assignment policy is not acceptable. The reasons have been stated many times, but I'm happy to do one more round and explain to you as simply or in as much detail as you would like. > I think use of solr.xml for this is pragmatic/realistic with the Solr we have today. This is absolutely untrue. Pragmatic/realistic is a joke. Using solr.xml means new configuration cannot take effect unless that file is hand-edited and node is restarted. Even bigger problem, in context of replica assignment/placement, is the possibility of different nodes having different configurations for the placement plugins, and hence cause cluster wide inconsistency. > And it's flexible -- can go on the node or in ZK as you desire, and can hold structured > information (not just a bag of name-value pairs). This matter can be improved in the future. Oh, my goodness! Such thinking is the reason why our project is in a total mess today. Putting solr.xml in ZK is a hack, which we don't need to carry forward any longer. Even though I have expressed my reluctant agreement to carry on with solr.xml for now and switch to clusterprops.json before release, such flawed reasoning in favour of using solr.xml pushes me towards -1'ing this issue on this basis. > the other one [requirement] was efficiency: being able to fetch ALL needed properties > from a remote Node at once Efficiency is a matter of implementation, and not a characteristic of the interfaces. Lets look at two approaches where the plugin requests for 10 properties. Design 1 (your design): Interface contains 10 different classes for the 10 properties, and solr-core returns 10 different objects for those properties to the plugin. Design 2 (proper design): Interface contains 1 class containing (at least) 10 getter methods for those properties (all typed as Optional). Solr-core returns that 1 object containing those 10 optional values filled in. In terms of interface design, design 2 is clearly more efficient because it requires the creation of 10 less objects. Lets look at efficiency from implementation point of view. In the design 2, solr-core (on a remote node) can populate all properties in the same object (even if it needs 10 multi-threaded calls). If there are multiple nodes from where those 10 properties need to be fetched, the plugin can make n multi-threaded calls to n remote nodes and receive n objects (each containing those 10 properties that are needed). How is any of this inefficient? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
