The annotation framework is just a single class and it is made to work well with Solr instead of changing Solr to suit the needs of some external framework. There are no known issues with the current system and there are unknown issues that you'll face introducing a new framework.
Solr is not a general purpose web server where people write APIs every day. It's a finished product where people use the publicly available APIs . We should focus our efforts on making our APIs work well . On Wed, Dec 1, 2021, 2:14 AM Jason Gerlowski <gerlowsk...@gmail.com> wrote: > > We need to make an API work on a per core basis and cores can come up > and go down randomly > > Ah, yeah. I'll admit I was overlooking some of the wrinkles around > custom request handlers etc. That is a problem for switching to some > off-the-shelf framework for sure. It may still be possible, but it's > definitely a big hurdle. I wonder how commonly users still take > advantage of the custom requestHandler stuff these days? I haven't > run across many customers who used them extensively, but maybe that's > self-selecting in some way? > The only reason why anyone would need to use this framework is to write a custom request handler. You're also saying nobody writes custom request handlers. So, what problem are you trying to solve if you think nobody uses them? > > If SolrJ has a Jackson dependency, it can have a conflict (different > versions) with the Jackson used by the client app > > SolrJ has a handful of dependencies - is there a reason we take this > approach with Jackson, but not with any of SolrJ's other deps? > (apache-httpcomponents, apache-commons, zookeeper-client, etc.) > > Granted, I get that we want SolrJ to be as painless as possible for > its users, and jar-conflicts are a part of that and really suck. But > there are good tools out there for handling jar conflicts, and > reimplementing chunks of library functionality just to avoid a > gradle-dep strikes me as unsustainable from a maintenance/community > perspective. Not to mention that forking/mimicking Jackson code > deprives our users of any efficiency/security improvements that > Jackson might get tomorrow or next week or next year. > We did not fork Jackson. We are using Jackson itself. Jackson supports custom annotations. There was a huge discussion why we should do it this way. > > Unless there's some reason specific to Jackson I guess I just don't > get it. But if I'm the minority opinion on that, fair enough I guess. > > On Tue, Nov 30, 2021 at 5:05 AM Noble Paul <noble.p...@gmail.com> wrote: > > > > The @JsonProperty annotations are added because it's a dependency in > SolrJ as well > > If SolrJ has a Jackson dependency, it can have a conflict (different > versions) with the Jackson used by the client app. > > > > On Tue, Nov 30, 2021, 5:21 PM Noble Paul <noble.p...@gmail.com> wrote: > >> > >> Also keep in mind that the same endpoint can be accessed with a core > name and a collection name prefixes. > >> > >> On Tue, Nov 30, 2021 at 3:55 PM Noble Paul <noble.p...@gmail.com> > wrote: > >>> > >>> True Gus, Almost every framework works outside of SolrDispatchFilter+ > HttpSolrCall. A lot of our initializations occur there. > >>> > >>> We need to make an API work on a per core basis and cores can come up > and go down randomly. So we need to register these endpoints on a core. > >>> > >>> I'm not sure if any framework can achieve the same. > >>> > >>> On Tue, Nov 30, 2021 at 5:39 AM Gus Heck <gus.h...@gmail.com> wrote: > >>>> > >>>> IIRC last time I looked restlet had the unsavory property of existing > outside of the SolrDispatchFilter, unlike everything else which made for > special cases because several things that probably ought to be their own > siervlet filters are glommed into SolrDispatchFilter, like security, > tracing and MDC setup/teardown per request. Restlet wouldn't be so bad if > one could just wrap such filters around it too... > >>>> > >>>> On Mon, Nov 29, 2021 at 9:42 AM Jason Gerlowski < > gerlowsk...@gmail.com> wrote: > >>>>> > >>>>> > These are minor improvements compared to a full rewrite of the > entire framework > >>>>> > >>>>> If you think data type support is minor, fair enough. But to clarify > >>>>> I'm not suggesting a rewrite - I'm suggesting using something that > >>>>> already exists off the shelf. Jersey (e.g.) itself provides the > >>>>> framework - there would be no "rewrite". > >>>>> > >>>>> re: past restlet use > >>>>> > >>>>> > It was not playing well with our security framework. The framework > was not working well with Solr APIs > >>>>> > >>>>> Ah, very interesting! Security isn't something Eric or I tackled in > >>>>> our little spike branch, but it's definitely a concern. Do you > >>>>> remember the specific concerns? Or recall where any of the > discussion > >>>>> around this happened? > >>>>> > >>>>> Without the context of that past discussion, it seems like the > >>>>> "PermissionNameProvider" interface could be implemented just as well > >>>>> by a class with (e.g.) Jersey annotations as one with our own custom > >>>>> annotations. Certainly there'd need to be some > >>>>> RuleBasedAuthorizationPlugin changes or other integration code, but > >>>>> nothing that feels insurmountable. > >>>>> > >>>>> Maybe I can try spiking it out soon and find the issues myself, but > >>>>> it'd be much easier if someone happens to remember and can save me > the > >>>>> trouble :-p > >>>>> > >>>>> Best, > >>>>> > >>>>> Jason > >>>>> > >>>>> On Fri, Nov 26, 2021 at 12:05 AM Noble Paul <noble.p...@gmail.com> > wrote: > >>>>> > > >>>>> > The Annotations framework was written after playing with other > frameworks. There were many shortcomings which were hard to overcome. > >>>>> > > >>>>> > The best example is a per collection API . How do you register an > endpoint for a collection/core ? > >>>>> > > >>>>> > On Fri, Nov 26, 2021 at 3:42 PM Noble Paul <noble.p...@gmail.com> > wrote: > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> On Fri, Nov 26, 2021 at 1:03 AM Jason Gerlowski < > gerlowsk...@gmail.com> wrote: > >>>>> >>> > >>>>> >>> > Is there some problem with our annotations that we hope to > solve using third party dependencies? > >>>>> >>> > >>>>> >>> I guess so yeah. Third-party deps are just fuller, more robust > >>>>> >>> solutions, whereas our annotations still need support added now > and > >>>>> >>> then for even primitive data types like "long" (see SOLR-15619). > >>>>> >> > >>>>> >> These are minor improvements compared to a full rewrite of the > entire framework > >>>>> >> > >>>>> >> > >>>>> >>> > >>>>> >>> Every JIRA spent doing basic stuff like that is time away from > >>>>> >>> improving Solr in some other way. > >>>>> >>> > >>>>> >>> So there are feature-gap/capabilities arguments for moving to a > >>>>> >>> third-party dep, sure. But, even if our annotations did > everything > >>>>> >>> Jersey+Jackson do today, I think switching would still be worth > it. > >>>>> >>> Every LOC in our code base brings along with it some maintenance > cost: > >>>>> >>> it might have bugs, needs tested, takes time for new > contributors to > >>>>> >>> "grok", etc. Using off-the-shelf here would nuke a whole bunch > of > >>>>> >>> that. If off-the-shelf is available for some given > functionality, we > >>>>> >>> should need a compelling reason to NOT use it. > >>>>> >>> > >>>>> >>> Lastly, I think there's an "approachability" argument for using > >>>>> >>> off-the-shelf. Thousands of developers out there are familiar > with > >>>>> >>> (e.g.) Jersey, compared to maybe 15 or 20 (in the world) > familiar with > >>>>> >>> Solr's custom annotations. Using a well-known technology like > Jersey > >>>>> >>> would make Solr all the easier to approach and contribute to for > that > >>>>> >>> pool of developers. > >>>>> >>> > >>>>> >>> > By the way, we have used Restlet in the past and that has been > a regrettable decision. > >>>>> >> > >>>>> >> > >>>>> >>> > >>>>> >>> Ah, yeah, that's just the context I'm missing. Anyone have a > pointer > >>>>> >>> to related discussions, or remember what made this > "regrettable"? All > >>>>> >>> the theoretical benefits in the world don't matter much if we've > >>>>> >>> already tried something like this in the past and decided > against it. > >>>>> >> > >>>>> >> > >>>>> >> It was not playing well with our security framework. The > framework was not working well with Solr APIs > >>>>> >> > >>>>> >>> > >>>>> >>> (Unrelated - Happy Thanksgiving all!) > >>>>> >>> > >>>>> >>> Best, > >>>>> >>> > >>>>> >>> Jason > >>>>> >>> > >>>>> >>> On Thu, Nov 25, 2021 at 7:32 AM Noble Paul <noble.p...@gmail.com> > wrote: > >>>>> >>> > > >>>>> >>> > Have you gone through an API written using the @EndPoint > annotation? > >>>>> >>> > > >>>>> >>> > I strongly recommend that you do > >>>>> >>> > > >>>>> >>> > On Thu, Nov 25, 2021, 11:30 PM Eric Pugh < > ep...@opensourceconnections.com> wrote: > >>>>> >>> >> > >>>>> >>> >> I have found our V2 API code to be very impenetrable to > understand. Part of it is how the code is intertwined with support for > V1, however it’s also because there aren’t really resources to go look at > to understand how it should work! Maintaining the API should be very > simple work, as they just exist as a translation. The home grown stuff > may make sense if you are a super knowledgable Solr developer, but if you > are just a new person, it’s a lot harder to contribute. > >>>>> >>> >> > >>>>> >>> >> I was interested in the Jersey stuff because I’ve seen lots > of projects use it very successfully, and if I want to implement something, > well, there are lots of blogs and resources out there! > >>>>> >>> >> > >>>>> >>> >> Can anyone recap briefly why we dropped RESTlet? And what > lessons learned there might apply to adopting Jersey for API support? > Looking at https://issues.apache.org/jira/browse/SOLR-14659, it was > partly deprecated because we were not using it to support all the API, only > the ManagedResource ones, and > https://issues.apache.org/jira/browse/SOLR-14766 suggests that RESTlet > maybe was no longer being updated? One reason why we spiked out Jersey > was because of the broad support in the Java world! Looking at how much > work we have to do in the V2 API world, we need a much broader pool of > developers contributing to get there! > >>>>> >>> >> > >>>>> >>> >> Related, are there specific features/aspects of our > annotations that enable things in Solr that couldn’t be done otherwise? > >>>>> >>> >> > >>>>> >>> >> On Nov 25, 2021, at 2:12 AM, Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >>>>> >>> >> > >>>>> >>> >> Is there some problem with our annotations that we hope to > solve using third party dependencies? > >>>>> >>> >> By the way, we have used Restlet in the past and that has > been a regrettable decision. > >>>>> >>> >> > >>>>> >>> >> On Thu, Nov 25, 2021 at 10:10 AM Jason Gerlowski < > gerlowsk...@gmail.com> wrote: > >>>>> >>> >>> > >>>>> >>> >>> Solr's custom annotation framework ('@Endpoint', '@Command', > etc.) has > >>>>> >>> >>> cropped up a few times over the past week or two. [1] [2]. > Having them > >>>>> >>> >>> on top of mind, I've been wondering - is there a reason we > use our own > >>>>> >>> >>> annotations here instead of something off the shelf? > >>>>> >>> >>> > >>>>> >>> >>> What we have works well enough, but anything homegrown comes > with more > >>>>> >>> >>> maintenance burden than we'd have if we used something off > the shelf. > >>>>> >>> >>> There are plenty of well-used, active projects out there > whose whole > >>>>> >>> >>> purpose is facilitating the whole "annotation based API" > thing > >>>>> >>> >>> (Jersey, Restlet, RESTEasy, etc.) - why not use one of them? > >>>>> >>> >>> > >>>>> >>> >>> Does anyone know of any technical reasons why we can't go > this route? > >>>>> >>> >>> Or have any subjective reasons why we shouldn't? Or any > context on > >>>>> >>> >>> why we wrote our own Endpoint, Command, JsonProperty > annotations > >>>>> >>> >>> originally? > >>>>> >>> >>> > >>>>> >>> >>> FWIW, Eric Pugh and I spiked out a small POC recently, and > got > >>>>> >>> >>> Jersey+Jackson working for a few simple APIs without too > much trouble. > >>>>> >>> >>> [3] Obviously nothing production-ready there, and there's > still a lot > >>>>> >>> >>> of open questions (e.g. how would javabin be supported?), > but we both > >>>>> >>> >>> came away convinced that it seemed feasible, at least. Best > of all, > >>>>> >>> >>> APIs using our current homegrown annotation framework the > switchover > >>>>> >>> >>> seems blessedly straightforward, and it doesn't look like > Jersey > >>>>> >>> >>> (which we chose mostly arbitrarily) bloats our dist all that > much. > >>>>> >>> >>> > >>>>> >>> >>> Curious if anyone has thoughts or context on how we ended up > with the > >>>>> >>> >>> annotation setup we use today! > >>>>> >>> >>> > >>>>> >>> >>> Best, > >>>>> >>> >>> > >>>>> >>> >>> Jason > >>>>> >>> >>> > >>>>> >>> >>> [1] https://issues.apache.org/jira/browse/SOLR-15182 (and > children) > >>>>> >>> >>> [2] > http://mail-archives.apache.org/mod_mbox/solr-dev/202111.mbox/%3CCABEwPvENL41Pm6%2BOmjXb6Sx5N2XjUtnbWhgKOZSrnLjWBA8tcA%40mail.gmail.com%3E > >>>>> >>> >>> [3] > https://github.com/gerlowskija/solr/tree/jersey_jaxrs_jackson_solr_apis. > >>>>> >>> >>> > >>>>> >>> >>> > --------------------------------------------------------------------- > >>>>> >>> >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > >>>>> >>> >>> For additional commands, e-mail: dev-h...@solr.apache.org > >>>>> >>> >>> > >>>>> >>> >> > >>>>> >>> >> _______________________ > >>>>> >>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy > >>>>> >>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > >>>>> >>> >> This e-mail and all contents, including attachments, is > considered to be Company Confidential unless explicitly stated otherwise, > regardless of whether attachments are marked as such. > >>>>> >>> >> > >>>>> >>> > >>>>> >>> > --------------------------------------------------------------------- > >>>>> >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > >>>>> >>> For additional commands, e-mail: dev-h...@solr.apache.org > >>>>> >>> > >>>>> >> > >>>>> >> > >>>>> >> -- > >>>>> >> ----------------------------------------------------- > >>>>> >> Noble Paul > >>>>> > > >>>>> > > >>>>> > > >>>>> > -- > >>>>> > ----------------------------------------------------- > >>>>> > Noble Paul > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > >>>>> For additional commands, e-mail: dev-h...@solr.apache.org > >>>>> > >>>> > >>>> > >>>> -- > >>>> http://www.needhamsoftware.com (work) > >>>> http://www.the111shift.com (play) > >>> > >>> > >>> > >>> -- > >>> ----------------------------------------------------- > >>> Noble Paul > >> > >> > >> > >> -- > >> ----------------------------------------------------- > >> Noble Paul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > For additional commands, e-mail: dev-h...@solr.apache.org > >
