Ishan: can you be more specific please? How is it less secure or harder to secure than, say, a configSet upload (internally multiple files)?
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Sep 24, 2020 at 1:53 PM Ishan Chattopadhyaya < [email protected]> wrote: > Single file update capability is a security nightmare. Even if it can be > done, it should be supported only once authc/authz have been enabled. > > On Thu, 24 Sep, 2020, 10:16 pm Tomás Fernández Löbbe, < > [email protected]> wrote: > >> I won't step in the way of a single file update. I haven't needed it so >> far though. I usually have the configsets in a Git repo (all the configset >> together) and I have a simple bash script that essentialy what's described >> in the docs[1]: Generate the zip on the fly and upload (optionally set the >> auth too). This can become a problem with big zips, but then again, >> ZooKeeper limits the size of the configs, so far it hasn't been an issue >> for me. >> >> >> [1] >> https://lucene.apache.org/solr/guide/8_6/configsets-api.html#configsets-upload >> >> On Thu, Sep 24, 2020 at 7:13 AM Eric Pugh < >> [email protected]> wrote: >> >>> It would be great if we had a simple API for updating a file in a >>> configset that didn’t assume you were just uploading a zip file. >>> >>> As an example use case, if you use the Querqy library, you need to >>> deploy a “rules.txt” file, which in olden days just went on the filesystem >>> and you would click reload on a core. In the SolrCloud world, we do this >>> awkward “Let me stick the file in Zookeeper directly, avoiding Solr, and >>> then do a collection RELOAD” to push out the file everywhere. [1] It >>> works! But it’s just awkward. >>> >>> It’s great to know that I’ll be able to change out this awkward process >>> using these magic parameters to configSet. Even nicer would be to just >>> wrap the overwrite=true&cleanup=false and the Zip requirement into >>> something that sets those. >>> >>> >>> [1] >>> https://github.com/querqy/chorus/blob/master/smui/conf/smui2solrcloud.sh#L37 >>> >>> On Sep 23, 2020, at 11:57 PM, Tomás Fernández Löbbe < >>> [email protected]> wrote: >>> >>> > Hmmm; seems the configSet API doesn't have an API to update a single >>> file! I wonder if uploading a configSet to the same name effectively >>> overwrites newly updated files but does not delete the existing files? >>> I've been working on this recently. As of 8.7, the UPLOAD command >>> supports overwriting (before, an UPLOAD on an existing configset name would >>> fail with BAD_REQUEST) and you can choose to cleanup or not the extra files >>> with the "cleanup" parameter. >>> You could upload a single file if you say overwrite=true&cleanup=false, >>> but it would still need to be in a zip file (and needs to be located in the >>> right path of the zip, for example, a synonyms file may be in >>> lang/synonyms_foo.txt or something) >>> >>> On Wed, Sep 23, 2020 at 8:10 PM David Smiley <[email protected]> wrote: >>> >>>> +1 to deprecate managed resources in lieu of easier to maintain (and >>>> more flexible) file based GET/PUT into the configset. >>>> >>>> > I don't know if 9 is too soon from a deprecation stand point >>>> >>>> IMO it's never too soon as long as there is a deprecated release. >>>> Users take their time upgrading to major versions. >>>> >>>> > How much harder are the use-cases currently covered by managed >>>> resources, if that module was removed? >>>> >>>> I believe in practice, users synchronize one-way from their DB to Solr >>>> if they have dynamic resources like this. This is true where I work. >>>> Otherwise, they would probably be using Solr as the source of truth, which >>>> doesn't seem architecturally-sound for most apps IMO. Those users >>>> (hopefully few) would have to spend some time re-engineering the approach. >>>> Given one-way sync, the transition here is pretty easy: serialize the >>>> client-managed data to the right Solr format (stopwords vs synonyms vs ...) >>>> and then a file upload to Solr/ZK and then telling Solr which collections >>>> to "reload". >>>> >>>> Hmmm; seems the configSet API doesn't have an API to update a single >>>> file! I wonder if uploading a configSet to the same name effectively >>>> overwrites newly updated files but does not delete the existing files? >>>> >>>> ~ David Smiley >>>> Apache Lucene/Solr Search Developer >>>> http://www.linkedin.com/in/davidwsmiley >>>> >>>> >>>> On Wed, Sep 23, 2020 at 10:28 AM Timothy Potter <[email protected]> >>>> wrote: >>>> >>>>> I agree we should deprecate the managed resources feature, it was the >>>>> first thing I was asked to build by LW nearly 7 years ago, before I was a >>>>> committer. Restlet was already in place and I built on top of that, not >>>>> sure who introduced it originally (nor do I care). Clearly from the >>>>> vantage >>>>> point of looking back, JAX-RS and Jersey won the day with REST in Java but >>>>> that simply wasn't the case back then. What's important is how we move >>>>> forward vs. bestowing judgement backed by wisdom of hindsight on decisions >>>>> made many years ago. >>>>> >>>>> In the short term, does Apache have an Artifactory (or similar) where >>>>> we can host the Restlet dependencies for Github to pull them from? If not, >>>>> then we can port the code that's using Restlet over to using JAX-RS / >>>>> Jersey. Personally I'd prefer we remove Managed Resources support from 9 >>>>> instead of porting the Restlet code but I don't know if 9 is too soon from >>>>> a deprecation stand point? >>>>> >>>>> Tim >>>>> >>>>> >>>>> On Mon, Sep 21, 2020 at 11:33 PM Noble Paul <[email protected]> >>>>> wrote: >>>>> >>>>>> We should deprecate that feature and remove restlet dependency >>>>>> altogether >>>>>> >>>>>> On Mon, Sep 21, 2020 at 10:20 PM Joel Bernstein <[email protected]> >>>>>> wrote: >>>>>> > >>>>>> > Restlet again!!!!!!! >>>>>> > >>>>>> > >>>>>> > >>>>>> > Joel Bernstein >>>>>> > http://joelsolr.blogspot.com/ >>>>>> > >>>>>> > >>>>>> > On Mon, Sep 21, 2020 at 7:18 AM Eric Pugh < >>>>>> [email protected]> wrote: >>>>>> >> >>>>>> >> Do we have a community blessed alternative to restlet already? >>>>>> >> >>>>>> >> On Sep 20, 2020, at 9:40 AM, Noble Paul <[email protected]> >>>>>> wrote: >>>>>> >> >>>>>> >> Haha. >>>>>> >> >>>>>> >> In fact schema APIs don't use restlet. Only the managed resources >>>>>> use it >>>>>> >> >>>>>> >> On Sat, Sep 19, 2020, 3:35 PM Ishan Chattopadhyaya < >>>>>> [email protected]> wrote: >>>>>> >>> >>>>>> >>> If I were talend, I'd immediately start publishing to maven >>>>>> central. If I were the developer who built the schema APIs, I would never >>>>>> have used restlet to begin with. >>>>>> >>> >>>>>> >>> On Sat, 19 Sep, 2020, 1:13 am Uwe Schindler, <[email protected]> >>>>>> wrote: >>>>>> >>>> >>>>>> >>>> I was thinking the same. Because GitHub does not cache the >>>>>> downloaded artifacts like our jenkins servers. >>>>>> >>>> >>>>>> >>>> It seems to run it in a new VM or container every time, so it >>>>>> downloads all artifacts. If I were Talend, I'd also block this. >>>>>> >>>> >>>>>> >>>> Uwe >>>>>> >>>> >>>>>> >>>> Am September 18, 2020 7:32:47 PM UTC schrieb Dawid Weiss < >>>>>> [email protected]>: >>>>>> >>>>> >>>>>> >>>>> I don't think it's http/https - I believe restlet repository >>>>>> simply >>>>>> >>>>> bans github servers because of excessive traffic? These URLs >>>>>> work for >>>>>> >>>>> me locally... >>>>>> >>>>> >>>>>> >>>>> Dawid >>>>>> >>>>> >>>>>> >>>>> On Fri, Sep 18, 2020 at 6:35 PM Christine Poerschke (BLOOMBERG/ >>>>>> >>>>> LONDON) <[email protected]> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> This sounds vaguely familiar. "http works, https does not >>>>>> work" and https://issues.apache.org/jira/browse/SOLR-13756 possibly >>>>>> related? >>>>>> >>>>>> >>>>>> >>>>>> From: [email protected] At: 09/18/20 10:01:29 >>>>>> >>>>>> To: [email protected] >>>>>> >>>>>> Subject: Re: restlet dependencies >>>>>> >>>>>> >>>>>> >>>>>> I don't think it is, sadly. >>>>>> >>>>>> https://repo1.maven.org/maven2/org/restlet >>>>>> >>>>>> >>>>>> >>>>>> The link you provided (mvnrepository) aggregates from several >>>>>> maven >>>>>> >>>>>> repositories. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> D. >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Sep 18, 2020 at 10:46 AM Ishan Chattopadhyaya >>>>>> >>>>>> <[email protected]> wrote: >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> Sorry, afk, but I heard (*hearsay*) that restlet is also on >>>>>> maven central >>>>>> >>>>>> >>>>>> >>>>>> these days. Can we confirm and switch to that? Sorry, if >>>>>> that's not the case. >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> On Fri, 18 Sep, 2020, 1:15 pm Dawid Weiss, < >>>>>> [email protected]> wrote: >>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>>>> >>>>>>>> Just FYI: can't get PR builds on github to work recently >>>>>> because of this: >>>>>> >>>>>>>> >>>>>> >>>>>>>>> Could not resolve all files for configuration >>>>>> >>>>>> >>>>>> >>>>>> ':solr:core:compileClasspath'. >>>>>> >>>>>>>> >>>>>> >>>>>>>> 350 > Could not download org.restlet.ext.servlet-2.4.3.jar >>>>>> >>>>>>>> (org.restlet.jee:org.restlet.ext.servlet:2.4.3) >>>>>> >>>>>>>> 351 > Could not get resource >>>>>> >>>>>>>> >>>>>> >>>>>> ' >>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>> >>>>>>>> >>>>>> >>>>>>>> 352 > Could not GET >>>>>> >>>>>>>> >>>>>> >>>>>> ' >>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet.ext.servlet/2.4.3/org.res >>>>>> >>>>>> tlet.ext.servlet-2.4.3.jar'. >>>>>> >>>>>>>> >>>>>> >>>>>>>> 353 > Connection reset >>>>>> >>>>>>>> 354 > Could not download org.restlet-2.4.3.jar >>>>>> >>>>>>>> (org.restlet.jee:org.restlet:2.4.3) >>>>>> >>>>>>>> 355 > Could not get resource >>>>>> >>>>>>>> >>>>>> >>>>>> ' >>>>>> https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.j >>>>>> >>>>>> ar'. >>>>>> >>>>>>>> >>>>>> >>>>>>>> 356 > Could not GET >>>>>> >>>>>>>> >>>>>> >>>>>> ' >>>>>> https://maven.restlet.talend.com/org/restlet/jee/org.restlet/2.4.3/org.restlet- >>>>>> >>>>>> 2.4.3.jar'. >>>>>> >>>>>>>> >>>>>> >>>>>>>> 357 > Connection reset >>>>>> >>>>>>>> >>>>>> >>>>>>>> D. >>>>>> >>>>>>>> ________________________________ >>>>>> >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>> >>>>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>>>> >>>>>> >>>>>> ________________________________ >>>>>> >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> ________________________________ >>>>>> >>>>> To unsubscribe, e-mail: [email protected] >>>>>> >>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>>> >>>> >>>>>> >>>> -- >>>>>> >>>> Uwe Schindler >>>>>> >>>> Achterdiek 19, 28357 Bremen >>>>>> >>>> https://www.thetaphi.de >>>>>> >> >>>>>> >> >>>>>> >> _______________________ >>>>>> >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >>>>>> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >>>>>> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>>>>> >> This e-mail and all contents, including attachments, is considered >>>>>> to be Company Confidential unless explicitly stated otherwise, regardless >>>>>> of whether attachments are marked as such. >>>>>> >> >>>>>> >>>>>> >>>>>> -- >>>>>> ----------------------------------------------------- >>>>>> Noble Paul >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>> _______________________ >>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >>> | http://www.opensourceconnections.com | My Free/Busy >>> <http://tinyurl.com/eric-cal> >>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >>> This e-mail and all contents, including attachments, is considered to be >>> Company Confidential unless explicitly stated otherwise, regardless >>> of whether attachments are marked as such. >>> >>>
