[
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661362#comment-14661362
]
Noble Paul commented on SOLR-7576:
----------------------------------
bq.There are other features like sources for Suggesters that I look forward to
seeing getting upgraded to use the blob store
Exactly. That was the reason why I created that API. We need a place where we
can store MBs of files and multiple versions of each of them. So a developer
should be able to edit without fear and play with his production cluster
without impacting his actual users
bq.For this specific feature (request handlers in scripting language) why
should the executable be put in the Blob store at all? Generally these scripts
are going to be very small.
I don't know if they are going to be very small and neither do you. In general
we need a place which can store a lot of files. Moreover, whether the file is
big or small the user experience is going to be exactly same, he will have to
run a command line to edit this. Keeping multiple versions of the same script
in the conf directory is not what I call elegant. We are just thinking of ZK as
a kitchen sink because it used to be file system for standlone solr and size
was never a problem
Standalone Solr is not going to be the primary target for any new feature that
we develop. In a not too distant future we will make the standalone Solr also
use a ZK and other stuff to unify dev/user experience. After all a standalone
solr can easily be a single collection/ single shard/single replica version of
SolrCloud.
bq. A REST API on top could add another convenient access mechanism to "conf"
dir stuff
Please , we are not going to add more stuff to conf dir. That is a slippery
slope. conf dir should have well known standard files. The rest of the stuff
should move away. Bloating up ZK just because we used to store things in conf
dir is just inertia. We must get over it.
Having said that there is nothing that stops us from configuring the
JsRequestHandler to load scripts from a local file system directory which can
be useful for certain usecases.
bq.i.e. exactly how the Script URP works
IIRC the script URP does not reload the script automatically by watching the ZK
node. You need to go and restart core or other dirty stuff. In a large
SolrCloud cluster that amounts to 100's of cores getting reloaded and caches
trashed and what not (all queries getting slow/ a lot of GC etc). Do you still
like that idea? Let's not assume that whatever we have today is optimized to
run on cloud mode. There are a lot of things that got carried forward from the
legacy things which are suboptimal to cloud. Instead of clinging on to the old
way of life we must modernize our system to adopt to the new paradigm
> Implement RequestHandler in Javascript
> --------------------------------------
>
> Key: SOLR-7576
> URL: https://issues.apache.org/jira/browse/SOLR-7576
> Project: Solr
> Issue Type: New Feature
> Reporter: Noble Paul
> Attachments: SOLR-7576.patch, SOLR-7576.patch
>
>
> Solr now support dynamic loading (SOLR-7073) of components and it is secured
> in SOLR-7126
> We can extend the same functionality with JS as well
> the handler {{/js}} is implicitly registered
> To make this work
> * Solr should be started with {{-Denable.js.loading=true}}
> * The javascript must be loaded to the {{.system}} collection using the blob
> store API
> * Sign the javascript and pass the signature in a param called {{_sig}}
> The {{JSRequestHandler}} is implicitly defined and it can be accessed by
> hitting {{/js/<jsname>/<version>}}
> Steps for developing scripts
> # start the cluster with the {{enable.js.loading}} . If you are starting
> using our script it would be {{bin/solr start -e cloud -a
> "-Denable.js.loading=true"}} . You would not need security during development
> , so don't add the private keys to Solr
> # create {{.system}} collection {{bin/solr create -c .system}}
> # Write your javascript code . (say {{test.js}} )
> # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type:
> application/octet-stream' --data-binary @test.js
> http://localhost:8983/solr/.system/blob/test}}
> # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
> # Edit your script and repeat from step #4 . Keep in mind that the version
> would be bumped up every time you post a new script . So, the second time the
> url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and
> so forth
> sample programs
> 1) writes a val to output
> {code:javascript}
> //empty line
> $.response().add('testkey','Test Val');
> {code}
> 2) manipulate the output to add an extra field to each doc
> {code}
> //empty line
> var l = [];
> $.query({
> q: '*:*',
> qt: '/select',
> start:0,
> }).forEach('response', function(doc) {
> doc.put('script', 'Added this
> value');
> l.push(doc);
> });
> $.response().add('alldocs', l);
> {code}
> 3) stream through all the docs
> {code:Javascript}
> //empty line
> $.query({
> q: '*:*',
> qt: '/select',
> start:0,
> distrib:'false'
> }).pipe('response', 'docs', function(doc) { // the pipe function is
> executed right before the response writer and right after the transformers
> if('IT'== doc.get('genre_s')) return
> null;
> doc.put('script', 'Added this
> value');
> return doc;
> });
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
