[
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661282#comment-14661282
]
Noble Paul commented on SOLR-7576:
----------------------------------
[~dsmiley]
bq. is the annoyance of having to update a file in ZooKeeper from one's file
system
I can see that pain point.
But the current model is totally broken and we MUST move away from from that
* we can't keep large amounts data in ZK
* We should not give a new feature out which requires a developer direct access
to write to ZK.We are talking about the same thing for a toy single node system
to large clusters with 1000's of nodes. Any screw up in ZK will have huge
impact . It is not like a solr node going down, it is going to blow up a huge
cluster
* Editing files and picking them up immediately may sound ok if you don't cache
stuff. Which is terrible for performance. So there has to be a way to tell Solr
that a particular resource is changed
* In real production systems we need to rollback stuff if things are broken.
This design lets you do that. You always upload a new version of the same file
like a version control system. Uploading a new version will have no impact till
you push it to production. Play with the new version till you are happy and
then push it to production. The point is , the developer can't screw up the
cluster by introducing a small syntax error
I'm not against adding an easier dev sandbox for such features. This is just
the first step. First of all we need to get the feature right for a deployed
large cluster. That is the story that we will want to discuss first. This just
one extra step for a developer in that phase. Ease of use stuff for playing
with things is a secondary thing and we should be able to do it as a separate
ticket. Understand that a standalone process and a huge cluster with 1000's of
nodes will have some difference in the way stuff is deployed
guys , editing ZK directly is fraught with huge risk. Please don't recommend it
to users. I understand that we have a lot of legacy stuff without proper APIs
and we need to edit things directly. We MUST have a safe way of achieving the
same through APIs (eventually).
So please do not introduce any new feature that needs ZK write.
> Implement RequestHandler in Javascript
> --------------------------------------
>
> Key: SOLR-7576
> URL: https://issues.apache.org/jira/browse/SOLR-7576
> Project: Solr
> Issue Type: New Feature
> Reporter: Noble Paul
> Attachments: SOLR-7576.patch, SOLR-7576.patch
>
>
> Solr now support dynamic loading (SOLR-7073) of components and it is secured
> in SOLR-7126
> We can extend the same functionality with JS as well
> the handler {{/js}} is implicitly registered
> To make this work
> * Solr should be started with {{-Denable.js.loading=true}}
> * The javascript must be loaded to the {{.system}} collection using the blob
> store API
> * Sign the javascript and pass the signature in a param called {{_sig}}
> The {{JSRequestHandler}} is implicitly defined and it can be accessed by
> hitting {{/js/<jsname>/<version>}}
> Steps for developing scripts
> # start the cluster with the {{enable.js.loading}} . If you are starting
> using our script it would be {{bin/solr start -e cloud -a
> "-Denable.js.loading=true"}} . You would not need security during development
> , so don't add the private keys to Solr
> # create {{.system}} collection {{bin/solr create -c .system}}
> # Write your javascript code . (say {{test.js}} )
> # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type:
> application/octet-stream' --data-binary @test.js
> http://localhost:8983/solr/.system/blob/test}}
> # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
> # Edit your script and repeat from step #4 . Keep in mind that the version
> would be bumped up every time you post a new script . So, the second time the
> url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and
> so forth
> sample programs
> 1) writes a val to output
> {code:javascript}
> //empty line
> $.response().add('testkey','Test Val');
> {code}
> 2) manipulate the output to add an extra field to each doc
> {code}
> //empty line
> var l = [];
> $.query({
> q: '*:*',
> qt: '/select',
> start:0,
> }).forEach('response', function(doc) {
> doc.put('script', 'Added this
> value');
> l.push(doc);
> });
> $.response().add('alldocs', l);
> {code}
> 3) stream through all the docs
> {code:Javascript}
> //empty line
> $.query({
> q: '*:*',
> qt: '/select',
> start:0,
> distrib:'false'
> }).pipe('response', 'docs', function(doc) { // the pipe function is
> executed right before the response writer and right after the transformers
> if('IT'== doc.get('genre_s')) return
> null;
> doc.put('script', 'Added this
> value');
> return doc;
> });
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
