[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Upayavira (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660288#comment-14660288
 ] 

Upayavira commented on SOLR-7576:
-

We've had JS, we've had XSLT, etc, in our config directory for a long time. 
What is different about this new approach that it demands a more secure 
approach in all situations?

Please understand - I'm really keen on this feature, and am supportive of the 
auth/signing functionality.

I teach Solr a lot, and I know that a lot of normal users will find the .system 
and the HTTP post part hard to understand when first getting started, hence 
wanting to make it really simple to do the simplest things, and possible to do 
the powerful things.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Upayavira (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660681#comment-14660681
 ] 

Upayavira commented on SOLR-7576:
-

For sure - is suspect we're more on the same page than you think.

I would wholly support us moving towards such a situation where ZK is 
protected, and everything goes via authenticated UIs. My concern is merely 
about how we get there.

e.g. allow filesystem based JS (for non-cloud) and JS in ZK as now for 5.x, but 
for 6.x remove that ability and require an API across the board.

i.e. keep things as consistent as we can?

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660690#comment-14660690
 ] 

Noble Paul commented on SOLR-7576:
--

bq.e.g. allow filesystem based JS (for non-cloud) and JS in ZK as now for 5.x

A lot of APIs we build of late do not have Standalone Solr support. For lack of 
time and resources we are unable to build that. If possible I would like to see 
it supported. 

bq.i.e. keep things as consistent as we can?

yes, eventually consistent

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661362#comment-14661362
 ] 

Noble Paul commented on SOLR-7576:
--

bq.There are other features like sources for Suggesters that I look forward to 
seeing getting upgraded to use the blob store

Exactly. That was the reason why I created that API. We need a place where we 
can store MBs of files and multiple versions of each of them. So a developer 
should be able to edit without fear and play with his production cluster 
without impacting his actual users

bq.For this specific feature (request handlers in scripting language) why 
should the executable be put in the Blob store at all? Generally these scripts 
are going to be very small.

I don't know if they are going to be very small and neither do you. In general 
we need a place which can store a lot of files. Moreover, whether the file is 
big or small the user experience is going to be exactly same, he will have to 
run a command line to edit this. Keeping multiple versions of the same script 
in the conf directory is not what I call elegant. We are just thinking of ZK as 
a kitchen sink because it used to be file system for standlone solr and size 
was never a problem

Standalone Solr is not going to be the primary target for any new feature that 
we develop. In a not too distant future we will make the standalone Solr also 
use a ZK and other stuff to unify dev/user experience. After all a standalone  
solr can easily be a single collection/ single shard/single replica version of 
SolrCloud. 

bq. A REST API on top could add another convenient access mechanism to conf 
dir stuff 
Please , we are not going to add more stuff to conf dir. That is a slippery 
slope. conf dir should have well known standard files. The rest of the stuff 
should move away. Bloating up ZK just because we used to store things in conf 
dir is just inertia. We must get over it. 

Having said that there is nothing that stops us from configuring the 
JsRequestHandler to load scripts from a local file system directory which can 
be useful for certain usecases. 

bq.i.e. exactly how the Script URP works

IIRC the script URP does not reload the script automatically by watching the ZK 
node. You need to go and restart core or other dirty stuff. In a large 
SolrCloud cluster that amounts to 100's of cores getting reloaded and caches 
trashed and what not (all queries getting slow/ a lot of GC etc). Do you still 
like that idea? Let's not assume that whatever we have today is optimized to 
run on cloud mode. There are a lot of things that got carried forward from the 
legacy things which are suboptimal to cloud. Instead of clinging on to the old 
way of life we must modernize our system to adopt to the new paradigm






 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  

[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Erick Erickson (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661318#comment-14661318
 ] 

Erick Erickson commented on SOLR-7576:
--

bq: guys , editing ZK directly is fraught with huge risk. Please don't 
recommend it to users.

Of course it is. Of course not. That's why I said  Totally unsuitable for 
production of course.

But for prototyping/quick experiments on an experimental system it's waay 
cool.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661214#comment-14661214
 ] 

David Smiley commented on SOLR-7576:


I think the pain point here that Upayavira is getting at, and which I see, is 
the annoyance of having to update a file in ZooKeeper from one's file system.  
In stand-alone, there is no extra step between saving certain types of files in 
a conf/ dir on your file system and utilizing it from Solr (perhaps also 
needing a core reload or similar action).  That file list includes script URPs 
(JS and other interpreted languages supported), XSLTs, Velocity templates, 
sometimes DIH config files considering the debug mode, and file based 
Spellcheck  Suggester sources, and ExternalFileField's source file.  Perhaps 
there are others I have overlooked.  To some extent we could say absolutely 
everything in a conf/, but there are now RESTful APIs to modify various things 
in ways that don't involve the need to update the schema or solrconfig.xml 
directly.

What I'd love to see is a file system to ZooKeeper updater that works 
automatically once something is saved.  This might be a utility tool only used 
in development, or perhaps built-in to bin/solr -- either way it'd be super 
helpful.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661256#comment-14661256
 ] 

David Smiley commented on SOLR-7576:


Nice tip Erick!   -- for reference: 
https://plugins.jetbrains.com/plugin/7364?pr=idea_ce
I was just playing around with it now.  The only annoyance I've seen is that 
when editing a file I need to right-click and choose to update ZK; closing the 
window loses any edits, it appears.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661282#comment-14661282
 ] 

Noble Paul commented on SOLR-7576:
--

[~dsmiley]
bq. is the annoyance of having to update a file in ZooKeeper from one's file 
system 

I can see that pain point.

But the current model is totally broken and we MUST move away from from that
* we can't keep large amounts data in ZK
* We should not give a new feature out which requires a developer direct access 
to write to ZK.We are talking about the same thing for a toy single node system 
to large clusters with 1000's of nodes. Any screw up in ZK will have huge 
impact . It is not like a solr node going down, it is going to blow up a huge 
cluster 
* Editing files and picking them up immediately may sound ok if you don't cache 
stuff. Which is terrible for performance. So there has to be a way to tell Solr 
that a particular resource is changed
* In real production systems we need to rollback stuff if things are broken. 
This design lets you do that. You always upload a new version of the same file 
like a version control system. Uploading a new version will have no impact till 
you push it to production. Play with the new version till you are happy and 
then push it to production. The point is , the developer can't screw up the 
cluster by introducing a small syntax error 

I'm not against adding an easier dev sandbox for such features. This is just 
the first step. First of all we need to get the feature right for a deployed 
large cluster. That is the story that we will want to discuss first. This just 
one extra step for a developer in that phase. Ease of use stuff for playing 
with things is a secondary thing and we should be able to do it as a separate 
ticket. Understand that a standalone process and a huge cluster with 1000's of 
nodes will have some difference in the way stuff is deployed

guys , editing ZK directly is fraught with huge risk. Please don't recommend it 
to users. I understand that we have a lot of legacy stuff without proper APIs 
and we need to edit things directly. We MUST have a safe way of achieving the 
same through APIs (eventually). 

So please do not introduce any new feature that needs ZK write.  

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  

[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Erick Erickson (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661228#comment-14661228
 ] 

Erick Erickson commented on SOLR-7576:
--

[~dsmi...@mac.com] 
re: working with Zookeeper during development

FWIW, there's a nifty Zookeeper plugin for IntelliJ that lets you edit files 
directly on the ZK nodes. Totally unsuitable for production of course, you want 
all that in some kind of version control. But it removes the PITA of dropping 
to a command window every time you want to change _anything_ in ZK. Not exactly 
sure how it'd work with this JIRA though.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661328#comment-14661328
 ] 

David Smiley commented on SOLR-7576:


bq. we can't keep large amounts of data in ZK

Yes I am aware.  I see now why you mention this: it's because the blob store 
feature is itself a Solr collection and it keeps all versions.  I didn't know 
that until now (or lapse of memory ;-P); I had guessed (wrongly) it was just a 
REST API into ZooKeeper.   I think I better understand where you're coming from 
now.  There are other features like sources for Suggesters that I look forward 
to seeing getting upgraded to use the blob store, since it's bad to put large 
files in ZooKeeper in your conf dir.

For this specific feature (request handlers in scripting language) why should 
the executable be put in the Blob store at all?  Generally these scripts are 
going to be very small.   Instead, why not simply access it via the 
SolrResourceLoader abstraction, thus it'll work for stand-alone file-system  
ZooKeeper if SolrCloud (i.e. exactly how the Script URP works)?  A REST API on 
top could add another convenient access mechanism to conf dir stuff -- 
obviously with access control required by default though disable'able at the 
command-line for developers working on scripts locally.  AFAIK ZooKeeper 
clients (e.g. Solr) can monitor ZK paths for updates; and likewise this is 
possible of the file system in modern Java.  In summary, what I'm advocating is 
that the script be treated like the rest of the configuration for a core, even 
though technically it's code not configuration.  I don't think that distinction 
is pertinent.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660560#comment-14660560
 ] 

Noble Paul commented on SOLR-7576:
--

bq.For sure, offer the more secure, enterprise friendly version, but is there a 
valid reason not to support the method we are already using e.g. for update 
processors?

So you suggest that the way we are doing it today is good and we should follow 
the same path. Can you just list the steps involved for doing development 
process for the same ?  if I'm doing iterative development of JS how do I do it

bq.I dislike the idea that we are going to split ourselves in half - JS for an 
update processor can just be edited in place, or can be pushed to ZK via zkcli

I would like to give a big -1 for any feature which requires direct access to 
ZK. In all our security feaatures we assume that ZK writes are protected. So 
anyone who has access to ZK can compromise security .If it is already there we 
should slowly remove it. This JS RequestHandler can actually make a 
scriptupdateprocessor redundant

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660392#comment-14660392
 ] 

Noble Paul commented on SOLR-7576:
--

bq.We've had JS, we've had XSLT, etc, in our config directory for a long time. 
What is different about this new approach that it demands a more secure 
approach in all situations

We don't give an HTTP API to upload any executable (XSLT/JS). You need direct 
access to ZK. Solr is moving deeper and deeper into the enterprise and security 
cannot be an afterthought. So, if there are security hole sin our system, we 
should close them instead of opening new ones.

bq. I know that a lot of normal users will find the .system and the HTTP post 
part hard to understand

I miss that point. Users do not need to know anything about {{.system}} 
collection or anything. They should just blindly run a curl command to upload a 
new version of their JS. That is just one step. The problem starts when you 
start explaining the innards of the system. You should just say 
If you want to upload a new version of your JS, this is the command. {{curl 
-X POST -H 'Content-Type: application/octet-stream' --data-binary @test.js 
http://localhost:8983/solr/.system/blob/test}}
It does not matter to the user whether the JS is going to live in the cloud, ZK 
, filesystem or whatever, it just works. Solr is managing it for them

BTW I'm not saying this is the best design possible. Please suggest a simpler 
way to submit code to SolrCloud and we can adopt that

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Upayavira (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660489#comment-14660489
 ] 

Upayavira commented on SOLR-7576:
-

I dislike the idea that we are going to split ourselves in half - JS for an 
update processor can just be edited in place, or can be pushed to ZK via zkcli, 
whereas now, request handler JS must use an entirely different mechanism. 
That's making Solr much harder to comprehend and understand.

I understand how Solr is moving more into enterprises, but don't see how 
repeating a pattern we have been using for years is making things considerably 
worse - that's what I'd like to understand. 

For sure, offer the more secure, enterprise friendly version, but is there a 
valid reason *not* to support the method we are already using e.g. for update 
processors?

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Upayavira (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660150#comment-14660150
 ] 

Upayavira commented on SOLR-7576:
-

To most people who are used to using Solr, this development process will seem 
weird and over-complex - because the blob store functionality isn't yet much 
used.

Can't we have a mode where you just treat your JS the same as the rest of your 
configs? And also a more advanced mode to handle dynamically uploaded/signed JS 
like the one you are proposing above?

What does this development process give us that we wouldn't have with a file on 
the filesystem?

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-08-06 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660185#comment-14660185
 ] 

Noble Paul commented on SOLR-7576:
--

bq.Can't we have a mode where you just treat your JS the same as the rest of 
your configs?

JS is not configuration. I tis executable code. 

bq.To most people who are used to using Solr, this development process will 
seem weird and over-complex - because the blob store functionality isn't yet 
much used.

Overly complex ? 
If you look at the steps, most of it is one time task (that is because of the 
security implications of running executable code in your VM). and the only 
thing you need to do is a {{curl}} command to upload your JS step #4 in the 
description.
I can't imagine how can anything be simpler than this

bq.What does this development process give us that we wouldn't have with a file 
on the filesystem?

filesystem? Which file system when you are using solrcloud ?

If you wish to use it in standalone mode, then it is not supported, it can be 
another JIRA
 

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595083#comment-14595083
 ] 

David Smiley commented on SOLR-7576:


I've vaguely heard of the new blob store  signing.  I took a look at 
https://cwiki.apache.org/confluence/display/solr/Adding+Custom+Plugins+in+SolrCloud+Mode
   I get that it's secure but following that would be a real PITA during 
development.  Is there a faster way to to redeploy a changed file during 
development?  Maybe a simple script could do that?  Or better yet, a mode 
during development that would simply pick up the change from the file system.  
Oh wait; in a SolrCloud/ZooKeeper world, that may not make sense.  H :-/  
Related to this is the possibility of introducing some sort of 
AbstractSolrTestCase thingamagic (or new subclass) to test that a request 
employing some specific JavaScript (but defined in the test or as a test 
resource) is working.  That could lead to a faster dev cycle, and it encourages 
testing.

How does one go about logging? (again, not necessarily for real/production but 
for development to inspect what the heck some object is or its members).

The examples look nice; I like the use of '$'.  It'd be nice to see an example 
of taking the response of a query to pull out it's contents.

I understand that JS is a first-class-citizen but if I want to use something 
else, like Groovy, is that possible?  What remains?  Did you look at SOLR-5005?

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595091#comment-14595091
 ] 

David Smiley commented on SOLR-7576:


bq.  Groovy is a security issue . JavaScript is easy to lock down

Can you please elaborate how that is so on the JVM?  I was unaware that some 
JVM languages were more secure than others, or frankly that it even matters 
since this is trusted code (we even have to sign it).

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595089#comment-14595089
 ] 

Noble Paul commented on SOLR-7576:
--

All you need is just a curl command to load the new  script. I'll post the steps

I don't need a generic framework for all platforms . I wish to do JavaScript 
well and worry about  other languages later. Groovy is a security issue . 
JavaScript is easy to lock down
Logging and debugging is something that needs too be added.codebase has changed 
a lot and the patch in 5005 is not very relevant.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595173#comment-14595173
 ] 

Noble Paul commented on SOLR-7576:
--

Groovy has all permissions a normal java program may have . So it is equivalent 
to native java code. nashorn JS engine has some nice sandboxing features 
http://mail.openjdk.java.net/pipermail/nashorn-dev/2013-September/002010.html. 
recently ES dropped groovy citing security concerns

 I wish to develop the SDK that is tailored for a given language. It has to 
work well for that language. If we need to support other languages , we should 
invest some time and thought in providing a nice set of APIs for that instead 
of trying to provide a sub par experience by catering to all languages.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595180#comment-14595180
 ] 

David Smiley commented on SOLR-7576:


Interesting -- thanks for sharing.  I promise I won't hassle you any more on 
this issue about supporting other languages -- that can in another issue. :-)   
But I find the security issue here quite moot since the code is signed.  -- 
i.e. I could very well *want* to do whatever I want without restrictions.  
Furthermore, I read the link you pointed to and much of it doesn't apply unless 
you both run with a security manager and remove various Java things in global 
scope.  And I don't think we should do those things.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 1
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-06-21 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595185#comment-14595185
 ] 

Noble Paul commented on SOLR-7576:
--

I have changed the description to explain the dev process

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch, SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 the handler {{/js}} is implicitly registered
 To make this work
 * Solr should be started with {{-Denable.js.loading=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Sign the javascript and pass the signature in a param called {{_sig}}
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 
 Steps for developing scripts
 # start the cluster with the {{enable.js.loading}} . If you are starting 
 using our script it would be {{bin/solr start -e cloud -a 
 -Denable.js.loading=true}} . You would not need security during development 
 , so don't add the private keys to Solr
 # create {{.system}} collection {{bin/solr create -c .system}}
 # Write your javascript code . (say {{test.js}} )
 # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: 
 application/octet-stream' --data-binary @test.js 
 http://localhost:8983/solr/.system/blob/test}}
 # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
 # Edit your script and repeat from step #4 . Keep in mind that the version 
 would be bumped up every time you post a new script . So, the second time the 
 url would be {{http://host:8983/solr/gettingstarted/js/test/2}} . So on and 
 so forth
 sample programs
 1) writes a val to output
 {code:javascript}
 //empty line
 $.response().add('testkey','Test Val');
 {code}
 2)  manipulate the output to add an extra field to each doc 
 {code}
 //empty line
 var l = [];
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   }).forEach('response', function(doc) {
  doc.put('script', 'Added this 
 value');
  l.push(doc);
   });
  $.response().add('alldocs', l);
 {code}
 3)  stream through all the docs
 {code:Javascript}
 //empty line
 $.query({
   q: '*:*',
   qt: '/select',
   start:0,
   distrib:'false'
   }).pipe('response', 'docs', function(doc) { // the pipe function is 
 executed right before the response writer and right after the transformers   
  if('IT'== doc.get('genre_s')) return 
 null;
  doc.put('script', 'Added this 
 value');
  return doc;
   });
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-05-29 Thread Noble Paul (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564284#comment-14564284
 ] 

Noble Paul commented on SOLR-7576:
--

 I have missed SOLR-5005
I'm mostly done with this. Planning to commit it soon

Do you think anything is missing  in this patch you wish to include.

The objective is not exactly to make  just a JS handler. The idea is to provide 
a comprehensive API set which the functional nature of Javascript can leverage 
on

It also should have the security mechanisms which loading  executable code to 
Solr must adhere to. I'll add security to this before committing

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 example of creating a RequestHandler 
 {code:javascript}
 curl http://localhost:8983/solr/collection1/config -H 
 'Content-type:application/json'  -d '{
 create-requesthandler : {name: jshandler ,
 class:solr.JSRequestHandler, 
 defaults: {
 js: myreqhandlerjs, //this is the name of the blob in .system collection
 version:3,
 sig:mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=
 }
  }  
 }'
 {code}
 To make this work
 * Solr should be started with {{-Denable.runtime.lib=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and configure the signature if security is enabled
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-05-29 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565815#comment-14565815
 ] 

David Smiley commented on SOLR-7576:


I particularly like that in SOLR-5005 I didn't hard-code the choice of 
JavaScript.  You could perhaps borrow the script execution mechanism there.  I 
thought through the multi-threading model of Java's scripting API, or lack 
thereof ;-P,  when I wrote that.

Without having to review the patch, I think it's important to show here some 
JavaScript samples that do little interesting things, thereby showing off 
whatever utilities provided by this script offer.  It'd be good to get some 
peer review on that because it in effect creates an API that is somewhat 
public, and therefore not trivial to change on a whim later.

Good point on security.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 example of creating a RequestHandler 
 {code:javascript}
 curl http://localhost:8983/solr/collection1/config -H 
 'Content-type:application/json'  -d '{
 create-requesthandler : {name: jshandler ,
 class:solr.JSRequestHandler, 
 defaults: {
 js: myreqhandlerjs, //this is the name of the blob in .system collection
 version:3,
 sig:mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=
 }
  }  
 }'
 {code}
 To make this work
 * Solr should be started with {{-Denable.runtime.lib=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and configure the signature if security is enabled
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript

2015-05-28 Thread David Smiley (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564099#comment-14564099
 ] 

David Smiley commented on SOLR-7576:


This duplicates SOLR-5005; but the approach appears a little different.  
Perhaps you forgot about that issue?  I admit I've been too busy to finish that 
up; not that it doesn't work but could use more polish  tests.

 Implement RequestHandler in Javascript
 --

 Key: SOLR-7576
 URL: https://issues.apache.org/jira/browse/SOLR-7576
 Project: Solr
  Issue Type: New Feature
Reporter: Noble Paul
 Attachments: SOLR-7576.patch


 Solr now support dynamic loading (SOLR-7073) of components and it is secured 
 in SOLR-7126
 We can extend the same functionality with JS as well
 example of creating a RequestHandler 
 {code:javascript}
 curl http://localhost:8983/solr/collection1/config -H 
 'Content-type:application/json'  -d '{
 create-requesthandler : {name: jshandler ,
 class:solr.JSRequestHandler, 
 defaults: {
 js: myreqhandlerjs, //this is the name of the blob in .system collection
 version:3,
 sig:mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=
 }
  }  
 }'
 {code}
 To make this work
 * Solr should be started with {{-Denable.runtime.lib=true}}
 * The javascript must be loaded to the {{.system}} collection using the blob 
 store API
 * Configure the requesthandler with the JS blob name and version
 * Sign the javascript and configure the signature if security is enabled
 The {{JSRequestHandler}} is implicitly defined and it can be accessed by 
 hitting {{/js/jsname/version}} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org