RE: suggestion for new custom atomic update

2014-10-28 Thread Elran Dvir 
Shalin and Matthew, 
Thank you very much.

-Original Message-
From: Matthew Nigl [mailto:matthew.n...@gmail.com] 
Sent: Monday, October 27, 2014 7:24 PM
To: solr-user@lucene.apache.org
Subject: Re: suggestion for new custom atomic update

No problem Elran. As Shalin mentioned, you will need to do it like this:

processor class=solr.DistributedUpdateProcessorFactory/
processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / 
processor class=solr.LogUpdateProcessorFactory / processor 
class=solr.RunUpdateProcessorFactory /


On 28 October 2014 03:22, Shalin Shekhar Mangar shalinman...@gmail.com
wrote:

 Hi Elran,

 You need to explicitly specify the DistributedUpdateProcessorFactory 
 in the chain and then add your custom processor after it.

 On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote:

  Thank you very much for your suggestion.
 
  I created an update processor factory with my logic.
  I changed the update processor chain to be:
  processor class=solr.LogUpdateProcessorFactory / processor 
  class=solr.RunUpdateProcessorFactory / processor 
  class=mycode.solr_plugins.FieldManipulationProcessorFactory
 /
 
  But nothing seems to happen.
  When I move my class to be the first in the chain, the logic is 
  running (not as I want, of course. It's calculated based on the 
  update value
 rather
  than the stored value) .
  How can I define a custom update processor factory that will run 
  after DistributedUpdateProcessorFactory?
 
  Thank you very much.
 
 
  -Original Message-
  From: Matthew Nigl [mailto:matthew.n...@gmail.com]
  Sent: Monday, October 27, 2014 12:10 PM
  To: solr-user@lucene.apache.org
  Subject: Re: suggestion for new custom atomic update
 
  You can get the summed value, 13, if you add a processor after 
  DistributedUpdateProcessorFactory in the URP chain. Then one 
  possibility would be to clone this value to another field, such as 
  field_b, and run other processors on that field.
 
  Or for something more customized, you can use the 
  StatelessScriptUpdateProcessorFactory, and retrieve the value of 
  field_a
  with:
 
  var doc = cmd.solrDoc;  // org.apache.solr.common.SolrInputDocument
  var field_a = doc.getFieldValue(field_a);
 
  Note if you try to get the value of field_a before 
  DistributedUpdateProcessorFactory, then using your example with an 
  atomic update, the value would be 5 (the value of the increment from 
  the input document).
 
 
  On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote:
 
   I will explain with an example.
   Let's say field_a is sent in the update with the value of 5. 
   field_a is already stored in the document with the value 8.
   After the update field_a should have the value 13 (sum).
   The value of field_b will be based on the value of 13 and not 5.
   Is there a way in URP to know what is the value which is already 
   stored in field_a?
  
   Thank you very much.
  
   -Original Message-
   From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
   Sent: Sunday, October 26, 2014 6:07 PM
   To: solr-user
   Subject: Re: suggestion for new custom atomic update
  
   I am not sure what the problem is. URP catches all operations. So, 
   you can modify the source document to add the calculation when the 
   field_a is either new or updated.
  
   Or are you trying to calculate things across multiple documents? 
   In that case, neither mine nor your solution will work, I think.
  
   Regards,
  Alex.
   Personal: http://www.outerthoughts.com/ and @arafalov Solr 
   resources and
   newsletter: http://www.solr-start.com/ and @solrstart Solr 
   popularizers
   community: https://www.linkedin.com/groups?gid=6713853
  
  
   On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
Thanks for your response.
   
If the calculation is based on the most recent summed value of 
field_a
   and the value of field_a in the update, how can I?
   
Thanks.
   
-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
Sent: Sunday, October 26, 2014 2:11 PM
To: solr-user
Subject: RE: suggestion for new custom atomic update
   
Can't you do the calculation in custom UpdateRequestProcessor?
   
Regards,
Alex
On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:
   
Hi all,
   
Did anyone have a chance to review my idea?
   
Thanks.
   
-Original Message-
From: Elran Dvir
Sent: Monday, October 20, 2014 12:42 PM
To: solr-user
Subject: suggestion for new custom atomic update
   
Hi all,
   
This is my use  case:
I have a stored field, field_a, which is atomic updated (let's 
say by inc). field_a is stored but not indexed due to the 
large number of distinct values it can have.
I need to index field_b (I need facet and stats on it) which is 
not in the document but its value is based on a calculation

RE: suggestion for new custom atomic update

2014-10-27 Thread Elran Dvir 
I will explain with an example.
Let's say field_a is sent in the update with the value of 5. field_a is already 
stored in the document with the value 8.
After the update field_a should have the value 13 (sum).
The value of field_b will be based on the value of 13 and not 5.
Is there a way in URP to know what is the value which is already stored in 
field_a?

Thank you very much.

-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] 
Sent: Sunday, October 26, 2014 6:07 PM
To: solr-user
Subject: Re: suggestion for new custom atomic update

I am not sure what the problem is. URP catches all operations. So, you can 
modify the source document to add the calculation when the field_a is either 
new or updated.

Or are you trying to calculate things across multiple documents? In that case, 
neither mine nor your solution will work, I think.

Regards,
   Alex.
Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and 
newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers 
community: https://www.linkedin.com/groups?gid=6713853


On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
 Thanks for your response.

 If the calculation is based on the most recent summed value of field_a and 
 the value of field_a in the update, how can I?

 Thanks.

 -Original Message-
 From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
 Sent: Sunday, October 26, 2014 2:11 PM
 To: solr-user
 Subject: RE: suggestion for new custom atomic update

 Can't you do the calculation in custom UpdateRequestProcessor?

 Regards,
 Alex
 On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:

 Hi all,

 Did anyone have a chance to review my idea?

 Thanks.

 -Original Message-
 From: Elran Dvir
 Sent: Monday, October 20, 2014 12:42 PM
 To: solr-user
 Subject: suggestion for new custom atomic update

 Hi all,

 This is my use  case:
 I have a stored field, field_a, which is atomic updated (let's say by 
 inc). field_a is stored but not indexed due to the large number of 
 distinct values it can have.
 I need to index field_b (I need facet and stats on it) which is not 
 in the document but its value is based on a calculation of the recent (e.g.
 summed) value of field_a.
 There is no way to do it nowadays.
 So I thought of a new method: custom atomic update.

 There will be a new interface in Solr:

 public interface CustomAtomicUpdater {
 public void update(SolrInputDocument oldDoc, String fieldName, 
 Object
 fieldVal) ;  }

 There will be a new attribute for fields in schema.xml called 
 customAtomicUpdateClass (and all support in code, of course).
 The value is a class which is an implementation of  CustomAtomicUpdater.
 In our example it will be defined for field_a.

 In method getUpdatedDocument in DistributedUpdateProcessor.java, we 
 will add handling of custom case:

} else if (custom.equals(key)) {
 updateField = true;
 SchemaField sf = schema.getField(sif.getName());
 String customAtomicUpdaterClassName = 
 sf.getCustomAtomicUpdaterClass();
 if (customAtomicUpdaterClassName == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, There 
 is no customAtomicUpdaterClass defined for  + sif + .);
 }
 CustomAtomicUpdater updater = schema.getResourceLoader()
 .newInstance(customAtomicUpdaterClassName,
 CustomAtomicUpdater.class);
 if (updater == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, Was 
 unable to create instance of  + customAtomicUpdaterClassName + .);
 }
 updater.update(oldDoc, sif.getName(), fieldVal);

   }

 In my implementation I will sum field_a (oldvalue + newvalue) and 
 update field_b according to my logic.

 Example of use:
 add
   doc
 field name=field_a update=custom128/field
   /doc
 /add

 What do say about my suggestion?

 Thanks.



 Email secured by Check Point

Email secured by Check Point

Re: suggestion for new custom atomic update

2014-10-27 Thread Matthew Nigl 
You can get the summed value, 13, if you add a processor after
DistributedUpdateProcessorFactory in the URP chain. Then one possibility
would be to clone this value to another field, such as field_b, and run
other processors on that field.

Or for something more customized, you can use
the StatelessScriptUpdateProcessorFactory, and retrieve the value of
field_a with:

var doc = cmd.solrDoc;  // org.apache.solr.common.SolrInputDocument
var field_a = doc.getFieldValue(field_a);

Note if you try to get the value of field_a before
DistributedUpdateProcessorFactory, then using your example with an atomic
update, the value would be 5 (the value of the increment from the input
document).


On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote:

 I will explain with an example.
 Let's say field_a is sent in the update with the value of 5. field_a is
 already stored in the document with the value 8.
 After the update field_a should have the value 13 (sum).
 The value of field_b will be based on the value of 13 and not 5.
 Is there a way in URP to know what is the value which is already stored in
 field_a?

 Thank you very much.

 -Original Message-
 From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
 Sent: Sunday, October 26, 2014 6:07 PM
 To: solr-user
 Subject: Re: suggestion for new custom atomic update

 I am not sure what the problem is. URP catches all operations. So, you can
 modify the source document to add the calculation when the field_a is
 either new or updated.

 Or are you trying to calculate things across multiple documents? In that
 case, neither mine nor your solution will work, I think.

 Regards,
Alex.
 Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and
 newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers
 community: https://www.linkedin.com/groups?gid=6713853


 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
  Thanks for your response.
 
  If the calculation is based on the most recent summed value of field_a
 and the value of field_a in the update, how can I?
 
  Thanks.
 
  -Original Message-
  From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
  Sent: Sunday, October 26, 2014 2:11 PM
  To: solr-user
  Subject: RE: suggestion for new custom atomic update
 
  Can't you do the calculation in custom UpdateRequestProcessor?
 
  Regards,
  Alex
  On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:
 
  Hi all,
 
  Did anyone have a chance to review my idea?
 
  Thanks.
 
  -Original Message-
  From: Elran Dvir
  Sent: Monday, October 20, 2014 12:42 PM
  To: solr-user
  Subject: suggestion for new custom atomic update
 
  Hi all,
 
  This is my use  case:
  I have a stored field, field_a, which is atomic updated (let's say by
  inc). field_a is stored but not indexed due to the large number of
  distinct values it can have.
  I need to index field_b (I need facet and stats on it) which is not
  in the document but its value is based on a calculation of the recent
 (e.g.
  summed) value of field_a.
  There is no way to do it nowadays.
  So I thought of a new method: custom atomic update.
 
  There will be a new interface in Solr:
 
  public interface CustomAtomicUpdater {
  public void update(SolrInputDocument oldDoc, String fieldName,
  Object
  fieldVal) ;  }
 
  There will be a new attribute for fields in schema.xml called
  customAtomicUpdateClass (and all support in code, of course).
  The value is a class which is an implementation of  CustomAtomicUpdater.
  In our example it will be defined for field_a.
 
  In method getUpdatedDocument in DistributedUpdateProcessor.java, we
  will add handling of custom case:
 
 } else if (custom.equals(key)) {
  updateField = true;
  SchemaField sf = schema.getField(sif.getName());
  String customAtomicUpdaterClassName =
  sf.getCustomAtomicUpdaterClass();
  if (customAtomicUpdaterClassName == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, There
  is no customAtomicUpdaterClass defined for  + sif + .);
  }
  CustomAtomicUpdater updater = schema.getResourceLoader()
  .newInstance(customAtomicUpdaterClassName,
  CustomAtomicUpdater.class);
  if (updater == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, Was
  unable to create instance of  + customAtomicUpdaterClassName + .);
  }
  updater.update(oldDoc, sif.getName(), fieldVal);
 
}
 
  In my implementation I will sum field_a (oldvalue + newvalue) and
  update field_b according to my logic.
 
  Example of use:
  add
doc
  field name=field_a update=custom128/field
/doc
  /add
 
  What do say about my suggestion?
 
  Thanks.
 
 
 
  Email secured by Check Point

 Email secured by Check Point

RE: suggestion for new custom atomic update

2014-10-27 Thread Elran Dvir 
Thank you very much for your suggestion.

I created an update processor factory with my logic. 
I changed the update processor chain to be:
processor class=solr.LogUpdateProcessorFactory /
processor class=solr.RunUpdateProcessorFactory /
processor class=mycode.solr_plugins.FieldManipulationProcessorFactory /

But nothing seems to happen.
When I move my class to be the first in the chain, the logic is running (not as 
I want, of course. It's calculated based on the update value rather than the 
stored value) .
How can I define a custom update processor factory that will run after  
DistributedUpdateProcessorFactory?

Thank you very much.


-Original Message-
From: Matthew Nigl [mailto:matthew.n...@gmail.com] 
Sent: Monday, October 27, 2014 12:10 PM
To: solr-user@lucene.apache.org
Subject: Re: suggestion for new custom atomic update

You can get the summed value, 13, if you add a processor after 
DistributedUpdateProcessorFactory in the URP chain. Then one possibility would 
be to clone this value to another field, such as field_b, and run other 
processors on that field.

Or for something more customized, you can use the 
StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with:

var doc = cmd.solrDoc;  // org.apache.solr.common.SolrInputDocument
var field_a = doc.getFieldValue(field_a);

Note if you try to get the value of field_a before 
DistributedUpdateProcessorFactory, then using your example with an atomic 
update, the value would be 5 (the value of the increment from the input 
document).


On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote:

 I will explain with an example.
 Let's say field_a is sent in the update with the value of 5. field_a 
 is already stored in the document with the value 8.
 After the update field_a should have the value 13 (sum).
 The value of field_b will be based on the value of 13 and not 5.
 Is there a way in URP to know what is the value which is already 
 stored in field_a?

 Thank you very much.

 -Original Message-
 From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
 Sent: Sunday, October 26, 2014 6:07 PM
 To: solr-user
 Subject: Re: suggestion for new custom atomic update

 I am not sure what the problem is. URP catches all operations. So, you 
 can modify the source document to add the calculation when the field_a 
 is either new or updated.

 Or are you trying to calculate things across multiple documents? In 
 that case, neither mine nor your solution will work, I think.

 Regards,
Alex.
 Personal: http://www.outerthoughts.com/ and @arafalov Solr resources 
 and
 newsletter: http://www.solr-start.com/ and @solrstart Solr 
 popularizers
 community: https://www.linkedin.com/groups?gid=6713853


 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
  Thanks for your response.
 
  If the calculation is based on the most recent summed value of 
  field_a
 and the value of field_a in the update, how can I?
 
  Thanks.
 
  -Original Message-
  From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
  Sent: Sunday, October 26, 2014 2:11 PM
  To: solr-user
  Subject: RE: suggestion for new custom atomic update
 
  Can't you do the calculation in custom UpdateRequestProcessor?
 
  Regards,
  Alex
  On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:
 
  Hi all,
 
  Did anyone have a chance to review my idea?
 
  Thanks.
 
  -Original Message-
  From: Elran Dvir
  Sent: Monday, October 20, 2014 12:42 PM
  To: solr-user
  Subject: suggestion for new custom atomic update
 
  Hi all,
 
  This is my use  case:
  I have a stored field, field_a, which is atomic updated (let's say 
  by inc). field_a is stored but not indexed due to the large 
  number of distinct values it can have.
  I need to index field_b (I need facet and stats on it) which is not 
  in the document but its value is based on a calculation of the 
  recent
 (e.g.
  summed) value of field_a.
  There is no way to do it nowadays.
  So I thought of a new method: custom atomic update.
 
  There will be a new interface in Solr:
 
  public interface CustomAtomicUpdater {
  public void update(SolrInputDocument oldDoc, String fieldName, 
  Object
  fieldVal) ;  }
 
  There will be a new attribute for fields in schema.xml called 
  customAtomicUpdateClass (and all support in code, of course).
  The value is a class which is an implementation of  CustomAtomicUpdater.
  In our example it will be defined for field_a.
 
  In method getUpdatedDocument in DistributedUpdateProcessor.java, 
  we will add handling of custom case:
 
 } else if (custom.equals(key)) {
  updateField = true;
  SchemaField sf = schema.getField(sif.getName());
  String customAtomicUpdaterClassName = 
  sf.getCustomAtomicUpdaterClass();
  if (customAtomicUpdaterClassName == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, There 
  is no customAtomicUpdaterClass defined

Re: suggestion for new custom atomic update

2014-10-27 Thread Shalin Shekhar Mangar 
Hi Elran,

You need to explicitly specify the DistributedUpdateProcessorFactory in the
chain and then add your custom processor after it.

On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote:

 Thank you very much for your suggestion.

 I created an update processor factory with my logic.
 I changed the update processor chain to be:
 processor class=solr.LogUpdateProcessorFactory /
 processor class=solr.RunUpdateProcessorFactory /
 processor class=mycode.solr_plugins.FieldManipulationProcessorFactory /

 But nothing seems to happen.
 When I move my class to be the first in the chain, the logic is running
 (not as I want, of course. It's calculated based on the update value rather
 than the stored value) .
 How can I define a custom update processor factory that will run after
 DistributedUpdateProcessorFactory?

 Thank you very much.


 -Original Message-
 From: Matthew Nigl [mailto:matthew.n...@gmail.com]
 Sent: Monday, October 27, 2014 12:10 PM
 To: solr-user@lucene.apache.org
 Subject: Re: suggestion for new custom atomic update

 You can get the summed value, 13, if you add a processor after
 DistributedUpdateProcessorFactory in the URP chain. Then one possibility
 would be to clone this value to another field, such as field_b, and run
 other processors on that field.

 Or for something more customized, you can use the
 StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a
 with:

 var doc = cmd.solrDoc;  // org.apache.solr.common.SolrInputDocument
 var field_a = doc.getFieldValue(field_a);

 Note if you try to get the value of field_a before
 DistributedUpdateProcessorFactory, then using your example with an atomic
 update, the value would be 5 (the value of the increment from the input
 document).


 On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote:

  I will explain with an example.
  Let's say field_a is sent in the update with the value of 5. field_a
  is already stored in the document with the value 8.
  After the update field_a should have the value 13 (sum).
  The value of field_b will be based on the value of 13 and not 5.
  Is there a way in URP to know what is the value which is already
  stored in field_a?
 
  Thank you very much.
 
  -Original Message-
  From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
  Sent: Sunday, October 26, 2014 6:07 PM
  To: solr-user
  Subject: Re: suggestion for new custom atomic update
 
  I am not sure what the problem is. URP catches all operations. So, you
  can modify the source document to add the calculation when the field_a
  is either new or updated.
 
  Or are you trying to calculate things across multiple documents? In
  that case, neither mine nor your solution will work, I think.
 
  Regards,
 Alex.
  Personal: http://www.outerthoughts.com/ and @arafalov Solr resources
  and
  newsletter: http://www.solr-start.com/ and @solrstart Solr
  popularizers
  community: https://www.linkedin.com/groups?gid=6713853
 
 
  On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
   Thanks for your response.
  
   If the calculation is based on the most recent summed value of
   field_a
  and the value of field_a in the update, how can I?
  
   Thanks.
  
   -Original Message-
   From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
   Sent: Sunday, October 26, 2014 2:11 PM
   To: solr-user
   Subject: RE: suggestion for new custom atomic update
  
   Can't you do the calculation in custom UpdateRequestProcessor?
  
   Regards,
   Alex
   On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:
  
   Hi all,
  
   Did anyone have a chance to review my idea?
  
   Thanks.
  
   -Original Message-
   From: Elran Dvir
   Sent: Monday, October 20, 2014 12:42 PM
   To: solr-user
   Subject: suggestion for new custom atomic update
  
   Hi all,
  
   This is my use  case:
   I have a stored field, field_a, which is atomic updated (let's say
   by inc). field_a is stored but not indexed due to the large
   number of distinct values it can have.
   I need to index field_b (I need facet and stats on it) which is not
   in the document but its value is based on a calculation of the
   recent
  (e.g.
   summed) value of field_a.
   There is no way to do it nowadays.
   So I thought of a new method: custom atomic update.
  
   There will be a new interface in Solr:
  
   public interface CustomAtomicUpdater {
   public void update(SolrInputDocument oldDoc, String fieldName,
   Object
   fieldVal) ;  }
  
   There will be a new attribute for fields in schema.xml called
   customAtomicUpdateClass (and all support in code, of course).
   The value is a class which is an implementation of
 CustomAtomicUpdater.
   In our example it will be defined for field_a.
  
   In method getUpdatedDocument in DistributedUpdateProcessor.java,
   we will add handling of custom case:
  
  } else if (custom.equals(key)) {
   updateField = true

Re: suggestion for new custom atomic update

2014-10-27 Thread Matthew Nigl 
No problem Elran. As Shalin mentioned, you will need to do it like this:

processor class=solr.DistributedUpdateProcessorFactory/
processor class=mycode.solr_plugins.FieldManipulationProcessorFactory /
processor class=solr.LogUpdateProcessorFactory /
processor class=solr.RunUpdateProcessorFactory /


On 28 October 2014 03:22, Shalin Shekhar Mangar shalinman...@gmail.com
wrote:

 Hi Elran,

 You need to explicitly specify the DistributedUpdateProcessorFactory in the
 chain and then add your custom processor after it.

 On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote:

  Thank you very much for your suggestion.
 
  I created an update processor factory with my logic.
  I changed the update processor chain to be:
  processor class=solr.LogUpdateProcessorFactory /
  processor class=solr.RunUpdateProcessorFactory /
  processor class=mycode.solr_plugins.FieldManipulationProcessorFactory
 /
 
  But nothing seems to happen.
  When I move my class to be the first in the chain, the logic is running
  (not as I want, of course. It's calculated based on the update value
 rather
  than the stored value) .
  How can I define a custom update processor factory that will run after
  DistributedUpdateProcessorFactory?
 
  Thank you very much.
 
 
  -Original Message-
  From: Matthew Nigl [mailto:matthew.n...@gmail.com]
  Sent: Monday, October 27, 2014 12:10 PM
  To: solr-user@lucene.apache.org
  Subject: Re: suggestion for new custom atomic update
 
  You can get the summed value, 13, if you add a processor after
  DistributedUpdateProcessorFactory in the URP chain. Then one possibility
  would be to clone this value to another field, such as field_b, and run
  other processors on that field.
 
  Or for something more customized, you can use the
  StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a
  with:
 
  var doc = cmd.solrDoc;  // org.apache.solr.common.SolrInputDocument
  var field_a = doc.getFieldValue(field_a);
 
  Note if you try to get the value of field_a before
  DistributedUpdateProcessorFactory, then using your example with an atomic
  update, the value would be 5 (the value of the increment from the input
  document).
 
 
  On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote:
 
   I will explain with an example.
   Let's say field_a is sent in the update with the value of 5. field_a
   is already stored in the document with the value 8.
   After the update field_a should have the value 13 (sum).
   The value of field_b will be based on the value of 13 and not 5.
   Is there a way in URP to know what is the value which is already
   stored in field_a?
  
   Thank you very much.
  
   -Original Message-
   From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
   Sent: Sunday, October 26, 2014 6:07 PM
   To: solr-user
   Subject: Re: suggestion for new custom atomic update
  
   I am not sure what the problem is. URP catches all operations. So, you
   can modify the source document to add the calculation when the field_a
   is either new or updated.
  
   Or are you trying to calculate things across multiple documents? In
   that case, neither mine nor your solution will work, I think.
  
   Regards,
  Alex.
   Personal: http://www.outerthoughts.com/ and @arafalov Solr resources
   and
   newsletter: http://www.solr-start.com/ and @solrstart Solr
   popularizers
   community: https://www.linkedin.com/groups?gid=6713853
  
  
   On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
Thanks for your response.
   
If the calculation is based on the most recent summed value of
field_a
   and the value of field_a in the update, how can I?
   
Thanks.
   
-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
Sent: Sunday, October 26, 2014 2:11 PM
To: solr-user
Subject: RE: suggestion for new custom atomic update
   
Can't you do the calculation in custom UpdateRequestProcessor?
   
Regards,
Alex
On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:
   
Hi all,
   
Did anyone have a chance to review my idea?
   
Thanks.
   
-Original Message-
From: Elran Dvir
Sent: Monday, October 20, 2014 12:42 PM
To: solr-user
Subject: suggestion for new custom atomic update
   
Hi all,
   
This is my use  case:
I have a stored field, field_a, which is atomic updated (let's say
by inc). field_a is stored but not indexed due to the large
number of distinct values it can have.
I need to index field_b (I need facet and stats on it) which is not
in the document but its value is based on a calculation of the
recent
   (e.g.
summed) value of field_a.
There is no way to do it nowadays.
So I thought of a new method: custom atomic update.
   
There will be a new interface in Solr:
   
public interface CustomAtomicUpdater {
public void update(SolrInputDocument

RE: suggestion for new custom atomic update

2014-10-26 Thread Elran Dvir 
Hi all,

Did anyone have a chance to review my idea?

Thanks.

-Original Message-
From: Elran Dvir 
Sent: Monday, October 20, 2014 12:42 PM
To: solr-user
Subject: suggestion for new custom atomic update

Hi all,

This is my use  case:
I have a stored field, field_a, which is atomic updated (let's say by inc). 
field_a is stored but not indexed due to the large number of distinct values it 
can have.
I need to index field_b (I need facet and stats on it) which is not in the 
document but its value is based on a calculation of the recent (e.g. summed) 
value of field_a.
There is no way to do it nowadays.
So I thought of a new method: custom atomic update.

There will be a new interface in Solr:

public interface CustomAtomicUpdater {
public void update(SolrInputDocument oldDoc, String fieldName, Object 
fieldVal) ;  }

There will be a new attribute for fields in schema.xml called 
customAtomicUpdateClass (and all support in code, of course). 
The value is a class which is an implementation of  CustomAtomicUpdater.
In our example it will be defined for field_a.

In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add 
handling of custom case:

   } else if (custom.equals(key)) {
updateField = true;
SchemaField sf = schema.getField(sif.getName());
String customAtomicUpdaterClassName = 
sf.getCustomAtomicUpdaterClass();
if (customAtomicUpdaterClassName == null) {
  throw new SolrException(ErrorCode.BAD_REQUEST, There is no 
customAtomicUpdaterClass defined for  + sif + .);
}
CustomAtomicUpdater updater = schema.getResourceLoader()
.newInstance(customAtomicUpdaterClassName, 
CustomAtomicUpdater.class);
if (updater == null) {
  throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to 
create instance of  + customAtomicUpdaterClassName + .);
}
updater.update(oldDoc, sif.getName(), fieldVal);
  
  }

In my implementation I will sum field_a (oldvalue + newvalue) and update 
field_b according to my logic.

Example of use:
add
  doc
field name=field_a update=custom128/field
  /doc
/add

What do say about my suggestion?

Thanks.

RE: suggestion for new custom atomic update

2014-10-26 Thread Alexandre Rafalovitch 
Can't you do the calculation in custom UpdateRequestProcessor?

Regards,
Alex
On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:

 Hi all,

 Did anyone have a chance to review my idea?

 Thanks.

 -Original Message-
 From: Elran Dvir
 Sent: Monday, October 20, 2014 12:42 PM
 To: solr-user
 Subject: suggestion for new custom atomic update

 Hi all,

 This is my use  case:
 I have a stored field, field_a, which is atomic updated (let's say by
 inc). field_a is stored but not indexed due to the large number of
 distinct values it can have.
 I need to index field_b (I need facet and stats on it) which is not in the
 document but its value is based on a calculation of the recent (e.g.
 summed) value of field_a.
 There is no way to do it nowadays.
 So I thought of a new method: custom atomic update.

 There will be a new interface in Solr:

 public interface CustomAtomicUpdater {
 public void update(SolrInputDocument oldDoc, String fieldName, Object
 fieldVal) ;  }

 There will be a new attribute for fields in schema.xml called
 customAtomicUpdateClass (and all support in code, of course).
 The value is a class which is an implementation of  CustomAtomicUpdater.
 In our example it will be defined for field_a.

 In method getUpdatedDocument in DistributedUpdateProcessor.java, we will
 add handling of custom case:

} else if (custom.equals(key)) {
 updateField = true;
 SchemaField sf = schema.getField(sif.getName());
 String customAtomicUpdaterClassName =
 sf.getCustomAtomicUpdaterClass();
 if (customAtomicUpdaterClassName == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, There is no
 customAtomicUpdaterClass defined for  + sif + .);
 }
 CustomAtomicUpdater updater = schema.getResourceLoader()
 .newInstance(customAtomicUpdaterClassName,
 CustomAtomicUpdater.class);
 if (updater == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, Was unable
 to create instance of  + customAtomicUpdaterClassName + .);
 }
 updater.update(oldDoc, sif.getName(), fieldVal);

   }

 In my implementation I will sum field_a (oldvalue + newvalue) and update
 field_b according to my logic.

 Example of use:
 add
   doc
 field name=field_a update=custom128/field
   /doc
 /add

 What do say about my suggestion?

 Thanks.

RE: suggestion for new custom atomic update

2014-10-26 Thread Elran Dvir 
Thanks for your response.

If the calculation is based on the most recent summed value of field_a and the 
value of field_a in the update, how can I?

Thanks.

-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] 
Sent: Sunday, October 26, 2014 2:11 PM
To: solr-user
Subject: RE: suggestion for new custom atomic update

Can't you do the calculation in custom UpdateRequestProcessor?

Regards,
Alex
On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:

 Hi all,

 Did anyone have a chance to review my idea?

 Thanks.

 -Original Message-
 From: Elran Dvir
 Sent: Monday, October 20, 2014 12:42 PM
 To: solr-user
 Subject: suggestion for new custom atomic update

 Hi all,

 This is my use  case:
 I have a stored field, field_a, which is atomic updated (let's say by 
 inc). field_a is stored but not indexed due to the large number of 
 distinct values it can have.
 I need to index field_b (I need facet and stats on it) which is not in 
 the document but its value is based on a calculation of the recent (e.g.
 summed) value of field_a.
 There is no way to do it nowadays.
 So I thought of a new method: custom atomic update.

 There will be a new interface in Solr:

 public interface CustomAtomicUpdater {
 public void update(SolrInputDocument oldDoc, String fieldName, 
 Object
 fieldVal) ;  }

 There will be a new attribute for fields in schema.xml called 
 customAtomicUpdateClass (and all support in code, of course).
 The value is a class which is an implementation of  CustomAtomicUpdater.
 In our example it will be defined for field_a.

 In method getUpdatedDocument in DistributedUpdateProcessor.java, we 
 will add handling of custom case:

} else if (custom.equals(key)) {
 updateField = true;
 SchemaField sf = schema.getField(sif.getName());
 String customAtomicUpdaterClassName = 
 sf.getCustomAtomicUpdaterClass();
 if (customAtomicUpdaterClassName == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, There is 
 no customAtomicUpdaterClass defined for  + sif + .);
 }
 CustomAtomicUpdater updater = schema.getResourceLoader()
 .newInstance(customAtomicUpdaterClassName,
 CustomAtomicUpdater.class);
 if (updater == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, Was 
 unable to create instance of  + customAtomicUpdaterClassName + .);
 }
 updater.update(oldDoc, sif.getName(), fieldVal);

   }

 In my implementation I will sum field_a (oldvalue + newvalue) and 
 update field_b according to my logic.

 Example of use:
 add
   doc
 field name=field_a update=custom128/field
   /doc
 /add

 What do say about my suggestion?

 Thanks.



Email secured by Check Point

Re: suggestion for new custom atomic update

2014-10-26 Thread Alexandre Rafalovitch 
I am not sure what the problem is. URP catches all operations. So, you
can modify the source document to add the calculation when the field_a
is either new or updated.

Or are you trying to calculate things across multiple documents? In
that case, neither mine nor your solution will work, I think.

Regards,
   Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853


On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote:
 Thanks for your response.

 If the calculation is based on the most recent summed value of field_a and 
 the value of field_a in the update, how can I?

 Thanks.

 -Original Message-
 From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
 Sent: Sunday, October 26, 2014 2:11 PM
 To: solr-user
 Subject: RE: suggestion for new custom atomic update

 Can't you do the calculation in custom UpdateRequestProcessor?

 Regards,
 Alex
 On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote:

 Hi all,

 Did anyone have a chance to review my idea?

 Thanks.

 -Original Message-
 From: Elran Dvir
 Sent: Monday, October 20, 2014 12:42 PM
 To: solr-user
 Subject: suggestion for new custom atomic update

 Hi all,

 This is my use  case:
 I have a stored field, field_a, which is atomic updated (let's say by
 inc). field_a is stored but not indexed due to the large number of
 distinct values it can have.
 I need to index field_b (I need facet and stats on it) which is not in
 the document but its value is based on a calculation of the recent (e.g.
 summed) value of field_a.
 There is no way to do it nowadays.
 So I thought of a new method: custom atomic update.

 There will be a new interface in Solr:

 public interface CustomAtomicUpdater {
 public void update(SolrInputDocument oldDoc, String fieldName,
 Object
 fieldVal) ;  }

 There will be a new attribute for fields in schema.xml called
 customAtomicUpdateClass (and all support in code, of course).
 The value is a class which is an implementation of  CustomAtomicUpdater.
 In our example it will be defined for field_a.

 In method getUpdatedDocument in DistributedUpdateProcessor.java, we
 will add handling of custom case:

} else if (custom.equals(key)) {
 updateField = true;
 SchemaField sf = schema.getField(sif.getName());
 String customAtomicUpdaterClassName =
 sf.getCustomAtomicUpdaterClass();
 if (customAtomicUpdaterClassName == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, There is
 no customAtomicUpdaterClass defined for  + sif + .);
 }
 CustomAtomicUpdater updater = schema.getResourceLoader()
 .newInstance(customAtomicUpdaterClassName,
 CustomAtomicUpdater.class);
 if (updater == null) {
   throw new SolrException(ErrorCode.BAD_REQUEST, Was
 unable to create instance of  + customAtomicUpdaterClassName + .);
 }
 updater.update(oldDoc, sif.getName(), fieldVal);

   }

 In my implementation I will sum field_a (oldvalue + newvalue) and
 update field_b according to my logic.

 Example of use:
 add
   doc
 field name=field_a update=custom128/field
   /doc
 /add

 What do say about my suggestion?

 Thanks.



 Email secured by Check Point

suggestion for new custom atomic update

2014-10-20 Thread Elran Dvir 
Hi all,

This is my use  case:
I have a stored field, field_a, which is atomic updated (let's say by inc). 
field_a is stored but not indexed due to the large number of distinct values it 
can have.
I need to index field_b (I need facet and stats on it) which is not in the 
document but its value is based on a calculation of the recent (e.g. summed) 
value of field_a.
There is no way to do it nowadays.
So I thought of a new method: custom atomic update.

There will be a new interface in Solr:

public interface CustomAtomicUpdater {
public void update(SolrInputDocument oldDoc, String fieldName, Object 
fieldVal) ;
 }

There will be a new attribute for fields in schema.xml called 
customAtomicUpdateClass (and all support in code, of course).
The value is a class which is an implementation of  CustomAtomicUpdater.
In our example it will be defined for field_a.

In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add 
handling of custom case:

   } else if (custom.equals(key)) {
updateField = true;
SchemaField sf = schema.getField(sif.getName());
String customAtomicUpdaterClassName = 
sf.getCustomAtomicUpdaterClass();
if (customAtomicUpdaterClassName == null) {
  throw new SolrException(ErrorCode.BAD_REQUEST, There is no 
customAtomicUpdaterClass defined for  + sif + .);
}
CustomAtomicUpdater updater = schema.getResourceLoader()
.newInstance(customAtomicUpdaterClassName, 
CustomAtomicUpdater.class);
if (updater == null) {
  throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to 
create instance of  + customAtomicUpdaterClassName + .);
}
updater.update(oldDoc, sif.getName(), fieldVal);

  }

In my implementation I will sum field_a (oldvalue + newvalue) and update 
field_b according to my logic.

Example of use:
add
  doc
field name=field_a update=custom128/field
  /doc
/add

What do say about my suggestion?

Thanks.