Re: QueryElevationComponent : hot update of elevate.xml
Hi, On May 12, 2009, at 12:33 , Nicolas Pastorino wrote: Hi, On May 7, 2009, at 6:03 , Noble Paul നോബിള് नोब्ळ् wrote: going forward the java based replication is going to be the preferred means replicating index. It does not support replicating files in the dataDir , it only supports replicating index files and conf files (files in conf dir). I was unaware of the fact that it was possible to put the elevate.xml in dataDir. reloading on commit is a trivial for a search component. it can register itself to be an even listener for commit and do a reload of elevate.xml. This can be a configuration parameter. str name=refreshOnCommmittrue/str Thanks for these nice tips and recommendations. I attached a new version of this requestHandler here : https:// issues.apache.org/jira/browse/SOLR-1147. As proposed in the comments of this issue, it was rescoped to a more generic feature : Updating configuration files through HTTP ( https://issues.apache.org/jira/browse/SOLR-1147 ). Discussion continues in the issue tracker and right here ! Regards, -- Nicolas Pastorino eZ Systems ( Labs )
Re: QueryElevationComponent : hot update of elevate.xml
Hi, On May 7, 2009, at 6:03 , Noble Paul നോബിള് नोब्ळ् wrote: going forward the java based replication is going to be the preferred means replicating index. It does not support replicating files in the dataDir , it only supports replicating index files and conf files (files in conf dir). I was unaware of the fact that it was possible to put the elevate.xml in dataDir. reloading on commit is a trivial for a search component. it can register itself to be an even listener for commit and do a reload of elevate.xml. This can be a configuration parameter. str name=refreshOnCommmittrue/str Thanks for these nice tips and recommendations. I attached a new version of this requestHandler here : https:// issues.apache.org/jira/browse/SOLR-1147. Would this requestHandler be of any general use and could be part of Solr's trunk ? Thanks in advance, -- Nicolas Pastorino - eZ Labs On Wed, May 6, 2009 at 7:08 PM, Nicolas Pastorino n...@ez.no wrote: On May 6, 2009, at 15:17 , Noble Paul നോബിള് नोब्ळ् wrote: Why would you want to write it to the data dir? why can't it be in the same place (conf) ? Well, fact is that the QueryElevationComponent loads the configuration file ( elevate.xml ) either from the data dir, either from the conf dir. Which means that existing setups using this component maybe using either location. That is the only reason why i judged necessary to keep supporting this flexibility. But this could be simplified, forcing the elevate.xml file to be in the conf dir, and having a system ( the one you proposed, or the request handler attached to the issue ) to reload the configuration from the conf dir ( which is currently not possible. While when elevate.xml is stored in the dataDir, triggering a commit would reload it ). I was just unsure about all ins and outs of the Elevation system, and then did not remove this flexibility. Thanks for your expert eye on this ! On Wed, May 6, 2009 at 6:43 PM, Nicolas Pastorino n...@ez.no wrote: Hello, On May 6, 2009, at 15:02 , Noble Paul നോബിള് नोब्ळ् wrote: The elevate.xml is loaded from conf dir when the core is reloaded . if you post the new xml you will have to reload the core. A simple solution would be to write a RequestHandler which extends QueryElevationComponent which can be a listener for commit and call an super.inform() on that event You may want to have a look at this issue : https://issues.apache.org/jira/browse/SOLR-1147 The proposed solution ( new request handler, attached to the ticket ), solves the issue in both cases : * when elevate.xml is in the DataDir. * when elevate.xml is in the conf dir. Basically this new request handler receives, as XML, the new configuration, writes it to the right place ( some logic was copied from the QueryElevationComponent.inform() code ), and then calls the inform() method on the QueryElevationComponent for the current core, as you suggested above, to reload the Elevate configuration. -- Nicolas On Fri, Apr 10, 2009 at 5:18 PM, Nicolas Pastorino n...@ez.no wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no --
Re: QueryElevationComponent : hot update of elevate.xml
Hi, On Apr 10, 2009, at 16:51 , Ryan McKinley wrote: On Apr 10, 2009, at 7:48 AM, Nicolas Pastorino wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Perhaps look at implementing custom RequestHandler: http://wiki.apache.org/solr/SolrRequestHandler maybe it could POST the new elevate.xm and then save it to the right place and call commit... Thanks Ryan for your answer. Here is the related issue in JIRA : https://issues.apache.org/jira/ browse/SOLR-1147 It includes a Request Handler, as you advised, which takes care of all this. I guess we can follow up on this on JIRA from now on. -- Nicolas eZ Systems
Re: QueryElevationComponent : hot update of elevate.xml
The elevate.xml is loaded from conf dir when the core is reloaded . if you post the new xml you will have to reload the core. A simple solution would be to write a RequestHandler which extends QueryElevationComponent which can be a listener for commit and call an super.inform() on that event On Fri, Apr 10, 2009 at 5:18 PM, Nicolas Pastorino n...@ez.no wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: QueryElevationComponent : hot update of elevate.xml
Hello, On May 6, 2009, at 15:02 , Noble Paul നോബിള് नोब्ळ् wrote: The elevate.xml is loaded from conf dir when the core is reloaded . if you post the new xml you will have to reload the core. A simple solution would be to write a RequestHandler which extends QueryElevationComponent which can be a listener for commit and call an super.inform() on that event You may want to have a look at this issue : https://issues.apache.org/ jira/browse/SOLR-1147 The proposed solution ( new request handler, attached to the ticket ), solves the issue in both cases : * when elevate.xml is in the DataDir. * when elevate.xml is in the conf dir. Basically this new request handler receives, as XML, the new configuration, writes it to the right place ( some logic was copied from the QueryElevationComponent.inform() code ), and then calls the inform() method on the QueryElevationComponent for the current core, as you suggested above, to reload the Elevate configuration. -- Nicolas On Fri, Apr 10, 2009 at 5:18 PM, Nicolas Pastorino n...@ez.no wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no
Re: QueryElevationComponent : hot update of elevate.xml
On May 6, 2009, at 15:17 , Noble Paul നോബിള് नोब्ळ् wrote: Why would you want to write it to the data dir? why can't it be in the same place (conf) ? Well, fact is that the QueryElevationComponent loads the configuration file ( elevate.xml ) either from the data dir, either from the conf dir. Which means that existing setups using this component maybe using either location. That is the only reason why i judged necessary to keep supporting this flexibility. But this could be simplified, forcing the elevate.xml file to be in the conf dir, and having a system ( the one you proposed, or the request handler attached to the issue ) to reload the configuration from the conf dir ( which is currently not possible. While when elevate.xml is stored in the dataDir, triggering a commit would reload it ). I was just unsure about all ins and outs of the Elevation system, and then did not remove this flexibility. Thanks for your expert eye on this ! On Wed, May 6, 2009 at 6:43 PM, Nicolas Pastorino n...@ez.no wrote: Hello, On May 6, 2009, at 15:02 , Noble Paul നോബിള് नोब्ळ् wrote: The elevate.xml is loaded from conf dir when the core is reloaded . if you post the new xml you will have to reload the core. A simple solution would be to write a RequestHandler which extends QueryElevationComponent which can be a listener for commit and call an super.inform() on that event You may want to have a look at this issue : https://issues.apache.org/jira/browse/SOLR-1147 The proposed solution ( new request handler, attached to the ticket ), solves the issue in both cases : * when elevate.xml is in the DataDir. * when elevate.xml is in the conf dir. Basically this new request handler receives, as XML, the new configuration, writes it to the right place ( some logic was copied from the QueryElevationComponent.inform() code ), and then calls the inform () method on the QueryElevationComponent for the current core, as you suggested above, to reload the Elevate configuration. -- Nicolas On Fri, Apr 10, 2009 at 5:18 PM, Nicolas Pastorino n...@ez.no wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no
Re: QueryElevationComponent : hot update of elevate.xml
going forward the java based replication is going to be the preferred means replicating index. It does not support replicating files in the dataDir , it only supports replicating index files and conf files (files in conf dir). I was unaware of the fact that it was possible to put the elevate.xml in dataDir. reloading on commit is a trivial for a search component. it can register itself to be an even listener for commit and do a reload of elevate.xml. This can be a configuration parameter. str name=refreshOnCommmittrue/str On Wed, May 6, 2009 at 7:08 PM, Nicolas Pastorino n...@ez.no wrote: On May 6, 2009, at 15:17 , Noble Paul നോബിള് नोब्ळ् wrote: Why would you want to write it to the data dir? why can't it be in the same place (conf) ? Well, fact is that the QueryElevationComponent loads the configuration file ( elevate.xml ) either from the data dir, either from the conf dir. Which means that existing setups using this component maybe using either location. That is the only reason why i judged necessary to keep supporting this flexibility. But this could be simplified, forcing the elevate.xml file to be in the conf dir, and having a system ( the one you proposed, or the request handler attached to the issue ) to reload the configuration from the conf dir ( which is currently not possible. While when elevate.xml is stored in the dataDir, triggering a commit would reload it ). I was just unsure about all ins and outs of the Elevation system, and then did not remove this flexibility. Thanks for your expert eye on this ! On Wed, May 6, 2009 at 6:43 PM, Nicolas Pastorino n...@ez.no wrote: Hello, On May 6, 2009, at 15:02 , Noble Paul നോബിള് नोब्ळ् wrote: The elevate.xml is loaded from conf dir when the core is reloaded . if you post the new xml you will have to reload the core. A simple solution would be to write a RequestHandler which extends QueryElevationComponent which can be a listener for commit and call an super.inform() on that event You may want to have a look at this issue : https://issues.apache.org/jira/browse/SOLR-1147 The proposed solution ( new request handler, attached to the ticket ), solves the issue in both cases : * when elevate.xml is in the DataDir. * when elevate.xml is in the conf dir. Basically this new request handler receives, as XML, the new configuration, writes it to the right place ( some logic was copied from the QueryElevationComponent.inform() code ), and then calls the inform() method on the QueryElevationComponent for the current core, as you suggested above, to reload the Elevate configuration. -- Nicolas On Fri, Apr 10, 2009 at 5:18 PM, Nicolas Pastorino n...@ez.no wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no -- - Noble Paul | Principal Engineer| AOL | http://aol.com -- Nicolas Pastorino Consultant - Trainer - System Developer Phone : +33 (0)4.78.37.01.34 eZ Systems ( Western Europe ) | http://ez.no -- - Noble Paul | Principal Engineer| AOL |
QueryElevationComponent : hot update of elevate.xml
Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Thanks a lot in advance for your thoughts, -- Nicolas
Re: QueryElevationComponent : hot update of elevate.xml
On Apr 10, 2009, at 7:48 AM, Nicolas Pastorino wrote: Hello ! Browsing the mailing-list's archives did not help me find the answer, hence the question asked directly here. Some context first : Integrating Solr with a CMS ( eZ Publish ), we chose to support Elevation. The idea is to be able to 'elevate' any object from the CMS. This can be achieved through eZ Publish's back office, with a dedicated Elevate administration GUI, the configuration is stored in the CMS temporarily, and then synchronized frequently and/or on demand onto Solr. This synchronisation is currently done as follows : 1. Generate the elevate.xml based on the stored configuration 2. Replace elevate.xml in Solr's dataDir 3. Commit. It appears that when having elevate.xml in Solr's dataDir, and solely in this case, commiting triggers a reload of elevate.xml. This does not happen when elevate.xml is stored in Solr's conf dir. This method has one main issue though : eZ Publish needs to have access to the same filesystem as the one on which Solr's dataDir is stored. This is not always the case when the CMS is clustered for instance -- show stopper :( Hence the following idea / RFC : How about extending the Query Elevation system with the possibility to push an updated elevate.xml file/XML through HTTP ? This would update the file where it is actually located, and trigger a reload of the configuration. Not being very knowledgeable about Solr's API ( yet ! ), i cannot figure out whether this would be possible, how this would be achievable ( which type of plugin for instance ) or even be valid ? Perhaps look at implementing custom RequestHandler: http://wiki.apache.org/solr/SolrRequestHandler maybe it could POST the new elevate.xm and then save it to the right place and call commit... ryan
