[jira] [Commented] (SOLR-4978) Time is stripped from datetime column when imported into Solr date field
[ https://issues.apache.org/jira/browse/SOLR-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701129#comment-13701129 ] Bill Au commented on SOLR-4978: --- This will only have an effect if convertType is enabled and will only affect date field type. convertType is disabled by default. I guess most people use the default setting, which is probably why no one has noticed this bug before. The current behavior for date field type is incorrect when converType is enabled. Making the change will fix the bug. So date fields indexed by DIH when convertType is enabled will actually have a time portion. Time is stripped from datetime column when imported into Solr date field Key: SOLR-4978 URL: https://issues.apache.org/jira/browse/SOLR-4978 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Reporter: Bill Au I discovered that all dates I imported into a Solr date field from a MySQL datetime column have the time stripped (ie time portion is always 00:00:00). After double checking my DIH config and trying different things, I decided to take a look at the DIH code. When I looked at the source code of DIH JdbcDataSource class, I discovered that it is using java.sql.ResultSet and its getDate() method to handle date field. The getDate() method returns java.sql.Date. The java api doc for java.sql.Date http://docs.oracle.com/javase/6/docs/api/java/sql/Date.html states that: To conform with the definition of SQL DATE, the millisecond values wrapped by a java.sql.Date instance must be 'normalized' by setting the hours, minutes, seconds, and milliseconds to zero in the particular time zone with which the instance is associated. I am so surprise at my finding that I think I may not be right. What am I doing wrong here? This is such a big hole in DIH, how could it be possible that no one has noticed this until now? Has anyone successfully imported a datetime column into a Solr date field using DIH? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-4978) Time is stripped from datetime column when imported into Solr date field
[ https://issues.apache.org/jira/browse/SOLR-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au closed SOLR-4978. - Resolution: Invalid we have convertType=true in our dih config. Time is stripped from datetime column when imported into Solr date field Key: SOLR-4978 URL: https://issues.apache.org/jira/browse/SOLR-4978 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Reporter: Bill Au I discovered that all dates I imported into a Solr date field from a MySQL datetime column have the time stripped (ie time portion is always 00:00:00). After double checking my DIH config and trying different things, I decided to take a look at the DIH code. When I looked at the source code of DIH JdbcDataSource class, I discovered that it is using java.sql.ResultSet and its getDate() method to handle date field. The getDate() method returns java.sql.Date. The java api doc for java.sql.Date http://docs.oracle.com/javase/6/docs/api/java/sql/Date.html states that: To conform with the definition of SQL DATE, the millisecond values wrapped by a java.sql.Date instance must be 'normalized' by setting the hours, minutes, seconds, and milliseconds to zero in the particular time zone with which the instance is associated. I am so surprise at my finding that I think I may not be right. What am I doing wrong here? This is such a big hole in DIH, how could it be possible that no one has noticed this until now? Has anyone successfully imported a datetime column into a Solr date field using DIH? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-4978) Time is stripped from datetime column when imported into Solr date field
[ https://issues.apache.org/jira/browse/SOLR-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reopened SOLR-4978: --- setting convertType=false is a workaround for my problem. But the bug still exists when convertType is set to true. DIH should not call ResultSet.getDate() for solr date field because ResultSet.getDate() returns a java.sql.Date which has not time portion. Since sold date field does have a time portion, DIH should call ResultSet.getTimestamp(). Time is stripped from datetime column when imported into Solr date field Key: SOLR-4978 URL: https://issues.apache.org/jira/browse/SOLR-4978 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Reporter: Bill Au I discovered that all dates I imported into a Solr date field from a MySQL datetime column have the time stripped (ie time portion is always 00:00:00). After double checking my DIH config and trying different things, I decided to take a look at the DIH code. When I looked at the source code of DIH JdbcDataSource class, I discovered that it is using java.sql.ResultSet and its getDate() method to handle date field. The getDate() method returns java.sql.Date. The java api doc for java.sql.Date http://docs.oracle.com/javase/6/docs/api/java/sql/Date.html states that: To conform with the definition of SQL DATE, the millisecond values wrapped by a java.sql.Date instance must be 'normalized' by setting the hours, minutes, seconds, and milliseconds to zero in the particular time zone with which the instance is associated. I am so surprise at my finding that I think I may not be right. What am I doing wrong here? This is such a big hole in DIH, how could it be possible that no one has noticed this until now? Has anyone successfully imported a datetime column into a Solr date field using DIH? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4978) Time is stripped from datetime column when imported into Solr date field
Bill Au created SOLR-4978: - Summary: Time is stripped from datetime column when imported into Solr date field Key: SOLR-4978 URL: https://issues.apache.org/jira/browse/SOLR-4978 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Reporter: Bill Au I discovered that all dates I imported into a Solr date field from a MySQL datetime column have the time stripped (ie time portion is always 00:00:00). After double checking my DIH config and trying different things, I decided to take a look at the DIH code. When I looked at the source code of DIH JdbcDataSource class, I discovered that it is using java.sql.ResultSet and its getDate() method to handle date field. The getDate() method returns java.sql.Date. The java api doc for java.sql.Date http://docs.oracle.com/javase/6/docs/api/java/sql/Date.html states that: To conform with the definition of SQL DATE, the millisecond values wrapped by a java.sql.Date instance must be 'normalized' by setting the hours, minutes, seconds, and milliseconds to zero in the particular time zone with which the instance is associated. I am so surprise at my finding that I think I may not be right. What am I doing wrong here? This is such a big hole in DIH, how could it be possible that no one has noticed this until now? Has anyone successfully imported a datetime column into a Solr date field using DIH? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-3986) index version and generation not changed in admin UI after delete by query on master
Bill Au created SOLR-3986: - Summary: index version and generation not changed in admin UI after delete by query on master Key: SOLR-3986 URL: https://issues.apache.org/jira/browse/SOLR-3986 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 4.0 Reporter: Bill Au Priority: Minor Here are the steps to reproduce this: - follow steps in Solr 4.0 tutorial to set up a master and a slave to use Java/HTTP replication - index example documents on master: java -jar post.jar *.xml - make a note of the index version and generation the on both the replication section of the summary screen of core collection1 and the replication screen on both the master and slave - run a delete by query on the master java -Ddata=args -jar post.jar deletequeryname:DDR/query/delete - on master reload the summary screen for core collection1. The Num Docs field decreased but the index version and generation are unchanged in the replication section. The index version and generation are also unchanged in the replication screen. - on the slave, wait for replication to kick in or trigger it manually. On the summary screen for core collection1, the Num DOcs field decreased to match what's on the master. The index version and generation of the master remain unchanged but the index version and generation of the slave both changed. The same goes for the index version and generation of the master and slave on the replication screen. The replication handler on the master does report changed index version and generation: localhost:8983/solr/collection1/replication?command=indexversion It is only the admin UI that reporting the older index version and generation on both the core summary screen and replication screen. This only happens with delete by query. There is no problem with delete with id or add. Both the index version and generation do get updated on subsequent delete by query but both remain one cycle behind on the master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3986) index version and generation not changed in admin UI after delete by query on master
[ https://issues.apache.org/jira/browse/SOLR-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483273#comment-13483273 ] Bill Au commented on SOLR-3986: --- By the way, the request URL for the summary screen for core collection1: localhost:8983/solr/#/collection1 the request URL for the replication screen: localhost:8983/solr/#/collection1/replication index version and generation not changed in admin UI after delete by query on master Key: SOLR-3986 URL: https://issues.apache.org/jira/browse/SOLR-3986 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 4.0 Reporter: Bill Au Priority: Minor Here are the steps to reproduce this: - follow steps in Solr 4.0 tutorial to set up a master and a slave to use Java/HTTP replication - index example documents on master: java -jar post.jar *.xml - make a note of the index version and generation the on both the replication section of the summary screen of core collection1 and the replication screen on both the master and slave - run a delete by query on the master java -Ddata=args -jar post.jar deletequeryname:DDR/query/delete - on master reload the summary screen for core collection1. The Num Docs field decreased but the index version and generation are unchanged in the replication section. The index version and generation are also unchanged in the replication screen. - on the slave, wait for replication to kick in or trigger it manually. On the summary screen for core collection1, the Num DOcs field decreased to match what's on the master. The index version and generation of the master remain unchanged but the index version and generation of the slave both changed. The same goes for the index version and generation of the master and slave on the replication screen. The replication handler on the master does report changed index version and generation: localhost:8983/solr/collection1/replication?command=indexversion It is only the admin UI that reporting the older index version and generation on both the core summary screen and replication screen. This only happens with delete by query. There is no problem with delete with id or add. Both the index version and generation do get updated on subsequent delete by query but both remain one cycle behind on the master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Updated: (SOLR-1734) Add pid file to snappuller to skip script overruns, and recover from failure
[ https://issues.apache.org/jira/browse/SOLR-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1734: -- Attachment: SOLR-1734-2.patch The original patch has a slight race condition within the time window between testing for the presence and creation of the lock file. Since snappuller is a bash script, the revised patch uses the noclobber option to prevent that. Add pid file to snappuller to skip script overruns, and recover from failure Key: SOLR-1734 URL: https://issues.apache.org/jira/browse/SOLR-1734 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: SOLR-1734-2.patch, SOLR-1734.patch The pid file will allow snappuller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snappuller process no longer be running. The same has already been done to snapinstaller in SOLR-990. Overlapping snappuller could cause replication traffic to saturate the network if a large Solr index is being replicated to a large number of clients. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1734) Add pid file to snappuller to skip script overruns, and recover from failure
[ https://issues.apache.org/jira/browse/SOLR-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1734: -- Attachment: (was: SOLR-1734-2.patch) Add pid file to snappuller to skip script overruns, and recover from failure Key: SOLR-1734 URL: https://issues.apache.org/jira/browse/SOLR-1734 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: SOLR-1734-2.patch, SOLR-1734.patch The pid file will allow snappuller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snappuller process no longer be running. The same has already been done to snapinstaller in SOLR-990. Overlapping snappuller could cause replication traffic to saturate the network if a large Solr index is being replicated to a large number of clients. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1734) Add pid file to snappuller to skip script overruns, and recover from failure
[ https://issues.apache.org/jira/browse/SOLR-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1734: -- Attachment: SOLR-1734-2.patch Add pid file to snappuller to skip script overruns, and recover from failure Key: SOLR-1734 URL: https://issues.apache.org/jira/browse/SOLR-1734 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: SOLR-1734-2.patch, SOLR-1734.patch The pid file will allow snappuller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snappuller process no longer be running. The same has already been done to snapinstaller in SOLR-990. Overlapping snappuller could cause replication traffic to saturate the network if a large Solr index is being replicated to a large number of clients. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1755) race condition in snappuller
race condition in snappuller Key: SOLR-1755 URL: https://issues.apache.org/jira/browse/SOLR-1755 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor There is a slight race condition in snapinstaller within the time window between testing for the presence and creating the lock file. When the race condition is met there can be more than one instance of snapinstaller running. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1755) race condition in snappuller
[ https://issues.apache.org/jira/browse/SOLR-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1755: -- Attachment: SOLR-1755.patch Since snapinstaller is a bash script. The attached patch uses the noclobber option to prevent the race condition. race condition in snappuller Key: SOLR-1755 URL: https://issues.apache.org/jira/browse/SOLR-1755 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: SOLR-1755.patch There is a slight race condition in snapinstaller within the time window between testing for the presence and creating the lock file. When the race condition is met there can be more than one instance of snapinstaller running. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1734) Add pid file to snappuller to skip script overruns, and recover from failure
Add pid file to snappuller to skip script overruns, and recover from failure Key: SOLR-1734 URL: https://issues.apache.org/jira/browse/SOLR-1734 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor The pid file will allow snappuller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snappuller process no longer be running. The same has already been done to snapinstaller in SOLR-990. Overlapping snappuller could cause replication traffic to saturate the network if a large Solr index is being replicated to a large number of clients. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1734) Add pid file to snappuller to skip script overruns, and recover from failure
[ https://issues.apache.org/jira/browse/SOLR-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1734: -- Attachment: SOLR-1734.patch I am reusing the code from SOLR-990 which adds the same feature to snapinstaller. I have added a -f command line argument to force the snappuller to run even if one is already running. That will be useful if network capacity is not an issue. Add pid file to snappuller to skip script overruns, and recover from failure Key: SOLR-1734 URL: https://issues.apache.org/jira/browse/SOLR-1734 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: SOLR-1734.patch The pid file will allow snappuller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snappuller process no longer be running. The same has already been done to snapinstaller in SOLR-990. Overlapping snappuller could cause replication traffic to saturate the network if a large Solr index is being replicated to a large number of clients. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
[ https://issues.apache.org/jira/browse/SOLR-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-1594: -- Attachment: solr-1594.patch SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
[ https://issues.apache.org/jira/browse/SOLR-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12781496#action_12781496 ] Bill Au commented on SOLR-1594: --- Try running this query: solr/select/?q=title:scriptalert(xss)/script SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
[ https://issues.apache.org/jira/browse/SOLR-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12781507#action_12781507 ] Bill Au commented on SOLR-1594: --- Jetty is sanitizing both the HTTP response line and the response body so it is OK. I know of at least one appserver that is not doing that. SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
[ https://issues.apache.org/jira/browse/SOLR-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12781522#action_12781522 ] Bill Au commented on SOLR-1594: --- I just tried it and Jetty does double-escape: org.apache.lucene.queryParser.ParseException: Cannot parse 'amp;lt;scriptamp;gt;alert(xss)amp;lt;/scriptamp;gt;': Lexical error at line 1, column 31. Encountered: amp;lt;EOFamp;gt; after : \)amp;lt;/scriptamp;gt; So should we leave it up to the appserver to do the right thing or should Solr be more proactive? To me double-escaping is a lesser evil than being vulnerable to xss attack. SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1594) SolrDispatchFilter needs to sanitize exception message
[ https://issues.apache.org/jira/browse/SOLR-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-1594. --- Resolution: Invalid This same problem had been reported against and fixed in Tomcat. I have reported it to the vendor of my appserver and they are working on a fix. Marking as invalid. SolrDispatchFilter needs to sanitize exception message -- Key: SOLR-1594 URL: https://issues.apache.org/jira/browse/SOLR-1594 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Bill Au Assignee: Bill Au Fix For: 1.5 Attachments: solr-1594.patch SolrDispatchFIlter needs to sanitize exception messages before using them in the response. I will attach a patch shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1545) add support for sort to MoreLikeThis
[ https://issues.apache.org/jira/browse/SOLR-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12779652#action_12779652 ] Bill Au commented on SOLR-1545: --- Has anyone had a change to review/test this patch? I have been testing it with my app and have not found any problem. add support for sort to MoreLikeThis Key: SOLR-1545 URL: https://issues.apache.org/jira/browse/SOLR-1545 Project: Solr Issue Type: Improvement Components: search Affects Versions: 1.4 Reporter: Bill Au Priority: Minor Fix For: 1.5 Attachments: solr-1545.patch Add support for sort to MoreLikeThis. I will attach a patch with more info shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1294) SolrJS/Javascript client fails in IE8!
[ https://issues.apache.org/jira/browse/SOLR-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12762604#action_12762604 ] Bill Au commented on SOLR-1294: --- jscalendar.js is LGPL so I am not sure if it can be included into Solr. The latest patch does not apply cleanly to the trunk. After I manually fix things. The reuters example now works for IE8. But I am still getting one error for IE7: Line: 61 Error: Expected Identifier, string or number But if I ignore that error, the auto suggestion and searches do seem to work correctly. It is an improvement since those were not working before. SolrJS/Javascript client fails in IE8! -- Key: SOLR-1294 URL: https://issues.apache.org/jira/browse/SOLR-1294 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Eric Pugh Assignee: Ryan McKinley Fix For: 1.4 Attachments: jscalendar.tar, SOLR-1294-full.patch, SOLR-1294-IE8.patch, SOLR-1294.patch, solrjs-ie8-html-syntax-error.patch SolrJS seems to fail with 'jQuery.solrjs' is null or not an object errors under IE8. I am continuing to test if this occurs in IE 6 and 7 as well. This happens on both the Sample online site at http://solrjs.solrstuff.org/test/reuters/ as well as the /trunk/contrib/javascript library. Seems to be a show stopper from the standpoint of really using this library! I have posted a screenshot of the error at http://img.skitch.com/20090717-jejm71u6ghf2dpn3mwrkarigwm.png The error is just a whole bunch of repeated messages in the vein of: Message: 'jQuery.solrjs' is null or not an object Line: 24 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/QueryItem.js Message: 'jQuery.solrjs' is null or not an object Line: 37 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/Manager.js Message: 'jQuery.solrjs' is null or not an object Line: 24 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/AbstractSelectionView.js Message: 'jQuery.solrjs' is null or not an object Line: 27 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/AbstractWidget.js -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1485) PayloadTermQuery support
[ https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761931#action_12761931 ] Bill Au commented on SOLR-1485: --- I am +0 on including/excluding this from 1.4. FYI, Solr 1.4 already has a DelimitedPayloadTokenFilterFactory which uses the DelimitedPayloadTokenFIlter in Lucene. If we include this, I think we should also include a Similarity class for payload, either as part of this JIRA or a separate one. There is also a similar JIRA on query support: https://issues.apache.org/jira/browse/SOLR-1337 PayloadTermQuery support Key: SOLR-1485 URL: https://issues.apache.org/jira/browse/SOLR-1485 Project: Solr Issue Type: New Feature Components: search Reporter: Erik Hatcher Attachments: PayloadTermQueryPlugin.java Solr currently has no support for Lucene's PayloadTermQuery, yet it has support for indexing payloads. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1481) phps writer ignores omitHeader parameter
[ https://issues.apache.org/jira/browse/SOLR-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reassigned SOLR-1481: - Assignee: Bill Au phps writer ignores omitHeader parameter Key: SOLR-1481 URL: https://issues.apache.org/jira/browse/SOLR-1481 Project: Solr Issue Type: Bug Components: search Reporter: Koji Sekiguchi Assignee: Bill Au Priority: Trivial Fix For: 1.4 Attachments: SOLR-1481.patch My co-worker found this one. I'm expecting a patch will be attached soon by him. :) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1481) phps writer ignores omitHeader parameter
[ https://issues.apache.org/jira/browse/SOLR-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761592#action_12761592 ] Bill Au commented on SOLR-1481: --- I can take this one. phps writer ignores omitHeader parameter Key: SOLR-1481 URL: https://issues.apache.org/jira/browse/SOLR-1481 Project: Solr Issue Type: Bug Components: search Reporter: Koji Sekiguchi Assignee: Bill Au Priority: Trivial Fix For: 1.4 Attachments: SOLR-1481.patch My co-worker found this one. I'm expecting a patch will be attached soon by him. :) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1481) phps writer ignores omitHeader parameter
[ https://issues.apache.org/jira/browse/SOLR-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-1481. --- Resolution: Fixed The patch looks good. I have committed it: SendingCHANGES.txt Sendingsrc/java/org/apache/solr/request/PHPSerializedResponseWriter.java Transmitting file data .. Committed revision 821076. Thanks, Jun. phps writer ignores omitHeader parameter Key: SOLR-1481 URL: https://issues.apache.org/jira/browse/SOLR-1481 Project: Solr Issue Type: Bug Components: search Reporter: Koji Sekiguchi Assignee: Bill Au Priority: Trivial Fix For: 1.4 Attachments: SOLR-1481.patch My co-worker found this one. I'm expecting a patch will be attached soon by him. :) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1482) Solr master and slave freeze after query
[
https://issues.apache.org/jira/browse/SOLR-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761654#action_12761654
]
Bill Au commented on SOLR-1482:
---
You probably want to take a JVM thread dump (kill -3) while the JVM is hung to
find out what's going on.
Is your webapp app being reloaded? You can check the appserver log file to see
if that's happening. One common way of running out of PermGen space is a
classloader link which occurs when a webapp is reloaded.
Solr master and slave freeze after query
Key: SOLR-1482
URL: https://issues.apache.org/jira/browse/SOLR-1482
Project: Solr
Issue Type: Bug
Affects Versions: 1.4
Environment: Nightly 9/28/09.
14 individual instances per server, using JNDI.
replicateAfter commit, 5 min interval polling.
All caches are currently commented out, on both slave and master.
Lots of ongoing commits - large chunks of data, each accompanied by a commit.
This is to guarantee that anything we think is now in Solr remains there in
case the server crashes.
Reporter: Artem Russakovskii
Priority: Critical
We're having issues with the deployment of 2 master-slave setups.
One of the master-slave setups is OK (so far) but on the other both the
master and the slave keep freezing, but only after I send a query to them.
And by freezing I mean indefinite hanging, with almost no output to log, no
errors, nothing. It's as if there's some sort of a deadlock. The hanging
servers need to be killed with -9, otherwise they keep hanging.
The query I send queries all instances at the same time using the ?shards=
syntax.
On the slave, the logs just stop - nothing shows up anymore after the query
is issued. On the master, they're a bit more descriptive. This information
seeps through very-very slowly, as you can see from the timestamps:
{quote}
SEVERE: java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:16:00 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:19:37 PM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request
processing
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:19:37 PM org.apache.coyote.http11.Http11Processor process
SEVERE: Error processing request
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:19:39 PM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request
processing
java.lang.OutOfMemoryError: PermGen space
Exception in thread ContainerBackException in thread pool-29-threadOct 1,
2009 2:21:47 PM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request
processing
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
-22 java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
Exception in thread http-8080-42 Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
Oct 1, 2009 2:21:47 PM
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler process
SEVERE: Error reading request, ignored
java.lang.OutOfMemoryError: PermGen space
Exception in thread http-8080-26 Exception in thread http-8080-32
Exception in thread http-8080-25 Exception in thread http-8080-22
Exception in thread http-8080-15 Exception in thread http-8080-45
Exception in thread http-8080-13 Exception in thread http-8080-48
Exception in thread http-8080-7 Exception in thread http-8080-38
Exception in thread http-8080-39 Exception in thread http-8080-28
Exception in thread http-8080-1 Exception in thread http-8080-2 Exception
in thread http-8080-12 Exception in thread http-8080-44 Exception in
thread http-8080-47 Exception in thread http-8080-29 Exception in thread
http-8080-33 Exception in thread http-8080-27 Exception in thread
http-8080-36 Exception in thread http-8080-113 Exception in thread
http-8080-112
[jira] Commented: (SOLR-1485) PayloadTermQuery support
[ https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761687#action_12761687 ] Bill Au commented on SOLR-1485: --- Eric, have you started on this? I recently wrote a QParserPlugin that supports PayloadTermQuery. It is very bear-bone but could be a good starting point. I can attach my code here to get things started. PayloadTermQuery support Key: SOLR-1485 URL: https://issues.apache.org/jira/browse/SOLR-1485 Project: Solr Issue Type: New Feature Components: search Reporter: Erik Hatcher Fix For: 1.4 Attachments: PayloadTermQueryPlugin.java Solr currently has no support for Lucene's PayloadTermQuery, yet it has support for indexing payloads. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1485) PayloadTermQuery support
[ https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761688#action_12761688 ] Bill Au commented on SOLR-1485: --- Never mind. I just saw you update. Your code looks good. PayloadTermQuery support Key: SOLR-1485 URL: https://issues.apache.org/jira/browse/SOLR-1485 Project: Solr Issue Type: New Feature Components: search Reporter: Erik Hatcher Fix For: 1.4 Attachments: PayloadTermQueryPlugin.java Solr currently has no support for Lucene's PayloadTermQuery, yet it has support for indexing payloads. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1485) PayloadTermQuery support
[ https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12761696#action_12761696 ] Bill Au commented on SOLR-1485: --- Eric, do you think we should support default field and default operator in the QParser used? PayloadTermQuery support Key: SOLR-1485 URL: https://issues.apache.org/jira/browse/SOLR-1485 Project: Solr Issue Type: New Feature Components: search Reporter: Erik Hatcher Fix For: 1.4 Attachments: PayloadTermQueryPlugin.java Solr currently has no support for Lucene's PayloadTermQuery, yet it has support for indexing payloads. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1294) SolrJS/Javascript client fails in IE8!
[ https://issues.apache.org/jira/browse/SOLR-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12760637#action_12760637 ] Bill Au commented on SOLR-1294: --- I just tried the patch from 8/14 with the reuters example in trunk but it does not work for IE8. Eric, http://www.newswise.com/search does work for both IE7 and IE8. So you think you can come up with a patch this week? If not I think we should postpone this bug to 1.5. SolrJS/Javascript client fails in IE8! -- Key: SOLR-1294 URL: https://issues.apache.org/jira/browse/SOLR-1294 Project: Solr Issue Type: Bug Affects Versions: 1.4 Reporter: Eric Pugh Assignee: Ryan McKinley Fix For: 1.4 Attachments: SOLR-1294-IE8.patch, SOLR-1294.patch, solrjs-ie8-html-syntax-error.patch SolrJS seems to fail with 'jQuery.solrjs' is null or not an object errors under IE8. I am continuing to test if this occurs in IE 6 and 7 as well. This happens on both the Sample online site at http://solrjs.solrstuff.org/test/reuters/ as well as the /trunk/contrib/javascript library. Seems to be a show stopper from the standpoint of really using this library! I have posted a screenshot of the error at http://img.skitch.com/20090717-jejm71u6ghf2dpn3mwrkarigwm.png The error is just a whole bunch of repeated messages in the vein of: Message: 'jQuery.solrjs' is null or not an object Line: 24 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/QueryItem.js Message: 'jQuery.solrjs' is null or not an object Line: 37 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/Manager.js Message: 'jQuery.solrjs' is null or not an object Line: 24 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/AbstractSelectionView.js Message: 'jQuery.solrjs' is null or not an object Line: 27 Char: 1 Code: 0 URI: file:///C:/dev/projects/lib/solr/contrib/javascript/src/core/AbstractWidget.js -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1203) We should add an example of setting the update.processor for a given RequestHandler
[ https://issues.apache.org/jira/browse/SOLR-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12749189#action_12749189 ] Bill Au commented on SOLR-1203: --- I had actually come across an example in the Wiki when I was looking for information on update processor: http://wiki.apache.org/solr/Deduplication?highlight=%28updateprocessor%29#head-177c1b1e490e1192f41d9ab0e037b05e1567a35d We should add an example of setting the update.processor for a given RequestHandler --- Key: SOLR-1203 URL: https://issues.apache.org/jira/browse/SOLR-1203 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 1.4 Attachments: SOLR-1203.patch a commented out example that points to the commented out example update chain or just as good: a comment above the current update chain example explaining how to attach it to a handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-346: - Attachment: solr-346-2.patch attaching new patch to fix snapinstaller need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.2, 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.4 Attachments: solr-346-2.patch, solr-346.patch http://www.mail-archive.com/solr-u...@lucene.apache.org/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-346. -- Resolution: Fixed I went ahead and committed the code change: Sendingsnapinstaller Transmitting file data . Committed revision 750048. need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.2, 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.4 Attachments: solr-346-2.patch, solr-346.patch http://www.mail-archive.com/solr-u...@lucene.apache.org/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-990) Add pid file to snapinstaller to skip script overruns, and recover from faliure
[ https://issues.apache.org/jira/browse/SOLR-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12667726#action_12667726 ] Bill Au commented on SOLR-990: -- The pid file should be move under the logs directory. I don't think it is a good idea to keep the pid file under /tmp. Add pid file to snapinstaller to skip script overruns, and recover from faliure --- Key: SOLR-990 URL: https://issues.apache.org/jira/browse/SOLR-990 Project: Solr Issue Type: Improvement Components: replication (scripts) Reporter: Dan Rosher Priority: Minor Attachments: SOLR-990.patch, SOLR-990.patch, SOLR-990.patch The pid file will allow snapinstaller to be run as fast as possible without overruns. Also it will recover from a last failed run should an older snapinstaller process no longer be running. Avoiding overruns means that snapinstaller can be run as fast as possible, but without suffering from the performance issue described here: http://wiki.apache.org/solr/SolrPerformanceFactors#head-fc7f22035c493431d58c5404ab22aef0ee1b9909 This means that one can do the following */1 * * * * /bin/snappuller/bin/snapinstaller Even with a 'properly tuned' setup, there can be times where snapinstaller can suffer from overruns due to a lack of resources, or an unoptimized index using more resources etc. currently the pid will live in /tmp ... perhaps it should be in the logs dir? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au closed SOLR-897.
Resolution: Fixed
Patch commited:
SendingCHANGES.txt
Sendingsrc/scripts/abc
Sendingsrc/scripts/abo
Sendingsrc/scripts/backupcleaner
Sendingsrc/scripts/snapcleaner
Transmitting file data .
Committed revision 727722.
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: solr-897-2.patch, SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Summary: abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long (was: snapcleaner, if removinmg
by number of snapshots, gets Argument list too long)
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-897) snapcleaner, if removinmg by number of snapshots, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12657071#action_12657071
]
Bill Au commented on SOLR-897:
--
The following scripts also have the same problem: abc, abo, backupcleaner
snapcleaner, if removinmg by number of snapshots, gets Argument list too long
-
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Attachment: solr-897.patch2
expanded patch to include fix for all four scripts with the same problem.
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Attachment: (was: solr-897.patch2)
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Attachment: solr-897-2.patch
reattach expanded patch for fixes to all four scritps with the same problem.
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Attachment: (was: solr-897-2.patch)
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-897) abo/abc/backupcleaner/snapcleaner, if removinmg by number of snapshots/backups, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-897:
-
Attachment: solr-897-2.patch
reattach expanded patch for all four scripts with the same problem. This time
with correct license.
abo/abc/backupcleaner/snapcleaner, if removinmg by number of
snapshots/backups, gets Argument list too long
---
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: solr-897-2.patch, SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-897) snapcleaner, if removinmg by number of snapshots, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12656542#action_12656542
]
Bill Au commented on SOLR-897:
--
I am going to see of other replication related scripts have this problem and
fix them once and for all.
snapcleaner, if removinmg by number of snapshots, gets Argument list too long
-
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-897) snapcleaner, if removinmg by number of snapshots, gets Argument list too long
[
https://issues.apache.org/jira/browse/SOLR-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au reassigned SOLR-897:
Assignee: Bill Au
snapcleaner, if removinmg by number of snapshots, gets Argument list too long
-
Key: SOLR-897
URL: https://issues.apache.org/jira/browse/SOLR-897
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.3
Reporter: Dan Rosher
Assignee: Bill Au
Fix For: 1.4
Attachments: SOLR-897.patch
ls -cd ${data_dir}/snapshot.* returns Argument list too long, use find instead
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-346. -- Resolution: Fixed Patch committed: SendingCHANGES.txt Sendingsrc/scripts/snapinstaller Transmitting file data .. Committed revision 719232. need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.2, 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.3.1 Attachments: solr-346.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au resolved SOLR-830.
--
Resolution: Fixed
Patch committed:
SendingCHANGES.txt
Sendingsrc/scripts/snappuller
Transmitting file data ..
Committed revision 719233.
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
Fix For: 1.3.1
Attachments: solr-830.patch
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12648260#action_12648260
]
Bill Au commented on SOLR-830:
--
It works on Mac OS X.
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
Fix For: 1.3.1
Attachments: solr-830.patch
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-830:
-
Attachment: solr-830.patch
patch to use perl regular expression to improve accuracy in finding latest
snapshot.
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
Fix For: 1.3.1
Attachments: solr-830.patch
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-346: - Attachment: solr-346.patch patch to use perl regular expression to improve accuracy in finding latest snapshot. need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.2, 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.3.1 Attachments: solr-346.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-857) Memory Leak during the indexing of large xml files
[ https://issues.apache.org/jira/browse/SOLR-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12648070#action_12648070 ] Bill Au commented on SOLR-857: -- HashMap$Entry is on top of your list there. I would look for a large HashMap in the heap dump. Memory Leak during the indexing of large xml files -- Key: SOLR-857 URL: https://issues.apache.org/jira/browse/SOLR-857 Project: Solr Issue Type: Bug Affects Versions: 1.3 Environment: Verified on Ubuntu 8.0.4 (1.7GB RAM, 2.4GHz dual core) and Windows XP (2GB RAM, 2GHz pentium) both with a Java5 SDK Reporter: Ruben Jimenez Attachments: OQ_SOLR_1.xml.zip, schema.xml, solr256MBHeap.jpg While indexing a set of SOLR xml files that contain 5000 document adds within them and are about 30MB each, SOLR 1.3 seems to continually use more and more memory until the heap is exhausted, while the same files are indexed without issue with SOLR 1.2. Steps used to reproduce. 1 - Download SOLR 1.3 2 - Modify example schema.xml to match fields required 3 - start example server with following command java -Xms512m -Xmx1024m -XX:MaxPermSize=128m -jar start.jar 4 - Index files as follow java -Xmx128m -jar .../examples/exampledocs/post.jar *.xml Directory with xml files contains about 100 xml files each of about 30MB each. While indexing after about the 25th file SOLR 1.3 runs out of memory, while SOLR 1.2 is able to index the entire set of files without any problems. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-857) Memory Leak during the indexing of large xml files
[ https://issues.apache.org/jira/browse/SOLR-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12647684#action_12647684 ] Bill Au commented on SOLR-857: -- Have you try starting your JVM with -XX:-HeapDumpOnOutOfMemoryError and then looking at the heap dump? Memory Leak during the indexing of large xml files -- Key: SOLR-857 URL: https://issues.apache.org/jira/browse/SOLR-857 Project: Solr Issue Type: Bug Affects Versions: 1.3 Environment: Verified on Ubuntu 8.0.4 (1.7GB RAM, 2.4GHz dual core) and Windows XP (2GB RAM, 2GHz pentium) both with a Java5 SDK Reporter: Ruben Jimenez Attachments: OQ_SOLR_1.xml.zip, schema.xml While indexing a set of SOLR xml files that contain 5000 document adds within them and are about 30MB each, SOLR 1.3 seems to continually use more and more memory until the heap is exhausted, while the same files are indexed without issue with SOLR 1.2. Steps used to reproduce. 1 - Download SOLR 1.3 2 - Modify example schema.xml to match fields required 3 - start example server with following command java -Xms512m -Xmx1024m -XX:MaxPermSize=128m -jar start.jar 4 - Index files as follow java -Xmx128m -jar .../examples/exampledocs/post.jar *.xml Directory with xml files contains about 100 xml files each of about 30MB each. While indexing after about the 25th file SOLR 1.3 runs out of memory, while SOLR 1.2 is able to index the entire set of files without any problems. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-857) Memory Leak during the indexing of large xml files
[ https://issues.apache.org/jira/browse/SOLR-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12647684#action_12647684 ] billa edited comment on SOLR-857 at 11/14/08 10:24 AM: - Have you tried starting your JVM with -XX:-HeapDumpOnOutOfMemoryError and then looking at the heap dump? was (Author: billa): Have you try starting your JVM with -XX:-HeapDumpOnOutOfMemoryError and then looking at the heap dump? Memory Leak during the indexing of large xml files -- Key: SOLR-857 URL: https://issues.apache.org/jira/browse/SOLR-857 Project: Solr Issue Type: Bug Affects Versions: 1.3 Environment: Verified on Ubuntu 8.0.4 (1.7GB RAM, 2.4GHz dual core) and Windows XP (2GB RAM, 2GHz pentium) both with a Java5 SDK Reporter: Ruben Jimenez Attachments: OQ_SOLR_1.xml.zip, schema.xml While indexing a set of SOLR xml files that contain 5000 document adds within them and are about 30MB each, SOLR 1.3 seems to continually use more and more memory until the heap is exhausted, while the same files are indexed without issue with SOLR 1.2. Steps used to reproduce. 1 - Download SOLR 1.3 2 - Modify example schema.xml to match fields required 3 - start example server with following command java -Xms512m -Xmx1024m -XX:MaxPermSize=128m -jar start.jar 4 - Index files as follow java -Xmx128m -jar .../examples/exampledocs/post.jar *.xml Directory with xml files contains about 100 xml files each of about 30MB each. While indexing after about the 25th file SOLR 1.3 runs out of memory, while SOLR 1.2 is able to index the entire set of files without any problems. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12647331#action_12647331
]
Bill Au commented on SOLR-830:
--
Steve, thanks for the perl code. I need to get rid of the \ before the $
in order to get it to work for me:
perl -e 'chdir q/${master_data_dir}/; print ((sort grep
{/^snapshot[.][1-9][0-9]{13}$/} *)[-1])'
I have tested this on Linux and FreeBSD. I will test on Mac OS X tonight. It
will be good if someone can do a quick test on Solaris. You really don't need
a full brown Solr installation to test it. Just create some dummy directory
with various names like:
snapshot.00080527124131
snapshot.20080527124131
snapshot.20080527124131-wip
snapshot.20080527140518
snapshot.20080527140610
snapshot.20081113113700
snapshot.2080527124131
temp-snapshot.20080527124131
and then run the perl command to make sure the right one is returned. With the
data set above, you should get:
snapshot.20081113113700
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
Fix For: 1.3.1
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12647384#action_12647384
]
Bill Au commented on SOLR-830:
--
Steven, thanks for testing on Solaris. It looks like on Linux and FreeBSD that
the '\' in front of '$' escape the special meaning of '$' so it is trying to
match against the literal '$' after all the digits (ie
snapshot.20080527124131$). Unless this does not work on Mac OS X, I will go
with perl without the '\' before the '$'. I will attach a patch here after I
test on my Mac tonight.
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
Fix For: 1.3.1
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12644004#action_12644004
]
Bill Au commented on SOLR-830:
--
I think we should use a regular expression and match against the naming
convention of the snapshot (snapshot.mmddHHMMss). Here is what I propose:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|egrep '^snapshot\.[123456789][0-9]{13}$'|sort -r|head
-1`
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-830) snappuller picks bad snapshot name
[
https://issues.apache.org/jira/browse/SOLR-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au reassigned SOLR-830:
Assignee: Bill Au
snappuller picks bad snapshot name
--
Key: SOLR-830
URL: https://issues.apache.org/jira/browse/SOLR-830
Project: Solr
Issue Type: Bug
Components: replication (scripts)
Affects Versions: 1.2, 1.3
Reporter: Hoss Man
Assignee: Bill Au
as mentioned on the mailing list...
http://www.nabble.com/FileNotFoundException-on-slave-after-replication---script-bug--to20111313.html#a20111313
{noformat}
We're seeing strange behavior on one of our slave nodes after replication.
When the new searcher is created we see FileNotFoundExceptions in the log
and the index is strangely invalid/corrupted.
We may have identified the root cause but wanted to run it by the community.
We figure there is a bug in the snappuller shell script, line 181:
snap_name=`ssh -o StrictHostKeyChecking=no ${master_host} ls
${master_data_dir}|grep 'snapshot\.'|grep -v wip|sort -r|head -1`
This line determines the directory name of the latest snapshot to download
to the slave from the master. Problem with this line is that it grab the
temporary work directory of a snapshot in progress. Those temporary
directories are prefixed with temp and as far as I can tell should never
get pulled from the master so its easy to disambiguate. It seems that this
temp directory, if it exists will be the newest one so if present it will be
the one replicated: FAIL.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12644007#action_12644007 ] Bill Au commented on SOLR-346: -- snappuller has the same problem. See SOLR-830 (https://issues.apache.org/jira/browse/SOLR-830) for details. The proposed solution there should work here also. need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Reporter: Bill Au Assignee: Bill Au Priority: Minor http://www.mail-archive.com/[EMAIL PROTECTED]/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-346) need to improve snapinstaller to ignore non-snapshots in data directory
[ https://issues.apache.org/jira/browse/SOLR-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-346: - Affects Version/s: 1.2 1.3 need to improve snapinstaller to ignore non-snapshots in data directory --- Key: SOLR-346 URL: https://issues.apache.org/jira/browse/SOLR-346 Project: Solr Issue Type: Improvement Components: replication (scripts) Affects Versions: 1.2, 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor http://www.mail-archive.com/[EMAIL PROTECTED]/msg05734.html latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already installed A directory in the Solr data directory is causing snapinstaller to fail. Snapinstaller should be improved to ignore any much non-snapshot as possible. It can use a regular expression to look for snapshot.dd where d is a digit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-624) patch: Don't take snapshot if there are no differences
[ https://issues.apache.org/jira/browse/SOLR-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12612458#action_12612458 ] Bill Au commented on SOLR-624: -- Yes, the '*' will cost problem if there are many snapshots. The snappuller script has a shell command to determine the latest snapshot on the master. We can use the same command in snapshooter. patch: Don't take snapshot if there are no differences -- Key: SOLR-624 URL: https://issues.apache.org/jira/browse/SOLR-624 Project: Solr Issue Type: Improvement Components: replication Affects Versions: 1.3 Reporter: Richard Trey Hyde Fix For: 1.3 Attachments: solr.check.patch This is similar in concept to a change I made several years ago in Solar. Cronned snapshooters can quickly generate a lot of snaps which will then be unnecessarily distributed to the slaves if there hasn't been any changes in that period. Adds a check argument to make sure there where changes to the index before taking the snap. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-590) Limitation in pgrep on Linux platform breaks script-utils fixUser
[ https://issues.apache.org/jira/browse/SOLR-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reassigned SOLR-590: Assignee: Bill Au Limitation in pgrep on Linux platform breaks script-utils fixUser -- Key: SOLR-590 URL: https://issues.apache.org/jira/browse/SOLR-590 Project: Solr Issue Type: Bug Affects Versions: 1.2 Environment: Linux 2.6.18-53.1.14.el5 #1 SMP Wed Mar 5 11:37:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux procps-3.2.7-8.1.el5 Reporter: Hannes Schmidt Assignee: Bill Au Priority: Minor Attachments: fixUser.patch The fixUser function in script-utils uses two methods to determine the username of the parent process (oldwhoami). If the first method fails for certain reasons it will fallback to the second method. For most people the first method will succeed but I know that in my particular installation the first method fails so I need the second method to succeed. Unfortunately, that fallback method doesn't work because it uses pgrep to lookup the current script's name and on my Linux 2.6.18 platform pgrep is limited to 15 characters. The names of many scripts in the SOLR distribution are longer than that, causing pgrep to return nothing and the subsequent ps invocation to fail with an error: ERROR: List of process IDs must follow -p. You can easily reproduce that behaviour with /app/solr/solr/bin/snappuller-enable /dev/null The redirection of stdin from /dev/null causes fixUser to fallback to the second method but there are other, more realistic scenarios in which the fallback happens, like ssh [EMAIL PROTECTED] /app/solr/solr/bin/snappuller-enable The fix is to use the -f option which causes pgrep to compare the full path of the executable. Interestingly, that method is not subject to the 15 character length limit. The limit is not actually enforced by jetty but rather by the procfs file system of the linux kernel. If you look at /proc/*/stat you will notice that the second column is limited to 15 characters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-590) Limitation in pgrep on Linux platform breaks script-utils fixUser
[ https://issues.apache.org/jira/browse/SOLR-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-590. -- Resolution: Fixed Patch commited and CHANGES.txt updated. Thanks Hannes. SendingCHANGES.txt Sendingsrc/scripts/scripts-util Transmitting file data .. Committed revision 663089. Limitation in pgrep on Linux platform breaks script-utils fixUser -- Key: SOLR-590 URL: https://issues.apache.org/jira/browse/SOLR-590 Project: Solr Issue Type: Bug Affects Versions: 1.2 Environment: Linux 2.6.18-53.1.14.el5 #1 SMP Wed Mar 5 11:37:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux procps-3.2.7-8.1.el5 Reporter: Hannes Schmidt Assignee: Bill Au Priority: Minor Attachments: fixUser.patch The fixUser function in script-utils uses two methods to determine the username of the parent process (oldwhoami). If the first method fails for certain reasons it will fallback to the second method. For most people the first method will succeed but I know that in my particular installation the first method fails so I need the second method to succeed. Unfortunately, that fallback method doesn't work because it uses pgrep to lookup the current script's name and on my Linux 2.6.18 platform pgrep is limited to 15 characters. The names of many scripts in the SOLR distribution are longer than that, causing pgrep to return nothing and the subsequent ps invocation to fail with an error: ERROR: List of process IDs must follow -p. You can easily reproduce that behaviour with /app/solr/solr/bin/snappuller-enable /dev/null The redirection of stdin from /dev/null causes fixUser to fallback to the second method but there are other, more realistic scenarios in which the fallback happens, like ssh [EMAIL PROTECTED] /app/solr/solr/bin/snappuller-enable The fix is to use the -f option which causes pgrep to compare the full path of the executable. Interestingly, that method is not subject to the 15 character length limit. The limit is not actually enforced by jetty but rather by the procfs file system of the linux kernel. If you look at /proc/*/stat you will notice that the second column is limited to 15 characters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-139) Support updateable/modifiable documents
[ https://issues.apache.org/jira/browse/SOLR-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12601408#action_12601408 ] Bill Au commented on SOLR-139: -- I noticed that this bug is no longer included in the 1.3 release. Are there any outstanding issues if all the fields are stored? Requiring that all fields are stored for a document to be update-able seems like reasonable to me. This feature will simplify things for Solr users who are doing a query to get all the fields following by an add when they only want to update a very small number of fields. Support updateable/modifiable documents --- Key: SOLR-139 URL: https://issues.apache.org/jira/browse/SOLR-139 Project: Solr Issue Type: New Feature Components: update Reporter: Ryan McKinley Assignee: Ryan McKinley Attachments: Eriks-ModifiableDocument.patch, Eriks-ModifiableDocument.patch, Eriks-ModifiableDocument.patch, Eriks-ModifiableDocument.patch, Eriks-ModifiableDocument.patch, Eriks-ModifiableDocument.patch, getStoredFields.patch, getStoredFields.patch, getStoredFields.patch, getStoredFields.patch, getStoredFields.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-IndexDocumentCommand.patch, SOLR-139-ModifyInputDocuments.patch, SOLR-139-ModifyInputDocuments.patch, SOLR-139-ModifyInputDocuments.patch, SOLR-139-ModifyInputDocuments.patch, SOLR-139-XmlUpdater.patch, SOLR-269+139-ModifiableDocumentUpdateProcessor.patch It would be nice to be able to update some fields on a document without having to insert the entire document. Given the way lucene is structured, (for now) one can only modify stored fields. While we are at it, we can support incrementing an existing value - I think this only makes sense for numbers. for background, see: http://www.nabble.com/loading-many-documents-by-ID-tf3145666.html#a8722293 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-531) rsyncd-start and snappuller should exit with different exit code when disabled
[ https://issues.apache.org/jira/browse/SOLR-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reassigned SOLR-531: Assignee: Bill Au rsyncd-start and snappuller should exit with different exit code when disabled -- Key: SOLR-531 URL: https://issues.apache.org/jira/browse/SOLR-531 Project: Solr Issue Type: Improvement Components: replication Affects Versions: 1.3 Reporter: Thomas Peuss Assignee: Bill Au Priority: Trivial Attachments: SOLR-531.patch When the rsyncd-start and snappuller scripts get executed the scripts check if they are enabled or not. If they are disabled they exit with 1. The scripts exit with 1 on error as well. So you cannot decide in upstream scripts if really an error occurred or if they are only disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-531) rsyncd-start and snappuller should exit with different exit code when disabled
[ https://issues.apache.org/jira/browse/SOLR-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-531. -- Resolution: Fixed Patch committed: SendingCHANGES.txt Sendingsrc/scripts/rsyncd-start Sendingsrc/scripts/snappuller Transmitting file data ... Committed revision 645608. Thanks, Thomas. rsyncd-start and snappuller should exit with different exit code when disabled -- Key: SOLR-531 URL: https://issues.apache.org/jira/browse/SOLR-531 Project: Solr Issue Type: Improvement Components: replication Affects Versions: 1.3 Reporter: Thomas Peuss Assignee: Bill Au Priority: Trivial Attachments: SOLR-531.patch When the rsyncd-start and snappuller scripts get executed the scripts check if they are enabled or not. If they are disabled they exit with 1. The scripts exit with 1 on error as well. So you cannot decide in upstream scripts if really an error occurred or if they are only disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-535) Small typo in schema.jsp (Tokenzied - Tokenized)
[ https://issues.apache.org/jira/browse/SOLR-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reassigned SOLR-535: Assignee: Bill Au Small typo in schema.jsp (Tokenzied - Tokenized) - Key: SOLR-535 URL: https://issues.apache.org/jira/browse/SOLR-535 Project: Solr Issue Type: Bug Components: web gui Reporter: Thomas Peuss Assignee: Bill Au Priority: Trivial Original Estimate: 0.02h Remaining Estimate: 0.02h Small typo in schema.jsp: Tokenzied - Tokenized (line 274) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-535) Small typo in schema.jsp (Tokenzied - Tokenized)
[ https://issues.apache.org/jira/browse/SOLR-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-535. -- Resolution: Fixed Typo fixed: SendingCHANGES.txt Sendingsrc/webapp/web/admin/schema.jsp Transmitting file data .. Committed revision 645614. Thanks, Thomas. Small typo in schema.jsp (Tokenzied - Tokenized) - Key: SOLR-535 URL: https://issues.apache.org/jira/browse/SOLR-535 Project: Solr Issue Type: Bug Components: web gui Reporter: Thomas Peuss Assignee: Bill Au Priority: Trivial Original Estimate: 0.02h Remaining Estimate: 0.02h Small typo in schema.jsp: Tokenzied - Tokenized (line 274) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-531) rsyncd-start and snappuller should exit with different exit code when disabled
[ https://issues.apache.org/jira/browse/SOLR-531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12585100#action_12585100 ] Bill Au commented on SOLR-531: -- The patch looks good to me. I will apply it if no one objects. rsyncd-start and snappuller should exit with different exit code when disabled -- Key: SOLR-531 URL: https://issues.apache.org/jira/browse/SOLR-531 Project: Solr Issue Type: Improvement Components: replication Affects Versions: 1.3 Reporter: Thomas Peuss Priority: Trivial Attachments: SOLR-531.patch When the rsyncd-start and snappuller scripts get executed the scripts check if they are enabled or not. If they are disabled they exit with 1. The scripts exit with 1 on error as well. So you cannot decide in upstream scripts if really an error occurred or if they are only disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-524) snappuller has limitation w/r/t/ handling multiple web apps
[
https://issues.apache.org/jira/browse/SOLR-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12584150#action_12584150
]
Bill Au commented on SOLR-524:
--
In the command line in question:
rsync -Wa${verbose}${compress} --delete ${sizeonly} \
${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip
The string solr IS NOT the webapp name. II is the name used by rsyncd to map
to a file system path.
Here is the content of rsyncd.conf, which is generated by rsyncd-start
dynamically:
rsyncd.conf file
uid = $(whoami)
gid = $(whoami)
use chroot = no
list = no
pid file = ${solr_root}/logs/rsyncd.pid
log file = ${solr_root}/logs/rsyncd.log
[solr]
path = ${data_dir}
comment = Solr
snappuller has limitation w/r/t/ handling multiple web apps
---
Key: SOLR-524
URL: https://issues.apache.org/jira/browse/SOLR-524
Project: Solr
Issue Type: Improvement
Components: replication
Affects Versions: 1.2
Environment: Linux (CentOS release 5 (Final))
Java JDK 6
Reporter: Ezra Epstein
Priority: Minor
Original Estimate: 0.5h
Remaining Estimate: 0.5h
The snappuller has a limitation which makes it hard to use for replicating
the indices for multiple webapps. In particular, by changing:
# rsync over files that have changed
rsync -Wa${verbose}${compress} --delete ${sizeonly} \
${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip
to:
# rsync over files that have changed
rsync -Wa${verbose}${compress} --delete ${sizeonly} \
${stats} rsync://${master_host}:${rsyncd_port}/${rsync_module_path}/${name}/
${data_dir}/${name}-wip
and adding an rsync_module_path variable to scripts.conf, plus giving it a
default value of solr before the 'unset' commands at the top of the
snappuller script, I've worked around the issue. Still, it seems better to
not hard-code the module name ([solr]) and also to allow some flexibility in
the location of the data files under that module. This is req'd for multiple
webapps since they won't share a data folder.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-463) script commit failed
[ https://issues.apache.org/jira/browse/SOLR-463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12561205#action_12561205 ] Bill Au commented on SOLR-463: -- This looks like a duplicate of solr-426. Can you either apply the patch contained in solr-426 or try the latest version of the commit script? script commit failed Key: SOLR-463 URL: https://issues.apache.org/jira/browse/SOLR-463 Project: Solr Issue Type: Bug Affects Versions: 1.3 Environment: Linux, Ubantu Reporter: Eric Fix For: 1.3 commit.log - 2008/01/22 09:33:03 started by root 2008/01/22 09:33:03 command: ./solr/bin/commit 2008/01/22 09:33:03 commit request to Solr at http://localhost:8080/solr/update failed: 2008/01/22 09:33:03 ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint name=status0/intint name=QTime12/int/lststr name=WARNINGThis response format is experimental. It is likely to change in the future./str /response 2008/01/22 09:33:03 failed (elapsed time: 0 sec) -- when i execute snapinstaller or commit scripts in sor/bin, get above commit log, and snapinstaller shows snapshot installed but Solr server has not open a new Searcher, in fact solr has already opened new searcher and updated contented can be searched;here the os is ubantu; But under federa, i can get same xml content in commit.log and without failed tip. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-393) contentType is set twice with conflicting values in raw-schema.jsp
[ https://issues.apache.org/jira/browse/SOLR-393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-393. -- Resolution: Fixed patch committed. Also updated CHANGES.txt. contentType is set twice with conflicting values in raw-schema.jsp -- Key: SOLR-393 URL: https://issues.apache.org/jira/browse/SOLR-393 Project: Solr Issue Type: Bug Components: web gui Reporter: Bill Au Attachments: solr-393.patch Content type is set twice in raw-schema.jsp with conflicting values, causing a compiling error in Resin: %@ page contentType=text/html; charset=utf-8 pageEncoding=UTF-8% %-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the License); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --% %@ page import=org.apache.solr.core.SolrCore, org.apache.solr.schema.IndexSchema% %@ page import=java.io.InputStreamReader% %@ page import=java.io.Reader% %@ page contentType=text/plain;charset=UTF-8 language=java % -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-393) contentType is set twice with conflicting values in raw-schema.jsp
contentType is set twice with conflicting values in raw-schema.jsp -- Key: SOLR-393 URL: https://issues.apache.org/jira/browse/SOLR-393 Project: Solr Issue Type: Bug Components: web gui Reporter: Bill Au Content type is set twice in raw-schema.jsp with conflicting values, causing a compiling error in Resin: %@ page contentType=text/html; charset=utf-8 pageEncoding=UTF-8% %-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the License); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --% %@ page import=org.apache.solr.core.SolrCore, org.apache.solr.schema.IndexSchema% %@ page import=java.io.InputStreamReader% %@ page import=java.io.Reader% %@ page contentType=text/plain;charset=UTF-8 language=java % -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-393) contentType is set twice with conflicting values in raw-schema.jsp
[ https://issues.apache.org/jira/browse/SOLR-393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-393: - Attachment: solr-393.patch The attached patch remove duplicates contentType settings, keeping the last one: %@ page contentType=text/plain;charset=UTF-8 language=java % contentType is set twice with conflicting values in raw-schema.jsp -- Key: SOLR-393 URL: https://issues.apache.org/jira/browse/SOLR-393 Project: Solr Issue Type: Bug Components: web gui Reporter: Bill Au Attachments: solr-393.patch Content type is set twice in raw-schema.jsp with conflicting values, causing a compiling error in Resin: %@ page contentType=text/html; charset=utf-8 pageEncoding=UTF-8% %-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the License); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --% %@ page import=org.apache.solr.core.SolrCore, org.apache.solr.schema.IndexSchema% %@ page import=java.io.InputStreamReader% %@ page import=java.io.Reader% %@ page contentType=text/plain;charset=UTF-8 language=java % -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-365) improper handling of user login greater than 8 characters
[ https://issues.apache.org/jira/browse/SOLR-365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530711 ] Bill Au commented on SOLR-365: -- Paul, what OS are you running? The who command on Red Hat Linux does not seems to have this problem. improper handling of user login greater than 8 characters - Key: SOLR-365 URL: https://issues.apache.org/jira/browse/SOLR-365 Project: Solr Issue Type: Bug Components: replication Environment: linux and probably other unix operating systems Reporter: Paul Sundling Priority: Minor to reproduce, create a user account that is more than 8 characters long. Then try to do a command like snappuller and even though the config is setup properly, it attempts to do a sudo. The reason is that 2 different methods are used to calculate the user, one of which truncates accounts to 8 characters. While user logins used to be limited to 8 characters, this may not be the case on modern UNIX. Here is a snippet I get by adding the -x debug flag to bash. Note how oldwhoami is a truncated version (psundlin) of the full login (psundling). + fixUser + [[ -z psundling ]] ++ whoami + [[ psundling != psundling ]] ++ who -m ++ cut '-d ' -f1 ++ sed '-es/^.*!//' + oldwhoami=psundlin + [[ psundlin == '' ]] -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (SOLR-353) default values of UpdateRequest not supported by Solr
[ https://issues.apache.org/jira/browse/SOLR-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au closed SOLR-353. Resolution: Fixed Fix Version/s: 1.3 Patch committed (Committed revision 574920). default values of UpdateRequest not supported by Solr - Key: SOLR-353 URL: https://issues.apache.org/jira/browse/SOLR-353 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.3 Attachments: solr-353.patch The default values of allowDups, overwriteCommitted, and overwritePending are all false in UpdateRequest. This combination is not supported in Solr. The default should be changed to overwrite = true: allowDups = false; overwriteCommitted = true; overwritePending = true; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit and optimize requests
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526463 ] Bill Au commented on SOLR-352: -- Thanks Ryan. I got it to work but I have to add req.setOverwrite(true); Otherwise, a got an exception from Solr: org.apache.solr.common.SolrException: unsupported param combo:add:,allowDups=false,overwritePending=false,overwriteCommitted=false unsupported param combo:add:,allowDups=false,overwritePending=false,overwriteCommitted=false request: http://cn-ewr1-dev40-pi2.cnet.com:7905/solr/update?commit=truewaitFlush=falsewaitSearcher=falsewt=xmlversion=2.2 So it looks like the combination of default values of allowDups, overwriteCommitted, and overwritePending (all false) in UpdateRequest is not supported by Solr. Should we change the default to something that is supported (setting overwirte to true)? I can open a separate bug and take care of that. UpdateRequest is duplicating commit and optimize requests - Key: SOLR-352 URL: https://issues.apache.org/jira/browse/SOLR-352 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-352.patch, solr-352.patch UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-335) solrj and ping / PingRequestHandler
[
https://issues.apache.org/jira/browse/SOLR-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526464
]
Bill Au commented on SOLR-335:
--
Since I am returning a null NamedList in SolrPingResponse, I have also put some
checks into SolrResponseBase to guide against NPE:
---
client/java/solrj/src/org/apache/solr/client/solrj/response/SolrResponseBase.java
(revision 574346)
+++
client/java/solrj/src/org/apache/solr/client/solrj/response/SolrResponseBase.java
(working copy)
@@ -63,11 +63,23 @@
// these two methods are based on the logic in
SolrCore.setResponseHeaderValues(...)
public int getStatus() {
-return (Integer) getResponseHeader().get(status);
+NamedList header = getResponseHeader();
+if (header != null) {
+return (Integer) header.get(status);
+}
+else {
+return 0;
+}
}
public int getQTime() {
-return (Integer) getResponseHeader().get(QTime);
+NamedList header = getResponseHeader();
+if (header != null) {
+return (Integer) header.get(QTime);
+}
+else {
+return 0;
+}
}
public String getRequestUrl() {
solrj and ping / PingRequestHandler
---
Key: SOLR-335
URL: https://issues.apache.org/jira/browse/SOLR-335
Project: Solr
Issue Type: Bug
Components: clients - java
Reporter: Ryan McKinley
Priority: Minor
Attachments: solr-335-workaround.patch
Solrj needs to talk to a PingRequestHandler
see:
http://www.nabble.com/-Solrj--Documentation---SolrServer-Ping-tf4246988.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-353) default values of UpdateRequest not supported by Solr
default values of UpdateRequest not supported by Solr - Key: SOLR-353 URL: https://issues.apache.org/jira/browse/SOLR-353 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor The default values of allowDups, overwriteCommitted, and overwritePending are all false in UpdateRequest. This combination is not supported in Solr. The default should be changed to overwrite = true: allowDups = false; overwriteCommitted = true; overwritePending = true; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit and optimize requests
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526497 ] Bill Au commented on SOLR-352: -- I have opened SOLR-353 for setting default overwrite=true in UpdateRequest. UpdateRequest is duplicating commit and optimize requests - Key: SOLR-352 URL: https://issues.apache.org/jira/browse/SOLR-352 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.3 Attachments: solr-352.patch, solr-352.patch UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-335) solrj and ping / PingRequestHandler
[ https://issues.apache.org/jira/browse/SOLR-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526195 ] Bill Au commented on SOLR-335: -- Until there is a PingRequestHandler, a simple work-around would be to pass in the required parameter for the standard request handler, ie q=solrpingquery. This is what the admin jsp does. I would like to provide a patch to fix the broken ping function in SolrJ. Should I attach it here or open a new bug. If I attach it here, we can keep this bug open until we have a PingRequestHandler. solrj and ping / PingRequestHandler --- Key: SOLR-335 URL: https://issues.apache.org/jira/browse/SOLR-335 Project: Solr Issue Type: Bug Components: clients - java Reporter: Ryan McKinley Priority: Minor Solrj needs to talk to a PingRequestHandler see: http://www.nabble.com/-Solrj--Documentation---SolrServer-Ping-tf4246988.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-352) UpdateRequest is duplicating commit and optimize requests
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-352: - Attachment: solr-352.patch Patch to remove commit/optimize query args since the request already contain an commit/optimize XML message in the POST body. UpdateRequest is duplicating commit and optimize requests - Key: SOLR-352 URL: https://issues.apache.org/jira/browse/SOLR-352 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-352.patch UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-352) UpdateRequest is duplicating commit and optimize requests
[ https://issues.apache.org/jira/browse/SOLR-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526213 ] Bill Au commented on SOLR-352: -- Ryan, I am gettign a NullPointerException when I tried you patch: java.lang.NullPointerException at org.apache.solr.common.util.ContentStreamBase$StringStream.init(ContentStreamBase.java:137) at org.apache.solr.client.solrj.util.ClientUtils.toContentStreams(ClientUtils.java:59) at org.apache.solr.client.solrj.request.UpdateRequest.getContentStreams(UpdateRequest.java:134) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:113) at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:199) at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:79) at org.apache.solr.client.solrj.impl.BaseSolrServer.commit(BaseSolrServer.java:68) at _jsp._solrjCommit__jsp._jspService(solrjCommit.jsp:12) at com.caucho.jsp.JavaPage.service(JavaPage.java:60) at com.caucho.jsp.Page.pageservice(Page.java:570) at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:179) at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:209) at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:173) at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:229) at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:274) at com.caucho.server.port.TcpConnection.run(TcpConnection.java:511) at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:520) at com.caucho.util.ThreadPool.run(ThreadPool.java:442) at java.lang.Thread.run(Thread.java:595) UpdateRequest is duplicating commit and optimize requests - Key: SOLR-352 URL: https://issues.apache.org/jira/browse/SOLR-352 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 1.3 Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-352.patch, solr-352.patch UpdateRequest current sets both query args and a update XML message in the POST body. This causes Solr to do two commit/optimize for each commit/optimize request sent in by SolrJ. I will be attaching a patch to remove the commit/optimize query args. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-335) solrj and ping / PingRequestHandler
[ https://issues.apache.org/jira/browse/SOLR-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526237 ] Bill Au commented on SOLR-335: -- It turns out that in the Solr response for a ping is neither a response or result but a solr: ?xml-stylesheet type=text/xsl href=ping.xsl? solr ping /ping /solr Since the response has no useful information, I will put an empty NamedList in SolrPingResponse. solrj and ping / PingRequestHandler --- Key: SOLR-335 URL: https://issues.apache.org/jira/browse/SOLR-335 Project: Solr Issue Type: Bug Components: clients - java Reporter: Ryan McKinley Priority: Minor Solrj needs to talk to a PingRequestHandler see: http://www.nabble.com/-Solrj--Documentation---SolrServer-Ping-tf4246988.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-136) snappuller - date -d and locales don't mix
[
https://issues.apache.org/jira/browse/SOLR-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-136:
-
Affects Version/s: 1.3
snappuller - date -d and locales don't mix
Key: SOLR-136
URL: https://issues.apache.org/jira/browse/SOLR-136
Project: Solr
Issue Type: Bug
Components: replication
Environment: SuSE 9.1
Reporter: Jürgen Hermann
Fix For: 1.3
In snappuller, the output of $(date) is fed back into date -d, which
doesn't work in some (non-US) locales:
date -d$(date)
date: ungültiges Datum ,,Fr Feb 2 13:39:04 CET 2007
date -d$(date +'%Y-%m-%d %H:%M:%S')
Fr Feb 2 13:39:10 CET 2007
This is the fix:
--- snappuller (revision 1038)
+++ snappuller (working copy)
@@ -214,7 +214,7 @@
ssh -o StrictHostKeyChecking=no ${master_host} mkdir -p ${master_status_dir}
# start new distribution stats
-rsyncStart=`date`
+rsyncStart=`date +'%Y-%m-%d %H:%M:%S'`
startTimestamp=`date -d $rsyncStart +'%Y%m%d-%H%M%S'`
rsyncStartSec=`date -d $rsyncStart +'%s'`
startStatus=rsync of `basename ${name}` started:$startTimestamp
@@ -226,7 +226,7 @@
${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip
rc=$?
-rsyncEnd=`date`
+rsyncEnd=`date +'%Y-%m-%d %H:%M:%S'`
endTimestamp=`date -d $rsyncEnd +'%Y%m%d-%H%M%S'`
rsyncEndSec=`date -d $rsyncEnd +'%s'`
elapsed=`expr $rsyncEndSec - $rsyncStartSec`
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-136) snappuller - date -d and locales don't mix
[
https://issues.apache.org/jira/browse/SOLR-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au updated SOLR-136:
-
Fix Version/s: (was: 1.1.0)
1.3
Affects Version/s: (was: 1.3)
snappuller - date -d and locales don't mix
Key: SOLR-136
URL: https://issues.apache.org/jira/browse/SOLR-136
Project: Solr
Issue Type: Bug
Components: replication
Environment: SuSE 9.1
Reporter: Jürgen Hermann
Fix For: 1.3
In snappuller, the output of $(date) is fed back into date -d, which
doesn't work in some (non-US) locales:
date -d$(date)
date: ungültiges Datum ,,Fr Feb 2 13:39:04 CET 2007
date -d$(date +'%Y-%m-%d %H:%M:%S')
Fr Feb 2 13:39:10 CET 2007
This is the fix:
--- snappuller (revision 1038)
+++ snappuller (working copy)
@@ -214,7 +214,7 @@
ssh -o StrictHostKeyChecking=no ${master_host} mkdir -p ${master_status_dir}
# start new distribution stats
-rsyncStart=`date`
+rsyncStart=`date +'%Y-%m-%d %H:%M:%S'`
startTimestamp=`date -d $rsyncStart +'%Y%m%d-%H%M%S'`
rsyncStartSec=`date -d $rsyncStart +'%s'`
startStatus=rsync of `basename ${name}` started:$startTimestamp
@@ -226,7 +226,7 @@
${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip
rc=$?
-rsyncEnd=`date`
+rsyncEnd=`date +'%Y-%m-%d %H:%M:%S'`
endTimestamp=`date -d $rsyncEnd +'%Y%m%d-%H%M%S'`
rsyncEndSec=`date -d $rsyncEnd +'%s'`
elapsed=`expr $rsyncEndSec - $rsyncStartSec`
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-282) snapshooter does not work under solaris
[ https://issues.apache.org/jira/browse/SOLR-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-282: - Fix Version/s: 1.3 snapshooter does not work under solaris Key: SOLR-282 URL: https://issues.apache.org/jira/browse/SOLR-282 Project: Solr Issue Type: Bug Components: replication Affects Versions: 1.2 Environment: solaris Reporter: Xuesong Luo Assignee: Bill Au Fix For: 1.3 Attachments: solr-282-solaris-and-osx.patch, solr-282.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg04761.html solr is able to find snapshooter but didn't generate any snapshot files after the index is updated. I checked the log, everything looks fine, then I run snapshooter from command line. It failed because Solaris doesn't support -l option for cp command. I changed command cp -lr dir1 dir2 to: mkdir dir2 ln dir1/* dir2 It seems working. Otis suggested to create an issue so that Bill Au Co. can fix this problem. Please note: several other commands under solr/bin also have this problem. You can use grep cp -lr to find all of them and make similar changes. I'm also curious why there is no error log when solr failed running snapshooter. Shouldn't solr log an error message? Thanks Xuesong -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-294) scripts fail to log elapsed time on Solaris
[ https://issues.apache.org/jira/browse/SOLR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-294: - Fix Version/s: 1.3 scripts fail to log elapsed time on Solaris --- Key: SOLR-294 URL: https://issues.apache.org/jira/browse/SOLR-294 Project: Solr Issue Type: Bug Components: replication Environment: Solaris Reporter: Bill Au Assignee: Bill Au Priority: Minor Fix For: 1.3 Attachments: solr-294.patch The code in the scritps to determine the elapsed time does not work on Solaris because the date command there does not support the %s output format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-333) distributiondump.jsp gets the path to logs wrong
[ https://issues.apache.org/jira/browse/SOLR-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-333: - Attachment: solr-333.patch It turns out to be a very simple change in the JSP as it already has obtained solr home from solr.core.Config. Attached in the patch. I will apply it tomorrow if no one objects. distributiondump.jsp gets the path to logs wrong Key: SOLR-333 URL: https://issues.apache.org/jira/browse/SOLR-333 Project: Solr Issue Type: Bug Environment: Tomcat 5.5 Reporter: Ken Prows Assignee: Bill Au Fix For: 1.2 Attachments: solr-333.patch I can't seem to get the Distribution Info page to work. I think it is because I have a custom path to solr. My path to the logs directory looks like this: /apps/search/solr/catalog/logs (The base url to solr ends up being http://domain.com/catalog) I have to create my path like this in order to have multiple instances of solr running in the same container. distributiondump.jsp is creating the path to the status file using: cwd + /solr/logs/snapshot.current, which ends up being wrong. Would it be possible to make it use a rootdir variable like: rootdir + /logs/snapshot.current? If there is no way to determine the rootdir, perhaps it could be added to solrconfig.xml (or at least let it be overridden there.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-333) distributiondump.jsp gets the path to logs wrong
[ https://issues.apache.org/jira/browse/SOLR-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au reassigned SOLR-333: Assignee: Bill Au distributiondump.jsp gets the path to logs wrong Key: SOLR-333 URL: https://issues.apache.org/jira/browse/SOLR-333 Project: Solr Issue Type: Bug Environment: Tomcat 5.5 Reporter: Ken Prows Assignee: Bill Au Fix For: 1.2 I can't seem to get the Distribution Info page to work. I think it is because I have a custom path to solr. My path to the logs directory looks like this: /apps/search/solr/catalog/logs (The base url to solr ends up being http://domain.com/catalog) I have to create my path like this in order to have multiple instances of solr running in the same container. distributiondump.jsp is creating the path to the status file using: cwd + /solr/logs/snapshot.current, which ends up being wrong. Would it be possible to make it use a rootdir variable like: rootdir + /logs/snapshot.current? If there is no way to determine the rootdir, perhaps it could be added to solrconfig.xml (or at least let it be overridden there.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-333) distributiondump.jsp gets the path to logs wrong
[ https://issues.apache.org/jira/browse/SOLR-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519509 ] Bill Au commented on SOLR-333: -- I am going to update the JSP to behave the same way as Solr does in determining solr home: 1) JNDI lookup 2) system property solr.solr.home 3) default to cwd Ken, so you will need to do either 1) or 2) to set Solr home. distributiondump.jsp gets the path to logs wrong Key: SOLR-333 URL: https://issues.apache.org/jira/browse/SOLR-333 Project: Solr Issue Type: Bug Environment: Tomcat 5.5 Reporter: Ken Prows Assignee: Bill Au Fix For: 1.2 I can't seem to get the Distribution Info page to work. I think it is because I have a custom path to solr. My path to the logs directory looks like this: /apps/search/solr/catalog/logs (The base url to solr ends up being http://domain.com/catalog) I have to create my path like this in order to have multiple instances of solr running in the same container. distributiondump.jsp is creating the path to the status file using: cwd + /solr/logs/snapshot.current, which ends up being wrong. Would it be possible to make it use a rootdir variable like: rootdir + /logs/snapshot.current? If there is no way to determine the rootdir, perhaps it could be added to solrconfig.xml (or at least let it be overridden there.) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-327) shell scripts failed to run on Solaris 8 (and probably other non recent UNIX)
[ https://issues.apache.org/jira/browse/SOLR-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517996 ] Bill Au commented on SOLR-327: -- The latest version of the scripts no longer use cp -l on Solaris: https://issues.apache.org/jira/browse/SOLR-282 I no longer have access to a Solaris machine so I cannot check myself but according to the Solaris 8 documentation published by SUN pgrep is available on Solaris 8: http://docs.sun.com/app/docs/doc/806-0624/6j9vek5cu?a=view What other incompatibilities have you come across? shell scripts failed to run on Solaris 8 (and probably other non recent UNIX) - Key: SOLR-327 URL: https://issues.apache.org/jira/browse/SOLR-327 Project: Solr Issue Type: Bug Reporter: Paul Sundling Priority: Minor There are several places where commands used are incompatible with older UNIX versions, even though these capabilities are present. There are ways to rewrite the shell scripts to be compatible with these older versions. The first example is the use of pgrep. Older machines will have grep, egrep and fgrep, but NOT pgrep. I've been doing UNIX for well over a decade and never heard of pgrep, although it is installed on my home server apparently. :) There are also enhancements like the use of 'cp -l'. This could be accomplished with the 'ln' command instead on older UNIX versions. Since it's also used recursively in snapshooter, which 'ln' doesn't support, it makes the command more complex. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-294) scripts fail to log elapsed time on Solaris
[ https://issues.apache.org/jira/browse/SOLR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-294. -- Resolution: Fixed Patch has been committed. scripts fail to log elapsed time on Solaris --- Key: SOLR-294 URL: https://issues.apache.org/jira/browse/SOLR-294 Project: Solr Issue Type: Bug Components: replication Environment: Solaris Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-294.patch The code in the scritps to determine the elapsed time does not work on Solaris because the date command there does not support the %s output format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-136) snappuller - date -d and locales don't mix
[
https://issues.apache.org/jira/browse/SOLR-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Au resolved SOLR-136.
--
Resolution: Fixed
Patch commited. Thanks Jürgen.
snappuller - date -d and locales don't mix
Key: SOLR-136
URL: https://issues.apache.org/jira/browse/SOLR-136
Project: Solr
Issue Type: Bug
Components: replication
Environment: SuSE 9.1
Reporter: Jürgen Hermann
Fix For: 1.1.0
In snappuller, the output of $(date) is fed back into date -d, which
doesn't work in some (non-US) locales:
date -d$(date)
date: ungültiges Datum ,,Fr Feb 2 13:39:04 CET 2007
date -d$(date +'%Y-%m-%d %H:%M:%S')
Fr Feb 2 13:39:10 CET 2007
This is the fix:
--- snappuller (revision 1038)
+++ snappuller (working copy)
@@ -214,7 +214,7 @@
ssh -o StrictHostKeyChecking=no ${master_host} mkdir -p ${master_status_dir}
# start new distribution stats
-rsyncStart=`date`
+rsyncStart=`date +'%Y-%m-%d %H:%M:%S'`
startTimestamp=`date -d $rsyncStart +'%Y%m%d-%H%M%S'`
rsyncStartSec=`date -d $rsyncStart +'%s'`
startStatus=rsync of `basename ${name}` started:$startTimestamp
@@ -226,7 +226,7 @@
${stats} rsync://${master_host}:${rsyncd_port}/solr/${name}/
${data_dir}/${name}-wip
rc=$?
-rsyncEnd=`date`
+rsyncEnd=`date +'%Y-%m-%d %H:%M:%S'`
endTimestamp=`date -d $rsyncEnd +'%Y%m%d-%H%M%S'`
rsyncEndSec=`date -d $rsyncEnd +'%s'`
elapsed=`expr $rsyncEndSec - $rsyncStartSec`
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-282) snapshooter does not work under solaris
[ https://issues.apache.org/jira/browse/SOLR-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-282. -- Resolution: Fixed I have committed the patch. snapshooter does not work under solaris Key: SOLR-282 URL: https://issues.apache.org/jira/browse/SOLR-282 Project: Solr Issue Type: Bug Components: replication Affects Versions: 1.2 Environment: solaris Reporter: Xuesong Luo Assignee: Bill Au Attachments: solr-282-solaris-and-osx.patch, solr-282.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg04761.html solr is able to find snapshooter but didn't generate any snapshot files after the index is updated. I checked the log, everything looks fine, then I run snapshooter from command line. It failed because Solaris doesn't support -l option for cp command. I changed command cp -lr dir1 dir2 to: mkdir dir2 ln dir1/* dir2 It seems working. Otis suggested to create an issue so that Bill Au Co. can fix this problem. Please note: several other commands under solr/bin also have this problem. You can use grep cp -lr to find all of them and make similar changes. I'm also curious why there is no error log when solr failed running snapshooter. Shouldn't solr log an error message? Thanks Xuesong -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-213) snapshooter link creation doesn't work on OS X
[ https://issues.apache.org/jira/browse/SOLR-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au resolved SOLR-213. -- Resolution: Duplicate I have committed the patch for SOLR-282 which will take care of this problem for both OS X and Solaris. snapshooter link creation doesn't work on OS X -- Key: SOLR-213 URL: https://issues.apache.org/jira/browse/SOLR-213 Project: Solr Issue Type: Bug Components: replication Environment: OS X 10.4.9 Reporter: Grant Ingersoll Assignee: Bill Au Priority: Minor The snapshooter script fails on the cp -l command when trying to create the hard links for a snapshot. Should be able to use ln instead. Also should look into if there are other issues on OS X. There may be some relation to https://issues.apache.org/jira/browse/SOLR-93 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-294) scripts fail to log elapsed time on Solaris
[ https://issues.apache.org/jira/browse/SOLR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-294: - Attachment: solr-294.patch Here is the patch to using perl -e print time; to get the timestamp in seconds on Solaris. I have added a new function in scripts-util to set the start time and updated all the scripts to use that. This way the code to get timestamp and to calculate elapsed time is centralized in scripts.util. We will just have to update a single source if we need to change it to support additional OSes. scripts fail to log elapsed time on Solaris --- Key: SOLR-294 URL: https://issues.apache.org/jira/browse/SOLR-294 Project: Solr Issue Type: Bug Components: replication Environment: Solaris Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-294.patch The code in the scritps to determine the elapsed time does not work on Solaris because the date command there does not support the %s output format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-294) scripts fail to log elapsed time on Solaris
[ https://issues.apache.org/jira/browse/SOLR-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512085 ] Bill Au commented on SOLR-294: -- By the way, I have lost my access to Solaris. While I did tested the code snipplet the patch is using before when I still had access, I am now not able to test the patch itself on Solaris. It will be good if someone using Solaris check this out. scripts fail to log elapsed time on Solaris --- Key: SOLR-294 URL: https://issues.apache.org/jira/browse/SOLR-294 Project: Solr Issue Type: Bug Components: replication Environment: Solaris Reporter: Bill Au Assignee: Bill Au Priority: Minor Attachments: solr-294.patch The code in the scritps to determine the elapsed time does not work on Solaris because the date command there does not support the %s output format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-213) snapshooter link creation doesn't work on OS X
[ https://issues.apache.org/jira/browse/SOLR-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511406 ] Bill Au commented on SOLR-213: -- I have attached an updated patch to SOLR-282 that works for both Solaris and OS X. snapshooter link creation doesn't work on OS X -- Key: SOLR-213 URL: https://issues.apache.org/jira/browse/SOLR-213 Project: Solr Issue Type: Bug Components: replication Environment: OS X 10.4.9 Reporter: Grant Ingersoll Assignee: Bill Au Priority: Minor The snapshooter script fails on the cp -l command when trying to create the hard links for a snapshot. Should be able to use ln instead. Also should look into if there are other issues on OS X. There may be some relation to https://issues.apache.org/jira/browse/SOLR-93 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-282) snapshooter does not work under solaris
[ https://issues.apache.org/jira/browse/SOLR-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Au updated SOLR-282: - Attachment: solr-282.patch Attached is a patch for using cpio instead of cp -l when the scripts are run on SunOS. While I was at it, I also updated the commit and optimize related scripts to be able to handle both the old and new response format. snapshooter does not work under solaris Key: SOLR-282 URL: https://issues.apache.org/jira/browse/SOLR-282 Project: Solr Issue Type: Bug Components: replication Affects Versions: 1.2 Environment: solaris Reporter: Xuesong Luo Assignee: Bill Au Attachments: solr-282.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg04761.html solr is able to find snapshooter but didn't generate any snapshot files after the index is updated. I checked the log, everything looks fine, then I run snapshooter from command line. It failed because Solaris doesn't support -l option for cp command. I changed command cp -lr dir1 dir2 to: mkdir dir2 ln dir1/* dir2 It seems working. Otis suggested to create an issue so that Bill Au Co. can fix this problem. Please note: several other commands under solr/bin also have this problem. You can use grep cp -lr to find all of them and make similar changes. I'm also curious why there is no error log when solr failed running snapshooter. Shouldn't solr log an error message? Thanks Xuesong -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-282) snapshooter does not work under solaris
[ https://issues.apache.org/jira/browse/SOLR-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511236 ] Bill Au commented on SOLR-282: -- I have tested my patch on the only Solaris box that I have access to. That is running Solaris 8. It will be good if someone with access to a more recent version of Solaris check it out too. snapshooter does not work under solaris Key: SOLR-282 URL: https://issues.apache.org/jira/browse/SOLR-282 Project: Solr Issue Type: Bug Components: replication Affects Versions: 1.2 Environment: solaris Reporter: Xuesong Luo Assignee: Bill Au Attachments: solr-282.patch http://www.mail-archive.com/[EMAIL PROTECTED]/msg04761.html solr is able to find snapshooter but didn't generate any snapshot files after the index is updated. I checked the log, everything looks fine, then I run snapshooter from command line. It failed because Solaris doesn't support -l option for cp command. I changed command cp -lr dir1 dir2 to: mkdir dir2 ln dir1/* dir2 It seems working. Otis suggested to create an issue so that Bill Au Co. can fix this problem. Please note: several other commands under solr/bin also have this problem. You can use grep cp -lr to find all of them and make similar changes. I'm also curious why there is no error log when solr failed running snapshooter. Shouldn't solr log an error message? Thanks Xuesong -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-213) snapshooter link creation doesn't work on OS X
[ https://issues.apache.org/jira/browse/SOLR-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511238 ] Bill Au commented on SOLR-213: -- This same problem occurs on Solaris (see bug SOLR-282). I have just attached a patch there. I will see if the solution for Solaris will also work for OS X or not later tonight when I have access to OS X. snapshooter link creation doesn't work on OS X -- Key: SOLR-213 URL: https://issues.apache.org/jira/browse/SOLR-213 Project: Solr Issue Type: Bug Components: replication Environment: OS X 10.4.9 Reporter: Grant Ingersoll Priority: Minor The snapshooter script fails on the cp -l command when trying to create the hard links for a snapshot. Should be able to use ln instead. Also should look into if there are other issues on OS X. There may be some relation to https://issues.apache.org/jira/browse/SOLR-93 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
