Re: Pylucene release
On Nov 29, 2012, at 5:37, Shawn Grant shawn.gr...@orcatec.com wrote: Hi Andi, thanks for the explanation. The main problem I've come across so far is that it looks like the main branch lucene has a lucene41 codec in it that does not appear to be part of the 4.0 release and (I think) is causing problems creating and/or retrieving term vectors. I'm not a lucene expert and it's been hard to diagnose. I also can't use Luke due to the codec. PyLucene trunk is currently tracking Lucene's branch 4.x. I'd expect the lucene41 codec to be available there. I tried to set the default codec to lucene40 but then my index writer complained that lucene40 was only for reading. You should ask on the lucene-user@ list. There are more people listening there who would know the details. I'll try to contribute to porting the unit tests to help move the release along. Cool ! Andi.. On 11/13/2012 02:18 PM, Andi Vajda wrote: Hi Shawn, On Tue, 13 Nov 2012, Shawn Grant wrote: Hi Andi, I was just wondering if Pylucene is on its usual schedule to release 4-6 weeks after Lucene. I didn't see any discussion of it on the mailing list or elsewhere. I'm looking forward to 4.0! Normally, PyLucene is released a few days after a Lucene release but 4.0 has seen so many API changes and removals that all tests and samples need to be ported to the new API. Last week-end, I ported a few but lots remain to be. If no one helps, it either means that no one cares enough or that everyone is willing to be patient :-) The PyLucene trunk svn repository is currently tracking the Lucene Core 4.x branch and you're welcome to use it out of svn. In the ten or so unit tests I ported so far, I didn't find any issues with PyLucene proper (or JCC). All changes were due to the tests being out of date or using deprecated APIs now removed. You might find that PyLucene out-of-trunk is quite usable. If people want to help with porting PyLucene unit tests, the ones under its 'test' directory not yet ported, feel free to ask questions here. The gist of it is: - fix the imports (look at the first few tests for example, alphabetically) - fix the tests to pass by looking at the original Java tests for changes as most of these tests were originally ported from Java Lucene. Once you're familiar with the new APIs, porting the sample code in samples and in LuceneInAction should fairly straightforward. It's just that there is a lot to port. Andi..
AW: Pylucene release
Hi Andi, I still wanted to check the API changes related to 4.0 and could then help with porting the example code (and/or unit tests). I hope there are more people interested in helping to port PyLucene (or at least the 'related' Python code) to the Lucene 4.0 level... How can we best proceed? I assume you checked in the code that's adapted already to SVN. Is there a list of code that needs to be ported (and can be used to distribute tasks)? As said I don't have a an idea of the API changes yet, so it's hard to estimate the time needed to get used to 4.0 (and fix the code), but as you did that already maybe you can share your experience with us. As with any new major release (e.g. Python 3.x) I guess many of us are afraid to move forward to the new release and change our code base, but certainly that's just a matter of time ... Cheers, Thomas -Ursprüngliche Nachricht- Von: Andi Vajda [mailto:va...@apache.org] Gesendet: Dienstag, 13. November 2012 23:18 An: Shawn Grant Cc: pylucene-dev@lucene.apache.org Betreff: Re: Pylucene release Hi Shawn, On Tue, 13 Nov 2012, Shawn Grant wrote: Hi Andi, I was just wondering if Pylucene is on its usual schedule to release 4-6 weeks after Lucene. I didn't see any discussion of it on the mailing list or elsewhere. I'm looking forward to 4.0! Normally, PyLucene is released a few days after a Lucene release but 4.0 has seen so many API changes and removals that all tests and samples need to be ported to the new API. Last week-end, I ported a few but lots remain to be. If no one helps, it either means that no one cares enough or that everyone is willing to be patient :-) The PyLucene trunk svn repository is currently tracking the Lucene Core 4.x branch and you're welcome to use it out of svn. In the ten or so unit tests I ported so far, I didn't find any issues with PyLucene proper (or JCC). All changes were due to the tests being out of date or using deprecated APIs now removed. You might find that PyLucene out-of-trunk is quite usable. If people want to help with porting PyLucene unit tests, the ones under its 'test' directory not yet ported, feel free to ask questions here. The gist of it is: - fix the imports (look at the first few tests for example, alphabetically) - fix the tests to pass by looking at the original Java tests for changes as most of these tests were originally ported from Java Lucene. Once you're familiar with the new APIs, porting the sample code in samples and in LuceneInAction should fairly straightforward. It's just that there is a lot to port. Andi..
Re: AW: Pylucene release
Hi Thomas, On Wed, 14 Nov 2012, Thomas Koch wrote: I still wanted to check the API changes related to 4.0 and could then help with porting the example code (and/or unit tests). I hope there are more people interested in helping to port PyLucene (or at least the 'related' Python code) to the Lucene 4.0 level... How can we best proceed? 1. Pick a test that fails (for example: python test/test_FuzzyQuery.py) 2. Announce you're working on it on the list (so that only you does) 3. Fix it 4. Send in a patch I assume you checked in the code that's adapted already to SVN. Yes, all current code is checked in, including fixed or broken tests. Is there a list of code that needs to be ported (and can be used to distribute tasks)? Currently, all tests in test up to test_FilteredQuery.py (alphabetically) pass. The test_ICU* ones also pass. You should use these as examples on how to fix failing ones. As said I don't have a an idea of the API changes yet, so it's hard to estimate the time needed to get used to 4.0 No time estimated is expected from you. It's best to proceed by example. Look at the tests that pass already (and thus that have been fixed) as examples. The steps to fix a failing test are as follows: - fix import statements first (they're all changed since PyLucene 4.0 no longer uses a flat namespace but strictly follows the original Java package structure now) for example: from lucene import Document becomes from org.apache.lucene.document import Document If you don't know where a class is (and the Lucene tree is deeply nested), find lucene src -name ClassName.java will usually give you an idea of the package structure to import - when it makes sense (most of the time), use PyLuceneTestCase as the parent test class. This will help with the complexities/boilerplate in creating a test IndexWriter/Reader/Searcher using a RAMDirectory - if the tests still fails, look at the original Java test code for possible changes in the API or the expected that behaviour that occurred since the first port. The original Java test file is usually named TestName.java when the Python test is named test_Name.py Andi.. (and fix the code), but as you did that already maybe you can share your experience with us. As with any new major release (e.g. Python 3.x) I guess many of us are afraid to move forward to the new release and change our code base, but certainly that's just a matter of time ... Cheers, Thomas -Ursprüngliche Nachricht- Von: Andi Vajda [mailto:va...@apache.org] Gesendet: Dienstag, 13. November 2012 23:18 An: Shawn Grant Cc: pylucene-dev@lucene.apache.org Betreff: Re: Pylucene release Hi Shawn, On Tue, 13 Nov 2012, Shawn Grant wrote: Hi Andi, I was just wondering if Pylucene is on its usual schedule to release 4-6 weeks after Lucene. I didn't see any discussion of it on the mailing list or elsewhere. I'm looking forward to 4.0! Normally, PyLucene is released a few days after a Lucene release but 4.0 has seen so many API changes and removals that all tests and samples need to be ported to the new API. Last week-end, I ported a few but lots remain to be. If no one helps, it either means that no one cares enough or that everyone is willing to be patient :-) The PyLucene trunk svn repository is currently tracking the Lucene Core 4.x branch and you're welcome to use it out of svn. In the ten or so unit tests I ported so far, I didn't find any issues with PyLucene proper (or JCC). All changes were due to the tests being out of date or using deprecated APIs now removed. You might find that PyLucene out-of-trunk is quite usable. If people want to help with porting PyLucene unit tests, the ones under its 'test' directory not yet ported, feel free to ask questions here. The gist of it is: - fix the imports (look at the first few tests for example, alphabetically) - fix the tests to pass by looking at the original Java tests for changes as most of these tests were originally ported from Java Lucene. Once you're familiar with the new APIs, porting the sample code in samples and in LuceneInAction should fairly straightforward. It's just that there is a lot to port. Andi..
Re: Pylucene release
Hi Shawn, On Tue, 13 Nov 2012, Shawn Grant wrote: Hi Andi, I was just wondering if Pylucene is on its usual schedule to release 4-6 weeks after Lucene. I didn't see any discussion of it on the mailing list or elsewhere. I'm looking forward to 4.0! Normally, PyLucene is released a few days after a Lucene release but 4.0 has seen so many API changes and removals that all tests and samples need to be ported to the new API. Last week-end, I ported a few but lots remain to be. If no one helps, it either means that no one cares enough or that everyone is willing to be patient :-) The PyLucene trunk svn repository is currently tracking the Lucene Core 4.x branch and you're welcome to use it out of svn. In the ten or so unit tests I ported so far, I didn't find any issues with PyLucene proper (or JCC). All changes were due to the tests being out of date or using deprecated APIs now removed. You might find that PyLucene out-of-trunk is quite usable. If people want to help with porting PyLucene unit tests, the ones under its 'test' directory not yet ported, feel free to ask questions here. The gist of it is: - fix the imports (look at the first few tests for example, alphabetically) - fix the tests to pass by looking at the original Java tests for changes as most of these tests were originally ported from Java Lucene. Once you're familiar with the new APIs, porting the sample code in samples and in LuceneInAction should fairly straightforward. It's just that there is a lot to port. Andi..
Re: preparing for a PyLucene release
Andi Vajda wrote: On Fri, 13 Feb 2009, Michael McCandless wrote: I think this is a good idea! There were discussions late last year to do a 2.9 release for Lucene, but realistically it's probably a ways out still. Even a 2.4.1 release, just bug fixes ? Possibly we'll do a 2.4.1 before 2.9, but 2.4.0 has proven stable and I don't think there've been discussions/requests for a 2.4.1. Mike
Re: preparing for a PyLucene release
I think this is a good idea! There were discussions late last year to do a 2.9 release for Lucene, but realistically it's probably a ways out still. The Lucene java release checklist might be a good place to check for the typical things: http://wiki.apache.org/lucene-java/ReleaseTodo Alot of that is specific to Lucene java, but a good amount is generic and should be helpful. I'd recommend keeping notes of what you did and then posting to the Wiki. Once you have a release candidate posted, then you should call for a vote; you should definitely encourage anyone everyone to test/vote (not just PMC members). Mike Andi Vajda wrote: Hi, Apart from the JIRA project (issue INFRA-1861) which seems stuck, the transition for PyLucene from OSAF to Apache is now complete. Historically, PyLucene has had official releases closely tracking Java Lucene's. At the moment, it doesn't appear that Java Lucene is preparing for a release anytime soon. I'd like to propose creating an offical Apache PyLucene source release now instead of waiting for the next Java Lucene release for the following purposes: - practising the process (new to me) - getting a release out there for people looking for it under the not-yet-created 'releases' menu under 'resources' - announcing the move to a larger audience such as Freshmeat From a functionality standpoint, this release would still be built around Java Lucene 2.4.0 but would include the latest bug fixes since the last release made at OSAF. I propose calling this release PyLucene-2.4.0-3. I read the documents at [1] and [2] and I'm ready to kick-off the process. In particular, I fixed the NOTICE and LICENSE files again. At what time in that process do I need a vote from the Lucene PMC ? Is there anything else required besides the vote and what's described at [1] and [2] ? Andi.. [1] http://www.apache.org/dev/mirror-step-by-step.html?Step-By-Step [2] http://www.apache.org/dev/#releases
