Re: pylucene on Ubuntu 18.04
> JCC monkeypatches as needed. I don't think monkey patching worked on Ubuntu 18.04, for any version of JCC that I tried. When running make, I ran into errors discussing how JCC was not built with shared support. > Does Lucene 4 even build and run with Java 11 ? No idea. I halted my efforts once I got PyLucene 4.10.1 working with openjdk-8-jdk.
Re: pylucene on Ubuntu 18.04
> On May 29, 2018, at 22:26, Jeff Breidenbach wrote: > > I'd previously been using 4.9.0. Making a huge jump in release numbers is > scary. Will it invalidate existing indexes? Yes, Lucene N (where N is a major release) only supports N, N - 1 indexes and maybe (?) N - 2 read only (unsure). So yes, your Lucene 4 indexes are dead in Lucene 7. > Has the Lucene API changed a lot? Yes, for every major release, there are deprecations and API breakages. Changes are well documented but still a pain to migrate to. > Will I be forced to migrate to python 3? No, PyLucene (JCC in fact) supports both Python 2 and Python 3. This, since JCC 3 was released with PyLucene 6. I see no reason to drop support for Python 2. > That said, I was desperate and tried > all the official releases (4.10.1, 6.2.0, 6.4.1, 6.5.0). It just so happened > that > 4.10.1 was the first thing I got to work. > > Here are things I ran into. Just mentioning them, not asking for changes. > > The distribution packages for python-lucene is totally busted. > I made the original package, someone else updated it in 2013, > but it was all built against openjdk-7-jdk which is long gone. Nowadays, Java major releases happen a lot faster and Lucene keeps up. > I think python-setuptools that ship with Ubuntu 18.04 do not include > shared support. The jcc/patches no longer apply. I ended up hacking > the heck out of it (including copying files from older versions of Ubuntu) > to add shared support. These patch files shouldn't be necessary anymore, JCC monkeypatches as needed. > I think had some trouble with conflicting JVMs and really only got things > working after removing all but one from the system. Yes, that's expected. > I got confused by python setup.py build; I kept modifying setup.py and > trying to rebuild, but didn't notice it didn't actually rebuild. Ended up > doing > a lot of rm -rf once I realized this. Yes, rm -rf build is necessary. > There are a lot of if statements in jcc/setup.py that try to reason about > shared support. I often got confused what it thought. Ended up removing > all the ifs and locking everything to shared. They try to figure what to monkeypatch. > Due to sleepiness, for a while I mistakenly thought this line was a > typo in the Makefile and was supposed to be two separate lines. > ANT=JAVA_HOME=/usr/lib/jvm/java-8-oracle /usr/bin/ant > > It looks like libjava.so and libjvm.so in openjdk-11-jdk are in different > directories compared to openjdk-8-jdk. Quite possibly so, but I haven't tried that version yet. Does Lucene 4 even build and run with Java 11 ? Andi.. > > > > > > > > >
Re: pylucene on Ubuntu 18.04
I'd previously been using 4.9.0. Making a huge jump in release numbers is scary. Will it invalidate existing indexes? Has the Lucene API changed a lot? Will I be forced to migrate to python 3? That said, I was desperate and tried all the official releases (4.10.1, 6.2.0, 6.4.1, 6.5.0). It just so happened that 4.10.1 was the first thing I got to work. Here are things I ran into. Just mentioning them, not asking for changes. The distribution packages for python-lucene is totally busted. I made the original package, someone else updated it in 2013, but it was all built against openjdk-7-jdk which is long gone. I think python-setuptools that ship with Ubuntu 18.04 do not include shared support. The jcc/patches no longer apply. I ended up hacking the heck out of it (including copying files from older versions of Ubuntu) to add shared support. I think had some trouble with conflicting JVMs and really only got things working after removing all but one from the system. I got confused by python setup.py build; I kept modifying setup.py and trying to rebuild, but didn't notice it didn't actually rebuild. Ended up doing a lot of rm -rf once I realized this. There are a lot of if statements in jcc/setup.py that try to reason about shared support. I often got confused what it thought. Ended up removing all the ifs and locking everything to shared. Due to sleepiness, for a while I mistakenly thought this line was a typo in the Makefile and was supposed to be two separate lines. ANT=JAVA_HOME=/usr/lib/jvm/java-8-oracle /usr/bin/ant It looks like libjava.so and libjvm.so in openjdk-11-jdk are in different directories compared to openjdk-8-jdk.
Re: pylucene on Ubuntu 18.04
On Mon, 28 May 2018, Jeff Breidenbach wrote: Hi Andi, I am happy to report that I just got 4.10.1 working, using some extreme measures. Whew. Will give a clearer report after getting some sleep. But I ask again, why the old release ? Andi.. Jeff
Re: pylucene on Ubuntu 18.04
Hi Andi, I am happy to report that I just got 4.10.1 working, using some extreme measures. Whew. Will give a clearer report after getting some sleep. Jeff
Re: pylucene on Ubuntu 18.04
Hi Jeff, On Mon, 28 May 2018, Jeff Breidenbach wrote: To be a little more specific, here's what happens with version 4.9.0 which I've had good luck with in the past. The system contains the following shared libraries. Why are you using such an old release ? I'm afraid that old release is going to depend on equally old everything else to function. I don't think Lucene 4.9.0 is even supported with Java 8. How about you use the pylucene 7.2.0 rc1 release candidate available from https://dist.apache.org/repos/dist/dev/lucene/pylucene/7.2.0-rc1/ instead ? Andi.. /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so It looks like LFLAGS in jcc/setup.py should find them. I'm on the linux2/X86_64 platform, using '/usr/lib/jvm/java-8-openjdk-amd64' for the JDK. JCC builds and installs without complaint, but fails at runtime. Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main mod_name, _Error) File "/usr/lib/python2.7/runpy.py", line 111, in _get_module_details __import__(mod_name) # Do not catch exceptions initializing package File "/usr/local/lib/python2.7/dist-packages/JCC-2.20-py2.7-linux-x86_64.egg/jcc/__init__.py", line 31, in from jcc import _jcc ImportError: libjvm.so: cannot open shared object file: No such file or directory
Re: pylucene on Ubuntu 18.04
And for what it is worth, python-setuptools claims to be version 39.0.1. I've probably spent about 10 to 12 hours trying to get something to work. It is not clear to me if it needs patching to not.
Re: pylucene on Ubuntu 18.04
To be a little more specific, here's what happens with version 4.9.0 which I've had good luck with in the past. The system contains the following shared libraries. /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so It looks like LFLAGS in jcc/setup.py should find them. I'm on the linux2/X86_64 platform, using '/usr/lib/jvm/java-8-openjdk-amd64' for the JDK. JCC builds and installs without complaint, but fails at runtime. Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main mod_name, _Error) File "/usr/lib/python2.7/runpy.py", line 111, in _get_module_details __import__(mod_name) # Do not catch exceptions initializing package File "/usr/local/lib/python2.7/dist-packages/JCC-2.20-py2.7-linux-x86_64.egg/jcc/__init__.py", line 31, in from jcc import _jcc ImportError: libjvm.so: cannot open shared object file: No such file or directory
pylucene on Ubuntu 18.04
I'm having all sorts of trouble getting PyLucene to run on Ubuntu 18.04, which has openjdk-8, openjdk-11, python 2.7.15. Has anyone had success, and if so, with which version of pylucene? Thanks, Jeff
