[jira] [Resolved] (PYLUCENE-70) JCC --generate missing additional \ on windows

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-70?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-70.

Resolution: Fixed

fixed in rev 1916468.

> JCC --generate missing additional \ on windows
> --
>
> Key: PYLUCENE-70
> URL: https://issues.apache.org/jira/browse/PYLUCENE-70
> Project: PyLucene
>  Issue Type: Bug
> Environment: Windows11, conda python package
>Reporter: Petrus Hyvönen
>Priority: Blocker
> Attachments: issue_escape_package_dir.patch, jcc_python.patch
>
>
> The --generate seems to be missing double 
> in package_dir parameter on windows platform
> In tests at JCC conda build feedstock 
> ([https://github.com/conda-forge/jcc-feedstock/tree/main/recipe/test/java-example-test-parameters])
> --generates gives a setup.py with row:
> 
>package_dir={"test1": "build\test1"},
>  
> But should be:
>   package_dir={"test1": "build\\test1"},



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-70) JCC --generate missing additional \ on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827781#comment-17827781
 ] 

Andi Vajda commented on PYLUCENE-70:


Hi Petrus, thank you for the bug report and patch.
Please, test the attached jcc_python.patch file that handles more than just 
package_dir and switches to using r"strings" where \ chars are not a problem.
I don't have access to Windows, your help in verifying this patch is 
appreciated !

> JCC --generate missing additional \ on windows
> --
>
> Key: PYLUCENE-70
> URL: https://issues.apache.org/jira/browse/PYLUCENE-70
> Project: PyLucene
>  Issue Type: Bug
> Environment: Windows11, conda python package
>Reporter: Petrus Hyvönen
>Priority: Blocker
> Attachments: issue_escape_package_dir.patch, jcc_python.patch
>
>
> The --generate seems to be missing double 
> in package_dir parameter on windows platform
> In tests at JCC conda build feedstock 
> ([https://github.com/conda-forge/jcc-feedstock/tree/main/recipe/test/java-example-test-parameters])
> --generates gives a setup.py with row:
> 
>package_dir={"test1": "build\test1"},
>  
> But should be:
>   package_dir={"test1": "build\\test1"},



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (PYLUCENE-70) JCC --generate missing additional \ on windows

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-70?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda updated PYLUCENE-70:
---
Attachment: jcc_python.patch

> JCC --generate missing additional \ on windows
> --
>
> Key: PYLUCENE-70
> URL: https://issues.apache.org/jira/browse/PYLUCENE-70
> Project: PyLucene
>  Issue Type: Bug
> Environment: Windows11, conda python package
>Reporter: Petrus Hyvönen
>Priority: Blocker
> Attachments: issue_escape_package_dir.patch, jcc_python.patch
>
>
> The --generate seems to be missing double 
> in package_dir parameter on windows platform
> In tests at JCC conda build feedstock 
> ([https://github.com/conda-forge/jcc-feedstock/tree/main/recipe/test/java-example-test-parameters])
> --generates gives a setup.py with row:
> 
>package_dir={"test1": "build\test1"},
>  
> But should be:
>   package_dir={"test1": "build\\test1"},



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release PyLucene 9.10.0-rc1

[Andi Vajda]

Thank you all who voted.
Thank you Dawid and Mike for your PMC +1 votes as well.

This vote has passed !
Expect a release shortly...

Andi..

On Wed, 21 Feb 2024, Andi Vajda wrote:


The PyLucene 9.10.0 (rc1) release tracking the recent release of
Apache Lucene 9.10.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.10.0-rc1/

PyLucene 9.10.0 is built with JCC 3.14, included in these release artifacts.

Apart from the catch-up to Lucene 9.10.0, the other major new feature in this 
release candidate is that JCC can now generate a setup.py file instead of 
calling Setup() directly. This makes it possible to use modern Python 
packaging without falling afoul of "python setup.py install" being
deprecated. Setup.py itself is not deprecated, only some of its associated 
commands are; see [1] for more information about this.


In PyLucene's Makefile, there now is a new MODERN_PACKAGING variable, which 
can be set to true so that "python -m build" and "python -m pip install" are 
used for building and installing PyLucene.


JCC 3.14 supports Python 3.3 up to Python 3.12.
PyLucene may also be built with Python 2 but this configuration is no longer
tested.

Please vote to release these artifacts as PyLucene 9.10.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[1] https://packaging.python.org/en/latest/discussions/setup-py-deprecated/

Re: The future of the PyLucene project

[Andi Vajda]

So it does look like there are users of PyLucene who would like the project 
to continue, after all. As long as there is interest I'm happy to continue 
with it as well.


Thank you all who responded to this thread !

Andi..

On Wed, 28 Feb 2024, Andi Vajda wrote:



Hi PyLucene users and Lucene PMC,

A week ago, on Wednesday February 21st, I started a voting thread for 
qualifying a new PyLucene release candidate to catch-up with the recent 
Lucene 9.10.0 release and fix a bug in JCC.


Usually these voting threads get a couple of +1 for PyLucene users before 
getting votes from a couple of people on the Lucene PMC, always the same ones 
;-) Three PMC +1 votes -> a release can happen.


This time, crickets, the voting thread has been completely quiet.

If there are no PyLucene users anymore, maybe it's time to shut the project 
down ? Personally, I think that the "software value" in the project is all in 
JCC. PyLucene itself is 99% machine generated by JCC around Java Lucene.


Of course, having Java Lucene available that way from Python is pretty cool 
so I don't want diminish PyLucene's "usage value", but from a software 
engineering standpoint, the itch, if you prefer, all the cool stuff is done 
in JCC.


If the Lucene PMC agrees and no PyLucene users come forward, I propose the 
following:

 - shutdown the PyLucene project
 - fork JCC to my gitlab (https://gitlab.pyicu.org/main) where it can
   get the occasional fix or improvement before being released to PyPI.
   JCC has been distributed from PyPI forever,
 https://pypi.org/project/JCC/#history
   so JCC users shouldn't even notice this...

What do you all think ?
This message is not a vote, I'm just trying to gauge interest in PyLucene and 
JCC.


Andi..

ps: for those who have never heard of PyLucene, it is a sub-project of
   Apache Lucene hosted here:
 https://lucene.apache.org/pylucene/index.html
pps: for those who have never heard of JCC, it is a sub-project of PyLucene
 hosted here: https://lucene.apache.org/pylucene/jcc/index.html

Re: The future of the PyLucene project

[Andi Vajda]

On Wed, 28 Feb 2024, Erik Groeneveld LPV wrote:

I always followed new releases and checked the change log for both 
PyLucene and Lucene. I never felt entitled to vote however.


This seems to be a common misconception.
Everyone can vote on a release, everyone is entitled to.
It's just an Apache Rule that 3 PMC +1 votes are required to make a release.

Look at it this way: if a non-PMC user votes +1, it's a sign of interest.
If such a user votes -1, it's even more: a sign of participation and a 
non-binding veto to the release.
Sure, according to the rules, one can ignore it and go ahead with the 
release but what does that say to your user community ?


Of course anyone can vote !
Anyone interested in this project can and should vote !
If no one does, how do we know anyone cares ?

Andi..



I can still vote, but I think it would be more appropriate if Thijs does 
that.


Keep up the good work!
Erik

On Wed, Feb 28, 2024 at 21:08, Dawid Weiss <[dawid.we...@gmail.com](mailto:On Wed, Feb 
28, 2024 at 21:08, Dawid Weiss < wrote:


Hi Andi,

This time, crickets, the voting thread has been completely quiet.




For me - and it's not an excuse at all - you hit winter holidays, I'm
really sorry!


If the Lucene PMC agrees and no PyLucene users come forward, I propose the
following:
- shutdown the PyLucene project
- fork JCC to my gitlab (https://gitlab.pyicu.org/main) where it can
get the occasional fix or improvement before being released to PyPI.
JCC has been distributed from PyPI forever,
https://pypi.org/project/JCC/#history
so JCC users shouldn't even notice this...



I think open source is mostly about the community and folks coding together
for fun... And not many of us seem to be
able to help you with PyLucene development - I can't, for that matter,
because my Python is really limited.

Your plan sounds good to me. And you'd get more freedom from procedural
release
requirements at Apache too, which sounds like an added benefit?... :)

I also hope that, regardless of the status of PyLucene and JCC, you remain
with the Lucene project.

Dawid

--
Seecr is een kleine groep zeer ervaren full cycle software engineers. We
specialiseren ons in Linux, search en dataverwerking met de laatste
technieken. Wilt u weten meer weten? Kijk op seecr.nl .

The future of the PyLucene project

[Andi Vajda]

 Hi PyLucene users and Lucene PMC,

A week ago, on Wednesday February 21st, I started a voting thread for 
qualifying a new PyLucene release candidate to catch-up with the recent 
Lucene 9.10.0 release and fix a bug in JCC.


Usually these voting threads get a couple of +1 for PyLucene users before 
getting votes from a couple of people on the Lucene PMC, always the same 
ones ;-) Three PMC +1 votes -> a release can happen.


This time, crickets, the voting thread has been completely quiet.

If there are no PyLucene users anymore, maybe it's time to shut the project 
down ? Personally, I think that the "software value" in the project is all 
in JCC. PyLucene itself is 99% machine generated by JCC around Java Lucene.


Of course, having Java Lucene available that way from Python is pretty cool 
so I don't want diminish PyLucene's "usage value", but from a software 
engineering standpoint, the itch, if you prefer, all the cool stuff is done 
in JCC.


If the Lucene PMC agrees and no PyLucene users come forward, I propose the 
following:

  - shutdown the PyLucene project
  - fork JCC to my gitlab (https://gitlab.pyicu.org/main) where it can
get the occasional fix or improvement before being released to PyPI.
JCC has been distributed from PyPI forever,
  https://pypi.org/project/JCC/#history
so JCC users shouldn't even notice this...

What do you all think ?
This message is not a vote, I'm just trying to gauge interest in PyLucene 
and JCC.


Andi..

ps: for those who have never heard of PyLucene, it is a sub-project of
Apache Lucene hosted here:
  https://lucene.apache.org/pylucene/index.html
pps: for those who have never heard of JCC, it is a sub-project of PyLucene
  hosted here: https://lucene.apache.org/pylucene/jcc/index.html

[VOTE] Release PyLucene 9.10.0-rc1

[Andi Vajda]

The PyLucene 9.10.0 (rc1) release tracking the recent release of
Apache Lucene 9.10.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.10.0-rc1/

PyLucene 9.10.0 is built with JCC 3.14, included in these release artifacts.

Apart from the catch-up to Lucene 9.10.0, the other major new feature in 
this release candidate is that JCC can now generate a setup.py file instead 
of calling Setup() directly. This makes it possible to use modern Python 
packaging without falling afoul of "python setup.py install" being
deprecated. Setup.py itself is not deprecated, only some of its associated 
commands are; see [1] for more information about this.


In PyLucene's Makefile, there now is a new MODERN_PACKAGING variable, which 
can be set to true so that "python -m build" and "python -m pip install" are 
used for building and installing PyLucene.


JCC 3.14 supports Python 3.3 up to Python 3.12.
PyLucene may also be built with Python 2 but this configuration is no longer
tested.

Please vote to release these artifacts as PyLucene 9.10.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[1] https://packaging.python.org/en/latest/discussions/setup-py-deprecated/

[jira] [Commented] (PYLUCENE-65) Support the default java on debian in `setup.py`.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815420#comment-17815420
 ] 

Andi Vajda commented on PYLUCENE-65:


on my rather old debian install and also on ubuntu 22.04, I get these packages 
when I search for default-java

apt search default-java
Sorting... Done
Full Text Search... Done
libplexus-container-default-java/jammy 2.1.0-1 all
  Plexus Inversion-of-control Container

libplexus-container-default1.5-java/jammy 2.1.0-1 all
  Plexus Inversion-of-control Container (transitional package)

which java are you referring to ?
Does the fix you propose apply only to Debian and its derivatives ?

> Support the default java on debian in `setup.py`.
> -
>
> Key: PYLUCENE-65
> URL: https://issues.apache.org/jira/browse/PYLUCENE-65
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: A. Coady
>Priority: Major
>
> On debian, the `default-java` package does not have `jre/lib/amd64` in its 
> path, so it breaks the `linux/x86_64` build. The `temurin` flags have the 
> correct paths, so one easy fix would be to change the [temurin 
> check|https://svn.apache.org/viewvc/lucene/pylucene/trunk/jcc/setup.py?revision=1900087=markup#l194]
>  to:
> {code:python}
>     if 'temurin' in JDK['linux'] or 'default' in JDK['linux']:
> {code}
> That would also support `linux/aarch64` without any further changes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-69) Linking libjvm seems to prevent PyPi upload

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815416#comment-17815416
 ] 

Andi Vajda commented on PYLUCENE-69:


Try with the latest but I doubt you'll be able to build a wheel that includes 
the JVM.

> Linking libjvm seems to prevent PyPi upload
> ---
>
> Key: PYLUCENE-69
> URL: https://issues.apache.org/jira/browse/PYLUCENE-69
> Project: PyLucene
>  Issue Type: Question
>Reporter: Clément Jonglez
>Priority: Major
>
> As mentioned in 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] , I am 
> trying to package the Orekit Python wrapper from [~petrush] (which uses JCC) 
> to PyPi.
> I used the --wheel option to compile a wheel and not an egg, and I tried to 
> upload the wheel to PyPi.
> But PyPi refuses my wheel with the answer :
> {noformat}
> Binary wheel 'orekit-11.3.3-cp312-cp312-linux_x86_64.whl' has an unsupported 
> platform tag 'linux_x86_64'. {noformat}
> After reading about why this error occurs ( 
> [https://peps.python.org/pep-0513/#rationale] )  , I tried to convert the 
> wheel to a manylinux wheel by using:
>  
> {code:java}
> auditwheel repair dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned the following error:
>  
> {noformat}
> auditwheel: error: cannot repair 
> "dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl" to "manylinux_2_5_x86_64" 
> ABI because of the presence of too-recent versioned symbols. You'll need to 
> compile the wheel on an older toolchain.{noformat}
>  
> I then ran the following command to get more information about which symbols 
> are problematic in the wheel:
>  
> {code:java}
> auditwheel-symbols --manylinux 2_34 
> dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned:
>  
> {noformat}
> orekit/_orekit.cpython-312-x86_64-linux-gnu.so is not manylinux_2_34 
> compliant because it links the following forbidden libraries:
> libjvm.so{noformat}
>  
>  
> I tried removing -ljvm from the LFLAGS list in jcc's config.py, but as 
> expected the Python program then fails on starting the JVM:
> {noformat}
> Traceback (most recent call last):
>   File "/media/ssd/git/orekit_python_artifacts/test/AbstractDetectorTest.py", 
> line 3, in 
>     import orekit
>   File 
> "/home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/__init__.py",
>  line 7, in 
>     from . import _orekit
> ImportError: 
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/_orekit.cpython-312-x86_64-linux-gnu.so:
>  undefined symbol: JNI_GetDefaultJavaVMInitArgs{noformat}
>  
> I don't have any more clues... Alternatively, I could try to package the 
> Orekit Python wrapper as a source distribution (using 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-27] and 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] ), but 
> this would mean that a user would need to wait approx. 10 minutes for the 
> wheel to compile when running pip install...
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (PYLUCENE-68) setup.py install and easy_install command are deprecated

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-68.

Resolution: Fixed

Fixed in HEAD.

> setup.py install and easy_install command are deprecated
> 
>
> Key: PYLUCENE-68
> URL: https://issues.apache.org/jira/browse/PYLUCENE-68
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Clément Jonglez
>Priority: Major
>
> When compiling with jcc to create a wheel, I get the following warnings (see 
> below).
> This is still working but eventually the `setup.py` installation and 
> `easy_install` command will be removed.
> It would be nice to adapt JCC to modern pypa build tools, but I don't know 
> enough about JCC to do these changes.
>  
> {noformat}
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/dist.py:947:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
>   self.initialize_options()
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  EasyInstallDeprecationWarning: easy_install command is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` and ``easy_install``.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://github.com/pypa/setuptools/issues/917 for details.
>         
> 
> !!
>   self.initialize_options()
>  
> {noformat}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (PYLUCENE-68) setup.py install and easy_install command are deprecated

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815414#comment-17815414
 ] 

Andi Vajda edited comment on PYLUCENE-68 at 2/7/24 7:51 PM:


I looked into this issue a bit more now and setup.py is _not_ deprecated, see 
this 
[doc|https://packaging.python.org/en/latest/discussions/setup-py-deprecated/] 
for more about this topic.
What is deprecated are _some_ of the commands usually passed to setup.py, such 
as install, which JCC uses, in some cases, when it invokes setup() directly (at 
the bottom of python.py).
I fixed this now by adding support for a new --generate flag that supercedes 
--build and other such flags and, instead of calling setup() JCC produces a 
setup.py file for the extension. This setup.py file can then be used with 
modern python packaging tools such as build and pip.
The pylucene build is being switched to this and the process to build PyLucene 
is then:
  - build and install jcc using the modern tools:
  - python -m build
  - python -m pip install --force

  - build pylucene:
  - set MODERN_PACKAGING to true in Makefile
  - make all
 which runs:
  - jcc is invoked with --generate
  - python -m build -nw
  - python -m pip install --force



was (Author: vajda):
I looked into this issue a bit more now and setup.py is _not_ deprecated, see 
this 
[doc|https://packaging.python.org/en/latest/discussions/setup-py-deprecated/] 
for more about this topic.
What is deprecated are _some_ of the commands usually passed to setup.py, such 
as install, which JCC uses, in some cases, when it invokes setup() directly (at 
the bottom of python.py).
I fixed this now by adding support for a new --generate flag that supercedes 
--build and other such flags and, instead of calling setup() JCC produces a 
setup.py file for the extension. This setup.py file can then be used with 
modern python packaging tools such as build and pip.
The pylucene build is being switched to this and the process to build PyLucene 
is then:
  - build and install jcc using the modern tools:
  - python -m build
  - python -m pip install --force
  - build pylucene:
  - set MODERN_PACKAGING to true in Makefile
  - make all
 which runs:
  - jcc is invoked with --generate
  - python -m build -nw
  - python -m pip install --force


> setup.py install and easy_install command are deprecated
> 
>
> Key: PYLUCENE-68
> URL: https://issues.apache.org/jira/browse/PYLUCENE-68
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Clément Jonglez
>Priority: Major
>
> When compiling with jcc to create a wheel, I get the following warnings (see 
> below).
> This is still working but eventually the `setup.py` installation and 
> `easy_install` command will be removed.
> It would be nice to adapt JCC to modern pypa build tools, but I don't know 
> enough about JCC to do these changes.
>  
> {noformat}
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/dist.py:947:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
>   self.initialize_options()
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  EasyInstallDeprecationWarning: easy_install command is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` and ``easy_install``.
>      

[jira] [Commented] (PYLUCENE-68) setup.py install and easy_install command are deprecated

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815414#comment-17815414
 ] 

Andi Vajda commented on PYLUCENE-68:


I looked into this issue a bit more now and setup.py is _not_ deprecated, see 
this 
[doc|https://packaging.python.org/en/latest/discussions/setup-py-deprecated/] 
for more about this topic.
What is deprecated are _some_ of the commands usually passed to setup.py, such 
as install, which JCC uses, in some cases, when it invokes setup() directly (at 
the bottom of python.py).
I fixed this now by adding support for a new --generate flag that supercedes 
--build and other such flags and, instead of calling setup() JCC produces a 
setup.py file for the extension. This setup.py file can then be used with 
modern python packaging tools such as build and pip.
The pylucene build is being switched to this and the process to build PyLucene 
is then:
  - build and install jcc using the modern tools:
  - python -m build
  - python -m pip install --force
  - build pylucene:
  - set MODERN_PACKAGING to true in Makefile
  - make all
 which runs:
  - jcc is invoked with --generate
  - python -m build -nw
  - python -m pip install --force


> setup.py install and easy_install command are deprecated
> 
>
> Key: PYLUCENE-68
> URL: https://issues.apache.org/jira/browse/PYLUCENE-68
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Clément Jonglez
>Priority: Major
>
> When compiling with jcc to create a wheel, I get the following warnings (see 
> below).
> This is still working but eventually the `setup.py` installation and 
> `easy_install` command will be removed.
> It would be nice to adapt JCC to modern pypa build tools, but I don't know 
> enough about JCC to do these changes.
>  
> {noformat}
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/dist.py:947:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
>   self.initialize_options()
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  EasyInstallDeprecationWarning: easy_install command is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` and ``easy_install``.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://github.com/pypa/setuptools/issues/917 for details.
>         
> 
> !!
>   self.initialize_options()
>  
> {noformat}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Fwd: Where to report issues with lucene/pylucene

[Andi Vajda]

Begin forwarded message:

> From: Andi Vajda 
> Date: January 8, 2024 at 15:59:29 GMT+1
> To: d...@lucene.apache.org
> Subject: Re: Where to report issues with lucene/pylucene
> Reply-To: d...@lucene.apache.org
> 
> 
>> On Sun, 7 Jan 2024, Bob Kline wrote:
>> 
>> On Sun, Jan 7, 2024 at 7:10 AM Bob Kline  wrote:
>> 
>> I figured out how to get past all the problems I ran into, and was finally
>> able to get the package built and installed. Afterwards I captured notes on
>> the process so if I (or a colleague) needs to do this again, we'll have a
>> head start. I'm sharing these notes (attached) in case any of the
>> information might suggest improvements to the instructions page
>> <https://lucene.apache.org/pylucene/install.html>. Feel free to use or
>> ignore any of these notes as you think appropriate.
> 
> Thank you for your notes. Replies inline:
> 
>> Skip over the "For the Impatient Ones" section. Those steps won't work
>> because the Makefile must be edited first.
> 
> Well, the steps said to edit setup.py and Makefile to match your environment 
> except that the markup on the site was bogus for those lines and these 
> instructions were not rendered :-(
> I fixed that now: https://lucene.apache.org/pylucene/install.html
> 
>> The build will not be able to use the Ubuntu distro's jcc, because it's
>> not built to support the --shared option.
> 
> I wasn't aware that Ubuntu has a prebuilt package for JCC now. If it is not 
> built with --shared then you can't use shared mode for PyLucene. If all you 
> need/want is build PyLucene and no other wrappers later around other JARs for 
> use in the same VM, then you can skip shared mode and drop --shared from the 
> JCC invocations.
> 
>> The makefile needs help finding the JDK libraries.
> 
> This shouldn't be necessary as JCC, when setup.py is edited as required, will 
> populate that information.
> 
>> sudo make test
> 
> I would stronly recommend using a python virtual environment (venv) so that 
> you don't pollute the system python with your work. This doesn't require root 
> nor later careful cleanup of the system python installation should something 
> go wrong. Yes, 20 years ago, when PyLucene/JCC were started, venv wasn't as 
> much of a thing but nowadays it's very good practice.
> 
>> All of the tests pass except one:
> 
> I wonder what version of PyICU you have installed. I just verified that tests 
> do all pass if you do _not_ have PyICU installed.
> 
> Andi..
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (PYLUCENE-31) JCC Parallel/Multiprocess Compilation + Caching

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-31.
--
Resolution: Fixed

distutils is on its way out with Python 3.12

> JCC Parallel/Multiprocess Compilation + Caching
> ---
>
> Key: PYLUCENE-31
> URL: https://issues.apache.org/jira/browse/PYLUCENE-31
> Project: PyLucene
>  Issue Type: Improvement
> Environment: Linux 3.11.0-19-generic #33-Ubuntu SMP x86_64 GNU/Linux
>Reporter: Lee Skillen
>Priority: Minor
>  Labels: build, cache, ccache, distutils, jcc, parallel
> Attachments: feature-parallel-build.patch
>
>
> JCC utilises distutils.Extension() in order to build JCC itself and the 
> packages that it generates for Java wrapping - Unfortunately distutils 
> performs its build sequentially and doesn't take advantage of any additional 
> free cores for parallel building.  As discussed on the list this is likely a 
> design decision due to potential issues that may arise when building projects 
> with awkward, cyclic or recursive dependencies.
> These issues shouldn't appear within JCC-based projects because of the 
> generative nature of the build; i.e. all dependencies are resolved and 
> generated prior to building, and the build process itself is about 
> compilation and construction of the wrapper alone, of which the wrapper files 
> are contained to a sequence of flattened compilation units.
> Enabling this requires monkey patching of distutils, which was also discussed 
> on the list as being a potential source of issues, although we feel that the 
> risk is likely lower than the current setuptools patching utilised.  This 
> would be optional functionality that is also only enabled if the 
> monkey-patching succeeds.  Distutils itself is also part of the standard 
> library and might be less susceptible to change than setuptools, and the area 
> of code monkey patched almost hasn't changed since 2002 (see: 
> http://hg.python.org/cpython/file/tip/Lib/distutils/ccompiler.py).
> In addition to the distutils changes this patch also includes changes to the 
> wrapper class generation to make it more cache friendly, with the target 
> being that no changes in the wrapped code equals no changes in the wrapper 
> code.  So any changes that minimally change the wrapped code mean that with a 
> tool such as ccache the rebuild time would be significantly reduced (almost 
> to a nth, where n is the number of files and only one has changed).
> Obviously the maintainers would have to assess this risk and decide whether 
> they would like to accept the patch or not.  Code has only been tested on 
> Linux with Python 2.7.5 but should gracefully fail and prevent 
> parallelisation if one of the requirements hasn't been met (not on linux, no 
> multiprocessing support, or monkey patching somehow fails).  The change to 
> caching should still benefit everyone regardless.
> Please note that an additional dependency on orderedset has been added to 
> achieve the more deterministic ordering - This may not be desirable (i.e. 
> another package might be desired, such as ordered-set, or the code might be 
> inlined into the package instead), as per maintainer comments.
> --- [following repeated from mailing list] ---
> Performance Statistics :-
> The following are some quick and dirty statistics for building the jcc 
> pylucene itself (incl. java lucene which accounts for about 30-ish seconds 
> upfront) - The JCC files are split using --files 8, and each build is 
> preceded with a make clean:
> Serial (unpatched):
> real5m1.502s
> user5m22.887s
> sys 0m7.749s
> Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs):
> real1m37.382s
> user7m16.658s
> sys 0m8.697s
> Furthermore, some additional changes were made to the wrapped file generation 
> to make the generated code more ccache friendly (additional deterministic 
> sorting for methods and some usage of an ordered set).  With these in place 
> and the CC and CCACHE_COMPILERCHECK environment variables set to "ccache gcc" 
> and "content" respectively, and ensuring ccache is installed, subsequent 
> compilation time is reduced again as follows:
> Parallel (patched, 4 physical cores, 8 hyperthreads, 8 parallel jobs, ccache 
> enabled):
> real0m43.051s
> user1m10.392s
> sys 0m4.547s
> This was a run in which nothing changed between runs, so a realistic run in 
> which changes occur it'll be a figure between 0m43.051s and 1m37.382s, 
> depending on how drastic the change was. If many

[jira] [Closed] (PYLUCENE-8) lucene.JCC_VERSION isn't set properly on Windows

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-8.
-
Resolution: Fixed

> lucene.JCC_VERSION isn't set properly on Windows
> 
>
> Key: PYLUCENE-8
> URL: https://issues.apache.org/jira/browse/PYLUCENE-8
> Project: PyLucene
>  Issue Type: Bug
> Environment: Windows XP, MinGW gcc 4.5, Python 2.6.6, PyLucene 3.0.2
>Reporter: Bill Janssen
>Priority: Minor
>
> After installing 3.0.2 on Win XP with MinGW/msys, I see the following:
> Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)] 
> on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import lucene
> >>> lucene.initVM(classpath=lucene.CLASSPATH)
> 
> >>> lucene.JCC_VERSION
> 'JCC_VER'
> >>> quit()



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Closed] (PYLUCENE-18) Version 3.6.0 lacks readme

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-18.
--
Resolution: Fixed

> Version 3.6.0 lacks readme
> --
>
> Key: PYLUCENE-18
> URL: https://issues.apache.org/jira/browse/PYLUCENE-18
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Dion Gillard
>Priority: Minor
>
> Hi,
> It looks like pylucene version 3.6.0 lacks CHANGES in tarball. Also README 
> has followed content:
> {code}
>   Please see doc/documentation/readme.html
> {code}
> At the same there is no docs directory in tarball at all.
> These files are important for linux distributions (like Debian) where they 
> are usually installed to /usr/share/doc



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Closed] (PYLUCENE-64) ModuleNotFoundError: No module named 'lucene' when intalling PyLucene

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-64.
--
Resolution: Fixed

seems to work fine

> ModuleNotFoundError: No module named 'lucene' when intalling PyLucene
> -
>
> Key: PYLUCENE-64
> URL: https://issues.apache.org/jira/browse/PYLUCENE-64
> Project: PyLucene
>  Issue Type: Bug
>Reporter: anis amirouche
>Priority: Major
>
> I'm trying to install Pylucene 8.11.0 on mac os. the jcc is built and 
> installed successfully, but from pylucene folder when running make test i'm 
> encountering the following error : ModuleNotFoundError: No module named 
> 'lucene', please note the make command is successfull.
> please can anyone help me with solving this error. I don't even understand 
> what i'm doing wrong, it's like lucene is not installed, and i'm having this 
> error with all the files of pylucene. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Closed] (PYLUCENE-56) Can't build JCC on Mac

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-56.
--
Resolution: Fixed

building JCC on M2 mac works fine

> Can't build JCC on Mac
> --
>
> Key: PYLUCENE-56
> URL: https://issues.apache.org/jira/browse/PYLUCENE-56
> Project: PyLucene
>  Issue Type: Bug
> Environment: MacOSX 10.15.7, Intel Core i7, Python 3.8.2, gcc 
> (Homebrew GCC 10.2.0_3) 10.2.0
> following the instructions here:
> http://lucene.apache.org/pylucene/jcc/install.html
> svn co https://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc jcc
> Checked out revision 1886645.
>Reporter: Clem Wang
>Priority: Major
>  Labels: build
>
> This is puzzling to me, as I can't figure out how the failing gcc command 
> line gets its arguments (mostly).  I found one problem in the setup.py, but I 
> can't find the error causing strings anywhere in the files in the jcc 
> directory of sub directory.
>  
> Steps:
>  
> {code:java}
> svn co https://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc jcc
> Checked out revision 1886645.
> cd jcc
> python setup.py build
>  
> {code}
> ...
> /opt/local/bin/gcc -Wno-unused-result -Wsign-compare -Wunreachable-code 
> -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall 
> {color:#ff}*-iwithsysroot/*{color}System/Library/Frameworks/System.framework/PrivateHeaders
>  
> {color:#ff}*-iwithsysroot/*{color}Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers
>  {color:#ff}*-arch arm64*{color} -arch x86_64 
> -I/usr/local/opt/libomp/include -Xpreprocessor -fopenmp -dynamiclib 
> -D_jcc_lib -DJCC_VER="3.8" 
> -I/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/include 
> -I/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/include/darwin
>  -I_jcc3 -Ijcc3/sources -I/Users/cwang/3.7/include 
> -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8
>  -c jcc3/sources/jcc.cpp -o 
> build/temp.macosx-10.14.6-x86_64-3.8/jcc3/sources/jcc.o -DPYTHON 
> -fno-strict-aliasing -Wno-write-strings -mmacosx-version-min=10.9 -std=c++11  
> {color:#ff}*-stdlib=libc++*{color}
>  
> which generates 4 errors due to the parts marked above in red bold:
>  
> *gcc:* *error:* this compiler does not support arm64
> *gcc:* *error:* unrecognized command-line option 
> '*-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders*'
> *gcc:* *error:* unrecognized command-line option 
> '*-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers*'
> *gcc:* *error:* unrecognized command-line option '*-stdlib=libc++*'
> error: command '/opt/local/bin/gcc' failed with exit status 1
>  
> Obviously, the contradictory 
> *-arch arm64*
> needs to be removed but I can't find arm64 anywhere.
>  
> The unnecessary
> *-stdlib=libc++*
>  
> can be removed from setup.py:
> {code:java}
> CFLAGS = {
>  'darwin': ['-fno-strict-aliasing', '-Wno-write-strings',
>  '-mmacosx-version-min=10.9', '-std=c++11', '-stdlib=libc++'],{code}
>  
> After poking around, I figured out that gcc uses
> {code:java}
> -I {code}
> not
> {code:java}
> -i {code}
> for includes.
>  
> Making these modifications (and adding
> {code:java}
> -Wno-attributes{code}
> to remove warnings)
>  
> I came up with this line that does successfully compile without errors:
> /opt/local/bin/gcc -Wno-attributes -Wno-unused-result -Wsign-compare 
> -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall 
> -I/System/Library/Frameworks/System.framework/PrivateHeaders 
> -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers
>  -arch x86_64 -I/usr/local/opt/libomp/include -Xpreprocessor -fopenmp 
> -dynamiclib -D_jcc_lib -DJCC_VER="3.8" 
> -I/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/include 
> -I/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/include/darwin
>  -I_jcc3 -Ijcc3/sources -I/Users/cwang/3.7/include 
> -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8
>  -c jcc3/sources/jcc.cpp -o 
> build/temp.macosx-10.14.6-x86_64-3.8/jcc3/sources/jcc.o -DPYTHON 
> -fno-strict-aliasing -Wno-write-strings -mmacosx-version-min=10.9 -std=c++11
>  
> But other than removing  *'-stdlib=libc++'*  from the setup.py file I have no 
> idea how to modify things to fix the compile errors by the line generated 
> some how by setup.py



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-27) JCC should be able to create sdist archives

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-27?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785231#comment-17785231
 ] 

Andi Vajda commented on PYLUCENE-27:


Yes, with the upcoming demise of setuptools/distutils, this issue may get 
resolution.
Instead of invoking setuptools.setup() produce something that pypa/build can 
use as input, some yaml maybe ?

> JCC should be able to create sdist archives
> ---
>
> Key: PYLUCENE-27
> URL: https://issues.apache.org/jira/browse/PYLUCENE-27
> Project: PyLucene
>  Issue Type: Wish
> Environment: jcc-svn-head
>Reporter: Martin Scherer
>Priority: Major
> Attachments: createSetupScript.patch
>
>
> I was not able to create a complete (in terms one is able to compile and 
> install the desired wrapper) source distribution.
> I've tried following calls:
>   python -m jcc --jar foo  --egg-info --extra-setup-arg sdist
> and
>  python -m jcc --jar foo --extra-setup-arg sdist
> Both create archives only containing the egg-info and setup.py but no source 
> code at all.
> I really need this feature for my testing environment with tox, since this 
> heavily depends on the sdist feature.
> thanks,
> best,
> Martin



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (PYLUCENE-69) Linking libjvm seems to prevent PyPi upload

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785230#comment-17785230
 ] 

Andi Vajda edited comment on PYLUCENE-69 at 11/11/23 10:54 PM:
---

JCC is a python/c++ code generator for wrapping Java libraries using JNI. Any 
JCC-built Python extension embeds a Java Virtual Machine. If PyPI doesn't allow 
you to upload libjvm.so then you cannot ship JCC-produced binaries on PyPI. 
Only source distributions are then possible. And yes, building binaries can 
take time and requires that at least a C++ compiler be present as well. 
Knowledge how to operate a C++ compiler and linker is also useful.
I think you should reset your expectation a bit here.
Even _if_ you could do this, the wheel would be enormous: the number of Python 
* C++ * Java * OS versions possible makes for a lot of combos. There is a 
reason I don't ever ship binaries, it's just too much (!)


was (Author: vajda):
JCC is a python/c++ code generator for wrapping Java libraries using JNI. Any 
JCC-built Python extension embeds a Java Virtual Machine. If PyPI doesn't allow 
you to upload libjvm.so then you cannot ship JCC-produced binaries on PyPI. 
Only source distributions are then possible. And yes, building binaries can 
take time and requires that at least a C++ compiler be present as well. 
Knowledge how to operate a C++ compiler and linker is also useful.
I think you should reset your expectation a bit here.
Even _if_ could do this, the wheel would be enormous: the number of Python * 
C++ * Java * OS versions possible makes for a lot of combos. There is a reason 
I don't ever ship binaries, it's just too much (!)

> Linking libjvm seems to prevent PyPi upload
> ---
>
> Key: PYLUCENE-69
> URL: https://issues.apache.org/jira/browse/PYLUCENE-69
> Project: PyLucene
>  Issue Type: Question
>Reporter: Clément Jonglez
>Priority: Major
>
> As mentioned in 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] , I am 
> trying to package the Orekit Python wrapper from [~petrush] (which uses JCC) 
> to PyPi.
> I used the --wheel option to compile a wheel and not an egg, and I tried to 
> upload the wheel to PyPi.
> But PyPi refuses my wheel with the answer :
> {noformat}
> Binary wheel 'orekit-11.3.3-cp312-cp312-linux_x86_64.whl' has an unsupported 
> platform tag 'linux_x86_64'. {noformat}
> After reading about why this error occurs ( 
> [https://peps.python.org/pep-0513/#rationale] )  , I tried to convert the 
> wheel to a manylinux wheel by using:
>  
> {code:java}
> auditwheel repair dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned the following error:
>  
> {noformat}
> auditwheel: error: cannot repair 
> "dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl" to "manylinux_2_5_x86_64" 
> ABI because of the presence of too-recent versioned symbols. You'll need to 
> compile the wheel on an older toolchain.{noformat}
>  
> I then ran the following command to get more information about which symbols 
> are problematic in the wheel:
>  
> {code:java}
> auditwheel-symbols --manylinux 2_34 
> dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned:
>  
> {noformat}
> orekit/_orekit.cpython-312-x86_64-linux-gnu.so is not manylinux_2_34 
> compliant because it links the following forbidden libraries:
> libjvm.so{noformat}
>  
>  
> I tried removing -ljvm from the LFLAGS list in jcc's config.py, but as 
> expected the Python program then fails on starting the JVM:
> {noformat}
> Traceback (most recent call last):
>   File "/media/ssd/git/orekit_python_artifacts/test/AbstractDetectorTest.py", 
> line 3, in 
>     import orekit
>   File 
> "/home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/__init__.py",
>  line 7, in 
>     from . import _orekit
> ImportError: 
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/_orekit.cpython-312-x86_64-linux-gnu.so:
>  undefined symbol: JNI_GetDefaultJavaVMInitArgs{noformat}
>  
> I don't have any more clues... Alternatively, I could try to package the 
> Orekit Python wrapper as a source distribution (using 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-27] and 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] ), but 
> this would mean that a user would need to wait approx. 10 minutes for the 
> wheel to compile when running pip install...
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-69) Linking libjvm seems to prevent PyPi upload

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785230#comment-17785230
 ] 

Andi Vajda commented on PYLUCENE-69:


JCC is a python/c++ code generator for wrapping Java libraries using JNI. Any 
JCC-built Python extension embeds a Java Virtual Machine. If PyPI doesn't allow 
you to upload libjvm.so then you cannot ship JCC-produced binaries on PyPI. 
Only source distributions are then possible. And yes, building binaries can 
take time and requires that at least a C++ compiler be present as well. 
Knowledge how to operate a C++ compiler and linker is also useful.
I think you should reset your expectation a bit here.
Even _if_ could do this, the wheel would be enormous: the number of Python * 
C++ * Java * OS versions possible makes for a lot of combos. There is a reason 
I don't ever ship binaries, it's just too much (!)

> Linking libjvm seems to prevent PyPi upload
> ---
>
> Key: PYLUCENE-69
> URL: https://issues.apache.org/jira/browse/PYLUCENE-69
> Project: PyLucene
>  Issue Type: Question
>Reporter: Clément Jonglez
>Priority: Major
>
> As mentioned in 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] , I am 
> trying to package the Orekit Python wrapper from [~petrush] (which uses JCC) 
> to PyPi.
> I used the --wheel option to compile a wheel and not an egg, and I tried to 
> upload the wheel to PyPi.
> But PyPi refuses my wheel with the answer :
> {noformat}
> Binary wheel 'orekit-11.3.3-cp312-cp312-linux_x86_64.whl' has an unsupported 
> platform tag 'linux_x86_64'. {noformat}
> After reading about why this error occurs ( 
> [https://peps.python.org/pep-0513/#rationale] )  , I tried to convert the 
> wheel to a manylinux wheel by using:
>  
> {code:java}
> auditwheel repair dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned the following error:
>  
> {noformat}
> auditwheel: error: cannot repair 
> "dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl" to "manylinux_2_5_x86_64" 
> ABI because of the presence of too-recent versioned symbols. You'll need to 
> compile the wheel on an older toolchain.{noformat}
>  
> I then ran the following command to get more information about which symbols 
> are problematic in the wheel:
>  
> {code:java}
> auditwheel-symbols --manylinux 2_34 
> dist/orekit-11.3.3-cp312-cp312-linux_x86_64.whl{code}
>  
> Which returned:
>  
> {noformat}
> orekit/_orekit.cpython-312-x86_64-linux-gnu.so is not manylinux_2_34 
> compliant because it links the following forbidden libraries:
> libjvm.so{noformat}
>  
>  
> I tried removing -ljvm from the LFLAGS list in jcc's config.py, but as 
> expected the Python program then fails on starting the JVM:
> {noformat}
> Traceback (most recent call last):
>   File "/media/ssd/git/orekit_python_artifacts/test/AbstractDetectorTest.py", 
> line 3, in 
>     import orekit
>   File 
> "/home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/__init__.py",
>  line 7, in 
>     from . import _orekit
> ImportError: 
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/orekit-12.0-py3.12-linux-x86_64.egg/orekit/_orekit.cpython-312-x86_64-linux-gnu.so:
>  undefined symbol: JNI_GetDefaultJavaVMInitArgs{noformat}
>  
> I don't have any more clues... Alternatively, I could try to package the 
> Orekit Python wrapper as a source distribution (using 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-27] and 
> [https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-68] ), but 
> this would mean that a user would need to wait approx. 10 minutes for the 
> wheel to compile when running pip install...
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-68) setup.py install and easy_install command are deprecated

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785219#comment-17785219
 ] 

Andi Vajda commented on PYLUCENE-68:


I was afraid of that...
I did try to build jcc with pypa/build and it seems to work just fine:
{code:java}
$ cd jcc
$ python3 -m build
$ python3 -m pip install dist/JCC-3.13-cp312-cp312-macosx_13_0_arm64.whl{code}
worked fine.
Then, I get the warnings you reported when running "jcc ... --install" from 
pylucene's Makefile are due to jcc's calling setuptools.setup() function to 
build the extension it generated.
Do you know of a migration guide from setuptools.setup() to calling similar 
stuff in pypa/build ?

> setup.py install and easy_install command are deprecated
> 
>
> Key: PYLUCENE-68
> URL: https://issues.apache.org/jira/browse/PYLUCENE-68
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Clément Jonglez
>Priority: Major
>
> When compiling with jcc to create a wheel, I get the following warnings (see 
> below).
> This is still working but eventually the `setup.py` installation and 
> `easy_install` command will be removed.
> It would be nice to adapt JCC to modern pypa build tools, but I don't know 
> enough about JCC to do these changes.
>  
> {noformat}
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/dist.py:947:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  SetuptoolsDeprecationWarning: setup.py install is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` directly.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html 
> for details.
>         
> 
> !!
>   self.initialize_options()
> /home/yzokras/Documents/orekit-pip/orekit312/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66:
>  EasyInstallDeprecationWarning: easy_install command is deprecated.
> !!
>         
> 
>         Please avoid running ``setup.py`` and ``easy_install``.
>         Instead, use pypa/build, pypa/installer or other
>         standards-based tools.
>         See https://github.com/pypa/setuptools/issues/917 for details.
>         
> 
> !!
>   self.initialize_options()
>  
> {noformat}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release PyLucene 9.7.0-rc1

[Andi Vajda]

This vote has passed !
Thank you everyone who voted, release artifacts should be available shortly...

Andi..

> On Jul 12, 2023, at 21:45, Michael McCandless  
> wrote:
> 
> +1
> 
> I ran the same exciting smoke test -- indexing first 100K enwiki docs,
> running a few political searches, force merging, searching again.
> Everything ran fine!
> 
> Arch Linux kernel 6.3.2, Java 17.0.7+7, Python 3.11.3.
> 
> Sorry for the delay!
> 
> Mike
> 
>> On Sun, Jul 9, 2023 at 3:28 PM Dawid Weiss  wrote:
>> 
>> 
>> +1 to release. Thanks Andi.
>> 
>>> On Thu, Jul 6, 2023 at 9:47 AM Andi Vajda  wrote:
>>> 
>>> 
>>> The PyLucene 9.7.0 (rc1) release tracking the recent release of
>>> Apache Lucene 9.7.0 is ready.
>>> 
>>> A release candidate is available from:
>>>https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.7.0-rc1/
>>> 
>>> PyLucene 9.7.0 is built with JCC 3.13, included in these release
>>> artifacts.
>>> 
>>> JCC 3.13 supports Python 3.3 up to Python 3.11.
>>> PyLucene may also be built with Python 2 but this configuration is no
>>> longer
>>> tested.
>>> 
>>> Please vote to release these artifacts as PyLucene 9.7.0.
>>> Anyone interested in this release can and should vote !
>>> 
>>> Thanks !
>>> 
>>> Andi..
>>> 
>>> ps: the KEYS file for PyLucene release signing is at:
>>> https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
>>> https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS
>>> 
>>> pps: here is my +1
>>> 
>> --
> Mike McCandless
> 
> http://blog.mikemccandless.com

Re: Unable to install Pylucene

[Andi Vajda]

> On Jul 8, 2023, at 12:58, Subinay Adhikary  wrote:
> 
> 
> Dear sir,
>  I encountered difficulties while attempting to install 
> Pylucene (version 8.10.0) on my Linux machine. Despite having installed JCC, 
> I encountered an unresolved error even after multiple attempts. I have 
> attached a screenshot also.

Screenshots don't make it to this list. 

Andi..

> Thanks and regards,
> Subinay Adhikary

Re: New release for PyLucene?

[Andi Vajda]

On Wed, 5 Jul 2023, Benjamin Trent wrote:


Lucene 9.7 was just released and contains multiple desirable improvements:
https://lucene.apache.org/core/9_7_0/changes/Changes.html

Could we kick off a new release for PyLucene?


We prepared a 9.7.0 release candidate here [1] and kicked off a release 
vote thread. Please vote !


Thanks !

Andi..

[1] https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.7.0-rc1/

[VOTE] Release PyLucene 9.7.0-rc1

[Andi Vajda]

The PyLucene 9.7.0 (rc1) release tracking the recent release of
Apache Lucene 9.7.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.7.0-rc1/

PyLucene 9.7.0 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2 but this configuration is no longer
tested.

Please vote to release these artifacts as PyLucene 9.7.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Commented] (PYLUCENE-28) JCC reuses JVM instances in impl, if compile() is called twice.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736851#comment-17736851
 ] 

Andi Vajda commented on PYLUCENE-28:


I see 2 things:

  - if a constructor takes no arguments, like 
org.apache.lucene.document.Document(), then anything passed to it is ignored.
  - all constructor wrappers have a kwds parameter but it's ignored.

Both could be fixed at the cost of extra code that raises an error if kwds is 
not empty.
Also, extra code would have to be generated for checking that args is empty for 
contructores not taking any args.
Is it worth it ?

> JCC reuses JVM instances in impl, if compile() is called twice.
> ---
>
> Key: PYLUCENE-28
> URL: https://issues.apache.org/jira/browse/PYLUCENE-28
> Project: PyLucene
>  Issue Type: Improvement
> Environment: jcc-svn-head (2.18-pre)
>Reporter: Martin Scherer
>Priority: Blocker
>  Labels: patch
> Attachments: jvm_instance_check.patch
>
>
> If you import jcc.cpp to call compile yourself (a wrapped setup.py script to 
> generate a wrapper on the fly, which seems to be a common use case), the 
> current version complains about the JVM already being initialized before.
> The patch first checks for a running instance and creates one, if none is 
> being found.
> For myself, this is a blocker, since it raises otherwise.
> Best, 
> Martin



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release PyLucene 9.6.0-rc1

[Andi Vajda]

On Fri, 2 Jun 2023, Dawid Weiss wrote:


Late, but +1 to release.

Thank you Andi!


Thank you Dawid and Michael, this vote has passed !
Preparing release...

Andi..



Dawid


On Tue, May 30, 2023 at 1:44 AM Andi Vajda  wrote:



The PyLucene 9.6.0 (rc1) release tracking the recent release of
Apache Lucene 9.6.0 is ready.

A release candidate is available from:
https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.6.0-rc1/

PyLucene 9.6.0 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2 but this configuration is no
longer
tested.

Please vote to release these artifacts as PyLucene 9.6.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 9.6.0-rc1

[Andi Vajda]

The PyLucene 9.6.0 (rc1) release tracking the recent release of
Apache Lucene 9.6.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.6.0-rc1/

PyLucene 9.6.0 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2 but this configuration is no longer
tested.

Please vote to release these artifacts as PyLucene 9.6.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: PyLucene installation - make failing

[Andi Vajda]

On Tue, 28 Mar 2023, S Less wrote:


I'm able to successfully perform indexing and searching within Python.
I was hoping to try the Monitor

class however this line fails

from org.apache.lucene.monitor import Monitor

Traceback (most recent call last):
 File "/Users/akeers/Documents/git_work/pylucene-test/test2.py", line 2,
in 
   from org.apache.lucene.monitor import Monitor
ModuleNotFoundError: No module named 'org.apache.lucene.monitor';
'org.apache.lucene' is not a package


Indeed, the org.apache.lucene.monitor package is not built into PyLucene.
Why ? I didn't know about it.

Here is a diff you can apply to Makefile to get the Monitor stuff included 
in PyLucene:


-- SNIP --
Index: Makefile
===
--- Makefile(revision 1904983)
+++ Makefile(working copy)
@@ -97,6 +97,7 @@
 JARS+=$(KUROMOJI_JAR)   # japanese analyzer module
 JARS+=$(MEMORY_JAR) # single-document memory index
 JARS+=$(MISC_JAR)   # misc
+JARS+=$(MONITOR_JAR)# monitor
 JARS+=$(MORFOLOGIK_JAR) # morfologik analyzer module
 JARS+=$(NORI_JAR)   # korean analyzer module
 #JARS+=$(PHONETIC_JAR)   # phonetic analyzer module
@@ -135,6 +136,7 @@
 
KUROMOJI_JAR=$(LUCENE)/analysis/kuromoji/build/runtimeJars/lucene-analysis-kuromoji-$(LUCENE_VER)-SNAPSHOT.jar
 
MEMORY_JAR=$(LUCENE)/memory/build/runtimeJars/lucene-memory-$(LUCENE_VER)-SNAPSHOT.jar
 
MISC_JAR=$(LUCENE)/misc/build/runtimeJars/lucene-misc-$(LUCENE_VER)-SNAPSHOT.jar
+MONITOR_JAR=$(LUCENE)/monitor/build/runtimeJars/lucene-monitor-$(LUCENE_VER)-SNAPSHOT.jar
 
NORI_JAR=$(LUCENE)/analysis/nori/build/runtimeJars/lucene-analysis-nori-$(LUCENE_VER)-SNAPSHOT.jar
 
PHONETIC_JAR=$(LUCENE)/analysis/phonetic/build/runtimeJars/lucene-analysis-phonetic-$(LUCENE_VER)-SNAPSHOT.jar
 
QUERIES_JAR=$(LUCENE)/queries/build/runtimeJars/lucene-queries-$(LUCENE_VER)-SNAPSHOT.jar
-- SNIP --

Andi..

Re: PyLucene installation - make failing

[Andi Vajda]

I suspect you didn't edit Makefile to enable the variables corresponding to 
your environment ?

Andi..

> On Mar 23, 2023, at 01:02, S Less  wrote:
> 
> Hi, as you can see below, make and make test are reporting *Illegal option:
> l*
> 
> and make test an additional error.
> 
> macOS Ventura Version 13.2.1
> 
> Thanks, Al
> 
> 
> 
> username@MacBook-Pro pylucene-9.4.1 % make
> 
> (cd lucene-java-9.4.1; ./gradlew collectRuntimeJars)
> 
> Downloading https://services.gradle.org/distributions/gradle-7.3.3-all.zip
> 
> ...10%...20%...30%...40%...50%...60%70%...80%...90%...100%
> 
> 
> Welcome to Gradle 7.3.3!
> 
> 
> Here are the highlights of this release:
> 
> - Easily declare new test suites in Java projects
> 
> - Support for Java 17
> 
> - Support for Scala 3
> 
> 
> For more details see https://docs.gradle.org/7.3.3/release-notes.html
> 
> 
> To honour the JVM settings for this build a single-use Daemon process will
> be forked. See
> https://docs.gradle.org/7.3.3/userguide/gradle_daemon.html#sec:disabling_the_daemon
> .
> 
> Daemon will be stopped at the end of the build
> 
> 
> *> Task :localSettings*
> 
> 
> IMPORTANT. This is the first time you ran the build. I wrote some sane
> defaults (for this machine) to 'gradle.properties', they will be picked up
> on consecutive gradle invocations (not this one).
> 
> 
> Run gradlew :helpLocalSettings for more information.
> 
> 
> *> Task :errorProneSkipped*
> 
> WARNING: errorprone disabled (skipped on builds not running inside CI
> environments, pass -Pvalidation.errorprone=true to enable)
> 
> 
> *> Task :lucene:core:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:backward-codecs:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:facet:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:memory:compileJava*
> 
> Note:
> /Users/username/Downloads/pylucene-9.4.1/lucene-java-9.4.1/lucene/memory/src/java/org/apache/lucene/index/memory/MemoryIndex.java
> uses or overrides a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:queries:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:sandbox:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:analysis:common:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:codecs:compileJava*
> 
> Note:
> /Users/username/Downloads/pylucene-9.4.1/lucene-java-9.4.1/lucene/codecs/src/java/org/apache/lucene/codecs/simpletext/SimpleTextDocValuesReader.java
> uses or overrides a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:extensions:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:misc:compileJava*
> 
> Note:
> /Users/username/Downloads/pylucene-9.4.1/lucene-java-9.4.1/lucene/misc/src/java/org/apache/lucene/misc/util/fst/UpToTwoPositiveIntOutputs.java
> uses or overrides a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *> Task :lucene:test-framework:compileJava*
> 
> Note: Some input files use or override a deprecated API.
> 
> Note: Recompile with -Xlint:deprecation for details.
> 
> 
> *BUILD SUCCESSFUL* in 4m 26s
> 
> 123 actionable tasks: 123 executed
> 
> ICU not installed
> 
> jar
> lucene-java-9.4.1/lucene/core/build/runtimeJars/lucene-core-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/analysis/common/build/runtimeJars/lucene-analysis-common-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/backward-codecs/build/runtimeJars/lucene-backward-codecs-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/classification/build/runtimeJars/lucene-classification-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/codecs/build/runtimeJars/lucene-codecs-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/expressions/build/runtimeJars/lucene-expressions-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/extensions/build/runtimeJars/lucene-extensions-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/facet/build/runtimeJars/lucene-facet-9.4.1-SNAPSHOT.jar
> --jar
> lucene-java-9.4.1/lucene/grouping/build/runtimeJars/lucene-grouping-9.4.1-SNAPSHOT.jar
> --jar
> 

Re: JCC Compilation Error

[Andi Vajda]

On Thu, 23 Mar 2023, S Less wrote:


*jcc3/sources/functions.cpp:1742:28: **error: **expression is not
assignable*

   Py_TYPE(*type) = PY_TYPE(FinalizerClass);

*~~ ^*


This is https://issues.apache.org/jira/browse/PYLUCENE-66 and was fixed for 
PyLucene 9.4.1.
You say "I am running setup.py from the latest PyLucene download", is that 
9.4.1 ?


Andi..

Re: Adding new extension point

[Andi Vajda]

On Thu, 10 Nov 2022, Benjamin Trent wrote:


Hey y'all,

I am new to this type of workflow, I am used to github and Pull-requests.

What is the process for adding a new extension point? With a recent foray
into getting Lucene KNN into the ann-benchmarks repository, I found the
need to adjust the current codec (Lucene94Codec) to allow us to adjust some
values.

I think the extension will only need to allow us to optionally override the
`getFormat...ForField` parameters.


Technically:
  - see how it's done by looking at the code tree in PyLucene's extensions
directory tree where the native extension points are defined and add
yours there - then see how they're used in PyLucene's Python unit tests
where the actual extension points are implemented in Python

Procedurally (assuming you want your change maintained in the PyLucene code 
base):

  - explain the benefits for it to be part of the PyLucene codebase and
how it'll be maintained as Lucene's codebase evolves
  - open an issue on Jira [1] and attach a patch or send the code changes as
a patch file to the list directly

Andi..

[1] 
https://issues.apache.org/jira/projects/PYLUCENE/issues/PYLUCENE-28?filter=allopenissues

Re: [VOTE] Release PyLucene 9.4.1-rc3

[Andi Vajda]

This vote has passed.
Thank you all who voted !

The PyLucene 9.4.1 is now available (or when the download mirrors show it).

Andi..

On Tue, 1 Nov 2022, Andi Vajda wrote:



The PyLucene 9.4.1 (rc3) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc3/

PyLucene 9.4.1 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2, although Python 2 support is now 
untested.


Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Comment Edited] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629184#comment-17629184
 ] 

Andi Vajda edited comment on PYLUCENE-66 at 11/4/22 7:52 PM:
-

See code for installType() function in this file:
[https://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/jcc3/sources/functions.cpp]

 


was (Author: vajda):
The installType function in jcc/jcc3/sources/functions.cpp is supposed to be

{{
void installType(PyTypeObject **type, PyType_Def *def,
 PyObject *module, char *name, int isExtension)
{
if (*type == NULL)
{
*type = makeType(def);

if (isExtension)
{
#if PY_VERSION_HEX >= 0x030b
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));
#else
Py_TYPE(*type) = PY_TYPE(FinalizerClass);
#endif
Py_INCREF(PY_TYPE(FinalizerClass));
}

PyObject *module_name = PyModule_GetNameObject(module);

if (module_name != NULL)
{
PyDict_SetItemString((*type)->tp_dict, "__module__", module_name);
Py_DECREF(module_name);
}

PyModule_AddObject(module, name, (PyObject *) *type);
}
}
}}

> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629184#comment-17629184
 ] 

Andi Vajda edited comment on PYLUCENE-66 at 11/4/22 7:50 PM:
-

The installType function in jcc/jcc3/sources/functions.cpp is supposed to be

{{
void installType(PyTypeObject **type, PyType_Def *def,
 PyObject *module, char *name, int isExtension)
{
if (*type == NULL)
{
*type = makeType(def);

if (isExtension)
{
#if PY_VERSION_HEX >= 0x030b
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));
#else
Py_TYPE(*type) = PY_TYPE(FinalizerClass);
#endif
Py_INCREF(PY_TYPE(FinalizerClass));
}

PyObject *module_name = PyModule_GetNameObject(module);

if (module_name != NULL)
{
PyDict_SetItemString((*type)->tp_dict, "__module__", module_name);
Py_DECREF(module_name);
}

PyModule_AddObject(module, name, (PyObject *) *type);
}
}
}}


was (Author: vajda):
The installType function in jcc/jcc3/sources/functions.cpp is supposed to be
```
void installType(PyTypeObject **type, PyType_Def *def,
 PyObject *module, char *name, int isExtension)
{
if (*type == NULL)
{
*type = makeType(def);

if (isExtension)
{
#if PY_VERSION_HEX >= 0x030b
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));
#else
Py_TYPE(*type) = PY_TYPE(FinalizerClass);
#endif
Py_INCREF(PY_TYPE(FinalizerClass));
}

PyObject *module_name = PyModule_GetNameObject(module);

if (module_name != NULL)
{
PyDict_SetItemString((*type)->tp_dict, "__module__", module_name);
Py_DECREF(module_name);
}

PyModule_AddObject(module, name, (PyObject *) *type);
}
}
```



> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629184#comment-17629184
 ] 

Andi Vajda commented on PYLUCENE-66:


The installType function in jcc/jcc3/sources/functions.cpp is supposed to be
```
void installType(PyTypeObject **type, PyType_Def *def,
 PyObject *module, char *name, int isExtension)
{
if (*type == NULL)
{
*type = makeType(def);

if (isExtension)
{
#if PY_VERSION_HEX >= 0x030b
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));
#else
Py_TYPE(*type) = PY_TYPE(FinalizerClass);
#endif
Py_INCREF(PY_TYPE(FinalizerClass));
}

PyObject *module_name = PyModule_GetNameObject(module);

if (module_name != NULL)
{
PyDict_SetItemString((*type)->tp_dict, "__module__", module_name);
Py_DECREF(module_name);
}

PyModule_AddObject(module, name, (PyObject *) *type);
}
}
```



> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629183#comment-17629183
 ] 

Andi Vajda commented on PYLUCENE-66:


Use the JCC in the 9.4.1 release candidate or get the diff for rev 1904960.

> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] Release PyLucene 9.4.1-rc3

[Andi Vajda]

The PyLucene 9.4.1 (rc3) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc3/

PyLucene 9.4.1 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2, although Python 2 support is now 
untested.


Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.4.1-rc2

[Andi Vajda]

This vote has now failed too as Py_SET_TYPE is not available in Python 3.6.
rc3 is ready.

Sorry for the noise,

Andi..

On Tue, 1 Nov 2022, Andi Vajda wrote:


The PyLucene 9.4.1 (rc2) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc2/

PyLucene 9.4.1 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2, although Python 2 support is now 
untested.


Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.4.1-rc2

[Andi Vajda]

On Tue, 1 Nov 2022, Benjamin Trent wrote:


Andi,

I pulled down the rc-2 and tested in Docker on Ubuntu 18.04.6 LTS (Bionic
Beaver), with Python 3.6.9, I get the following error attempting to build
JCC

jcc3/sources/functions.cpp: In function 'void installType(PyTypeObject**,
PyType_Def*, PyObject*, char*, int)':
jcc3/sources/functions.cpp:1742:13: error: 'Py_SET_TYPE' was not declared
in this scope
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));


Ok, that says that this macro doesn't exist in Python 3.6.
I can add some conditionals around this.
I guess we're now onto rc3...

Andi..


^~~
jcc3/sources/functions.cpp:1742:13: note: suggested alternative:
'__S64_TYPE'
Py_SET_TYPE(*type, PY_TYPE(FinalizerClass));
^~~
__S64_TYPE
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1

I am thinking https://issues.apache.org/jira/browse/PYLUCENE-66 is related.

Thanks!

On Tue, Nov 1, 2022 at 2:13 PM Andi Vajda  wrote:



The PyLucene 9.4.1 (rc2) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc2/

PyLucene 9.4.1 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2, although Python 2 support is now
untested.

Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 9.4.1-rc2

[Andi Vajda]

The PyLucene 9.4.1 (rc2) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc2/

PyLucene 9.4.1 is built with JCC 3.13, included in these release artifacts.

JCC 3.13 supports Python 3.3 up to Python 3.11.
PyLucene may also be built with Python 2, although Python 2 support is now 
untested.


Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.4.1

[Andi Vajda]

This vote has failed - we need to release a new version of JCC after all in 
order to support Python 3.11, which came out last week.


Preparing rc2.

Andi..

On Mon, 31 Oct 2022, Andi Vajda wrote:



The PyLucene 9.4.1 (rc1) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc1/

PyLucene 9.4.1 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3, although Python 2 support is
now untested.

Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Comment Edited] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626895#comment-17626895
 ] 

Andi Vajda edited comment on PYLUCENE-66 at 11/1/22 1:12 AM:
-

I switched that line to use Py_SET_TYPE() and that should fix this bug.
I have not yet upgrade to Python 3.11 yet, could you please try this fix and 
see if there are other issues with Python 3.11 ?
Thanks !
(The fix is in rev 1904960)


was (Author: vajda):
I switched that line to use Py_SET_TYPE() and that should fix this bug.
I have not yet upgrade to Python 3.11 yet, could you please try this fix and 
see if there are other issues with Python 3.11 ?
Thanks !
(The fix is in rev 1904960)

> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626895#comment-17626895
 ] 

Andi Vajda commented on PYLUCENE-66:


I switched that line to use Py_SET_TYPE() and that should fix this bug.
I have not yet upgrade to Python 3.11 yet, could you please try this fix and 
see if there are other issues with Python 3.11 ?
Thanks !
(The fix is in rev 1904960)

> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (PYLUCENE-66) JCC doesn't build with Python 3.11.

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-66.

Resolution: Fixed

fixed in rev 1904960.

> JCC doesn't build with Python 3.11.
> ---
>
> Key: PYLUCENE-66
> URL: https://issues.apache.org/jira/browse/PYLUCENE-66
> Project: PyLucene
>  Issue Type: Bug
> Environment: Python 3.11, PyLucene 9.4.1-rc
>Reporter: A. Coady
>Priority: Major
>
> {{#10 10.86 In file included from /usr/local/include/python3.11/Python.h:44,}}
> {{#10 10.86 from _jcc3/java/lang/Object.h:18,}}
> {{#10 10.86 from jcc3/sources/functions.cpp:23:}}
> {{#10 10.86 jcc3/sources/functions.cpp: In function ‘void 
> installType(PyTypeObject*{*}, PyType_Def{*}, PyObject*, char*, int)’:}}
> {{#10 10.86 /usr/local/include/python3.11/object.h:136:30: error: lvalue 
> required as left operand of assignment}}
> {{#10 10.86 136 | # define Py_TYPE(ob) Py_TYPE(_PyObject_CAST(ob))}}
> {{#10 10.86 | ~{~}{{~}}^{{~}}{~}~}}
> {{#10 10.86 jcc3/sources/functions.cpp:1742:13: note: in expansion of macro 
> ‘Py_TYPE’}}
> {{#10 10.86 1742 | Py_TYPE(*type) = PY_TYPE(FinalizerClass);}}
> {{#10 10.86 | ^~~}}
> {{#10 10.89 error: command '/usr/bin/gcc' failed with exit code 1}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] Release PyLucene 9.4.1

[Andi Vajda]

The PyLucene 9.4.1 (rc1) release tracking the recent release of
Apache Lucene 9.4.1 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.4.1-rc1/

PyLucene 9.4.1 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3, although Python 2 support is
now untested.

Please vote to release these artifacts as PyLucene 9.4.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: New release of PyLucene?

[Andi Vajda]

On Mon, 31 Oct 2022, Benjamin Trent wrote:


Thank you very much Andi!

Once I wrapped my head around the basics, I found PyLucene
excellent, intuitive, and just like writing in Lucene, but with Python!

It would be marvelous if we could automate the process or document the
steps for building newer versions.


Sadly, the marvelous documentation goes as follows:
  - point PyLucene's Makefile at the version of Lucene you wish to wrap
  - build PyLucene as usual
  - fix the breakages

For example, for Lucene 9.4.1, the only breakage was updating the hppc 
version. Trivial, this time.


Andi..

Re: New release of PyLucene?

[Andi Vajda]

> On Oct 31, 2022, at 07:55, Benjamin Trent  wrote:
> 
> Lucene 9.4.1 was recently released. The last version of PyLucene released
> was 9.1.0. There have been some improvements to Lucene since then. My
> particular concern is around KNN search.
> 
> What is the process to start a new release of PyLucene?

Showing interest, which you just did !
Let me get one going...

Andi..

> 
> 
> Thank you!
> 
> Ben

[jira] [Commented] (PYLUCENE-64) ModuleNotFoundError: No module named 'lucene' when intalling PyLucene

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545615#comment-17545615
 ] 

Andi Vajda commented on PYLUCENE-64:


I believe there is an issue with running 'make test' in PyLucene 8.x. This is 
hopefully solved in PyLucene 9.1.0, which was released on 4/27/22.
To see if your PyLucene install works, you can try these steps in Python:
  >>> import lucene
  >>> lucene.initVM()
  >>> from org.apache.lucene.document import Document

If you need more help, please include the portion of the PyLucene Makefile you 
edited before you ran 'make' and 'make install'.



> ModuleNotFoundError: No module named 'lucene' when intalling PyLucene
> -
>
> Key: PYLUCENE-64
> URL: https://issues.apache.org/jira/browse/PYLUCENE-64
> Project: PyLucene
>  Issue Type: Bug
>Reporter: anis amirouche
>Priority: Major
>
> I'm trying to install Pylucene 8.11.0 on mac os. the jcc is built and 
> installed successfully, but from pylucene folder when running make test i'm 
> encountering the following error : ModuleNotFoundError: No module named 
> 'lucene', please note the make command is successfull.
> please can anyone help me with solving this error. I don't even understand 
> what i'm doing wrong, it's like lucene is not installed, and i'm having this 
> error with all the files of pylucene. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Commented] (PYLUCENE-63) can not install pylucene on linux with proxy

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529472#comment-17529472
 ] 

Andi Vajda commented on PYLUCENE-63:


The first 'make print-GENERATE' shows that you didn't follow the instructions 
at the top of the Makefile. You must uncomment one of the example groups so 
that PREFIX_PYTHON, PYTHON, JCC and NUM_FILES are set. And they must be set 
correctly for your environment.
You then did this, apparently, and your next print-GENERATE is making more 
sense (!).

The next issue then is that you didn't rebuild JCC (it shows you using version 
3.10 but PyLucene 9.1.0 comes with 3.12) and the earlier JCC build you made was 
built with an incompatible version of Java. Please rebuild JCC so that you are 
using version 3.12.

The last issue is that your java installation's layout (in particular its paths 
to libraries libjava and libjvm) is not covered in JCC's setup.py. This is a 
known issue, there are just so many different layouts that it's difficult to 
guess the correct one. You need to edit jcc's setup.py at line 172,173,174 and 
set the values for the -L flags to match your installation. I suspect that you 
don't need the final amd64 parts in there, ie, change -L%(linux)s/jre/lib/amd64 
into -L%(linux)s/jre/lib, change -L%(linux)s/jre/lib/amd64/server to 
-L%(linux)s/jre/lib/server and 
'-Wl,-rpath=%(linux)s/jre/lib/amd64:%(linux)s/jre/lib/amd64/server to 
'-Wl,-rpath=%(linux)s/jre/lib:%(linux)s/jre/lib/server.

Last but not least, you seem to be using python 2.7. It's still supported by 
PyLucene and JCC but not by the Python people anymore and it is not going to be 
supported by PyLucene and JCC much longer. I strongly recommend you install 
python3. Python 2.7 has been dead and deprecated for years now. If you are just 
getting started with Python now, please start with python 3.

And finally, this entire conversation should be had on the 
pylucene-dev@lucene.apache.org mailing list instead of buried in JIRA. This 
issue is not a bug.

> can not install pylucene on linux  with proxy
> -
>
> Key: PYLUCENE-63
> URL: https://issues.apache.org/jira/browse/PYLUCENE-63
> Project: PyLucene
>  Issue Type: Bug
> Environment: ubuntu 2020.4
>Reporter: lisa.shi
>Priority: Major
>
> i try to install pyylucene on linux  with proxy. i have already set 
> http_proxy on linux . i  have change Makefile .
> from 
> cd $(LUCENE); ($(ANT) ivy-availability-check || $(ANT) ivy-bootstrap )
> to
> cd $(LUCENE); ($(ANT) ivy-availability-check -autoproxy|| $(ANT) 
> ivy-bootstrap -autoproxy)
> then i can get ivy-2.4.0.jar in  ~/.ant/lib/
> when i try to make pyluence. i meet a bug.
>  
>  
> ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .pom
> [ivy:cachepath]           – artifact org.codehaus.groovy#groovy-all;2.4  
> .17!groovy-all.jar:
> [ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .jar
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 ::          UNRESOLVED DEPENDENCIES           
> ::
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 :: org.codehaus.groovy#groovy-all;2.4.17  : 
> not found
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]  ERRORS
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.pom (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.jar (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.pom (java.net.ConnectException: Connection timed out (  
> Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.jar (java.net.ConnectException: Connection timed out (  
> Connection timed out))
>

[jira] [Commented] (PYLUCENE-63) can not install pylucene on linux with proxy

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529212#comment-17529212
 ] 

Andi Vajda commented on PYLUCENE-63:


What does 'make print-GENERATE' return for you ?
Also, please include the values you uncommmented in Makefile for:
  PREFIX_PYTHON, PYTHON, JCC, NUM_FILES


> can not install pylucene on linux  with proxy
> -
>
> Key: PYLUCENE-63
> URL: https://issues.apache.org/jira/browse/PYLUCENE-63
> Project: PyLucene
>  Issue Type: Bug
> Environment: ubuntu 2020.4
>Reporter: lisa.shi
>Priority: Major
>
> i try to install pyylucene on linux  with proxy. i have already set 
> http_proxy on linux . i  have change Makefile .
> from 
> cd $(LUCENE); ($(ANT) ivy-availability-check || $(ANT) ivy-bootstrap )
> to
> cd $(LUCENE); ($(ANT) ivy-availability-check -autoproxy|| $(ANT) 
> ivy-bootstrap -autoproxy)
> then i can get ivy-2.4.0.jar in  ~/.ant/lib/
> when i try to make pyluence. i meet a bug.
>  
>  
> ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .pom
> [ivy:cachepath]           – artifact org.codehaus.groovy#groovy-all;2.4  
> .17!groovy-all.jar:
> [ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .jar
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 ::          UNRESOLVED DEPENDENCIES           
> ::
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 :: org.codehaus.groovy#groovy-all;2.4.17  : 
> not found
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]  ERRORS
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.pom (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.jar (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.pom (java.net.ConnectException: Connection timed out (  
> Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.jar (java.net.ConnectException: Connection timed out (  
> Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repository.cl|https://repository.cl/]  
> oudera.com/artifactory/libs-release-local/org/codehaus/groovy/groovy-all  
> /2.4.17/groovy-all-2.4.17.pom (java.net.ConnectException: Connection tim  ed 
> out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repository.cl|https://repository.cl/]  
> oudera.com/artifactory/libs-release-local/org/codehaus/groovy/groovy-all  
> /2.4.17/groovy-all-2.4.17.jar (java.net.ConnectException: Connection tim  ed 
> out (Connection timed out))
> [ivy:cachepath]
> [ivy:cachepath] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> BUILD FAILED
>  
>  
> how can i fix this bug?
> i am sure the file ( 
> [https://repo1.maven.o|https://repo1.maven.o/]rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.jar)
>  is not exist



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (PYLUCENE-63) can not install pylucene on linux with proxy

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-63?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-63.

Resolution: Information Provided

I cannot help you with proxy issues and this bug is obsolete by now as Lucene 
is now at version 9.1.0 and is no longer using Ant nor Ivy but switched its 
build to Gradle.
Just 10 minutes ago, I released [PyLucene 
9.1.0|https://lucene.apache.org/pylucene/index.html] which also switched to 
using Gradle for building the Lucene part. If you have issues with building the 
Lucene part of PyLucene with Gradle, please contact the Lucene project at 
java-u...@lucene.apache.org.
Thank you !

> can not install pylucene on linux  with proxy
> -
>
> Key: PYLUCENE-63
> URL: https://issues.apache.org/jira/browse/PYLUCENE-63
> Project: PyLucene
>  Issue Type: Bug
> Environment: ubuntu 2020.4
>Reporter: lisa.shi
>Priority: Major
>
> i try to install pyylucene on linux  with proxy. i have already set 
> http_proxy on linux . i  have change Makefile .
> from 
> cd $(LUCENE); ($(ANT) ivy-availability-check || $(ANT) ivy-bootstrap )
> to
> cd $(LUCENE); ($(ANT) ivy-availability-check -autoproxy|| $(ANT) 
> ivy-bootstrap -autoproxy)
> then i can get ivy-2.4.0.jar in  ~/.ant/lib/
> when i try to make pyluence. i meet a bug.
>  
>  
> ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .pom
> [ivy:cachepath]           – artifact org.codehaus.groovy#groovy-all;2.4  
> .17!groovy-all.jar:
> [ivy:cachepath]           [https://repository.cloudera.com/artifactory/li]  
> bs-release-local/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17  .jar
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 ::          UNRESOLVED DEPENDENCIES           
> ::
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]                 :: org.codehaus.groovy#groovy-all;2.4.17  : 
> not found
> [ivy:cachepath]                   
> ::
> [ivy:cachepath]  ERRORS
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.pom (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repo1.maven.o|https://repo1.maven.o/]  
> rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.jar (j  
> ava.net.ConnectException: Connection timed out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.pom (java.net.ConnectException: Connection timed out (  
> Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://oss.sonatype|https://oss.sonatype/].  
> org/content/repositories/releases/org/codehaus/groovy/groovy-all/2.4.17/  
> groovy-all-2.4.17.jar (java.net.ConnectException: Connection timed out (  
> Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repository.cl|https://repository.cl/]  
> oudera.com/artifactory/libs-release-local/org/codehaus/groovy/groovy-all  
> /2.4.17/groovy-all-2.4.17.pom (java.net.ConnectException: Connection tim  ed 
> out (Connection timed out))
> [ivy:cachepath]         Server access error at url 
> [https://repository.cl|https://repository.cl/]  
> oudera.com/artifactory/libs-release-local/org/codehaus/groovy/groovy-all  
> /2.4.17/groovy-all-2.4.17.jar (java.net.ConnectException: Connection tim  ed 
> out (Connection timed out))
> [ivy:cachepath]
> [ivy:cachepath] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> BUILD FAILED
>  
>  
> how can i fix this bug?
> i am sure the file ( 
> [https://repo1.maven.o|https://repo1.maven.o/]rg/maven2/org/codehaus/groovy/groovy-all/2.4.17/groovy-all-2.4.17.jar)
>  is not exist



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Re: [VOTE] Release PyLucene 9.1.0 (rc4)

[Andi Vajda]

This vote has passed.
Thank you all who voted !

Andi..

On Fri, 22 Apr 2022, Andi Vajda wrote:



The PyLucene 9.1.0 (rc4) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc4/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0 (rc4)

[Andi Vajda]

 Hi Mike,

On Tue, 26 Apr 2022, Andi Vajda wrote:


I have not yet tested PyLucene and JCC with python 3.10.
It could be that the heuristics for modern setuptools need to be updated 
again...


I don't think it's worth respinning but I need to have python 3.10 and this 
setuptools stuff resolved for the next release.


I installed python 3.10.4 on macos from sources.
It came with setuptools 58.1.0.

I was able to build jcc shared without forcing with_modern_setuptools.
I then ran pip install --upgrade setuptools and got version 62.1.0.
I was still able to build jcc shared without forcing with_modern_setuptools.

What version of setuptools do you have installed ?

Andi..

Re: [VOTE] Release PyLucene 9.1.0 (rc4)

[Andi Vajda]

 Hi Mike,

Thank you for your vote.
Replies inline...

On Tue, 26 Apr 2022, Michael McCandless wrote:


+1 to release.

I ran my usual smoke test, indexing first 100K Wikipedia docs, running a
couple queries, force merging down to one segment and running them again.

I discovered in this process that IndexWriter.getReader() had been removed
and tracked down the change that did that (it's good, thanks Dawid!) :)

I did hit one wrinkle during installation -- JCC was not built with a
shared lib by default, and poking around in setup.py, I found that it had
detected with_modern_setuptools = False, which was odd since I'm using
Python 3.10.2 (and JDK 17 on Arch Linux).  I tried "pip3 install
setuptools" but it was already up to date.  Then I edited setup.py and
"hardwired" the with_modern_setuptools to True, and then everything
installed fine.  Maybe I did something silly wrong in editing setup.py /
Makefile, so I don't think this has to block release ...


I have not yet tested PyLucene and JCC with python 3.10.
It could be that the heuristics for modern setuptools need to be updated 
again...


I don't think it's worth respinning but I need to have python 3.10 and this 
setuptools stuff resolved for the next release.


Andi..



Mike McCandless

http://blog.mikemccandless.com


On Mon, Apr 25, 2022 at 8:20 AM Nelia Vb  wrote:


+1


On 22 Apr 2022, at 23:51, Andi Vajda  wrote:


The PyLucene 9.1.0 (rc4) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc4/

PyLucene 9.1.0 is built with JCC 3.12, included in these release

artifacts.


JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python

2.3+).

PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 9.1.0 (rc4)

[Andi Vajda]

The PyLucene 9.1.0 (rc4) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc4/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0 (rc3)

[Andi Vajda]

This vote has now also failed.
There is still an issue with the PyLucene gradle setup

Andi..

On Wed, 20 Apr 2022, Andi Vajda wrote:



The PyLucene 9.1.0 (rc3) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc3/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 9.1.0 (rc3)

[Andi Vajda]

The PyLucene 9.1.0 (rc3) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc3/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0

[Andi Vajda]

On Wed, 20 Apr 2022, Andi Vajda wrote:

Ok, so the layout has changed with temurin I guess. This was also the case on 
Mac. I need to add yet another entry into LFLAGS for linux/temurin

in setup.py that reflects this new layout.


So it seems to be some sort of expected-packaging problem?


I need to debug this by installing temurin into a linux VM of mine.
To be continued...


I created a brand new Debian 11.3 virtual machine and set it up for PyLucene 
development with the Temurin 17 JDK. I had to add a new linux/temurin entry 
in LFLAGS since the layout of that JDK is different from the others.


Then I changed the JDK['linux'] in jcc's setup.py to the Temurin java home 
as it looked on my install: /usr/lib/jvm/temurin-17-jdk-amd64
It may be different on your system. You edit in yours or override it via the 
JCC_JDK env variable.


Then building jcc with:
  $ python setup.py build install
just worked (tm)

For PyLucene, I also refreshed the first Linux config to be for Temurin JDK.
Uncomment that one and fix it to reflect your environment.
Then building pylucene with:
  $ make all test
just worked (tm).

I'm preparing an rc3 now...

Andi..

Re: [VOTE] Release PyLucene 9.1.0 (rc2)

[Andi Vajda]

This vote has now failed too, linux support needs to be improved...

Andi..

> On Apr 19, 2022, at 17:29, Andi Vajda  wrote:
> 
> 
> The PyLucene 9.1.0 (rc2) release tracking last month's release of
> Apache Lucene 9.1.0 is ready.
> 
> A release candidate is available from:
>   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc2/
> 
> PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.
> 
> JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
> PyLucene may be built with Python 2 or Python 3.
> 
> Please vote to release these artifacts as PyLucene 9.1.0.
> Anyone interested in this release can and should vote !
> 
> Thanks !
> 
> Andi..
> 
> ps: the KEYS file for PyLucene release signing is at:
> https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
> https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS
> 
> pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0

[Andi Vajda]

 Hi Dawid,

On Wed, 20 Apr 2022, Dawid Weiss wrote:


If you tell me what OS you are on and what the error actually is, I can
help you a bit better. But assuming you're on a Mac, you do not need to 
export



I'm actually on Windows but I tried to compile everything on Linux - an
older Ubuntu with Java 17 installed. Here is what I see:


echo $JAVA_HOME

/[...]/jvms/jdk17


java -version

openjdk version "17" 2021-09-14
OpenJDK Runtime Environment Temurin-17+35 (build 17+35)
OpenJDK 64-Bit Server VM Temurin-17+35 (build 17+35, mixed mode, sharing)


cd jcc
python3 setup.py build install

Traceback (most recent call last):
 File "setup.py", line 100, in 
   ''' %(JDK[platform]))
RuntimeError:

Java JDK directory '/usr/lib/jvm/java-8-oracle' does not exist.


Yeah, on Linux, there is no support implemented in JCC for finding the 
proper JAVA_HOME so the hardcoded value in jcc's setup.py is old and wrong

and must be overriden via the JCC_JDK variable (or edited in).


Please set the environment variable JCC_JDK to the correct location before
running setup.py.

When I set JCC_JDK:


export JCC_JDK=$JAVA_HOME
python3 setup.py build install

[lots of messages]
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro
-g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time
-D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/jcc3/sources/jcc.o
build/temp.linux-x86_64-3.6/jcc3/sources/JCCEnv.o -o
build/lib.linux-x86_64-3.6/libjcc3.so -L/[...]/jvms/jdk17/jre/lib/amd64
-ljava -L/ [...]/jvms/jdk17/jre/lib/amd64/server -ljvm -Wl,-rpath=/
[...]/jvms/jdk17/jre/lib/amd64:/ [...]/jvms/jdk17/jre/lib/amd64/server
-Wl,-S -lpython3.6m

/usr/bin/ld: cannot find -ljava
/usr/bin/ld: cannot find -ljvm
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

When you look at the compilation line, Java-relative paths are wrong as
it's a JDK installation and it has lib/ and include/ directly underneath
(not under 'jre/lib/amd64', for example).


Ok, so the layout has changed with temurin I guess. This was also the case 
on Mac. I need to add yet another entry into LFLAGS for linux/temurin

in setup.py that reflects this new layout.


So it seems to be some sort of expected-packaging problem?


I need to debug this by installing temurin into a linux VM of mine.
To be continued...

Andi..

[VOTE] Release PyLucene 9.1.0 (rc2)

[Andi Vajda]

The PyLucene 9.1.0 (rc2) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc2/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0

[Andi Vajda]

This vote has failed as the release artifacts were found to contain 
unnecessary cruft.


Thank you Dawid for reporting the issue.
I'm calling for an rc2 vote in the next message.

Andi..

On Sun, 17 Apr 2022, Andi Vajda wrote:



The PyLucene 9.1.0 (rc1) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc1/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 9.1.0

[Andi Vajda]

 Hi Dawid,

Thank you for checking the rc out !
More replies inline.

On Tue, 19 Apr 2022, Dawid Weiss wrote:


I downloaded the release - these files and folders should be probably
excluded from the distribution as it's gradle's binary
throw-away caches and generated stuff:

pylucene-9.1.0/lucene-java-9.1.0/.gradle
pylucene-9.1.0/lucene-java-9.1.0/gradle.properties

Perhaps you should use "git clean -xfd ." on Lucene's checkout prior
to assembling the distribution?


I'm now excluding all .[a-z]* files as well as gradle.properties.
And I'm respinning an rc2 for this change.


Also, the installation instructions seem a bit outdated (Java version, etc.):
https://lucene.apache.org/pylucene/install.html

I tried compiling everything from scratch by exporting JAVA_HOME and
JCC_JDK but failed. :(  Any hints as to what I may be doing wrong?


The instructions are very outdated indeed (!)

If you tell me what OS you are on and what the error actually is, I can help 
you a bit better. But assuming you're on a Mac, you do not need to export 
JAVA_HOME, but you can, it overrides what jcc is trying to do without it 
being set.

The $ /usr/libexec/java_home command works, it returns this
  /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
for me, which is correct.
If you do *not* set JAVA_HOME, jcc will try to figure it out for you, and 
report what it found (or not).


For me, on Mac, the build instructions are as simple as this:
I have python 3 in a virtual environment in my pylucene directory:
  $ cd ~/apache/pylucene/jcc
  $ ../_install3/bin/python setup.py build install
  $ cd ..
edit Makefile to uncommment/edit the configuration that corresponds to
your setup
  $ make all test

Please, let me know how it goes !

Andi..



Dawid

On Sun, Apr 17, 2022 at 9:48 PM Andi Vajda  wrote:



The PyLucene 9.1.0 (rc1) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc1/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 9.1.0

[Andi Vajda]

The PyLucene 9.1.0 (rc1) release tracking last month's release of
Apache Lucene 9.1.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/9.1.0-rc1/

PyLucene 9.1.0 is built with JCC 3.12, included in these release artifacts.

JCC 3.12 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 9.1.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Resolved] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-62.

Resolution: Fixed

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, 
> fixes_with_inclusion _of_windows_module.patch, jvm_dll.diff, small_fixes.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523433#comment-17523433
 ] 

Andi Vajda commented on PYLUCENE-62:


Thank you for having verified the changes again.
The 'import jcc' case, as you say, is not very useful but can be made to work 
by calling os.add_dll_directory(jvm_dll_path) manually first. If one is import 
the jcc package this way, they must know what they're up to.
I can't connect the %PREFIX%\lib\site-packages\jcc3\jcc3.lib issue to code in 
jcc other than the extra_link_args being set in python.py around line 1843. I 
don't see, at first glance, where %prefix% would be coming from.
I think this bug can now be marked fixed, please reopen if you think it needs 
more work.
Thank you again for all your work on this !

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, 
> fixes_with_inclusion _of_windows_module.patch, jvm_dll.diff, small_fixes.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523248#comment-17523248
 ] 

Andi Vajda commented on PYLUCENE-62:


Thank you for the fixes and explanations. I think I integrated your changes and 
even simplified things a bit more in python.py, now that we're including 
windows.py when --find-jvm-dll is set.
I committed the changes to HEAD. Please, try this out again by setting 
--find-jvm-dll server and let me know if it all works. Thank you !

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, 
> fixes_with_inclusion _of_windows_module.patch, jvm_dll.diff, small_fixes.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522433#comment-17522433
 ] 

Andi Vajda commented on PYLUCENE-62:


Does it work as expected if you pass --find-jvm-dll server to your command line 
?

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521968#comment-17521968
 ] 

Andi Vajda commented on PYLUCENE-62:


Hi Petrus, this issue seems to have stalled. 
Is it actually solved by the latest jvm_dll.diff patch attached here ?
I'm preparing to release PyLucene 9.1.0 and it'd be nice to know if it still 
works on Windows !

(thanks)

Andi..

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495042#comment-17495042
 ] 

Andi Vajda commented on PYLUCENE-62:


I'd like which kind of jvm.dll is used to not be random with regards to client 
or server kind. But if you prefer both to be searchable as well, we can add 
support for a "both" --find_jvm_dll value, that searches "client", then 
"server" (or the opposite, but again, not random)The default should remain 
"client" since that is what it's been doing for years. Also, it would be good 
to emit a log statement of some sort that says what jvm.dll (its full path) was 
found and added to the DLL path.
Bullet 2: I can't bet that people wouldn't copy source code around. It's also 
easier to debug if the same code is emitted everywhere, regardless of platform, 
using conditional code as necessary.
The stuff after SHARED is unchanged, your patch didn't touch it, nor did my 
changes. I'm wondering what to do about it, though, as it is also messing with 
PATH. Are you using --shared mode ?
About "it doesn't work": I can't debug this as I don't have access to Windows. 
Here, on mac, the following works fine, of course: python -c "import jcc; 
print(jcc.__file__)". Could you please take the current patch and "make it 
work" or, if you'd like my help with debugging it, tell me more details about 
what doesn't actually work ? In particular, please, add print statements in the 
__init__.py  and windows.py files as to what is going on until it eventually 
fails and send me the output. Thanks !

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492230#comment-17492230
 ] 

Andi Vajda commented on PYLUCENE-62:


I also not sure what to about the code in __init__.py, after line 28, in the 
SHARED case ?
Should the eggpath be added via os.add_dll_directory() as well ?
Are you using shared mode ?


> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492229#comment-17492229
 ] 

Andi Vajda commented on PYLUCENE-62:


I took a look at your patch and attached a version that should be less 
intrusive and work, hopefully, too:
  - you do not need to add windows.py to modules, it is already included and 
installed
  - you do need to keep code conditional to windows, you can't assume people 
are not
going to carry code around between machines
  - in windows.py, I renamed get_jvm_dll_directory() to 
get_jvm_dll_directory_from_registry()
   and otherwise kept it unchanged
  - in windows.py, I added get_jvm_dll_directory_from_env() based on your code.
  - in windows.py, I also changed add_jvm_dll_directory_to_path() to do the 
right thing, like
you did
  - I changed the --find-jvm-dll command line flag from a boolean to a string 
that must be one
of "client" or "server" and made it default to "client" for python >= 3.8, 
None otherwise.
Please, try the attached patch out by passing it --find-jvm-dll client (or 
server), I have no access to Windows, so I could not try it out myself. I 
verified that it didn't break on Mac.
Thank you !


> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda updated PYLUCENE-62:
---
Attachment: jvm_dll.diff

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win-1.patch, add_dll_win.patch, jvm_dll.diff
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491669#comment-17491669
 ] 

Andi Vajda commented on PYLUCENE-62:


As is, --find-jvm-dll shouldn't solve the issue because it modifies PATH and 
doesn't use the new os.add_dll_directory() function. 
If, on the other hand, you modify windows.py's add_jvm_dll_directory_to_path() 
to call os.add_dll_directory() instead of messing with Path, does 
--find-jvm-dll then solve the problem ? 
I have no access to Windows so I can't test any fix myself.
Currently, windows.py's get_jvm_dll_directory() does not look on JAVA_HOME. I 
think that it's reasonable that it should do so, as well as looking in the 
Windows registry like it does already. Maybe it should look at JAVA_HOME first, 
then continue looking in the registry if nothing is found ? Your JAVA_HOME 
traversal code should be added to get_jvm_dll_directory() so that all this 
logic is in one place.

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491471#comment-17491471
 ] 

Andi Vajda edited comment on PYLUCENE-62 at 2/12/22, 11:34 PM:
---

It looks like finding the jvm.dll is an old problem on Windows and python 3.8 
just made it a bit worse. If you look in jcc3/__init__.py, which you modified 
in this patch, you can see that something similar is done when the 
--find-jvm-dll command line flag is set.
The code in jcc3/windows.py has something a bit more involved than just relying 
on JAVA_HOME to locate this DLL. I think you should update the 
add_jvm_dll_directory_to_path() to use the new python 3.8 add_dll_directory() 
function instead of its current messing with the Path environment variable. If 
you think of making --find-jvm-dll the default on Windows/python3.8, then the 
add_jvm_dll_directory_to_path() function in windows.py should verify that 
jvm.dll isn't already findable, before adding another DLL path if necessary.
Same issue with the changes in jcc3/python.py: you should not depend on 
JAVA_HOME but share the same logic in jcc3/windows.py (assuming that logic is 
still correct, of course).
In other words, does --find-jvm-dll work for you ? (without your patch).


was (Author: vajda):
It looks like finding the jvm.dll is an old problem on Windows and python 3.8 
just made it a bit worse. If you look in jcc3/__init__.py, which you modified 
in this patch, you can see that something similar is done when the 
--find-jvm-dll command line flag is set.
The code in jcc3/windows.py has something a bit more involved than just relying 
on JAVA_HOME to locale this DLL. I think you should update the 
add_jvm_dll_directory_to_path() to use the new python 3.8 add_dll_directory() 
function instead of its current messing with the Path environment variable. If 
you think of making --find-jvm-dll the default on Windows/python3.8, then the 
add_jvm_dll_directory_to_path() function in windows.py should verify that 
jvm.dll isn't already findable, before adding a DLL path if necessary.
Same issue with the changes in jcc3/python.py: you should not depend on 
JAVA_HOME but share the same logic in jcc3/windows.py (assuming that logic is 
still correct, of course).
In other words, does --find-jvm-dll work for you ? (without your patch).

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (PYLUCENE-62) Not finding jvm.dll on windows

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491471#comment-17491471
 ] 

Andi Vajda commented on PYLUCENE-62:


It looks like finding the jvm.dll is an old problem on Windows and python 3.8 
just made it a bit worse. If you look in jcc3/__init__.py, which you modified 
in this patch, you can see that something similar is done when the 
--find-jvm-dll command line flag is set.
The code in jcc3/windows.py has something a bit more involved than just relying 
on JAVA_HOME to locale this DLL. I think you should update the 
add_jvm_dll_directory_to_path() to use the new python 3.8 add_dll_directory() 
function instead of its current messing with the Path environment variable. If 
you think of making --find-jvm-dll the default on Windows/python3.8, then the 
add_jvm_dll_directory_to_path() function in windows.py should verify that 
jvm.dll isn't already findable, before adding a DLL path if necessary.
Same issue with the changes in jcc3/python.py: you should not depend on 
JAVA_HOME but share the same logic in jcc3/windows.py (assuming that logic is 
still correct, of course).
In other words, does --find-jvm-dll work for you ? (without your patch).

> Not finding jvm.dll on windows
> --
>
> Key: PYLUCENE-62
> URL: https://issues.apache.org/jira/browse/PYLUCENE-62
> Project: PyLucene
>  Issue Type: Bug
>Reporter: Petrus Hyvönen
>Priority: Major
> Attachments: add_dll_win.patch
>
>
> On recent versions of Python, the dll's seems to require to be added via the 
> os.add_dll_directory() function.
>  
> Apparently something changed in Python 3.8 regarding this, "Python 3.8 
> changed the DLL resolution order 
> [https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew];
> Thanks to:
> [https://github.com/conda-forge/python-feedstock/issues/552]
>  
> Proposed fix in a patch below. It can likely be rewritten in some more neat 
> way. :)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [VOTE] Release PyLucene 8.11.0 rc2

[Andi Vajda]

This vote has passed !
Thank you all who voted.
The PyLucene 8.11.0 release artifacts should be available shortly.

Andi..

On Fri, 10 Dec 2021, Andi Vajda wrote:



The rc1 vote failed because of a bug fix in JCC that helps with detecting
the Temurin JDK - available from https://adoptium.net.
IIUC, the Temurin JDK supercedes AdoptOpenJDK.

Please vote on PyLucene 8.11.0 rc2 instead. These release artifacts were 
built

and tested with Temurin JDK 17.

  

The PyLucene 8.11.0 (rc2) release tracking the recent release of
Apache Lucene 8.11.0 is ready.

This should be the last of the PyLucene 8.x releases (!) since Lucene 9.0 is 
now available.


A release candidate is available from:
 https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.11.0-rc2/

PyLucene 8.11.0 is built with JCC 3.11, included in these release artifacts.

JCC 3.11 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
JCC 3.11 supports building PyLucene with JDK 17 Apple M1.
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.11.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: [VOTE] Release PyLucene 8.11.0 rc2

[Andi Vajda]

On Mon, 13 Dec 2021, Dawid Weiss wrote:


The notice file may need to be updated at some point - the date reads
(c) -> 2013? :)

Apache PyLucene
 Copyright 2009-2013 The Apache Software Foundation


Thanks, done in rev 1895915.

Andi..



+1.

Dawid

On Fri, Dec 10, 2021 at 11:33 PM Andi Vajda  wrote:



The rc1 vote failed because of a bug fix in JCC that helps with detecting
the Temurin JDK - available from https://adoptium.net.
IIUC, the Temurin JDK supercedes AdoptOpenJDK.

Please vote on PyLucene 8.11.0 rc2 instead. These release artifacts were built
and tested with Temurin JDK 17.



The PyLucene 8.11.0 (rc2) release tracking the recent release of
Apache Lucene 8.11.0 is ready.

This should be the last of the PyLucene 8.x releases (!) since Lucene 9.0 is
now available.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.11.0-rc2/

PyLucene 8.11.0 is built with JCC 3.11, included in these release artifacts.

JCC 3.11 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
JCC 3.11 supports building PyLucene with JDK 17 Apple M1.
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.11.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 8.11.0 rc2

[Andi Vajda]

The rc1 vote failed because of a bug fix in JCC that helps with detecting
the Temurin JDK - available from https://adoptium.net.
IIUC, the Temurin JDK supercedes AdoptOpenJDK.

Please vote on PyLucene 8.11.0 rc2 instead. These release artifacts were built
and tested with Temurin JDK 17.

   

The PyLucene 8.11.0 (rc2) release tracking the recent release of
Apache Lucene 8.11.0 is ready.

This should be the last of the PyLucene 8.x releases (!) since Lucene 9.0 is 
now available.


A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.11.0-rc2/

PyLucene 8.11.0 is built with JCC 3.11, included in these release artifacts.

JCC 3.11 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
JCC 3.11 supports building PyLucene with JDK 17 Apple M1.
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.11.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Resolved] (PYLUCENE-61) `adoptopenjdk` succeeded by `temurin` on macOS.

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-61?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-61.

Resolution: Fixed

Committed revision 1895796.

> `adoptopenjdk` succeeded by `temurin` on macOS.
> ---
>
> Key: PYLUCENE-61
> URL: https://issues.apache.org/jira/browse/PYLUCENE-61
> Project: PyLucene
>  Issue Type: Bug
> Environment: macOS
>Reporter: A. Coady
>Priority: Major
>
> {code:java}
> % brew info adoptopenjdk
> adoptopenjdk: 16.0.1,9
> https://adoptopenjdk.net/
> /usr/local/Caskroom/adoptopenjdk/16.0.1,9 (196.9MB)
> From: 
> https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/adoptopenjdk.rb
> ==> Name
> AdoptOpenJDK Java Development Kit
> ==> Description
> JDK from the Java User Group (JUG)
> ==> Artifacts
> OpenJDK16U-jdk_x64_mac_hotspot_16.0.1_9.pkg (Pkg)
> ==> Caveats
> Temurin is the official successor to this software:
>   brew install --cask temurin
> adoptopenjdk has been officially discontinued upstream.
> It may stop working correctly (or at all) in recent versions of macOS.
> {code}
> The support for `darwin/adoptopenjdk` in `setup.py` is based on checking 
> whether `adoptopenjdk` is in `JAVAHOME`. Under `temurin`, the check defaults 
> to `darwin/home`, breaking the paths for `INCLUDES` and `LFLAGS`.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[VOTE] Release PyLucene 8.11.0

[Andi Vajda]

The PyLucene 8.11.0 (rc1) release tracking the recent release of
Apache Lucene 8.11.0 is ready.

This should be the last of the PyLucene 8.x releases (!) since Lucene 9.0 
is now available.


A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.11.0-rc1/

PyLucene 8.11.0 is built with JCC 3.11, included in these release artifacts.

JCC 3.11 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
JCC 3.11 supports building PyLucene with JDK 17 early access on Apple M1.
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.11.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Closed] (PYLUCENE-60) we are providing the projects for mtech and btech students.

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda closed PYLUCENE-60.
--
Resolution: Invalid

This is spam, right ?

> we are providing the projects for mtech and btech students.
> ---
>
> Key: PYLUCENE-60
> URL: https://issues.apache.org/jira/browse/PYLUCENE-60
> Project: PyLucene
>  Issue Type: Blog - New Blog Request
> Environment: python
>Reporter: Nasima Takeoff
>Priority: Trivial
>  Labels: btech, final, mtech, projects, python
> Attachments: Python projects.png
>
>
> h4. Python has been within the top 10 popular programming languages for an 
> extended time because the community of Python programmers has grown tons 
> thanks to its easy syntax and library support. during this article, Takeoff 
> will be able to introduce you to 60 amazing Python projects with ASCII text 
> files solved and explained by experts in [Takeoff 
> Projects|https://takeoffprojects.com/].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: PyLucene for python3-dbg and libjcc3.so (Debian)

[Andi Vajda]

 Hi Erik,

On Fri, 30 Jul 2021, Erik Groeneveld - Seecr wrote:


Hi Andy,

Thanks for your answer. I can now formulate my question.

We use at least 3 extensions generated by JCC and we use shared mode.


Thus, you need shared mode.


My actual question is.

Why does JCC generate extensions that use the shared library
“libjcc3.so, while all that is needed is also in jcc/_
jcc3.cpython-37m-x86_64-linux-gnu.so? In short, what is the use of
libjcc3.so?


The code in that .so is not the same code in libjcc3.so which contains the 
global registry of java objects shared between the python VM and the java 
VM, iirc. That piece needs to be unique for the process, thus the extra 
shared library, which is not a python extension, actually, but a plain 
shared library.



The point is that generated extensions always link to libjcc3.so,
regardless if these extensions are build for debug or not. Why does JCC do
that?


It's is just a name. When you expect to use multiple incompatible versions 
of this library, you need to use a link flag such as 'rpath' (I don't 
remember the exact syntax) that builds into the libraries using libjcc3.so 
the full path of where the correct version of this library is installed (or 
you switch your LD_LIBRARY_PATH at runtime to guide the dynamic linker to 
pickup the right one at extension load time).


For macosx, you can see on line 377 in jcc's setup.py that I'm setting 
@rpath/libjcc%s.dylib, which is a similar trick for its linker to achieve 
this.


On linux, you need to make a similar change, or use the right version 
of LD_LIBRARY_PATH at runtime.
On Linux, to see where the dynamic linker is going to pickup libjcc3.so 
from, you can use the 'ldd' utility (on macosx, the similar utility is 
called 'otool').


Andi..



Best regards,
Erik



On Thu, 29 Jul 2021 at 18:47, Andi Vajda  wrote:




On Jul 29, 2021, at 14:51, Erik Groeneveld - Seecr 

wrote:


L.S.,

We make Debian packages for JCC and PyLucene.

When building debug extensions for JCC and Lucene, we run into the

problem

that JCC links the generated extension against libjcc3.so.


That is the case if you use 'shared mode' which is the default but is
optional. Shared mode lets you load more than one jcc-built extension in a
given python process.


However, libjcc3.so defines Python objects and because the Python object
header defines additional fields in DEBUG mode, there need to be a

separate

version of libjcc3.so to run with python3-dbg.


Then you need to link that against a dbg version of jcc too. You should
not mix shared libraries built with different python defines.


However, it looks like it is sufficient if the generated extension links
against _jcc3.cpython-37m-x86_64-linux-gnu.so, because all sources

compiled

into libjcc3.so are also in this shared library.

This means that the debug version should link against the ...-37dm-...
version.

This would require some changes in the way JCC generates new extensions.

Is my reasoning correct or do I miss some things?


I don't know what all these extra suffixes mean but you should not mix
shared libraries built with different defines (ie debug and non debug). If
you don't need shared mode then you can skip the shared library mess
altogether, otherwise you have to get it right: use a dbg version jcc to
build a dbg version of pylucene.

Andi..



Best regards,
Erik

--
Seecr helpt informatieprofessionals met het consistent integreren en
verbinden van decentrale metadata zodat zij zich helemaal kunnen

focussen

op de inhoud.
Meer weten? Kijk op seecr.nl <https://seecr.nl>.





--
Seecr helpt informatieprofessionals met het consistent integreren en
verbinden van decentrale metadata zodat zij zich helemaal kunnen focussen
op de inhoud.

?? Meer weten? Kijk op seecr.nl <https://seecr.nl>.

Re: PyLucene for python3-dbg and libjcc3.so (Debian)

[Andi Vajda]

> On Jul 29, 2021, at 14:51, Erik Groeneveld - Seecr  wrote:
> 
> L.S.,
> 
> We make Debian packages for JCC and PyLucene.
> 
> When building debug extensions for JCC and Lucene, we run into the problem
> that JCC links the generated extension against libjcc3.so.

That is the case if you use 'shared mode' which is the default but is optional. 
Shared mode lets you load more than one jcc-built extension in a given python 
process.

> However, libjcc3.so defines Python objects and because the Python object
> header defines additional fields in DEBUG mode, there need to be a separate
> version of libjcc3.so to run with python3-dbg.

Then you need to link that against a dbg version of jcc too. You should not mix 
shared libraries built with different python defines.

> However, it looks like it is sufficient if the generated extension links
> against _jcc3.cpython-37m-x86_64-linux-gnu.so, because all sources compiled
> into libjcc3.so are also in this shared library.
> 
> This means that the debug version should link against the ...-37dm-...
> version.
> 
> This would require some changes in the way JCC generates new extensions.
> 
> Is my reasoning correct or do I miss some things?

I don't know what all these extra suffixes mean but you should not mix shared 
libraries built with different defines (ie debug and non debug). If you don't 
need shared mode then you can skip the shared library mess altogether, 
otherwise you have to get it right: use a dbg version jcc to build a dbg 
version of pylucene.

Andi..

> 
> Best regards,
> Erik
> 
> -- 
> Seecr helpt informatieprofessionals met het consistent integreren en 
> verbinden van decentrale metadata zodat zij zich helemaal kunnen focussen 
> op de inhoud. 
> Meer weten? Kijk op seecr.nl .
> 

[jira] [Resolved] (PYLUCENE-59) Python warns about missing __module__, but means that type names have no '.' in them.

[Andi Vajda (Jira)]

 [ 
https://issues.apache.org/jira/browse/PYLUCENE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andi Vajda resolved PYLUCENE-59.

Resolution: Workaround

> Python warns about missing __module__, but means that type names have no '.' 
> in them.
> -
>
> Key: PYLUCENE-59
> URL: https://issues.apache.org/jira/browse/PYLUCENE-59
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Erik Groeneveld
>Priority: Trivial
>
> When starting JCC, Python emits warnings such as
> {code:java}
> DeprecationWarning: builtin type Object has no __module__ attribute
> {code}
> It does this because, early in de process of creating types, it does not find 
> a '.' in de name of the type. The warning is somewhat misleading. The code 
> from Python is (fragment from typeobject.c): 
> {code:java}
>     /* Set type.__module__ */
>     s = strrchr(spec->name, '.');
>     if (s != NULL) {
>         int err;
>         modname = PyUnicode_FromStringAndSize(
>                 spec->name, (Py_ssize_t)(s - spec->name));
>         if (modname == NULL) {
>             goto fail;
>         }
>         err = _PyDict_SetItemId(type->tp_dict, ___module__, modname);
>         Py_DECREF(modname);
>         if (err != 0)
>             goto fail;
>     } else {
>         if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1,
>                 "builtin type %.200s has no __module__ attribute",
>                 spec->name))
>             goto fail;
>     }
> {code}
> The name of the types in JCC do not include a package name and hence no dot.
> Python 3.10 still does it like this.
> The __module__ is set correctly later on in the JCC code!
> Maybe you could add a package name (and a dot) to the typename to avoid these 
> warning?
> I am just reporting this for your convenience and maybe it helps others 
> seeing these warnings. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (PYLUCENE-59) Python warns about missing __module__, but means that type names have no '.' in them.

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383336#comment-17383336
 ] 

Andi Vajda commented on PYLUCENE-59:


I did 'fix' that as much as possible by setting it later as you saw in the
code already. The warning is harmless and a proper fix to workaround what
looks like a kludge in Python is more involved than the problem is worth.


> Python warns about missing __module__, but means that type names have no '.' 
> in them.
> -
>
> Key: PYLUCENE-59
> URL: https://issues.apache.org/jira/browse/PYLUCENE-59
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Erik Groeneveld
>Priority: Trivial
>
> When starting JCC, Python emits warnings such as
> {code:java}
> DeprecationWarning: builtin type Object has no __module__ attribute
> {code}
> It does this because, early in de process of creating types, it does not find 
> a '.' in de name of the type. The warning is somewhat misleading. The code 
> from Python is (fragment from typeobject.c): 
> {code:java}
>     /* Set type.__module__ */
>     s = strrchr(spec->name, '.');
>     if (s != NULL) {
>         int err;
>         modname = PyUnicode_FromStringAndSize(
>                 spec->name, (Py_ssize_t)(s - spec->name));
>         if (modname == NULL) {
>             goto fail;
>         }
>         err = _PyDict_SetItemId(type->tp_dict, ___module__, modname);
>         Py_DECREF(modname);
>         if (err != 0)
>             goto fail;
>     } else {
>         if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1,
>                 "builtin type %.200s has no __module__ attribute",
>                 spec->name))
>             goto fail;
>     }
> {code}
> The name of the types in JCC do not include a package name and hence no dot.
> Python 3.10 still does it like this.
> The __module__ is set correctly later on in the JCC code!
> Maybe you could add a package name (and a dot) to the typename to avoid these 
> warning?
> I am just reporting this for your convenience and maybe it helps others 
> seeing these warnings. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [jira] [Created] (PYLUCENE-59) Python warns about missing __module__, but means that type names have no '.' in them.

[Andi Vajda]

I did 'fix' that as much as possible by setting it later as you saw in the code 
already. The warning is harmless and a proper fix to workaround what looks like 
a kludge in Python is more involved than the problem is worth.

Andi..

> On Jul 19, 2021, at 12:03, Erik Groeneveld (Jira)  wrote:
> 
> Erik Groeneveld created PYLUCENE-59:
> ---
> 
> Summary: Python warns about missing __module__, but means that 
> type names have no '.' in them.
> Key: PYLUCENE-59
> URL: https://issues.apache.org/jira/browse/PYLUCENE-59
> Project: PyLucene
>  Issue Type: Improvement
>Reporter: Erik Groeneveld
> 
> 
> When starting JCC, Python emits warnings such as
> {code:java}
> DeprecationWarning: builtin type Object has no __module__ attribute
> {code}
> It does this because, early in de process of creating types, it does not find 
> a '.' in de name of the type. The warning is somewhat misleading. The code 
> from Python is (fragment from typeobject.c): 
> {code:java}
> /* Set type.__module__ */
> s = strrchr(spec->name, '.');
> if (s != NULL) {
> int err;
> modname = PyUnicode_FromStringAndSize(
> spec->name, (Py_ssize_t)(s - spec->name));
> if (modname == NULL) {
> goto fail;
> }
> err = _PyDict_SetItemId(type->tp_dict, ___module__, modname);
> Py_DECREF(modname);
> if (err != 0)
> goto fail;
> } else {
> if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1,
> "builtin type %.200s has no __module__ attribute",
> spec->name))
> goto fail;
> }
> {code}
> The name of the types in JCC do not include a package name and hence no dot.
> 
> Python 3.10 still does it like this.
> 
> The __module__ is set correctly later on in the JCC code!
> 
> Maybe you could add a package name (and a dot) to the typename to avoid these 
> warning?
> 
> I am just reporting this for your convenience and maybe it helps others 
> seeing these warnings. 
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)

[ANNOUNCE] Apache PyLucene 8.9.0

[Andi Vajda]

I am pleased to announce the availability of Apache PyLucene 8.9.0.

Apache PyLucene, a subproject of Apache Lucene, is a Python extension for
accessing Apache Lucene Core. Its goal is to allow you to use Lucene's text
indexing and searching capabilities from Python. It is API compatible with
Lucene 8.x Core, version 8.9.0.

For changes in this release, please review:
http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_8_9_0/CHANGES
http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_8_9_0/jcc/CHANGES
http://lucene.apache.org/core/8_9_0/changes/Changes.html

Apache PyLucene is available from the following download page:
http://www.apache.org/dyn/closer.cgi/lucene/pylucene/pylucene-8.9.0-src.tar.gz

When downloading from a mirror site, please remember to verify the downloads
using signatures found on the Apache site:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS

For more information on Apache PyLucene, visit the project home page:
  http://lucene.apache.org/pylucene

Andi..

Re: [VOTE] Release PyLucene 8.9.0

[Andi Vajda]

This vote now has passed.
Thank you all who voted, PyLucene users and PMC members !

Andi..

On Thu, 17 Jun 2021, Andi Vajda wrote:



The PyLucene 8.9.0 (rc1) release tracking today's release of
Apache Lucene 8.9.0 is ready.

A release candidate is available from:
  https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.9.0-rc1/

PyLucene 8.9.0 is built with JCC 3.10, included in these release artifacts.

JCC 3.10 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.9.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[VOTE] Release PyLucene 8.9.0

[Andi Vajda]

The PyLucene 8.9.0 (rc1) release tracking today's release of
Apache Lucene 8.9.0 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.9.0-rc1/

PyLucene 8.9.0 is built with JCC 3.10, included in these release artifacts.

JCC 3.10 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.9.0.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

[jira] [Commented] (PYLUCENE-58) SEGV on import lucene

[Andi Vajda (Jira)]

[ 
https://issues.apache.org/jira/browse/PYLUCENE-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359016#comment-17359016
 ] 

Andi Vajda commented on PYLUCENE-58:


I did not reproduce the issue you reported but I still fixed a couple of bugs, 
one serious, that I found by running JCC with Python 3.9.5 configured 
--with-pydebug. Namely, the makeType() function was
not INCREF'ing types it returned when they existed already, which caused their 
refcount to drop to 0, not a good thing at all. I hadn't run with dbg python in 
a long time, thank you for pointing me to it.
The fixes are checked into rev 1890581., please try them out and let me know if 
you still see the original issue.
Thanks !

> SEGV on import lucene
> -
>
> Key: PYLUCENE-58
> URL: https://issues.apache.org/jira/browse/PYLUCENE-58
> Project: PyLucene
>  Issue Type: Bug
> Environment: Debian Buster, Python 3.7
>Reporter: Erik Groeneveld
>Priority: Critical
>
> Hi Andy,
> Thanks again for your great work on PyLucene and JCC!
> Recently, after porting everything to python3, we get occasional SEGV's on 
> shutdown. It happens very late, when the garbage collector starts cleaning up.
> Using python3-dbg exposed another problem however. With python3-dbg, "import 
> lucene" already triggers SEGV. Here is the top of the backtrace:
>  
> {code:bash}
> #0  0x0060 in ?? ()
> #1  0x7fe8aee51d6e in unicode_fromformat_write_cstr 
> (writer=writer@entry=0x7ffdc0dcd170, str=, 
> width=width@entry=-1, precision=) at 
> ../Objects/unicodeobject.c:2596
> #2  0x7fe8aee525ec in unicode_fromformat_arg (vargs=0x7ffdc0dcd150, 
> f=, writer=0x7ffdc0dcd170) at ../Objects/unicodeobject.c:2797
> #3  PyUnicode_FromFormatV (format=, vargs=) at 
> ../Objects/unicodeobject.c:2914
> #4  0x7fe8aedca3dd in PyErr_FormatV (exception=, 
> format=0x7fe8aefe2568 "%s:%d: bad argument to internal function", 
> vargs=vargs@entry=0x7ffdc0dcd210) at ../Python/errors.c:835
> #5  0x7fe8aedca4a4 in PyErr_Format (exception=, 
> format=) at ../Python/errors.c:852
> #6  0x7fe8aee89fcd in PyDict_SetItem (op=, key= out>, value=) at ../Objects/dictobject.c:1448
> #7  PyDict_SetItem (op=, key=, value= out>, op=, key=, value=) at 
> ../Objects/dictobject.c:1443
> #8  0x7fe8aee76f4a in module_init_dict (md_dict= 0x7fe8ae9f6060>, name=name@entry=, 
> doc=None, doc@entry=0x0, mod=) at ../Objects/moduleobject.c:72
> #9  0x7fe8aee7da83 in PyModule_NewObject (name=name@entry= remote 0x7fe8ae9f5030>) at ../Objects/moduleobject.c:103
> #10 0x7fe8aee7de2a in PyModule_New (name=name@entry=0x7fe8b32bfa20 
> "lucene._lucene") at ../Objects/moduleobject.c:120
> #11 0x7fe8aee7deec in _PyModule_CreateInitialized (module=0x7fe8b2612080 
> <_lucene_def>, module_api_version=) at 
> ../Objects/moduleobject.c:215
> #12 0x7fe8b1238de7 in PyInit__lucene () from 
> /data/bouwen/van_kras/pylucene-8.6.1/build/test/lucene-8.6.1-py3.7-linux-x86_64.egg/lucene/_lucene.cpython-37m-x86_64-linux-gnu.so
> {code}
> It could be that this goes undetected with normal python, yet causes an SEGV 
> on shutdown.
>  
> The error above can be reproduced with the following script that downloads 
> the sources, builds JCC and PyLucene and the executes: python3-dbg -c "import 
> lucene"
>  
> {code:bash}
> # Environment
> # debian buster
> # ant 1.10.5-2
> export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
> export PYTHON=/usr/bin/python3
> export PYLUCENE="pylucene-8.6.1"
> rm ${PYLUCENE}-src.tar.gz ${PYLUCENE} -rf
> wget 
> https://ftp.nluug.nl/internet/apache/lucene/pylucene/${PYLUCENE}-src.tar.gz
> tar xzf ${PYLUCENE}-src.tar.gz
> (cd ${PYLUCENE}
> (cd jcc
> export JCC_JDK=${JAVA_HOME}
> export 
> JCC_INCLUDES=/usr/include/python3.7m:${JAVA_HOME}/include:${JAVA_HOME}/include/linux
> ${PYTHON} setup.py build
> )
> export NUM_FILES=10
> export ANT=/usr/bin/ant
> export JCC="${PYTHON} -m jcc --shared"
> make
> make test
> )
> PYTHONPATH='pylucene-8.6.1/build/test/lucene-8.6.1-py3.7-linux-x86_64.egg'
> ${PYTHON}-dbg -c "import lucene"
> {code}
> Would you be as kind as to look into this? Perhaps our problem is solved, or 
> it enables us to find an other problem at shutdown.
> Best regards,
> Erik



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: StoreField with long value using python3

[Andi Vajda]

On Tue, 20 Apr 2021, antony wrote:



Hello,

Anyone help me how to use StoredField(String name, long value). Its not
working because there is no 'long integer' in Python 3.


Not sure that's the problem.
It looks like the StoredField(int) overload is called before the 
StoredField(long). This is a bug.


A possible workaround would be to edit the StoredField class to add a 
constructor accepting a java.lang.Long() and then passing it a Long(cdate) ?

especially since all it does is set Object fieldsData on Field anyway...

I didn't try this out...

Andi..



I am using Python 3.8.0 and pylucene 8.6.1.

Source code:

cdate = int(datetime.datetime.now().strftime("%Y%m%d%H%M"))

print(1, cdate)

doc = Document()
doc.add(LongPoint('cdatetime', cdate))
doc.add(StoredField('cdatetime', cdate))

print(2, doc)

writer.addDocument(doc)
writer.commit()
writer.close()

Output:

1, cdate = 202104202312

2, Document
*stored*>


Thanks,
Antony



--
Sent from: http://pylucene-users-developers-list.2474766.n2.nabble.com/

[ANNOUNCE] Apache PyLucene 8.8.1

[Andi Vajda]

I am pleased to announce the availability of Apache PyLucene 8.8.1.

Apache PyLucene, a subproject of Apache Lucene, is a Python extension for
accessing Apache Lucene Core. Its goal is to allow you to use Lucene's text
indexing and searching capabilities from Python. It is API compatible with
Lucene 8.x Core, version 8.8.1.

For changes in this release, please review:
http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_8_8_1/CHANGES
http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_8_8_1/jcc/CHANGES
http://lucene.apache.org/core/8_8_1/changes/Changes.html

Apache PyLucene is available from the following download page:
http://www.apache.org/dyn/closer.cgi/lucene/pylucene/pylucene-8.8.1-src.tar.gz

When downloading from a mirror site, please remember to verify the downloads
using signatures found on the Apache site:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS

For more information on Apache PyLucene, visit the project home page:
  http://lucene.apache.org/pylucene

Andi..

Re: [VOTE] Release PyLucene 8.8.1

[Andi Vajda]

This vote has passed.
Thank you all who voted !

Andi..

On Mon, 8 Mar 2021, Michael McCandless wrote:


+1

I ran my usual smoke test: install JCC, PyLucene, then index and optimize
the first 100K documents from a Wikipedia English snapshot, and run a
couple queries.

Sorry for being late to the party too!

Mike McCandless

http://blog.mikemccandless.com


On Mon, Mar 1, 2021 at 9:35 PM Andi Vajda  wrote:



The PyLucene 8.8.1 (rc1) release tracking the recent release of
Apache Lucene 8.8.1 is ready.

A release candidate is available from:
https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.8.1-rc1/

PyLucene 8.8.1 is built with JCC 3.9, included in these release artifacts.

JCC 3.9 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.8.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: Amazon Sponsor Product(Ads) Search team is looking for talents with expertise in Solr/Lucene at all levels

[Andi Vajda]

On Wed, 3 Mar 2021, Alexander Yaworsky wrote:


All the best, pylucene is nice, especially JCC which we still use to
make other java libs friends with python.


Thank you for the kind words.
Good luck in your next adventure !

Andi..



Alexander.


On Wed, Mar 3, 2021 at 12:27 PM Alexander Yaworsky
 wrote:


Hi Pengcheng Xiong,

out of interest, why pylucene-dev?

Although I'm not a fan of ads and also recently moved all my personal
projects away from AWS, I almost lost my job so I'm free for a while.

I was working for a tiny US-based company remotely for last 15 years.
The business has declined because the owner had a stroke more than
year ago so my objective since then was to help him to maintain the
infrastructure till the acquisition by other company.

We used pylucene in one of our products, which indexed news from
various sources and allowed users to track their searches in real time
(we called them watches). Plus some analytics.

Not sure I can work on similar things, I need to check
non-solicitation section of my contract, there are some restrictions.
But general things are okay.

Alexander.

On Wed, Mar 3, 2021 at 10:37 AM  wrote:


Hi folks,

Are you interested in Amazon’s Advertising (that employs state of art
solutions involving Search, Digital Advertising, AWS technologies etc)
but wondering which team can best leverage your expertise in
Solr/Lucene? Please continue to read:

My team is Sponsored Search Delivery (SSD) where we directly use Solr to
efficiently render relevant Ads for shoppers. We deliver billions of ad
impressions and millions of clicks daily and are breaking fresh ground
to create world-class products. Powered with Solr, our systems scan
billions of records in milliseconds to return relevant ads. The
architectural excellence is evident from the fact that despite we’ve
such high-throughput-low-tps Tier-1 system, our team enjoys an
exceptionally low operations burden. Our solutions employ latest
‘information retrieval(IR)’ algorithms and we try to leverage latest
research in IR to ensure superior shopper experience. It’s a great mix
of ‘engineering’ and ‘applied-science’ to render relevant ads within
milliseconds.

My team is making tremendous investment in Solr research, development
and application this year and the future. For example, we plan to apply
vector search in Solr (SOLR-12890) to extend power of current
keyword-based search to improve ads recall. We are experimenting
approximate nearest vector search (LUCENE-9004) to achieve accuracy at a
reasonable cost. We also plan to leverage rich features of Solr to
return ads based on 'themes' (viz. brand-similar, economical, 4-star
above etc.). We also plan to share the experience and lessons to
Solr/Lucene community and find ways to contribute back to the community.

With a broad mandate to experiment and innovate, we are growing at an
unprecedented rate together with Amazon Ads business. There is a
seemingly endless range of new opportunities ahead of us. We’re looking
for amazing minds at various positions in team for New York Location
(Primary goal is to expand the team in NYC, However, Location
preferences of Boulder, Seattle, Toronto: can also be considered on case
by case basis). Prior experience in Solr/Lucene would be a great value add.
SDM: www.amazon.jobs/jobs/1427445?no_int_redir=1

SDE2: www.amazon.jobs/jobs/1451662?no_int_redir=1

SDE3:www.amazon.jobs/jobs/1357591?no_int_redir=1

ASII: www.amazon.jobs/jobs/1427080?no_int_redir=1

ASIII: www.amazon.jobs/jobs/1379670?no_int_redir=1

TPM: www.amazon.jobs/jobs/1353298?no_int_redir=1


If you are interested, please reply this email directly or reach out to
the hiring manager Vikas Dhaka (dhavi...@amazon.com
) Thanks!

Best

Pengcheng Xiong

(Apache Hive PMC member and committer working at Amazon)

[VOTE] Release PyLucene 8.8.1

[Andi Vajda]

The PyLucene 8.8.1 (rc1) release tracking the recent release of
Apache Lucene 8.8.1 is ready.

A release candidate is available from:
   https://dist.apache.org/repos/dist/dev/lucene/pylucene/8.8.1-rc1/

PyLucene 8.8.1 is built with JCC 3.9, included in these release artifacts.

JCC 3.9 supports Python 3.3 up to Python 3.9 (in addition to Python 2.3+).
PyLucene may be built with Python 2 or Python 3.

Please vote to release these artifacts as PyLucene 8.8.1.
Anyone interested in this release can and should vote !

Thanks !

Andi..

ps: the KEYS file for PyLucene release signing is at:
https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS

pps: here is my +1

Re: jcc - Output unbuilt package

[Andi Vajda]

 Hi Phil,

On Mon, 1 Mar 2021, Phil wrote:


Great - I've attached a one-line change that outputs the missing info to
stdout.  I haven't added a command line switch at this stage as there is
no functional change to the output - just an extra line of logging.


Yes, a flag is not necessary for this change.


Note this is necessary because although the program already outputs the
setup args which are also required, the Extension class doesn't render
its args as text on printing the setup args, so I also need to output
the inputs to the Extension so that I can capture all the inputs to setup().

This would be hugely useful for me if we could even this simple change.


I incorporated your change and committed into rev 1887063.


I will look at automating the full generation of a source package, and
adding an optional switch to control this - but this is a larger undertaking,
so I'll have to add it to the to-do list.  I'll let you know when I have
had a chance to implement this.


Great, thank you !

Andi..