[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183577#comment-17183577
]
Andreas Vajda commented on PYLUCENE-55:
---
All right, marking this as fixed then !
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182258#comment-17182258
]
Andrea Sterbini commented on PYLUCENE-55:
-
What a beautiful solution! Thanks a lot for your help!
Now it works, I had just to add the missing packages.
There is even no need to --use-full-names
My Makefile now looks like
{code:java}
// Makefile
CPLUS_INCLUDE_PATH:=/opt/anaconda3/envs/MLC/include:/opt/anaconda3/envs/MLC/include/linux
PREFIX_PYTHON=/opt/anaconda3/envs/MLC
PYTHON=$(PREFIX_PYTHON)/bin/python
JCC=CPLUS_INCLUDE_PATH=$(CPLUS_INCLUDE_PATH) $(PYTHON) -m jcc --shared --arch
x86_64 --wheel
NUM_FILES=10
JARS= --jar babelnet-api-4.0.1.jar
JARS+= --jar lib/babelscape-data-commons-1.0.jar
INCLUDE+=--include lib/commons-beanutils-1.7.0.jar
INCLUDE+=--include lib/commons-beanutils-core-1.7.0.jar
INCLUDE+=--include lib/commons-codec-1.8.jar
INCLUDE+=--include lib/commons-collections-3.2.jar
INCLUDE+=--include lib/commons-configuration-1.5.jar
INCLUDE+=--include lib/commons-digester-1.8.jar
INCLUDE+=--include lib/commons-lang-2.3.jar
INCLUDE+=--include lib/commons-logging-1.1.jar
INCLUDE+=--include lib/gson-2.8.2.jar
INCLUDE+=--include lib/guava-23.0.jar
INCLUDE+=--include lib/httpclient-4.3.6.jar
INCLUDE+=--include lib/httpcore-4.3.3.jar
INCLUDE+=--include lib/icu4j-56.1.jar
INCLUDE+=--include lib/jwi-2.2.3.jar
INCLUDE+=--include lib/lcl-jlt-2.4.jar
INCLUDE+=--include lib/logback-classic-1.2.3.jar
INCLUDE+=--include lib/logback-core-1.2.3.jar
INCLUDE+=--include lib/lucene-analyzers-common-7.2.0.jar
INCLUDE+=--include lib/lucene-core-7.2.0.jar
INCLUDE+=--include lib/lucene-queries-7.2.0.jar
INCLUDE+=--include lib/lucene-queryparser-7.2.0.jar
INCLUDE+=--include lib/lucene-sandbox-7.2.0.jar
INCLUDE+=--include lib/slf4j-api-1.7.25.jar
PACKAGES+=--package it.uniroma1.lcl.babelnet
PACKAGES+=--package it.uniroma1.lcl.jlt.util
PACKAGES+=--package it.uniroma1.lcl.jlt
PACKAGES+=--package com.babelscape
PACKAGES+=--package com.babelscape.pipeline.annotation.maps
PACKAGES+=--package com.babelscape.pipeline.annotation
PACKAGES+=--package com.babelscape.pipeline
PACKAGES+=--package java.util
MAPPING=
compile:
$(JCC) $(INCLUDE) $(JARS) $(MAPPING) $(PACKAGES) --python babelnet
--build --version 4.0.1
install:
$(JCC) $(INCLUDE) $(JARS) $(MAPPING) $(PACKAGES) --python babelnet
--install --version 4.0.1
{code}
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182184#comment-17182184
]
Andreas Vajda commented on PYLUCENE-55:
---
All right, I wrote JCC a long time ago so it takes a bit of time to page its
inner workings back in.
The issue is that JCC tries really hard not to generate wrappers for the entire
universe of Java classes reachable, transitively, from your requested set of
wrappers. This process is recursive and order dependent, thus bugs in it can be
tricky to reproduce as we just saw.
The issue that triggered this bug is that you have some generics on classes or
methods you want wrapped but with generic parameters that were not in the set
of things to be wrapped either directly (via --jar or a direct classname) or
indirectly (via --package).
These missing dependencies cause JCC to drop methods using them. This is an
important feature as it
gives you control over the the amount of wrappers generated.
The situation here is a mix or found dependency (a generic class) and not found
dependency (a parameter to that generic class). I made it an error to be in
that situation and this new error can be remedied via additions of --package
parameters. In your case, you are missing --package java.util and --package
it.uniroma1.lcl.jlt.util.
Once these two package flags are added to the JCC invocation the BabelNet class
gets all its 28 getSynsets() methods wrapped. This also explains why you didn't
see any of your mappings since java.util defines a lot of that.
The fix is checked into the PyLucene trunk svn repository at apache:
https://svn.apache.org/viewvc/lucene/pylucene/trunk/jcc/
Please, try this out and let me know how it works for you.
Thanks !
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182181#comment-17182181
]
Andreas Vajda commented on PYLUCENE-55:
---
Wow, there are 28 overloads for getSynsets() in
it.uniroma1.lcl.babelnet.BabelNet (!)
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182162#comment-17182162
]
Andrea Sterbini commented on PYLUCENE-55:
-
Very good, thanks!
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182110#comment-17182110
]
Andreas Vajda commented on PYLUCENE-55:
---
I tried a bunch of times to build and I'm now in a situation where I reproduced
the issue.
I see only getSynsets() overloads instead of the 9 expected ones.
Now that I can reproduce this, I can debug this...
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182107#comment-17182107
]
Andreas Vajda commented on PYLUCENE-55:
---
I'm using java --version
openjdk 11.0.7 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.7+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.7+10, mixed mode)
My python is version 3.8.2, I'm on macos 10.15.6.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182098#comment-17182098
]
Andrea Sterbini commented on PYLUCENE-55:
-
Could you list your test environment?
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182094#comment-17182094
]
Andrea Sterbini commented on PYLUCENE-55:
-
Fift test: same than the fourth but with line 694 of cpp.py changed to scan the
classes contained in 'todo' in name order
Success: now the BabelNet class has all her 9 getSynsets methods (in BabelNet.h)
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182091#comment-17182091
]
Andrea Sterbini commented on PYLUCENE-55:
-
Fourth test: empty virtualenv with Python 3.8.5 (Fedora fc32), java jdk
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.265.b01-1.fc32.x86_64, jcc 3.7 installed
with pip, make with --use-full-names
Same result (only 3 getSynsets methods with wrong signatures in BabelNet.h)
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182059#comment-17182059
]
Andrea Sterbini commented on PYLUCENE-55:
-
Third test: pip-installed jcc, build babelnet with --use-full-names
Same result
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182058#comment-17182058
]
Andrea Sterbini commented on PYLUCENE-55:
-
Second test: disinstalled the pre-packeged JCC and reinstalled with pip
Same result
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182054#comment-17182054
]
Andrea Sterbini commented on PYLUCENE-55:
-
First test: Anaconda with a freshly minted new environment (python 3.8.5, jcc
3.7 as distributed by conda-forge)
* conda create -n bn jcc # new
* . activate bn
* make
The resulting BabelNet class has only 3 getSynsets methods instead than plenty.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182030#comment-17182030
]
Andreas Vajda commented on PYLUCENE-55:
---
> Did you try to build it more than once?
No, I just looked into the build directory from yesterday's build, which I
built only once.
> What version of jcc are you using? My version has no --wheel option
JCC version 3.7, which was released with PyLucene 8.3.0 (the latest release of
PyLucene/JCC).
Also available from PyPI: https://pypi.org/project/JCC/
You can find the version of JCC you're using: from babelnet import JCC_VERSION
> I'll try again with a completely clean python environment, both with pip and
> Anaconda and let you know.
Note that I'm not using Anaconda (what is that ?) but regular python 3.8.2.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182007#comment-17182007
]
Andrea Sterbini commented on PYLUCENE-55:
-
Did you try to build it more than once?
What version of jcc are you using? My version has no --wheel option
I'll try again with a completely clean python environment, both with pip and
Anaconda and let you know.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181996#comment-17181996
]
Andreas Vajda commented on PYLUCENE-55:
---
You ask:
> What is the reason for so different generated modules?
I don't know yet since I've not been able to reproduce the problem as reported.
> How the order of class wrapper generation influences the resulting methods?
The order of method generation should not matter.
> Why some methods sometimes are NON generated?
So far, all methods you told me about were generated in my build. I have not
been able to
reproduce the problem as reported yet. However, by default, only public methods
get wrappers.
It seems that all the methods in question so far are indeed public.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181993#comment-17181993
]
Andreas Vajda commented on PYLUCENE-55:
---
I see 9 getSynsets() methods declared in BabelNet.h in my build:
{code}
::java::util::List getSynsets(const JArray<
::it::uniroma1::lcl::kb::ResourceID > &) const;
::java::util::List getSynsets(const ::java::lang::String &) const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &) const;
::java::util::List getSynsets(const ::java::lang::String &, jboolean)
const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &, jboolean) const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &, const ::com::babelscape::util::POS
&) const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &, const ::com::babelscape::util::POS
&, const JArray< ::it::uniroma1::lcl::babelnet::data::BabelSenseSource > &)
const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &, const ::com::babelscape::util::POS
&, jboolean) const;
::java::util::List getSynsets(const ::java::lang::String &, const
::it::uniroma1::lcl::jlt::util::Language &, const ::com::babelscape::util::POS
&, jboolean, const JArray<
::it::uniroma1::lcl::babelnet::data::BabelSenseSource > &) const;
{code}
In other words, I have not been able to reproduce the problem you reported.
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181772#comment-17181772
]
Andrea Sterbini commented on PYLUCENE-55:
-
Thanks for the solution.
Still sometimes I find that something is missing. E.g. one of the methods I
need is it.uniroma1.lcl.babelnet.BabelNet.getSynsets(...), which has many
implementations (I need at least the first three)
{code:java}
// from it/uniroma1/lcl/babelnet/BabelNet.class
public java.util.List
getSynsets(java.lang.String);
public java.util.List
getSynsets(java.lang.String, it.uniroma1.lcl.jlt.util.Language);
public java.util.List
getSynsets(java.lang.String, it.uniroma1.lcl.jlt.util.Language,
com.babelscape.util.POS);
public java.util.List
getSynsets(java.lang.String,
java.util.Collection);
public java.util.List
getSynsets(java.lang.String,
java.util.Collection,
com.babelscape.util.POS);
public java.util.List
getSynsets(it.uniroma1.lcl.kb.ResourceID...);
{code}
Yet, after building and installing the module the resulting BabelNet class
shows only 3 versions of the method
{code:java}
// from ... build/_babelnet/it/uniroma1/lcl/babelnet/BabelNet.h
::java::util::List getSynsets(const JArray<
::it::uniroma1::lcl::kb::ResourceID > &) const;
::java::util::List getSynsets(const ::java::lang::String &) const;
::java::util::List getSynsets(const ::java::lang::String &, jboolean)
const;
{code}
I have tried also to add the methods I need to the mappings but the result is
the same, non-deterministic, the resulting module could have or could have not
the methods.
{code:java}
// from the Makefile
MAPPING=
MAPPING+=--mapping it/uniroma1/lcl/babelnet/BabelNet
'getSynsets:(Ljava/lang/String;Lit/uniroma1/lcl/jlt/util/Language;)Ljava/lang/List;'
MAPPING+=--mapping it/uniroma1/lcl/babelnet/BabelNet
'getSynsets:(Ljava/lang/String;Lit/uniroma1/lcl/jlt/util/Language;com/babelscape/util/POS;)Ljava/lang/List;'
MAPPING+=--mapping it/uniroma1/lcl/babelnet/BabelSynset
'getLemmas:()Ljava/lang/List;'
{code}
What is the reason for so different generated modules?
How the order of class wrapper generation influences the resulting methods?
Why some methods sometimes are NON generated?
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179317#comment-17179317
]
Andreas Vajda commented on PYLUCENE-55:
---
Thank you, that's very useful for me to try to reproduce this !
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179256#comment-17179256
]
Andrea Sterbini commented on PYLUCENE-55:
-
Hi,
with "not working" I mean that some of the methods of the java classes are
missing in the resulting wrapper depending on the order the classes are
processed by cpp.py .
I agree with you that my patch is NOT a solution, I wrote it only to see if the
order mattered, and for this specific library the order seems to matter.
To reproduce the error: (my environment is: 16G RAM, Fedora Linux 32, Anaconda
python 3.7 64 bit, jcc 3.7)
* download and unzip [https://babelnet.org/data/4.0/BabelNet-API-4.0.1.zip]
* copy the Makefile below in the resulting directory and build and install
the module (make)
* The resulting module sometimes comes *without* the method getLemmas of the
BabelSynset class, sometimes *with*
** *ipython*
** *from babelnet import **
** *BabelSynset.getLemmas() # sometimes this method is missing, sometimes
not*
(notice, the code above just checks if the method is present)
By sorting the classes at line 696 of cpp.py I get the method always (but I
have not checked if other stuff is missing)
I am not so well versed in all jcc details but I hope this bug-report will tell
you where to look.
This is my Makefile
{code:java}
JNI_DIR:=/usr/lib/jvm/java
CPLUS_INCLUDE_PATH:=$(JNI_DIR)/include:$(JNI_DIR)/include/linux
JARS= --jar babelnet-api-4.0.1.jar
JARS+= --jar lib/babelscape-data-commons-1.0.jar
CP=config:lib:lib/commons-beanutils-1.7.0.jar:lib/commons-beanutils-core-1.7.0.jar:lib/commons-codec-1.8.jar:lib/commons-collections-3.2.jar:lib/commons-configuration-1.5.jar:lib/commons-digester-1.8.jar:lib/commons-lang-2.3.jar:lib/commons-logging-1.1.jar:lib/gson-2.8.2.jar:lib/guava-23.0.jar:lib/httpclient-4.3.6.jar:lib/httpcore-4.3.3.jar:lib/icu4j-56.1.jar:lib/jwi-2.2.3.jar:lib/lcl-jlt-2.4.jar:lib/logback-classic-1.2.3.jar:lib/logback-core-1.2.3.jar:lib/lucene-analyzers-common-7.2.0.jar:lib/lucene-core-7.2.0.jar:lib/lucene-queries-7.2.0.jar:lib/lucene-queryparser-7.2.0.jar:lib/lucene-sandbox-7.2.0.jar:lib/slf4j-api-1.7.25.jar
INCLUDE+=--include lib/commons-beanutils-1.7.0.jar
INCLUDE+=--include lib/commons-beanutils-core-1.7.0.jar
INCLUDE+=--include lib/commons-codec-1.8.jar
INCLUDE+=--include lib/commons-collections-3.2.jar
INCLUDE+=--include lib/commons-configuration-1.5.jar
INCLUDE+=--include lib/commons-digester-1.8.jar
INCLUDE+=--include lib/commons-lang-2.3.jar
INCLUDE+=--include lib/commons-logging-1.1.jar
INCLUDE+=--include lib/gson-2.8.2.jar
INCLUDE+=--include lib/guava-23.0.jar
INCLUDE+=--include lib/httpclient-4.3.6.jar
INCLUDE+=--include lib/httpcore-4.3.3.jar
INCLUDE+=--include lib/icu4j-56.1.jar
INCLUDE+=--include lib/jwi-2.2.3.jar
INCLUDE+=--include lib/lcl-jlt-2.4.jar
INCLUDE+=--include lib/logback-classic-1.2.3.jar
INCLUDE+=--include lib/logback-core-1.2.3.jar
INCLUDE+=--include lib/lucene-analyzers-common-7.2.0.jar
INCLUDE+=--include lib/lucene-core-7.2.0.jar
INCLUDE+=--include lib/lucene-queries-7.2.0.jar
INCLUDE+=--include lib/lucene-queryparser-7.2.0.jar
INCLUDE+=--include lib/lucene-sandbox-7.2.0.jar
INCLUDE+=--include lib/slf4j-api-1.7.25.jar
MAPPING+=--mapping it/uniroma1/lcl/babelnet/BabelNetQuery/Builder
'from:(Lit/uniroma1/lcl/babelnet/Language;)Lit/uniroma1/lcl/babelnet/BabelNetQuery/Builder;'
MAPPING+=--mapping it/uniroma1/lcl/babelnet/BabelNet
'getSynsets:(Lit/uniroma1/lcl/babelnet/BabelNetQuery;)Ljava/lang/List;'
JCC:=python -m jcc
all:
CPLUS_INCLUDE_PATH=$(CPLUS_INCLUDE_PATH) $(JCC) $(INCLUDE) $(JARS)
$(MAPPING) --python babelnet --build --install --version 4.0.1
{code}
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (PYLUCENE-55) JCC creates the classes in non-deterministic order
[
https://issues.apache.org/jira/browse/PYLUCENE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179215#comment-17179215
]
Andreas Vajda commented on PYLUCENE-55:
---
Can you be more specific about "not working" ?
Why does the order of classes matter ?
As it is, why should I accept your patch, maybe someone else is depending on
the order to be random,
or the reverse of yours ? Seriously, I'd like to better understand what is
going on here.
Please, help me help you by:
- describing or quoting the error, "not working" is not good enough.
- include a way to reproduce this, with your complete JCC command line.
There has to go be something more broken here than that the sorted order you're
proposing is hiding.
Thanks !
> JCC creates the classes in non-deterministic order
> --
>
> Key: PYLUCENE-55
> URL: https://issues.apache.org/jira/browse/PYLUCENE-55
> Project: PyLucene
> Issue Type: Bug
>Reporter: Andrea Sterbini
>Priority: Major
>
> I am trying to wrap the BabelNet API code.
> The resulting module is non-deterministically not-working (once every 5 I get
> it OK).
> This seems to be related to the order the classes are handled, because they
> are kept in a set, which has nondeterministic order.
> By changing cpp.py at line 696 to sort the class names I get a working module.
> {code:java}
> // changed from
> for cls in todo:
> {code}
> {code:java}
> // to
> for cls in sorted(todo, key=lambda c: c.getName()):{code}
> I have been luky with this way to order the classes. Possibly a better
> algorithm exists to fix this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
