[jira] [Commented] (PYLUCENE-28) JCC reuses JVM instances in impl, if compile() is called twice.
[ https://issues.apache.org/jira/browse/PYLUCENE-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736851#comment-17736851 ] Andi Vajda commented on PYLUCENE-28: I see 2 things: - if a constructor takes no arguments, like org.apache.lucene.document.Document(), then anything passed to it is ignored. - all constructor wrappers have a kwds parameter but it's ignored. Both could be fixed at the cost of extra code that raises an error if kwds is not empty. Also, extra code would have to be generated for checking that args is empty for contructores not taking any args. Is it worth it ? > JCC reuses JVM instances in impl, if compile() is called twice. > --- > > Key: PYLUCENE-28 > URL: https://issues.apache.org/jira/browse/PYLUCENE-28 > Project: PyLucene > Issue Type: Improvement > Environment: jcc-svn-head (2.18-pre) >Reporter: Martin Scherer >Priority: Blocker > Labels: patch > Attachments: jvm_instance_check.patch > > > If you import jcc.cpp to call compile yourself (a wrapped setup.py script to > generate a wrapper on the fly, which seems to be a common use case), the > current version complains about the JVM already being initialized before. > The patch first checks for a running instance and creates one, if none is > being found. > For myself, this is a blocker, since it raises otherwise. > Best, > Martin -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [jira] [Commented] (PYLUCENE-28) JCC reuses JVM instances in impl, if compile() is called twice.
Hi Andi, do you suppose that adding to the classpath by runtime is possible via python or a java workaround? I think it is very error prone if the user has to maintain his own classpath and can not rely on jcc. Best, Martin Am 04.11.2013 16:02, schrieb Andi Vajda (JIRA): [ https://issues.apache.org/jira/browse/PYLUCENE-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13812882#comment-13812882 ] Andi Vajda commented on PYLUCENE-28: A couple of comments. Your patch introduces a silent failure: if the second invocation of initVM() is made with different parameters (in particular, with a different classpath) people are going to be tearing their hair out with odd class loading errors because the 'second' classpath didn't take. Your patch should at least make sure that the classpath in the second invocation either matches the first or that the second classpath is forced into the existing VM (possible but hacky). I didn't intend jcc to be used this way because of the one Java VM per process limitation and I thus consider your bug a feature request. It's a worthwhile feature nonetheless but it needs to be implemented more solidly. Maybe a better way to support this would be to introduce a new entrypoint in cpp.py that takes a VMEnv. That way, there is no silent failure possible, the user knows what VMEnv they have and passed in. Or maybe the jcc() function could take an optional VMEnv parameter - and check that no VM configuration params are passed in via args (and error if there are) in that case ? JCC reuses JVM instances in impl, if compile() is called twice. --- Key: PYLUCENE-28 URL: https://issues.apache.org/jira/browse/PYLUCENE-28 Project: PyLucene Issue Type: Improvement Environment: jcc-svn-head (2.18-pre) Reporter: Martin Scherer Priority: Blocker Labels: patch Attachments: jvm_instance_check.patch If you import jcc.cpp to call compile yourself (a wrapped setup.py script to generate a wrapper on the fly, which seems to be a common use case), the current version complains about the JVM already being initialized before. The patch first checks for a running instance and creates one, if none is being found. For myself, this is a blocker, since it raises otherwise. Best, Martin -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: [jira] [Commented] (PYLUCENE-28) JCC reuses JVM instances in impl, if compile() is called twice.
Hi,
I've implemented your second suggestion and made the env an optional
parameter to the jcc function. In addition there are performed some
checks for classpath and vmargs for sanity.
This fixes the silent failure, when a different classpath should be
used. I didn't figure out, how to add dynamically to the existing
runtime, which is like you said very hackish.
Best,
Martin
Am 04.11.2013 16:02, schrieb Andi Vajda (JIRA):
[
https://issues.apache.org/jira/browse/PYLUCENE-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13812882#comment-13812882
]
Andi Vajda commented on PYLUCENE-28:
A couple of comments. Your patch introduces a silent failure: if the second
invocation of initVM() is made with different parameters (in particular, with
a different classpath) people are going to be tearing their hair out with odd
class loading errors because the 'second' classpath didn't take. Your patch
should at least make sure that the classpath in the second invocation either
matches the first or that the second classpath is forced into the existing VM
(possible but hacky).
I didn't intend jcc to be used this way because of the one Java VM per
process limitation and I thus consider your bug a feature request. It's a
worthwhile feature nonetheless but it needs to be implemented more solidly.
Maybe a better way to support this would be to introduce a new entrypoint in
cpp.py that takes a VMEnv. That way, there is no silent failure possible, the
user knows what VMEnv they have and passed in. Or maybe the jcc() function
could take an optional VMEnv parameter - and check that no VM configuration
params are passed in via args (and error if there are) in that case ?
JCC reuses JVM instances in impl, if compile() is called twice.
---
Key: PYLUCENE-28
URL: https://issues.apache.org/jira/browse/PYLUCENE-28
Project: PyLucene
Issue Type: Improvement
Environment: jcc-svn-head (2.18-pre)
Reporter: Martin Scherer
Priority: Blocker
Labels: patch
Attachments: jvm_instance_check.patch
If you import jcc.cpp to call compile yourself (a wrapped setup.py script to
generate a wrapper on the fly, which seems to be a common use case), the
current version complains about the JVM already being initialized before.
The patch first checks for a running instance and creates one, if none is
being found.
For myself, this is a blocker, since it raises otherwise.
Best,
Martin
--
This message was sent by Atlassian JIRA
(v6.1#6144)
Index: jcc/cpp.py
===
--- jcc/cpp.py (revision 1537185)
+++ jcc/cpp.py (working copy)
@@ -338,9 +338,8 @@
jar.close()
+def jcc(args, env = None):
-def jcc(args):
-
classNames = set()
listedClassNames = set()
listedMethodNames = {}
@@ -363,7 +362,6 @@
sequences = {}
renames = {}
use_full_names = False
-env = None
wrapperFiles = 1
prefix = None
root = None
@@ -405,9 +403,13 @@
i += 1
libpath.append(args[i])
elif arg == '--vmarg':
+if env != None:
+raise ValueError('option --vmarg ignored, since env given')
i += 1
vmargs.append(args[i])
elif arg == '--maxheap':
+if env != None:
+raise ValueError('option --maxheap ignored, since env given')
i += 1
initvm_args['maxheap'] = args[i]
elif arg == '--python':
@@ -517,9 +519,16 @@
initvm_args['maxstack'] = '512k'
initvm_args['vmargs'] = vmargs
+
+if env is None: # init only if not given
+env = initVM(os.pathsep.join(classpath) or None, **initvm_args)
+else: # check cp's from current env and given by arguments are matching
+env_cp = env.classpath.split(':')
+from os.path import abspath
+if [abspath(f) for f in env_cp] != [abspath(f) for f in classpath]:
+raise RuntimeError('classpath differs since last call:\n'
+ 'previous = %s;\n now = %s' % (env_cp, classpath))
-env = initVM(os.pathsep.join(classpath) or None, **initvm_args)
-
typeset = set()
excludes = set(excludes)
Re: [jira] [Commented] (PYLUCENE-28) JCC reuses JVM instances in impl, if compile() is called twice.
On Wed, 18 Dec 2013, Martin Scherer wrote: Hi Andi, do you suppose that adding to the classpath by runtime is possible via python or a java workaround? You can see the horrible kludge implemented in JCCEnv::setClassPath(). It is possible to modify the classpath at runtime. I think it is very error prone if the user has to maintain his own classpath and can not rely on jcc. Yes, classpaths are very error prone. Andi.. Best, Martin Am 04.11.2013 16:02, schrieb Andi Vajda (JIRA): [ https://issues.apache.org/jira/browse/PYLUCENE-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13812882#comment-13812882 ] Andi Vajda commented on PYLUCENE-28: A couple of comments. Your patch introduces a silent failure: if the second invocation of initVM() is made with different parameters (in particular, with a different classpath) people are going to be tearing their hair out with odd class loading errors because the 'second' classpath didn't take. Your patch should at least make sure that the classpath in the second invocation either matches the first or that the second classpath is forced into the existing VM (possible but hacky). I didn't intend jcc to be used this way because of the one Java VM per process limitation and I thus consider your bug a feature request. It's a worthwhile feature nonetheless but it needs to be implemented more solidly. Maybe a better way to support this would be to introduce a new entrypoint in cpp.py that takes a VMEnv. That way, there is no silent failure possible, the user knows what VMEnv they have and passed in. Or maybe the jcc() function could take an optional VMEnv parameter - and check that no VM configuration params are passed in via args (and error if there are) in that case ? JCC reuses JVM instances in impl, if compile() is called twice. --- Key: PYLUCENE-28 URL: https://issues.apache.org/jira/browse/PYLUCENE-28 Project: PyLucene Issue Type: Improvement Environment: jcc-svn-head (2.18-pre) Reporter: Martin Scherer Priority: Blocker Labels: patch Attachments: jvm_instance_check.patch If you import jcc.cpp to call compile yourself (a wrapped setup.py script to generate a wrapper on the fly, which seems to be a common use case), the current version complains about the JVM already being initialized before. The patch first checks for a running instance and creates one, if none is being found. For myself, this is a blocker, since it raises otherwise. Best, Martin -- This message was sent by Atlassian JIRA (v6.1#6144)
