[Help-glpk] RE: GLPK on 64 bit Linux
Hi,
You may recall some time back that I had issues with
numerical differences running GLPK-Java on 64 bit Linux that could not be
explained by simply a different floating point precision. Some months ago, I
believe I found an issue, and wanted to share it with everybody, sorry didn
#8217;t
post it sooner, just slipped my mind.
If you look at the GKPK JNI #8216;C #8217; bridging file
in GLPK Java, called glpk_jni.c, you can see casting in the methods, for
example:
JNIEXPORT jdouble JNICALL
Java_org_gnu_glpk_GlpkSolver_getRowDual(JNIEnv *env,
jobject obj, jint i)
{
return
(jint)lpx_get_row_dual(get_lpx(env, obj), (int)i);
}
Notice that it returns a jdouble, but the casting is to a
jint.
If we look in the jni.h file, and in turn the jni_md.h
which it includes, we can see that jint is typedefed to a jsize and in turn to
an int. On a 64 bit platform, sometimes an int can remain at 32 bit.
If you look at all the methods, you will notice that
whether the methods return a jdouble, or a jint, they are *always* cast to a
jint, perhaps a cut and paste error. The casting to jint in 32 bit does not
seem to be an issue, not sure why as I thought that casting to an int would
lose the decimal part regardless but it seems to be passing through the bit
pattern unchanged, which is then returned and interpreted as a floating point
number. Even on a 32 bit platform, doubles are normally 64 bit in 2x32 bit
words, so one would think that would be still an issue in 32 bit but it seems
to work on 32 bit.
Here are some of the problematic JNI signatures,
interesting bits are bolded. When I changed the (jint) casts to (jdouble)
casts, recompiled and reran on 64 bit, I did not get the errors I got before,
it seemed to solve the problem.
JNIEXPORT jdouble JNICALL
Java_org_gnu_glpk_GlpkSolver_getRowDual(JNIEnv *env,
jobject obj, jint i)
{
return (jint)lpx_get_row_dual(get_lpx(env,
obj), (int)i);
}
JNIEXPORT jdouble JNICALL
Java_org_gnu_glpk_GlpkSolver_getColPrim(JNIEnv *env,
jobject obj, jint j)
{
return (jint)lpx_get_col_prim(get_lpx(env,
obj), (int)j);
}
JNIEXPORT jdouble JNICALL
Java_org_gnu_glpk_GlpkSolver_getColDual(JNIEnv *env,
jobject obj, jint j)
{
return (jint)lpx_get_col_dual(get_lpx(env,
obj), (int)j);
}
Paul
From: Hammond, Paul
(IDEAS)
Sent: 25 March 2010 11:24
To: 'help-glpk@gnu.org'
Cc: 'bug-g...@gnu.org'
Subject: RE: GLPK on 64 bit Linux
Correcting the bug-glpk address,
I had a stray dot at the end.
From: Hammond, Paul
(IDEAS)
Sent: 25 March 2010 11:01
To: 'help-glpk@gnu.org'
Cc: 'bug-g...@gnu.org.'
Subject: GLPK on 64 bit Linux
Hi,
I #8217;m including bug-glpk here as this
is potentially a bug, but it could be something we are doing wrong.
We are using GLPK with Java
(glpk-java-1.0.5), but to date have been using it in a 32 bit Linux environment
and it #8217;s been fine (apart from crashes which we put down to the fact
it #8217;s not thread safe)
Now we are trying to use it in a 64 bit
Linux environment and encountering some strange issues and I #8217;m just
wondering if anybody else has encountered these and how we might solve it.
To move to 64 bit, we recompiled the C code
on 64 bit to produce a new shared object linked to it. It runs and executes no
problem. For most of our inputs, it gives the same outputs as before, save for
some very small diffs which I put down to the different floating point
rounding.
However, there are cases where a few of the
numbers differ by amounts that cannot be put down to rounding. We have
sometimes a fraction like .8xxx going to 0, or we have large numbers differing
by 10-15%. Again, this is only in very few of the output variables out of quite
a lot (all the others match fine) and many sets of inputs have no problem but
it #8217;s enough to pose an issue if it goes wrong for even some of our
inputs.
We have dumped the inputs to a .dat file to
confirm the inputs are the same going in for both 32 bit and 64 bit. It #8217;s
the same Java software calling the solver so this should be the case.
Does anybody know about such issues with 64
bit and the cause? Is there some particular compile flags or switches we need
to use? Is there anywhere on the net a confirmed working binary version of GLPK
for Java on 64 bit?
If this is completely new and you would
like to investigate, what would you need to be supplied with as a test case to
look into the problem?
Thanks,
Paul
NOTICE: If you have received this communication in error, please destroy all
electronic and paper copies and notify the sender immediately. Mistransmission
is not intended to waive confidentiality or privilege. Morgan Stanley reserves
the right, to the extent permitted under applicable law, to monitor electronic
communications. This message is subject to terms available at the following
link: http://www.morganstanley.com/disclaimers. If you
RE: [Help-glpk] Thread safety of GLPK
Andrew. I'm thinking since it is written in C, and C is source compatible with C++, since C++ does support locking in threads, if one was to treat it as a C++ app written mostly in C, it may be possible to multi-thread it without a huge rewrite? I say this without any knowledge of the GLPK source at all or how it works internally, so please bear my comments in that light, but in terms of locking, if the data structures being used for each computation are completely separate copies in each thread, I'm not sure where the demand for locking arises, perhaps you could elaborate a little? To get thread safety now, I can synchronize access to the JNI lib in Java to enforce single-threadness, but of course it means if I have multiple computations wanting to use GLPK, which I do, they have to queue for it and I have a central bottleneck. So I guess I just don't mean thread safe, I mean thread hot, as in I can have multiple GLPK computations going in separate threads simultaneously which don't need to synchronize on anything or if they do, it's for a very small part of the computation rather than the entire duration. Thx. Paul -Original Message- From: Andrew Makhorin [mailto:m...@gnu.org] Sent: 13 April 2010 20:05 To: Hammond, Paul (IDEAS) Cc: help-glpk@gnu.org Subject: Re: [Help-glpk] Thread safety of GLPK I #8217;d like to know if GLPK (the C lib) is not thread safe? No, glpk is not thread safe. If not, are there any plans to ever make it thread safe? We get occasional core dumps in a multi-threaded environment which we think are related to thread safety as we get a SIGSEGV inside GLPK. Thread safety (unlike re-enterability) requires locking/unlocking program objects used by concurrent threads. Since the standard C does not provide such a feature, there are no plans in this direction. I'm just wondering how you would use glpk if it were thread safe? Thanks. Andrew Makhorin -- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. ___ Help-glpk mailing list Help-glpk@gnu.org http://lists.gnu.org/mailman/listinfo/help-glpk
RE: [Help-glpk] Thread safety of GLPK
Yes, I was referred to separate C++ threading libraries that give you threading support in C++. Paul -Original Message- From: Michael Hennebry [mailto:henne...@web.cs.ndsu.nodak.edu] Sent: 14 April 2010 14:51 To: Hammond, Paul (IDEAS) Cc: help-glpk@gnu.org; Andrew Makhorin Subject: RE: [Help-glpk] Thread safety of GLPK On Wed, 14 Apr 2010, Hammond, Paul wrote: I'm thinking since it is written in C, and C is source compatible with C++, since C++ does support locking in threads, if one was to treat it as a C++ app written mostly in C, it may be possible to multi-thread it without a huge rewrite? C++ doesn't know about threads either. Perhaps you are thinking of some C++-only thread implementation? -- Michael henne...@web.cs.ndsu.nodak.edu Pessimist: The glass is half empty. Optimist: The glass is half full. Engineer: The glass is twice as big as it needs to be. -- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. ___ Help-glpk mailing list Help-glpk@gnu.org http://lists.gnu.org/mailman/listinfo/help-glpk
[Help-glpk] GLPK on 64 bit Linux
Hi, I #8217;m including bug-glpk here as this is potentially a bug, but it could be something we are doing wrong. We are using GLPK with Java (glpk-java-1.0.5), but to date have been using it in a 32 bit Linux environment and it #8217;s been fine (apart from crashes which we put down to the fact it #8217;s not thread safe) Now we are trying to use it in a 64 bit Linux environment and encountering some strange issues and I #8217;m just wondering if anybody else has encountered these and how we might solve it. To move to 64 bit, we recompiled the C code on 64 bit to produce a new shared object linked to it. It runs and executes no problem. For most of our inputs, it gives the same outputs as before, save for some very small diffs which I put down to the different floating point rounding. However, there are cases where a few of the numbers differ by amounts that cannot be put down to rounding. We have sometimes a fraction like .8xxx going to 0, or we have large numbers differing by 10-15%. Again, this is only in very few of the output variables out of quite a lot (all the others match fine) and many sets of inputs have no problem but it #8217;s enough to pose an issue if it goes wrong for even some of our inputs. We have dumped the inputs to a .dat file to confirm the inputs are the same going in for both 32 bit and 64 bit. It #8217;s the same Java software calling the solver so this should be the case. Does anybody know about such issues with 64 bit and the cause? Is there some particular compile flags or switches we need to use? Is there anywhere on the net a confirmed working binary version of GLPK for Java on 64 bit? If this is completely new and you would like to investigate, what would you need to be supplied with as a test case to look into the problem? Thanks, Paul NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. Hi, Im including bug-glpk here as this is potentially a bug, but it could be something we are doing wrong. We are using GLPK with Java (glpk-java-1.0.5), but to date have been using it in a 32 bit Linux environment and its been fine (apart from crashes which we put down to the fact its not thread safe) Now we are trying to use it in a 64 bit Linux environment and encountering some strange issues and Im just wondering if anybody else has encountered these and how we might solve it. To move to 64 bit, we recompiled the C code on 64 bit to produce a new shared object linked to it. It runs and executes no problem. For most of our inputs, it gives the same outputs as before, save for some very small diffs which I put down to the different floating point rounding. However, there are cases where a few of the numbers differ by amounts that cannot be put down to rounding. We have sometimes a fraction like .8xxx going to 0, or we have large numbers differing by 10-15%. Again, this is only in very few of the output variables out of quite a lot (all the others match fine) and many sets of inputs have no problem but its enough to pose an issue if it goes wrong for even some of our inputs. We have dumped the inputs to a .dat file to confirm the inputs are the same going in for both 32 bit and 64 bit. Its the same Java software calling the solver so this should be the case. Does anybody know about such issues with 64 bit and the cause? Is there some particular compile flags or switches we need to use? Is there anywhere on the net a confirmed working binary version of GLPK for Java on 64 bit? If this is completely new and you would like to investigate, what would you need to be supplied with as a test case to look into the problem? Thanks, Paul NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.We may monitor and store emails to the extent permitted by applicable law. ___ Help-glpk mailing list Help-glpk@gnu.org http://lists.gnu.org/mailman/listinfo/help-glpk
[Help-glpk] RE: GLPK on 64 bit Linux
Hi Andrew, Thanks for replying. Yes you're right, a given input resource may be allocated differently and is the case. But one thing that should remain constant though is the totals, what I'm talking about is that the totals of the allocations at the end of a given input do not equal that input. By way of illustration, take the often quoted example where the farmer is allocating his land to grow wheat or barley subject to constraints. In the output results, even if there are multiple optima, all the land should be allocated to either wheat or barley, but I find cases where the total land in the output is less or greater than the amount coming in, and it's too great to be a rounding error. Now my resource allocation problem is clearly different than the simple example above, I have many different resources to be allocated subject to more constraints, but the problem I have is essentially the same as this analogy. Paul -Original Message- From: Andrew Makhorin [mailto:m...@gnu.org] Sent: 25 March 2010 13:56 To: Hammond, Paul (IDEAS) Cc: help-glpk@gnu.org; bug-g...@gnu.org Subject: Re: GLPK on 64 bit Linux I #8217;m including bug-glpk here as this is potentially a bug, but it could be something we are doing wrong. We are using GLPK with Java (glpk-java-1.0.5), but to date have been using it in a 32 bit Linux environment and it #8217;s been fine (apart from crashes which we put down to the fact it #8217;s not thread safe) Now we are trying to use it in a 64 bit Linux environment and encountering some strange issues and I #8217;m just wondering if anybody else has encountered these and how we might solve it. To move to 64 bit, we recompiled the C code on 64 bit to produce a new shared object linked to it. It runs and executes no problem. For most of our inputs, it gives the same outputs as before, save for some very small diffs which I put down to the different floating point rounding. However, there are cases where a few of the numbers differ by amounts that cannot be put down to rounding. We have sometimes a fraction like .8xxx going to 0, or we have large numbers differing by 10-15%. Again, this is only in very few of the output variables out of quite a lot (all the others match fine) and many sets of inputs have no problem but it #8217;s enough to pose an issue if it goes wrong for even some of our inputs. We have dumped the inputs to a .dat file to confirm the inputs are the same going in for both 32 bit and 64 bit. It #8217;s the same Java software calling the solver so this should be the case. Does anybody know about such issues with 64 bit and the cause? Is there some particular compile flags or switches we need to use? Is there anywhere on the net a confirmed working binary version of GLPK for Java on 64 bit? If this is completely new and you would like to investigate, what would you need to be supplied with as a test case to look into the problem? Thank you for your report. Are the optimal objective values found by the 32- and 64-bit version of the solver identical or close to each other? If so, this is not a bug. If your lp/mip instance has multiple optima, it may happen that the 32-bit glpk solver finds one optimal solution while its 64-bit version finds another solution, which is optimal too. This is normal. -- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. ___ Help-glpk mailing list Help-glpk@gnu.org http://lists.gnu.org/mailman/listinfo/help-glpk
[Help-glpk] RE: GLPK on 64 bit Linux
Andrew, I did not write the original code here I'm debugging, but it's certainly intended that there is a constraint that all the input must be allocated. I can double check that this is the case. In terms of presenting a test case, I'm quite new to GLPK, I'd have to read up n the formats you have below. We do make some calls when we're debugging to the following: solver.writeCpxlp(file + .dat); . solver.printSol(file + .sol); Would any of these files be sufficient? If not is there a call I can make on the solver from Java to give you the problem in the format you would need? Paul -Original Message- From: Andrew Makhorin [mailto:m...@gnu.org] Sent: 25 March 2010 14:37 To: Hammond, Paul (IDEAS) Cc: help-glpk@gnu.org Subject: Re: GLPK on 64 bit Linux Yes you're right, a given input resource may be allocated differently and is the case. But one thing that should remain constant though is the totals, what I'm talking about is that the totals of the allocations at the end of a given input do not equal that input. By way of illustration, take the often quoted example where the farmer is allocating his land to grow wheat or barley subject to constraints. In the output results, even if there are multiple optima, all the land should be allocated to either wheat or barley, but I find cases where the total land in the output is less or greater than the amount coming in, and it's too great to be a rounding error. It may happen that it is not profitable to allocate all the land until you require all the land to be allocated by introducing an appropriate equiality constraint. Now my resource allocation problem is clearly different than the simple example above, I have many different resources to be allocated subject to more constraints, but the problem I have is essentially the same as this analogy. You can write the solution in a text file with glp_print_sol and see if the optimality conditions are satisfied. If they are, probably you missed some essential constraints. If not, please write your instance in mps or better in glpk lp/mip format and post it to me. Thanks. -- NOTICE: If received in error, please destroy, and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. We may monitor and store emails to the extent permitted by applicable law. ___ Help-glpk mailing list Help-glpk@gnu.org http://lists.gnu.org/mailman/listinfo/help-glpk
