Replying, to my own question.
Indeed, it seems to be the case that for Ignite 2.0, everything will be
off heap. Denis Magda just (23 hours ago) updated the relevant wiki
page.
https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=67638199&selectedPageVersions=10&selectedPageVersions=8
Sounds cool to me :).
Kamil
On 2017-04-19 00:43, Kamil Misuth wrote:
Sure, Andrey.
In mean time, a fact-checking question:
I haven't really gone through the new memory policy functionality in
depth, but it does seem that for Ignite 2.0 your aim is to move data
off heap implicitly for all caches (and possibly backup the off heap
region by memory mapped file(s)). Is this what you meant by "many
memory related fixes"?
Thanks.
Kamil
On 2017-04-13 13:08, Andrey Gura wrote:
Kamil,
thanks a lot for your help. As I told before I need some time for
problem analyse. After it I'll create JIRA ticket and share link to it
in this thread.
Thank you again!
On Wed, Apr 12, 2017 at 11:56 PM, Kamil Misuth <ki...@ethome.sk>
wrote:
Hi Andrey,
I've built ignite-3477-master (commit hash 5839f481b7) today.
Apart from the fact that some other Configuration APIs changed,
CacheMemoryMode disapeared. I guess this is related to
http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-4758-introducing-cache-memory-policies-td14957.html
Anyway, I've got the following exception on the first try, since
ignite-indexing 2.0.0 is compiled against H2 1.4.191 and Spring Boot
1.5.2.RELEASE managed H2 version is 1.4.193 and apparently
org.h2.result.Row
changed from abstract class to an interface between the two versions.
Caused by: java.lang.IncompatibleClassChangeError: class
org.apache.ignite.internal.processors.query.h2.opt.GridH2Row has
interface
org.h2.result.Row as super class
After fixing the dependency issue, I've tried to create two node
cluster and
JVM SIGSEGVed in DirectNioClientWorker just as before.
Attaching crash logs from both nodes (running on the same machine
FreeBSD
11) to this e-mail.
Kamil
On 2017-04-12 00:49, Kamil Misuth wrote:
Sure thing.
I will check out ignite-3477-master as soon as I have some time
tomorrow.
Kamil
On 04/11/2017 05:48 PM, Andrey Gura wrote:
Thanks for provided information. I need additional time for problem
investigation.
You can also try code from ignite-3477-master branch. This branch
contains many memory related fixes but it isn't stable yet.
On Mon, Apr 10, 2017 at 11:37 PM, kimec.ethome.sk <ki...@ethome.sk>
wrote:
Hi Andrey,
sorry, I've got ahead of my self.
I am on FreeBSD 11.0-RELEASE-p1 amd64
With OpenJDK Runtime Environment 1.8.0_121-b13 Oracle Corporation
OpenJDK
64-Bit Server VM 25.121-b13
hw.model: Intel(R) Core(TM) i7-4702MQ CPU @ 2.20GHz
hw.machine_arch: amd64
hw.ncpu: 8
hw.physmem: 8251813888
Core dump is 1 GB, so I guess that is no go. I am attaching crash
log to
this e-mail.
I have uploaded the project I've used during my testing here
https://github.com/kimec/ignite-spring-boot .
The sample works perfectly well with stock ignite-core on Linux
OpenJDK
8
xs64 CentOS 7 .
Kamil
On 2017-04-10 12:24, Andrey Gura wrote:
Hi,
could you please share core dump file? If not, it would be
helpful to
know what is CPU architecture on this server.
On Mon, Apr 10, 2017 at 2:53 AM, Kamil Misuth <ki...@ethome.sk>
wrote:
Greetings,
OpenJDK (7 and 8) HotSpot JVM SIGSEGVs on FreeBSD 11 as soon as
node
joins a
topology and starts to communicate via DirectNioClientWorker.
The root cause is DirectByteBufferStreamImpl (both versions)
which
uses
GridUnsafe.getXXX/putXXX(Object object, offset, value) methods
to
manipulate
DirectByteBuffer, whereas it should really be using
GridUnsafe.getXXX/putXXX(address, value), since DirectByteBuffer
is
allocated on C heap (off java heap).
Notice that at least one instance of the same problem is known
to
exist
in
another project using Unsafe
https://issues.apache.org/jira/browse/CASSANDRA-8325 .
The OpenJDK source of Unsafe is more or less clear on this
http://hg.openjdk.java.net/jdk8u/jdk8u60/jdk/file/935758609767/src/share/classes/sun/misc/Unsafe.java#l391
I have prepared a simple fix here
https://github.com/apache/ignite/compare/1.9.0-rc2...kimec:freebsd-support
.
However, I am not sure if the solution is right in regard to
overall
ignite
performance.
I've tried to compile ignite-core with tests and after applying
my
changes
was able to pass all the basic stuff until the performance test
stage
at
which point my machine run out of RAM and swap space (some 10
GB)...
Not
sure if this is how the tests are supposed to be. After
compiling with
-DskipTests I was able to create FreeBSD 11 - CentOS 7 two node
cluster
and
everything seemed OK (the two nodes shared an IGFS instance
backed by
replicated caches).
Please note that OpenJDK on different systems as well as Oracle
JDK
(via
Linux compatility layer) on FreeBSD seem to be more forgiving
and does
not
SIGSEGV.
I've based my branch on 1.9.0-rc2 since tag 1.9.0 has already
POM with
version 2.0.
Kamil
