Hi,
Actually if a processor is being deployed on a remote node all other
classes, the processor refers to from its code, are deployed as well.
In my understanding you should take into account the deployment mode
that is used on your side.
By default SHARED mode is activated which means for you that if you had
several nodes that were sending the processor via compute.affinityRun
(or other method) then either all of them has to be restarted once the
processor is modified (no need to restart the nodes that executed the
task and deployed the processor class remotely) or restart only a single
node that modified the processor but provide updated class version in
META-INF/ignite.xml. Please refer to DeploymentMode.SHARED documentation
for more info.
--
Denis
On 12/6/2015 3:51 AM, alpha centauri wrote:
Hello,
Thanks for your answers, guys.
It works now but I struggled a lot with the serialization process.
Maybe somewhere strongly underline the fact in the documentation that
ONLY certain type of classes (I think it's closures and some other
inner tasks) are part of the zero deployment mechanisms. For like two
hours, I didn't understood why any object inside my tasks would not be
automagically deployed on the cluster nodes.
It is still a bit annoying. Once I create a new processor, I have to
restart all my nodes to have it properly deployed everywhere. Is there
something I'm missing ?
Anyway thanks, Ignite is a really powerfull thing!
------------------------------------------------------------------------
Date: Fri, 27 Nov 2015 13:55:08 +0300
Subject: Re: From one cache to another
From: [email protected]
To: [email protected]
Hi,
You should not serialize cache instances. Actually your cache is
already created on all nodes of your cluster and you can obtain it
using ignite.cache(<name>) or ignite.getOrCreateCache(<name>) methods.
I assume that you try to start Ignite instance with default name from
your ComplexObjectProcessor but Ignite is started already. You can
declare Ignite reference in ComplexObjectProcessor and annotate it
with @IgniteInstanceResource. When your processor will be deserialized
on remote node Ignite instance will be injected automatically. It
looks like this:
public class ComplexObjectProcessor {
@IgniteInstanceResource
private Ignite ignite
...
}
Make sure that your ComplexObjectProcessor is regular class or inner
static class. Non-static inner class instances have reference to
enclosing class that will be serialized with inner class.
On Thu, Nov 26, 2015 at 9:08 PM, alpha centauri <[email protected]
<mailto:[email protected]>> wrote:
Hello,
I'm a new Apache Ignite User. I'm trying to prototype a simple use
case :
1. create two caches
2. read strings from a file and put them in the first cache
3. compute strings into complex objects put the complex objects
into second cache
My code looks like this :
// Part 1
final IgniteCache<Integer, String> cacheString =
ignite.getOrCreateCache("somestrings");
final IgniteCache<String, SomeObjects> cache =
ignite.getOrCreateCache("someobjects");
// Part 2
try (IgniteDataStreamer<Integer, String> stmr =
ignite.dataStreamer(cacheString.getName())) {
stmr.allowOverwrite(true);
try (BufferedReader br = Files.newReader(file,
StandardCharsets.UTF_8)) {
for (String line; (line = br.readLine()) !=
null; ) {
if (line !=null && line.length() > 0) {
stmr.addData(index, line);
}
}
}
}
// Part 3 is where I struggle and need help
IgniteCompute compute = ignite.compute();
for (int i = 0; i < cacheString.size(); i++) {
final int key = i;
compute.affinityRun(cacheString.getName(), key, new
ComplexObjectProcessor(cache, cacheString.get(key)));
}
The issue is that the ComplexObjectProcessor is serialized to my
grid nodes and if I put the destination cache, it will crash
because the cache cannot be serialized. If I initialize the cache
from my ComplexObjectProcessor, it will complain about the fact
that Ignite has already been instanciated outside.
ComplexObjectProcessor has a lot of code so an inside/inline class
is not an option.
The cacheString contains 100Millions lines so I really want to
split the complex object processor with compute tasks.
Any solution? Or code sample would greatly help.
Thks ahead,
AC
--
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com <http://www.gridgain.com/>
