Hello, Thanks for your answers, guys.
It works now but I struggled a lot with the serialization process. Maybe somewhere strongly underline the fact in the documentation that ONLY certain type of classes (I think it's closures and some other inner tasks) are part of the zero deployment mechanisms. For like two hours, I didn't understood why any object inside my tasks would not be automagically deployed on the cluster nodes. It is still a bit annoying. Once I create a new processor, I have to restart all my nodes to have it properly deployed everywhere. Is there something I'm missing ? Anyway thanks, Ignite is a really powerfull thing! Date: Fri, 27 Nov 2015 13:55:08 +0300 Subject: Re: From one cache to another From: [email protected] To: [email protected] Hi, You should not serialize cache instances. Actually your cache is already created on all nodes of your cluster and you can obtain it using ignite.cache(<name>) or ignite.getOrCreateCache(<name>) methods. I assume that you try to start Ignite instance with default name from your ComplexObjectProcessor but Ignite is started already. You can declare Ignite reference in ComplexObjectProcessor and annotate it with @IgniteInstanceResource. When your processor will be deserialized on remote node Ignite instance will be injected automatically. It looks like this: public class ComplexObjectProcessor { @IgniteInstanceResource private Ignite ignite ... } Make sure that your ComplexObjectProcessor is regular class or inner static class. Non-static inner class instances have reference to enclosing class that will be serialized with inner class. On Thu, Nov 26, 2015 at 9:08 PM, alpha centauri <[email protected]> wrote: Hello, I'm a new Apache Ignite User. I'm trying to prototype a simple use case : 1. create two caches 2. read strings from a file and put them in the first cache 3. compute strings into complex objects put the complex objects into second cache My code looks like this : // Part 1 final IgniteCache<Integer, String> cacheString = ignite.getOrCreateCache("somestrings"); final IgniteCache<String, SomeObjects> cache = ignite.getOrCreateCache("someobjects"); // Part 2 try (IgniteDataStreamer<Integer, String> stmr = ignite.dataStreamer(cacheString.getName())) { stmr.allowOverwrite(true); try (BufferedReader br = Files.newReader(file, StandardCharsets.UTF_8)) { for (String line; (line = br.readLine()) != null; ) { if (line !=null && line.length() > 0) { stmr.addData(index, line); } } } } // Part 3 is where I struggle and need help IgniteCompute compute = ignite.compute(); for (int i = 0; i < cacheString.size(); i++) { final int key = i; compute.affinityRun(cacheString.getName(), key, new ComplexObjectProcessor(cache, cacheString.get(key))); } The issue is that the ComplexObjectProcessor is serialized to my grid nodes and if I put the destination cache, it will crash because the cache cannot be serialized. If I initialize the cache from my ComplexObjectProcessor, it will complain about the fact that Ignite has already been instanciated outside. ComplexObjectProcessor has a lot of code so an inside/inline class is not an option. The cacheString contains 100Millions lines so I really want to split the complex object processor with compute tasks. Any solution? Or code sample would greatly help. Thks ahead, AC -- Andrey Gura GridGain Systems, Inc. www.gridgain.com
