Hello,
I'm a new Apache Ignite User. I'm trying to prototype a simple use case :
1. create two caches
2. read strings from a file and put them in the first cache
3. compute strings into complex objects put the complex objects into second
cache
My code looks like this :
// Part 1
final IgniteCache<Integer, String> cacheString =
ignite.getOrCreateCache("somestrings");
final IgniteCache<String, SomeObjects> cache =
ignite.getOrCreateCache("someobjects");
// Part 2
try (IgniteDataStreamer<Integer, String> stmr =
ignite.dataStreamer(cacheString.getName())) {
stmr.allowOverwrite(true);
try (BufferedReader br = Files.newReader(file,
StandardCharsets.UTF_8)) {
for (String line; (line = br.readLine()) != null; ) {
if (line !=null && line.length() > 0) {
stmr.addData(index, line);
}
}
}
}
// Part 3 is where I struggle and need help
IgniteCompute compute = ignite.compute();
for (int i = 0; i < cacheString.size(); i++) {
final int key = i;
compute.affinityRun(cacheString.getName(), key, new
ComplexObjectProcessor(cache, cacheString.get(key)));
}
The issue is that the ComplexObjectProcessor is serialized to my grid nodes and
if I put the destination cache, it will crash because the cache cannot be
serialized. If I initialize the cache from my ComplexObjectProcessor, it will
complain about the fact that Ignite has already been instanciated outside.
ComplexObjectProcessor has a lot of code so an inside/inline class is not an
option.
The cacheString contains 100Millions lines so I really want to split the
complex object processor with compute tasks.
Any solution? Or code sample would greatly help.
Thks ahead,
AC
