I fixed the issue with dataframe api and am getting all columns now.
However, I am not able to perform grouping + udaf operations as it tries to
perform these on ignite.
setting OPTION_DISABLE_SPARK_SQL_OPTIMIZATION = true is not helping.
How so we tell ignite to just fetch data and perform all ot
Hi,
Thanks for the answer.
Unfortunately, we cannot remove Cassandra as it is being used elsewhere as
well. We will have to write directly in ignite and sync with cassandra.
We had a few other issues while getting data from spark:
1) cacherdd.sql("select * from table") is giving me heap memory (G
Hi, Shrey!
Just as idea - Ignite now has persistence (see
https://apacheignite.readme.io/docs/distributed-persistent-store),
may be you can completely replace Cassandra with Ignite?
In this case all data always be actual, no need to sync with external db.
--
Alexey Kuznetsov
Hello!
1) There is no generic way of pulling updates from 3rd party database and
there is no API support for it usually, so it's not obvious how we could
implement that even if we wanted.
2) By default cache store will process data in parallel on all nodes.
However if will not align data distribu
Hi, we are using Ignite as a cache layer over Cassandra for faster read
queries using spark. Our cluster has 10 nodes running an instance of
Cassandra and Ignite. However, we came across a few issues:
1) We currently store the data from spark to cassandra. Hence to load data,
we need to call .loa
