Thank you for the response I am so sorry for the repeated trouble. On Tue, 4 Jul, 2023, 2:37 pm Stephen Darlington, < [email protected]> wrote:
> This is a community user forum where people are volunteering their time. > I’m afraid you can’t expect immediate responses. > > On 4 Jul 2023, at 08:31, Arunima Barik <[email protected]> wrote: > > Any updates on this please... > > On Mon, 3 Jul 2023 at 18:01, Arunima Barik <[email protected]> > wrote: > >> I am reading a parquet file using Spark dataframe as df >> >> I want to write some part of this data to ignite cache >> >> Assume I want to write df2 to the cache >> >> I used df2.write.format('ignite') >> Is there a better way to do this or this is the only way?? >> >> Regards >> Arunima >> >> On Mon, 3 Jul, 2023, 1:19 pm Stephen Darlington, < >> [email protected]> wrote: >> >>> Commercial options are available, but otherwise help would generally be >>> limited to email lists and Stack Overflow. >>> >>> On 1 Jul 2023, at 06:59, Arunima Barik <[email protected]> wrote: >>> >>> Are there any provisions wherein I can discuss about my project >>> implementation with someone from the Ignite team to clarify some doubts? >>> >>> Preferably through a small online meet? >>> >>> Regards >>> Arunima >>> >>> On Sat, 1 Jul, 2023, 12:03 am Jeremy McMillan, < >>> [email protected]> wrote: >>> >>>> Python doesn't at this time go anywhere near Ignite CacheStore. You >>>> would need to implement the CacheStore in Java or some other language which >>>> compiles to JVM runtime/jar. There's a talk from the most recent summit on >>>> using Groovy, if you want a higher level language than Java, but >>>> theoretically you could use Jython (if you are willing to experiment and >>>> can find a compatible JVM that runs both Ignite and Jython). >>>> >>>> Ignite can operate like a federated query proxy if different caches are >>>> implemented with different external persistence for each cache. CacheStore >>>> is the interface Ignite would use to send a cache miss to a backend >>>> database. In your original question you intended to use Parquet files as a >>>> backend database, but Ignite does not (yet) provide one for Parquet. If >>>> someone were to donate a supportable Java implementation, I suspect the >>>> community would adopt and support it. Since Parquet is columnar, I also >>>> suspect it would need to target Ignite 3 to adopt conventions around >>>> columnar data, and then might be backported to Ignite 2. >>>> >>>> >>>> On Fri, Jun 30, 2023 at 12:13 PM Arunima Barik < >>>> [email protected]> wrote: >>>> >>>>> Which do you think would be a better option? >>>>> >>>>> Federated queries or CacheStore >>>>> >>>>> And is CacheStore supported in Python? >>>>> >>>>> On Fri, 30 Jun, 2023, 1:50 pm Stephen Darlington, < >>>>> [email protected]> wrote: >>>>> >>>>>> You’d need to implement your own Cache Store. >>>>>> https://ignite.apache.org/docs/latest/persistence/custom-cache-store >>>>>> >>>>>> On 30 Jun 2023, at 06:46, Arunima Barik <[email protected]> >>>>>> wrote: >>>>>> >>>>>> >>>>>> ---------- Forwarded message --------- >>>>>> From: Arunima Barik <[email protected]> >>>>>> Date: Fri, 30 Jun, 2023, 10:52 am >>>>>> Subject: Ignite for Parquet files >>>>>> To: <[email protected]> >>>>>> >>>>>> >>>>>> Hello Team >>>>>> >>>>>> I have my data stored as parquet files. I want a caching layer on top >>>>>> of this existing file system. I am going to use Ignite for that but I do >>>>>> not need native persistence for that. >>>>>> >>>>>> I want that any changes to database should be reflected in both cache >>>>>> and file. >>>>>> And same for read queries. It should automatically read from disk if >>>>>> data is not present in cache. >>>>>> >>>>>> I want to do all this is python. Please let me know how the same can >>>>>> be done. >>>>>> Resources if any as well. >>>>>> >>>>>> Thank you and looking forward to hearing from you. >>>>>> >>>>>> Regard, >>>>>> Arunima Barik >>>>>> >>>>>> >>>>>> >>> >
