This is a community user forum where people are volunteering their time. I’m afraid you can’t expect immediate responses.
> On 4 Jul 2023, at 08:31, Arunima Barik <arunimabari...@gmail.com> wrote: > > Any updates on this please... > > On Mon, 3 Jul 2023 at 18:01, Arunima Barik <arunimabari...@gmail.com > <mailto:arunimabari...@gmail.com>> wrote: >> I am reading a parquet file using Spark dataframe as df >> >> I want to write some part of this data to ignite cache >> >> Assume I want to write df2 to the cache >> >> I used df2.write.format('ignite') >> Is there a better way to do this or this is the only way?? >> >> Regards >> Arunima >> >> On Mon, 3 Jul, 2023, 1:19 pm Stephen Darlington, >> <stephen.darling...@gridgain.com <mailto:stephen.darling...@gridgain.com>> >> wrote: >>> Commercial options are available, but otherwise help would generally be >>> limited to email lists and Stack Overflow. >>> >>>> On 1 Jul 2023, at 06:59, Arunima Barik <arunimabari...@gmail.com >>>> <mailto:arunimabari...@gmail.com>> wrote: >>>> >>>> Are there any provisions wherein I can discuss about my project >>>> implementation with someone from the Ignite team to clarify some doubts? >>>> >>>> Preferably through a small online meet? >>>> >>>> Regards >>>> Arunima >>>> >>>> On Sat, 1 Jul, 2023, 12:03 am Jeremy McMillan, >>>> <jeremy.mcmil...@gridgain.com <mailto:jeremy.mcmil...@gridgain.com>> wrote: >>>>> Python doesn't at this time go anywhere near Ignite CacheStore. You would >>>>> need to implement the CacheStore in Java or some other language which >>>>> compiles to JVM runtime/jar. There's a talk from the most recent summit >>>>> on using Groovy, if you want a higher level language than Java, but >>>>> theoretically you could use Jython (if you are willing to experiment and >>>>> can find a compatible JVM that runs both Ignite and Jython). >>>>> >>>>> Ignite can operate like a federated query proxy if different caches are >>>>> implemented with different external persistence for each cache. >>>>> CacheStore is the interface Ignite would use to send a cache miss to a >>>>> backend database. In your original question you intended to use Parquet >>>>> files as a backend database, but Ignite does not (yet) provide one for >>>>> Parquet. If someone were to donate a supportable Java implementation, I >>>>> suspect the community would adopt and support it. Since Parquet is >>>>> columnar, I also suspect it would need to target Ignite 3 to adopt >>>>> conventions around columnar data, and then might be backported to Ignite >>>>> 2. >>>>> >>>>> >>>>> On Fri, Jun 30, 2023 at 12:13 PM Arunima Barik <arunimabari...@gmail.com >>>>> <mailto:arunimabari...@gmail.com>> wrote: >>>>>> Which do you think would be a better option? >>>>>> >>>>>> Federated queries or CacheStore >>>>>> >>>>>> And is CacheStore supported in Python? >>>>>> >>>>>> On Fri, 30 Jun, 2023, 1:50 pm Stephen Darlington, >>>>>> <stephen.darling...@gridgain.com >>>>>> <mailto:stephen.darling...@gridgain.com>> wrote: >>>>>>> You’d need to implement your own Cache Store. >>>>>>> https://ignite.apache.org/docs/latest/persistence/custom-cache-store >>>>>>> >>>>>>>> On 30 Jun 2023, at 06:46, Arunima Barik <arunimabari...@gmail.com >>>>>>>> <mailto:arunimabari...@gmail.com>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> ---------- Forwarded message --------- >>>>>>>> From: Arunima Barik <arunimabari...@gmail.com >>>>>>>> <mailto:arunimabari...@gmail.com>> >>>>>>>> Date: Fri, 30 Jun, 2023, 10:52 am >>>>>>>> Subject: Ignite for Parquet files >>>>>>>> To: <user@ignite.apache.org <mailto:user@ignite.apache.org>> >>>>>>>> >>>>>>>> >>>>>>>> Hello Team >>>>>>>> >>>>>>>> I have my data stored as parquet files. I want a caching layer on top >>>>>>>> of this existing file system. I am going to use Ignite for that but I >>>>>>>> do not need native persistence for that. >>>>>>>> >>>>>>>> I want that any changes to database should be reflected in both cache >>>>>>>> and file. >>>>>>>> And same for read queries. It should automatically read from disk if >>>>>>>> data is not present in cache. >>>>>>>> >>>>>>>> I want to do all this is python. Please let me know how the same can >>>>>>>> be done. >>>>>>>> Resources if any as well. >>>>>>>> >>>>>>>> Thank you and looking forward to hearing from you. >>>>>>>> >>>>>>>> Regard, >>>>>>>> Arunima Barik >>>>>>> >>>
