OK, that appears to be part of the way, but they caches are not queryable from any SQL tools like Tableau or DBeaver. What am i missing?
On Fri, Mar 15, 2019 at 5:29 AM Ilya Kasnacheev <[email protected]> wrote: > Hello! > > Yes, I understand your confusuion here. > > Take a look at following elaborate snippet: > > package org.apache.ignite.examples; > import java.util.Collections; > import java.util.LinkedHashMap; > import org.apache.ignite.Ignite; > import org.apache.ignite.IgniteCache; > import org.apache.ignite.IgniteDataStreamer; > import org.apache.ignite.Ignition; > import org.apache.ignite.cache.QueryEntity; > import org.apache.ignite.cache.query.SqlFieldsQuery; > import org.apache.ignite.configuration.CacheConfiguration; > > public class LoadTableWithDataStreamer { > public static void main(String[] args) { > try (Ignite ignite = Ignition.start()) { > IgniteCache<Object, Object> personCache = > ignite.getOrCreateCache(new CacheConfiguration<>("person") > .setQueryEntities(Collections.singleton( > new QueryEntity(Integer.class, > String.class).setTableName("person_table")))); > > IgniteCache<Object, Object> placeCache = > ignite.getOrCreateCache(new CacheConfiguration<>("place") > .setQueryEntities(Collections.singleton( > new QueryEntity(Integer.class, > String.class).setTableName("place_table") > // more decoration > .setKeyFieldName("id").setValueFieldName("name") > .setFields(new LinkedHashMap<String, String>() {{ > // Note that extending LinkedHashMap isn't > production-ready > put("id", Integer.class.getCanonicalName()); > put("name", String.class.getCanonicalName()); > }})))); > > try (IgniteDataStreamer<Object, Object> ds = > ignite.dataStreamer("person")) { > ds.addData(1, "John"); > } > > try (IgniteDataStreamer<Object, Object> ds = > ignite.dataStreamer("place")) { > ds.addData(1, "Siberia"); > } > > System.err.println("Query result"); > personCache.query(new SqlFieldsQuery("select _key, _val from > person_table")).getAll().forEach(System.err::println); > > // refer to different cache's table > personCache.query(new SqlFieldsQuery("select id, name from > \"place\".place_table")).getAll().forEach(System.err::println); > } > } > } > > Regards, > -- > Ilya Kasnacheev > > > чт, 14 мар. 2019 г. в 22:59, Mike Needham <[email protected]>: > >> Hi, >> >> Here is the code I am using >> >> package org.apache.ignite.examples; >> import java.util.Collections; >> import org.apache.ignite.Ignite; >> import org.apache.ignite.IgniteCache; >> import org.apache.ignite.IgniteDataStreamer; >> import org.apache.ignite.Ignition; >> import org.apache.ignite.cache.QueryEntity; >> import org.apache.ignite.cache.query.SqlFieldsQuery; >> import org.apache.ignite.configuration.CacheConfiguration; >> import org.apache.ignite.configuration.IgniteConfiguration; >> >> public class LoadTableWithDataStreamer { >> public static void main(String[] args) { >> try (Ignite ignite = >> Ignition.start("E:\\ignite\\apache-ignite-2.7.0-src\\examples\\config\\example-ignite.xml")) >> { >> IgniteCache<Object, Object> personCache = >> ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC") >> .setQueryEntities(Collections.singleton( >> new QueryEntity(Integer.class, >> String.class).setTableName("person_table")))); >> >> IgniteCache<Object, Object> placeCache = >> ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC") >> .setQueryEntities(Collections.singleton( >> new QueryEntity(Integer.class, >> String.class).setTableName("place_table")))); >> try (IgniteDataStreamer<Object, Object> ds = >> ignite.dataStreamer("person")) { >> ds.addData(1, "John"); >> } >> >> System.err.println("Query result"); >> personCache.query(new SqlFieldsQuery("select * from >> person_table")).getAll().forEach(System.err::println); >> } >> } >> } >> >> I want to create to queryable tables that I can load using the data >> streamer. The config is the default config in the java examples >> >> >> On Mon, Mar 11, 2019 at 3:06 AM Ilya Kasnacheev < >> [email protected]> wrote: >> >>> Hello! >>> >>> Can you perhaps post your config and code? Much easier than writing my >>> own boilerplate. >>> >>> Regards, >>> -- >>> Ilya Kasnacheev >>> >>> >>> пт, 8 мар. 2019 г. в 00:36, Mike Needham <[email protected]>: >>> >>>> Would it be possible for someone to provide a sample that uses the >>>> DataStreamer for multiple large caches and the resulting caches are >>>> queryable from ODBC tools like Tableau and DBeaver? I can get one to work, >>>> but cannot get the cache naming to work for a second one. >>>> >>>> On Thu, Mar 7, 2019 at 7:53 AM Ilya Kasnacheev < >>>> [email protected]> wrote: >>>> >>>>> Hello! >>>>> >>>>> JDBC with SET STREAMING ON should work reasonably well. You could also >>>>> use CacheStore. >>>>> >>>>> With .Net I guess you will have to stick to DataStreamer & learn how >>>>> to integrate it with Indexing properly. Or use JDBC from .Net. >>>>> >>>>> Regards, >>>>> -- >>>>> Ilya Kasnacheev >>>>> >>>>> >>>>> чт, 7 мар. 2019 г. в 16:31, Mike Needham <[email protected]>: >>>>> >>>>>> Is that the most efficient way to load millions of rows from a >>>>>> database table into a cache? I guess I am not seeing how this would work >>>>>> for millions of rows in the source database. That also appears to be a >>>>>> JAVA only solution and it would be preferable to have it be usable for >>>>>> .NET >>>>>> as well as java and hopefully python. >>>>>> >>>>>> On Thu, Mar 7, 2019 at 6:55 AM Ilya Kasnacheev < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello! >>>>>>> >>>>>>> What prevents you from reading rows from DB and feeding them to JDBC >>>>>>> prepared statement? >>>>>>> >>>>>>> Regards, >>>>>>> -- >>>>>>> Ilya Kasnacheev >>>>>>> >>>>>>> >>>>>>> чт, 7 мар. 2019 г. в 15:51, Mike Needham <[email protected]>: >>>>>>> >>>>>>>> And what if the data is already in a database table? I do not want >>>>>>>> to read from the table to write to a file to load a cache. >>>>>>>> >>>>>>>> On Tue, Mar 5, 2019 at 4:06 AM Ilya Kasnacheev < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hello! >>>>>>>>> >>>>>>>>> The preferred approach is to use Thin JDBC client: >>>>>>>>> https://apacheignite-sql.readme.io/docs/jdbc-driver >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> -- >>>>>>>>> Ilya Kasnacheev >>>>>>>>> >>>>>>>>> >>>>>>>>> пн, 4 мар. 2019 г. в 19:39, Mike Needham <[email protected]>: >>>>>>>>> >>>>>>>>>> Thanks for the links, If SET STREAMING ON is the preferred >>>>>>>>>> method, how would you do this in code rather than from a file with >>>>>>>>>> all the >>>>>>>>>> insert statements. >>>>>>>>>> >>>>>>>>>> On Mon, Mar 4, 2019 at 1:44 AM Ilya Kasnacheev < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> You can see at this page: >>>>>>>>>>> https://apacheignite-sql.readme.io/docs/sql-and-key-value-usage >>>>>>>>>>> >>>>>>>>>>> And then substitite cache.put() with addData() and cache name >>>>>>>>>>> will be SQL_PUBLIC_{table name in caps} >>>>>>>>>>> >>>>>>>>>>> There are not many examples since this is being discouraged in >>>>>>>>>>> favor of SET STREAMING ON, actually: >>>>>>>>>>> https://apacheignite-sql.readme.io/docs/set >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> -- >>>>>>>>>>> Ilya Kasnacheev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> пт, 1 мар. 2019 г. в 22:48, Mike Needham <[email protected]>: >>>>>>>>>>> >>>>>>>>>>>> I have looked at the documentation and the code samples and >>>>>>>>>>>> nothing is doing what I am trying to do. I want to be able to use >>>>>>>>>>>> the >>>>>>>>>>>> datastreamer to load 3 or 4 TABLES in a cache for an application >>>>>>>>>>>> that we >>>>>>>>>>>> use. If I create the tables using a create table syntax how do >>>>>>>>>>>> attach a >>>>>>>>>>>> datastreamer to the different caches if the cache name is PUBLIC >>>>>>>>>>>> for all of >>>>>>>>>>>> them? >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Feb 28, 2019 at 8:13 AM Ilya Kasnacheev < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello! >>>>>>>>>>>>> >>>>>>>>>>>>> I have linked the documentation page, there are also some code >>>>>>>>>>>>> examples in distribution. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> -- >>>>>>>>>>>>> Ilya Kasnacheev >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> чт, 28 февр. 2019 г. в 17:10, Mike Needham <[email protected] >>>>>>>>>>>>> >: >>>>>>>>>>>>> >>>>>>>>>>>>>> Is there any examples that show the steps to do this >>>>>>>>>>>>>> correctly? I stumbled upon this but have no idea if it is the >>>>>>>>>>>>>> best way to >>>>>>>>>>>>>> do this >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Feb 28, 2019 at 6:27 AM Ilya Kasnacheev < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> There's no restriction on cache name but setting it up for >>>>>>>>>>>>>>> the first time may be tricky indeed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Ilya Kasnacheev >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ср, 27 февр. 2019 г. в 19:48, needbrew99 <[email protected] >>>>>>>>>>>>>>> >: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> OK, was able to get it working. Apparently the cache name >>>>>>>>>>>>>>>> has to be PUBLIC >>>>>>>>>>>>>>>> and it will create a table based on the object definition >>>>>>>>>>>>>>>> that I have. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Some days it just not worth chewing through the restraints* >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Some days it just not worth chewing through the restraints* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Some days it just not worth chewing through the restraints* >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Some days it just not worth chewing through the restraints* >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> *Some days it just not worth chewing through the restraints* >>>>>> >>>>> >>>> >>>> -- >>>> *Some days it just not worth chewing through the restraints* >>>> >>> >> >> -- >> *Some days it just not worth chewing through the restraints* >> > -- *Some days it just not worth chewing through the restraints*
