Hello!
Yes, I understand your confusuion here.
Take a look at following elaborate snippet:
package org.apache.ignite.examples;
import java.util.Collections;
import java.util.LinkedHashMap;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteDataStreamer;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.QueryEntity;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
public class LoadTableWithDataStreamer {
public static void main(String[] args) {
try (Ignite ignite = Ignition.start()) {
IgniteCache<Object, Object> personCache =
ignite.getOrCreateCache(new CacheConfiguration<>("person")
.setQueryEntities(Collections.singleton(
new QueryEntity(Integer.class,
String.class).setTableName("person_table"))));
IgniteCache<Object, Object> placeCache =
ignite.getOrCreateCache(new CacheConfiguration<>("place")
.setQueryEntities(Collections.singleton(
new QueryEntity(Integer.class,
String.class).setTableName("place_table")
// more decoration
.setKeyFieldName("id").setValueFieldName("name")
.setFields(new LinkedHashMap<String, String>() {{
// Note that extending LinkedHashMap isn't production-ready
put("id", Integer.class.getCanonicalName());
put("name", String.class.getCanonicalName());
}}))));
try (IgniteDataStreamer<Object, Object> ds =
ignite.dataStreamer("person")) {
ds.addData(1, "John");
}
try (IgniteDataStreamer<Object, Object> ds =
ignite.dataStreamer("place")) {
ds.addData(1, "Siberia");
}
System.err.println("Query result");
personCache.query(new SqlFieldsQuery("select _key, _val
from person_table")).getAll().forEach(System.err::println);
// refer to different cache's table
personCache.query(new SqlFieldsQuery("select id, name from
\"place\".place_table")).getAll().forEach(System.err::println);
}
}
}
Regards,
--
Ilya Kasnacheev
чт, 14 мар. 2019 г. в 22:59, Mike Needham <[email protected]>:
> Hi,
>
> Here is the code I am using
>
> package org.apache.ignite.examples;
> import java.util.Collections;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.IgniteDataStreamer;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.QueryEntity;
> import org.apache.ignite.cache.query.SqlFieldsQuery;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
>
> public class LoadTableWithDataStreamer {
> public static void main(String[] args) {
> try (Ignite ignite =
> Ignition.start("E:\\ignite\\apache-ignite-2.7.0-src\\examples\\config\\example-ignite.xml"))
> {
> IgniteCache<Object, Object> personCache =
> ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(Integer.class,
> String.class).setTableName("person_table"))));
>
> IgniteCache<Object, Object> placeCache =
> ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC")
> .setQueryEntities(Collections.singleton(
> new QueryEntity(Integer.class,
> String.class).setTableName("place_table"))));
> try (IgniteDataStreamer<Object, Object> ds =
> ignite.dataStreamer("person")) {
> ds.addData(1, "John");
> }
>
> System.err.println("Query result");
> personCache.query(new SqlFieldsQuery("select * from
> person_table")).getAll().forEach(System.err::println);
> }
> }
> }
>
> I want to create to queryable tables that I can load using the data
> streamer. The config is the default config in the java examples
>
>
> On Mon, Mar 11, 2019 at 3:06 AM Ilya Kasnacheev <[email protected]>
> wrote:
>
>> Hello!
>>
>> Can you perhaps post your config and code? Much easier than writing my
>> own boilerplate.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 8 мар. 2019 г. в 00:36, Mike Needham <[email protected]>:
>>
>>> Would it be possible for someone to provide a sample that uses the
>>> DataStreamer for multiple large caches and the resulting caches are
>>> queryable from ODBC tools like Tableau and DBeaver? I can get one to work,
>>> but cannot get the cache naming to work for a second one.
>>>
>>> On Thu, Mar 7, 2019 at 7:53 AM Ilya Kasnacheev <
>>> [email protected]> wrote:
>>>
>>>> Hello!
>>>>
>>>> JDBC with SET STREAMING ON should work reasonably well. You could also
>>>> use CacheStore.
>>>>
>>>> With .Net I guess you will have to stick to DataStreamer & learn how to
>>>> integrate it with Indexing properly. Or use JDBC from .Net.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> чт, 7 мар. 2019 г. в 16:31, Mike Needham <[email protected]>:
>>>>
>>>>> Is that the most efficient way to load millions of rows from a
>>>>> database table into a cache? I guess I am not seeing how this would work
>>>>> for millions of rows in the source database. That also appears to be a
>>>>> JAVA only solution and it would be preferable to have it be usable for
>>>>> .NET
>>>>> as well as java and hopefully python.
>>>>>
>>>>> On Thu, Mar 7, 2019 at 6:55 AM Ilya Kasnacheev <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> What prevents you from reading rows from DB and feeding them to JDBC
>>>>>> prepared statement?
>>>>>>
>>>>>> Regards,
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>>
>>>>>> чт, 7 мар. 2019 г. в 15:51, Mike Needham <[email protected]>:
>>>>>>
>>>>>>> And what if the data is already in a database table? I do not want
>>>>>>> to read from the table to write to a file to load a cache.
>>>>>>>
>>>>>>> On Tue, Mar 5, 2019 at 4:06 AM Ilya Kasnacheev <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Hello!
>>>>>>>>
>>>>>>>> The preferred approach is to use Thin JDBC client:
>>>>>>>> https://apacheignite-sql.readme.io/docs/jdbc-driver
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> --
>>>>>>>> Ilya Kasnacheev
>>>>>>>>
>>>>>>>>
>>>>>>>> пн, 4 мар. 2019 г. в 19:39, Mike Needham <[email protected]>:
>>>>>>>>
>>>>>>>>> Thanks for the links, If SET STREAMING ON is the preferred
>>>>>>>>> method, how would you do this in code rather than from a file with
>>>>>>>>> all the
>>>>>>>>> insert statements.
>>>>>>>>>
>>>>>>>>> On Mon, Mar 4, 2019 at 1:44 AM Ilya Kasnacheev <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> Hello!
>>>>>>>>>>
>>>>>>>>>> You can see at this page:
>>>>>>>>>> https://apacheignite-sql.readme.io/docs/sql-and-key-value-usage
>>>>>>>>>>
>>>>>>>>>> And then substitite cache.put() with addData() and cache name
>>>>>>>>>> will be SQL_PUBLIC_{table name in caps}
>>>>>>>>>>
>>>>>>>>>> There are not many examples since this is being discouraged in
>>>>>>>>>> favor of SET STREAMING ON, actually:
>>>>>>>>>> https://apacheignite-sql.readme.io/docs/set
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> --
>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> пт, 1 мар. 2019 г. в 22:48, Mike Needham <[email protected]>:
>>>>>>>>>>
>>>>>>>>>>> I have looked at the documentation and the code samples and
>>>>>>>>>>> nothing is doing what I am trying to do. I want to be able to use
>>>>>>>>>>> the
>>>>>>>>>>> datastreamer to load 3 or 4 TABLES in a cache for an application
>>>>>>>>>>> that we
>>>>>>>>>>> use. If I create the tables using a create table syntax how do
>>>>>>>>>>> attach a
>>>>>>>>>>> datastreamer to the different caches if the cache name is PUBLIC
>>>>>>>>>>> for all of
>>>>>>>>>>> them?
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 28, 2019 at 8:13 AM Ilya Kasnacheev <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello!
>>>>>>>>>>>>
>>>>>>>>>>>> I have linked the documentation page, there are also some code
>>>>>>>>>>>> examples in distribution.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> --
>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> чт, 28 февр. 2019 г. в 17:10, Mike Needham <[email protected]
>>>>>>>>>>>> >:
>>>>>>>>>>>>
>>>>>>>>>>>>> Is there any examples that show the steps to do this
>>>>>>>>>>>>> correctly? I stumbled upon this but have no idea if it is the
>>>>>>>>>>>>> best way to
>>>>>>>>>>>>> do this
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Feb 28, 2019 at 6:27 AM Ilya Kasnacheev <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There's no restriction on cache name but setting it up for
>>>>>>>>>>>>>> the first time may be tricky indeed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ср, 27 февр. 2019 г. в 19:48, needbrew99 <[email protected]
>>>>>>>>>>>>>> >:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> OK, was able to get it working. Apparently the cache name
>>>>>>>>>>>>>>> has to be PUBLIC
>>>>>>>>>>>>>>> and it will create a table based on the object definition
>>>>>>>>>>>>>>> that I have.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Some days it just not worth chewing through the restraints*
>>>>>
>>>>
>>>
>>> --
>>> *Some days it just not worth chewing through the restraints*
>>>
>>
>
> --
> *Some days it just not worth chewing through the restraints*
>
