Re: count not updated through SQL using DataStreamer
Hello!
Can you show how you create FOO table? I guess expected cache type is NOT
, that's why no data is seen.
Regards,
--
Ilya Kasnacheev
чт, 16 мая 2019 г. в 23:49, Banias H :
> Hello Ignite experts,
>
> I am very new to Ignite. I am trying to ingest 15M rows of data using
> DataStreamer into a table in a two-node Ignite cluster (v2.7) but run into
> problems of not getting the data through running SQL on DBeaver.
>
> Here is the list of steps I took:
>
> 1. Start up two nodes using the following xml.
>
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
>http://www.springframework.org/schema/beans
>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>
>
>
>class="org.apache.ignite.configuration.DataStorageConfiguration">
>
>class="org.apache.ignite.configuration.DataRegionConfiguration">
>
>
>
>
>
>
>
>
>class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> IP1
> IP2
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 2. Use python thin client to get the cache SQL_PUBLIC_FOO and insert ten
> row of data. After this step, both thin client and DBeaver SQL client
> report the same count:
>
> - thin client:
>
> nodes = [
> (IP1, 10800),
> (IP2, 10800),
> ]
> client = Client()
> client.connect(nodes)
> cache = client.get_cache("SQL_PUBLIC_FOO")
> print(cache.get_size())
>
> returns 10
>
> - SQL through DBeaver
>
> SELECT COUNT(*) FROM FOO
>
> returns 10
>
> 3. However when I tried using DataStreamer to ingest 100 rows into the
> cache SQL_PUBLIC_FOO, only thin client showed new count value and SQL
> returned old count value:
>
> - ingesting through DataStreamer
> //I ran the jar on one of the Ignite nodes
> String CONFIG_FILE = ;
> Ignition.setClientMode(true);
> Ignite ignite = Ignition.start(CONFIG_FILE);
> IgniteDataStreamer stmr =
> ignite.dataStreamer("SQL_PUBLIC_FOO");
> stmr.addData(rowCount, value);
>
> - thin client:
>
> nodes = [
> (IP1, 10800),
> (IP2, 10800),
> ]
> client = Client()
> client.connect(nodes)
> cache = client.get_cache("SQL_PUBLIC_FOO")
> cache.get_size()
>
> returns 110
>
> - SQL through DBeaver
>
> SELECT COUNT(*) FROM FOO
>
> returns 10
>
> Would anyone shed some lights on what I did wrong? I would love to use
> DataStreamer to put much more data into the cache so that I would wan to be
> able to query them through SQL.
>
> Thanks for the help. I appreciate it.
>
> Regards,
> Calvin
>
>
Re: count not updated through SQL using DataStreamer
Hi Calvin,
Cache.size and SELECT COUNT(*) are not always equal in Ignite. Could you please
tell what arguments did you pass to IgniteDataStreamer.addData method?
Sent from my iPhone
> On 16 May 2019, at 23:40, Banias H wrote:
>
> Hello Ignite experts,
>
> I am very new to Ignite. I am trying to ingest 15M rows of data using
> DataStreamer into a table in a two-node Ignite cluster (v2.7) but run into
> problems of not getting the data through running SQL on DBeaver.
>
> Here is the list of steps I took:
>
> 1. Start up two nodes using the following xml.
>
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
>http://www.springframework.org/schema/beans
>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>
>
>
>
>
>class="org.apache.ignite.configuration.DataRegionConfiguration">
>
>
>
>
>
>
>
>
>class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
> IP1
> IP2
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 2. Use python thin client to get the cache SQL_PUBLIC_FOO and insert ten row
> of data. After this step, both thin client and DBeaver SQL client report the
> same count:
>
> - thin client:
>
> nodes = [
> (IP1, 10800),
> (IP2, 10800),
> ]
> client = Client()
> client.connect(nodes)
> cache = client.get_cache("SQL_PUBLIC_FOO")
> print(cache.get_size())
>
> returns 10
>
> - SQL through DBeaver
>
> SELECT COUNT(*) FROM FOO
>
> returns 10
>
> 3. However when I tried using DataStreamer to ingest 100 rows into the cache
> SQL_PUBLIC_FOO, only thin client showed new count value and SQL returned old
> count value:
>
> - ingesting through DataStreamer
> //I ran the jar on one of the Ignite nodes
> String CONFIG_FILE = ;
> Ignition.setClientMode(true);
> Ignite ignite = Ignition.start(CONFIG_FILE);
> IgniteDataStreamer stmr =
> ignite.dataStreamer("SQL_PUBLIC_FOO");
> stmr.addData(rowCount, value);
>
> - thin client:
>
> nodes = [
> (IP1, 10800),
> (IP2, 10800),
> ]
> client = Client()
> client.connect(nodes)
> cache = client.get_cache("SQL_PUBLIC_FOO")
> cache.get_size()
>
> returns 110
>
> - SQL through DBeaver
>
> SELECT COUNT(*) FROM FOO
>
> returns 10
>
> Would anyone shed some lights on what I did wrong? I would love to use
> DataStreamer to put much more data into the cache so that I would wan to be
> able to query them through SQL.
>
> Thanks for the help. I appreciate it.
>
> Regards,
> Calvin
>
