Re: Using Ignite as blob store?
Thanks, Ilya. The blob size varies from a few KBs to a few MBs. Cheers, Shane On Thu, Aug 22, 2019 at 5:02 AM Ilya Kasnacheev wrote: > Hello! > > How large are these blobs? Ignite is going to divide blobs into <4k > chunks. We have no special optimizations for storing large key-value pairs. > > Regards, > -- > Ilya Kasnacheev > > > чт, 22 авг. 2019 г. в 02:53, Shane Duan : > >> Hi Igniters, is it a good idea to use Ignite(with persistence) as a blob >> store? I did run some testing with a small dataset, and it looks performing >> okay, even with a small off-heap mem for the data region. >> >> Thanks! >> >> Shane >> >
Using Ignite as blob store?
Hi Igniters, is it a good idea to use Ignite(with persistence) as a blob store? I did run some testing with a small dataset, and it looks performing okay, even with a small off-heap mem for the data region. Thanks! Shane
Re: Set Expiry Policies When Creating Cache Using Java Thin Client
Yes, I know I can set the template in SQL (via JDBC) or REST API. But how can I set the template on the Java thin client? Thanks! On Mon, Jul 8, 2019 at 5:55 AM Ilya Kasnacheev wrote: > Hello! > > Yes, you can declare cache configuration templates, refer to them when > creating tables from thin client/JDBC, as per documentation: > https://apacheignite.readme.io/docs/cache-template > > Regards, > -- > Ilya Kasnacheev > > > пт, 5 июл. 2019 г. в 21:22, Shane Duan : > >> Thanks, Denis. Alternatively, does ignite thin client provide a way to >> use a server side pre-defined cache configurations, just like the TEMPLATE >> in the CREATE TABLE statement in SQL? >> >> Thanks, >> Shane >> >> On Wed, Jul 3, 2019 at 2:41 PM Denis Magda wrote: >> >>> Shane, >>> >>> That's unavailable on the thing clients end yet. My suggestion is to >>> configure caches with required expiration policies on the servers' >>> configuration end. >>> >>> - >>> Denis >>> >>> >>> On Tue, Jul 2, 2019 at 10:54 AM Shane Duan wrote: >>> >>>> I mean ClientCacheConfiguration >>>> >>>> On Tue, Jul 2, 2019 at 10:48 AM Shane Duan wrote: >>>> >>>>> Hi Igniters, >>>>> >>>>> How can I set expiry policies if I have to create cache using Java >>>>> Thin client. I did not see any API to do so in ClientConfiguration. >>>>> >>>>> Thanks, >>>>> Shane >>>>> >>>>
Re: Set Expiry Policies When Creating Cache Using Java Thin Client
Thanks, Denis. Alternatively, does ignite thin client provide a way to use a server side pre-defined cache configurations, just like the TEMPLATE in the CREATE TABLE statement in SQL? Thanks, Shane On Wed, Jul 3, 2019 at 2:41 PM Denis Magda wrote: > Shane, > > That's unavailable on the thing clients end yet. My suggestion is to > configure caches with required expiration policies on the servers' > configuration end. > > - > Denis > > > On Tue, Jul 2, 2019 at 10:54 AM Shane Duan wrote: > >> I mean ClientCacheConfiguration >> >> On Tue, Jul 2, 2019 at 10:48 AM Shane Duan wrote: >> >>> Hi Igniters, >>> >>> How can I set expiry policies if I have to create cache using Java Thin >>> client. I did not see any API to do so in ClientConfiguration. >>> >>> Thanks, >>> Shane >>> >>
Re: Thick client/thin client difference and how to get thin client port
Thank a lot, Alex. You saved my day :) .
On Wed, Jul 3, 2019 at 12:14 AM Alex Plehanov
wrote:
> Hello,
>
> There was a bug [1] with one node thin client configuration, which is
> fixed now but was not released yet.
> You can try to add one more server address (the same address) as a
> workaround. For example:
>
> ClientConfiguration cfg = new ClientConfiguration().setAddresses(hostName +
> ":" + portNumber, hostName + ":" + portNumber);
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-11649
>
> вт, 2 июл. 2019 г. в 20:36, Shane Duan :
>
>> I am having similar problem: I am testing with a one-node-cluster. After
>> restart the Ignite cluster(server), Java thin client is not reconnecting.
>> There is no port change on server for my case since I defined client port
>> in the Ignite configuration:
>>
>> org.apache.ignite.client.ClientConnectionException: Ignite cluster is
>> unavailable
>> at
>> org.apache.ignite.internal.client.thin.TcpClientChannel.read(TcpClientChannel.java:333)
>> at
>> org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:154)
>> at
>> org.apache.ignite.internal.client.thin.ReliableChannel.service(ReliableChannel.java:126)
>> at
>> org.apache.ignite.internal.client.thin.TcpClientCache.get(TcpClientCache.java:82)
>> at
>> com.esri.arcgis.datastore.model.cachestore.BinaryTest.main(BinaryTest.java:69)
>>
>> import org.apache.ignite.IgniteBinary;
>> import org.apache.ignite.Ignition;
>> import org.apache.ignite.binary.BinaryObject;
>> import org.apache.ignite.client.ClientCache;
>> import org.apache.ignite.client.IgniteClient;
>> import org.apache.ignite.configuration.ClientConfiguration;
>>
>> public class BinaryTest {
>>
>> public static void main(String[] args) {
>>
>> IgniteClient igniteClient = null;
>>
>> try{
>>
>> // Prepare a Ignite thin client
>> String hostName = args[0];
>> Integer portNumber = Integer.parseInt(args[1]);
>> String userName = args[2];
>> String password = args[3];
>>
>> ClientConfiguration cfg = new
>> ClientConfiguration().setAddresses(hostName + ":" + portNumber);
>> cfg.setUserName(userName);
>> cfg.setUserPassword(password);
>>
>> igniteClient = Ignition.startClient(cfg);
>>
>> // Get IgniteBinary object using the ignite thin client.
>> IgniteBinary binary = igniteClient.binary();
>>
>> // Build two test binary object.
>> // Note: we are defining the attributes for the binary object on the
>> fly.
>> BinaryObject val1 = binary.builder("person")
>> .setField("id", 1, int.class)
>> .setField("name", "Tom J", String.class)
>> .build();
>> BinaryObject val2 = binary.builder("person")
>> .setField("id", 2, int.class)
>> .setField("name", "Jeffson L", String.class)
>> .build();
>>
>> // Create a cache, keep it as binary.
>> ClientCache cache =
>> igniteClient.getOrCreateCache("persons").withKeepBinary();
>>
>> // Store the testing objects.
>> cache.put(1, val1);
>> cache.put(2, val2);
>>
>>
>>
>> // Please restart Ignite server here.
>>
>>
>> // Get the objects.
>> BinaryObject cachedVal1 = cache.get(1);
>> System.out.println("Person1");
>> System.out.println("\tID = " + cachedVal1.field("id"));
>> System.out.println("\tName = " + cachedVal1.field("name"));
>>
>> BinaryObject cachedVal2 = cache.get(2);
>> System.out.println("Person2");
>> System.out.println("\tID = " + cachedVal2.field("id"));
>> System.out.println("\tName = " + cachedVal2.field("name"));
>>
>> // Destroy the cache.
>> System.out.print("Dropped caches...");
>> igniteClient.destroyCache("persons");
>>
>> } catch (Exception e) {
>> e.printStackTrace();
>> } finally {
>> if (igniteClient != null) {
>> try {
>> igniteClient.close();
>> } catch (Exception ignore) {
>> }
>> }
>> }
>> }
>>
>> }
>>
>>
>> On Mon, Jul 1, 2019 at 5:38 AM Alex Plehanov
>> wrote:
>>
>>> Hello, Rick
>>>
>>> There should be a message in the server log:
>>> "Client connector processor has started on TCP port x" with "INFO"
>>> level.
>>>
>>> пн, 1 июл. 2019 г. в 10:26, rick_tem :
>>>
>>>> Thanks for your response. I tried 10801 and it failed. My question is
>>>> how
>>>> do I find out which port it is using?
>>>>
>>>> Thanks,
>>>> Rick
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
Set Expiry Policies When Creating Cache Using Java Thin Client
Hi Igniters, How can I set expiry policies if I have to create cache using Java Thin client. I did not see any API to do so in ClientConfiguration. Thanks, Shane
Re: Thick client/thin client difference and how to get thin client port
I am having similar problem: I am testing with a one-node-cluster. After
restart the Ignite cluster(server), Java thin client is not reconnecting.
There is no port change on server for my case since I defined client port
in the Ignite configuration:
org.apache.ignite.client.ClientConnectionException: Ignite cluster is
unavailable
at
org.apache.ignite.internal.client.thin.TcpClientChannel.read(TcpClientChannel.java:333)
at
org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:154)
at
org.apache.ignite.internal.client.thin.ReliableChannel.service(ReliableChannel.java:126)
at
org.apache.ignite.internal.client.thin.TcpClientCache.get(TcpClientCache.java:82)
at
com.esri.arcgis.datastore.model.cachestore.BinaryTest.main(BinaryTest.java:69)
import org.apache.ignite.IgniteBinary;
import org.apache.ignite.Ignition;
import org.apache.ignite.binary.BinaryObject;
import org.apache.ignite.client.ClientCache;
import org.apache.ignite.client.IgniteClient;
import org.apache.ignite.configuration.ClientConfiguration;
public class BinaryTest {
public static void main(String[] args) {
IgniteClient igniteClient = null;
try{
// Prepare a Ignite thin client
String hostName = args[0];
Integer portNumber = Integer.parseInt(args[1]);
String userName = args[2];
String password = args[3];
ClientConfiguration cfg = new
ClientConfiguration().setAddresses(hostName + ":" + portNumber);
cfg.setUserName(userName);
cfg.setUserPassword(password);
igniteClient = Ignition.startClient(cfg);
// Get IgniteBinary object using the ignite thin client.
IgniteBinary binary = igniteClient.binary();
// Build two test binary object.
// Note: we are defining the attributes for the binary object on the fly.
BinaryObject val1 = binary.builder("person")
.setField("id", 1, int.class)
.setField("name", "Tom J", String.class)
.build();
BinaryObject val2 = binary.builder("person")
.setField("id", 2, int.class)
.setField("name", "Jeffson L", String.class)
.build();
// Create a cache, keep it as binary.
ClientCache cache =
igniteClient.getOrCreateCache("persons").withKeepBinary();
// Store the testing objects.
cache.put(1, val1);
cache.put(2, val2);
// Please restart Ignite server here.
// Get the objects.
BinaryObject cachedVal1 = cache.get(1);
System.out.println("Person1");
System.out.println("\tID = " + cachedVal1.field("id"));
System.out.println("\tName = " + cachedVal1.field("name"));
BinaryObject cachedVal2 = cache.get(2);
System.out.println("Person2");
System.out.println("\tID = " + cachedVal2.field("id"));
System.out.println("\tName = " + cachedVal2.field("name"));
// Destroy the cache.
System.out.print("Dropped caches...");
igniteClient.destroyCache("persons");
} catch (Exception e) {
e.printStackTrace();
} finally {
if (igniteClient != null) {
try {
igniteClient.close();
} catch (Exception ignore) {
}
}
}
}
}
On Mon, Jul 1, 2019 at 5:38 AM Alex Plehanov
wrote:
> Hello, Rick
>
> There should be a message in the server log:
> "Client connector processor has started on TCP port x" with "INFO"
> level.
>
> пн, 1 июл. 2019 г. в 10:26, rick_tem :
>
>> Thanks for your response. I tried 10801 and it failed. My question is
>> how
>> do I find out which port it is using?
>>
>> Thanks,
>> Rick
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: How much data is actually in RAM?
It guess really depends how you use the cluster. For my use case, I have to bump up heap to 8G to support concurrent queries (JDBC); otherwise, Ignite could crash since Ignite need process results in heap. On Fri, Jun 14, 2019 at 2:02 PM John Smith wrote: > Hi, so I have 3 machines with 8GB RAM and 96GB disk each. > > I have configured the persistence as > > class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > > > > > > Looking at the logs: > Topology snapshot [ver=3, locNode=xx, servers=3, clients=0, > state=INACTIVE, CPUs=12, offheap=216.0GB, heap=3.0GB] > > 1- If I get this right... Each node is using about 1GB on-heap JVM memory > each for the ignite process to run. Correct? Should this be set a bit > higher or that's ok? > 2- There is total 216GB off-heap so about 72GB off-heap per node. So how > much of that is stored in actual RAM in the remainder of what is left from > 8GB of the physical host? Is that the 20% value indicated in the docs? Or > should maxSize be set to 6L and not 72L giving 2GB free to the OS and > ignite process? > > > > >
Ignite runs out heap during query (via SQL)
Just wondering, how JVM heap is used in Ignite (Version 2.7) during query. With persistence enabled, I am assuming everything will be stored in the off-heap memory or disk. But during query time, query is failing because Ignite is running heap space. In my test, I have to increase heap size for Ignite to 8Gb with about 5-10 concurrent queries. My guess is that Ignite need to process query results using heap. Is that right? If that is the case, any recommendation on JVM heap setting for Ignite? Thanks, Shane
Re: JDK 11 support.
Very much appreciated, Dmitriy! On Tue, Mar 12, 2019 at 2:15 PM Dmitriy Pavlov wrote: > Hi Shane, > > These flags are required to access JVM internals and are used by Ignite. > And it is not related to production readiness. > > A number of projects require these flags. In theory in some future release > Ignite can get rid of the mandatory specification of extra flags, but it > will anyway affect performance. So in this scenario (if community accept > it), Ignite will recommend to set it up but will be (much) slower without > it. > > There are a number of open discussions at dev@ related to Java 11, > modularity support. So AFAIK there are no exact plans. > > Sincerely, > Dmitriy Pavlov > > вт, 12 мар. 2019 г. в 20:48, Shane Duan : > >> Currently running Ignite 2.7 with OpenJDK11, with these additional JVM >> flags: >> >> --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED >> --add-exports=java.base/sun.nio.ch=ALL-UNNAMED >> --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED >> --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED--add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED--illegal-access=permit-Djdk.tls.client.protocols=TLSv1.2 >> >> >> It is working, but also brought some concerns whether Ignite is >> production-ready... >> >> Any plan to remove these dependency on these flags in next release? >> >> Thanks! >> >> -Shane >> >> >> >> >>
JDK 11 support.
Currently running Ignite 2.7 with OpenJDK11, with these additional JVM flags: --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED--add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED--illegal-access=permit-Djdk.tls.client.protocols=TLSv1.2 It is working, but also brought some concerns whether Ignite is production-ready... Any plan to remove these dependency on these flags in next release? Thanks! -Shane
