Re: Ignite startu is very slow

2018-12-03 Thread Evgenii Zhuravlev
Hi, You have a pretty small amount of heap and 30 caches. Each cache creates some overhead for node startup, because node should read states for each partition of each cache(and each partition is a separate file). You can reduce this overhead by configuring the same cache group:

Re: Ignite benchmarking with YCSB

2018-12-03 Thread summasumma
Hi Ilya, In early execution the writesyncmode=prim_sync was not set. I made this correction and enabled and able to get this performance. Even now CPU is not going above 50% in 3 node ignite cluster. network bandwidhth: 700 mbps rx and 350 tx. Not sure if there is any way now to improve above

Re: Ignite startu is very slow

2018-12-03 Thread kvenkatramtreddy
hi Team, any update or clue on above issue. Thanks & Regards, Venkat -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

iginte REST API using HTTPS

2018-12-03 Thread Shesha Nanda
Hi, I am trying add ssl security for the ignite to access iginte REST API using HTTPS. I have fallowed below steps: I have enabled the ssl by adding below configurations.

Re: GridProcessorAdapter fails to start due to failure to initialise WAL segment on Ignite startup

2018-12-03 Thread Raymond Wilson
Hi Ilya, I check the state of the WAL file in question (0008.wal) - it is a zero byte WAL file. The only other WAL file present in the same location is .wal (65536kb in size), which seems odd as WALfiles 0001.wal through 0007.wal are not

RE: When will Apache Ignite support Java 11?

2018-12-03 Thread Stanislav Lukyanov
The support for Java 9 *should* mean support for Java 11, the compatibility gap between the two is not big. Moreover, I would (and going to) push for almost completely skipping the testing on Java 9 – it is in end-of-life already, so providing support for it is kind of pointless. Java 11 is

RE: How to filter ip interfaces in TcpDiscoveryJdbcIpFinder

2018-12-03 Thread Stanislav Lukyanov
Currently you can only use IGNITE_LOCAL_HOST or TcpDiscoverySpi.localAddress for this. You can automate setting these addresses via an external script, like MY_IP=`ifconfig | grep ` java -DIGNITE_LOCAL_HOST=$MY_IP or put it into the Ignite config like Stan From: Luckyman Sent: 3

RE: Create index got stuck and freeze whole cluster.

2018-12-03 Thread Stanislav Lukyanov
Hi, The only thing I can say is that your troubles seem to have started way before. I see a bunch of “Found long running cache future” repeating, and then exchange for stopping SQL_PUBLIC_USERLEVEL cache that never completes. Would need logs going further (at least minutes) into the past to

RE: Avoiding Docker Bridge network when using S3 discovery

2018-12-03 Thread Stanislav Lukyanov
Hi, Have you been able to solve this? I think specifying TcpDiscoverySpi.localAddress should work. Stan From: Dave Harvey Sent: 17 октября 2018 г. 20:10 To: user@ignite.apache.org Subject: Avoiding Docker Bridge network when using S3 discovery When we use S3 discovery and Ignite containers

RE: Ignite can't activate

2018-12-03 Thread Stanislav Lukyanov
Hi, Reproduced that and filed https://issues.apache.org/jira/browse/IGNITE-10516. Thanks for reporting. Stan From: yangjiajun Sent: 29 ноября 2018 г. 10:52 To: user@ignite.apache.org Subject: Re: Ignite can't activate Hello. Here is a reproducer for my case: 1.Start a node with persistence

Re: Ignite Performance Issues when seeding data from Spark

2018-12-03 Thread kellan
I'm Using 2.6 on AWS. Like I mentioned, my Ignite cluster is running on i3 instances which have local storage, so burst shouldn't be a problem. The trend I've noticed is that my writes-per-second increases, while the size of each write decreases, and the number of PUT operations per second and

Re: [RESOLVED] JDBC Streaming

2018-12-03 Thread Ilya Kasnacheev
Hello! Yes, thin client has different approach to failover. However, it should not freeze. Please collect thread dumps so that we can see why it froze. The expectation here is that connection will be eventually dropped. I have found a reason for your Thick driver streaming troubles. Turns out,

RE: Fair queue polling policy?

2018-12-03 Thread Stanislav Lukyanov
I think what you’re talking about isn’t fairness, it’s round-robinness. You can’t distribute a single piece of work among multiple nodes fairly – one gets it and others don’t. Yes, it could be using different node each time, but it I don’t really a use case for that. The queue itself isn’t a

RE: Ignite not scaling as expected !!! (Thread dump provided)

2018-12-03 Thread Stanislav Lukyanov
How many cores does each node have? Numbers of which threads do you increase? The ones doing the get() calls? Thread dump from the client isn’t that interesting. Better to look what’s going on the servers. You need to monitor your resources – CPU, Network IO, Disk IO. You may hit the limit on

[RESOLVED] JDBC Streaming

2018-12-03 Thread joseheitor
Hooray!!! - It works. Thanks, Ilya. Please continue your investigation of the JDBC Client Driver (thick-client), and let me know what you find...? What follows should perhaps be posted separately...but here's something I noticed, which I don't fully understand or know how to deal with: While

Re: GridProcessorAdapter fails to start due to failure to initialise WAL segment on Ignite startup

2018-12-03 Thread Ilya Kasnacheev
Hello! It seems that WAL file got truncated or something like that. Can you post this file to some file storage so that we could look? You can also try to change this node's WAL mode to LOG_ONLY and try to start it again (after backing up data, of course). Checks are less strict in this case.

Re: Failed to read data from remote connection

2018-12-03 Thread wangsan
Do you shut down C++ node properly prior killing the process? Yeath, c++ node was killed by kill -9 .not sighup. It is a wrong ops,And I will use kill ops. Does this exceptions impacts cluster's functionality anyhow? I am not sure about the exceptions. My cluster will crash with oom

Re: Ignite distribution configuration.

2018-12-03 Thread Ilya Kasnacheev
Hello! It is a separate deliverable and not a part of Apache Ignite distribution. You can find it in Nightly Builds, for example: https://ci.ignite.apache.org/viewLog.html?buildId=lastSuccessful=Releases_NightlyRelease_RunApacheIgniteNightlyRelease=artifacts=1 I'm not sure if you can get it as

Re: Invalid property 'statisticsEnabled' is not writable

2018-12-03 Thread Ilya Kasnacheev
Hello! I'm pretty sure that I was able to run SQL queries on Apache Ignite. If there's some specific configuration (such as Spark), please prepare detailed steps to reproduce, reproducer project or film a video :) Regards, -- Ilya Kasnacheev пн, 3 дек. 2018 г. в 18:05, ApacheUser : > Hi

Re: Query regarding Ignite unit tests

2018-12-03 Thread Ilya Kasnacheev
Hello! There is no scenario where you would run all tests during mvn clean install. Normally, tests are run on per test suite basis with specific suite settings. It will make more than a day to run all tests and they might interfere each other. The proper way of running tests is

Re: Invalid property 'statisticsEnabled' is not writable

2018-12-03 Thread ApacheUser
Hi Ilya, I am able to start and run SQL Queryies but not able to write, while loading data this error is thrown. please try to write some data in any dummy table with couple fields. I am using affinity key and backups=1 . Thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Client stucks and doesn't connect

2018-12-03 Thread Dmitry Lazurkin
OK. I have found steps to reproduce. For reproducing we need Ignite with slow disk. Or I known how to emulate slow hard disk: Add to partitions cycle in GridCacheDatabaseSharedManager#restorePartitionStates: //...     for (int i = 0; i < grp.affinity().partitions(); i++) {    

Re: JDBC Streaming

2018-12-03 Thread Ilya Kasnacheev
Hello! Indeed, you will need to reconnect if node that you're connected to fails. It always supported returning cluster-wide data retrieval. Regarda, -- Ilya Kasnacheev пн, 3 дек. 2018 г. в 15:25, joseheitor : > Hi Ilya, > > Thanks for the response. > > My understanding was that Thin JDBC

Re: Ignite distribution configuration.

2018-12-03 Thread Viraj Rathod
I’m unable to find ignite-web-agent. I have stored space ignite in /usr/share/apache-ignite directory. Using RHEL 7 with CENTOS as the OS. On Mon, 3 Dec 2018 at 1:14 PM, Mikael wrote: > Hi! > > I don't think there is an easy answer, the configuration depends on so > many things, try the default

RE: Ignite cache.getAll takes a long time

2018-12-03 Thread Stanislav Lukyanov
I guess it could be caused by https://issues.apache.org/jira/browse/IGNITE-5003 which is mentioned in that thread. Also, make sure that your cache store code doesn’t cause you troubles – that you don’t open a new connection every time, don’t have unnecessary blocking, etc. Stan From: Justin

RE: Query regarding Ignite unit tests

2018-12-03 Thread Stanislav Lukyanov
Hi, This is better to be asked on the dev-list – added that to the To, and Bcc’ed user-list. I actually don’t think you can run tests for a specific module – either a single test, or a single test suite, or all of them. I would usually either run a single test from IDEA or run all tests via

Re: JDBC Streaming

2018-12-03 Thread joseheitor
Hi Ilya, Thanks for the response. My understanding was that Thin JDBC driver was only able to connect to a single node (not a cluster), so that if that node failed - it was not able to continue operating on the cluster... It would also only return data residing on that node (not records residing

Re: JDBC Streaming

2018-12-03 Thread Ilya Kasnacheev
Hello! Apache Ignite SQL should be accessed by Ignite JDBC Thin driver. This is the preferred way. JDBC Thin driver also has streaming mode in the form of SET STREAMING ON/OFF. Please see attached file where I have introduced "thin" mode. As for client mode streaming not working, I will look

Query regarding Ignite unit tests

2018-12-03 Thread Namrata Bhave
Hi, I have recently started working with Apache Ignite. Build on x86 Ubuntu 16.04 is complete. However, while running tests using `mvn test` command, the execution gets stuck while running `ignite-core` module. Hence started running tests on individual modules, where similar behavior was seen

Re: Ignite benchmarking with YCSB

2018-12-03 Thread Ilya Kasnacheev
Hello! Looks like your numbers have improved. What change would lead to such improvement? What's the current CPU utilization? Regards, -- Ilya Kasnacheev пн, 3 дек. 2018 г. в 12:48, summasumma : > HI Ilya, > > Thanks for all the inputs. > > Latest update is: > > Attempted Insert: 120k > 2

RE: Ignite cache.getAll takes a long time

2018-12-03 Thread Justin Ji
Stan - Thank for your reply! Yes, the getAll and putAll(async) executed in parallel(amount of operations executed at the same time). But I think it may be caused by the write-behind, when I disabled the write-behind the timeout disappeared, and I enabled the write-behind the timeout appeared.

Re: Ignite benchmarking with YCSB

2018-12-03 Thread summasumma
HI Ilya, Thanks for all the inputs. Latest update is: Attempted Insert: 120k 2 YCSB each with 80 threads giving: 54.5k / 54.5k = total 109k - Failed to touch target 120k - This is with Primary_Sync enabled in ignite but without Threadpool=64 or connection pair. Increasing System/Public