How to make full use of network bandwidth?

[Michael Jay]

Hi,
I have two physical hosts, each host has a sever node, node A and node B. I
created a table and inserted data into table via JDBCthin driver. But the
performance of inserting is poor. When the bandwidth of network is 100Mbps,
the throughput from node A to node B is only about 2Mbps, 1 million rows
need 800s to be finished. Then I only changed the bandwidth of network to
1Gbps, the throughput from node A to node B is up to 10Mbps, 1 million rows
need 250s to be finished. Moreover, the cacheMode is "partitioned". 
How to explain this case? Is there some limitaiton or configuration on
network? What did I miss? How can How can we  use all network bandwidth?
Thanks!!
  default-config.xml
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: JDBC Thin Client with Transactions

[Teki]

Hi Denis,

When this feature will be available? Is this why I am getting the error
mentioned in the below post?
Please assist. Thanks in advance !!

http://apache-ignite-users.70518.x6.nabble.com/insert-into-ignite-cache-from-Netezza-database-using-talend-tp19295.html



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

insert into ignite cache from Netezza database using talend

[Kavin]

Hi,

I am using talend to insert data from netezza database to ignite cache. The
Netezza SQL query has joins with many tables. I am getting the below error.
Are inserts supported? The server was started using a java code. The
attached image contains the connection details.
*Error:*

[statistics] connecting to socket on port 3343
[statistics] connected
Jan 04, 2018 8:35:54 PM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file:
config/java.util.logging.properties
Exception in component tJDBCOutput_1 (NzToCache)
*java.sql.SQLFeatureNotSupportedException: Transactions are not supported.*
at
org.apache.ignite.internal.jdbc.JdbcConnection.setAutoCommit(JdbcConnection.java:170)
at
local_project.nztocache_0_1.NzToCache.tNetezzaInput_1Process(NzToCache.java:463)
at 
local_project.nztocache_0_1.NzToCache.runJobInTOS(NzToCache.java:1027)
at local_project.nztocache_0_1.NzToCache.main(NzToCache.java:876)
[statistics] disconnected
Job NzToCache ended at 20:35 04/01/2018. [exit code=1]
 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

XML Format for Expiry Policies

[rizal123]

Hi all,

Would you mind to create xml format for this java syntax.

https://apacheignite.readme.io/v2.3/docs/expiry-policies

cfg.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Duration.ONE_MINUTE));

thanks for your help.

regards.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Connection problem between client and server

[Jeff Jiao]

Hi Denis,

Thanks for the analysis.

Today we tried to reduce the Java class path(.../lib/*) when start a server
node, but it still sends all the jar files(absolute path) to client.

and we also find that, it looks like client node sends its info(path...) to
server node, and then server node sends the client's info back to client
again...

for the TCP window increasing, does Ignite has some attribute that we can
configure? or we may need to modify our production system.

btw, we are using Ignite 2.3.0.

Thanks,
Jeff



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: java.lang.IllegalStateException: Cache has been closed:

[Denis Mekhanikov]

Hi Rajarshi!

Well, it must be, that you are actually closing the cache.
Check, that you didn't put cache creation into a try-with-resources.
When *IgniteCache.close()* is called, then cache is destroyed.

Denis

чт, 4 янв. 2018 г. в 18:17, Rajarshi Pain :

> Hi,
>
> I am using Ignite 2.3 for my application. But got Cache closed exception
> while running.
>
> This is a multi threaded application through testing on single thread. it
> will load the reference number from database to cache during startup and
> while processing a message, it will check if that particular reference
> number is present or not on that cache.
>
> if there is no entry it will add refernce_number-approved else
> refernce_number-rejected.
>
> I was testing with 2000 message but for some of the message during the
> validation it's throwing *Cache has been closed *exception.
>
> while tested the same code with 10 messages didnt get any exception.
>
> couldnt figure it out why I am getting this exception while the cache is
> active ?
>
> *java.lang.IllegalStateException: Cache has been closed:
> ReferenceStatusCache*
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.checkProxyIsValid(GatewayProtectedCacheProxy.java:1639)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.onEnter(GatewayProtectedCacheProxy.java:1654)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:869)
> at
> sample.example.ProcessingConsumerThread.populateMsgObj(ProcessingConsumerThread.java:232)
> at
> sample.example.ProcessingConsumerThread..run(ProcessingConsumerThread.java:164)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4.execute(GridClosureProcessor.java:1944)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> --
> Regards,
> Raj
>

Re: off heap memory usage

[Denis Mekhanikov]

Hi!

Looks like this metric shows only non-heap memory, consumed by JVM itself.
Looks like a bug.
If you want to get a picture of how much memory Ignite pages consumed, you
can get *DataRegionMetrics#PhysicalMemoryPages* and multiply it by a size
of a data page.
To do this you should enable memory metrics first. Here you can find how:
https://apacheignite.readme.io/docs/memory-metrics
Note, that pages may be not full, will still be included into this metric.

Denis

чт, 4 янв. 2018 г. в 11:05, shawn.du :

> Hi community,
>
> I want monitor ignite cluster off heap memory usage. if off heap available
> memory is less than 1G then alert.
> I went through ignite MBean,  find ClusterLocalNodeMetricsMXBeanImpl's
> NonHeapMemoryUsed.
> The value is too small. I don't think it is.  How can I get it?
>
> Thanks
> Shawn
>

java.lang.IllegalStateException: Cache has been closed:

[Rajarshi Pain]

Hi,

I am using Ignite 2.3 for my application. But got Cache closed exception
while running.

This is a multi threaded application through testing on single thread. it
will load the reference number from database to cache during startup and
while processing a message, it will check if that particular reference
number is present or not on that cache.

if there is no entry it will add refernce_number-approved else
refernce_number-rejected.

I was testing with 2000 message but for some of the message during the
validation it's throwing *Cache has been closed *exception.

while tested the same code with 10 messages didnt get any exception.

couldnt figure it out why I am getting this exception while the cache is
active ?

*java.lang.IllegalStateException: Cache has been closed:
ReferenceStatusCache*
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.checkProxyIsValid(GatewayProtectedCacheProxy.java:1639)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.onEnter(GatewayProtectedCacheProxy.java:1654)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:869)
at
sample.example.ProcessingConsumerThread.populateMsgObj(ProcessingConsumerThread.java:232)
at
sample.example.ProcessingConsumerThread..run(ProcessingConsumerThread.java:164)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4.execute(GridClosureProcessor.java:1944)
at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631)
at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

-- 
Regards,
Raj

Re: H2 console - GridH2QueryContext is not initialized

[Rajarshi Pain]

Hi Slava,

Yes it might be the root cause as i have set QueryParallelism as 4.
On Thu 4 Jan, 2018, 20:25 slava.koptilin,  wrote:

> Hi Rajarshi,
>
> I was able to reproduce the issue.
> It seems that using CacheConfiguration#QueryParallelism greater than 1 may
> lead to this behavior.
> I will investigate this use case further (will create a JIRA ticket for
> that
> bug).
>
> Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
-- 

Thanks
Rajarshi

Re: H2 console - GridH2QueryContext is not initialized

[slava.koptilin]

Hi Rajarshi,

I was able to reproduce the issue.
It seems that using CacheConfiguration#QueryParallelism greater than 1 may
lead to this behavior.
I will investigate this use case further (will create a JIRA ticket for that
bug).

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

dotnet thin client - multiple hosts?

[Colin Green]

IgniteClientConfiguration.Host is currently a string property that appears
to accept a single dotted IP address, or single host name.

However, in the event that the single configured host/node goes offline, it
would be useful if the client would attempt to dynamically connect to one
or more additional nodes/hosts. I understand the dotnet redis client
operates in this manner.

Thoughts?

Colin

Re: Rest API GET command syntax with a composite primary key

[Evgenii Zhuravlev]

Hi,

According to these tickets, now REST protocol interpret everything as String
:
https://issues.apache.org/jira/browse/IGNITE-3345
https://issues.apache.org/jira/browse/IGNITE-962

At this moment you can use *ConnectorMessageInterceptor *interface. Once
you implement and configure it, you will start receiving a callback for all
keys and values passed back and forth. So you can encode your object as a
String somehow and then convert it to a real object inside interceptor. And
the opposite: before returning object from the cache you can convert it to
some String form.

This is not very convenient, though it will allow you to go further without
awaiting any tickets to be implemented.

Evgenii


2018-01-04 14:34 GMT+03:00 Naveen :

> Hi
>
> Am using 2.3
>
> What is the GET syntax for REST API for a cache which has a composite key.
>
> I have a cache which has a composite key - consisting of 2 columns
>
> Thanks
> Naveen
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Cursor in TextQuery - first hasNex() is slow

[zbyszek]

Thank you Val,

I am afraid it is still not clear to me

What I was trying to explain above it that regardless the page size (1 or 10
or default), the duration of first next() or hasNext() is always equal to
duration of fetching all data with getAll().
 - in my example above pageSize is 10.

This is a big problem for me as with 10 mln. entities  sometimes (depending
on term) I need to wait 5 sec. to retrieve first row. This disqualifies
TextQuery as a solution for our auto-complete feature.

Is there a way to utilize the fact that I need only first 10 records, so the
response time is a small fraction of the time needed to fetch all data?

thanx,
zbyszek





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Rest API GET command syntax with a composite primary key

[Naveen]

Hi

Am using 2.3

What is the GET syntax for REST API for a cache which has a composite key.

I have a cache which has a composite key - consisting of 2 columns

Thanks
Naveen



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Connection problem between client and server

[Denis Mekhanikov]

Jeff,

Your have indeed a lot of data sent over discovery. Almost all of it is
taken by node attributes. Try reducing Java classpath and PATH environment
variable. Also increasing of TCP window could help, because now a lot of
small messages are sent instead of a few big ones.

Cross-sending it to dev list:
Guys, turns out, we are sending a lot of irrelevant information in
discovery messages. Some messages contain *TcpDiscoveryNode* objects, which
in turn have such attributes like *PATH, java.class.path,
sun.boot.class.path, java.library.path, org.apache.ignite.jvm.args, *etc.
Some of these attributes may contain huge strings, that can sum up to
megabytes of data.

I think, we should filter-out such attributes, because they are not used
anywhere, but  make messages grow enormous and slow down discovery.

What do you think?

Denis

чт, 4 янв. 2018 г. в 10:42, Jeff Jiao :

> dev2.cap <
> http://apache-ignite-users.70518.x6.nabble.com/file/t757/dev2.cap>
>
> Hi Denis,
>
> the attachment is the packages that server transport to client when
> connect,
> we catched it using tool (Microsoft network monitor), can you please take a
> look and see what you can find?
>
> we only configured 12 caches in our Ignite, most of them have 3 or 4
> indexes, only few has 7 or 8 indexes configured...
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

off heap memory usage

[shawn.du]

Hi community,I want monitor ignite cluster off heap memory usage. if off heap available memory is less than 1G then alert.
I went through ignite MBean,  find ClusterLocalNodeMetricsMXBeanImpl's NonHeapMemoryUsed. The value is too small. I don't think it is.  How can I get it?






ThanksShawn