Customized affinity function

[Prasad Bhalerao]

Hi,

I have following case.

I have around 1 subscriptions. Each subscription has data varying from
1 to 10 million.
Currently the affinity key is set on subscriptionId.

e.g:

Subscription1: 2 million rows
Subscription2: 10 million rows
Subscription3: 50 million rows
Subscription5: 30 million rows
Subscription6: 2 rows

Now I want make sure that all big subscription should be evenly distributed
across all nodes.

I meant Subscription2,Subscription3,Subscription4,Subscription5 should not
land on same node based on their affinity id.

Is there any way with which I can distribute all big subscriptions across
all node evenly or in round robin fashion?


Thanks,
Prasad

Issue IGNITE-3471

[Prasad Bhalerao]

https://issues.apache.org/jira/browse/IGNITE-3471

Can we please have a fix for this issue in ignite 2.5 version?


Reason:

I am trying to test the distributed transaction support using following
piece of code. While debugging the code I observed that code executes on
client node first and after doing commit the code executes on a node which
owns that kay.

What I am trying to do is, to collocate the data to avoid the network call
as my data in real use case is going to big. But while debugging the code,
I observed that entry processor first executes on client node, gets all the
data executes the task. and after commit executes the same code on remote
node.

Can someone please explain this behavior? My use case to execute the task
on nodes which owns the data in single transaction.

private static void executeEntryProcessorTransaction(IgniteCache cache) {
Person val=null;
try (Transaction tx = Ignition.ignite().transactions().txStart(
TransactionConcurrency.OPTIMISTIC,TransactionIsolation.SERIALIZABLE)) {
  long myid =6l;
CacheEntryProcessor entryProcessor = new MyEntryProcessor();
cache.invoke(myid, entryProcessor);
System.out.println("Overwrote old value: " + val);
val = cache.get(myid);
System.out.println("Read value: " + val);

tx.commit();
System.out.println("Read value after commit: " +   cache.get(myid));
}
}

Thanks,
Prasad

Re: Readonly nodes

[Alew]

That is ok, I don't need Ignite to do authentication. Just prevent data 
modification.


Denis told about node filters, but i didn't get how it is working


On 11/04/2018 02:56, vkulichenko wrote:

Ignite doesn't have authentication/authorization capabilities at the moment,
however there are certain plans for that as far as I know.

In the meantime, you can take a look 3rd party vendors like GridGain that
have paid offerings for this:
https://docs.gridgain.com/docs/security-and-audit

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Slow invoke call

[vkulichenko]

Hi Sam,

What does this byte array represent? You have to array copies in the entry
processor, so I'm not surprised it doesn't perform very well. That's also
the reason why it gets worse when size of the array is increased. The fact
that you're using off-heap also make it worse in this case as it basically
adds another two copies to read from off-heap and then write back.

I would consider revisiting your data model. Is it possible, for example, to
create a new entry instead of appending to an existing array?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Readonly nodes

[vkulichenko]

Ignite doesn't have authentication/authorization capabilities at the moment,
however there are certain plans for that as far as I know.

In the meantime, you can take a look 3rd party vendors like GridGain that
have paid offerings for this:
https://docs.gridgain.com/docs/security-and-audit

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to query cross-node Cache data with JDBC correlation?

[vkulichenko]

Hi,

Unfortunately, I doubt this description would be efficient to understand
what's going on. Do you have a reproducer that you can share with us?

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: JMX-MBean Reports OffHeapAllocatedSize as zero

[vkulichenko]

Hi Christoph,

Can you try 2.4.0? I know there were some fixes for memory metrics, so there
is a big chance you will not reproduce the issue with the latest version.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Readonly nodes

[Alew]

I use Ignite as a session store and need to allow write access only for 
authentication services.



On 11/04/2018 00:45, Denis Magda wrote:
The filter excludes the nodes from the list of those which can store 
data. That's why those nodes would never receive any requests.


Considering your follow-up questions, I guess you are looking for 
multi-tenancy capabilities. Do you want to prevent some of your 
applications to update data stored in the cluster?


--
Denis

On Tue, Apr 10, 2018 at 11:49 AM, Alew > wrote:


Hi, Denis

Thank you for your answer.

What effect will filter have? Does it mean that read-only node
doesn't have own data copy? Is it like partitioned cache? If so,
why is it read-only?



On 10/04/2018 02:37, Denis Magda wrote:

Hi,

Yes, you can tap into NodeFilter interface applying it for your
CacheConfiguration in a way similar to ServiceConfiguration as
shown here:

https://apacheignite.readme.io/docs/service-grid#section-node-filter-based-deployment



--
Denis

On Mon, Apr 9, 2018 at 4:23 PM, Alew > wrote:

Hi!
 I have a relicated cache. Is there a way to forbid insert
and update data in the cache for some nodes in a cluster?

Regards

Re: Readonly nodes

[Denis Magda]

The filter excludes the nodes from the list of those which can store data.
That's why those nodes would never receive any requests.

Considering your follow-up questions, I guess you are looking for
multi-tenancy capabilities. Do you want to prevent some of your
applications to update data stored in the cluster?

--
Denis

On Tue, Apr 10, 2018 at 11:49 AM, Alew  wrote:

> Hi, Denis
>
> Thank you for your answer.
>
> What effect will filter have? Does it mean that read-only node doesn't
> have own data copy? Is it like partitioned cache? If so, why is it
> read-only?
>
>
>
> On 10/04/2018 02:37, Denis Magda wrote:
>
> Hi,
>
> Yes, you can tap into NodeFilter interface applying it for your
> CacheConfiguration in a way similar to ServiceConfiguration as shown here:
> https://apacheignite.readme.io/docs/service-grid#section-
> node-filter-based-deployment
>
> --
> Denis
>
> On Mon, Apr 9, 2018 at 4:23 PM, Alew  wrote:
>
>> Hi!
>>  I have a relicated cache. Is there a way to forbid insert and update
>> data in the cache for some nodes in a cluster?
>>
>> Regards
>>
>>
>
>

Re: Ignite Sql query problem using IN clause

[mike-griggs]

There is a suggested workaround in
https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-sql-performance-and-usability-considerations,
item 2

 As a workaround...you can rewrite the query in the following way: select
p.name from Person p join table(id bigint = ?) i on p.id = i.id.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Readonly nodes

[Alew]

Hi, Denis

Thank you for your answer.

What effect will filter have? Does it mean that read-only node doesn't 
have own data copy? Is it like partitioned cache? If so, why is it 
read-only?




On 10/04/2018 02:37, Denis Magda wrote:

Hi,

Yes, you can tap into NodeFilter interface applying it for your 
CacheConfiguration in a way similar to ServiceConfiguration as shown here:

https://apacheignite.readme.io/docs/service-grid#section-node-filter-based-deployment

--
Denis

On Mon, Apr 9, 2018 at 4:23 PM, Alew > wrote:


Hi!
 I have a relicated cache. Is there a way to forbid insert and
update data in the cache for some nodes in a cluster?

Regards

JMX-MBean Reports OffHeapAllocatedSize as zero

[Christoph]

Hi,
I use a partitioned cache with off heap in Ignite 2.3.0. After putting some
values I see in JMX MBean
org.apache"org.apache.ignite.internal.processors.cache.CacheClusterMetricsMXBeanImpl"
some entries with prefix OffHeap.
Value OffHeapEntriesCount is the number of stored entries and now 3178.
*Value OffHeapAllocatedSize is 0.*
But CachePuts is 3178, AveragePutTime is 328.502 and
OffHeapPrimaryEntriesCount is 3178.
The 3178 values can be retrieved from the cache. So I believe they are
stored in off heap.

I also run a test with 1 million entries and OffHeapAllocatedSize states
zero.

*Is this expected behaviour?*

Kind regards,
Christoph

Re: Delete SQL is failing with IN clause for a table which has composite key

[dkarachentsev]

Hi Naveen,

Unfortunately I'm unable to reproduce that error. Could you please attach
simple code/project that fails with specified exception?

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Sql query problem using IN clause

[Naveen]

Has it been fixed in 2.4 ?

Looks like it is not, when I generate the explain plan for a query which has
a composite key

Here are the queries I ran and explain plan results.
When I use IN, it uses the full table scan, otherwise It does look up on
primary key

explain SELECT * FROM "MapLonglistCache".MAP_LONGLIST  WHERE 
(ENTITY_ID,RELATIONSHIP,SEQUENCE_NO) = ('12345', 'OutageID_ServiceIDs', '9') 

SELECT
__Z0.ENTITY_ID AS __C0_0,
__Z0.MAPPING_ID_LIST AS __C0_1,
__Z0.RELATIONSHIP AS __C0_2,
__Z0.UPDATEDDATETIME AS __C0_3,
__Z0.UPDATEDBY AS __C0_4,
__Z0.SEQUENCE_NO AS __C0_5,
__Z0.TUPLE_COUNT AS __C0_6
FROM "MapLonglistCache".MAP_LONGLIST __Z0
/* "MapLonglistCache".MAP_LONGLIST.__SCAN_ */
WHERE (__Z0.ENTITY_ID, __Z0.RELATIONSHIP, __Z0.SEQUENCE_NO) = ('12345',
'OutageID_ServiceIDs', '9')

explain SELECT * FROM "MapLonglistCache".MAP_LONGLIST  WHERE 
ENTITY_ID = '12345' AND 
RELATIONSHIP = 'OutageID_ServiceIDs' AND 
SEQUENCE_NO='9' 


SELECT
__Z0.ENTITY_ID AS __C0_0,
__Z0.MAPPING_ID_LIST AS __C0_1,
__Z0.RELATIONSHIP AS __C0_2,
__Z0.UPDATEDDATETIME AS __C0_3,
__Z0.UPDATEDBY AS __C0_4,
__Z0.SEQUENCE_NO AS __C0_5,
__Z0.TUPLE_COUNT AS __C0_6
FROM "MapLonglistCache".MAP_LONGLIST __Z0
/* "MapLonglistCache".MAP_LONGLIST_PK: SEQUENCE_NO = '9'
AND ENTITY_ID = '12345'
AND RELATIONSHIP = 'OutageID_ServiceIDs'
 */
WHERE (__Z0.SEQUENCE_NO = '9')
AND ((__Z0.ENTITY_ID = '12345')
AND (__Z0.RELATIONSHIP = 'OutageID_ServiceIDs'))

Any suggestions on this.

Thanks
naveen



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Delete SQL is failing with IN clause for a table which has composite key

[Naveen]

Hi

AM using 2.4

My delete statement is failing when I pass multiple keys, table has a
composite key with 3 columns. 

Have mentioned the table structure along with a failed query and successful
query and the error I am seeing the 

Here is the table created Java API

CREATE TABLE MapLonglistCache.MAP_LONGLIST (
ENTITY_ID VARCHAR,
MAPPING_ID_LIST VARCHAR,
RELATIONSHIP VARCHAR,
UPDATEDDATETIME TIMESTAMP,
UPDATEDBY VARCHAR,
SEQUENCE_NO VARCHAR,
TUPLE_COUNT VARCHAR,
CONSTRAINT PK_MapLonglistCache_MAP_LONGLIST PRIMARY KEY
(ENTITY_ID,RELATIONSHIP,SEQUENCE_NO)
) ;
CREATE INDEX MAP_LONGLIST_PK ON MapLonglistCache.MAP_LONGLIST (RELATIONSHIP
DESC,SEQUENCE_NO DESC) ;

Here is query which is failing

DELETE FROM "MapLonglistCache".MAP_LONGLIST  WHERE ENTITY_ID IN
('12345','12345') 
AND RELATIONSHIP  IN ('OutageID_ServiceIDs','OutageID_ServiceIDs') 
AND SEQUENCE_NO IN ('6','7')

And, when I pass only one key, it seems to be working like 

DELETE FROM "MapLonglistCache".MAP_LONGLIST  WHERE ENTITY_ID IN ('12345') 
AND RELATIONSHIP  IN ('OutageID_ServiceIDs') 
AND SEQUENCE_NO IN ('6')

So is it, that delete will not work with IN clause which has a composite
key. 

And I do not have POJO classes on the classpath, not sure how it is trying
load the key class "'com.jio.digitalapi.edif.customer.model.MapLonglistKey"


SEVERE: Failed to execute SQL query [reqId=0, req=JdbcQueryExecuteRequest
[schemaName=PUBLIC, pageSize=1024, maxRows=0, sqlQry=DELETE FROM
"MapLonglistCache".MAP_LONGLIST  WHERE ENTITY_ID IN ('12345','12345')
AND RELATIONSHIP  IN ('OutageID_ServiceIDs','OutageID_ServiceIDs')
AND SEQUENCE_NO IN ('6','7'), args=[], stmtType=ANY_STATEMENT_TYPE]]
class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed
to process key 'com.jio.digitalapi.edif.customer.model.MapLonglistKey
[idHash=15804671, hash=156851519, entityId=12345,
relationship=OutageID_ServiceIDs, sequenceNo=7]'
at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.doDelete(DmlStatementsProcessor.java:681)
at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.processDmlSelectResult(DmlStatementsProcessor.java:582)
at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:534)
at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:160)
at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsDistributed(DmlStatementsProcessor.java:334)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunPrepared(IgniteH2Indexing.java:1613)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1577)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2039)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2034)
at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2555)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2048)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:1979)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:310)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:169)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:148)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:41)
at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Failed to process key
'com.jio.digitalapi.edif.customer.model.MapLonglistKey [idHash=15804671,
hash=156851519, entityId=12345, 

Re: Slow invoke call

[Pavel Vinokurov]

Sam,

>>
Why does invoking remote method is taking 50% of time? Is it because of
concurrency and entry processor will execute under a lock internally?
>>
Could you please share a small reproducer project.

>> Is there any better way to deal with this usecase?
Have you tested with only heap memory?

Thanks,
Pavel




2018-04-06 2:39 GMT+07:00 javastuff@gmail.com :

> Thanks Pavel for reply.
>
> /"When you store entries into the off-heap, any update operation requires
> copying value from the off-heap to the heap." /
> I am updating using remote entry processor, does that also need to copy
> value to calling heap? If yes then no benefit using entry processor here, I
> can fetch and put, probably with a lock.
>
> Why does invoking remote method is taking 50% of time? Is it because of
> concurrency and entry processor will execute under a lock internally?
>
> Coping existing and incoming bytes, must be generating a lot for GC.
>
> Is there any better way to deal with this usecase? right now it seems
> slower
> than DB updates.
>
> Thanks,
> -Sam
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 

Regards

Pavel Vinokurov

Re: Ignite cache get operation not working, client application stucks at the initializaiton

[Evgenii Zhuravlev]

In object com.interactcrm.qm.queuegroup.QueueGroup you have the field of
type com.interactcrm.qm.dao.ContactDAO and you need to make sure that this
field won't be serialized.

Evgenii

2018-04-10 12:32 GMT+03:00 Priyanka Shinde :

> Hi,
>
> I have provided the absolute path of the log4j.xml and logs were generated
> in the application logs itself.Attaching the same.
>
> C‪lient_logs.txt
>  t1594/C%E2%80%AAlient_logs.txt>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite cache get operation not working, client application stucks at the initializaiton

[Priyanka Shinde]

Hi,

I have provided the absolute path of the log4j.xml and logs were generated
in the application logs itself.Attaching the same.

C‪lient_logs.txt

  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cache get operation not working, client application stucks at the initializaiton

[Priyanka Shinde]

Hi Evengii,

I have provided the Absolute path for log4j.xml and logs are generated.A
C‪lient_logs.txt

 
ttaching the same



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Strange node fail

[dkarachentsev]

Duplicates
http://apache-ignite-users.70518.x6.nabble.com/Strange-node-fail-td21078.html.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Strange node fail

[dkarachentsev]

Hi Ray,

If your JVM process consumes more memory, then started swapping may cause
JVM freeze, and as a consequence, throwing it out from the cluster. Check
your free memory, disable swapping, if possible, or increase
IgniteConfiguration.failureDetectionTimeout.

To check that guess you may use dstat and add
-XX:+PrintGCApplicationStoppedTime property, it will log additional process
stop times.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Regarding Ignite Slack Channel

[Roman Guseinov]

Hi Rick,

If I am not wrong, there is no Apache Ignite channel in Slack. Besides
User-list, you can use Gitter [1] or Stackoverflow [2] to contact the
community as well.

Best Regards,
Roman

[1] https://gitter.im/apacheignite/ignite
[2] https://stackoverflow.com/questions/tagged/ignite



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/