Re: Binary recovery for a very long time

[38797715]

Hi,

the following log message:

[2020-05-12T18:17:57,071][INFO ][main][GridCacheProcessor] Started cache 
in recovery mode [name=CO_CO_LINE_NEW, id=1742991829, 
dataRegionName=default, mode=PARTITIONED, atomicity=ATOMIC, backups=1, 
mvcc=false]


I have the following questions:

1.What has been done in the startup cache in recovery mode?

2.After testing, if the node stops normally (non abnormal shutdown), the 
recovery process will also be performed during startup. Why?


在 2020/5/18 下午9:58, Ilya Kasnacheev 写道:

Hello!

Direct IO module is experimental and should not be used unless 
performance is tested first, in your specific use case.


Regards,
--
Ilya Kasnacheev


пн, 18 мая 2020 г. в 16:47, 38797715 <38797...@qq.com 
>:


Hi,

If direct IO is disabled, the startup speed will be doubled,
including some other tests. I find that direct IO has a great
impact on the read performance.

在 2020/5/14 上午5:16, Evgenii Zhuravlev 写道:

Can you share full logs from all nodes?

вт, 12 мая 2020 г. в 18:24, 38797715 <38797...@qq.com
>:

Hi Evgenii,

The storage used is not SSD.

We will use different versions of ignite for further testing,
such as ignite2.8.
Ignite is configured as follows:


http://www.springframework.org/schema/beans";

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";

xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd";>














































在 2020/5/13 上午4:45, Evgenii Zhuravlev 写道:

Hi,

Can you share full logs and configuration? What disk so you use?

Evgenii

вт, 12 мая 2020 г. в 06:49, 38797715 <38797...@qq.com
>:

Among them:
CO_CO_NEW: ~ 48 minutes(partitioned,backup=1,33M)

Ignite sys cache: ~ 27 minutes

PLM_ITEM:~3 minutes(repicated,1.9K)


在 2020/5/12 下午9:08, 38797715 写道:


Hi community,

We have 5 servers, 16 cores, 256g memory, and 200g
off-heap memory.
We have 7 tables to test, and the data volume is

respectively:31.8M,495.2M,552.3M,33M,873.3K,28M,1.9K(replicated),others
are partitioned(backup = 1)

VM args:-server -Xms20g -Xmx20g -XX:+AlwaysPreTouch
-XX:+UseG1GC -XX:+ScavengeBeforeFullGC
-XX:+DisableExplicitGC -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps
-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=100M -Xloggc:/data/gc/logs/gclog.txt
-Djava.net.preferIPv4Stack=true
-XX:MaxDirectMemorySize=256M -XX:+PrintAdaptiveSizePolicy

Today, one of the servers was restarted(kill and then
start ignite.sh) for some reason, but the node took 1.5
hours to start, which was much longer than expected.

After analyzing the log, the following information is
found:


[2020-05-12T17:00:05,138][INFO][main][GridCacheDatabaseSharedManager]
Found last checkpoint marker
[cpId=7a0564f2-43e5-400b-9439-746fc68a6ccb,
pos=FileWALPointer [idx=10511, fileOff=5134,
len=61193]]

[2020-05-12T17:00:05,151][INFO][main][GridCacheDatabaseSharedManager]
Binary memory state restored at node startup
[restoredPtr=FileWALPointer [idx=10511,
fileOff=51410110, len=0]]
[2020-05-12T17:00:05,152][INFO][main][FileWriteAheadLogManager]
Resuming logging to WAL segment
[file=/appdata/ignite/db/wal/24/0001.wal,
offset=51410110, ver=2]
[2020-05-12T17:00:06,448][INFO][main][PageMemoryImpl]
Started page memory [memoryAllocated=200.0GiB,
pages=50821088, tableSize=3.9GiB, checkpointBuffer=2.0GiB]
[2020-05-12T17:02:08,528][INFO][main][GridCacheProcessor]
Started cache in recovery mode [name=CO_CO_NEW,
id=-189779360, dataRegionName=default,
mode=PARTITIONED, atomicity=ATOMIC, backups=1, mvcc=false]
[2020-05-12T17:50:44,341][INFO][main][GridCacheProcessor]
Started cache in recovery mode [name=CO_CO_LINE,
id=-1588248812, dataRegionName=default,
mode=PARTITIONED, at

Ordered initial query for continuous query in Ignite .Net

[Barney Pippin]

Hi,

I'm just upgrading to Ignite 2.8 and see that SqlQuery has been deprecated
directing us to SqlFieldsQuery, ScanQuery or Text. Previously I had a
continuous query running that took an SqlQuery which had "order by xyz" as
part of the initial query so I could get ordered results back upfront. 

Can anyone advise how I can achieve the same in 2.8 given that
SqlFieldsQuery isn't derived from QueryBase and I've yet to spot an ordering
option for text or scan queries.

Hopefully I've just missed something in the docs!

Many thanks,

James 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: join question

[Ilya Kasnacheev]

Hello!

It is possible that Data Streamer is not actually aware of affinity column
of your table. Can you try marking it with @AffinityKeyMapped?

Regards,
-- 
Ilya Kasnacheev


вт, 19 мая 2020 г. в 01:48, narges saleh :

> It seems the issue exist only if one uses data streamer with binaryobject
> builder. If I use straight JDBC to insert data, the issue goes away. Any
> idea what one needs to do to get this working with binary objects?
> Everything else is the same between the two scenarios.
>
> On Mon, May 18, 2020 at 4:39 PM narges saleh  wrote:
>
>> It turned out that I'd get partial results in some cases, when joining
>> partitioned caches. But I still don't understand why I am not getting all
>> the rows that the joined query should return.
>> My assumption is that if you have caches with primary keys, containing
>> the affinity key, then the related entries  (by affinity key) in these
>> caches should be collocated and a join among these caches based on the
>> leading part of the primary keys (including the affinity key) which is
>> shared across all the keys, should return all the rows which satisfy the
>> where clause. Even if this is not the case, a distributed join should be
>> possible and I still should get all the rows. But this is not happening
>> either.
>> What could be the issue here? What am I missing here?
>>
>> On Mon, May 18, 2020 at 9:30 AM narges saleh 
>> wrote:
>>
>>> No error. Just no records is returned, as opposed to the join between
>>> the replicated and partitioned cache which returns ass applicable rows.
>>> Sorry, for not being clear.
>>>
>>> On Mon, May 18, 2020 at 9:00 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Fails how? Is the result set incorrect? Any specific error message?
 Please share details.

 Regards,
 --
 Ilya Kasnacheev


 пн, 18 мая 2020 г. в 16:49, narges saleh :

> Hi All,
> I have encountered a puzzling join case.
> I have 3 tables on a cluster of two ignite server nodes:
> table-A (id + org = primary), replicated
> id
> org. <-- affinity
> other fields
>
> table-B (id, org, add-id=primary key), partitioned
> id
> org <- affinity
> addr-id
> other fields
>
> table-C (id, org, comp-id=primary key), partitioned
> id
> org <- affinity
> comp-id
> other fields
>
> joins between table-A and table-B (on id, and org) succeeds.
> joins between table-A and table-C (of id and org) succeeds.
> joins between table-B and table-C (on id and org) fails.
>
> all three joins succeed if the cluster has only one server node.
> Why the join between the partitioned caches fail in a distributed mode?
>
> I am using JDBC connection for select statements. The join fails
> whether dealing with thick or thin client.
>
> thanks
>
>

Re: Near Cache Support For Thin Clients

[Pavel Tupitsyn]

Can you please describe the use case in more detail?
What do you expect from such a feature?

On Tue, May 19, 2020 at 2:01 AM martybjo...@gmail.com 
wrote:

> I wanted to see if there are any plans to support near caches for thin
> clients? I think it would be a great feature. I know I could use it right
> now.
> --
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Igor Sapego]

Which thin client are you referring to?

Best Regards,
Igor


On Mon, May 18, 2020 at 5:09 PM scriptnull 
wrote:

> I would like to break down this question into two questions.
>
> 1. Can we have key-values with different expire times in the same cache? (I
> think the answer for this is yes, because the redis layer in ignite allows
> for this)
>
> 2. I am trying to build a ruby thin client for Apache Ignite and got a
> basic
> prototype in place. But I couldn't find an operation in binary protocol
> documentation that will enable us to set TTL for a key-value pair.  So, any
> idea on how to set TTL via the binary protocol?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Data loading during startup

[38797715]

Hi community,

I know that during the startup process of ignite node, cached data will 
not be loaded, that is, there is no warm up process.


However, we can see from the top command that during the startup of 
ignite, the memory usage has been increasing, which will increase by 
more than 10g, what data has been loaded in the start process?

Ignite Node Metrics using JMX

[kay]

Hello, I have 4 remote node for server mode.

I'd like to monitoring that nodes using JMX.
I want to get CPU, THREAD, HEAP, OFFHEAP, GC and so on..

Can I get thoes things( CPU, THREAD, Heap, offheap, gc) each node??

I figured out there are MXBean(CacheMetrics, CacheGroup, DataRegion,
DataStorage)
but I don't know what attribute should I going to use like
(PhysicalMemorySize, TotalAllocatedSize)

Please help me..
Thank you  




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ordered initial query for continuous query in Ignite .Net

[Pavel Tupitsyn]

Hi James,

This is a very good catch!
I've filed a ticket, we'll fix it in the next version:
https://issues.apache.org/jira/browse/IGNITE-13031

For now please continue using SqlQuery, it is functional as before.

Thank you,
Pavel

On Tue, May 19, 2020 at 10:42 AM Barney Pippin <
james.pri...@uk.bnpparibas.com> wrote:

> Hi,
>
> I'm just upgrading to Ignite 2.8 and see that SqlQuery has been deprecated
> directing us to SqlFieldsQuery, ScanQuery or Text. Previously I had a
> continuous query running that took an SqlQuery which had "order by xyz" as
> part of the initial query so I could get ordered results back upfront.
>
> Can anyone advise how I can achieve the same in 2.8 given that
> SqlFieldsQuery isn't derived from QueryBase and I've yet to spot an
> ordering
> option for text or scan queries.
>
> Hopefully I've just missed something in the docs!
>
> Many thanks,
>
> James
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Exporting SQL DB

[NSquare]

Hi,

I have a 3 node ignite cluster which stores data in SQL tables. All the
tables and table contents can be seen with sqline.sh tool.

How can I export these tables as an SQL DB file for future reference?

Cheers,
NSquare

Re: Scheduling Cache Refresh

[Ilya Kasnacheev]

Hello!

See below inline:

пт, 15 мая 2020 г. в 23:59, nithin91 <
nithinbharadwaj.govindar...@franklintempleton.com>:

> Hi
>
> Although this seems to be working.
> I have following queries with this approach.Can you please
> provide your inputs/suggestions.
>
>
>
> We are going to expose this SpringBoot
> App as a RestFul API where we are trying to
> refresh the Ignite Cache by sending  request to
> this
> Application whenever the data in Oracle changes.
> .
> As we are trying to expose this APP as a API,
> which means that this  Ignite Client Node
> instance launched from Spring boot Application
> will
> be active for ever.Now my queries are
>
> 1. Is it ok to have a client node which
> is continously in connected state forever.
> If Ok,then how many such client node can exist.
>
Yes, it's OK, you can have dozens of clients connected at all times.

2. Can this Application, be hosted on the
> one of the server node with different port.
>
Yes. No additional configuration is necessary.


> 3. Also will client node connect automatically
> to the cluster if the client node has  lost
> communication with server because if the client
> node is down and if there is an API request
> , then the application will throw an error.
>
Client will try to reconnect automatically. It will throw ClientNode
IgniteClientDisconnectedException in the meantime.


> 4. Is this way of refreshing the cache is
> efficient.
> If no, can you please suggest a better way.
>
What do you mean by refreshing cache?


> 5. Also take provide an example on the possibility
> of  having the REST API calling a
> compute job which would
> execute in a separate server
> https://apacheignite.readme.io/docs/compute-grid
> Also i have few more queries related to Ignite
> Behaviour.It would be really helpful if you can
> provide some inputs.
>
> 1. Currently we are using
> ignite.cache("CacheName").loadCache(null) method
> to load  the cache initially and also increamentally as
> this method loads data very fast almost 0.1 million
> in 1 minute. For increamental load also  we are using this
> method
> by passing an additional argument i.e. Custom
> Sql  to load/update only the records that have
> created/changed.
> but what is happening is if the key doesn't
> exist it is inserting the data  but if the key is present
> it is not updating its value.
> Why is the value not updated if the key is present.
>
This is by design.


>
> 2. To overcome this, i have to build my own custom
> cache store as mentioned in the following link
> https://apacheignite.readme.io/v1.9/docs/data-loading.
> But the problem with this approach is  i need to use JDBC
> result set to load the data which is very slow as
> we have to insert row by row.
> Is there a better way to update the keys.
>
Consider using DataStreamer.


>
>
> 3. How to load data faster with custom cache store
> as loading the data which is very slow as
> we have to insert row by row.
>
See above.

Regards,
-- 
Ilya.


>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Near Cache Support For Thin Clients

[Marty Jones]

The use case is having a local cache that stores most widely used cache
items in memory on server instead of having the network expense of pulling
them down every time they are requested.  The main thing is the near cache
has to support removing cache items that have expired on the server.

The best use case I have is a web application that needs a cache item per
request.  we would not want to pull the cache item from the cluster every
request.   It would be way more efficient for the thin client to have a
near cache that would hold "hot" cache items that are requested frequently.

On Tue, May 19, 2020 at 3:43 AM Pavel Tupitsyn  wrote:

> Can you please describe the use case in more detail?
> What do you expect from such a feature?
>
> On Tue, May 19, 2020 at 2:01 AM martybjo...@gmail.com <
> martybjo...@gmail.com> wrote:
>
>> I wanted to see if there are any plans to support near caches for thin
>> clients? I think it would be a great feature. I know I could use it right
>> now.
>> --
>> Sent from the Apache Ignite Users mailing list archive
>>  at Nabble.com.
>>
>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[scriptnull]

Hi,

I am referring to any of the thin clients (java/.net/c++/node.js/python/php)
documented at https://apacheignite.readme.io/docs/thin-clients

I wonder if any of those thin clients have an API to set TTL for a key-value
in a cache. If so I can reverse engineer the implementation of it and
implement it in the ruby thin client implementation that I am currently
working on.

More specifically, I am interested in knowing if there is an operation in
binary protocol like OP_CACHE_UPDATE_TTL

If there is such an operation, I could call it via a ruby thin client
implementation that I am working on currently.

Regards,
Vishnu Bharathi P



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Pavel Tupitsyn]

Hi,

Yes, every cache request has Flags byte right after cacheId.
There is FLAG_WITH_EXPIRY_POLICY = 0x04.
When this flag is set, server expects 3 long values (3x8 bytes) after flags
byte,
representing TTL for Create, Update, and Access, in milliseconds

For example, if we want to store a cache entry that should expire in 3
seconds,
we execute OP_CACHE_PUT (1001) operation, set FLAG_WITH_EXPIRY_POLICY,
and pass 3000, 0, 0 as TTL values.

Let me know if you need more details.

Thanks,
Pavel

On Tue, May 19, 2020 at 3:27 PM scriptnull 
wrote:

> Hi,
>
> I am referring to any of the thin clients
> (java/.net/c++/node.js/python/php)
> documented at https://apacheignite.readme.io/docs/thin-clients
>
> I wonder if any of those thin clients have an API to set TTL for a
> key-value
> in a cache. If so I can reverse engineer the implementation of it and
> implement it in the ruby thin client implementation that I am currently
> working on.
>
> More specifically, I am interested in knowing if there is an operation in
> binary protocol like OP_CACHE_UPDATE_TTL
>
> If there is such an operation, I could call it via a ruby thin client
> implementation that I am working on currently.
>
> Regards,
> Vishnu Bharathi P
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Near Cache Support For Thin Clients

[Pavel Tupitsyn]

Ok, thanks for the explanation.
Yes, this is a good feature, and I've had this in mind for some time.

Ticket filed: https://issues.apache.org/jira/browse/IGNITE-13037
There are no immediate plans, but I think there is a possibility to achieve
this by the end of the year.

On Tue, May 19, 2020 at 2:52 PM Marty Jones  wrote:

> The use case is having a local cache that stores most widely used cache
> items in memory on server instead of having the network expense of pulling
> them down every time they are requested.  The main thing is the near cache
> has to support removing cache items that have expired on the server.
>
> The best use case I have is a web application that needs a cache item per
> request.  we would not want to pull the cache item from the cluster every
> request.   It would be way more efficient for the thin client to have a
> near cache that would hold "hot" cache items that are requested frequently.
>
> On Tue, May 19, 2020 at 3:43 AM Pavel Tupitsyn 
> wrote:
>
>> Can you please describe the use case in more detail?
>> What do you expect from such a feature?
>>
>> On Tue, May 19, 2020 at 2:01 AM martybjo...@gmail.com <
>> martybjo...@gmail.com> wrote:
>>
>>> I wanted to see if there are any plans to support near caches for thin
>>> clients? I think it would be a great feature. I know I could use it right
>>> now.
>>> --
>>> Sent from the Apache Ignite Users mailing list archive
>>>  at Nabble.com.
>>>
>>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[scriptnull]

Awesome, that's exactly the information that I needed. So we will have to set
the flag byte while doing OP_CACHE_PUT.

Do you know by any chance if there is support for setting expire times for
multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004) operation?
I am guessing the answer is no.

Regards,
Vishnu Bharathi P



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Pavel Tupitsyn]

> if there is support for setting expire times for multiple key-values
> while doing a OP_CACHE_PUT_ALL

The answer is yes. All key-val operations support expiration, namely:
OP_CACHE_PUT
OP_CACHE_PUT_IF_ABSENT
OP_CACHE_PUT_ALL
OP_CACHE_GET_AND_PUT
OP_CACHE_GET_AND_REPLACE
OP_CACHE_GET_AND_PUT_IF_ABSENT
OP_CACHE_REPLACE
OP_CACHE_REPLACE_IF_EQUALS



On Tue, May 19, 2020 at 4:11 PM scriptnull 
wrote:

> Awesome, that's exactly the information that I needed. So we will have to
> set
> the flag byte while doing OP_CACHE_PUT.
>
> Do you know by any chance if there is support for setting expire times for
> multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004)
> operation?
> I am guessing the answer is no.
>
> Regards,
> Vishnu Bharathi P
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Marty Jones]

I am not seeing that option in the ASP.NET thin client.   This is how I
accomplished setting an expiration policy:

ICacheClient cache = igniteClient.GetCache("testCache");

cache.WithExpiryPolicy(new ExpiryPolicy(TimeSpan.FromSeconds(15), null,
null))
 .Put("test1", new CacheItemEntry() { Entry = "test" });

On Tue, May 19, 2020 at 8:17 AM Pavel Tupitsyn  wrote:

> > if there is support for setting expire times for multiple key-values
> > while doing a OP_CACHE_PUT_ALL
>
> The answer is yes. All key-val operations support expiration, namely:
> OP_CACHE_PUT
> OP_CACHE_PUT_IF_ABSENT
> OP_CACHE_PUT_ALL
> OP_CACHE_GET_AND_PUT
> OP_CACHE_GET_AND_REPLACE
> OP_CACHE_GET_AND_PUT_IF_ABSENT
> OP_CACHE_REPLACE
> OP_CACHE_REPLACE_IF_EQUALS
>
>
>
> On Tue, May 19, 2020 at 4:11 PM scriptnull 
> wrote:
>
>> Awesome, that's exactly the information that I needed. So we will have to
>> set
>> the flag byte while doing OP_CACHE_PUT.
>>
>> Do you know by any chance if there is support for setting expire times for
>> multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004)
>> operation?
>> I am guessing the answer is no.
>>
>> Regards,
>> Vishnu Bharathi P
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Pavel Tupitsyn]

Marty, can you please elaborate?
We are talking about protocol-level details in this thread.
Your code for .NET Thin Client seems to be fine, does it work as expected?

On Tue, May 19, 2020 at 4:39 PM Marty Jones  wrote:

> I am not seeing that option in the ASP.NET thin client.   This is how I
> accomplished setting an expiration policy:
>
> ICacheClient cache = igniteClient.GetCache CacheItemEntry>("testCache");
>
> cache.WithExpiryPolicy(new ExpiryPolicy(TimeSpan.FromSeconds(15), null,
> null))
>  .Put("test1", new CacheItemEntry() { Entry = "test" });
>
> On Tue, May 19, 2020 at 8:17 AM Pavel Tupitsyn 
> wrote:
>
>> > if there is support for setting expire times for multiple key-values
>> > while doing a OP_CACHE_PUT_ALL
>>
>> The answer is yes. All key-val operations support expiration, namely:
>> OP_CACHE_PUT
>> OP_CACHE_PUT_IF_ABSENT
>> OP_CACHE_PUT_ALL
>> OP_CACHE_GET_AND_PUT
>> OP_CACHE_GET_AND_REPLACE
>> OP_CACHE_GET_AND_PUT_IF_ABSENT
>> OP_CACHE_REPLACE
>> OP_CACHE_REPLACE_IF_EQUALS
>>
>>
>>
>> On Tue, May 19, 2020 at 4:11 PM scriptnull 
>> wrote:
>>
>>> Awesome, that's exactly the information that I needed. So we will have
>>> to set
>>> the flag byte while doing OP_CACHE_PUT.
>>>
>>> Do you know by any chance if there is support for setting expire times
>>> for
>>> multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004)
>>> operation?
>>> I am guessing the answer is no.
>>>
>>> Regards,
>>> Vishnu Bharathi P
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Marty Jones]

Pavel,

My bad, I misunderstood the question.  The code I am using works fine.  I
thought the previous discussion was stating that you could specify the
expiration policy on the actual "Put" call itself.

On Tue, May 19, 2020 at 8:53 AM Pavel Tupitsyn  wrote:

> Marty, can you please elaborate?
> We are talking about protocol-level details in this thread.
> Your code for .NET Thin Client seems to be fine, does it work as expected?
>
> On Tue, May 19, 2020 at 4:39 PM Marty Jones  wrote:
>
>> I am not seeing that option in the ASP.NET thin client.   This is how I
>> accomplished setting an expiration policy:
>>
>> ICacheClient cache =
>> igniteClient.GetCache("testCache");
>>
>> cache.WithExpiryPolicy(new ExpiryPolicy(TimeSpan.FromSeconds(15), null,
>> null))
>>  .Put("test1", new CacheItemEntry() { Entry = "test" });
>>
>> On Tue, May 19, 2020 at 8:17 AM Pavel Tupitsyn 
>> wrote:
>>
>>> > if there is support for setting expire times for multiple key-values
>>> > while doing a OP_CACHE_PUT_ALL
>>>
>>> The answer is yes. All key-val operations support expiration, namely:
>>> OP_CACHE_PUT
>>> OP_CACHE_PUT_IF_ABSENT
>>> OP_CACHE_PUT_ALL
>>> OP_CACHE_GET_AND_PUT
>>> OP_CACHE_GET_AND_REPLACE
>>> OP_CACHE_GET_AND_PUT_IF_ABSENT
>>> OP_CACHE_REPLACE
>>> OP_CACHE_REPLACE_IF_EQUALS
>>>
>>>
>>>
>>> On Tue, May 19, 2020 at 4:11 PM scriptnull 
>>> wrote:
>>>
 Awesome, that's exactly the information that I needed. So we will have
 to set
 the flag byte while doing OP_CACHE_PUT.

 Do you know by any chance if there is support for setting expire times
 for
 multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004)
 operation?
 I am guessing the answer is no.

 Regards,
 Vishnu Bharathi P



 --
 Sent from: http://apache-ignite-users.70518.x6.nabble.com/

>>>

Re: Can we set TTL (expiry time) of a key-value from thin client?

[Pavel Tupitsyn]

> you could specify the expiration policy on the actual "Put" call itself
That is true on the protocol level.
User-facing API has WithExpiryPolicy so we don't have to add an additional
overload for every cache operation.

On Tue, May 19, 2020 at 5:00 PM Marty Jones  wrote:

> Pavel,
>
> My bad, I misunderstood the question.  The code I am using works fine.  I
> thought the previous discussion was stating that you could specify the
> expiration policy on the actual "Put" call itself.
>
> On Tue, May 19, 2020 at 8:53 AM Pavel Tupitsyn 
> wrote:
>
>> Marty, can you please elaborate?
>> We are talking about protocol-level details in this thread.
>> Your code for .NET Thin Client seems to be fine, does it work as expected?
>>
>> On Tue, May 19, 2020 at 4:39 PM Marty Jones 
>> wrote:
>>
>>> I am not seeing that option in the ASP.NET thin client.   This is how I
>>> accomplished setting an expiration policy:
>>>
>>> ICacheClient cache =
>>> igniteClient.GetCache("testCache");
>>>
>>> cache.WithExpiryPolicy(new ExpiryPolicy(TimeSpan.FromSeconds(15), null,
>>> null))
>>>  .Put("test1", new CacheItemEntry() { Entry = "test" });
>>>
>>> On Tue, May 19, 2020 at 8:17 AM Pavel Tupitsyn 
>>> wrote:
>>>
 > if there is support for setting expire times for multiple key-values
 > while doing a OP_CACHE_PUT_ALL

 The answer is yes. All key-val operations support expiration, namely:
 OP_CACHE_PUT
 OP_CACHE_PUT_IF_ABSENT
 OP_CACHE_PUT_ALL
 OP_CACHE_GET_AND_PUT
 OP_CACHE_GET_AND_REPLACE
 OP_CACHE_GET_AND_PUT_IF_ABSENT
 OP_CACHE_REPLACE
 OP_CACHE_REPLACE_IF_EQUALS



 On Tue, May 19, 2020 at 4:11 PM scriptnull 
 wrote:

> Awesome, that's exactly the information that I needed. So we will have
> to set
> the flag byte while doing OP_CACHE_PUT.
>
> Do you know by any chance if there is support for setting expire times
> for
> multiple key-values while doing a OP_CACHE_PUT_ALL (opcode: 1004)
> operation?
> I am guessing the answer is no.
>
> Regards,
> Vishnu Bharathi P
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Scheduling Cache Refresh

[nithin91]

Hi 

Thanks For the inputs.

To the question is this way of refreshing the cache is efficient, i mean 
refreshing the cache using the following process.

We have created a REST-API using Spring Boot which refreshes a particular
cache when a GET Request is triggered.

*Sample REST-API Url :*

http://domainname.com/api/v1/refresh?cachename="MyCache";.

To refresh the cache, in the Spring Boot Application, we have started a
client node which stays connected to the cluster as long as the Application
forever.

Also can you please let me know whether DataSteamer API provided by Ignite
can be accessed through any of the thin clients like JAVA,Python,JS.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Deploying Ignite Code

[nithin91]

Hi

Thanks for the inputs.

I have few queries.

For example i have few caches which have custom key a JAVA POJO Class and
custom Value which is also a JAVA POJO Class.Currently i am unable to do
Cache.invoke and Cache.invokeall operations with peer class loading(i.e
facing class Not Found Exception),so decided to deploy the jar files in each
of the server nodes.

In this case do  i need to create a JAR file for the POJO Classes alone or
for the entire eclipse project and should  i include the maven dependencies
also in the JAR File(i.e is Uber Jar needed).






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Can you create a cache using Ignite Visor CLI?

[Andrew Munn]

It looks like the Visor CLI can only operate on existing caches, not create
new ones.  Is that correct?

Re: Ignite Node Metrics using JMX

[Evgenii Zhuravlev]

Hi,

Some node information, like CPU & heap memory, can be found in
ClusterLocalNodeMetricsMXBeanImpl.

As per memory metrics - they are described here:
https://apacheignite.readme.io/docs/memory-metrics#getting-metrics

Evgenii

вт, 19 мая 2020 г. в 01:52, kay :

> Hello, I have 4 remote node for server mode.
>
> I'd like to monitoring that nodes using JMX.
> I want to get CPU, THREAD, HEAP, OFFHEAP, GC and so on..
>
> Can I get thoes things( CPU, THREAD, Heap, offheap, gc) each node??
>
> I figured out there are MXBean(CacheMetrics, CacheGroup, DataRegion,
> DataStorage)
> but I don't know what attribute should I going to use like
> (PhysicalMemorySize, TotalAllocatedSize)
>
> Please help me..
> Thank you
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Two specific warnings everytime

[adipro]

I always get these two warnings. Persistence is on.

Warning 1:

```
[WARNING][checkpoint-runner-#569][GridCacheDatabaseSharedManager] 3
checkpoint pages were not written yet due to unsuccessful page write lock
acquisition and will be retried
[WARNING][checkpoint-runner-#569][GridCacheDatabaseSharedManager] 2
checkpoint pages were not written yet due to unsuccessful page write lock
acquisition and will be retried
```

Warning 2:

```
[WARNING][sys-stripe-193-#194][GridContinuousProcessor] Failed to wait for
ack message. [node=a1cdcd89-51a7-42c9-b999-c3adfef26600,
routine=8ee5510d-e8ca-47e8-9960-4688b2007ebb]
```


Can someone please explain why these are popping and are we doing something
wrong? Or are these harmless?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can you create a cache using Ignite Visor CLI?

[akorensh]

yes. That is correct:
https://apacheignite-tools.readme.io/docs/command-line-interface

use the "help cache" command inside the visor to see all of the
capabilities.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data loading during startup

[akorensh]

Hi,
  If you've enabled persistence then some data will be loaded.
  Use metrics to see what/how much was loaded:

see:  https://apacheignite.readme.io/docs/distributed-persistent-store
https://apacheignite.readme.io/docs/durable-memory#persistence-features
https://apacheignite.readme.io/docs/memory-metrics#data-region-metrics
example:
https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/persistentstore

  If there is a complex config/many nodes then a lot of messages will be
exchanged which leads to memory usage:

Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Exporting SQL DB

[akorensh]

Hi,
  You can record data output from sqlline
  0: jdbc:ignite:thin://127.0.0.1/> !outputformat csv
   jdbc:ignite:thin://127.0.0.1/> !record data.csv
  Saving all output to "data.csv". Enter "record" with no arguments to stop
it.
  0: jdbc:ignite:thin://127.0.0.1/> select * from Person

  here the output from the "select" will go into data.csv

  You can then clean up the csv and use it as need be.

  see:  https://apacheignite-sql.readme.io/docs/sqlline
Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Two specific warnings everytime

[akorensh]

Hi,
  Like it says, these are warnings.

   Warning 1: This means that a lock couldn't be acquired when starting a
check-pointing operation.
   This operation will be re-tried. see:
https://apacheignite.readme.io/docs/persistence-checkpointing


  Warning 2: This means that the continuous query you are subscribing to
sent an update to the listener
   but did not get an acknowledgement within 100ms. Check
that the network is working correctly
   and that all your continuous query updates have reached
their destinations.
Thanks, Alex 





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Cache Item Invalidation

[Marty Jones]

How do you guys handle invalidating cache items when the underlying data is
changed within say a database you are using to create cache items with?

Re: Suggest a better way to access a particular K-V store

[adipro]

My query is 

SELECT * FROM URLS WHERE APPNAME_ID = ? ORDER BY SCORE LIMIT ?

That is giving me 0.7-1.0 msec read performance for a test run for about
some time. But the index with URLS (SCORE ASC, APPNAME_ID), I'm getting a
read performance of about 0.3 to 0.5 msec. i found it to be constant when
data grows.. whereas the index which you provided is getting delayed with
growing data. Ours is write intensive application.

But the thing is in deployment servers we are constantly receiving these
warnings

```
[22:26:31,046][WARNING][query-#48349][IgniteH2Indexing] Long running query
is finished [duration=3006ms, type=MAP, distributedJoin=false,
enforceJoinOrder=false, lazy=false, schema=PUBLIC, sql='SELECT
"__Z0"."ID" "__C0_0",
"__Z0"."URL" "__C0_1",
"__Z0"."SCORE" "__C0_2",
"__Z0"."APPNAME_ID" "__C0_3"
FROM "PUBLIC"."URLS" "__Z0"
WHERE "__Z0"."APPNAME_ID" = ?1
ORDER BY 3 FETCH FIRST ?2 ROWS ONLY', plan=SELECT
__Z0.ID AS __C0_0,
__Z0.URL AS __C0_1,
__Z0.SCORE AS __C0_2,
__Z0.APPNAME_ID AS __C0_3
FROM PUBLIC.URLS __Z0
/* PUBLIC.IDX_2_URLS */
/* scanCount: 1106428 */
WHERE __Z0.APPNAME_ID = ?1
ORDER BY 3
FETCH FIRST ?2 ROWS ONLY
/* index sorted */, node=TcpDiscoveryNode
[id=9a62f525-98a8-43d6-85da-2270f1ee4e7a,
consistentId=9a62f525-98a8-43d6-85da-2270f1ee4e7a, addrs=ArrayList
[127.0.0.1, 172.20.42.17], sockAddrs=HashSet [/172.20.42.17:0,
/127.0.0.1:0], discPort=0, order=3, intOrder=3,
lastExchangeTime=1589893316075, loc=false,
ver=8.7.10#20191227-sha1:c481441d, isClient=true], reqId=262124, segment=0]
```


data metric at the point of warning ->

```
[22:34:05,866][INFO][grid-timeout-worker-#407][IgniteKernal] 
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=4f2b808c, uptime=04:19:01.321]
^-- H/N/C [hosts=4, nodes=4, CPUs=192]
^-- CPU [cur=0.03%, avg=13.79%, GC=0%]
^-- PageMemory [pages=927396]
^-- Heap [used=1521MB, free=62.86%, comm=4096MB]
^-- Off-heap [used=3665MB, free=56.84%, comm=8392MB]
^--   sysMemPlc region [used=0MB, free=99.98%, comm=100MB]
^--   default region [used=3665MB, free=55.26%, comm=8192MB]
^--   metastoreMemPlc region [used=0MB, free=99.94%, comm=0MB]
^--   TxLog region [used=0MB, free=100%, comm=100MB]
^-- Ignite persistence [used=3622MB]
^--   sysMemPlc region [used=0MB]
^--   default region [used=3622MB]
^--   metastoreMemPlc region [used=0MB]
^--   TxLog region [used=0MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=0, idle=200, qSize=0]
^-- Striped thread pool [active=2, idle=198, qSize=0]
```

Can you please tell why these warnings are coming. Although in client side,
I added a check if query delays to print a warning. But it didn't through
any warnings in client machine. It's weird why this warning is coming in
server logs.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/