Re: ignite stops expiring caches after a period of time

[Scott Feldstein]

Ah, interesting. Thanks for the suggestion!
On Sat, Dec 29, 2018 at 03:04 Ilya Kasnacheev 
wrote:

> Hello!
>
> Yes, you are right, you will have use some other means of expiry. However,
> you can also have a separate data region without persistence for data which
> should be prone to expiry.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
>
> сб, 29 дек. 2018 г. в 00:48, Scott Feldstein :
>
>> Hi Ilya,
>> Thanks. I assume since I’m using persistence that expiry of the data is
>> the only way to ensure that it is eventually purged. Is that correct or is
>> there something else I should look at?
>>
>> Yep, I’m fine with upgrading to 2.7, we like to be as close to the
>> bleeding edge with our technology stack anyway :)
>> On Fri, Dec 28, 2018 at 06:41 Ilya Kasnacheev 
>> wrote:
>>
>>> Hello!
>>>
>>> Since Ignite 2.0 works with OffHeap pages mostly, Page Eviction works
>>> best if you don't care about data (pure cache mode):
>>> https://apacheignite.readme.io/docs/evictions
>>>
>>> If it works for 2.7.0 I recommend you to stick with is, since if one
>>> would be fixing 2.6 they will yield 2.7 anyway :)
>>>
>>> Regards,
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 26 дек. 2018 г. в 22:25, Scott Feldstein :
>>>
 hi Ilya,
 Why wouldn't many people use Expiry with 2.x?  Is there another feature
 that I should be using that accomplishes the same thing?

 I've reproduced the issue for 2.6.0, but with 2.7.0 it looks like it is
 working.  Here is the repro if anyone is interested -

 https://github.com/scottmf/ignite-expiry-repro

 On Tue, Dec 25, 2018 at 5:30 AM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:

> Hello!
>
> Not many people are using Expiry with 2.x.
>
> If you can share a reproducer, please create a ticket right away and I
> hope that somebody will look into it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 25 дек. 2018 г. в 01:36, Scott Feldstein :
>
>> Thanks Ilya, I’ll upgrade and see if that works soon. Right now we’ve
>> switched to only using partitioned caches to avoid this issue. If it’s
>> something that isn’t fixed with 2.7 then I’ll create a bug and attach 
>> code
>> to reproduce it.
>>
>> I’m really surprised that no one else is complaining about it. The
>> repro is very straight forward..
>>
>>
>> On Mon, Dec 24, 2018 at 06:36 Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> There could be fixes for expiration between 2.6 and 2.7.
>>>
>>> However, if it will still be the case, we would be glad if you could
>>> post reproducer for this behavior.
>>>
>>> Note that you can almost emulate replicated cache by creating
>>> partitioned cache with large number of backups, maybe you should be 
>>> using
>>> that in the meantime.
>>>
>>> Regards,
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 21 дек. 2018 г. в 23:28, scottmf :
>>>
 I forgot to mention this is on Ignite 2.6.0 and i'm running with a
 3 node
 cluster.

 I've been experimenting some more and I'm finding that using
 PARTITIONED
 caches works fine, but replicated caches stop expiring usually
 within 10
 minutes.  The behavior is very consistent.

 I plan on trying this with Ignite 2.7.0...



 --
 Sent from: http://apache-ignite-users.70518.x6.nabble.com/

>>>

Re: Query Execution very slow

[Ilya Kasnacheev]

Hello!

The answer to all your questions is "try it and see". I think that I have
shown you enough tools so that you are now ready for research of your own.

Regards,
-- 
Ilya Kasnacheev


сб, 29 дек. 2018 г. в 15:59, Prasad Bhalerao :

> Hi Ilya,
>
> Thank you for the help, I tried your suggestion and it worked fine.  For
> Second query h2 was selecting idx1 which was slowing it down.
>
> I have couple of questions.
> 1) Is there any limit for JOIN temp table clause? In some cases I have
> 17000 ids and it was causing sql execution to hang but it was before
> implementing you suggestion. So I had to split the 17K list into batch of
> 50 ids.
> But now I have to execute this sql 17000/50 times in a loop which is again
> slowing down the process. One thing is I can increase the batch size.
> Is there any way to handle it in better way?
>
> 2) Is it necessary to push the ids first to the temp table and then join
> it with necessary table like we did in this case?
>
> 3) When I executed the same sql with 900 ids in temp table, it took 30
> seconds and fetched 385332 rows. Does ignite suppose to take this much time
> when number ids in temp table join case increases?
>
>
> Regards,
> Prasad
>
> On Sat, Dec 29, 2018 at 4:58 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> I have updated my Pull Request with USE INDEX:
>> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>> JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
>> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
>> (ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
>> getAffectedIPRange_2 :: TimeTakenToComplete=9 :: Size=2
>>
>> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>> JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
>> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
>> (ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
>> getAffectedIPRange_2 :: TimeTakenToComplete=25 :: Size=1260
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 28 дек. 2018 г. в 18:59, Prasad Bhalerao <
>> prasadbhalerao1...@gmail.com>:
>>
>>> Hi,
>>>
>>> After setting enforceJoinOrder to true it worked. Can you please explain
>>> how did it work... pushing the data to temp table first and then
>>> enforeOrder?
>>>
>>> Is it documented in ignite docs?
>>>
>>> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>>> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
>>> WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId
>>> getAffectedIPRange_3 :: *TimeTakenToComplete=10 ::* Size=1295
>>>
>>>
>>> *But second sql is still taking time. The only difference is it has
>>> ipStart and ipEnd filter in where clause.*
>>>
>>> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>>> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
>>> WHERE subscriptionId = ? AND (ipStart <= ? AND ipEnd >= ?) ORDER BY
>>> ipv4agd.assetGroupId
>>> getAffectedIPRange_2 :: *TimeTakenToComplete=25436* :: Size=1260
>>>
>>> Thanks,
>>> Prasad
>>>
>>> On Fri, Dec 28, 2018 at 9:02 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Did you set enforceJoinOrder to true?

 Regards,
 --
 Ilya Kasnacheev


 пт, 28 дек. 2018 г. в 18:19, Prasad Bhalerao <
 prasadbhalerao1...@gmail.com>:

> Hi,
>
> I tried your suggestion but it did not work. It is taking 22.8 seconds.
>
> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id,
> ipv4agd.assetGroupId, ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE
> (assetGroupId bigint = ? ) temp JOIN IpV4AssetGroupData ipv4agd ON
> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? ORDER BY
> ipv4agd.assetGroupId
>
> getAffectedIPRange_3 :: TimeTakenToComplete=22891 :: Size=1295
> Thanks,
> Prasad
>
> On Fri, Dec 28, 2018 at 8:32 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> I have created a PR for you:
>> https://github.com/prasadbhalerao1983/IgniteTestPrj/pull/1
>> With it, I can see:
>>
>> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id,
>> ipv4agd.assetGroupId, ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE
>> (assetGroupId bigint = ? ) temp JOIN IpV4AssetGroupData ipv4agd ON
>> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? ORDER 
>> BY
>> ipv4agd.assetGroupId
>> getAffectedIPRange_3 :: TimeTakenToComplete=11 :: Size=1295
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>

Re: javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Some of DataStreamer operations failed [failedCount=1]

[aealexsandrov]

Hi,

Could you please attach the full log file with:

Caused by: javax.cache.CacheException: class 
org.apache.ignite.IgniteCheckedException: Some of DataStreamer operations 
failed [failedCount=1] 
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
 
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1287)
 
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1388)
 
at com.XXX.datagrid.DataGridClient.writeAll(DataGridClient.java:190) 

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Query Execution very slow

[Prasad Bhalerao]

Hi Ilya,

Thank you for the help, I tried your suggestion and it worked fine.  For
Second query h2 was selecting idx1 which was slowing it down.

I have couple of questions.
1) Is there any limit for JOIN temp table clause? In some cases I have
17000 ids and it was causing sql execution to hang but it was before
implementing you suggestion. So I had to split the 17K list into batch of
50 ids.
But now I have to execute this sql 17000/50 times in a loop which is again
slowing down the process. One thing is I can increase the batch size.
Is there any way to handle it in better way?

2) Is it necessary to push the ids first to the temp table and then join it
with necessary table like we did in this case?

3) When I executed the same sql with 900 ids in temp table, it took 30
seconds and fetched 385332 rows. Does ignite suppose to take this much time
when number ids in temp table join case increases?


Regards,
Prasad

On Sat, Dec 29, 2018 at 4:58 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I have updated my Pull Request with USE INDEX:
> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
> JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
> (ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
> getAffectedIPRange_2 :: TimeTakenToComplete=9 :: Size=2
>
> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
> JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
> (ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
> getAffectedIPRange_2 :: TimeTakenToComplete=25 :: Size=1260
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 28 дек. 2018 г. в 18:59, Prasad Bhalerao  >:
>
>> Hi,
>>
>> After setting enforceJoinOrder to true it worked. Can you please explain
>> how did it work... pushing the data to temp table first and then
>> enforeOrder?
>>
>> Is it documented in ignite docs?
>>
>> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
>> WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId
>> getAffectedIPRange_3 :: *TimeTakenToComplete=10 ::* Size=1295
>>
>>
>> *But second sql is still taking time. The only difference is it has
>> ipStart and ipEnd filter in where clause.*
>>
>> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
>> WHERE subscriptionId = ? AND (ipStart <= ? AND ipEnd >= ?) ORDER BY
>> ipv4agd.assetGroupId
>> getAffectedIPRange_2 :: *TimeTakenToComplete=25436* :: Size=1260
>>
>> Thanks,
>> Prasad
>>
>> On Fri, Dec 28, 2018 at 9:02 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Did you set enforceJoinOrder to true?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 28 дек. 2018 г. в 18:19, Prasad Bhalerao <
>>> prasadbhalerao1...@gmail.com>:
>>>
 Hi,

 I tried your suggestion but it did not work. It is taking 22.8 seconds.

 getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
 ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
 JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
 WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId

 getAffectedIPRange_3 :: TimeTakenToComplete=22891 :: Size=1295
 Thanks,
 Prasad

 On Fri, Dec 28, 2018 at 8:32 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:

> Hello!
>
> I have created a PR for you:
> https://github.com/prasadbhalerao1983/IgniteTestPrj/pull/1
> With it, I can see:
>
> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id,
> ipv4agd.assetGroupId, ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE
> (assetGroupId bigint = ? ) temp JOIN IpV4AssetGroupData ipv4agd ON
> ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? ORDER BY
> ipv4agd.assetGroupId
> getAffectedIPRange_3 :: TimeTakenToComplete=11 :: Size=1295
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 28 дек. 2018 г. в 17:46, Prasad Bhalerao <
> prasadbhalerao1...@gmail.com>:
>
>> Can someone from community help me with t
>>
>> I have created a reproducer and uploaded it to GitHub. I have
>> created 3 cases to test the sql execution time.
>>
>> Please run *IgniteQueryTester_4* class to check the issue.
>> GitHub project: https://github
>> .com/prasadbhalerao1983/IgniteTestPrj.git

Re: error in running shared rdd example with intellij

[aealexsandrov]

Hi,

Possible that you change something in the source code. I attached the simple
maven project with SharedRDDExample.

SharedRddExample.zip

  

Could you please try to run it?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Query Execution very slow

[Ilya Kasnacheev]

Hello!

I have updated my Pull Request with USE INDEX:
getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
(ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
getAffectedIPRange_2 :: TimeTakenToComplete=9 :: Size=2

getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
JOIN IpV4AssetGroupData ipv4agd USE INDEX (ipv4_asset_group_data_idx2) ON
ipv4agd.assetGroupId = temp.assetGroupId WHERE subscriptionId = ? AND
(ipStart <= ? AND ipEnd >= ?) ORDER BY ipv4agd.assetGroupId
getAffectedIPRange_2 :: TimeTakenToComplete=25 :: Size=1260

Regards,
-- 
Ilya Kasnacheev


пт, 28 дек. 2018 г. в 18:59, Prasad Bhalerao :

> Hi,
>
> After setting enforceJoinOrder to true it worked. Can you please explain
> how did it work... pushing the data to temp table first and then
> enforeOrder?
>
> Is it documented in ignite docs?
>
> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
> WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId
> getAffectedIPRange_3 :: *TimeTakenToComplete=10 ::* Size=1295
>
>
> *But second sql is still taking time. The only difference is it has
> ipStart and ipEnd filter in where clause.*
>
> getAffectedIPRange_2 :: SQL_2=SELECT ipv4agd.id, ipv4agd.assetGroupId,
> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
> WHERE subscriptionId = ? AND (ipStart <= ? AND ipEnd >= ?) ORDER BY
> ipv4agd.assetGroupId
> getAffectedIPRange_2 :: *TimeTakenToComplete=25436* :: Size=1260
>
> Thanks,
> Prasad
>
> On Fri, Dec 28, 2018 at 9:02 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Did you set enforceJoinOrder to true?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 28 дек. 2018 г. в 18:19, Prasad Bhalerao <
>> prasadbhalerao1...@gmail.com>:
>>
>>> Hi,
>>>
>>> I tried your suggestion but it did not work. It is taking 22.8 seconds.
>>>
>>> getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
>>> ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
>>> JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
>>> WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId
>>>
>>> getAffectedIPRange_3 :: TimeTakenToComplete=22891 :: Size=1295
>>> Thanks,
>>> Prasad
>>>
>>> On Fri, Dec 28, 2018 at 8:32 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 I have created a PR for you:
 https://github.com/prasadbhalerao1983/IgniteTestPrj/pull/1
 With it, I can see:

 getAffectedIPRange_3 :: SQL_3=SELECT ipv4agd.id, ipv4agd.assetGroupId,
 ipv4agd.ipStart, ipv4agd.ipEnd FROM TABLE (assetGroupId bigint = ? ) temp
 JOIN IpV4AssetGroupData ipv4agd ON ipv4agd.assetGroupId = temp.assetGroupId
 WHERE subscriptionId = ? ORDER BY ipv4agd.assetGroupId
 getAffectedIPRange_3 :: TimeTakenToComplete=11 :: Size=1295

 Regards,
 --
 Ilya Kasnacheev


 пт, 28 дек. 2018 г. в 17:46, Prasad Bhalerao <
 prasadbhalerao1...@gmail.com>:

> Can someone from community help me with t
>
> I have created a reproducer and uploaded it to GitHub. I have created
> 3 cases to test the sql execution time.
>
> Please run *IgniteQueryTester_4* class to check the issue.
> GitHub project: https://github
> .com/prasadbhalerao1983/IgniteTestPrj.git
>
> Thanks,
> Prasad
>
>
> On Wed, Dec 26, 2018 at 11:18 PM Prasad Bhalerao <
> prasadbhalerao1...@gmail.com> wrote:
>
>> How to push the ids to temp table, can you please give any example?
>> Is it a in memory temp table created by ignite?
>>
>> Can you please explain how enforceJoinOrder will help in this case?
>> Thanks ,
>> Prasad
>>
>> On Wed 26 Dec, 2018, 9:37 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com wrote:
>>
>>> Hello!
>>>
>>> Can you try pushing temp table to 1st position and setting
>>> enforceJoinOrder=true?
>>>
>>> SELECT ipv4agd.id,
>>>   ipv4agd.assetGroupId,
>>>   ipv4agd.ipStart,
>>>   ipv4agd.ipEnd
>>> FROM TABLE (assetGroupId bigint = ? ) temp
>>> JOIN IpV4AssetGroupData ipv4agd
>>> ON ipv4agd.assetGroupId = temp.assetGroupId
>>> WHERE subscriptionId= ?
>>> AND (ipStart <= ? AND ipEnd >= ?)
>>> ORDER BY ipv4agd.assetGroupId
>>>
>>> See https://apacheignite.readme.io/docs/configuration-parameters
>>> for enforceJoinOrder.
>>>
>>> 

Re: Question about write speed.

[Ilya Kasnacheev]

Hello!

I have gathered some hints in this e-mail:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Performance-Issues-when-seeding-data-from-Spark-tp25551p25596.html

Regards,
-- 
Ilya Kasnacheev


сб, 29 дек. 2018 г. в 07:01, yangjiajun <1371549...@qq.com>:

> Hello.
>
> I use ignite 2.6  as a sql database and enable persistence(full_sync).I
> only
> start one node.All tables are replicated.I use streaming while insert
> data.The write speed is about 5~6k records per table.Is such speed
> normal?Can I improve it?
>
> Here is my config file：
> example-default.xml
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t2059/example-default.xml>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: ignite stops expiring caches after a period of time

[Ilya Kasnacheev]

Hello!

Yes, you are right, you will have use some other means of expiry. However,
you can also have a separate data region without persistence for data which
should be prone to expiry.

Regards,
-- 
Ilya Kasnacheev


сб, 29 дек. 2018 г. в 00:48, Scott Feldstein :

> Hi Ilya,
> Thanks. I assume since I’m using persistence that expiry of the data is
> the only way to ensure that it is eventually purged. Is that correct or is
> there something else I should look at?
>
> Yep, I’m fine with upgrading to 2.7, we like to be as close to the
> bleeding edge with our technology stack anyway :)
> On Fri, Dec 28, 2018 at 06:41 Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Since Ignite 2.0 works with OffHeap pages mostly, Page Eviction works
>> best if you don't care about data (pure cache mode):
>> https://apacheignite.readme.io/docs/evictions
>>
>> If it works for 2.7.0 I recommend you to stick with is, since if one
>> would be fixing 2.6 they will yield 2.7 anyway :)
>>
>> Regards,
>>
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 22:25, Scott Feldstein :
>>
>>> hi Ilya,
>>> Why wouldn't many people use Expiry with 2.x?  Is there another feature
>>> that I should be using that accomplishes the same thing?
>>>
>>> I've reproduced the issue for 2.6.0, but with 2.7.0 it looks like it is
>>> working.  Here is the repro if anyone is interested -
>>>
>>> https://github.com/scottmf/ignite-expiry-repro
>>>
>>> On Tue, Dec 25, 2018 at 5:30 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Not many people are using Expiry with 2.x.

 If you can share a reproducer, please create a ticket right away and I
 hope that somebody will look into it.

 Regards,
 --
 Ilya Kasnacheev


 вт, 25 дек. 2018 г. в 01:36, Scott Feldstein :

> Thanks Ilya, I’ll upgrade and see if that works soon. Right now we’ve
> switched to only using partitioned caches to avoid this issue. If it’s
> something that isn’t fixed with 2.7 then I’ll create a bug and attach code
> to reproduce it.
>
> I’m really surprised that no one else is complaining about it. The
> repro is very straight forward..
>
>
> On Mon, Dec 24, 2018 at 06:36 Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> There could be fixes for expiration between 2.6 and 2.7.
>>
>> However, if it will still be the case, we would be glad if you could
>> post reproducer for this behavior.
>>
>> Note that you can almost emulate replicated cache by creating
>> partitioned cache with large number of backups, maybe you should be using
>> that in the meantime.
>>
>> Regards,
>>
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 21 дек. 2018 г. в 23:28, scottmf :
>>
>>> I forgot to mention this is on Ignite 2.6.0 and i'm running with a 3
>>> node
>>> cluster.
>>>
>>> I've been experimenting some more and I'm finding that using
>>> PARTITIONED
>>> caches works fine, but replicated caches stop expiring usually
>>> within 10
>>> minutes.  The behavior is very consistent.
>>>
>>> I plan on trying this with Ignite 2.7.0...
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>

Re: Question about write speed.

[Mikael]

Hi!

5-6k records per what ? second ? anyway, it's not easy to answer, what 
kind of index if any, how big values you have and so on, 6000 records 
per second sound ok but could be faster.


Any normal usage would be with multiple nodes, if you are going to use 
just 1 node you do not have much use of Ignite, there are faster solutions.


Not sure I get the idea with the settings, a replicated cache will only 
have one copy anyway if you only have 1 node.


What is it you want to do and what kind of performance do you need ? can 
you use multiple nodes and load balance the writing between them ?


I have an application that writes around 15000+ records per second on a 
single node with streaming without problems (affinity string keys and 
small values, <100 bytes), and it can handle around 5000 entries per 
second with put( set), all with persistence enabled.


Mikael

Den 2018-12-29 kl. 05:01, skrev yangjiajun:

Hello.

I use ignite 2.6  as a sql database and enable persistence(full_sync).I only
start one node.All tables are replicated.I use streaming while insert
data.The write speed is about 5~6k records per table.Is such speed
normal?Can I improve it?

Here is my config file：
example-default.xml




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to define a cache template?

[Ilya Kasnacheev]

Hello!

It is also ommitted from doc, but you can put an asterisk '*' as the last
char of template name to make sure the cache is not started promptly:

CacheConfiguration cfg = new CacheConfiguration("myCacheTemplate*");


Regards,
-- 
Ilya Kasnacheev


сб, 29 дек. 2018 г. в 04:56, yangjiajun <1371549...@qq.com>:

> Hello!
>
> I want to make some cache settings global.It means I need to define a cache
> template and then use it,right?But I did not find any docs related to this.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: error in running shared rdd example with intellij

[Ilya Kasnacheev]

Hello!

I have no idea, but when I run this example from 2.7 source checkout it
runs to completion and there's no problems with client node.

Regards,
-- 
Ilya Kasnacheev


сб, 29 дек. 2018 г. в 09:33, Mehdi Seydali :

> how i could change client mode to server mode?
>
> On Sat, Dec 29, 2018 at 9:49 AM Mehdi Seydali 
> wrote:
>
>> i have run it in 4 nodes. but the same error has remained.
>>
>> On Fri, Dec 28, 2018 at 5:44 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Looks like this node is started in client node, but you don't have
>>> server nodes available for it to connect to.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 28 дек. 2018 г. в 15:59, mehdi sey :
>>>
 hi i want to run SharedRddExample in intellij IDE. but i have encounter
 error
 just as bellow . why?
 /usr/lib/jvm/java-8-oracle/bin/java
 -DIGNITE_HOME=/usr/local/apache-ignite-fabric-2.6.0-bin/

 -javaagent:/snap/intellij-idea-community/109/lib/idea_rt.jar=35933:/snap/intellij-idea-community/109/bin
 -Dfile.encoding=UTF-8 -classpath

 

Re: How to define a cache template?

[Павлухин Иван]

Hi,

Perhaps following docs section can help you [1].

[1] https://apacheignite.readme.io/docs/cache-template

сб, 29 дек. 2018 г. в 04:56, yangjiajun <1371549...@qq.com>:
>
> Hello!
>
> I want to make some cache settings global.It means I need to define a cache
> template and then use it,right?But I did not find any docs related to this.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/



-- 
Best regards,
Ivan Pavlukhin