Re: [akka-user] Implements CRDTs conters more than 100 000

2017-10-13 Thread Johan Andrén 

>
> I have a big tree of business objects each of them contain counter and 
> threshold when threshold reach then object change its state.


This may also be a reason why Distrubed Data isn't a perfect fit to your 
problem, the CRDTs are eventually consistent, that means that the counter 
could have +n added separately on every node of your cluster but the other 
nodes does not see this right away, if you want to make sure you act on the 
threshold being reached only once and act differently for all requests 
after that it will not be possible with Distriubted Data but you would 
rather need to represent each entity as an actor (perhaps together with 
sharding) to achieve consistency.

--
Johan
Akka Team

On Thursday, October 12, 2017 at 10:04:06 PM UTC+2, Patrik Nordwall wrote:
>
> Might be some misunderstanding of terminology here. The quote from the 
> documentation is about Distributed Data. In last post I read it as you talk 
> about an actor tree. Anyway, in distributed Data you would use many 
> ORCounterMap as top level entries, and each such map would hold a number of 
> named counters. E.g. 1 maps with 100 counters each. If it actually 
> scales to these numbers is something you have to try.
>
> /Patrik
> tors 12 okt. 2017 kl. 15:08 skrev Hsnamed :
>
>> Yes , i saw that limitation and yes i have 1m top entities.  As example , 
>> there are four level of tree
>>
>> 1 - Category of markets groups
>> 2 - Markets group
>> 3 - Market - eg. Derivative
>> 4 - selection in market 
>>
>> Each market may have one or more positions .
>> If counter of selection reaches threshold - position change the state e.g 
>> diactivated
>>
>> I think about it as create actor system which equals the tree, because 
>> increments must be ordered  in selection context.
>>
>>
>>
>>
>>
>> четверг, 12 октября 2017 г., 15:41:14 UTC+3 пользователь Konrad Malawski 
>> написал:
>>>
>>> It’s as the docs explain: 
>>>
>>> It is not intended for *Big Data*. The number of top level entries 
>>> should not exceed 10. When a new node is added to the cluster all these 
>>> entries are transferred (gossiped) to the new node. The entries are split 
>>> up in chunks and all existing nodes collaborate in the gossip, but it will 
>>> take a while (tens of seconds) to transfer all entries and this means that 
>>> you cannot have too many top level entries. The current recommended limit 
>>> is 10. We will be able to improve this if needed, but the design is 
>>> still not intended for billions of entries.
>>> It depends on your data architecture though.
>>> Are you sure it would be 1 million actual top level entries?
>>>
>>> -- 
>>> Cheers,
>>> Konrad 'ktoso ' Malawski
>>> Akka  @ Lightbend 
>>>
>>> On October 12, 2017 at 21:24:37, Hsnamed (hsn...@gmail.com) wrote:
>>>
>> Hello, how can i implement 1 million CRDTs counters to use akka without 
>>> big disadvantage ? 
>>>
>>> I have a big tree of business objects each of them contain counter and 
>>> threshold when threshold reach then object change its state.
>>>
>>> Tree have more than 1 million objects. Does it possible to implements 
>>> using akka ? 
>>>
>>> Thanks for advices.
>>>
>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: 
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >> Search the archives: 
>>> https://groups.google.com/group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google 
>>> Groups "Akka User List" group.
>>>
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>>
>>>
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails

Re: [akka-user] Implements CRDTs conters more than 100 000

2017-10-12 Thread Patrik Nordwall 
Might be some misunderstanding of terminology here. The quote from the
documentation is about Distributed Data. In last post I read it as you talk
about an actor tree. Anyway, in distributed Data you would use many
ORCounterMap as top level entries, and each such map would hold a number of
named counters. E.g. 1 maps with 100 counters each. If it actually
scales to these numbers is something you have to try.

/Patrik
tors 12 okt. 2017 kl. 15:08 skrev Hsnamed :

> Yes , i saw that limitation and yes i have 1m top entities.  As example ,
> there are four level of tree
>
> 1 - Category of markets groups
> 2 - Markets group
> 3 - Market - eg. Derivative
> 4 - selection in market
>
> Each market may have one or more positions .
> If counter of selection reaches threshold - position change the state e.g
> diactivated
>
> I think about it as create actor system which equals the tree, because
> increments must be ordered  in selection context.
>
>
>
>
>
> четверг, 12 октября 2017 г., 15:41:14 UTC+3 пользователь Konrad Malawski
> написал:
>>
>> It’s as the docs explain:
>>
>> It is not intended for *Big Data*. The number of top level entries
>> should not exceed 10. When a new node is added to the cluster all these
>> entries are transferred (gossiped) to the new node. The entries are split
>> up in chunks and all existing nodes collaborate in the gossip, but it will
>> take a while (tens of seconds) to transfer all entries and this means that
>> you cannot have too many top level entries. The current recommended limit
>> is 10. We will be able to improve this if needed, but the design is
>> still not intended for billions of entries.
>> It depends on your data architecture though.
>> Are you sure it would be 1 million actual top level entries?
>>
>> --
>> Cheers,
>> Konrad 'ktoso ' Malawski
>> Akka  @ Lightbend 
>>
>> On October 12, 2017 at 21:24:37, Hsnamed (hsn...@gmail.com) wrote:
>>
> Hello, how can i implement 1 million CRDTs counters to use akka without
>> big disadvantage ?
>>
>> I have a big tree of business objects each of them contain counter and
>> threshold when threshold reach then object change its state.
>>
>> Tree have more than 1 million objects. Does it possible to implements
>> using akka ?
>>
>> Thanks for advices.
>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>>
>>
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Implements CRDTs conters more than 100 000

2017-10-12 Thread Hsnamed 
Yes , i saw that limitation and yes i have 1m top entities.  As example , 
there are four level of tree

1 - Category of markets groups
2 - Markets group
3 - Market - eg. Derivative
4 - selection in market 

Each market may have one or more positions .
If counter of selection reaches threshold - position change the state e.g 
diactivated

I think about it as create actor system which equals the tree, because 
increments must be ordered  in selection context.





четверг, 12 октября 2017 г., 15:41:14 UTC+3 пользователь Konrad Malawski 
написал:
>
> It’s as the docs explain: 
>
> It is not intended for *Big Data*. The number of top level entries should 
> not exceed 10. When a new node is added to the cluster all these 
> entries are transferred (gossiped) to the new node. The entries are split 
> up in chunks and all existing nodes collaborate in the gossip, but it will 
> take a while (tens of seconds) to transfer all entries and this means that 
> you cannot have too many top level entries. The current recommended limit 
> is 10. We will be able to improve this if needed, but the design is 
> still not intended for billions of entries.
> It depends on your data architecture though.
> Are you sure it would be 1 million actual top level entries?
>
> -- 
> Cheers,
> Konrad 'ktoso ' Malawski
> Akka  @ Lightbend 
>
> On October 12, 2017 at 21:24:37, Hsnamed (hsn...@gmail.com ) 
> wrote:
>
> Hello, how can i implement 1 million CRDTs counters to use akka without 
> big disadvantage ? 
>
> I have a big tree of business objects each of them contain counter and 
> threshold when threshold reach then object change its state.
>
> Tree have more than 1 million objects. Does it possible to implements 
> using akka ? 
>
> Thanks for advices.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Implements CRDTs conters more than 100 000

2017-10-12 Thread Konrad “ktoso” Malawski 
It’s as the docs explain:

It is not intended for *Big Data*. The number of top level entries should
not exceed 10. When a new node is added to the cluster all these
entries are transferred (gossiped) to the new node. The entries are split
up in chunks and all existing nodes collaborate in the gossip, but it will
take a while (tens of seconds) to transfer all entries and this means that
you cannot have too many top level entries. The current recommended limit
is 10. We will be able to improve this if needed, but the design is
still not intended for billions of entries.
It depends on your data architecture though.
Are you sure it would be 1 million actual top level entries?

-- 
Cheers,
Konrad 'ktoso ' Malawski
Akka  @ Lightbend 

On October 12, 2017 at 21:24:37, Hsnamed (hsna...@gmail.com) wrote:

Hello, how can i implement 1 million CRDTs counters to use akka without big
disadvantage ?

I have a big tree of business objects each of them contain counter and
threshold when threshold reach then object change its state.

Tree have more than 1 million objects. Does it possible to implements using
akka ?

Thanks for advices.

--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Implements CRDTs conters more than 100 000

2017-10-12 Thread Hsnamed 
Hello, how can i implement 1 million CRDTs counters to use akka without big 
disadvantage ?

I have a big tree of business objects each of them contain counter and 
threshold when threshold reach then object change its state.

Tree have more than 1 million objects. Does it possible to implements using 
akka ? 

Thanks for advices.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.