Re: Capacity Planning - Calculating Memory Usage
Hi Val, I'm sorry, of course only @QuerySqlField(index = true) makes an index on objects field. Fields without indexes make none additional overhead. Group index on multiple fields is a one index (isn't it?) I don't understand what is still unclear. Entry footprint = key footprint + value footprint + entry overhead + index overhead. Index overhead depends on how many indices are enabled for the entry type. 2016-12-23 2:06 GMT+07:00 Valentin Kulichenko: > Alexandr, > > See my comments below. > > On Wed, Dec 21, 2016 at 7:01 PM, Alexandr Kuramshin > wrote: > > > Hi Val, > > > > the understanding is simple. > > > > When you enables the single index on entry class you get "First index > > overhead" per entry. > > > > When you enables two indices on entry class you get "First index > overhead" > > + "Next index overhead" per entry. > > > > With three indices you get "First index overhead" + 2 * "Next index > > overhead", and so on... > > > > This should be explained in more detail, probably with some trivial > example. Currently it's very unclear. > > > > > > Each annotated field with @QuerySqlField is an index, except multiple > > fields annotated with @QuerySqlField.Group. > > > > This actually confuses me a lot, because a field can be created with or > without index? Can you please clarify? How much overhead is introduced by a > field without index? With index? What about group indexes? > > > > > > Another way to defining indices is to use property "queryEntities" and > it's > > subproperty "indexes". See the article [1] > > > > [1] https://apacheignite.readme.io/docs/indexes > > > > 2016-12-20 8:38 GMT+07:00 Valentin Kulichenko < > > valentin.kuliche...@gmail.com > > >: > > > > > Alexandr, > > > > > > Can you please clarify what is "First index overhead" and "Next index > > > overhead"? Generally, I think overhead provided by indexes should be > > > described in more details, now it's not very clear what happens when > > > indexes are added. > > > > > > Also the calculation example should be a separate section. > > > > > > -Val > > > > > > On Wed, Dec 14, 2016 at 1:07 AM, Alexandr Kuramshin < > > ein.nsk...@gmail.com> > > > wrote: > > > > > > > Thank you, Andrey, > > > > > > > > I'll do additional tests with expire policy and update the article. > > > > > > > > 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov < > > andrey.mashen...@gmail.com > > > >: > > > > > > > > > Alexandr, > > > > > > > > > > In addition. If expire policy is configured, there is additional > > > overhead > > > > > to entries can be tracked by TtlManager. > > > > > This overhead is OnHeap and does not depend on cache MemoryMode > > (until > > > > > Ignite-3840 will be in master). > > > > > > > > > > For now overhead is about 32-40 bytes (EntryWrapper itself) + > (40-48) > > > > bytes > > > > > (ConcurrentSkipList node) per entry. > > > > > > > > > > > > > > > > > > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin < > > > > ein.nsk...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > Hello, Igniters, > > > > > > > > > > > > I'd like to represent updated article [1] about the subject. > > > > > > > > > > > > And I'll very appreciate your comments and questions about it. > > > > > > > > > > > > Please review. > > > > > > > > > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > > > > > > > > > > > -- > > > > > > Thanks, > > > > > > Alexandr Kuramshin > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > С уважением, > > > > > Машенков Андрей Владимирович > > > > > Тел. +7-921-932-61-82 > > > > > > > > > > Best regards, > > > > > Andrey V. Mashenkov > > > > > Cerr: +7-921-932-61-82 > > > > > > > > > > > > > > > > > > > > > -- > > > > Thanks, > > > > Alexandr Kuramshin > > > > > > > > > > > > > > > -- > > Thanks, > > Alexandr Kuramshin > > > -- Thanks, Alexandr Kuramshin
Re: Capacity Planning - Calculating Memory Usage
Alexandr, See my comments below. On Wed, Dec 21, 2016 at 7:01 PM, Alexandr Kuramshinwrote: > Hi Val, > > the understanding is simple. > > When you enables the single index on entry class you get "First index > overhead" per entry. > > When you enables two indices on entry class you get "First index overhead" > + "Next index overhead" per entry. > > With three indices you get "First index overhead" + 2 * "Next index > overhead", and so on... > This should be explained in more detail, probably with some trivial example. Currently it's very unclear. > > Each annotated field with @QuerySqlField is an index, except multiple > fields annotated with @QuerySqlField.Group. > This actually confuses me a lot, because a field can be created with or without index? Can you please clarify? How much overhead is introduced by a field without index? With index? What about group indexes? > > Another way to defining indices is to use property "queryEntities" and it's > subproperty "indexes". See the article [1] > > [1] https://apacheignite.readme.io/docs/indexes > > 2016-12-20 8:38 GMT+07:00 Valentin Kulichenko < > valentin.kuliche...@gmail.com > >: > > > Alexandr, > > > > Can you please clarify what is "First index overhead" and "Next index > > overhead"? Generally, I think overhead provided by indexes should be > > described in more details, now it's not very clear what happens when > > indexes are added. > > > > Also the calculation example should be a separate section. > > > > -Val > > > > On Wed, Dec 14, 2016 at 1:07 AM, Alexandr Kuramshin < > ein.nsk...@gmail.com> > > wrote: > > > > > Thank you, Andrey, > > > > > > I'll do additional tests with expire policy and update the article. > > > > > > 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov < > andrey.mashen...@gmail.com > > >: > > > > > > > Alexandr, > > > > > > > > In addition. If expire policy is configured, there is additional > > overhead > > > > to entries can be tracked by TtlManager. > > > > This overhead is OnHeap and does not depend on cache MemoryMode > (until > > > > Ignite-3840 will be in master). > > > > > > > > For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48) > > > bytes > > > > (ConcurrentSkipList node) per entry. > > > > > > > > > > > > > > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin < > > > ein.nsk...@gmail.com > > > > > > > > > wrote: > > > > > > > > > Hello, Igniters, > > > > > > > > > > I'd like to represent updated article [1] about the subject. > > > > > > > > > > And I'll very appreciate your comments and questions about it. > > > > > > > > > > Please review. > > > > > > > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > > > > > > > > > -- > > > > > Thanks, > > > > > Alexandr Kuramshin > > > > > > > > > > > > > > > > > > > > > -- > > > > С уважением, > > > > Машенков Андрей Владимирович > > > > Тел. +7-921-932-61-82 > > > > > > > > Best regards, > > > > Andrey V. Mashenkov > > > > Cerr: +7-921-932-61-82 > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Alexandr Kuramshin > > > > > > > > > -- > Thanks, > Alexandr Kuramshin >
Re: Capacity Planning - Calculating Memory Usage
Hi Val, the understanding is simple. When you enables the single index on entry class you get "First index overhead" per entry. When you enables two indices on entry class you get "First index overhead" + "Next index overhead" per entry. With three indices you get "First index overhead" + 2 * "Next index overhead", and so on... Each annotated field with @QuerySqlField is an index, except multiple fields annotated with @QuerySqlField.Group. Another way to defining indices is to use property "queryEntities" and it's subproperty "indexes". See the article [1] [1] https://apacheignite.readme.io/docs/indexes 2016-12-20 8:38 GMT+07:00 Valentin Kulichenko: > Alexandr, > > Can you please clarify what is "First index overhead" and "Next index > overhead"? Generally, I think overhead provided by indexes should be > described in more details, now it's not very clear what happens when > indexes are added. > > Also the calculation example should be a separate section. > > -Val > > On Wed, Dec 14, 2016 at 1:07 AM, Alexandr Kuramshin > wrote: > > > Thank you, Andrey, > > > > I'll do additional tests with expire policy and update the article. > > > > 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov >: > > > > > Alexandr, > > > > > > In addition. If expire policy is configured, there is additional > overhead > > > to entries can be tracked by TtlManager. > > > This overhead is OnHeap and does not depend on cache MemoryMode (until > > > Ignite-3840 will be in master). > > > > > > For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48) > > bytes > > > (ConcurrentSkipList node) per entry. > > > > > > > > > > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin < > > ein.nsk...@gmail.com > > > > > > > wrote: > > > > > > > Hello, Igniters, > > > > > > > > I'd like to represent updated article [1] about the subject. > > > > > > > > And I'll very appreciate your comments and questions about it. > > > > > > > > Please review. > > > > > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > > > > > > > -- > > > > Thanks, > > > > Alexandr Kuramshin > > > > > > > > > > > > > > > > -- > > > С уважением, > > > Машенков Андрей Владимирович > > > Тел. +7-921-932-61-82 > > > > > > Best regards, > > > Andrey V. Mashenkov > > > Cerr: +7-921-932-61-82 > > > > > > > > > > > -- > > Thanks, > > Alexandr Kuramshin > > > -- Thanks, Alexandr Kuramshin
Re: Capacity Planning - Calculating Memory Usage
What is the relationship of apacheignite.gridgain.org to the open source Apache Ignite project? Thanks! On Tue, Dec 13, 2016 at 2:37 AM, Alexandr Kuramshinwrote: > Hello, Igniters, > > I'd like to represent updated article [1] about the subject. > > And I'll very appreciate your comments and questions about it. > > Please review. > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > -- > Thanks, > Alexandr Kuramshin >
Re: Capacity Planning - Calculating Memory Usage
Alexandr, Can you please clarify what is "First index overhead" and "Next index overhead"? Generally, I think overhead provided by indexes should be described in more details, now it's not very clear what happens when indexes are added. Also the calculation example should be a separate section. -Val On Wed, Dec 14, 2016 at 1:07 AM, Alexandr Kuramshinwrote: > Thank you, Andrey, > > I'll do additional tests with expire policy and update the article. > > 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov : > > > Alexandr, > > > > In addition. If expire policy is configured, there is additional overhead > > to entries can be tracked by TtlManager. > > This overhead is OnHeap and does not depend on cache MemoryMode (until > > Ignite-3840 will be in master). > > > > For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48) > bytes > > (ConcurrentSkipList node) per entry. > > > > > > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin < > ein.nsk...@gmail.com > > > > > wrote: > > > > > Hello, Igniters, > > > > > > I'd like to represent updated article [1] about the subject. > > > > > > And I'll very appreciate your comments and questions about it. > > > > > > Please review. > > > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > > > > > -- > > > Thanks, > > > Alexandr Kuramshin > > > > > > > > > > > -- > > С уважением, > > Машенков Андрей Владимирович > > Тел. +7-921-932-61-82 > > > > Best regards, > > Andrey V. Mashenkov > > Cerr: +7-921-932-61-82 > > > > > > -- > Thanks, > Alexandr Kuramshin >
Re: Capacity Planning - Calculating Memory Usage
Thank you, Andrey, I'll do additional tests with expire policy and update the article. 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov: > Alexandr, > > In addition. If expire policy is configured, there is additional overhead > to entries can be tracked by TtlManager. > This overhead is OnHeap and does not depend on cache MemoryMode (until > Ignite-3840 will be in master). > > For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48) bytes > (ConcurrentSkipList node) per entry. > > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin > > wrote: > > > Hello, Igniters, > > > > I'd like to represent updated article [1] about the subject. > > > > And I'll very appreciate your comments and questions about it. > > > > Please review. > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > > > -- > > Thanks, > > Alexandr Kuramshin > > > > > > -- > С уважением, > Машенков Андрей Владимирович > Тел. +7-921-932-61-82 > > Best regards, > Andrey V. Mashenkov > Cerr: +7-921-932-61-82 > -- Thanks, Alexandr Kuramshin
Re: Capacity Planning - Calculating Memory Usage
Alexandr, In addition. If expire policy is configured, there is additional overhead to entries can be tracked by TtlManager. This overhead is OnHeap and does not depend on cache MemoryMode (until Ignite-3840 will be in master). For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48) bytes (ConcurrentSkipList node) per entry. On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshinwrote: > Hello, Igniters, > > I'd like to represent updated article [1] about the subject. > > And I'll very appreciate your comments and questions about it. > > Please review. > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > -- > Thanks, > Alexandr Kuramshin > -- С уважением, Машенков Андрей Владимирович Тел. +7-921-932-61-82 Best regards, Andrey V. Mashenkov Cerr: +7-921-932-61-82
Re: Capacity Planning - Calculating Memory Usage
Alexander, great update, thanks. Yakov, Alexey Scherbakov, Val, Please do the review first since you’re as the ones who deal a lot with performance and memory usage related questions. — Denis > On Dec 12, 2016, at 11:37 PM, Alexandr Kuramshinwrote: > > Hello, Igniters, > > I'd like to represent updated article [1] about the subject. > > And I'll very appreciate your comments and questions about it. > > Please review. > > [1] http://apacheignite.gridgain.org/docs/capacity-planning > > -- > Thanks, > Alexandr Kuramshin
Capacity Planning - Calculating Memory Usage
Hello, Igniters, I'd like to represent updated article [1] about the subject. And I'll very appreciate your comments and questions about it. Please review. [1] http://apacheignite.gridgain.org/docs/capacity-planning -- Thanks, Alexandr Kuramshin