Re: [pmacct-discussion] Realistic Scaling of pre_tag_map?
Hey Tim, Since it may take a few seconds to reload a large map, maybe i may discourage from - dunno - sub one minute reloading or suchs. The label, yes, can be as long as you like since it's malloc()'d - of course the longer you make it, the more space you burn and the more expensive it is to aggregate upon (since it's a key field). In theory the only char that you can't use is the default separator, the ',' sign. But i see you stumbled in some issues (both in this email i'm replying to and in the other): let me try to reproduce them at my end and come back to you. Paolo On Wed, Feb 12, 2020 at 08:23:30AM -0600, Tim Jackson wrote: > That's good news, since everything I've tested so far the maps_index has > worked with. Any worries about reloading the map often/quickly? > > Also is there a limit to how large the label can be in the pre_tag_map and > any characters that aren't supported? Seems as if '-' in any set_label > operation means the whole string gets ignored.. > > The use-case is just mapping ip+ifIndex -> downstream devices with a label, > but I've got a lot of interfaces to match there.. > > -- > Tim > > > On Wed, Feb 12, 2020, 12:47 AM Paolo Lucente wrote: > > > > > Hey Tim, > > > > It really depends whether you can leverage maps_index (*) or not. If yes > > then computations are O(1) and hence you can scale it as much as you > > like and i can confirm you there is people building maps of the same > > magnitude as you have in mind. If not then it's not going to work but > > then again i'd be interested in your use-case, how the map would look > > like, etc. > > > > Paolo > > > > (*) https://github.com/pmacct/pmacct/blob/1.7.4/CONFIG-KEYS#L1878-#L1891 > > > > On Tue, Feb 11, 2020 at 05:54:27PM -0600, Tim Jackson wrote: > > > Just curious, what's the realistic scaling of pre_tag_map? > > > > > > I'm looking to maybe put 50k+ entries in it and reload it every few > > > minutes.. > > > > > > Any real gotchas w/ that approach? > > > > > > -- > > > Tim > > > > > ___ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > > > > > ___ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Realistic Scaling of pre_tag_map?
That's good news, since everything I've tested so far the maps_index has worked with. Any worries about reloading the map often/quickly? Also is there a limit to how large the label can be in the pre_tag_map and any characters that aren't supported? Seems as if '-' in any set_label operation means the whole string gets ignored.. The use-case is just mapping ip+ifIndex -> downstream devices with a label, but I've got a lot of interfaces to match there.. -- Tim On Wed, Feb 12, 2020, 12:47 AM Paolo Lucente wrote: > > Hey Tim, > > It really depends whether you can leverage maps_index (*) or not. If yes > then computations are O(1) and hence you can scale it as much as you > like and i can confirm you there is people building maps of the same > magnitude as you have in mind. If not then it's not going to work but > then again i'd be interested in your use-case, how the map would look > like, etc. > > Paolo > > (*) https://github.com/pmacct/pmacct/blob/1.7.4/CONFIG-KEYS#L1878-#L1891 > > On Tue, Feb 11, 2020 at 05:54:27PM -0600, Tim Jackson wrote: > > Just curious, what's the realistic scaling of pre_tag_map? > > > > I'm looking to maybe put 50k+ entries in it and reload it every few > > minutes.. > > > > Any real gotchas w/ that approach? > > > > -- > > Tim > > > ___ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists > ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Realistic Scaling of pre_tag_map?
Hey Tim, It really depends whether you can leverage maps_index (*) or not. If yes then computations are O(1) and hence you can scale it as much as you like and i can confirm you there is people building maps of the same magnitude as you have in mind. If not then it's not going to work but then again i'd be interested in your use-case, how the map would look like, etc. Paolo (*) https://github.com/pmacct/pmacct/blob/1.7.4/CONFIG-KEYS#L1878-#L1891 On Tue, Feb 11, 2020 at 05:54:27PM -0600, Tim Jackson wrote: > Just curious, what's the realistic scaling of pre_tag_map? > > I'm looking to maybe put 50k+ entries in it and reload it every few > minutes.. > > Any real gotchas w/ that approach? > > -- > Tim > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Realistic Scaling of pre_tag_map?
Just curious, what's the realistic scaling of pre_tag_map? I'm looking to maybe put 50k+ entries in it and reload it every few minutes.. Any real gotchas w/ that approach? -- Tim ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
