It's not the while data store - snapshots occur per shard. But since everything probably goes into one shard, it essentially is the whole DS.
No behavior change - this limit hadn't been hit until now. On Wed, Jan 11, 2017 at 8:34 AM, Kochba, Alon <alo...@hpe.com> wrote: > + Daniel and Marcus > > > > Jumping in here with some questions.. > > Is the bottom line here that the *whole config DS* size cannot exceed 2GB > or did I get something wrong? > > Is it possible we never encountered such a scenario in the S3P performance > tests done in Beryllium/Boron [1], or did something change in the behavior? > > > > It seems that this limitation is extremely problematic for scale, and > should be easily recreated by doing NetVirt scale tests (loading ~200 > computes and creating a VM on each, or configuring a TZ for all of them > manually). > > > > I see 1800 OVSDB nodes were installed in [1], but not sure if any flows > were configured on the OpenFlow side which might cause the DS to scale.. > > We also need to monitor what part of the DS is taking so much space - we > might have redundant data being stored catching up space, but in any case > projects would hit 2GB at some point. > > > > [1] https://www.opendaylight.org/sites/www.opendaylight.org/files/odl_ > performancetechnicalreport_1-1_052716.pdf > > --alon > > > > *From:* netvirt-dev-boun...@lists.opendaylight.org [mailto: > netvirt-dev-boun...@lists.opendaylight.org] *On Behalf Of *Sela, Guy > *Sent:* Wednesday, 11 January 2017 11:23 > *To:* Tom Pantelis <tompante...@gmail.com> > *Cc:* odl netvirt dev <netvirt-...@lists.opendaylight.org>; > controller-dev@lists.opendaylight.org; Robert Varga <n...@hq.sk> > *Subject:* Re: [netvirt-dev] [controller-dev] [mdsal-dev] Netvirt Scale > tests: OutOfMemory from datastore > > > > Opened a bug: > > https://bugs.opendaylight.org/show_bug.cgi?id=7521 > > > > > > *From:* Tom Pantelis [mailto:tompante...@gmail.com <tompante...@gmail.com>] > > *Sent:* Wednesday, January 11, 2017 1:06 AM > *To:* Sela, Guy <guy.s...@hpe.com> > *Cc:* Robert Varga <n...@hq.sk>; controller-dev@lists.opendaylight.org; > odl netvirt dev <netvirt-...@lists.opendaylight.org> > *Subject:* Re: [controller-dev] [mdsal-dev] Netvirt Scale tests: > OutOfMemory from datastore > > > > operational won't snapshot if not persisted or replicated. > > > > Can you open a bug for tracking? > > > > On Tue, Jan 10, 2017 at 4:12 PM, Sela, Guy <guy.s...@hpe.com> wrote: > > Do the snapshot happen only to the configuration data store or to the > operational as well? > > -----Original Message----- > From: Robert Varga [mailto:n...@hq.sk] > Sent: Tuesday, January 10, 2017 10:21 PM > To: Tom Pantelis <tompante...@gmail.com>; Sela, Guy <guy.s...@hpe.com> > > Cc: controller-dev@lists.opendaylight.org; odl netvirt dev < > netvirt-...@lists.opendaylight.org> > Subject: Re: [controller-dev] [mdsal-dev] Netvirt Scale tests: OutOfMemory > from datastore > > On 01/10/2017 08:16 PM, Tom Pantelis wrote: > > Since the length field of an array is an int, it's constrained to > > Integer.MAX_VALUE (~2G). To handle snapshots larger than 2G we'll have > > to chuck it into multiple byte[]. > > That could get funky on the reassembly side of things. If we go that > route, we may as well switch to something more friendly, like > java.nio.ByteBuffer(), too. > > Guy, just out of curiosity: how much duplication is in your data set? > > Regards, > Robert > > >
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
