> On Jun 5, 2017, at 8:02 PM, Konstantin Boudnik <c...@apache.org> wrote: > > On Mon, Jun 05, 2017 at 07:41PM, Dmitriy Setrakyan wrote: >> On Mon, Jun 5, 2017 at 6:46 PM, Konstantin Boudnik <c...@apache.org> wrote: >> >>> Wow, hold on - as far as I remember there was a VOTE to accept the >>> contribution of the code into the project _on a branch_. We haven't vetted >>> its >>> inclusion into the next release, We are still at the phase of getting >>> familiar >>> with the code. >>> >> >> Cos, the community has been vetting the inclusion of the new code for over >> 3 weeks already (the process and dates are documented here [2]). To be >> honest, I am not sure what the appropriate time frame should be, but I >> would think that a month would be a good check-in point. > > I would think it should be as long as we, as the community, are comfortable > with the massive change coming into the release. We already been through the > discussion on the timing, etc. and I don't want to harp on that.
Just in case I’m working on the high-level storage documentation. Here is a draft for your reference: https://apacheignite.readme.io/docs/distributed-persistent-store <https://apacheignite.readme.io/docs/distributed-persistent-store> The page is invisible for wide audience at the moment. Only for Ignite committers. — Denis > On Jun 5, 2017, at 8:02 PM, Konstantin Boudnik <c...@apache.org> wrote: > > On Mon, Jun 05, 2017 at 07:41PM, Dmitriy Setrakyan wrote: >> On Mon, Jun 5, 2017 at 6:46 PM, Konstantin Boudnik <c...@apache.org> wrote: >> >>> Wow, hold on - as far as I remember there was a VOTE to accept the >>> contribution of the code into the project _on a branch_. We haven't vetted >>> its >>> inclusion into the next release, We are still at the phase of getting >>> familiar >>> with the code. >>> >> >> Cos, the community has been vetting the inclusion of the new code for over >> 3 weeks already (the process and dates are documented here [2]). To be >> honest, I am not sure what the appropriate time frame should be, but I >> would think that a month would be a good check-in point. > > I would think it should be as long as we, as the community, are comfortable > with the massive change coming into the release. We already been through the > discussion on the timing, etc. and I don't want to harp on that. > >> There is also an active stabilization thread for the persistence branch >> [3]. I encouraged the community to get involved and post any questions or >> concerns there as well. >> >> There is an upcoming in-memory computing conference that is coming up in >> June in Amsterdam [4], where there are many Ignite talks scheduled. It >> would be great to be able to talk about the persistence features of Ignite >> there as well. I would like to ask the community to mobilize with reviewing >> the donated code, so we could have something concrete to tell the audience >> on the conference. > > I am sure that having the code on the branch is good enough to be able to talk > about this. Having this in the release isn't really a contingency to be able > to make a presentation, right? > > Cos > >> [2] >> http://incubator.apache.org/ip-clearance/persistent-distributed-store-ignite.html >> [3] >> http://apache-ignite-developers.2346864.n4.nabble.com/Persistent-Store-Stabilization-for-release-td18288.html >> [4] https://imcsummit.org/ >> >> >> >>> And from what I am seeing in the discussions like this [1], we need to be >>> extra careful. >>> >> >> I would keep the discussion in [1] separate from the persistence store. >> These are 2 unrelated issues. I will respond on [1] either today or >> tomorrow, but I agree in general that it should be fixed ASAP. >> >> >>> BTW, you have sent this email 9 days before the vote had happened! A bit >>> too >>> soon, if you ask me. >>> >> >> Cos, this email was sent 1 week after the initial donated code branch was >> presented to the community (see [2] above). The developers involved were >> eager to make it available to the users as soon as possible, but no code >> has been merged to the master branch yet. >> >> I would like to encourage everyone in the community to participate in the >> persistence branch coding discussions, like the one in this thread, and >> familiarize themselves with the code. >> >> >>> >>> [1] https://is.gd/UQCr51 >>> >>> Cos >>> >>> On Wed, May 17, 2017 at 11:16AM, Dmitriy Govorukhin wrote: >>>> Folk, >>>> >>>> As you know, ignite 2.1 will contain new module (pds), it will be >>>> provide ability to store data on disk. Let's discuss what type of >>>> metrics we need for this? >>>> I think it must be metrics per memory policy, per cache, checkpoint, >>>> and global metrics which will be aggregate all metrics. >>>> >>>> I did sketch. >>>> >>>> PersistentStoreMetrics.java >>>> >>>> public interface PersistentStoreMetrics { >>>> >>>> // Global metrics. >>>> >>>> public long getMemorySize(); >>>> >>>> public long getDiskSize(); >>>> >>>> public long getPagesInMemory(); >>>> >>>> public long getPagesSizeInMemory(); >>>> >>>> public long getPagesOnDisk(); >>>> >>>> public long getPagesSizeOnDisk(); >>>> >>>> public long getFreePages(); >>>> >>>> public long getFreePagesSize(); >>>> >>>> public long getDirtyPages(); >>>> >>>> public long getDirtyPagesSize(); >>>> >>>> public long walLog(); >>>> >>>> public long walLogSize(); >>>> >>>> // Frequency. >>>> >>>> public long getPagesRead(); >>>> >>>> public long getPagesWrite(); >>>> >>>> public long getFsync(); >>>> >>>> public long getWal(); >>>> >>>> public long getAverageWalFsyncTime(); >>>> >>>> // Per cache. >>>> >>>> public PersistentStoreCacheMetrics cache(String name); >>>> >>>> public PersistentStoreCacheMetrics cache(int cacheId); >>>> >>>> // For last checkpoint. >>>> >>>> public PersistentStoreCheckpointMetrics getLastCheckPoint(); >>>> } >>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> >>>> PersistentStoreCacheMetrics.java >>>> >>>> public interface PersistentStoreCacheMetrics { >>>> >>>> public String name(); >>>> >>>> public double getFillFactor(); >>>> >>>> public double getFillFactor(int part); >>>> >>>> public long getMemorySize(); >>>> >>>> public long getDiskSize(); >>>> >>>> public long getPagesInMemory(); >>>> >>>> public long getPagesSizeInMemory(); >>>> >>>> public long getPagesOnDisk(); >>>> >>>> public long getPagesSizeOnDisk(); >>>> >>>> public long getFreePages(); >>>> >>>> public long getFreePagesSize(); >>>> >>>> public long getDirtyPages(); >>>> >>>> public long getDirtyPagesSize(); >>>> >>>> public long getPagesRead(); >>>> >>>> public long getPagesWritten(); >>>> } >>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> >>>> PersistentStoreCheckpointMetrics.java >>>> >>>> public interface PersistentStoreCheckpointMetrics { >>>> >>>> public long getTotalPages(); >>>> >>>> //TODO Page type is internal? >>>> public long[] pagesType(); >>>> >>>> public long getExecutingTime(); >>>> >>>> public long getFsyncTime(); >>>> >>>> public long getPagesCopyOnWrite(); >>>> } >>>
