[jira] [Created] (IGNITE-10048) Bounded iteration in standalone WAL iterator with compaction enabled may skip records

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-10048:
-----------------------------------

             Summary: Bounded iteration in standalone WAL iterator with 
compaction enabled may skip records
                 Key: IGNITE-10048
                 URL: https://issues.apache.org/jira/browse/IGNITE-10048
             Project: Ignite
          Issue Type: Improvement
            Reporter: Ivan Rakov
             Fix For: 2.8


Bounded iteration with non-zero start/end offsets may skip some records in 
intermediate segments. Reproducer (wal compaction should be enabled):
{noformat}
    /**
     *
     */
    public void testBoundedIterationOverSeveralSegments() throws Exception {
        walCompactionEnabled = true;

        IgniteEx ig = (IgniteEx)startGrid();

        String archiveWalDir = getArchiveWalDirPath(ig);

        ig.cluster().active(true);

        IgniteCache<Object, Object> cache = ig.getOrCreateCache(
            new CacheConfiguration<>().setName("c-n").setAffinity(new 
RendezvousAffinityFunction(false, 32)));

        IgniteCacheDatabaseSharedManager sharedMgr = 
ig.context().cache().context().database();

        IgniteWriteAheadLogManager walMgr = 
ig.context().cache().context().wal();

        WALPointer fromPtr = null;

        int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;

        for (int i = 0; i < recordsCnt; i++) {
            WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));

            if (i == 100)
                fromPtr = ptr;
        }

        assertNotNull(fromPtr);

        cache.put(1, 1);

        forceCheckpoint();

        // Generate WAL segments for filling WAL archive folder.
        for (int i = 0; i < 2 * 
ig.configuration().getDataStorageConfiguration().getWalSegments(); i++) {
            sharedMgr.checkpointReadLock();

            try {
                walMgr.log(new SnapshotRecord(i, false), 
RolloverType.NEXT_SEGMENT);
            }
            finally {
                sharedMgr.checkpointReadUnlock();
            }
        }

        cache.put(2, 2);

        forceCheckpoint();

        U.sleep(5000);

        stopGrid();

        WALIterator it = new IgniteWalIteratorFactory(log)
            .iterator(new 
IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));

        TreeSet<Integer> foundCounters = new TreeSet<>();

        it.forEach(x -> {
            WALRecord rec = x.get2();

            if (rec instanceof PartitionDestroyRecord)
                foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
        });

        assertEquals(new Integer(100), foundCounters.first());
        assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
        assertEquals(recordsCnt - 100, foundCounters.size());
    }

{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)