[
https://issues.apache.org/jira/browse/IOTDB-510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jialin Qiao closed IOTDB-510.
-----------------------------
Fix Version/s: 0.10.0-SNAPSHOT
Resolution: Fixed
> A NullPointException in the TsFileProcessor when flushing memtables
> -------------------------------------------------------------------
>
> Key: IOTDB-510
> URL: https://issues.apache.org/jira/browse/IOTDB-510
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Engine
> Affects Versions: 0.10.0-SNAPSHOT
> Reporter: Xiangdong Huang
> Assignee: Xiangdong Huang
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0-SNAPSHOT
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Sometimes when we run TravicCI, the TsFileProcessorTest will hang up.
> If we add log in TsFileProcessor, we can find logs like:
>
> WARN [2020-02-22 02:45:54,092] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 60s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
> WARN [2020-02-22 02:46:54,095] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 120s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
> WARN [2020-02-22 02:47:54,108] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 180s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
> WARN [2020-02-22 02:48:54,122] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 240s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
> WARN [2020-02-22 02:49:54,132] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 300s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
> WARN [2020-02-22 02:50:54,099] [main]
> org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 -
> C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile
> has spent 360s for waiting flushing one memtable; 1 left (first:
> org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f).
> FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
>
> Actually it is caused by a nullpointException but we omit the exception in
> the releaseFlushedMemTable() method...
>
> The NullPointException happens when:
> # A: there is an async flushing task;
> # B: users call TsFileProcessor.syncClose;
> The process is:
> # B set `shouldClose = true` and has not put the final memtable (maybe a
> signalMemtable) into the flushingMemtable in asyncClose() (which is called by
> syncClose()).
> # A checks the `shouldeClose == true && flushingMemtables.isEmpty() ` in
> flushOneMemtable(), then A call endFile() to close the fileWriter;
> # B then put the final memtable into the flushingMemtable. When begin to
> flush the memtable, nullPointException is thrown (because the file Writer was
> set to null in endFile()).
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
