[
https://issues.apache.org/jira/browse/ASTERIXDB-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994228#comment-15994228
]
Ian Maxon commented on ASTERIXDB-1900:
--------------------------------------
This is related to an issue with incorrect resource use counting on the
Metadata indexes. The methods in MetadataNode enter the
PrimaryIndexOperationTracker but only the index that is part of the
MetadataTransaction has its reference count decremented through the normal
callback via LogBuffer. I have a fix but I think the implications require some
thought as to how it affects the consistency design of the Metadata. A work
around is to simply restart after modifying anything in Metadata, this way the
reference counts are at 0 and soft checkpoints will proceed as intended.
> Transaction log files are not truncated.
> ----------------------------------------
>
> Key: ASTERIXDB-1900
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1900
> Project: Apache AsterixDB
> Issue Type: Bug
> Reporter: Taewoo Kim
> Assignee: Ian Maxon
>
> On the current master, transaction log files are not truncated when I ingest
> more than millions of records. I have a 10-nodes cluster (each has 4
> partitions). Before restarting, the transaction log files are never deleted
> on one node.
> For example, the following shows that the transaction files are never deleted
> since a feed ingestion has started on May 1, 17:37. This happens on only one
> node.
> {code}
> -rw-r--r-- 1 taewok2 grad 130 May 2 12:03 checkpoint_1493751802512
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 21:23 transaction_log_10
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 22:09 transaction_log_11
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 22:57 transaction_log_12
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 23:48 transaction_log_13
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 00:37 transaction_log_14
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 01:21 transaction_log_15
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 02:07 transaction_log_16
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 02:52 transaction_log_17
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 03:41 transaction_log_18
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 04:27 transaction_log_19
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 05:11 transaction_log_20
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 06:00 transaction_log_21
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 06:45 transaction_log_22
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 07:33 transaction_log_23
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 08:18 transaction_log_24
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 09:08 transaction_log_25
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 09:54 transaction_log_26
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 10:44 transaction_log_27
> -rw-r--r-- 1 taewok2 grad 1.0G May 2 11:32 transaction_log_28
> -rw-r--r-- 1 taewok2 grad 654M May 2 12:04 transaction_log_29
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 17:37 transaction_log_3
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 18:03 transaction_log_4
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 18:33 transaction_log_5
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 19:03 transaction_log_6
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 19:29 transaction_log_7
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 19:56 transaction_log_8
> -rw-r--r-- 1 taewok2 grad 1.0G May 1 20:39 transaction_log_9
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
