Hi!
I need to clean old data and use a query for that:
DELETE FROM IgniteTransfer WHERE _KEY IN (SELECT TOP ? _KEY FROM
IgniteTransfer WHERE AccountId = ? ORDER BY TimeStamp ASC)
But sometimes I get a warning
[00:05:47 WRN] Query execution is too long [time=6476 ms, sql='SELECT
_KEY,
_VAL
FROM "LocalTransfers2".IGNITETRANSFER
WHERE _KEY IN( SELECT
_KEY
FROM "LocalTransfers2".IGNITETRANSFER
WHERE ACCOUNTID = ?2
ORDER BY =TIMESTAMP LIMIT ?1 )', plan=
SELECT
_KEY,
_VAL
FROM "LocalTransfers2".IGNITETRANSFER
/* "LocalTransfers2"."_key_PK": _KEY IN(SELECT
_KEY
FROM "LocalTransfers2".IGNITETRANSFER
/++
"LocalTransfers2".IGNITETRANSFER_ACCOUNTID_ASC_TIMESTAMP_ASC_IDX:
ACCOUNTID = ?2 ++/
WHERE ACCOUNTID = ?2
ORDER BY =TIMESTAMP
LIMIT ?1)
*/
WHERE _KEY IN(
SELECT
_KEY
FROM "LocalTransfers2".IGNITETRANSFER
/*
"LocalTransfers2".IGNITETRANSFER_ACCOUNTID_ASC_TIMESTAMP_ASC_IDX:
ACCOUNTID = ?2 */
WHERE ACCOUNTID = ?2
ORDER BY =TIMESTAMP
LIMIT ?1)
, parameters=[2000, 1001]]
6 seconds to remove 2000 elements is too much. What is the complexity of
this operation?
What is a most efficient way to remove old data in FIFO order?
All inserts and deletes handled with a dedicated thread. Multithreaded
reading.
Cache config
var query =new QueryEntity(typeof(long),typeof(IgniteTransfer))
{
Fields =new[]
{
new QueryField {Name =nameof(IgniteTransfer.AccountId),FieldType
=typeof(int)},
new QueryField {Name =nameof(IgniteTransfer.TransferId),FieldType
=typeof(long)},
new QueryField {Name =nameof(IgniteTransfer.AssetId),FieldType
=typeof(int)},
new QueryField {Name =nameof(IgniteTransfer.Reason),FieldType
=typeof(int)},
new QueryField {Name =nameof(IgniteTransfer.TimeStamp),FieldType
=typeof(long)},
new QueryField {Name =nameof(IgniteTransfer.IsDeposit),FieldType
=typeof(bool)}, },
Indexes =new[]
{
new QueryIndex
{
Fields =new[]
{
new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
new QueryIndexField {Name =nameof(IgniteTransfer.AssetId)}
},
IndexType = QueryIndexType.Sorted },
new QueryIndex
{
Fields =new[]
{
new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
new QueryIndexField {Name =nameof(IgniteTransfer.Reason)}
},
IndexType = QueryIndexType.Sorted },
new QueryIndex
{
Fields =new[]
{
new QueryIndexField {Name =nameof(IgniteTransfer.AccountId)},
new QueryIndexField {Name =nameof(IgniteTransfer.TimeStamp)}
},
IndexType = QueryIndexType.Sorted }
}
};
var localCacheCfg =new CacheConfiguration("LocalTransfers2", query)
{
CacheMode = CacheMode.Local,
AtomicityMode = CacheAtomicityMode.Atomic };
var cache = ignite.GetOrCreateCache<long, IgniteTransfer>(localCacheCfg);