hanahmily opened a new pull request, #485: URL: https://github.com/apache/skywalking-banyandb/pull/485
### Fix a bug that the Stream module didn't support duplicated in index-based filtering and sorting In the index, the document id holds the timestamp before these changes. This can lead to several incorrect reads when using the index to filter or sort the data, especially when there are multiple duplicated timestamps in a single time series. In this pull request, I replaced the timestamp with the elementID, which is unique in a time series. The sorted elementID is now represented by the hashed uint64 value instead of the original string. The file system version is also updated due to this change. ### Fix missing data when a tag filter is applied When sorting data by an index, if the final result size exceeds the offset+limit and a tag filter is applied, some elements may be lost in the query result. In this pull request, the `limit` and `tag filter` executor can drain the parent source until no more data is returned or until the offset+limit is reached. This should help resolve the issue of missing data. A new `distributed limit`, which is the same as the previous `limit`, is applied to the distributed executor. This is because the liaison couldn't drain data nodes several times; it should interact with each data node only once in a query request. ### Reverted Index Enhancement The "Write" operation has been removed from the reverted index. Now, we can use the `Batch` operation to write index data. The timestamp has been moved from the document id to a field with a date-time type, and its value is stored in the index. @ButterBright Please check the query process to verify the change is expected. - [x] Add a unit test to verify that the fix works. - [x] Explain briefly why the bug exists and how to fix it. - [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Fixes apache/skywalking#12371. - [x] Update the [`CHANGES` log](https://github.com/apache/skywalking-banyandb/blob/main/CHANGES.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
