Re: [PR] Pipe: remove unnecessary fsync operations for file pieces' transferring [iotdb]
SteveYurongSu merged PR #12410: URL: https://github.com/apache/iotdb/pull/12410 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] update Export/ImportDataTestIT [iotdb]
2b3c511 commented on PR #12399: URL: https://github.com/apache/iotdb/pull/12399#issuecomment-2076287220 https://github.com/apache/iotdb/assets/43237967/5aab06c3-6da9-46a1-8838-74f0f0459cb1;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] update Export/ImportDataTestIT [iotdb]
2b3c511 commented on PR #12399: URL: https://github.com/apache/iotdb/pull/12399#issuecomment-2076280586   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Change log level and opt error msg for source path of view is deleted [iotdb]
Beyyes merged PR #12409: URL: https://github.com/apache/iotdb/pull/12409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] iotdb 压缩线程处理异常之后，会无限处理错误数据。手动删除对应的序列也没用 [iotdb]
lintao666 commented on issue #12360: URL: https://github.com/apache/iotdb/issues/12360#issuecomment-2076184512 [error.log](https://github.com/apache/iotdb/files/15102931/error.log) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Colin fix pipe snapshot parser. [iotdb]
shuolinl opened a new pull request, #12413: URL: https://github.com/apache/iotdb/pull/12413 In this pr: 1. auth model can create user with raw password in pipe. 2. fix pipe snapshot parser will lose aligned timeseries' tags and attributes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: remove unnecessary fsync operations for file pieces' transferring [iotdb]
codecov[bot] commented on PR #12410: URL: https://github.com/apache/iotdb/pull/12410#issuecomment-2075372608 ## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/12410?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 42.92%. Comparing base [(`f5cf782`)](https://app.codecov.io/gh/apache/iotdb/commit/f5cf782df5b8ad6aa23f060308f240ac7d5894bf?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`03c8815`)](https://app.codecov.io/gh/apache/iotdb/pull/12410?dropdown=coverage=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). Additional details and impacted files ```diff @@Coverage Diff@@ ## master #12410 +/- ## = Coverage 42.91% 42.92% Complexity 69 69 = Files 3102 3102 Lines191264 191260-4 Branches 2277122771 = + Hits 8208882090+2 + Misses 109176 109170-6 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/12410?dropdown=coverage=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Pipe: Support partial insert [iotdb]
MiniSho opened a new pull request, #12412: URL: https://github.com/apache/iotdb/pull/12412 ## Description As title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: remove unnecessary fsync operations for file pieces' transferring [iotdb]
sonarcloud[bot] commented on PR #12410: URL: https://github.com/apache/iotdb/pull/12410#issuecomment-2075270388 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12410) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb=12410=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_iotdb=12410=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb=12410=false=true)  No data about Coverage  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12410=new_duplicated_lines_density=list) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12410) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Fix Concurrency Instability When IoTConsensus LogDispatcher Exits [iotdb]
BUAAserein opened a new pull request, #12411: URL: https://github.com/apache/iotdb/pull/12411 ## Description - Immediately check `Thread.interrupted()` for interrupts after `getBatch()` - Fix configuration logic in ReplicateTest -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Implement new plan optimizer: AggregationPushDown [iotdb]
sonarcloud[bot] commented on PR #12398: URL: https://github.com/apache/iotdb/pull/12398#issuecomment-2075084336 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12398) **Quality Gate failed** Failed conditions  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12398=new_coverage=list) (required ≥ 80%)  [5.1% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12398=new_duplicated_lines_density=list) (required ≤ 3%)  [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb=12398) (required ≥ A) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12398) ##  Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: Two stage aggregate framework & count-point-processor plugin [iotdb]
SteveYurongSu merged PR #12328: URL: https://github.com/apache/iotdb/pull/12328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: refine procedure timeout message [iotdb]
SteveYurongSu merged PR #12402: URL: https://github.com/apache/iotdb/pull/12402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: Two stage aggregate framework & count-point-processor plugin [iotdb]
sonarcloud[bot] commented on PR #12328: URL: https://github.com/apache/iotdb/pull/12328#issuecomment-2074906017 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) **Quality Gate failed** Failed conditions  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12328=new_coverage=list) (required ≥ 80%)  [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) (required ≥ A) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) ##  Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Change log level and opt error msg for source path of view is deleted [iotdb]
sonarcloud[bot] commented on PR #12409: URL: https://github.com/apache/iotdb/pull/12409#issuecomment-2074802047 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12409) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb=12409=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_iotdb=12409=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb=12409=false=true)  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12409=new_coverage=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12409=new_duplicated_lines_density=list) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12409) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Change log level and opt error msg for source path of view is deleted [iotdb]
JackieTien97 opened a new pull request, #12409: URL: https://github.com/apache/iotdb/pull/12409 1. chaneg the log level from error to warn when https://jira.infra.timecho.com:8443/browse/TIMECHODB-689 2. opt error msg for source path of view is deleted https://jira.infra.timecho.com:8443/browse/TIMECHODB-693 3. delete useless log file -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Pipe: Added check for wal disable and ratis consensus when including insertNode in realtime mode in data transmission [iotdb]
Caideyipi opened a new pull request, #12407: URL: https://github.com/apache/iotdb/pull/12407 ## Description As the title said. This PR has: - [ ] been self-reviewed. - [ ] concurrent read - [ ] concurrent write - [ ] concurrent read and write - [ ] added documentation for new or modified features or behaviors. - [ ] added Javadocs for most classes and all non-trivial methods. - [ ] added or updated version, __license__, or notice information - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage. - [ ] added integration tests. - [ ] been tested in a test IoTDB cluster. # Key changed/added classes (or packages if there are too many classes) in this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: Two stage aggregate counter [iotdb]
sonarcloud[bot] commented on PR #12328: URL: https://github.com/apache/iotdb/pull/12328#issuecomment-2074631361 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) **Quality Gate failed** Failed conditions  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12328=new_coverage=list) (required ≥ 80%)  [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) (required ≥ A) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) ##  Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: Fixed FileNotFoundException caused by processing files deleted by compaction & Changed the "isEmpty()" judgement to avoid "getDevice()" attempts to read from a deleted file when using f
SteveYurongSu merged PR #12405: URL: https://github.com/apache/iotdb/pull/12405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix IoTDBTimezoneIT [iotdb]
JackieTien97 merged PR #12401: URL: https://github.com/apache/iotdb/pull/12401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix partition cache after region migration [iotdb]
OneSizeFitsQuorum merged PR #12404: URL: https://github.com/apache/iotdb/pull/12404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fixed rest service insertTablet lost data event return SUCCESS_STATUS [iotdb]
HTHou merged PR #12387: URL: https://github.com/apache/iotdb/pull/12387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Changed the ' [iotdb]
Caideyipi opened a new pull request, #12405: URL: https://github.com/apache/iotdb/pull/12405 ## Description ### Content1 ... ### Content2 ... ### Content3 ... This PR has: - [ ] been self-reviewed. - [ ] concurrent read - [ ] concurrent write - [ ] concurrent read and write - [ ] added documentation for new or modified features or behaviors. - [ ] added Javadocs for most classes and all non-trivial methods. - [ ] added or updated version, __license__, or notice information - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage. - [ ] added integration tests. - [ ] been tested in a test IoTDB cluster. # Key changed/added classes (or packages if there are too many classes) in this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fixed concurrency issues with RegionMigration thread pool [iotdb]
OneSizeFitsQuorum merged PR #12403: URL: https://github.com/apache/iotdb/pull/12403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fixed concurrency issues with RegionMigration thread pool [iotdb]
codecov[bot] commented on PR #12403: URL: https://github.com/apache/iotdb/pull/12403#issuecomment-2074323504 ## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/12403?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report Attention: Patch coverage is `0%` with `1 lines` in your changes are missing coverage. Please review. > Project coverage is 43.09%. Comparing base [(`1ce494b`)](https://app.codecov.io/gh/apache/iotdb/commit/1ce494b5f175f3c94f83bfdecd21ef8c33fa6c80?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`8f5df55`)](https://app.codecov.io/gh/apache/iotdb/pull/12403?dropdown=coverage=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 3 commits behind head on master. | [Files](https://app.codecov.io/gh/apache/iotdb/pull/12403?dropdown=coverage=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Patch % | Lines | |---|---|---| | [.../apache/iotdb/db/service/RegionMigrateService.java](https://app.codecov.io/gh/apache/iotdb/pull/12403?src=pr=tree=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fservice%2FRegionMigrateService.java_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc2VydmljZS9SZWdpb25NaWdyYXRlU2VydmljZS5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/12403?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #12403 +/- ## - Coverage 43.12% 43.09% -0.04% Complexity 69 69 Files 3084 3085 +1 Lines190299 190400 +101 Branches 2267122680 +9 - Hits 8206082046 -14 - Misses 108239 108354 +115 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/12403?dropdown=coverage=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: Two stage aggregate counter [iotdb]
sonarcloud[bot] commented on PR #12328: URL: https://github.com/apache/iotdb/pull/12328#issuecomment-2074309063 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) **Quality Gate failed** Failed conditions  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12328=new_coverage=list) (required ≥ 80%)  [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) (required ≥ A) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12328) ##  Catch issues before they fail your Quality Gate with our IDE extension  [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Try to fix timezone [iotdb]
sonarcloud[bot] commented on PR #12401: URL: https://github.com/apache/iotdb/pull/12401#issuecomment-2074257917 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12401) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb=12401=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_iotdb=12401=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb=12401=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12401) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fixed concurrency issues with RegionMigration thread pool [iotdb]
sonarcloud[bot] commented on PR #12403: URL: https://github.com/apache/iotdb/pull/12403#issuecomment-2074256098 ## [](https://sonarcloud.io/dashboard?id=apache_iotdb=12403) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb=12403=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_iotdb=12403=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb=12403=false=true)  [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12403=new_coverage=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb=12403=new_duplicated_lines_density=list) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb=12403) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Pipe: refine procedure timeout message [iotdb]
DanielWang2035 opened a new pull request, #12402: URL: https://github.com/apache/iotdb/pull/12402 As title -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Pipe: distinguish between old and new process subtasks before and after stuck restart to avoid unexpected hashmap override and removal [iotdb]
SteveYurongSu merged PR #12400: URL: https://github.com/apache/iotdb/pull/12400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Implementation of device TTL [iotdb]
choubenson commented on PR #12122: URL: https://github.com/apache/iotdb/pull/12122#issuecomment-2074191685 > meta sync support The TTL synchronization between clusters is a relatively independent feature, which will be implemented in a subsequent new PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Implementation of device TTL [iotdb]
choubenson commented on code in PR #12122:
URL: https://github.com/apache/iotdb/pull/12122#discussion_r1577350238
##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/task/InnerSpaceCompactionTask.java:
##
@@ -483,6 +476,13 @@ private void releaseAllLocks() {
@Override
public long getEstimatedMemoryCost() {
+if (innerSpaceEstimator == null) {
+ if (this.performer instanceof ReadChunkCompactionPerformer) {
+innerSpaceEstimator = new ReadChunkInnerCompactionEstimator();
+ } else if (this.performer instanceof FastCompactionPerformer) {
+innerSpaceEstimator = new FastCompactionInnerCompactionEstimator();
+ }
Review Comment:
Because the estimator is not simply determined by the `performer`, but also
depends on the compaction `task type`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Try to fix timezone [iotdb]
codecov[bot] commented on PR #12401: URL: https://github.com/apache/iotdb/pull/12401#issuecomment-2074174843 ## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/12401?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 43.10%. Comparing base [(`638be8a`)](https://app.codecov.io/gh/apache/iotdb/commit/638be8ae212ae7fff87a34d391a946705cf5749a?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`0310c5d`)](https://app.codecov.io/gh/apache/iotdb/pull/12401?dropdown=coverage=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). Additional details and impacted files ```diff @@Coverage Diff@@ ## master #12401 +/- ## = Coverage 43.09% 43.10% Complexity 69 69 = Files 3085 3085 Lines190398 190398 Branches 2267822678 = + Hits 8204582062 +17 + Misses 108353 108336 -17 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/12401?dropdown=coverage=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Implementation of device TTL [iotdb]
choubenson commented on code in PR #12122:
URL: https://github.com/apache/iotdb/pull/12122#discussion_r1577328532
##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/MultiTsFileDeviceIterator.java:
##
@@ -578,7 +583,27 @@ public String nextSeries() {
List modificationsInThisResource =
modificationCache.computeIfAbsent(
resource,
- r -> new
LinkedList<>(ModificationFile.getNormalMods(r).getModifications()));
+ r -> {
+List list =
+new
LinkedList<>(ModificationFile.getNormalMods(r).getModifications());
+// add outdated device mods by ttl
+for (IDeviceID device : r.getDevices()) {
+ long timeLowerBound =
+ CommonDateTimeUtils.currentTime()
+ - DataNodeTTLCache.getInstance()
+ .getTTL(((PlainDeviceID)
device).toStringID());
Review Comment:
Resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Implementation of device TTL [iotdb]
choubenson commented on code in PR #12122:
URL: https://github.com/apache/iotdb/pull/12122#discussion_r1577328114
##
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/impl/SettleSelectorImpl.java:
##
@@ -0,0 +1,328 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl;
+
+import org.apache.iotdb.commons.conf.IoTDBConstant;
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.commons.utils.CommonDateTimeUtils;
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import
org.apache.iotdb.db.queryengine.plan.analyze.cache.schema.DataNodeTTLCache;
+import
org.apache.iotdb.db.storageengine.dataregion.compaction.execute.performer.ICompactionPerformer;
+import
org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.SettleCompactionTask;
+import
org.apache.iotdb.db.storageengine.dataregion.compaction.execute.utils.CompactionUtils;
+import
org.apache.iotdb.db.storageengine.dataregion.compaction.selector.ISettleSelector;
+import org.apache.iotdb.db.storageengine.dataregion.modification.Deletion;
+import org.apache.iotdb.db.storageengine.dataregion.modification.Modification;
+import
org.apache.iotdb.db.storageengine.dataregion.modification.ModificationFile;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileManager;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import
org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResourceStatus;
+import
org.apache.iotdb.db.storageengine.dataregion.tsfile.timeindex.DeviceTimeIndex;
+import
org.apache.iotdb.db.storageengine.dataregion.tsfile.timeindex.ITimeIndex;
+import org.apache.iotdb.tsfile.file.metadata.IDeviceID;
+import org.apache.iotdb.tsfile.file.metadata.PlainDeviceID;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import static
org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl.SettleSelectorImpl.DirtyStatus.PARTIAL_DELETED;
+
+public class SettleSelectorImpl implements ISettleSelector {
+ private static final Logger LOGGER =
+ LoggerFactory.getLogger(IoTDBConstant.COMPACTION_LOGGER_NAME);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+
+ private final boolean heavySelect;
+ private final String storageGroupName;
+ private final String dataRegionId;
+ private final long timePartition;
+ private final TsFileManager tsFileManager;
+ private boolean isSeq;
+
+ public SettleSelectorImpl(
+ boolean heavySelect,
+ String storageGroupName,
+ String dataRegionId,
+ long timePartition,
+ TsFileManager tsFileManager) {
+this.heavySelect = heavySelect;
+this.storageGroupName = storageGroupName;
+this.dataRegionId = dataRegionId;
+this.timePartition = timePartition;
+this.tsFileManager = tsFileManager;
+ }
+
+ static class AllDirtyResource {
+List resources = new ArrayList<>();
+
+public void add(TsFileResource resource) {
+ resources.add(resource);
+}
+
+public List getResources() {
+ return resources;
+}
+ }
+
+ static class PartialDirtyResource {
+List resources = new ArrayList<>();
+long totalFileSize = 0;
+
+public boolean add(TsFileResource resource, long dirtyDataSize) {
+ resources.add(resource);
+ totalFileSize += resource.getTsFileSize();
+ totalFileSize -= dirtyDataSize;
+ return checkHasReachedThreshold();
+}
+
+public List getResources() {
+ return resources;
+}
+
+public boolean checkHasReachedThreshold() {
+ return resources.size() >= config.getFileLimitPerInnerTask()
+ || totalFileSize >= config.getTargetCompactionFileSize();
+}
+ }
+
+ @Override
+ public List selectSettleTask(List
tsFileResources) {
+if
Re: [PR] Implementation of device TTL [iotdb]
choubenson commented on code in PR #12122: URL: https://github.com/apache/iotdb/pull/12122#discussion_r1577311884 ## iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/schema/source/DeviceSchemaSource.java: ## @@ -98,6 +100,8 @@ public void transformToTsBlockColumns( .getColumnBuilder(0) .writeBinary(new Binary(device.getFullPath(), TSFileConfig.STRING_CHARSET)); int templateId = device.getTemplateId(); +long ttl = DataNodeTTLCache.getInstance().getTTL(device.getFullPath()); +String ttlStr = ttl == Long.MAX_VALUE ? IoTDBConstant.TTL_INFINITE : String.valueOf(ttl); Review Comment: Nice comment! The optimization can be implemented in a new pull request later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
