[PR] Test JDK-21 as OAP runtime. [skywalking]
wu-sheng opened a new pull request, #11828: URL: https://github.com/apache/skywalking/pull/11828 - [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #. - [ ] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking) branch jdk21-runtime created (now f482f163ed)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch jdk21-runtime in repository https://gitbox.apache.org/repos/asf/skywalking.git at f482f163ed Test JDK-21 as OAP runtime. This branch includes the following new commits: new f482f163ed Test JDK-21 as OAP runtime. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(skywalking) 01/01: Test JDK-21 as OAP runtime.
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch jdk21-runtime in repository https://gitbox.apache.org/repos/asf/skywalking.git commit f482f163ed912b5b968f920b363f6d42d8903f0c Author: Wu Sheng AuthorDate: Wed Jan 24 15:43:33 2024 +0800 Test JDK-21 as OAP runtime. --- .github/workflows/skywalking.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/skywalking.yaml b/.github/workflows/skywalking.yaml index 488588ac14..3e08f59b0e 100644 --- a/.github/workflows/skywalking.yaml +++ b/.github/workflows/skywalking.yaml @@ -196,7 +196,7 @@ jobs: timeout-minutes: 30 strategy: matrix: -java-version: [11, 17] +java-version: [11, 17, 21] steps: - uses: actions/checkout@v3 with: @@ -791,7 +791,7 @@ jobs: strategy: fail-fast: false matrix: -java-version: [11, 17] +java-version: [11, 17, 21] steps: - uses: actions/checkout@v3 with:
[PR] feat: add the not found page [skywalking-booster-ui]
Fine0830 opened a new pull request, #368: URL: https://github.com/apache/skywalking-booster-ui/pull/368 Screenshot https://github.com/apache/skywalking-booster-ui/assets/20871783/04f0a149-976f-4fde-97b5-093df680d53d;> Signed-off-by: Qiuxia Fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
hanahmily commented on code in PR #381: URL: https://github.com/apache/skywalking-banyandb/pull/381#discussion_r1464399046 ## banyand/stream/part.go: ## @@ -325,8 +325,7 @@ func (mp *memPart) mustInitFromElements(es *elements) { sidPrev = sid } - if uncompressedBlockSizeBytes >= maxUncompressedBlockSize || - (i-indexPrev) > maxBlockLength || sid != sidPrev { + if uncompressedBlockSizeBytes >= maxUncompressedBlockSize || sid != sidPrev { Review Comment: You should remove "indexPrev" as well ## banyand/stream/merger.go: ## @@ -300,20 +300,14 @@ func mergeBlocks(closeCh <-chan struct{}, bw *blockWriter, br *blockReader) (*pa tmpBlock.bm.seriesID = b.bm.seriesID br.loadBlockData(getDecoder()) mergeTwoBlocks(tmpBlock, pendingBlock, b) - if len(tmpBlock.timestamps) <= maxBlockLength && tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize { + if tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize { if len(tmpBlock.timestamps) == 0 { pendingBlockIsEmpty = true } pendingBlock, tmpBlock = tmpBlock, pendingBlock continue } - if len(tmpBlock.timestamps) <= maxBlockLength { - bw.mustWriteBlock(tmpBlock.bm.seriesID, ) - releaseDecoder() - continue - } - tmpBlock.idx = maxBlockLength Review Comment: You can't simply remove it. When the tmpBlock's size is larger than maxUncompressedBlockSize, it should be written. The rest of the pieces in the loop will be unnecessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch main updated: Fix boundary issue in findRange (#380)
This is an automated email from the ASF dual-hosted git repository. hanahmily pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git The following commit(s) were added to refs/heads/main by this push: new 9ead45d7 Fix boundary issue in findRange (#380) 9ead45d7 is described below commit 9ead45d7446a08f5a051ab5b92bb227fe0312fe8 Author: Huang Youliang <52878305+butterbri...@users.noreply.github.com> AuthorDate: Wed Jan 24 15:05:05 2024 +0800 Fix boundary issue in findRange (#380) * Fix boundary issue in findrange * Remove unnecessary query fields - Co-authored-by: 吴晟 Wu Sheng --- banyand/measure/block.go | 46 ++--- banyand/measure/block_test.go | 85 --- banyand/stream/block.go | 56 +--- banyand/stream/block_test.go | 93 - banyand/stream/query.go | 4 -- pkg/timestamp/range.go| 28 ++ pkg/timestamp/range_test.go | 116 ++ 7 files changed, 161 insertions(+), 267 deletions(-) diff --git a/banyand/measure/block.go b/banyand/measure/block.go index 0bdcdea4..707a1ae4 100644 --- a/banyand/measure/block.go +++ b/banyand/measure/block.go @@ -27,6 +27,7 @@ import ( "github.com/apache/skywalking-banyandb/pkg/fs" "github.com/apache/skywalking-banyandb/pkg/logger" pbv1 "github.com/apache/skywalking-banyandb/pkg/pb/v1" + "github.com/apache/skywalking-banyandb/pkg/timestamp" ) type block struct { @@ -542,11 +543,11 @@ func (bc *blockCursor) loadData(tmpBlock *block) bool { bc.bm.tagFamilies = tf tmpBlock.mustReadFrom(, bc.p, bc.bm) - start, end, ok := findRange(tmpBlock.timestamps, bc.minTimestamp, bc.maxTimestamp) + start, end, ok := timestamp.FindRange(tmpBlock.timestamps, bc.minTimestamp, bc.maxTimestamp) if !ok { return false } - bc.timestamps = append(bc.timestamps, tmpBlock.timestamps[start:end]...) + bc.timestamps = append(bc.timestamps, tmpBlock.timestamps[start:end+1]...) for _, cf := range tmpBlock.tagFamilies { tf := columnFamily{ @@ -563,7 +564,7 @@ func (bc *blockCursor) loadData(tmpBlock *block) bool { if len(cf.columns[i].values) != len(tmpBlock.timestamps) { logger.Panicf("unexpected number of values for tags %q: got %d; want %d", cf.columns[i].name, len(cf.columns[i].values), len(tmpBlock.timestamps)) } - column.values = append(column.values, cf.columns[i].values[start:end]...) + column.values = append(column.values, cf.columns[i].values[start:end+1]...) tf.columns = append(tf.columns, column) } bc.tagFamilies = append(bc.tagFamilies, tf) @@ -582,49 +583,12 @@ func (bc *blockCursor) loadData(tmpBlock *block) bool { valueType: tmpBlock.field.columns[i].valueType, } - c.values = append(c.values, tmpBlock.field.columns[i].values[start:end]...) + c.values = append(c.values, tmpBlock.field.columns[i].values[start:end+1]...) bc.fields.columns = append(bc.fields.columns, c) } return true } -func findRange(timestamps []int64, min int64, max int64) (int, int, bool) { - l := len(timestamps) - start, end := -1, -1 - - for i := 0; i < l; i++ { - if timestamps[i] > max || timestamps[l-i-1] < min { - break - } - if timestamps[i] >= min && start == -1 { - start = i - } - if timestamps[l-i-1] <= max && end == -1 { - end = l - i - } - if start != -1 && end != -1 { - break - } - } - - if start == -1 && end == -1 { - return 0, 0, false - } - - if start == -1 { - start = 0 - } - - if end == -1 { - end = l - } - - if start >= end { - return 0, 0, false - } - return start, end, true -} - var blockCursorPool sync.Pool func generateBlockCursor() *blockCursor { diff --git a/banyand/measure/block_test.go b/banyand/measure/block_test.go index 2431b735..655960dc 100644 --- a/banyand/measure/block_test.go +++ b/banyand/measure/block_test.go @@ -417,91 +417,6 @@ func Test_marshalAndUnmarshalBlock(t *testing.T) { } } -func Test_findRange(t *testing.T) { - type args struct { - timestamps []int64 - minint64 - maxint64 - } - tests := []struct { - name string - args args -
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
codecov-commenter commented on PR #381: URL: https://github.com/apache/skywalking-banyandb/pull/381#issuecomment-1907489247 ## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Comparison is base [(`97e9111`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/97e911121c4f9ec7ad07ac1580f50b39a03de644?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.31% compared to head [(`98e64d8`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.34%. > Report is 1 commits behind head on main. Additional details and impacted files ```diff @@Coverage Diff @@ ## main #381 +/- ## == + Coverage 47.31% 47.34% +0.02% == Files 172 172 Lines 2129621290 -6 == + Hits1007710079 +2 + Misses 1037710370 -7 + Partials 842 841 -1 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix boundary issue in findRange [skywalking-banyandb]
hanahmily merged PR #380: URL: https://github.com/apache/skywalking-banyandb/pull/380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking) branch master updated: Add hierarchy relations auto-matching for MYSQL/POSTGRESQL/SO11Y_OAP/VIRTUAL_DATABASE. Fix MQE in dashboards when using `Card widget`. (#11818)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/skywalking.git The following commit(s) were added to refs/heads/master by this push: new 2f4f4a1275 Add hierarchy relations auto-matching for MYSQL/POSTGRESQL/SO11Y_OAP/VIRTUAL_DATABASE. Fix MQE in dashboards when using `Card widget`. (#11818) 2f4f4a1275 is described below commit 2f4f4a12756eeebe9fed32fff2186edc35fa6144 Author: Wan Kai AuthorDate: Wed Jan 24 15:03:45 2024 +0800 Add hierarchy relations auto-matching for MYSQL/POSTGRESQL/SO11Y_OAP/VIRTUAL_DATABASE. Fix MQE in dashboards when using `Card widget`. (#11818) --- docs/en/changes/changes.md | 13 ++- .../service-hierarchy-configuration.md | 22 - docs/en/concepts-and-designs/service-hierarchy.md | 97 -- .../server/core/hierarchy/HierarchyService.java| 57 + .../src/main/resources/hierarchy-definition.yml| 20 - .../apisix/apisix-instance.json| 4 +- .../aws_s3/aws_s3-service.json | 8 +- .../elasticsearch/elasticsearch-cluster.json | 50 ++- .../elasticsearch/elasticsearch-index.json | 50 +++ .../elasticsearch/elasticsearch-node.json | 51 +--- .../ui-initialized-templates/k8s/k8s-cluster.json | 32 +++ .../mongodb/mongodb-cluster.json | 10 +-- .../mongodb/mongodb-node.json | 10 +-- .../mysql/mysql-instance.json | 65 +++ .../mysql/mysql-service.json | 32 ++- .../os_linux/linux-service.json| 10 +-- .../os_windows/windows-service.json| 6 +- .../postgresql/postgresql-instance.json| 96 - .../postgresql/postgresql-root.json| 5 +- .../postgresql/postgresql-service.json | 30 ++- .../rabbitmq/rabbitmq-node.json| 60 ++--- .../redis/redis-instance.json | 18 ++-- .../redis/redis-service.json | 8 +- .../so11y_oap/so11y-instance.json | 17 +++- .../so11y_oap/so11y-service.json | 3 +- .../virtual_database/virtual-database-service.json | 23 - .../e2e-v2/cases/kafka/kafka-monitoring/Dockerfile | 2 +- .../kafka/kafka-monitoring/docker-compose.yml | 6 +- 28 files changed, 585 insertions(+), 220 deletions(-) diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index bd70c2e44d..b1eed59df4 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -15,7 +15,11 @@ * Support build Service/Instance Hierarchy and query. * Change the string field in Elasticsearch storage from **keyword** type to **text** type if it set more than `32766` length. * [Break Change] Change the configuration field of `ui_template` and `ui_menu` in Elasticsearch storage from **keyword** type to **text**. -* Support Service Hierarchy auto matching. +* Support Service Hierarchy auto matching, add auto matching layer relationships (upper -> lower) as following: + - MESH -> MESH_DP + - MESH -> K8S_SERVICE + - MESH_DP -> K8S_SERVICE + - GENERAL -> K8S_SERVICE * Add `namespace` suffix for `K8S_SERVICE_NAME_RULE/ISTIO_SERVICE_NAME_RULE` and `metadata-service-mapping.yaml` as default. * Allow using a dedicated port for ALS receiver. * Fix log query by traceId in `JDBCLogQueryDAO`. @@ -24,6 +28,12 @@ * Remove unnecessary annotations and functions from Meter Functions. * Add `max` and `min` functions for MAL down sampling. * Fix critical bug of uncontrolled memory cost of TopN statistics. Change topN group key from `StorageId` to `entityId + timeBucket`. +* Add Service Hierarchy auto matching layer relationships (upper -> lower) as following: + - MYSQL -> K8S_SERVICE + - POSTGRESQL -> K8S_SERVICE + - SO11Y_OAP -> K8S_SERVICE + - VIRTUAL_DATABASE -> MYSQL + - VIRTUAL_DATABASE -> POSTGRESQL UI @@ -40,6 +50,7 @@ * Update Kubernetes related UI templates for adapt data from eBPF access log. * Fix dashboard `K8S-Service-Root` metrics expression. * Add dashboards for Service/Instance Hierarchy. +* Fix MQE in dashboards when using `Card widget`. Documentation diff --git a/docs/en/concepts-and-designs/service-hierarchy-configuration.md b/docs/en/concepts-and-designs/service-hierarchy-configuration.md index 9080d663e8..3daef37358 100644 --- a/docs/en/concepts-and-designs/service-hierarchy-configuration.md +++ b/docs/en/concepts-and-designs/service-hierarchy-configuration.md @@ -19,26 +19,39 @@ hierarchy: K8S_SERVICE: lower-short-name-remove-ns MYSQL: -K8S_SERVICE: ~ +K8S_SERVICE: short-name + + POSTGRESQL: +K8S_SERVICE: short-name + + SO11Y_OAP: +K8S_SERVICE: short-name VIRTUAL_DATABASE: -MYSQL: ~ +MYSQL:
Re: [PR] Add hierarchy relations auto-matching for MYSQL/POSTGRESQL/SO11Y_OAP/VIRTUAL_DATABASE. Fix MQE in dashboards when using `Card widget`. [skywalking]
wu-sheng merged PR #11818: URL: https://github.com/apache/skywalking/pull/11818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Remove maxBlockLength in stream [skywalking-banyandb]
ButterBright opened a new pull request, #381: URL: https://github.com/apache/skywalking-banyandb/pull/381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix stream startup warning [skywalking-banyandb]
wu-sheng commented on PR #378: URL: https://github.com/apache/skywalking-banyandb/pull/378#issuecomment-1907477063 CI fails. Please recheck. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix boundary issue in findRange [skywalking-banyandb]
codecov-commenter commented on PR #380: URL: https://github.com/apache/skywalking-banyandb/pull/380#issuecomment-1907463691 ## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report Attention: `4 lines` in your changes are missing coverage. Please review. > Comparison is base [(`97e9111`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/97e911121c4f9ec7ad07ac1580f50b39a03de644?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.31% compared to head [(`270d424`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.32%. | [Files](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Patch % | Lines | |---|---|---| | [pkg/timestamp/range.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-cGtnL3RpbWVzdGFtcC9yYW5nZS5nbw==) | 85.00% | [2 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/block.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vYmxvY2suZ28=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #380 +/- ## === Coverage 47.31% 47.32% === Files 172 172 Lines 2129621262 -34 === - Hits1007710062 -15 + Misses 1037710362 -15 + Partials 842 838-4 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/380?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Fix boundary issue in findRange [skywalking-banyandb]
ButterBright opened a new pull request, #380: URL: https://github.com/apache/skywalking-banyandb/pull/380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-banyandb) branch main updated: Introduce merge algorithm for stream (#379)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb.git The following commit(s) were added to refs/heads/main by this push: new 97e91112 Introduce merge algorithm for stream (#379) 97e91112 is described below commit 97e911121c4f9ec7ad07ac1580f50b39a03de644 Author: Huang Youliang <52878305+butterbri...@users.noreply.github.com> AuthorDate: Wed Jan 24 14:06:43 2024 +0800 Introduce merge algorithm for stream (#379) --- banyand/stream/merger.go| 40 +++- banyand/stream/merger_policy.go | 136 banyand/stream/service.go | 3 + banyand/stream/stream.go| 1 + banyand/stream/tstable.go | 2 +- banyand/stream/tstable_test.go | 16 - 6 files changed, 180 insertions(+), 18 deletions(-) diff --git a/banyand/stream/merger.go b/banyand/stream/merger.go index e2866b95..44013bb3 100644 --- a/banyand/stream/merger.go +++ b/banyand/stream/merger.go @@ -40,6 +40,8 @@ func (tst *tsTable) mergeLoop(merges chan *mergerIntroduction, flusherNotifier w return } + var pwsChunk []*partWrapper + for { select { case <-tst.loopCloser.CloseNotify(): @@ -50,7 +52,8 @@ func (tst *tsTable) mergeLoop(merges chan *mergerIntroduction, flusherNotifier w continue } if curSnapshot.epoch != epoch { - if err := tst.mergeSnapshot(curSnapshot, merges); err != nil { + var err error + if pwsChunk, err = tst.mergeSnapshot(curSnapshot, merges, pwsChunk[:0]); err != nil { if errors.Is(err, errClosed) { curSnapshot.decRef() return @@ -70,17 +73,18 @@ func (tst *tsTable) mergeLoop(merges chan *mergerIntroduction, flusherNotifier w } } -func (tst *tsTable) mergeSnapshot(curSnapshot *snapshot, merges chan *mergerIntroduction) error { - maxFanOut := tst.freeDiskSpace(tst.root) / 2 - partsToMerge, toBeMerged := tst.getPartsToMerge(curSnapshot, maxFanOut) - if len(partsToMerge) < 2 { - return nil +func (tst *tsTable) mergeSnapshot(curSnapshot *snapshot, merges chan *mergerIntroduction, dst []*partWrapper) ([]*partWrapper, error) { + freeDiskSize := tst.freeDiskSpace(tst.root) + var toBeMerged map[uint64]struct{} + dst, toBeMerged = tst.getPartsToMerge(curSnapshot, freeDiskSize, dst) + if len(dst) < 2 { + return nil, nil } - if _, err := tst.mergePartsThenSendIntroduction(snapshotCreatorMerger, partsToMerge, + if _, err := tst.mergePartsThenSendIntroduction(snapshotCreatorMerger, dst, toBeMerged, merges, tst.loopCloser.CloseNotify()); err != nil { - return err + return dst, err } - return nil + return dst, nil } func (tst *tsTable) mergePartsThenSendIntroduction(creator snapshotCreator, parts []*partWrapper, merged map[uint64]struct{}, merges chan *mergerIntroduction, @@ -177,18 +181,26 @@ func releaseDiskSpace(n uint64) { var reservedDiskSpace uint64 -func (tst *tsTable) getPartsToMerge(snapshot *snapshot, maxFanOut uint64) ([]*partWrapper, map[uint64]struct{}) { +func (tst *tsTable) getPartsToMerge(snapshot *snapshot, freeDiskSize uint64, dst []*partWrapper) ([]*partWrapper, map[uint64]struct{}) { var parts []*partWrapper - toBeMerged := make(map[uint64]struct{}) + for _, pw := range snapshot.parts { - if pw.mp != nil || pw.p.partMetadata.TotalCount < 1 || pw.p.partMetadata.CompressedSizeBytes > maxFanOut { + if pw.mp != nil || pw.p.partMetadata.TotalCount < 1 { continue } parts = append(parts, pw) + } + + dst = tst.option.mergePolicy.getPartsToMerge(dst, parts, freeDiskSize) + if len(dst) == 0 { + return nil, nil + } + + toBeMerged := make(map[uint64]struct{}) + for _, pw := range dst { toBeMerged[pw.ID()] = struct{}{} } - // TODO: select proper parts to generate the lowest write amplification - return parts, toBeMerged + return dst, toBeMerged } func (tst *tsTable) reserveSpace(parts []*partWrapper) uint64 { diff --git a/banyand/stream/merger_policy.go b/banyand/stream/merger_policy.go new file mode 100644 index ..4a284a4b --- /dev/null +++ b/banyand/stream/merger_policy.go @@ -0,0 +1,136 @@ +// Licensed to Apache Software Foundation (ASF) under one or more contributor +// license agreements. See the NOTICE file distributed with +// this work for additional information
Re: [PR] Introduce merge algorithm for stream [skywalking-banyandb]
wu-sheng merged PR #379: URL: https://github.com/apache/skywalking-banyandb/pull/379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Introduce merge algorithm for stream [skywalking-banyandb]
codecov-commenter commented on PR #379: URL: https://github.com/apache/skywalking-banyandb/pull/379#issuecomment-1907429888 ## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report Attention: `16 lines` in your changes are missing coverage. Please review. > Comparison is base [(`93408dc`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/93408dc3ddf8e5039d1cf6a9c5cb0764dfa87b5c?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.10% compared to head [(`be8a828`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.33%. | [Files](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Patch % | Lines | |---|---|---| | [banyand/stream/merger\_policy.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vbWVyZ2VyX3BvbGljeS5nbw==) | 82.27% | [9 Missing and 5 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/merger.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vbWVyZ2VyLmdv) | 91.30% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Additional details and impacted files ```diff @@Coverage Diff @@ ## main #379 +/- ## == + Coverage 47.10% 47.33% +0.22% == Files 171 172 +1 Lines 2120421296 +92 == + Hits 998810080 +92 + Misses 1037910375 -4 - Partials 837 841 +4 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/379?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix stream startup warning [skywalking-banyandb]
hanahmily commented on code in PR #378: URL: https://github.com/apache/skywalking-banyandb/pull/378#discussion_r1464340894 ## banyand/stream/tstable.go: ## @@ -180,6 +180,9 @@ func newTSTable(fileSystem fs.FileSystem, rootPath string, p common.Position, var needToDelete []string for i := range ee { if ee[i].IsDir() { + if ee[i].Name() == "element_idx" { Review Comment: You should exclude directories that do not start with "part-". If any directories do not meet this criterion, the condition will not be satisfied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Introduce merge algorithm for stream [skywalking-banyandb]
ButterBright opened a new pull request, #379: URL: https://github.com/apache/skywalking-banyandb/pull/379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
wankai123 commented on code in PR #11758: URL: https://github.com/apache/skywalking/pull/11758#discussion_r1464293445 ## oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-topic.json: ## @@ -0,0 +1,299 @@ +[ + { +"id": "RocketMQ-Topic", +"configuration": { + "children": [ +{ + "x": 0, + "y": 36, + "w": 12, + "h": 15, + "i": "0", + "type": "Widget", + "id": "0", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "graph": { +"type": "Bar", +"showBackground": true + }, + "expressions": [ +"meter_rocketmq_topic_producer_message_size/1024/1024" + ], + "typesOfMQE": [ +"TIME_SERIES_VALUES" + ], + "widget": { +"title": "Producer Message Size(MB)", +"tips": "The size of messages produced per second", +"name": "producer_message_size" + } +}, +{ + "x": 12, + "y": 36, + "w": 12, + "h": 15, + "i": "1", + "type": "Widget", + "id": "1", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "graph": { +"type": "Bar", +"showBackground": true + }, + "expressions": [ +"meter_rocketmq_topic_consumer_message_size/1024/1024" + ], + "typesOfMQE": [ +"TIME_SERIES_VALUES" + ], + "widget": { +"name": "consumer_message_size", +"title": "Consumer Message Size(MB)", +"tips": "The size of messages consumed per second " + } +}, +{ + "x": 6, + "y": 0, + "w": 6, + "h": 10, + "i": "2", + "type": "Widget", + "id": "2", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "metricConfig": [ +{ + "unit": "MB" +} + ], + "moved": false, + "expressions": [ +"latest(meter_rocketmq_topic_max_consumer_message_size)/1024/1024" + ], + "typesOfMQE": [ +"SINGLE_VALUE" + ], + "widget": { +"name": "max_consumer_message_size", +"title": "Max Consumer Message Size", +"tips": "The maximum number of messages consumed" + }, + "graph": { +"type": "Card", +"fontSize": 40, +"textAlign": "center", +"showUnit": true + } +}, +{ + "x": 0, + "y": 0, + "w": 6, + "h": 10, + "i": "3", + "type": "Widget", + "id": "3", + "metricTypes": [], + "metrics": [], + "metricConfig": [ +{ + "unit": "MB" +} + ], + "metricMode": "Expression", + "moved": false, + "expressions": [ +"latest(meter_rocketmq_topic_max_producer_message_size)/1024/1024" + ], + "typesOfMQE": [ +"SINGLE_VALUE" + ], + "widget": { +"name": "max_producer_message_size", +"title": "Max Producer Message Size", +"tips": "The maximum number of messages produced" + }, + "graph": { +"type": "Card", +"fontSize": 40, +"textAlign": "center", +"showUnit": true + } +}, +{ + "x": 12, + "y": 0, + "w": 12, + "h": 10, + "i": "5", + "type": "Widget", + "id": "5", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "expressions": [ +"avg(meter_rocketmq_topic_consumer_latency)/1000" Review Comment: ```suggestion "meter_rocketmq_topic_consumer_latency/1000" ``` this expression needs to be restored too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
wankai123 commented on code in PR #11758: URL: https://github.com/apache/skywalking/pull/11758#discussion_r1464293445 ## oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-topic.json: ## @@ -0,0 +1,299 @@ +[ + { +"id": "RocketMQ-Topic", +"configuration": { + "children": [ +{ + "x": 0, + "y": 36, + "w": 12, + "h": 15, + "i": "0", + "type": "Widget", + "id": "0", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "graph": { +"type": "Bar", +"showBackground": true + }, + "expressions": [ +"meter_rocketmq_topic_producer_message_size/1024/1024" + ], + "typesOfMQE": [ +"TIME_SERIES_VALUES" + ], + "widget": { +"title": "Producer Message Size(MB)", +"tips": "The size of messages produced per second", +"name": "producer_message_size" + } +}, +{ + "x": 12, + "y": 36, + "w": 12, + "h": 15, + "i": "1", + "type": "Widget", + "id": "1", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "graph": { +"type": "Bar", +"showBackground": true + }, + "expressions": [ +"meter_rocketmq_topic_consumer_message_size/1024/1024" + ], + "typesOfMQE": [ +"TIME_SERIES_VALUES" + ], + "widget": { +"name": "consumer_message_size", +"title": "Consumer Message Size(MB)", +"tips": "The size of messages consumed per second " + } +}, +{ + "x": 6, + "y": 0, + "w": 6, + "h": 10, + "i": "2", + "type": "Widget", + "id": "2", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "metricConfig": [ +{ + "unit": "MB" +} + ], + "moved": false, + "expressions": [ +"latest(meter_rocketmq_topic_max_consumer_message_size)/1024/1024" + ], + "typesOfMQE": [ +"SINGLE_VALUE" + ], + "widget": { +"name": "max_consumer_message_size", +"title": "Max Consumer Message Size", +"tips": "The maximum number of messages consumed" + }, + "graph": { +"type": "Card", +"fontSize": 40, +"textAlign": "center", +"showUnit": true + } +}, +{ + "x": 0, + "y": 0, + "w": 6, + "h": 10, + "i": "3", + "type": "Widget", + "id": "3", + "metricTypes": [], + "metrics": [], + "metricConfig": [ +{ + "unit": "MB" +} + ], + "metricMode": "Expression", + "moved": false, + "expressions": [ +"latest(meter_rocketmq_topic_max_producer_message_size)/1024/1024" + ], + "typesOfMQE": [ +"SINGLE_VALUE" + ], + "widget": { +"name": "max_producer_message_size", +"title": "Max Producer Message Size", +"tips": "The maximum number of messages produced" + }, + "graph": { +"type": "Card", +"fontSize": 40, +"textAlign": "center", +"showUnit": true + } +}, +{ + "x": 12, + "y": 0, + "w": 12, + "h": 10, + "i": "5", + "type": "Widget", + "id": "5", + "metricTypes": [], + "metrics": [], + "metricMode": "Expression", + "moved": false, + "expressions": [ +"avg(meter_rocketmq_topic_consumer_latency)/1000" Review Comment: ```suggestion "meter_rocketmq_topic_consumer_latency/1000" ``` the expression needs to be restored too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
peachisai commented on PR #11758: URL: https://github.com/apache/skywalking/pull/11758#issuecomment-1907355234 > > Should I restore to `Time series` ? > > Oops, sorry. you are right, then should restore both expression and typesOfMQE for this Widget. > > Should I restore to `Time series` ? > > Oops, sorry. you are right, then should restore both expression and typesOfMQE for this Widget. Okay > > Should I restore to `Time series` ? > > Oops, sorry. you are right, then should restore both expression and typesOfMQE for this Widget. That's ok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
wankai123 commented on PR #11758: URL: https://github.com/apache/skywalking/pull/11758#issuecomment-1907309182 > Should I restore to `Time series` ? Oops, sorry. you are right, then should restore both expression and type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
peachisai commented on PR #11758: URL: https://github.com/apache/skywalking/pull/11758#issuecomment-1907305115 > > > The Card Widget requires SINGLE_VALUE. Please check others UI templates. > > > > > > May I ask a question? Why should meter_rocketmq_topic_consumer_latency be as a single value even though it is a line graph? > > Are you use a Card widget? If so, we should use MQE to control which value should be picked, rather than UI. Because UI will pick the first value, which may be days ago, or minutes away, depending on the UI query condition. Okay, I got it, But it is a line widget ![image](https://github.com/apache/skywalking/assets/96932958/e2e99bea-5dcc-410f-8ae9-0a6655f16f41) @wankai123 Should I restore to ```Time series``` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
wu-sheng commented on PR #11758: URL: https://github.com/apache/skywalking/pull/11758#issuecomment-1907299245 > > The Card Widget requires SINGLE_VALUE. Please check others UI templates. > > May I ask a question? Why should meter_rocketmq_topic_consumer_latency be as a single value even though it is a line graph? Are you use a Card widget? If so, we should use MQE to control which value should be picked, rather than UI. Because UI will pick the first value, which may be days ago, or minutes away, depending on the UI query condition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support rocketmq monitoring [skywalking]
peachisai commented on PR #11758: URL: https://github.com/apache/skywalking/pull/11758#issuecomment-1907297905 > The Card Widget requires SINGLE_VALUE. Please check others UI templates. May I ask a question? Why should meter_rocketmq_topic_consumer_latency be as a single value even though it is a line graph? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix stream startup warning [skywalking-banyandb]
codecov-commenter commented on PR #378: URL: https://github.com/apache/skywalking-banyandb/pull/378#issuecomment-1907286492 ## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/378?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Comparison is base [(`93408dc`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/93408dc3ddf8e5039d1cf6a9c5cb0764dfa87b5c?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.10% compared to head [(`70139d8`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/378?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.01%. Additional details and impacted files ```diff @@Coverage Diff @@ ## main #378 +/- ## == - Coverage 47.10% 47.01% -0.09% == Files 171 171 Lines 2120421206 +2 == - Hits 9988 9971 -17 - Misses 1037910396 +17 - Partials 837 839 +2 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/378?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
wankai123 commented on PR #11819: URL: https://github.com/apache/skywalking/pull/11819#issuecomment-1907258919 > `So11y` seems have an error verify result with the latest `OTEL collector` version, I am tring to fix it here change `logLevel` to: `loglevel` https://github.com/apache/skywalking/blob/a9128e0cefbcba08c085b70ed8795d7fb416459c/test/e2e-v2/cases/so11y/otel-collector-config.yaml#L36 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Fix stream startup warning [skywalking-banyandb]
ButterBright opened a new pull request, #378: URL: https://github.com/apache/skywalking-banyandb/pull/378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature] add consul/etcd discover in skywalking-go [skywalking]
wu-sheng commented on issue #11825: URL: https://github.com/apache/skywalking/issues/11825#issuecomment-1907232109 More importantly, you are going to introduce more libs, that isn't our plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature] add consul/etcd discover in skywalking-go [skywalking]
wu-sheng commented on issue #11825: URL: https://github.com/apache/skywalking/issues/11825#issuecomment-1907229786 Sorry, we would accept service discovery in agent. In today's cloud native issue, dns and mesh service are main stream. You could consider that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-go) branch main updated: Add support ignore suffix (#163)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-go.git The following commit(s) were added to refs/heads/main by this push: new ffaa359 Add support ignore suffix (#163) ffaa359 is described below commit ffaa35950cf1bf1943d00c196042ec16d55ae24b Author: Starry AuthorDate: Wed Jan 24 10:04:49 2024 +0800 Add support ignore suffix (#163) --- CHANGES.md| 1 + docs/en/agent/tracing-metrics-logging.md | 7 --- plugins/core/tracer.go| 5 - plugins/core/tracing.go | 25 +++ tools/go-agent/config/agent.default.yaml | 2 ++ tools/go-agent/config/loader.go | 1 + tools/go-agent/instrument/agentcore/instrument.go | 3 ++- 7 files changed, 35 insertions(+), 9 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index a6ef940..88dc93a 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -5,6 +5,7 @@ Release Notes. 0.4.0 -- Features +* Add support ignore suffix. Plugins * Support setting a discard type of reporter. diff --git a/docs/en/agent/tracing-metrics-logging.md b/docs/en/agent/tracing-metrics-logging.md index 4d540eb..07ccf8d 100644 --- a/docs/en/agent/tracing-metrics-logging.md +++ b/docs/en/agent/tracing-metrics-logging.md @@ -22,9 +22,10 @@ If you wish to disable a particular plugin to prevent enhancements related to th The basic configuration is as follows: -| Name| Environment Key | Default Value | Description | -|-|---|---|---| -| agent.sampler | SW_AGENT_SAMPLER | 1 | Sampling rate of tracing data, which is a floating-point value that must be between 0 and 1. | +| Name| Environment Key| Default Value | Description | +|-||--|--| +| agent.sampler | SW_AGENT_SAMPLER | 1 | Sampling rate of tracing data, which is a floating-point value that must be between 0 and 1. | +| agent.ignore_suffix | SW_AGENT_IGNORE_SUFFIX | .jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg | If the operation name of the first span is included in this set, this segment should be ignored.(multiple split by ","). | ## Metrics diff --git a/plugins/core/tracer.go b/plugins/core/tracer.go index a0b797c..e10a56b 100644 --- a/plugins/core/tracer.go +++ b/plugins/core/tracer.go @@ -22,6 +22,7 @@ import ( defLog "log" "os" "reflect" + "strings" "sync" "github.com/apache/skywalking-go/plugins/core/operator" @@ -52,10 +53,11 @@ type Tracer struct { // for all metrics meterMap *sync.Map meterCollectListeners []func() + ignoreSuffix []string } func (t *Tracer) Init(entity *reporter.Entity, rep reporter.Reporter, samp Sampler, logger operator.LogOperator, - meterCollectSecond int, correlation *CorrelationConfig) error { + meterCollectSecond int, correlation *CorrelationConfig, ignoreSuffixStr string) error { t.ServiceEntity = entity t.Reporter = rep t.Sampler = samp @@ -66,6 +68,7 @@ func (t *Tracer) Init(entity *reporter.Entity, rep reporter.Reporter, samp Sampl t.initFlag = 1 t.initMetricsCollect(meterCollectSecond) t.correlation = correlation + t.ignoreSuffix = strings.Split(ignoreSuffixStr, ",") // notify the tracer been init success if len(GetInitNotify()) > 0 { for _, fun := range GetInitNotify() { diff --git a/plugins/core/tracing.go b/plugins/core/tracing.go index fb7b132..8fac32f 100644 --- a/plugins/core/tracing.go +++ b/plugins/core/tracing.go @@ -20,6 +20,7 @@ package core import ( "reflect" "runtime/debug" + "strings" "github.com/pkg/errors" @@ -42,7 +43,7 @@ func (t *Tracer) DebugStack() []byte { } func (t *Tracer) CreateEntrySpan(operationName string, extractor interface{}, opts ...interface{}) (s interface{}, err error) { - ctx,
Re: [PR] Add support ignore suffix [skywalking-go]
wu-sheng merged PR #163: URL: https://github.com/apache/skywalking-go/pull/163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
peachisai commented on PR #11819: URL: https://github.com/apache/skywalking/pull/11819#issuecomment-1907219345 ```So11y``` seems have an error verify result with the latest ```OTEL collector``` version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Feature] add consul/etcd discover in skywalking-go [skywalking]
vearne opened a new issue, #11825: URL: https://github.com/apache/skywalking/issues/11825 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement. ### Description skywalking-go uses grpc to report monitoring information. When it is used in production, it should support discovery of skywalking-oap-server from consul or etcd ### Use case 1. skywalking-oap-server is registered in consul or etcd 2. The business app discovers the skywalking-oap-server instance list from consul or etcd 3. The business app requests the skywalking-oap-server instance through grpc and reports monitoring data. ### Related issues _No response_ ### Are you willing to submit a pull request to implement this on your own? - [X] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Add support ignore suffix [skywalking-go]
CodePrometheus opened a new pull request, #163: URL: https://github.com/apache/skywalking-go/pull/163 ### summary add support ignore suffix ### what Provide users with the ability to ignore the span for the operationName dimension. ### why In the actual usage environment, there are a batch of requests such as /xx.jpg. We can reduce the signal-to-noise ratio of the collection to better analyze the trace. ### how I added `agent.ignore_suffix` in the default configuration file, which will add a layer of judgment when creating Entry, Local and Exit spans. ### additional There is no test code added for the time being. Since I am considering the operationName dimension, I would like to discuss the topic [11807](https://github.com/apache/skywalking/discussions/11807) with community members~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add R3 to showcase [skywalking-showcase]
kezhenxu94 commented on PR #156: URL: https://github.com/apache/skywalking-showcase/pull/156#issuecomment-1906147464 In case anyone who is interested, you can find the URIs that are recognized by the algorithm model in demo, (service `mesh-svr::rating.sample-services`): http://demo.skywalking.apache.org/dashboard/MESH/Service/bWVzaC1zdnI6OnJhdGluZy5zYW1wbGUtc2VydmljZXM=.1/Mesh-Service -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature] [TSDB]Stream column storage [skywalking]
hanahmily closed issue #11474: [Feature] [TSDB]Stream column storage URL: https://github.com/apache/skywalking/issues/11474 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Stream column-based storage [skywalking-banyandb]
hanahmily merged PR #377: URL: https://github.com/apache/skywalking-banyandb/pull/377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-showcase) branch main updated: Add namespace suffix in ebpf profiler job
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git The following commit(s) were added to refs/heads/main by this push: new 1aa56b2 Add namespace suffix in ebpf profiler job 1aa56b2 is described below commit 1aa56b238ae5f34a15357fb447b8ffbb43397d88 Author: kezhenxu94 AuthorDate: Tue Jan 23 21:54:58 2024 +0800 Add namespace suffix in ebpf profiler job --- .../kubernetes/templates/feature-rover/profiling.yaml | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/deploy/platform/kubernetes/templates/feature-rover/profiling.yaml b/deploy/platform/kubernetes/templates/feature-rover/profiling.yaml index d01fbbb..33b07fc 100644 --- a/deploy/platform/kubernetes/templates/feature-rover/profiling.yaml +++ b/deploy/platform/kubernetes/templates/feature-rover/profiling.yaml @@ -60,22 +60,22 @@ spec: - -c - > /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create fixed \ ---service-name mesh-svr::recommendation --labels mesh-envoy --duration=9m --target-type ON_CPU; +--service-name mesh-svr::recommendation.{{ .Values.sampleServices.namespace }} --labels mesh-envoy --duration=9m --target-type ON_CPU; /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create fixed \ ---service-name mesh-svr::app --labels mesh-envoy --duration=9m --target-type ON_CPU; +--service-name mesh-svr::app.{{ .Values.sampleServices.namespace }} --labels mesh-envoy --duration=9m --target-type ON_CPU; /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create fixed \ ---service-name mesh-svr::songs --labels mesh-envoy --duration=9m --target-type ON_CPU; +--service-name mesh-svr::songs.{{ .Values.sampleServices.namespace }} --labels mesh-envoy --duration=9m --target-type ON_CPU; /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create fixed \ ---service-name mesh-svr::gateway --labels mesh-envoy --duration=9m --target-type ON_CPU; +--service-name mesh-svr::gateway.{{ .Values.sampleServices.namespace }} --labels mesh-envoy --duration=9m --target-type ON_CPU; /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create fixed \ ---service-name mesh-svr::songs --labels mesh-envoy --duration=9m --target-type OFF_CPU; +--service-name mesh-svr::songs.{{ .Values.sampleServices.namespace }} --labels mesh-envoy --duration=9m --target-type OFF_CPU; apk update && apk add yq; /swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql profiling ebpf create network \ --sampling-config=/profiling/network-profiling-sampling.yaml \ ---service-name=mesh-svr::recommendation \ ---instance-name=$(/swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql instance ls --service-name mesh-svr::recommendation|yq e '.[0].name' -); +--service-name=mesh-svr::recommendation.{{ .Values.sampleServices.namespace }} \ +--instance-name=$(/swctl --base-url={{ template "skywalking.oap.address.http" . }}/graphql instance ls --service-name mesh-svr::recommendation.{{ .Values.sampleServices.namespace }}|yq e '.[0].name' -); volumeMounts: - name: profiling-config mountPath: /profiling
(skywalking-showcase) branch endpointmodel deleted (was 4fc1f9a)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch endpointmodel in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git was 4fc1f9a Add R3 to showcase The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(skywalking-showcase) branch main updated: Add R3 to showcase (#156)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git The following commit(s) were added to refs/heads/main by this push: new bd44ad8 Add R3 to showcase (#156) bd44ad8 is described below commit bd44ad8803e242043c59f5d3d4d9b1435ec95a1f Author: kezhenxu94 AuthorDate: Tue Jan 23 21:37:26 2024 +0800 Add R3 to showcase (#156) --- Makefile.in| 3 + deploy/platform/kubernetes/Makefile| 8 ++- deploy/platform/kubernetes/Makefile.in | 2 +- .../kubernetes/templates/feature-r3/resources.yaml | 83 ++ deploy/platform/kubernetes/values.yaml | 8 +++ docs/readme.md | 1 + 6 files changed, 103 insertions(+), 2 deletions(-) diff --git a/Makefile.in b/Makefile.in index b5f8852..d01c419 100644 --- a/Makefile.in +++ b/Makefile.in @@ -56,5 +56,8 @@ GRAFANA_IMAGE ?= grafana/grafana:9.5.2 OTEL_COLLECTOR_IMAGE ?= otel/opentelemetry-collector OTEL_COLLECTOR_IMAGE_TAG ?= 0.72.0 +SW_R3_IMAGE ?= ghcr.io/skyapm/r3 +SW_R3_IMAGE_TAG ?= 0.1.0 + # ALL variables should be listed above ^^ EXPORTED_VARS := $(filter-out <%,$(.VARIABLES)) diff --git a/deploy/platform/kubernetes/Makefile b/deploy/platform/kubernetes/Makefile index 13f071f..275c94b 100644 --- a/deploy/platform/kubernetes/Makefile +++ b/deploy/platform/kubernetes/Makefile @@ -39,6 +39,7 @@ HELM_OPTIONS := $(HELM_OPTIONS) --set features.javaAgentInjector.agentImage=$(SW HELM_OPTIONS := $(HELM_OPTIONS) --set features.rover.image=$(SW_ROVER_IMAGE) HELM_OPTIONS := $(HELM_OPTIONS) --set features.grafana.image=$(GRAFANA_IMAGE) HELM_OPTIONS := $(HELM_OPTIONS) --set skywalking.grafana.plugin.version=$(SW_GRAFANA_PLUGIN_VERSION) +HELM_OPTIONS := $(HELM_OPTIONS) --set features.r3.image=$(SW_R3_IMAGE):$(SW_R3_IMAGE_TAG) ifeq ($(DEBUG),true) HELM_OPTIONS := $(HELM_OPTIONS) --dry-run --debug @@ -99,7 +100,7 @@ feature-istiod-monitor: .PHONY: feature-als feature-als: $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set features.als.enabled=true) - $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set istio.enabled=true) + $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set istio.enabled=true --set istiod.global.istioNamespace=$(NAMESPACE) --set base.global.istioNamespace=$(NAMESPACE)) $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set istiod.meshConfig.defaultConfig.tracing.zipkin.address=$(RELEASE)-oap.$(NAMESPACE).svc:9411) $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set istiod.meshConfig.defaultConfig.envoyMetricsService.address=$(BACKEND_SERVICE)) $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set istiod.meshConfig.defaultConfig.envoyAccessLogService.address=$(BACKEND_SERVICE)) @@ -171,6 +172,11 @@ feature-grafana: feature-mesh-with-agent: $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set features.meshWithAgent.enabled=true) +.PHONY: feature-r3 +feature-r3: + $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set features.r3.enabled=true) + $(eval HELM_OPTIONS := $(HELM_OPTIONS) --set skywalking.oap.env.SW_AI_PIPELINE_URI_RECOGNITION_SERVER_ADDR=r3) + .PHONY: deploy deploy: $(features) helm dep up . diff --git a/deploy/platform/kubernetes/Makefile.in b/deploy/platform/kubernetes/Makefile.in index e126b9e..5eb6bbe 100644 --- a/deploy/platform/kubernetes/Makefile.in +++ b/deploy/platform/kubernetes/Makefile.in @@ -23,4 +23,4 @@ SAMPLE_SERVICES_NAMESPACE ?= sample-services RELEASE ?= demo AGENTLESS ?= false -FEATURE_FLAGS ?= java-agent-injector,cluster,elasticsearch,kubernetes-monitor,so11y,vm-monitor,als,event,istiod-monitor,satellite,rover,trace-profiling,mysql-monitor,postgresql-monitor,nginx-monitor,apisix-monitor,grafana,elasticsearch-monitor,rabbitmq-monitor,mongodb-monitor +FEATURE_FLAGS ?= java-agent-injector,cluster,elasticsearch,kubernetes-monitor,so11y,vm-monitor,als,event,istiod-monitor,satellite,rover,trace-profiling,mysql-monitor,postgresql-monitor,nginx-monitor,apisix-monitor,grafana,elasticsearch-monitor,rabbitmq-monitor,mongodb-monitor,r3 diff --git a/deploy/platform/kubernetes/templates/feature-r3/resources.yaml b/deploy/platform/kubernetes/templates/feature-r3/resources.yaml new file mode 100644 index 000..8becfa2 --- /dev/null +++ b/deploy/platform/kubernetes/templates/feature-r3/resources.yaml @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or
Re: [PR] Add R3 to showcase [skywalking-showcase]
kezhenxu94 merged PR #156: URL: https://github.com/apache/skywalking-showcase/pull/156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-showcase) branch endpointmodel updated (072a824 -> 4fc1f9a)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch endpointmodel in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git discard 072a824 Add R3 to showcase add 4fc1f9a Add R3 to showcase This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (072a824) \ N -- N -- N refs/heads/endpointmodel (4fc1f9a) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: deploy/platform/kubernetes/Makefile | 1 - 1 file changed, 1 deletion(-)
(skywalking-showcase) branch endpointmodel updated (8f94135 -> 072a824)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch endpointmodel in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git discard 8f94135 update OAP for hierarchy (#155) add 3196d8f update OAP for hierarchy (#155) add 072a824 Add R3 to showcase This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (8f94135) \ N -- N -- N refs/heads/endpointmodel (072a824) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes:
(skywalking-showcase) branch endpointmodel updated (1a63afd -> 8f94135)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch endpointmodel in repository https://gitbox.apache.org/repos/asf/skywalking-showcase.git omit 1a63afd Add R3 to showcase omit 3196d8f update OAP for hierarchy (#155) add 8f94135 update OAP for hierarchy (#155) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (1a63afd) \ N -- N -- N refs/heads/endpointmodel (8f94135) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: deploy/platform/kubernetes/templates/feature-r3/resources.yaml | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)
Re: [PR] Upgrade otel collector version [skywalking]
wu-sheng commented on PR #11819: URL: https://github.com/apache/skywalking/pull/11819#issuecomment-1906032592 Notice, Kafka case may fail, trying to fix in https://github.com/apache/skywalking/pull/11772. But not sure yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
wu-sheng commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463262285 ## .github/workflows/skywalking.yaml: ## @@ -583,16 +583,21 @@ jobs: config: test/e2e-v2/cases/vm/zabbix/e2e.yaml - name: VM Prometheus config: test/e2e-v2/cases/vm/prometheus-node-exporter/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: VM Telegraf config: test/e2e-v2/cases/vm/telegraf/e2e.yaml - name: So11y config: test/e2e-v2/cases/so11y/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: MySQL Prometheus and slowsql config: test/e2e-v2/cases/mysql/mysql-slowsql/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 Review Comment: No, we don't need to. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
peachisai commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463261313 ## .github/workflows/skywalking.yaml: ## @@ -583,16 +583,21 @@ jobs: config: test/e2e-v2/cases/vm/zabbix/e2e.yaml - name: VM Prometheus config: test/e2e-v2/cases/vm/prometheus-node-exporter/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: VM Telegraf config: test/e2e-v2/cases/vm/telegraf/e2e.yaml - name: So11y config: test/e2e-v2/cases/so11y/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: MySQL Prometheus and slowsql config: test/e2e-v2/cases/mysql/mysql-slowsql/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 Review Comment: > This could be on matrix level? I will set this as a global env variable with matrix level If we do not need assign values individually for each test. ## .github/workflows/skywalking.yaml: ## @@ -583,16 +583,21 @@ jobs: config: test/e2e-v2/cases/vm/zabbix/e2e.yaml - name: VM Prometheus config: test/e2e-v2/cases/vm/prometheus-node-exporter/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: VM Telegraf config: test/e2e-v2/cases/vm/telegraf/e2e.yaml - name: So11y config: test/e2e-v2/cases/so11y/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: MySQL Prometheus and slowsql config: test/e2e-v2/cases/mysql/mysql-slowsql/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 Review Comment: > This could be on matrix level? I will set this as a global env variable with matrix level If we do not need assign values individually for each test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
wu-sheng commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463253338 ## .github/workflows/skywalking.yaml: ## @@ -583,16 +583,21 @@ jobs: config: test/e2e-v2/cases/vm/zabbix/e2e.yaml - name: VM Prometheus config: test/e2e-v2/cases/vm/prometheus-node-exporter/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: VM Telegraf config: test/e2e-v2/cases/vm/telegraf/e2e.yaml - name: So11y config: test/e2e-v2/cases/so11y/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 - name: MySQL Prometheus and slowsql config: test/e2e-v2/cases/mysql/mysql-slowsql/e2e.yaml +env: OTEL_COLLECTOR_VERSION=0.92.0 Review Comment: This could be on matrix level? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
peachisai commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463251560 ## test/e2e-v2/cases/apisix/otel-collector/docker-compose.yml: ## @@ -25,7 +25,7 @@ services: networks: e2e: otel-collector: -image: otel/opentelemetry-collector:0.50.0 +image: otel/opentelemetry-collector:0.92.0 Review Comment: > @peachisai I think you could set a global env variable after here for the whole e2e-test job > > https://github.com/apache/skywalking/blob/a9128e0cefbcba08c085b70ed8795d7fb416459c/.github/workflows/skywalking.yaml#L332 > > ```yaml > env: > OTEL_COLLECTOR_VERSION: 0.92.0 > ``` Okay, I originally had in mind we can assign values individually for each test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
wankai123 commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463239316 ## test/e2e-v2/cases/apisix/otel-collector/docker-compose.yml: ## @@ -25,7 +25,7 @@ services: networks: e2e: otel-collector: -image: otel/opentelemetry-collector:0.50.0 +image: otel/opentelemetry-collector:0.92.0 Review Comment: @peachisai I think you could set a global env variable after here for the whole e2e-test job https://github.com/apache/skywalking/blob/a9128e0cefbcba08c085b70ed8795d7fb416459c/.github/workflows/skywalking.yaml#L332 ``` yaml env: OTEL_COLLECTOR_VERSION: 0.92.0 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fetch submodules when checking file changes [skywalking]
kezhenxu94 commented on code in PR #11772: URL: https://github.com/apache/skywalking/pull/11772#discussion_r1463074850 ## .github/workflows/skywalking.yaml: ## @@ -114,6 +114,7 @@ jobs: - uses: actions/checkout@v3 # required for push event with: fetch-depth: 0 + submodules: true Review Comment: @Superskyyy turns out this is the root cause... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] If 99% of the number of samples is less than 10ms, the percentiles value is confusing. [skywalking]
wu-sheng closed issue #11822: [Bug] If 99% of the number of samples is less than 10ms, the percentiles value is confusing. URL: https://github.com/apache/skywalking/issues/11822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] If 99% of the number of samples is less than 10ms, the percentiles value is confusing. [skywalking]
huangmougou opened a new issue, #11822: URL: https://github.com/apache/skywalking/issues/11822 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues. ### Apache SkyWalking Component OAP server (apache/skywalking) ### What happened In the 1-minute time window, 99% of the sampled values are less than 10ms, a small part of which are 0ms, and most of which are greater than 0ms and less than 10ms. But the percentiles p55-p99 are all 0ms. This #11063 is about the issue. But I didn't find a final solution to the problem. ### What you expected to happen When calculating percentiles, the default is to take 10ms as the bucket size. If the number of samples in the 0-10ms bucket exceeds 99%, the final value is obtained by multiplying the bucket index by 10ms, and the calculated value is 0ms.I personally think that the final calculated value can be the maximum value within the bucket range, or the median within the bucket range. rather than the minimum value within the bucket range. ![Snipaste_2024-01-23_18-36-22](https://github.com/apache/skywalking/assets/37232020/1ed776ae-41ac-4bc0-bd7e-1aa4c7953017) ![Snipaste_2024-01-23_18-37-06](https://github.com/apache/skywalking/assets/37232020/5c0096ef-cb7d-41e6-b1da-de3cf3dbdf94) ### How to reproduce The number of samples in the 0-10ms bucket exceeds 99%. ### Anything else _No response_ ### Are you willing to submit a pull request to fix on your own? - [X] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Upgrade otel collector version [skywalking]
peachisai commented on code in PR #11819: URL: https://github.com/apache/skywalking/pull/11819#discussion_r1463032187 ## test/e2e-v2/cases/apisix/otel-collector/docker-compose.yml: ## @@ -25,7 +25,7 @@ services: networks: e2e: otel-collector: -image: otel/opentelemetry-collector:0.50.0 +image: otel/opentelemetry-collector:0.92.0 Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(skywalking-website) branch asf-site updated: deploy: fe0471f578a9a1fae0f184d69a384e4680aa5aa5
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/skywalking-website.git The following commit(s) were added to refs/heads/asf-site by this push: new a4f6b945345 deploy: fe0471f578a9a1fae0f184d69a384e4680aa5aa5 a4f6b945345 is described below commit a4f6b94534542cf63118c6e5af39512cdb8f0fa5 Author: wu-sheng AuthorDate: Tue Jan 23 10:04:45 2024 + deploy: fe0471f578a9a1fae0f184d69a384e4680aa5aa5 --- contributors/index.html| 14 ++-- .../next/en/contribution/compiling/index.html | 2 +- .../en/contribution/release-java-agent/index.html | 2 +- .../next/en/faq/ext-dirs/index.html| 2 +- docs/skywalking-java/next/en/faq/osgi/index.html | 2 +- .../java-agent/advanced-features/index.html| 2 +- .../java-agent/advanced-reporters/index.html | 2 +- .../kotlin-coroutine-plugin/index.html | 2 +- .../oracle-resin-plugins/index.html| 2 +- .../spring-annotation-plugin/index.html| 2 +- .../trace-ignore-plugin/index.html | 2 +- .../application-toolkit-dependency/index.html | 2 +- .../application-toolkit-kafka/index.html | 2 +- .../application-toolkit-log4j-1.x/index.html | 2 +- .../application-toolkit-log4j-2.x/index.html | 2 +- .../application-toolkit-logback-1.x/index.html | 2 +- .../application-toolkit-meter/index.html | 2 +- .../application-toolkit-micrometer-1.10/index.html | 2 +- .../application-toolkit-micrometer/index.html | 2 +- .../index.html | 2 +- .../index.html | 2 +- .../index.html | 2 +- .../index.html | 2 +- .../application-toolkit-trace/index.html | 2 +- .../application-toolkit-tracer/index.html | 2 +- .../application-toolkit-webflux/index.html | 2 +- .../java-agent/bootstrap-plugins/index.html| 2 +- .../java-agent/configuration-discovery/index.html | 2 +- .../java-agent/configurations/index.html | 2 +- .../java-agent/containerization/index.html | 2 +- .../java-agent/customize-enhance-trace/index.html | 2 +- .../java-agent/how-to-disable-plugin/index.html| 2 +- .../how-to-tolerate-exceptions/index.html | 2 +- .../java-plugin-development-guide/index.html | 4 ++-- .../java-agent/logic-endpoint/index.html | 2 +- .../java-agent/opentracing/index.html | 2 +- .../java-agent/optional-plugins/index.html | 2 +- .../java-agent/plugin-list/index.html | 2 +- .../java-agent/plugin-test/index.html | 26 +++--- .../service-agent/java-agent/readme/index.html | 2 +- .../java-agent/setting-override/index.html | 2 +- .../java-agent/specified-agent-config/index.html | 2 +- .../java-agent/supported-list/index.html | 2 +- .../setup/service-agent/java-agent/tls/index.html | 4 ++-- .../service-agent/java-agent/token-auth/index.html | 2 +- docs/skywalking-java/next/readme/index.html| 2 +- index.json | 2 +- team/index.html| 7 +++--- 48 files changed, 70 insertions(+), 71 deletions(-) diff --git a/contributors/index.html b/contributors/index.html index d84bb19a5c8..6cea97c76d5 100644 --- a/contributors/index.html +++ b/contributors/index.html @@ -8218,7 +8218,7 @@ -408 +409 1 @@ -8805,7 +8805,7 @@ -2743 +2744 1 @@ -15281,16 +15281,16 @@ - + -https://github.com/acceli; target="_blank"> + - acceli + 32413353cooolr** -1 -25 +1 +25 diff --git a/docs/skywalking-java/next/en/contribution/compiling/index.html b/docs/skywalking-java/next/en/contribution/compiling/index.html index 828d1e21bd5..fd3360991c0 100644 --- a/docs/skywalking-java/next/en/contribution/compiling/index.html +++ b/docs/skywalking-java/next/en/contribution/compiling/index.html @@ -740,7 +740,7
Re: [PR] Stream column-based storage [skywalking-banyandb]
codecov-commenter commented on PR #377: URL: https://github.com/apache/skywalking-banyandb/pull/377#issuecomment-1905617984 ## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report Attention: `1637 lines` in your changes are missing coverage. Please review. > Comparison is base [(`08d8dc8`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/08d8dc81f676078bc7776ec69bd99ea8cfc811e4?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 44.75% compared to head [(`5023f99`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 47.08%. | [Files](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Patch % | Lines | |---|---|---| | [banyand/stream/query.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vcXVlcnkuZ28=) | 0.00% | [346 Missing :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/write.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vd3JpdGUuZ28=) | 2.20% | [222 Missing :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/block.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vYmxvY2suZ28=) | 58.97% | [189 Missing and 28 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/part.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vcGFydC5nbw==) | 58.52% | [122 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/tstable.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vdHN0YWJsZS5nbw==) | 55.97% | [107 Missing and 11 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/merger.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vbWVyZ2VyLmdv) | 67.58% | [82 Missing and 12 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/index.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vaW5kZXguZ28=) | 14.45% | [70 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/part\_iter.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vcGFydF9pdGVyLmdv) | 74.08% | [50 Missing and 14 partials :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | | [banyand/stream/iter\_builder.go](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-YmFueWFuZC9zdHJlYW0vaXRlcl9idWlsZGVyLmdv) | 0.00% | [56 Missing :warning: ](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/377?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | |
Re: [PR] Optimize service conflict error message [skywalking-java]
lujiajing1126 commented on PR #666: URL: https://github.com/apache/skywalking-java/pull/666#issuecomment-1905581882 > @lujiajing1126 You are right, it can't resolve that problem and don't have much to do with each other , but I hope this optimeize can help us to find the cause of the problem quickly when plugins conflict. I agree this PR is a kind of optimization. But the changes here are totally irrelevant to the so-called "plugins conflict". BootService is an internal service. Normally plugin developers and users do not implement their own BootServices. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Optimize service conflict error message [skywalking-java]
wongtp commented on PR #666: URL: https://github.com/apache/skywalking-java/pull/666#issuecomment-1905562891 > > > I am confused why this is related to the [apache/skywalking#10991](https://github.com/apache/skywalking/issues/10991)? > > > The linked issue may be caused by version compatibility of the Netty plugin. It must not relevant to the kernel. > > > > > > @wu-sheng @lujiajing1126 I am so sorry, this was my mistake, I checked again the relate issue [intercept failure](https://github.com/apache/skywalking/issues/10991), it should be relate to [issues/11205](https://github.com/apache/skywalking/issues/11205), when `ServiceManager.bootedServices` initial failure, it will print many of NPE error message in skywalking-api.log, I hope this optimeize can help us to find the cause of the problem quickly. > > As I comment here, [apache/skywalking#11205 (comment)](https://github.com/apache/skywalking/issues/11205#issuecomment-1902871724) is caused by misunderstanding of the webflux scenario. > > I am going to close the issue since it actually resolves neither of the issues the author linked here. > > Feel free to reopen this if you have further idea. @lujiajing1126 You are right, it can't resolve that problem and don't have much to do with each other , but I hope this optimeize can help us to find the cause of the problem quickly when plugins conflict. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [skywalking]
wu-sheng closed issue #11820: [Bug] URL: https://github.com/apache/skywalking/issues/11820 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] [skywalking]
YaoXina opened a new issue, #11820: URL: https://github.com/apache/skywalking/issues/11820 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues. ### Apache SkyWalking Component Java Agent (apache/skywalking-java) ### What happened I use in docker java-agent ### What you expected to happen ERROR 2024-01-23 15:29:15.188 3a23de32c3114df5aaed5f117047f960-0-C-1 InstMethodsInter : class[class org.apache.kafka.clients.consumer.KafkaConsumer] after method[pollForFetches] intercept failure java.lang.ClassCastException: org.apache.kafka.clients.consumer.internals.Fetch incompatible with java.util.Map at org.apache.skywalking.apm.plugin.kafka.KafkaConsumerInterceptor.afterMethod(KafkaConsumerInterceptor.java:64) at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97) at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1254) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1227) at org.apache.kafka.clients.consumer.KafkaConsumer$$FastClassBySpringCGLIB$$11173828.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704) at org.apache.kafka.clients.consumer.KafkaConsumer$$EnhancerBySpringCGLIB$$262c25f1.poll() at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollConsumer(KafkaMessageListenerContainer.java:1601) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1576) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke$original$Ndtu6fc3(KafkaMessageListenerContainer.java:1377) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke$original$Ndtu6fc3$accessor$d6AyCJFE(KafkaMessageListenerContainer.java) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$auxiliary$LUiXHu1Q.call(Unknown Source) at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:86) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1291) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:832) ### How to reproduce bi-server: image: /bi-base-image:2.2.0 deploy: restart_policy: condition: on-failure placement: constraints: [node.role == manager] depends_on: - bi-server-jar - bi-server-lib volumes: - ../file:/file - ../log:/logs - ../backup:/backup - ./config:/config - ../application/:/app - /data/skywalking-agent/:/home/skywalking-agent entrypoint: "sh /wait.sh -d pgsql:5432 -s 3 -c 'java -javaagent:/home/skywalking-agent/skywalking-agent.jar -Dskywalking.agent.service_name= -Dskywalking.collector.backend_service=*** -jar -Ddb.password=${DB_PASSWORD} -Dspring.profiles.active=prd -Dspring.config.additional-location=file:/config/ /app/bi-server.jar'" networks: - ${NETWORK_NAME} environment: - TZ=Asia/Shanghai ### Anything else _No response_ ### Are you willing to submit a pull request to fix on your own? - [ ] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org