This is an automated email from the ASF dual-hosted git repository.
laiyingchun pushed a change to branch dont_find_zpacks
in repository https://gitbox.apache.org/repos/asf/incubator-pegasus.git
discard 3201896aa update cmake
discard 24c1eb685 fix enc test failures
discard 810abcc45 keei zlib
discard dcad21e5d try
discard abf82e0fa 2
discard 47c74292e 1
discard 22d42ad7d Update FindLz4.cmake
discard 6e9e5da66 Update BaseFunctions.cmake
discard a71cf1c52 update docker
discard ca7657b52 add zstd and lz4 libs
discard 2183a9842 12
discard 393270b49 fix centos 7
discard ba3f524d2 fix snappy more
discard c0dc4aff5 Build snappy in the project
add 19876094e feat(HDFS): Add hdfs_service related configs in server.ini
and run.sh (#1705)
add 50563ea33 chore: add change log for version 2.5 (#1700)
add 3ffe54275 refactor(backup&restore): add more logs when backup or
restore failed (#1702)
add 05a90883c chore: switch to the standard Incubator DISCLAIMER instead
of current work-in-progress version (#1707)
add 63583da3e feat: source config_hdfs.sh automatically before starting
onebox (#1708)
add e15f9b275 feat(replica): add FileDataType as the parameter (#1703)
add 4b8b49a09 chore(CI): Use actions/labeler@v4 to keep compatiblity
(#1711)
add 18e378dc8 refactor(function_test): make [backup_]restore_test derive
from test_util (#1710)
add 50493b1fc chore(asf.yml): update collaborators (#1757)
add 4028c12f8 feat(new_metrics): add replica-level metric entity (#1345)
add 2d7cb40b8 feat(new_metrics): migrate replica-level metrics for write
service (#1351)
add c3f524002 feat(new_metrics): migrate replica-level metrics for
pegasus_server_impl (part 1) (#1374)
add e6ce28b87 feat(new_metrics): migrate replica-level metrics for
pegasus_server_impl (part 2) (#1386)
add f5eafcbf3 feat(new_metrics): migrate replica-level metrics for
capacity_unit_calculator (#1387)
add 522626048 feat(new_metrics): migrate replica-level metrics for replica
class (#1397)
add c263b8b51 feat(new_metrics): migrate replica-level metrics for
pegasus_event_listener (#1407)
add 688b43630 feat(new_metrics): migrate replica-level metrics for
pegasus_mutation_duplicator (#1413)
add b1730fba5 feat(new_metrics): add server-level metric entity (#1415)
add 2d362f1e1 feat(new_metrics): migrate built-in server-level metrics
(#1418)
add 14c8a102d feat(new_metrics): migrate server-level metrics for nfs
(#1421)
add 659743d6d feat(new_metrics): add disk-level metric entity and migrate
disk-level metrics for fs_manager (#1427)
add dbaa01abd feat(new_metrics): add table-level metric entity and migrate
table-level metrics for server_state of meta (#1431)
add e2243751a feat(new_metrics): add partition-level metric entity and
migrate partition-level metrics for greedy_load_balancer of meta (#1435)
add fb5b5b017 feat(new_metrics): migrate server-level metrics for
meta_service (#1437)
add 3e88153a1 feat(new_metrics): add backup-policy-level metric entity and
migrate backup-policy-level metrics for meta_backup_service (#1438)
add 6bd8b9568 feat(new_metrics): migrate partition-level metrics for
partition_guardian (#1440)
add 23debd2d2 feat(new_metrics): migrate replica-level metrics for
pegasus_manual_compact_service (#1443)
add a5c841e2b feat(new_metrics): migrate metrics for replica_stub (part 1)
(#1455)
add 30221094d feat(new_metrics): migrate metrics for replica_stub (part 2)
(#1459)
add 15ab7a1d0 feat(collector): migrate the collector from
pegasus-kv/collector (#1461)
add f959ba4b4 feat(new_metrics): migrate metrics for replica_stub (part 3)
(#1462)
add 43a0b0e8a feat(new_metrics): migrate metrics for replica_stub (part 4)
(#1463)
add a5ae84682 feat(new_metrics): migrate metrics for replica_stub (part 5)
(#1469)
add ec9a1ae62 feat(new_metrics): migrate metrics for replica_stub (part 6)
(#1474)
add 5249fdc9e feat(new_metrics): migrate metrics for replica_stub (part 7)
(#1475)
add c4c835149 feat(new_metrics): migrate metrics for some duplication
class (#1482)
add cbfc0e842 feat(new_metrics): migrate metrics for task queue (#1484)
add ade5adfde refactor(new_metrics): refactor enum definition for metric
types and units (#1491)
add 8abadfc4d feat(new_metrics): migrate metrics for failure detector
(#1502)
add 556c6ac62 feat(new_metrics): migrate metrics for network (#1504)
add 79b4e6aa3 feat(new_metrics): migrate server-level metrics of rocksdb
(#1506)
add 93391cb2f feat: Aggregate table/server level metrics (#1517)
add 568227956 feat(new_metrics): migrate metrics for profiler (#1524)
add 71e516251 fix(new_metrics): profiled tasks are measured by the wrong
metrics (#1528)
add 8e3b1f036 feat(new_metrics): remove all table-level perf-counters for
each replica (#1531)
add 2169d9c43 refactor(new_metrics): remove perf-counters that are still
used in shared log (#1533)
add 50cf91606 feat(new_metrics): migrate metrics for latency tracer (#1537)
add 641e97bfa fix(new_metrics): total_capacity_mb/total_available_mb are
not atomic (#1542)
add e724b62c6 feat(new_metrics): remove http service for perf counters
(#1540)
add dfb50dcc0 feat(new_metrics): remove pegasus_counter_reporter (#1548)
add 83eeb8c4a feat(new_metrics): remove perf counter since shared log has
been deprecated
add 6be5e286f feat(new_metrics): remove deleted header files introduced
in source files for new metrics
add db8e780bc fix(new_metrics): fix unit tests in verifying values of
backup_requests and write_size_exceed_threshold_requests
add fa82854a3 fix(IWYU): fix the suggestions reported by IWYU while rebase
merging branch migrate-metrics-dev into master such as including
absl/strings/string_view.h
add ade183428 Build snappy in the project
add efcdc13cc fix snappy more
add a0eab938f fix centos 7
add 1c3af7022 12
add a0a4f789f add zstd and lz4 libs
add 152caf1d6 update docker
add 1c651781c Update BaseFunctions.cmake
add c9db7df7a Update FindLz4.cmake
add 26ff927cc 1
add a91a9086c 2
add cb8ddbbe6 try
add 3db3fd48f keei zlib
add 8f09400f4 fix enc test failures
add 85370b277 update cmake
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3201896aa)
\
N -- N -- N refs/heads/dont_find_zpacks (85370b277)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 3 +-
.github/workflows/labeler.yml | 2 +-
...t_admin-cli.yml => lint_and_test_collector.yml} | 33 +-
.github/workflows/lint_and_test_cpp.yaml | 34 +-
.github/workflows/module_labeler_conf.yml | 2 +
.gitignore | 3 +
.licenserc.yaml | 4 +-
.rat-excludes | 4 +-
DISCLAIMER | 10 +
DISCLAIMER-WIP | 19 -
HISTORY.md | 40 ++
src/geo/CMakeLists.txt => collector/Makefile | 10 +-
collector/README.md | 28 +
collector/aggregate/aggregatable.go | 92 +++
collector/aggregate/aggregator.go | 173 ++++++
collector/aggregate/aggregator_test.go | 97 +++
collector/aggregate/history.go | 111 ++++
.../aggregate/history_test.go | 34 +-
collector/aggregate/hook.go | 69 +++
collector/aggregate/hook_test.go | 51 ++
collector/aggregate/perf_client.go | 237 ++++++++
collector/aggregate/perf_client_test.go | 49 ++
collector/aggregate/perf_counter_decoder.go | 62 ++
collector/aggregate/perf_counter_decoder_test.go | 61 ++
collector/aggregate/perf_session.go | 91 +++
collector/aggregate/table_stats.go | 134 +++++
collector/avail/detector.go | 152 +++++
.../start_onebox.sh => collector/config.yml | 45 +-
collector/go.mod | 47 ++
{admin-cli => collector}/go.sum | 179 +++---
.../hotspot/algo.go | 8 +-
collector/main.go | 101 ++++
.../metrics/meta_server_metrics.go | 9 +-
collector/metrics/metric_collector.go | 514 ++++++++++++++++
.../metrics/replica_server_metrics.go | 9 +-
collector/sink/falcon_sink.go | 144 +++++
collector/sink/prometheus_sink.go | 110 ++++
collector/sink/sink.go | 53 ++
collector/templates/index.html | 80 +++
collector/usage/usage_recorder.go | 115 ++++
collector/webui/index.go | 84 +++
src/utils/clock.cpp => collector/webui/tables.go | 49 +-
collector/webui/webserver.go | 61 ++
run.sh | 23 +-
scripts/config_hdfs.sh | 16 +-
src/CMakeLists.txt | 1 -
src/base/test/CMakeLists.txt | 8 +-
src/block_service/block_service_manager.cpp | 11 +-
src/common/fs_manager.cpp | 114 ++--
src/common/fs_manager.h | 63 +-
src/common/serialization_helper/thrift_helper.h | 1 -
src/common/test/CMakeLists.txt | 1 +
src/failure_detector/failure_detector.cpp | 17 +-
src/failure_detector/failure_detector.h | 4 +-
src/http/builtin_http_calls.cpp | 6 -
src/http/builtin_http_calls.h | 2 -
src/http/perf_counter_http_service.cpp | 66 ---
src/http/test/perf_counter_http_service_test.cpp | 76 ---
src/meta/greedy_load_balancer.cpp | 48 +-
src/meta/greedy_load_balancer.h | 7 -
src/meta/meta_backup_service.cpp | 77 ++-
src/meta/meta_backup_service.h | 28 +-
src/meta/meta_service.cpp | 43 +-
src/meta/meta_service.h | 8 +-
src/meta/meta_split_service.cpp | 5 +
src/meta/partition_guardian.cpp | 14 +-
src/meta/partition_guardian.h | 2 -
src/meta/server_state.cpp | 81 +--
src/meta/server_state.h | 15 +-
src/meta/server_state_restore.cpp | 2 +
src/meta/table_metrics.cpp | 389 ++++++++++++
src/meta/table_metrics.h | 257 ++++++++
src/meta/test/backup_test.cpp | 2 +-
src/meta/test/meta_split_service_test.cpp | 4 +
src/meta/test/state_sync_test.cpp | 2 +
src/meta/test/update_configuration_test.cpp | 2 +
src/nfs/nfs_client_impl.cpp | 56 +-
src/nfs/nfs_client_impl.h | 10 +-
src/nfs/nfs_server_impl.cpp | 34 +-
src/nfs/nfs_server_impl.h | 6 +-
src/nfs/test/CMakeLists.txt | 1 +
src/perf_counter/builtin_counters.cpp | 57 --
src/perf_counter/builtin_counters.h | 33 --
src/perf_counter/perf_counters.cpp | 13 -
src/perf_counter/perf_counters.h | 2 -
src/perf_counter/test/perf_counter_test.cpp | 1 +
src/perf_counter/test/perf_counters_test.cpp | 29 -
src/redis_protocol/proxy/CMakeLists.txt | 1 -
src/redis_protocol/proxy/main.cpp | 3 -
src/replica/backup/cold_backup_context.cpp | 28 +-
src/replica/backup/replica_backup_manager.cpp | 32 +-
src/replica/backup/replica_backup_manager.h | 9 +-
src/replica/bulk_load/replica_bulk_loader.cpp | 62 +-
src/replica/bulk_load/replica_bulk_loader.h | 9 +
src/replica/duplication/duplication_pipeline.cpp | 16 +-
src/replica/duplication/duplication_pipeline.h | 4 +-
src/replica/duplication/duplication_sync_timer.cpp | 7 +-
src/replica/duplication/load_from_private_log.cpp | 73 +--
src/replica/duplication/load_from_private_log.h | 14 +-
src/replica/duplication/mutation_batch.cpp | 21 +-
src/replica/duplication/mutation_batch.h | 4 +-
src/replica/duplication/replica_duplicator.cpp | 15 +-
src/replica/duplication/replica_duplicator.h | 6 +
.../duplication/replica_duplicator_manager.cpp | 16 +-
.../duplication/replica_duplicator_manager.h | 13 +-
.../test/load_from_private_log_test.cpp | 23 +-
src/replica/mutation_log.cpp | 6 -
src/replica/mutation_log.h | 16 +-
src/replica/replica.cpp | 357 +++++++----
src/replica/replica.h | 90 ++-
src/replica/replica_2pc.cpp | 15 +-
src/replica/replica_backup.cpp | 7 +-
.../replica_backup_manager.h => replica_base.cpp} | 53 +-
src/replica/replica_base.h | 21 +-
src/replica/replica_check.cpp | 5 +-
src/replica/replica_chkpt.cpp | 15 +-
src/replica/replica_disk_migrator.cpp | 2 +-
src/replica/replica_learn.cpp | 27 +-
src/replica/replica_stub.cpp | 660 +++++++--------------
src/replica/replica_stub.h | 116 +---
src/replica/replica_throttle.cpp | 11 +-
src/replica/replication_app_base.cpp | 46 +-
src/replica/replication_app_base.h | 35 ++
src/replica/split/replica_split_manager.cpp | 55 +-
src/replica/split/replica_split_manager.h | 8 +
src/replica/test/replica_disk_test_base.h | 1 -
src/replica/test/replica_test.cpp | 33 +-
src/reporter/CMakeLists.txt | 44 --
src/reporter/pegasus_counter_reporter.cpp | 381 ------------
src/reporter/pegasus_counter_reporter.h | 119 ----
src/reporter/pegasus_io_service.h | 65 --
src/runtime/profiler.cpp | 495 ++++++++--------
src/runtime/profiler_header.h | 69 ++-
src/runtime/rpc/network.cpp | 31 +-
src/runtime/rpc/network.h | 5 +-
src/runtime/rpc/rpc_message.cpp | 2 +-
src/runtime/task/task_queue.cpp | 80 ++-
src/runtime/task/task_queue.h | 19 +-
src/server/capacity_unit_calculator.cpp | 149 ++---
src/server/capacity_unit_calculator.h | 29 +-
src/server/config.ini | 10 +-
src/server/info_collector_app.cpp | 11 +-
src/server/info_collector_app.h | 1 -
src/server/pegasus_event_listener.cpp | 121 ++--
src/server/pegasus_event_listener.h | 26 +-
src/server/pegasus_manual_compact_service.cpp | 37 +-
src/server/pegasus_manual_compact_service.h | 6 +-
src/server/pegasus_mutation_duplicator.cpp | 33 +-
src/server/pegasus_mutation_duplicator.h | 6 +-
src/server/pegasus_server_impl.cpp | 412 ++++++-------
src/server/pegasus_server_impl.h | 96 +--
src/server/pegasus_server_impl_init.cpp | 406 +++++++------
src/server/pegasus_server_write.cpp | 24 +-
src/server/pegasus_server_write.h | 4 +-
src/server/pegasus_service_app.h | 47 +-
src/server/pegasus_write_service.cpp | 288 ++++-----
src/server/pegasus_write_service.h | 50 +-
src/server/pegasus_write_service_impl.h | 5 +-
src/server/rocksdb_wrapper.cpp | 9 +-
src/server/rocksdb_wrapper.h | 7 +-
src/server/test/CMakeLists.txt | 3 +-
.../test/pegasus_mutation_duplicator_test.cpp | 1 +
src/server/test/pegasus_server_impl_test.cpp | 8 +-
src/server/test/pegasus_server_write_test.cpp | 4 +-
.../function_test/backup_restore/CMakeLists.txt | 2 +-
.../backup_restore/test_backup_and_restore.cpp | 115 ++--
src/test/function_test/config.ini | 2 -
src/test/function_test/restore/test_restore.cpp | 87 +--
src/test/function_test/utils/test_util.cpp | 2 +-
src/test/kill_test/config.ini | 1 -
src/utils/CMakeLists.txt | 1 +
src/utils/builtin_metrics.cpp | 92 +++
src/utils/{preloadable.h => builtin_metrics.h} | 33 +-
src/utils/clock.cpp | 1 +
src/utils/enum_helper.h | 63 +-
src/utils/latency_tracer.cpp | 178 ++++--
src/utils/latency_tracer.h | 3 -
src/utils/metrics.cpp | 10 +-
src/utils/metrics.h | 405 ++++++++++++-
src/utils/test/enum_helper_test.cpp | 211 +++++++
src/utils/test/metrics_test.cpp | 184 +++++-
src/utils/time_utils.h | 26 +
182 files changed, 7495 insertions(+), 3809 deletions(-)
copy .github/workflows/{lint_and_test_admin-cli.yml =>
lint_and_test_collector.yml} (75%)
create mode 100644 DISCLAIMER
delete mode 100644 DISCLAIMER-WIP
create mode 100644 HISTORY.md
copy src/geo/CMakeLists.txt => collector/Makefile (90%)
create mode 100644 collector/README.md
create mode 100644 collector/aggregate/aggregatable.go
create mode 100644 collector/aggregate/aggregator.go
create mode 100644 collector/aggregate/aggregator_test.go
create mode 100644 collector/aggregate/history.go
copy src/runtime/ranger/access_type.cpp => collector/aggregate/history_test.go
(57%)
create mode 100644 collector/aggregate/hook.go
create mode 100644 collector/aggregate/hook_test.go
create mode 100644 collector/aggregate/perf_client.go
create mode 100644 collector/aggregate/perf_client_test.go
create mode 100644 collector/aggregate/perf_counter_decoder.go
create mode 100644 collector/aggregate/perf_counter_decoder_test.go
create mode 100644 collector/aggregate/perf_session.go
create mode 100644 collector/aggregate/table_stats.go
create mode 100644 collector/avail/detector.go
copy docker/pegasus-docker-compose/start_onebox.sh => collector/config.yml
(59%)
mode change 100755 => 100644
create mode 100644 collector/go.mod
copy {admin-cli => collector}/go.sum (87%)
copy src/http/service_version.cpp => collector/hotspot/algo.go (89%)
create mode 100644 collector/main.go
copy src/perf_counter/test/main.cpp =>
collector/metrics/meta_server_metrics.go (82%)
create mode 100644 collector/metrics/metric_collector.go
copy src/perf_counter/test/main.cpp =>
collector/metrics/replica_server_metrics.go (81%)
create mode 100644 collector/sink/falcon_sink.go
create mode 100644 collector/sink/prometheus_sink.go
create mode 100644 collector/sink/sink.go
create mode 100644 collector/templates/index.html
create mode 100644 collector/usage/usage_recorder.go
create mode 100644 collector/webui/index.go
copy src/utils/clock.cpp => collector/webui/tables.go (53%)
create mode 100644 collector/webui/webserver.go
delete mode 100644 src/http/perf_counter_http_service.cpp
delete mode 100644 src/http/test/perf_counter_http_service_test.cpp
create mode 100644 src/meta/table_metrics.cpp
create mode 100644 src/meta/table_metrics.h
delete mode 100644 src/perf_counter/builtin_counters.cpp
delete mode 100644 src/perf_counter/builtin_counters.h
copy src/replica/{backup/replica_backup_manager.h => replica_base.cpp} (50%)
delete mode 100644 src/reporter/CMakeLists.txt
delete mode 100644 src/reporter/pegasus_counter_reporter.cpp
delete mode 100644 src/reporter/pegasus_counter_reporter.h
delete mode 100644 src/reporter/pegasus_io_service.h
create mode 100644 src/utils/builtin_metrics.cpp
copy src/utils/{preloadable.h => builtin_metrics.h} (67%)
create mode 100644 src/utils/test/enum_helper_test.cpp
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
