Smityz opened a new issue #588: URL: https://github.com/apache/incubator-pegasus/issues/588
## Step 1: Communication protocol > Change: > > [src/base/rrdb_types.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-95830f82267c97703daff7260747d291) > > [src/idl/rrdb.thrift](https://github.com/apache/incubator-pegasus/pull/502/files#diff-757c132cde1d84f5ca93d5570e6e5884) > > [src/include/rrdb/rrdb_types.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-603c6423dd9eac54b8ff57fae445e54c) These codes construct the communication protocol hotkey collector needs. **No effect on the original function!** ## Step 2: Refactor the hot partition detection(split read/write hotkey) > Add: > > [src/server/hotspot_algo_qps_variance.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-aade56caef4710898031101d1b3e9a8c) > > [src/server/hotspot_algo_qps_variance.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-8b7096f3be2d1540ae479649f3b3a3f8) > Change: > > [src/server/hotspot_partition_data.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-071cd34eaa571441acbfe7dc41295609) > > [src/server/table_hotspot_policy.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-5fd118d0e84066931e24d8214154ae84) > > [src/server/table_hotspot_policy.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-757e94e52452d0f940a3295ae4276c3f) > > [src/shell/command_helper.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-1599c853dea1efbbb4c96417ea819e7c) > Unit test: > > [src/server/test/pegasus_tablehotspot_test.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-15111e043d98c9a1cdb514e9660be9e3) The original hot partition detection can not distinguish read and write hotspots, the current code adds this function. **Changed the original data structure, changed the log format, and did not bring additional performance burden!** ## Step 3: Add the ability of collector to send RPC to start hotkey detection > Change: > > [src/server/table_hotspot_policy.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-5fd118d0e84066931e24d8214154ae84) > > [src/server/table_hotspot_policy.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-757e94e52452d0f940a3295ae4276c3f) These codes can add the ability of collector to send RPC to start hotkey detection. **No effect on the original function!** ## Step 4: Add hotkey collector's main codes > Add: > > [src/server/hotkey_collector.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-5e4c9662fd28e62dc1361b95dd08066b) > > [src/server/hotkey_collector.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-bd4101c5f1445e41aace39b8ca759a7e) > > src/server/hotkey_coarse_data_collector.h > > [src/server/hotkey_coarse_data_collector.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-54e1f33fe670e7659c7d0f8c0c70bf89) > > src/server/hotkey_fine_data_collector.h > > [src/server/hotkey_fine_data_collector.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-3dc38b942ebab020d456a84a79c67dda) > Unit test: > > [src/server/test/hotkey_collector_test.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-4ce4c6a1959039c44ee2f9907ff7c229) These codes contain main functions of hotkey collector, but in this PR, it won't be starting. **No effect on the original function!** ## Step 5: Start timing task of hotkey_collector in server > Change: > > [src/server/pegasus_server_impl.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-b28d1f920e921008396bf24dcdf1834c) > > [src/server/pegasus_server_impl.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-3111b4bc6ec81c9104fc7716d27db4fa) > > [src/server/pegasus_server_impl_init.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-e8288a65f73989b96c120d49624c64ba) > > [src/server/pegasus_read_service.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-e8d13f3ebd402a905841fdb3c753e1f8) These codes integrate hotkey collector and pegasus_server_impl, and start timing time in the tasking queue. **Add a timing task, but no effect on the original function!** ## Step 6: Realize the function of capturing data > Change: > > [src/server/capacity_unit_calculator.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-1b76afc3c33a18c468bc9caeb40b7495) > > [src/server/capacity_unit_calculator.h](https://github.com/apache/incubator-pegasus/pull/502/files#diff-1ec0346e75c1b84666b2764aa6feaa98) > Unit test: > > [src/server/test/hotkey_collector_test.cpp](https://github.com/apache/incubator-pegasus/pull/502/files#diff-4ce4c6a1959039c44ee2f9907ff7c229) These codes will implement the ability to capture data. **May affect performance and stability, so be careful!** ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
