acelyc111 commented on PR #1841: URL: https://github.com/apache/incubator-pegasus/pull/1841#issuecomment-1883370089
> > What's the relationship between the newly added config and `duplicate_log_batch_bytes`? > > `duplicate_log_batch_bytes` is the batch size you can set,always set to 4096. The 'newly' added config `max_allowed_write_size` is the max size of a write mutation that a cluster could received. Not only duplication write,normal write will be limit by this config also. > > If we only consider `duplicate_log_batch_bytes`,master will send some mutations that backup cluster can not receive. Backup cluster will log: > > ``` > E2024-01-04 11:42:02.88 (1704339722088300009 6004) replica.replica7.0400174218c587bd: replica_stub.cpp:1645:response_client(): [email protected]:42801: write fail: client = xx.xx.xx.xxx:42801, code = RPC_RRDB_RRDB_DUPLICATE, timeout = 236, status = replication::partition_status::PS_PRIMARY, error = ERR_INVALID_DATA > > W2024-01-04 11:47:33.656 (1704340053656585710 6004) replica.replica7.0400174218c58cd1: replica_2pc.cpp:77:on_client_write(): [22.53@xxxx:42801] client from xxxx:42801 write request body size exceed threshold, request = [default], request_body_size = 1048716, max_allowed_write_size = 1048576, it will be rejected! > ``` > > We can see it clearly that request_body_size(master cluster send to backup cluster) is greater than max_allowed_write_size. 1. duplicate_log_batch_bytes is optional, how to comprehend "always" set to 4096? 2. I only saw max_allowed_write_size is used in duplication module, how it take effect on "normal write"? 3. What's the config value of duplicate_log_batch_bytes in your environment, if it's less than the config value of max_allowed_write_size in backup cluster (i.e. 1048576), how can this situaction happen? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
