ninsmiracle commented on PR #1841: URL: https://github.com/apache/incubator-pegasus/pull/1841#issuecomment-1884346769
> > > What's the relationship between the newly added config and `duplicate_log_batch_bytes`? > > > > > > `duplicate_log_batch_bytes` is the batch size you can set,always set to 4096. The 'newly' added config `max_allowed_write_size` is the max size of a write mutation that a cluster could received. Not only duplication write,normal write will be limit by this config also. > > If we only consider `duplicate_log_batch_bytes`,master will send some mutations that backup cluster can not receive. Backup cluster will log: > > ``` > > E2024-01-04 11:42:02.88 (1704339722088300009 6004) replica.replica7.0400174218c587bd: replica_stub.cpp:1645:response_client(): 22...@xx.xx.xx.xx:42801: write fail: client = xx.xx.xx.xxx:42801, code = RPC_RRDB_RRDB_DUPLICATE, timeout = 236, status = replication::partition_status::PS_PRIMARY, error = ERR_INVALID_DATA > > > > W2024-01-04 11:47:33.656 (1704340053656585710 6004) replica.replica7.0400174218c58cd1: replica_2pc.cpp:77:on_client_write(): [22.53@xxxx:42801] client from xxxx:42801 write request body size exceed threshold, request = [default], request_body_size = 1048716, max_allowed_write_size = 1048576, it will be rejected! > > ``` > > > > > > > > > > > > > > > > > > > > > > > > We can see it clearly that request_body_size(master cluster send to backup cluster) is greater than max_allowed_write_size. > > 1. duplicate_log_batch_bytes is optional, how to comprehend "always" set to 4096? > 2. I only saw dup_max_allowed_write_size is used in duplication module, how it take effect on "normal write"? > 3. What's the config value of duplicate_log_batch_bytes in your environment, if it's less than the config value of max_allowed_write_size in backup cluster (i.e. 1048576), how can this situaction happen? 1.Because`duplicate_log_batch_bytes` default value is 4096 2.I saw `max_allowed_write_size` in `replica_2pc_cpp`.Here it can effect on "normal write" 3.It's not easy for master cluster get the value of `duplicate_log_batch_bytes` config by backup cluster. I don't think it's necessary for master cluster to get this value,because when we doing duplication ,we usually keep the similarity config between two clusters. I'm sorry that what I said above caused ambiguity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pegasus.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pegasus.apache.org For additional commands, e-mail: dev-h...@pegasus.apache.org
