pan3793 commented on code in PR #4418: URL: https://github.com/apache/kyuubi/pull/4418#discussion_r1118541610
##########
docs/deployment/settings.md:
##########
@@ -403,9 +403,11 @@ You can configure the Kyuubi properties in
`$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.metadata.cleaner.interval | PT30M
| The interval to check and clean expired
metadata.
| duration | 1.6.0 |
| kyuubi.metadata.max.age | PT72H
| The maximum age of metadata, the metadata
exceeding the age will be cleaned.
|
duration | 1.6.0 |
| kyuubi.metadata.recovery.threads | 10
| The number of threads for recovery from the
metadata store when the Kyuubi server restarts.
|
int | 1.6.0 |
-| kyuubi.metadata.request.retry.interval | PT5S
| The interval to check and trigger the metadata
request retry tasks.
|
duration | 1.6.0 |
-| kyuubi.metadata.request.retry.queue.size | 65536
| The maximum queue size for buffering metadata
requests in memory when the external metadata storage is down. Requests will be
dropped if the queue exceeds.
| int
| 1.6.0 |
-| kyuubi.metadata.request.retry.threads | 10
| Number of threads in the metadata request retry
manager thread pool. The metadata store might be unavailable sometimes and the
requests will fail, tolerant for this case and unblock the main thread, we
support retrying the failed requests in an async way.
|
int | 1.6.0 |
+| kyuubi.metadata.request.retry.async.enabled | false
| Whether to retry in async when metadata request
failed. If false, when metadata request failed, block the request thread and
retry synchronously until success or reach the threshold; if true, return
immediately even the metadata request failed, and schedule it in background
until success. In async mode, we can not guarantee metadata consistency
eventually, but it's still valuable since we can tolerate long-time metadata
store outages w/o blocking the submission request.
| boolean | 1.7.0 |
+| kyuubi.metadata.request.retry.async.queue.size | 65536
| The maximum queue size for buffering metadata
requests in memory when the external metadata storage is down. Requests will be
dropped if the queue exceeds. Only take affect when
kyuubi.metadata.request.retry.async.enabled is `true`.
| int | 1.6.0 |
+| kyuubi.metadata.request.retry.async.threads | 10
| Number of threads in the metadata request retry
manager thread pool. Only take affect when
kyuubi.metadata.request.retry.async.enabled is `true`.
| int | 1.6.0 |
+| kyuubi.metadata.request.retry.interval | PT5S
| The interval for metadata request retry.
|
duration | 1.6.0 |
+| kyuubi.metadata.request.retry.maxAttempts | 3
| The max attempts for metadata request retry.
| int
| 1.7.0 |
Review Comment:
case 1:
- t1: metadata store outage
- t2: client opens batch, kyuubi inserts metadata failed, but returns a
successful response
- t3: server done
- t4: metadata store recovered.
case 2: kyuubi inserts metadata record but encounters "duplicated unique
key" error, which means the insert will never succeed, it blocks all following
update operations of this batch job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
