[
https://issues.apache.org/jira/browse/CASSANDRA-19827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-19827:
----------------------------------
Test and Documentation Plan: ci; unit
Status: Patch Available (was: Open)
PR: https://github.com/apache/cassandra-analytics/pull/73
CI:
https://app.circleci.com/pipelines/github/yifan-c/cassandra-analytics?branch=CASSANDRA-19827%2Ftrunk
> [Analytics] Add job_ideal_timeout_seconds writer option
> -------------------------------------------------------
>
> Key: CASSANDRA-19827
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19827
> Project: Cassandra
> Issue Type: Improvement
> Components: Analytics Library
> Reporter: Yifan Cai
> Assignee: Yifan Cai
> Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Option to specify the ideal timeout in seconds for bulk write jobs.
> It is only effective when the bulk write job is using S3_COMPACT data
> transport mode.
> When JOB_IDEAL_TIMEOUT_SECONDS is specified and less than the actual time the
> bulk write job
> needs to achieve the specified consistency level, it is ignored and job only
> exit after the desired consistency level has been satisfied.
> For example, a bulk write job indeed requires 1 hour to achieve LOCAL_QUORUM,
> it ignores
> any JOB_IDEAL_TIMEOUT_SECONDS that is less than 3600 seconds (1 hour), and
> only complete after 1 hour.
> If JOB_IDEAL_TIMEOUT_SECONDS is 5400 seconds (1.5 hours), the job after
> achieve LOCAL_QUORUM waits for at most 0.5 hours in addition. The effective
> wait time is the minimum of the remaining time to ideal timeout and the
> estimated wait time to finish all slice import (as estimated
> in org.apache.cassandra.spark.bulkwriter.ImportCompletionCoordinator).
> The ideal timeout is ignored in order to complete the bulk write job in some
> circumstances, hence named "ideal".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
