yifan-c opened a new pull request, #73: URL: https://github.com/apache/cassandra-analytics/pull/73
Option to specify the ideal timeout in seconds for bulk write jobs. It is only effective when the bulk write job is using S3_COMPACT data transport mode. When JOB_IDEAL_TIMEOUT_SECONDS is specified and less than the actual time the bulk write job needs to achieve the specified consistency level, it is ignored and job only exit after the desired consistency level has been satisfied. For example, a bulk write job indeed requires 1 hour to achieve LOCAL_QUORUM, it ignores any JOB_IDEAL_TIMEOUT_SECONDS that is less than 3600 seconds (1 hour), and only complete after 1 hour. If JOB_IDEAL_TIMEOUT_SECONDS is 5400 seconds (1.5 hours), the job after achieve LOCAL_QUORUM waits for at most 0.5 hours in addition. The effective wait time is the minimum of the remaining time to ideal timeout and the estimated wait time to finish all slice import (as estimated in org.apache.cassandra.spark.bulkwriter.ImportCompletionCoordinator). The ideal timeout is ignored in order to complete the bulk write j ob in some circumstances, hence named "ideal". Patch by Yifan Cai; Reviewed by TBD for CASSANDRA-19827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
