nagasrisai commented on code in PR #64519:
URL: https://github.com/apache/airflow/pull/64519#discussion_r3019894265
##########
providers/salesforce/src/airflow/providers/salesforce/operators/bulk.py:
##########
@@ -46,6 +47,9 @@ class SalesforceBulkOperator(BaseOperator):
:param batch_size: number of records to assign for each batch in the job
:param use_serial: Process batches in serial mode
:param salesforce_conn_id: The :ref:`Salesforce Connection id
<howto/connection:salesforce>`.
+ :param raise_on_failures: If True, raise an
:class:`~airflow.exceptions.AirflowException` when
+ any record in the result set reports ``success=False``. Defaults to
``False`` for backward
+ compatibility — failures are always logged as warnings regardless of
this setting.
Review Comment:
@eladkal Thanks for pointing me there. I had a look at
`BigQueryInsertJobOperator` and the key difference I noticed is that it retries
at the job level, specifically resubmitting the entire job when it hits a
rate-limit error from the API. Salesforce Bulk errors are different in that
they live at the record level inside a successful API response, so the retry
target would be a subset of the original payload rather than the whole request.
That said, the idea makes sense for certain Salesforce error codes that are
genuinely transient, things like `UNABLE_TO_LOCK_ROW` or
`API_TEMPORARILY_UNAVAILABLE`. For those, re-submitting just the failed records
after a short delay has a reasonable chance of succeeding. Errors like
`INVALID_FIELD` or `REQUIRED_FIELD_MISSING` are obviously not worth retrying.
Would you want me to add that to this PR, or keep this one focused on the
logging and open a follow-up for the retry behaviour? I am happy either way,
just want to make sure I do not make this PR harder to review by expanding its
scope without checking first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
