hewerthomn opened a new issue, #25676:
URL: https://github.com/apache/airflow/issues/25676
### Apache Airflow Provider(s)
oracle
### Versions of Apache Airflow Providers
apache-airflow-providers-oracle == 3.2.0
### Apache Airflow version
2.3.3
### Operating System
Debian GNU/Linux 11 (bullseye)
### Deployment
Docker-Compose
### Deployment details
_No response_
### What happened
When `source_sql` query returns more or equal rows than `rows_chunk` value,
an exception is raised and the transfer process stops, but the first chunk part
is inserted.
In some cases it worked fine because the `source_sql` returned a few rows
and the default `rows_chuck` value is `5000`.
### What you think should happen instead
Log of an limited query under default `rows_chuck` size:
```
[2022-08-11, 10:11:39 -04] {oracle.py:273} INFO - [TABLE_NAME] inserted 4999
rows
[2022-08-11, 10:11:39 -04] {oracle_to_oracle.py:81} INFO - Total inserted:
4999 rows
[2022-08-11, 10:11:39 -04] {oracle_to_oracle.py:83} INFO - Finished data
transfer.
[2022-08-11, 10:11:39 -04] {taskinstance.py:1420} INFO - Marking task as
SUCCESS.
```
Log without limit query results and need to chuck:
```
[2022-08-11, 10:07:01 -04] {oracle.py:266} INFO - [TABLE_NAME] inserted 5000
rows
[2022-08-11, 10:07:01 -04] {taskinstance.py:1909} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/oracle/transfers/oracle_to_oracle.py",
line 89, in execute
self._execute(src_hook, dest_hook, context)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/oracle/transfers/oracle_to_oracle.py",
line 78, in _execute
self.destination_table, rows, target_fields=target_fields,
commit_every=self.rows_chunk
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/oracle/hooks/oracle.py",
line 271, in bulk_insert_rows
cursor.executemany(None, row_chunk)
File
"/home/airflow/.local/lib/python3.7/site-packages/oracledb/cursor.py", line
440, in executemany
bool(arraydmlrowcounts))
File "src/oracledb/impl/thin/cursor.pyx", line 132, in
oracledb.thin_impl.ThinCursorImpl.executemany
File "src/oracledb/impl/thin/cursor.pyx", line 113, in
oracledb.thin_impl.ThinCursorImpl._preprocess_execute
File
"/home/airflow/.local/lib/python3.7/site-packages/oracledb/errors.py", line
103, in _raise_err
raise exc_type(_Error(message)) from cause
oracledb.exceptions.DatabaseError: DPY-4010: a bind variable replacement
value for placeholder ":1" was not provided
[2022-08-11, 10:07:01 -04] {taskinstance.py:1420} INFO - Marking task as
UP_FOR_RETRY.
```
When I changed the `rows_chunk` value to a higher value, it worked:
```
[2022-08-11, 10:12:11 -04] {oracle.py:273} INFO - [TABLE_NAME] inserted
106849 rows
[2022-08-11, 10:12:11 -04] {oracle_to_oracle.py:81} INFO - Total inserted:
106849 rows
[2022-08-11, 10:12:11 -04] {oracle_to_oracle.py:83} INFO - Finished data
transfer.
[2022-08-11, 10:12:11 -04] {taskinstance.py:1420} INFO - Marking task as
SUCCESS.
```
### How to reproduce
To reproduce this bug the `source_sql` needs to returns more or equal rows
than `rows_chunk` value.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
