HenryLinTw opened a new issue #9717:
URL: https://github.com/apache/airflow/issues/9717
**Apache Airflow version**:
1.10.10
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
NA
**Environment**:
Docker Container
- **Cloud provider or hardware configuration**:
Docker host - OCI VM.Standard2.8
- **OS** (e.g. from /etc/os-release):
Oracle Linux Server 7.8
- **Kernel** (e.g. `uname -a`):
Linux internal-report-airflow 4.14.35-1902.303.5.3.el7uek.x86_64 #2 SMP Wed
Jun 10 15:39:01 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**:
pip3 install -r requirements.txt
- **Others**:
**What happened**:
First time switching to Oracle DB (Oracle Autonomous Transaction Processing
DB), `airflow initdb` failed on:
```bash
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-22858: invalid
alteration of datatype
[SQL: ALTER TABLE xcom MODIFY value BLOB]
```
Removed tables and still failed on the same place. try `airflow resetdb` and
failed with table already exists.
<details><summary>failure trace log</summary>
bash-4.2# airflow initdb
DB: oracle+cx_oracle://Admin:***@intlairflowdb_tp
[2020-07-08 06:00:54,217] {db.py:378} INFO - Creating tables
/usr/local/lib64/python3.6/site-packages/sqlalchemy/dialects/oracle/base.py:1376:
SAWarning: Oracle version (19, 5, 0, 0, 0) is known to have a maximum
identifier length of 128, rather than the historical default of 30. SQLAlchemy
1.4 will use 128 for this database; please set max_identifier_length=128 in
create_engine() in order to test the application with this new length, or set
to 30 in order to assure that 30 continues to be used. In particular, pay
close attention to the behavior of database migrations as dynamically generated
names may change. See the section 'Max Identifier Lengths' in the SQLAlchemy
Oracle dialect documentation for background.
% ((self.server_version_info,))
INFO [alembic.runtime.migration] Context impl OracleImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade cc1e65623dc7 ->
bdaa763e6c56, Make xcom value column a large binary
Traceback (most recent call last):
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1278, in _execute_context
cursor, statement, parameters, context
File
"/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line
593, in do_execute
cursor.execute(statement, parameters)
cx_Oracle.DatabaseError: ORA-22858: invalid alteration of datatype
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line
1140, in initdb
db.initdb(settings.RBAC)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line
323, in initdb
upgradedb()
File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line
386, in upgradedb
command.upgrade(config, 'heads')
File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line
298, in upgrade
script.run_env()
File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line
489, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py",
line 98, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line
184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py",
line 96, in <module>
run_migrations_online()
File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py",
line 90, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File
"/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line
846, in run_migrations
self.get_context().run_migrations(**kw)
File
"/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line
520, in run_migrations
step.migration_fn(**kw)
File
"/usr/local/lib/python3.6/site-packages/airflow/migrations/versions/bdaa763e6c56_make_xcom_value_column_a_large_binary.py",
line 42, in upgrade
batch_op.alter_column('value', type_=sa.LargeBinary())
File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/usr/local/lib/python3.6/site-packages/alembic/operations/base.py",
line 354, in batch_alter_table
impl.flush()
File "/usr/local/lib/python3.6/site-packages/alembic/operations/batch.py",
line 83, in flush
fn(*arg, **kw)
File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line
204, in alter_column
existing_comment=existing_comment,
File "/usr/local/lib/python3.6/site-packages/alembic/ddl/oracle.py", line
36, in _exec
result = super(OracleImpl, self)._exec(construct, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line
140, in _exec
return conn.execute(construct, *multiparams, **params)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1014, in execute
return meth(self, multiparams, params)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/sql/ddl.py",
line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1076, in _execute_ddl
compiled,
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1318, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1512, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py",
line 178, in raise_
raise exception
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1278, in _execute_context
cursor, statement, parameters, context
File
"/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line
593, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-22858: invalid
alteration of datatype
[SQL: ALTER TABLE xcom MODIFY value BLOB]
</details>
**What you expected to happen**:
`airflow initdb` succeed. don't understand why initdb trying to alter the
schema of the table it just created.
**How to reproduce it**:
1. provision an [Oracle Autonomous Transaction Processing
database](https://docs.cloud.oracle.com/en-us/iaas/Content/Database/Tasks/adbcreating.htm)
- though, based on the error message, I think any oracle DB would have this
issue.
1. install airflow with oracle as extra package
1. install [oracle instant
client](https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html)
**Anything else we need to know**:
How often does this problem occur? Once? Every time etc?
Tried multiple times and occured every time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
