Lee-W commented on code in PR #59183:
URL: https://github.com/apache/airflow/pull/59183#discussion_r2613145239
##########
airflow-core/tests/unit/assets/test_manager.py:
##########
@@ -202,3 +206,38 @@ def test_create_assets_notifies_asset_listener(self,
session):
assert len(asset_listener.created) == 1
assert len(asms) == 1
assert asset_listener.created[0].uri == asset.uri == asms[0].uri
+
+ @pytest.mark.usefixtures("dag_maker", "testing_dag_bundle")
+ def test_get_or_create_apdr_race_condition(self, session):
+ asm = AssetModel(uri="test://asset1/", name="parition_asset",
group="asset")
+ testing_dag = DagModel(dag_id="testing_dag", is_stale=False,
bundle_name="testing")
+ session.add_all([asm, testing_dag])
+ session.commit()
+ session.flush()
+ assert session.query(AssetPartitionDagRun).count() == 0
+
+ def _get_or_create_apdr():
+ if TYPE_CHECKING:
+ assert Session
+ assert Session.session_factory
+
+ _session = Session.session_factory()
+ _session.begin()
+ print(_session.scalar(select(AssetModel)).id)
+ try:
+ return AssetManager._get_or_create_apdr(
+ target_key="test_partition_key",
+ target_dag=testing_dag,
+ asset_id=asm.id,
+ session=_session,
+ ).id
+ finally:
+ _session.commit()
+ _session.close()
+
Review Comment:
I understand your point now. If we decide to incorporate this, we'll need to
handle the three different databases separately. What we're trying to verify is
whether we can achieve the correct behavior with multiple simultaneous writes,
rather than just assessing if there's a lock in place. So, I'm not entirely
convinced that we should add this test.
As an alternative, I slightly improved the test and ran it with 100 threads
simultaneously, but we still observed the same behavior. Please let me know
what you think. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
