Re: Airflow 2.x graph view problem

Fri, 20 May 2022 03:56:25 -0700

One possible use case for this is a helper/factory function in a shared library that creates tasks for use in multiple DAGs. 

I think we should look at fixing this bug rather than raising an error.
(I _think_ we already error if you try to change the DAG a task is assigned to, so you can only set a task's dag this way, never change it.) 

-a
On Wed, May 18 2022 at 12:27:42 +0200, Jarek Potiuk <[email protected]> wrote: 
Thanks for the analysis Laszlo. Really helpful.

As usual my first thought is - yeah. "Laszlo\s problem is solved but
can we do something to prevent it from happening for others ?".

Just loudly thinking - I guess there is never a reason to set the dag
on the task after it has been created. I think there are some internal
cases like task mapping and others where we want to do it, but in
"Regular" dags it should never be needed. Maybe we could catch such
cases and raise exception - informing the user that they are doing it
wrong ?

J.
On Wed, May 18, 2022 at 11:03 AM Kovacs, Laszlo <[email protected] <mailto:[email protected]>> wrote: 

 It turns out I found the problem.
This code was written by someone who does not work with us anymore and some pockets are still a bit undiscovered.
Yesterday I spotted that some operators do not do retries as configured in the default_args of the DAG instance. Digging into it I noticed that on some of the operators the dag instance is set after construction. 



 So say instead of:



 Dag = DAG(…)

 op = DummyOperator(task_id=…, dag=dag)



 We have:



 Dag = DAG(…)

 op = DummyOperator(task_id=…)

 op.dag=dag
Using the first variant instead of the second with all operators seems to have fixed the graph view issue in Airflow 2.x. The graph view in Airflow 1.x works with the second variant as well.
Retries configured in the default_args of the DAG instance do not propagate to the operators with the second variant, they do with the first. 



 Thanks all for helping, this is sorted now.



 Laszlo



 From: Ash Berlin-Taylor <[email protected] <mailto:[email protected]>>
 Sent: Tuesday, May 17, 2022 3:35 PM
 To: [email protected] <mailto:[email protected]>
 Subject: RE: Airflow 2.x graph view problem
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. 

 ________________________________
Are you able to share the DAG that triggers this? (Privately if necessary) -- it would make debugging and fixing that error a lot easier. 



 Thanks,

 Ash
On Mon, May 16 2022 at 16:15:26 +0000, "Kovacs, Laszlo" <[email protected] <mailto:[email protected]>> wrote: 

 Hi Jed,



 Thanks for helping.
Yes, as it turns out there is an error there (thanks for the tip to look there). It does not tell me much though. 



 I pasted the error message below.



 Any help is much appreciated.



 Thanks,



 Laszlo
DevTools failed to load source map: Could not load content for https://<airflow server>/static/appbuilder/css/bootstrap.min.css.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
d3.min.js:1 Error: <g> attribute transform: Expected number, "translate(NaN,20)scale(0)". 

 i               @            d3.min.js:1

 (anonymous)     @            d3.min.js:3

 Y              @            d3.min.js:1

 Co.each @            d3.min.js:3

 Co.attr  @            d3.min.js:3

 (anonymous)     @            graph.b30d452пїЅ.js:1087

 t              @            d3.min.js:1

 (anonymous)     @            d3.min.js:1

 c              @            d3.min.js:3

 (anonymous)     @            d3.min.js:3

 (anonymous)     @            d3.min.js:3

 Y              @            d3.min.js:1

 Co.each @            d3.min.js:3

 n.event @            d3.min.js:3

 setUpZoomSupport        @            graph.b30d452пїЅ.js:1105

 draw      @            graph.b30d452пїЅ.js:1079

 (anonymous)     @            graph.b30d452пїЅ.js:1569

 __webpack_require__  @            graph.b30d452пїЅ.js:30

 (anonymous)     @            graph.b30d452пїЅ.js:892

 __webpack_require__  @            graph.b30d452пїЅ.js:30

 (anonymous)     @            graph.b30d452пїЅ.js:94

 (anonymous)     @            graph.b30d452пїЅ.js:97
webpackUniversalModuleDefinition @ graph.b30d452пїЅ.js:9 

 (anonymous)     @            graph.b30d452пїЅ.js:10
moment.26f1d83пїЅ.js:1147 Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged. Please refer to <http://momentjs.com/guides/#/warnings/js-date/> for more info. 

 Arguments:
[0] _isAMomentObject: true, _isUTC: false, _useUTC: false, _l: undefined, _i: 2022-05-16T04:57:57T+01:0016, _f: undefined, _strict: undefined, _locale: [object Object] 

 Error
at Function.createFromInputFallback (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:1183:25)
at configFromString (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3400:19)
at configFromInput (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3841:13)
at prepareConfig (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3824:13)
at createFromConfig (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3791:44)
at createLocalOrUTC (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3885:16)
at createLocal (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:3889:16)
at hooks (https://<airflow server>/static/dist/moment.26f1d83пїЅ.js:880:29)
at HTMLTimeElement.<anonymous> (https://<airflow server>/static/dist/graph.b30d452пїЅ.js:180:16)
at Function.each (https://<airflow server>/static/appbuilder/js/jquery-latest.js:2:3003) 

 warn      @            moment.26f1d83пїЅ.js:1147

 (anonymous)     @            moment.26f1d83пїЅ.js:1178
configFromString @ moment.26f1d83пїЅ.js:3400
configFromInput @ moment.26f1d83пїЅ.js:3841 

 prepareConfig   @            moment.26f1d83пїЅ.js:3824

 createFromConfig           @            moment.26f1d83пїЅ.js:3791
createLocalOrUTC @ moment.26f1d83пїЅ.js:3885 

 createLocal         @            moment.26f1d83пїЅ.js:3889

 hooks    @            moment.26f1d83пїЅ.js:880

 (anonymous)     @            graph.b30d452пїЅ.js:180

 each      @            jquery-latest.js:2

 each      @            jquery-latest.js:2

 updateAllDateTimes       @            graph.b30d452пїЅ.js:178

 setDisplayedTimezone  @            graph.b30d452пїЅ.js:201
changDisplayedTimezone @ graph.b30d452пїЅ.js:286 

 initializeUITimezone       @            graph.b30d452пїЅ.js:378

 (anonymous)     @            graph.b30d452пїЅ.js:427

 e             @            jquery-latest.js:2

 t              @            jquery-latest.js:2

 setTimeout (async)

 (anonymous)     @            jquery-latest.js:2

 c              @            jquery-latest.js:2

 fireWith               @            jquery-latest.js:2

 fire         @            jquery-latest.js:2

 c              @            jquery-latest.js:2

 fireWith               @            jquery-latest.js:2

 ready    @            jquery-latest.js:2

 B             @            jquery-latest.js:2

 From: Jed Cunningham <[email protected] <mailto:[email protected]>>
 Sent: Monday, May 16, 2022 3:51 PM
 To: [email protected] <mailto:[email protected]>
 Subject: Re: Airflow 2.x graph view problem
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. 

 ________________________________



 Hey Laszlo,
Are there any errors in your browsers console? I can't think of anything off the top of my head, but if there is an error that may help us figure out what went wrong. 



 Jed



 ________________________________
IMPORTANT: The information contained in this email and/or its attachments is confidential and may only be used by the person to whom it is addressed. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this email and/or any attachment by an unintended recipient is strictly prohibited. Neither this email nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this email and/or any of its attachments are free of viruses. Any views expressed are personal to the author, except where specifically stated to be the views of SIG. SIG accepts no liability of any kind either for any errors arising as a result of electronic transmission or for any loss or damage which may be sustained by any person as a result of this email and/or its attachments being communicated to any person other than the intended recipient.
Susquehanna International Group Limited is a private company limited by shares and registered in Ireland. Registration No. 445356. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland
Susquehanna International Securities Limited is a private company limited by shares and registered in Ireland with the Companies Registration Office. Registration No. 337946. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland. Issued share capital: пїЅ2.00. Susquehanna International Securities Limited is regulated by the Central Bank of Ireland and is in the Temporary Permissions Regime in the UK.
Susquehanna International Securities Limited (UK Branch). Registration No. FC033877. Registered Establishment: Salesforce Tower /110 Part Floor 34 (West), 110 Bishopsgate, London, EC2N 4AY, UK. Susquehanna International Securities Limited (UK Branch) is regulated by the Financial Conduct Authority for conduct of business rules.
Liffeyview Property Holdings Limited is a private company limited by shares and registered in Ireland. Registration No. 534083. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland
Heights Capital Ireland, LLC is a branch registered in Ireland under Registration No. 908306 of Heights Capital Ireland, LLC incorporated in Delaware USA under Registration No. 5762692
SAL Trading, LLC is a branch registered in Ireland under Registration No. 906667 of SAL Trading, LLC incorporated in Delaware USA under Registration No. 4984826
If this email relates to investment services and activities as defined in Directive 2014/65/EU of the European Parliament and of the Council of 15 May 2014 on Markets in Financial Instruments (MiFID II), you are receiving the email from Susquehanna International Securities Limited. 


 ________________________________
IMPORTANT: The information contained in this email and/or its attachments is confidential and may only be used by the person to whom it is addressed. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this email and/or any attachment by an unintended recipient is strictly prohibited. Neither this email nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this email and/or any of its attachments are free of viruses. Any views expressed are personal to the author, except where specifically stated to be the views of SIG. SIG accepts no liability of any kind either for any errors arising as a result of electronic transmission or for any loss or damage which may be sustained by any person as a result of this email and/or its attachments being communicated to any person other than the intended recipient.
Susquehanna International Group Limited is a private company limited by shares and registered in Ireland. Registration No. 445356. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland
Susquehanna International Securities Limited is a private company limited by shares and registered in Ireland with the Companies Registration Office. Registration No. 337946. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland. Issued share capital: €2.00. Susquehanna International Securities Limited is regulated by the Central Bank of Ireland and is in the Temporary Permissions Regime in the UK.
Susquehanna International Securities Limited (UK Branch). Registration No. FC033877. Registered Establishment: Salesforce Tower /110 Part Floor 34 (West), 110 Bishopsgate, London, EC2N 4AY, UK. Susquehanna International Securities Limited (UK Branch) is regulated by the Financial Conduct Authority for conduct of business rules.
Liffeyview Property Holdings Limited is a private company limited by shares and registered in Ireland. Registration No. 534083. Registered Office Address: International Centre Memorial Road IFSC Dublin 1 Ireland
Heights Capital Ireland, LLC is a branch registered in Ireland under Registration No. 908306 of Heights Capital Ireland, LLC incorporated in Delaware USA under Registration No. 5762692
SAL Trading, LLC is a branch registered in Ireland under Registration No. 906667 of SAL Trading, LLC incorporated in Delaware USA under Registration No. 4984826
If this email relates to investment services and activities as defined in Directive 2014/65/EU of the European Parliament and of the Council of 15 May 2014 on Markets in Financial Instruments (MiFID II), you are receiving the email from Susquehanna International Securities Limited.

Reply via email to