[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329340=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329340
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 19:02
Start Date: 16/Oct/19 19:02
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542845583
 
 
   Filed https://issues.apache.org/jira/browse/BEAM-8414 to reenable missing 
checks, I don't think we need another issue.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329340)
Time Spent: 8h 50m  (was: 8h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329319=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329319
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 18:27
Start Date: 16/Oct/19 18:27
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542831863
 
 
   Thanks, @chadrik !
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329319)
Time Spent: 8h 40m  (was: 8.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329317=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329317
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 18:26
Start Date: 16/Oct/19 18:26
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329317)
Time Spent: 8.5h  (was: 8h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329314
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 18:25
Start Date: 16/Oct/19 18:25
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542831016
 
 
   Ready to go!
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329314)
Time Spent: 8h 20m  (was: 8h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329284=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329284
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 17:29
Start Date: 16/Oct/19 17:29
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542808340
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329284)
Time Spent: 8h 10m  (was: 8h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329279=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329279
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 17:25
Start Date: 16/Oct/19 17:25
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542806630
 
 
   Run CommunityMetrics PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329279)
Time Spent: 8h  (was: 7h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329273=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329273
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 17:18
Start Date: 16/Oct/19 17:18
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542803906
 
 
   LGTM, thank you.
   
   We can merge after tests pass.
   
   I can file remaining Jiras.
   
   > I think we should exclude this, because mypy is much better at determining 
structural problems, 
   and it will catch this kind of thing with far fewer false positives.
   Sounds good. We can certainly use mypy if that's the case. I would be 
conservative with excluding the rules unless:
   - We have frequent false-positives for a rule.
   - We have checked that mypy catches the errors of this kind.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329273)
Time Spent: 7h 40m  (was: 7.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329274=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329274
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 17:18
Start Date: 16/Oct/19 17:18
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542803906
 
 
   LGTM, thank you.
   
   We can merge after tests pass.
   
   I can file remaining Jiras.
   
   > I think we should exclude this, because mypy is much better at determining 
structural problems, 
   and it will catch this kind of thing with far fewer false positives.
   
   Sounds good. We can certainly use mypy if that's the case. I would be 
conservative with excluding the rules unless:
   - We have frequent false-positives for a rule.
   - We have checked that mypy catches the errors of this kind.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329274)
Time Spent: 7h 50m  (was: 7h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329252=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329252
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 16:48
Start Date: 16/Oct/19 16:48
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542792382
 
 
   Run CommunityMetrics PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329252)
Time Spent: 7.5h  (was: 7h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=329243=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-329243
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 16/Oct/19 16:40
Start Date: 16/Oct/19 16:40
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542789129
 
 
   Ok, the last set of notes has been addressed. 
   
   A note on this:
   
   > invalid-overridden-method - I think this is worth fixing, we can file a 
Jira + a make separate PR to be safe. I think AI here is to replace deprecated 
decorator @abc.abstractproperty in filesystemio.py.
   
   I think we should exclude this, because mypy is much better at determining 
structural problems, and it will catch this kind of thing with far fewer false 
positives.  In my mypy/typing branch, which is just about ready, I've had to 
turn off a few more pylint warnings along these lines, where they overlap into 
mypy's territory, but with less accuracy.
   
   Assuming you agree with that, we just need the 2 separate followup Jira 
issues:
   
   1) 
   
   > Exclude but add a newbie task 1 (may help somebody learn bits of python):
   >
   > - consider-using-set-comprehension
   > - chained-comparison
   > - consider-using-sys-exit
   
   2) 
   
   > Exclude but add a newbie task 2:
   >
   > - unnecessary-comprehension. We should mention that in some places we 
should use list (some_iter), in some just remove the comprehension. May be 
better to do this in a separate task to avoid a blind approval during review as 
some place need to be fixed differently.
   
   Would you like to make the issue 2 so that you can get the wording how you 
want it, or do you want me to make it?
   
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 329243)
Time Spent: 7h 20m  (was: 7h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328721=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328721
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 18:12
Start Date: 15/Oct/19 18:12
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542337427
 
 
   I went over the warnings in your last message, my suggestions below.
   
   Fix in this PR (most changes affect one file):
   - expression-not-assigned (recent test, can be fixed via _ = ...)
   - singleton-comparison 
   - comparisons-with-literal 
   
   Exclude inline:
   - using-constant-test - I don't get this warning. Consider excluding in-line 
+ file a linter bug.
   - comparison-with-callable - potentially useful, let's fix inline in 2 
places.
   
   Fix in a separate PR:
   - invalid-overridden-method - I think this is worth fixing, we can file a 
Jira + a make separate PR to be safe. I think AI here is to replace deprecated 
decorator   @abc.abstractproperty in filesystemio.py. 
   
   Exclude from lint (seem debatable/harmless) code style warnings:
   - no-else-break
   - unnecessary-pass
   - consider-using-in 
   - try-except-raise
   
   Exclude but add a newbie task 1 (may help somebody learn bits of python):
   - consider-using-set-comprehension
   - chained-comparison
   - consider-using-sys-exit
   
   Exclude but add a newbie task 2:
   - unnecessary-comprehension. We should mention that in some places we should 
use list (some_iter), in some just remove the comprehension. May be better to 
do this in a separate task to avoid a blind approval during review as some 
place need to be fixed differently.
   
   Not sure. Let's exclude for now:
   - stop-iteration-return.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328721)
Time Spent: 7h 10m  (was: 7h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328716=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328716
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 18:10
Start Date: 15/Oct/19 18:10
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-542337427
 
 
   I went over the warnings, my suggestions below.
   
   Fix in this PR (most changes affect one file):
   - expression-not-assigned (recent test, can be fixed via _ = ...)
   - singleton-comparison 
   - comparisons-with-literal 
   
   Exclude inline:
   - using-constant-test - I don't get this warning. Consider excluding in-line 
+ file a linter bug.
   - comparison-with-callable - potentially useful, let's fix inline in 2 
places.
   
   Fix in a separate PR:
   - invalid-overridden-method - I think this is worth fixing, we can file a 
Jira + a make separate PR to be safe. I think AI here is to replace deprecated 
decorator   @abc.abstractproperty in filesystemio.py. 
   
   Exclude from lint (seem debatable/harmless) code style warnings:
   - no-else-break
   - unnecessary-pass
   - consider-using-in 
   - try-except-raise
   
   Exclude but add a newbie task 1 (may help somebody learn bits of python):
   - consider-using-set-comprehension
   - chained-comparison
   - consider-using-sys-exit
   
   Exclude but add a newbie task 2:
   - unnecessary-comprehension. We should mention that in some places we should 
use list (some_iter), in some just remove the comprehension. May be better to 
do this in a separate task to avoid a blind approval during review as some 
place need to be fixed differently.
   
   Not sure. Let's exclude for now:
   - stop-iteration-return.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328716)
Time Spent: 6h 50m  (was: 6h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328719
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 18:10
Start Date: 15/Oct/19 18:10
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334716983
 
 

 ##
 File path: sdks/python/apache_beam/examples/complete/juliaset/setup.py
 ##
 @@ -31,7 +31,8 @@
 import subprocess
 from distutils.command.build import build as _build
 
-import setuptools
+# workaround pylint bug: https://github.com/PyCQA/pylint/issues/3152
 
 Review comment:
   How about we file an issue and add a comment:
   `# TODO(BEAM-...): re-enable lint check. `
   We can tag the issue as newbie/starter/trivial and it can be someone's first 
contribution to beam.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328719)
Time Spent: 7h  (was: 6h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328333
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 05:12
Start Date: 15/Oct/19 05:12
Worklog Time Spent: 10m 
  Work Description: chadrik commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334741855
 
 

 ##
 File path: sdks/python/.pylintrc
 ##
 @@ -85,28 +85,41 @@ disable =
   bad-builtin,
   bad-super-call,
   broad-except,
+  chained-comparison,
 
 Review comment:
   > How did you choose which to rules to permanently exclude and which to fix 
in this PR?
   
   That's a good question. 
   
   Obvious errors to exclude:
   
   - `useless-object-inheritance`: obviously we need to keep this until we drop 
python2 support
   -  `import-outside-toplevel`: scoped imports are pretty fundamental.  I 
don't even understand why they added this as an error
   
   Probably worth solving now:
   - `expression-not-assigned`: used in a few places in one test.  Could be 
worth making explicit exclusions for the test cases so that the problem is not 
spread elsewhere
   - `singleton-comparison`:  e.g. `x == None`.  this only occurs in a few 
places, so would be easy to squash now
   
   Could be added to a ticket for later:
   - `no-else-raise/break/continue`:  this pattern is pretty prevalent 
throughout the code.  could be worth fixing, but pretty time consuming.  good 
newbie task.
   
   Most of the remaining errors seemed like insignificant or debatable style 
suggestions.
   
   Here's the full output (excluding the first two).  Let me know if there are 
other errors you'd like to fix in this PR or add to a follow-up ticket.
   
   ```
   * Module apache_beam.metrics.metricbase
   apache_beam/metrics/metricbase.py:92:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.pvalue
   apache_beam/pvalue.py:195:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:201:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:574:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.metrics.execution_test
   apache_beam/metrics/execution_test.py:129:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:131:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:140:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:142:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   * Module apache_beam.options.pipeline_options
   apache_beam/options/pipeline_options.py:181:6: R1723: Unnecessary "elif" 
after "break" (no-else-break)
   apache_beam/options/pipeline_options.py:520:9: R1714: Consider merging these 
comparisons with "in" to "runner in ('DataflowRunner', 'TestDataflowRunner')" 
(consider-using-in)
   * Module apache_beam.pipeline
   apache_beam/pipeline.py:243:10: R1720: Unnecessary "elif" after "raise" 
(no-else-raise)
   apache_beam/pipeline.py:682:23: R1718: Consider using a set comprehension 
(consider-using-set-comprehension)
   apache_beam/pipeline.py:720:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:724:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:728:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:732:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.coders.coder_impl
   apache_beam/coders/coder_impl.py:405:9: R1714: Consider merging these 
comparisons with "in" to 't in (LIST_TYPE, TUPLE_TYPE, SET_TYPE)' 
(consider-using-in)
   apache_beam/coders/coder_impl.py:459:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:461:9: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:470:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:472:9: R0123: Comparison to literal 
(literal-comparison)
   * Module apache_beam.coders.coders
   apache_beam/coders/coders.py:192:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   apache_beam/coders/coders.py:198:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   * Module apache_beam.pipeline_test
   apache_beam/pipeline_test.py:690:8: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/pipeline_test.py:695:8: R1718: Consider using a set 
comprehension 

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328332=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328332
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 05:11
Start Date: 15/Oct/19 05:11
Worklog Time Spent: 10m 
  Work Description: chadrik commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334741855
 
 

 ##
 File path: sdks/python/.pylintrc
 ##
 @@ -85,28 +85,41 @@ disable =
   bad-builtin,
   bad-super-call,
   broad-except,
+  chained-comparison,
 
 Review comment:
   > How did you choose which to rules to permanently exclude and which to fix 
in this PR?
   
   That's a good question. 
   
   Obvious errors to exclude:
   
   - `useless-object-inheritance`: obviously we need to keep this until we drop 
python2 support
   -  `import-outside-toplevel`: scoped imports are pretty fundamental.  I 
don't even understand why they added this as an error
   
   Probably worth solving now:
   - `expression-not-assigned`: used in a few places in one test.  Could be 
worth making explicit exclusions for the test cases so that the problem is not 
spread elsewhere
   - `singleton-comparison`:  e.g. `x == None`.  this only occurs in a few 
places, so would be easy to squash now
   
   Could be added to a ticket for later:
   - `no-else-raise/break/continue`:  this pattern is pretty prevalent 
throughout the code.  could be worth fixing, but pretty time consuming.  good 
newbie task.
   
   Here's the full output (excluding the first two).  Let me know if there are 
other errors you'd like to fix in this PR or add to a follow-up ticket.
   
   ```
   * Module apache_beam.metrics.metricbase
   apache_beam/metrics/metricbase.py:92:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.pvalue
   apache_beam/pvalue.py:195:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:201:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:574:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.metrics.execution_test
   apache_beam/metrics/execution_test.py:129:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:131:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:140:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:142:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   * Module apache_beam.options.pipeline_options
   apache_beam/options/pipeline_options.py:181:6: R1723: Unnecessary "elif" 
after "break" (no-else-break)
   apache_beam/options/pipeline_options.py:520:9: R1714: Consider merging these 
comparisons with "in" to "runner in ('DataflowRunner', 'TestDataflowRunner')" 
(consider-using-in)
   * Module apache_beam.pipeline
   apache_beam/pipeline.py:243:10: R1720: Unnecessary "elif" after "raise" 
(no-else-raise)
   apache_beam/pipeline.py:682:23: R1718: Consider using a set comprehension 
(consider-using-set-comprehension)
   apache_beam/pipeline.py:720:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:724:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:728:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:732:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.coders.coder_impl
   apache_beam/coders/coder_impl.py:405:9: R1714: Consider merging these 
comparisons with "in" to 't in (LIST_TYPE, TUPLE_TYPE, SET_TYPE)' 
(consider-using-in)
   apache_beam/coders/coder_impl.py:459:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:461:9: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:470:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:472:9: R0123: Comparison to literal 
(literal-comparison)
   * Module apache_beam.coders.coders
   apache_beam/coders/coders.py:192:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   apache_beam/coders/coders.py:198:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   * Module apache_beam.pipeline_test
   apache_beam/pipeline_test.py:690:8: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/pipeline_test.py:695:8: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   * Module apache_beam.io.hadoopfilesystem
   

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328299=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328299
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 03:47
Start Date: 15/Oct/19 03:47
Worklog Time Spent: 10m 
  Work Description: chadrik commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334741855
 
 

 ##
 File path: sdks/python/.pylintrc
 ##
 @@ -85,28 +85,41 @@ disable =
   bad-builtin,
   bad-super-call,
   broad-except,
+  chained-comparison,
 
 Review comment:
   > How did you choose which to rules to permanently exclude and which to fix 
in this PR?
   
   That's a good question. 
   
   Obvious errors to exclude:
   
   - `useless-object-inheritance`: obviously we need to keep this until we get 
to python3-only
   -  `import-outside-toplevel`: scoped imports are pretty fundamental.  I 
don't even understand why they added this as an error
   
   Probably worth solving now:
   - `expression-not-assigned`: used in a few places in tests.  Could be worth 
making explicit exclusions for the test cases so that the problem is not spread 
elsewhere
   - `singleton-comparison`:  e.g. `x == None`.  this only occurs in a few 
places, so would be easy to squash now
   
   Could be added to a ticket for later:
   - `no-else-raise/break/continue`:  this pattern is pretty prevalent 
throughout the code.  could be worth fixing, but pretty time consuming.  good 
newbie task.
   
   Here's the full output (excluding the first two).  Let me know if there are 
other errors you'd like to fix in this PR or add to a follow-up ticket.
   
   ```
   * Module apache_beam.metrics.metricbase
   apache_beam/metrics/metricbase.py:92:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.pvalue
   apache_beam/pvalue.py:195:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:201:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pvalue.py:574:2: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.metrics.execution_test
   apache_beam/metrics/execution_test.py:129:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:131:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:140:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/metrics/execution_test.py:142:21: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   * Module apache_beam.options.pipeline_options
   apache_beam/options/pipeline_options.py:181:6: R1723: Unnecessary "elif" 
after "break" (no-else-break)
   apache_beam/options/pipeline_options.py:520:9: R1714: Consider merging these 
comparisons with "in" to "runner in ('DataflowRunner', 'TestDataflowRunner')" 
(consider-using-in)
   * Module apache_beam.pipeline
   apache_beam/pipeline.py:243:10: R1720: Unnecessary "elif" after "raise" 
(no-else-raise)
   apache_beam/pipeline.py:682:23: R1718: Consider using a set comprehension 
(consider-using-set-comprehension)
   apache_beam/pipeline.py:720:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:724:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:728:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   apache_beam/pipeline.py:732:4: W0107: Unnecessary pass statement 
(unnecessary-pass)
   * Module apache_beam.coders.coder_impl
   apache_beam/coders/coder_impl.py:405:9: R1714: Consider merging these 
comparisons with "in" to 't in (LIST_TYPE, TUPLE_TYPE, SET_TYPE)' 
(consider-using-in)
   apache_beam/coders/coder_impl.py:459:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:461:9: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:470:7: R0123: Comparison to literal 
(literal-comparison)
   apache_beam/coders/coder_impl.py:472:9: R0123: Comparison to literal 
(literal-comparison)
   * Module apache_beam.coders.coders
   apache_beam/coders/coders.py:192:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   apache_beam/coders/coders.py:198:4: R1720: Unnecessary "else" after "raise" 
(no-else-raise)
   * Module apache_beam.pipeline_test
   apache_beam/pipeline_test.py:690:8: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   apache_beam/pipeline_test.py:695:8: R1718: Consider using a set 
comprehension (consider-using-set-comprehension)
   * Module apache_beam.io.hadoopfilesystem
   

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328252=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328252
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 01:19
Start Date: 15/Oct/19 01:19
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334717631
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -33,7 +33,8 @@
 from pkg_resources import DistributionNotFound
 from pkg_resources import get_distribution
 from setuptools.command.build_py import build_py
-from setuptools.command.develop import develop
+# workaround pylint bug: https://github.com/PyCQA/pylint/issues/3152
 
 Review comment:
   Same TODO here.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328252)
Time Spent: 6h 10m  (was: 6h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328250=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328250
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 01:19
Start Date: 15/Oct/19 01:19
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334718345
 
 

 ##
 File path: sdks/python/.pylintrc
 ##
 @@ -85,28 +85,41 @@ disable =
   bad-builtin,
   bad-super-call,
   broad-except,
+  chained-comparison,
 
 Review comment:
   What is your perspective on these exceptions: do we need cleanup TODOs to 
fix some of them in the future? How did you choose which to rules to 
permanently exclude and which to fix in this PR?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328250)
Time Spent: 6h  (was: 5h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328251
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 15/Oct/19 01:19
Start Date: 15/Oct/19 01:19
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#discussion_r334716983
 
 

 ##
 File path: sdks/python/apache_beam/examples/complete/juliaset/setup.py
 ##
 @@ -31,7 +31,8 @@
 import subprocess
 from distutils.command.build import build as _build
 
-import setuptools
+# workaround pylint bug: https://github.com/PyCQA/pylint/issues/3152
 
 Review comment:
   How about we file an issue and add a comment:
   # TODO(BEAM-...): re-enable lint check. 
   We can tag the issue as newbie/starter/trivial and it can be someone's first 
contribution to beam.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328251)
Time Spent: 6h 10m  (was: 6h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=328179=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328179
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 14/Oct/19 22:10
Start Date: 14/Oct/19 22:10
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541951747
 
 
   This is ready for another round of review.  All tests are passing.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 328179)
Time Spent: 5h 50m  (was: 5h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327582
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 13/Oct/19 23:56
Start Date: 13/Oct/19 23:56
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541471931
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327582)
Time Spent: 5h 40m  (was: 5.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327394=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327394
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 21:40
Start Date: 12/Oct/19 21:40
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541363691
 
 
   Ah, I forgot that Jenkins merges in master before running tests. Rebased and 
fixed!
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327394)
Time Spent: 5.5h  (was: 5h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327392=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327392
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 21:05
Start Date: 12/Oct/19 21:05
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541361380
 
 
   Run CommunityMetrics PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327392)
Time Spent: 5h 20m  (was: 5h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327353
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 15:24
Start Date: 12/Oct/19 15:24
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541334636
 
 
   Ah whoops.  I missed that error message (and I stupidly assumed that I 
didn't change anything related to the flink test).
   
   That's fixed but now I have a very strange issue:
   
   ```
   11:14:17 > Task :sdks:python:test-suites:tox:py37:lintPy37
   11:14:17 * Module 
apache_beam.runners.portability.fn_api_runner_test
   11:14:17 apache_beam/runners/portability/fn_api_runner_test.py:1614:4: 
W1505: Using deprecated method assertRegexpMatches() (deprecated-method)
   ```
   
   This test passes locally for me.  I searched the entire codebase for 
`assertRegexpMatches` on this commit and it doesn't show up.  Not only that, my 
version of `fn_api_runner_test.py` doesn't even have 1614 lines.  I'm going to 
be offline for awhile, but I guess I'll try pushing a dummy commit. 
   
   
   
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327353)
Time Spent: 5h 10m  (was: 5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327344=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327344
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 15:11
Start Date: 12/Oct/19 15:11
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541333516
 
 
   Run PythonLint PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327344)
Time Spent: 5h  (was: 4h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327164=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327164
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 00:40
Start Date: 12/Oct/19 00:40
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541264631
 
 
   Console output says:
   ```
   22:15:08 
==
   22:15:08 ERROR: test_assert_that (__main__.FlinkRunnerTest)
   22:15:08 
--
   22:15:08 Traceback (most recent call last):
   22:15:08   File "apache_beam/runners/portability/fn_api_runner_test.py", 
line 94, in test_assert_that
   22:15:08 with self.assertRaisesRegex(Exception, 'Failed assert'):
   22:15:08 AttributeError: 'FlinkRunnerTest' object has no attribute 
'assertRaisesRegex'
   22:15:08 
   22:15:08 
==
   22:15:08 ERROR: test_assert_that (__main__.FlinkRunnerTestOptimized)
   22:15:08 
--
   22:15:08 Traceback (most recent call last):
   22:15:08   File "apache_beam/runners/portability/fn_api_runner_test.py", 
line 94, in test_assert_that
   22:15:08 with self.assertRaisesRegex(Exception, 'Failed assert'):
   22:15:08 AttributeError: 'FlinkRunnerTestOptimized' object has no attribute 
'assertRaisesRegex'
   22:15:08 
   22:15:08 
--
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327164)
Time Spent: 4h 50m  (was: 4h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327163=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327163
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 12/Oct/19 00:38
Start Date: 12/Oct/19 00:38
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541264432
 
 
   > Ok, I can make a "mini" python2 lint job and restore the 2to3 check. 
   Sounds good.
   
   > Does anyone have any idea why Python2_PVR_Flink keeps failing? It's failed 
7 times in a row.
   
   Sorry to hear about this. My typical mode of operation in this situation is 
to:
   - Open test error message, search for the error in my inbox or Jira, to see 
if there is an open issue. 
   - If there is no issue, open an issue in test-failures component and add 
currently-failing label.
   - If owner is not obvious, ask on the mailing list who relevant people are.
   - If I have enough context & some time, try to fix the error.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327163)
Time Spent: 4h 40m  (was: 4.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327144
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 23:48
Start Date: 11/Oct/19 23:48
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541258257
 
 
   Does anyone have any idea why Python2_PVR_Flink keeps failing?  It's failed 
7 times in a row. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327144)
Time Spent: 4.5h  (was: 4h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327143=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327143
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 23:47
Start Date: 11/Oct/19 23:47
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541258151
 
 
   @tvalentyn 
   
   Ok, I can make a "mini" python2 lint job and restore the 2to3 check.  That 
basically puts us back to where we were, but swaps the "primary" and "mini" 
roles between python versions.  That seems reasonable.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327143)
Time Spent: 4h 20m  (was: 4h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327101=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327101
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 22:30
Start Date: 11/Oct/19 22:30
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541241040
 
 
   From development perspective, it would be easiest to run linter using the 
highest version we support (for example, no need to exclude tests using new 
syntax)
   
   From compatibility perspective, it would be more reliable to run the linter 
using the lowest version we support. The argument that compatibility errors 
will be caught by tests is a reasonable argument, however I am not sure that 
all modules are covered by tests. Advantage of linter is that it parses all the 
codebase and will be able to spot incompatible syntax. We can consider running 
python2 linter ignoring all/most linter warnings, but still checking that 
linter is able to parse the files. Unfortunately, we would have to keep some 
files excluded if we do so. I think this is the price we have to pay to ensure 
we support older version of python. 
   
   Also, I think syntax compatibility problem will remain beyond Python 2, as 
new minor versions of Python 3 may introduce incompatible syntactic features. 
Do we want to make sure that Beam core modules do not use such features? I 
think the answer is yes. Can we guarantee that this will be enforced by tests? 
I think not until we enforce code coverage criteria in precommits. 
   
   Python 2/3 compatibility checks also add value, and in my opinion should be 
kept while Beam supports Python 2. For example, such checks currently catch  
division statements without  `from __future__ import division`. These can cause 
subtle errors that in the past caused performance regressions 
(https://issues.apache.org/jira/browse/BEAM-4858).  
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327101)
Time Spent: 4h 10m  (was: 4h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327096
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 22:10
Start Date: 11/Oct/19 22:10
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541241040
 
 
   From development perspective, it would be easiest to run linter using the 
highest version we support (for example, no need to exclude tests using new 
syntax)
   
   From compatibility perspective, it would be more reliable to run the linter 
using the lowest version we support. The argument that compatibility errors 
will be caught by tests is a reasonable argument, however I am not sure that 
all modules are covered by tests. Advantage of linter is that it parses all the 
codebase and will be able to spot incompatible syntax. We can consider running 
python2 linter ignoring all/most linter warnings, but still checking that 
linter is able to parse the files? Unfortunately, we would have to keep some 
files excluded if we do so. I think this is the price we have to pay to ensure 
we support older version of python. 
   
   Also, I think syntax compatibility problem will remain beyond Python 2, as 
new minor versions of Python 3 may introduce incompatible syntactic features. 
Do we want to make sure that Beam core modules do not use such features? I 
think the answer is yes. Can we guarantee that this will be enforced by tests? 
I think not until we enforce code coverage criteria in precommits. 
   
   Python 2/3 compatibility checks also add value, and in my opinion should be 
kept while Beam supports Python 2. For example, such checks currently catch  
division statements without  `from __future__ import division`. These can cause 
subtle errors that in the past caused performance regressions 
(https://issues.apache.org/jira/browse/BEAM-4858).  
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327096)
Time Spent: 4h  (was: 3h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327091=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327091
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 21:59
Start Date: 11/Oct/19 21:59
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541238337
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327091)
Time Spent: 3h 50m  (was: 3h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327088=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327088
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 21:57
Start Date: 11/Oct/19 21:57
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541237653
 
 
   > Re: Py 2 deprecation date - I am not making a statement here on when. My 
argument was it has not happened yet, earliest it will happen in early 2020, 
and I would like to keep py2 linting support as long as py2 is supported in the 
codebase.
   
   Ah, sorry I misunderstood.  Yes, I agree we need to continue fully 
supporting python2 linting/tests for as long as the codebase supports python2.
   
   > Is the following correct for this change:
   > 
   > All py2 syntax errors will still be caught.
   
   yes, by virtue of the unittests.
   
   > There might be linting issues specific to py2 linter that might be missed 
but that is not harmful.
   
   Not quite.  I'm saying that, outside of syntax changes, there's not really a 
such thing as py2-specific linter issues.  I may be overlooking something here, 
so I welcome anyone fact checking me on this.  One thing that comes to mind is 
errors related to modules that were moved or renamed between python 2 and 3, 
but I did a quick check and I don't think pylint warns about this. 
   
   Lint rules are mostly about style, and Beam's style is the same on python 2 
as it is in python 3.  Outside of style, there are structural checks which 
apply equally to python 2 and 3.  So, since the lint rules are independent of 
python version (excluding syntax changes), nothing changes by virtue of running 
it in python3. 
   
   And lastly, there are additional style enforcements that can only be checked 
from python3 (like type annotations), hence this PR.  You'll notice that in 
this PR I had to fix some style issues in our `_py3*` suffixed files, because 
these were not being checked before, so that's another plus about this PR!
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327088)
Time Spent: 3h 40m  (was: 3.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=327030=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327030
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 18:27
Start Date: 11/Oct/19 18:27
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #9725: [BEAM-8350] Upgrade to 
Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541173859
 
 
   Re: Py 2 deprecation date - I am not making a statement here on when. My 
argument was it has not happened yet, earliest it will happen in early 2020, 
and I would like to keep py2 linting support as long as py2 is supported in the 
codebase.
   
   I understand your argument. Is the following correct for this change:
   - All py2 syntax errors will still be caught.
   - There might be linting issues specific to py2 linter that might be missed 
but that is not harmful.
   
   I guess for linting we need a target version in mind. We cannot potentially 
lint the same code base for different python versions with different rules.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327030)
Time Spent: 3.5h  (was: 3h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326995
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 17:01
Start Date: 11/Oct/19 17:01
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541142817
 
 
   My argument here is that there is no practical difference to the current
   approach and this new approach, other than the catching of syntax errors
   which will be easily caught by unit tests.  Besides, there were syntax
   changes made in 3.0, 3.5 and 3.6 (and probably other versions that I’m not
   aware of) and we are not currently linting these versions separately so we
   aren’t effectively using linting to catch syntax problems now anyway (other 
than 2.x vs all of 3.x).  We
   *are* testing minor versions separately with our unittests, so that will
   catch syntax errors per version.
   
   
   On Fri, Oct 11, 2019 at 12:30 PM Ahmet Altay 
   wrote:
   
   > *@aaltay* commented on this pull request.
   >
   > High level question, would it be better to keep py2/3 lint covered until
   > py2 is officially not supported in the beam code base? Currently we do not
   > exactly know when this will happen and it seems like it will be around
   > December/January.
   >
   > Or is it possible to keep 2 lint targets?
   >
   > /cc @tvalentyn 
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326995)
Time Spent: 3h 20m  (was: 3h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326993=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326993
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 17:00
Start Date: 11/Oct/19 17:00
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541144228
 
 
   Also, I thought the date of removing python 2 support was still very much up 
for debate, with the only decided target being “sometime in 2020”.  The 
download stats from the thread on this subject indicate that the majority of 
users are still using beam on 2.x, including my company.  I will personally be 
quite disappointed if Beam drops 2.x support so quickly and if you do so you 
will almost certainly cut me off from making future contributions. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326993)
Time Spent: 3h 10m  (was: 3h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326990=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326990
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 16:56
Start Date: 11/Oct/19 16:56
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-541142817
 
 
   My argument here is that there is no practical difference to the current
   approach and this new approach, other than the catching of syntax errors
   which will be easily caught by unit tests.  Besides, there were syntax
   changes made in 3.0, 3.5 and 3.6 (and probably other versions that I’m not
   aware of) and we are not currently linting these versions separately so we
   aren’t effectively using linting to catch syntax problems now anyway.  We
   *are* testing minor versions separately with our unittests, so that will
   catch syntax errors per version.
   
   
   On Fri, Oct 11, 2019 at 12:30 PM Ahmet Altay 
   wrote:
   
   > *@aaltay* commented on this pull request.
   >
   > High level question, would it be better to keep py2/3 lint covered until
   > py2 is officially not supported in the beam code base? Currently we do not
   > exactly know when this will happen and it seems like it will be around
   > December/January.
   >
   > Or is it possible to keep 2 lint targets?
   >
   > /cc @tvalentyn 
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326990)
Time Spent: 3h  (was: 2h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326679=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326679
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 02:58
Start Date: 11/Oct/19 02:58
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced style warnings that did not seem 
important
   - run the lint using python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was just testing futurization.  seems fine to do 
this all the time now that our code is python3 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326679)
Time Spent: 2h 50m  (was: 2h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326678=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326678
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 02:57
Start Date: 11/Oct/19 02:57
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced style warnings that did not seem 
important
   - run the lint using python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was just testing futurization.  seems fine to do 
this all the time now that our code is python3 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326678)
Time Spent: 2h 40m  (was: 2.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=326677=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326677
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 11/Oct/19 02:54
Start Date: 11/Oct/19 02:54
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-540878722
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 326677)
Time Spent: 2.5h  (was: 2h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=325395=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325395
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 09/Oct/19 00:10
Start Date: 09/Oct/19 00:10
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-539755932
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325395)
Time Spent: 2h 10m  (was: 2h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=325396=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325396
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 09/Oct/19 00:10
Start Date: 09/Oct/19 00:10
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-539756014
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325396)
Time Spent: 2h 20m  (was: 2h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323839
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 05/Oct/19 01:37
Start Date: 05/Oct/19 01:37
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538605375
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323839)
Time Spent: 2h  (was: 1h 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323602=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323602
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 17:38
Start Date: 04/Oct/19 17:38
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced style warnings that did not seem 
important
   - run the lint using python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was just testing futurization.  seems fine to do 
this all the time now that our code is python2 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323602)
Time Spent: 1h 50m  (was: 1h 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323592
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 17:24
Start Date: 04/Oct/19 17:24
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538488002
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323592)
Time Spent: 1.5h  (was: 1h 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323593=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323593
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 17:24
Start Date: 04/Oct/19 17:24
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538488071
 
 
   Python2_PVR_Flink seems incredibly flaky
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323593)
Time Spent: 1h 40m  (was: 1.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323502=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323502
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 15:59
Start Date: 04/Oct/19 15:59
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced style warnings that did not seem 
important
   - run the lint on python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was a test just for testing the futurization.  
seems fine to do this all the time now that our code is python2 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323502)
Time Spent: 1h 10m  (was: 1h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323503=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323503
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 15:59
Start Date: 04/Oct/19 15:59
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced style warnings that did not seem 
important
   - run the lint on python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was just testing futurization.  seems fine to do 
this all the time now that our code is python2 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323503)
Time Spent: 1h 20m  (was: 1h 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323498=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323498
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 15:58
Start Date: 04/Oct/19 15:58
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538458010
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323498)
Time Spent: 1h  (was: 50m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323497=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323497
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 04/Oct/19 15:58
Start Date: 04/Oct/19 15:58
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538457947
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323497)
Time Spent: 50m  (was: 40m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323074
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 03/Oct/19 23:45
Start Date: 03/Oct/19 23:45
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced error codes that did not seem important
   - run the lint on python-3.7: this ensures that it can run on test files 
that only work on python-37 due to syntax features
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was a test just for testing the futurization.  
seems fine to do this all the time now that our code is python2 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323074)
Time Spent: 40m  (was: 0.5h)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323057
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 03/Oct/19 23:37
Start Date: 03/Oct/19 23:37
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538169256
 
 
   Here's a breakdown of the changes required to get to pylint 2.4:
   
   - fix a bunch of warnings about deprecated methods.  mostly `logger.warn` 
and various unittest methods
   - update the names of a few error codes: `disable=unused-import` and 
`possibly-unused-variable`
   - ignore a bunch of newly introduced error codes that did not seem important
   - merge the lint tests into one test:
 - `run_pylint_2to3.sh` was a test just for testing the futurization.  
seems fine to do this all the time now that our code is python2 compliant
 - there was a "mini" test just for python3-compatibility.  not needed 
anymore now that everything is running on python3
 - stop running `pycodestyle`: it's run as part of `flake8`
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323057)
Time Spent: 0.5h  (was: 20m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323049=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323049
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 03/Oct/19 23:28
Start Date: 03/Oct/19 23:28
Worklog Time Spent: 10m 
  Work Description: chadrik commented on issue #9725: [BEAM-8350] Upgrade 
to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725#issuecomment-538167314
 
 
   R: @robertwb 
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323049)
Time Spent: 20m  (was: 10m)

> Upgrade to pylint 2.4
> -
>
> Key: BEAM-8350
> URL: https://issues.apache.org/jira/browse/BEAM-8350
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Chad Dombrova
>Assignee: Chad Dombrova
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> pylint 2.4 provides a number of new features and fixes, but the most 
> important/pressing one for me is that 2.4 adds support for understanding 
> python type annotations, which fixes a bunch of spurious unused import errors 
> in the PR I'm working on for BEAM-7746.
> As of 2.0, pylint dropped support for running tests in python2, so to make 
> the upgrade we have to move our lint jobs to python3.  Doing so will put 
> pylint into "python3-mode" and there is not an option to run in 
> python2-compatible mode.  That said, the beam code is intended to be python3 
> compatible, so in practice, performing a python3 lint on the Beam code-base 
> is perfectly safe.  The primary risk of doing this is that someone introduces 
> a python-3 only change that breaks python2, but these would largely be syntax 
> errors that would be immediately caught by the unit and integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-8350) Upgrade to pylint 2.4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-8350?focusedWorklogId=323047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323047
 ]

ASF GitHub Bot logged work on BEAM-8350:


Author: ASF GitHub Bot
Created on: 03/Oct/19 23:27
Start Date: 03/Oct/19 23:27
Worklog Time Spent: 10m 
  Work Description: chadrik commented on pull request #9725: [BEAM-8350] 
Upgrade to Pylint 2.4
URL: https://github.com/apache/beam/pull/9725
 
 
   pylint 2.4 provides a number of new features and fixes, but the most 
important/pressing one for me is that 2.4 adds support for understanding python 
type annotations, which fixes a bunch of spurious unused import errors in the 
PR I'm working on for BEAM-7746.
   
   As of 2.0, pylint dropped support for running tests in python2, so to make 
the upgrade we have to move our lint jobs to python3. Doing so will put pylint 
into "python3-mode" and there is not an option to run in python2-compatible 
mode. That said, the beam code is intended to be python3 compatible, so in 
practice, performing a python3 lint on the Beam code-base is perfectly safe. 
The primary risk of doing this is that someone introduces a python-3 only 
change that breaks python2, but these would largely be syntax errors that would 
be immediately caught by the unit and integration tests.
   
   
   
   Thank you for your contribution! Follow this checklist to help us 
incorporate your contribution quickly and easily:
   
- [ ] [**Choose 
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and 
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build