[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=85328=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-85328
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 28/Mar/18 17:16
Start Date: 28/Mar/18 17:16
Worklog Time Spent: 10m 
  Work Description: asfgit closed pull request #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/run_mini_py3lint.sh b/sdks/python/run_mini_py3lint.sh
index f5e9acd70b3..c4bd37fbb6c 100755
--- a/sdks/python/run_mini_py3lint.sh
+++ b/sdks/python/run_mini_py3lint.sh
@@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
+flake8 $MODULE --count --select=E9,F822,F823 --show-source --statistics
diff --git a/sdks/python/run_pylint.sh b/sdks/python/run_pylint.sh
index e3f9a8e267f..93f68d40a73 100755
--- a/sdks/python/run_pylint.sh
+++ b/sdks/python/run_pylint.sh
@@ -63,7 +63,8 @@ pylint -j8 "$MODULE" --ignore-patterns="$FILES_TO_IGNORE"
 echo "Running pycodestyle for module $MODULE:"
 pycodestyle "$MODULE" --exclude="$FILES_TO_IGNORE"
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
+flake8 $MODULE --count --select=E9,F822,F823 --show-source --statistics
 
 echo "Running isort for module $MODULE:"
 # Skip files where isort is behaving weirdly


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 85328)
Time Spent: 14h 50m  (was: 14h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 14h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=85264=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-85264
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 28/Mar/18 13:30
Start Date: 28/Mar/18 13:30
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-376887187
 
 
   https://issues.apache.org/jira/browse/BEAM-3959


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 85264)
Time Spent: 14h 40m  (was: 14.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 14h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=85017=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-85017
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 20:42
Start Date: 27/Mar/18 20:42
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177565468
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   It is fine to move that sentence to a JIRA, and add a todo referencing that 
JIRA. People reading that TODO once it is merged would not easily be able to 
find the PR and understand what is the todo for, and what needs to happen to 
resolve it.
   
   You can also reference the issues you have noticed with F821 as you 
mentioned in your other comment in that same JIRA. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 85017)
Time Spent: 14.5h  (was: 14h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 14.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84998=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84998
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 19:43
Start Date: 27/Mar/18 19:43
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177548977
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   Please reread the first sentence of description above which discusses the 
"showstopper" flake8 issues vs. the "style violations".  I have been focused on 
the former for the reasons described above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84998)
Time Spent: 14h 20m  (was: 14h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 14h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84997=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84997
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 19:42
Start Date: 27/Mar/18 19:42
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177548977
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   Please reread the first sentence of description above which discusses the 
"showstopper" flake8 issues vs. the "style violations".  I have been focused on 
the former for the reasons described above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84997)
Time Spent: 14h 10m  (was: 14h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 14h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84996=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84996
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 19:41
Start Date: 27/Mar/18 19:41
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-376649775
 
 
   F821 status (there are PRs open for most of these)...
   
   $ __python3 -m flake8 . --count --select=E9,F821,F822,F823 --show-source 
--statistics__
   ```
   ./sdks/python/apache_beam/io/vcfio_test.py:77:14: F821 undefined name 'cmp'
 return cmp(v1.end, v2.end)
^
   ./sdks/python/apache_beam/io/vcfio_test.py:78:12: F821 undefined name 'cmp'
   return cmp(v1.start, v2.start)
  ^
   ./sdks/python/apache_beam/io/vcfio_test.py:79:10: F821 undefined name 'cmp'
 return cmp(v1.reference_name, v2.reference_name)
^
   ./sdks/python/apache_beam/io/avroio.py:313:34: F821 undefined name 'buffer'
 result = snappy.decompress(buffer(data)[:-4])
^
   ./sdks/python/apache_beam/io/gcp/datastore/v1/helper.py:93:12: F821 
undefined name 'cmp'
 result = cmp(p1.kind, p2.kind)
  ^
   ./sdks/python/apache_beam/io/gcp/datastore/v1/helper.py:101:12: F821 
undefined name 'cmp'
   return cmp(p1.id, p2.id)
  ^
   ./sdks/python/apache_beam/io/gcp/datastore/v1/helper.py:106:10: F821 
undefined name 'cmp'
 return cmp(p1.name, p2.name)
^
   ./sdks/python/apache_beam/utils/windowed_value.py:196:14: F821 undefined 
name 'cmp'
 return cmp(type(left), type(right))
^
   ./sdks/python/apache_beam/utils/timestamp.py:100:12: F821 undefined name 
'cmp'
   return cmp(self.micros, other.micros)
  ^
   ./sdks/python/apache_beam/utils/timestamp.py:178:12: F821 undefined name 
'cmp'
   return cmp(self.micros, other.micros)
  ^
   ./sdks/python/apache_beam/testing/test_stream.py:51:14: F821 undefined name 
'cmp'
 return cmp(type(self), type(other))
^
   ./sdks/python/apache_beam/testing/test_stream.py:66:12: F821 undefined name 
'cmp'
   return cmp(self.timestamped_values, other.timestamped_values)
  ^
   ./sdks/python/apache_beam/testing/test_stream.py:76:12: F821 undefined name 
'cmp'
   return cmp(self.new_watermark, other.new_watermark)
  ^
   ./sdks/python/apache_beam/testing/test_stream.py:86:12: F821 undefined name 
'cmp'
   return cmp(self.advance_by, other.advance_by)
  ^
   ./sdks/python/apache_beam/transforms/window.py:195:12: F821 undefined name 
'cmp'
   return cmp(self.end, other.end) or cmp(hash(self), hash(other))
  ^
   ./sdks/python/apache_beam/transforms/window.py:195:40: F821 undefined name 
'cmp'
   return cmp(self.end, other.end) or cmp(hash(self), hash(other))
  ^
   ./sdks/python/apache_beam/transforms/window.py:250:14: F821 undefined name 
'cmp'
 return cmp(type(self), type(other))
^
   ./sdks/python/apache_beam/transforms/window.py:251:12: F821 undefined name 
'cmp'
   return cmp((self.value, self.timestamp), (other.value, other.timestamp))
  ^
   ./sdks/python/apache_beam/runners/portability/fn_api_runner_test.py:103:11: 
F821 undefined name 'MetricKey'
 MetricKey('mydofn',
 ^
   ./sdks/python/apache_beam/runners/portability/fn_api_runner_test.py:104:21: 
F821 undefined name 'MetricName'
   MetricName('ns1', 'elements')),
   ^
   ./sdks/python/apache_beam/runners/portability/fn_api_runner_test.py:105:11: 
F821 undefined name 'MetricKey'
 MetricKey('myotherdofn',
 ^
   ./sdks/python/apache_beam/runners/portability/fn_api_runner_test.py:106:21: 
F821 undefined name 'MetricName'
   MetricName('ns2', 'elementsplusone'))])
   ^
   ./sdks/python/apache_beam/runners/worker/statesampler_slow.py:54:12: F821 
undefined name 'StateSamplerInfo'
   return StateSamplerInfo(
  ^
   23F821 undefined name 'buffer'
   23
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84996)
Time Spent: 14h  (was: 13h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> 

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84986
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 19:21
Start Date: 27/Mar/18 19:21
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177520199
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   All showstopper issue tests pass __except__ for F821 which is what makes it 
special.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84986)
Time Spent: 13h 50m  (was: 13h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84982
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 19:19
Start Date: 27/Mar/18 19:19
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177542654
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   Please reread the first sentence of description above which discusses the 
"showstopper" flake8 issues vs. the "style violations".  I have been focused on 
the former for the reasons described above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84982)
Time Spent: 13h 40m  (was: 13.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84971
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 18:26
Start Date: 27/Mar/18 18:26
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177525760
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   There are other errors/violation codes that are not enabled including F821: 
http://flake8.pycqa.org/en/latest/user/error-codes.html


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84971)
Time Spent: 13.5h  (was: 13h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84967=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84967
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 18:09
Start Date: 27/Mar/18 18:09
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177520199
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   All tests pass __except__ for F821 which is what makes it special.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84967)
Time Spent: 13h 20m  (was: 13h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=84950=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84950
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 27/Mar/18 17:30
Start Date: 27/Mar/18 17:30
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4798: 
[BEAM-3738] Add more flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#discussion_r177508107
 
 

 ##
 File path: sdks/python/run_mini_py3lint.sh
 ##
 @@ -37,4 +37,5 @@ if test $# -gt 0; then
 fi
 
 echo "Running flake8 for module $MODULE:"
-flake8 $MODULE --count --select=E999 --show-source --statistics
+# TODO: Add F821 (undefined names) as soon as that test passes
 
 Review comment:
   Which test passes? Do we have a JIRA tracking this?
   
   Also why F821 is special compared to any other warning to be mentioned in 
this todo?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 84950)
Time Spent: 13h 10m  (was: 13h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83896=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83896
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 24/Mar/18 00:48
Start Date: 24/Mar/18 00:48
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-375833548
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83896)
Time Spent: 13h  (was: 12h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 13h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83895=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83895
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 24/Mar/18 00:47
Start Date: 24/Mar/18 00:47
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-375833548
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83895)
Time Spent: 12h 50m  (was: 12h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 12h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83759=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83759
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:50
Start Date: 23/Mar/18 18:50
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-375765483
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83759)
Time Spent: 12h 40m  (was: 12.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 12h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83758
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:49
Start Date: 23/Mar/18 18:49
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176833151
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   pip-compile can generate requirements.txt from setup.py, and the docker 
image can pull that in.
   The only problem is that we have a different set of requirements depending 
on the extras specified (such as [test] and [gcp]), or whether cython is used.
   So the solution is probably to use a script to generate a set of 
requirements files, using pip-compile output as an intermediate step. setup.py 
will be used as the single source of truth for version pinning.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83758)
Time Spent: 12.5h  (was: 12h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 12.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83757
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:43
Start Date: 23/Mar/18 18:43
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-375763430
 
 
   Now that #4877 is merged, can we get another review on these changes to 
avoid backslides?  @udim 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83757)
Time Spent: 12h 20m  (was: 12h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 12h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83749
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:36
Start Date: 23/Mar/18 18:36
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #4877: [BEAM-3738] Enable 
py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/typehints/typehints.py 
b/sdks/python/apache_beam/typehints/typehints.py
index 543641526f5..af9e1fef70b 100644
--- a/sdks/python/apache_beam/typehints/typehints.py
+++ b/sdks/python/apache_beam/typehints/typehints.py
@@ -1127,7 +1127,6 @@ def coerce_to_kv_type(element_type, label=None):
 Union[tuple(t.tuple_types[1] for t in union_types)]]
   else:
 # TODO: Possibly handle other valid types.
-print "element_type", element_type
 raise ValueError(
 "Input to %r must be compatible with KV[Any, Any]. "
 "Found %s." % (label, element_type))
diff --git a/sdks/python/build.gradle b/sdks/python/build.gradle
index 484adb3629e..2c6637a5373 100644
--- a/sdks/python/build.gradle
+++ b/sdks/python/build.gradle
@@ -64,7 +64,7 @@ build.dependsOn buildPython
 task lint (dependsOn: 'setupTest') {
   doLast {
 exec {
-  commandLine 'tox', '-e', 'lint', '-c', 'tox.ini'
+  commandLine 'tox', '-e', '{py27,py3}-lint', '-c', 'tox.ini'
 }
   }
 }
@@ -74,7 +74,7 @@ check.dependsOn lint
 task testCython (dependsOn: ['setupTest', 'testPython', 'testGcp']) {
   doLast {
 exec {
-  commandLine 'tox', '-e', 'py27cython', '-c', 'tox.ini'
+  commandLine 'tox', '-e', 'py27-cython2', '-c', 'tox.ini'
 }
   }
 }
@@ -83,7 +83,7 @@ test.dependsOn testCython
 task testGcp (dependsOn: 'setupTest') {
   doLast {
 exec {
-  commandLine 'tox', '-e', 'py27gcp', '-c', 'tox.ini'
+  commandLine 'tox', '-e', 'py27-gcp', '-c', 'tox.ini'
 }
   }
 }
diff --git a/sdks/python/run_tox_cleanup.sh b/sdks/python/run_tox_cleanup.sh
new file mode 100755
index 000..76b683cf588
--- /dev/null
+++ b/sdks/python/run_tox_cleanup.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+#
+#Licensed to the Apache Software Foundation (ASF) under one or more
+#contributor license agreements.  See the NOTICE file distributed with
+#this work for additional information regarding copyright ownership.
+#The ASF licenses this file to You under the Apache License, Version 2.0
+#(the "License"); you may not use this file except in compliance with
+#the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+
+# This script is used to remove generated files in previous Tox runs so that
+# they are not picked up by tests.
+
+set -e
+
+for dir in apache_beam target/build; do
+if [ -d "${dir}" ]; then
+for ext in pyc c so; do
+find ${dir} -type f -name "*.${ext}" -delete
+done
+fi
+done
diff --git a/sdks/python/setup.py b/sdks/python/setup.py
index 02c02f56c42..ffc4df78413 100644
--- a/sdks/python/setup.py
+++ b/sdks/python/setup.py
@@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
-]
-
 REQUIRED_TEST_PACKAGES = [
+'nose>=1.3.7',
 'pyhamcrest>=1.9,<2.0',
 ]
 
@@ -177,6 +174,9 @@ def generate_common_urns():
 + '\n')
 generate_common_urns()
 
+python_requires = '>=2.7'
+if os.environ.get('BEAM_EXPERIMENTAL_PY3') is None:
+  python_requires += ',<3.0'
 
 setuptools.setup(
 name=PACKAGE_NAME,
@@ -202,9 +202,8 @@ def generate_common_urns():
 'apache_beam/utils/counters.py',
 'apache_beam/utils/windowed_value.py',
 ]),
-setup_requires=REQUIRED_SETUP_PACKAGES,
 install_requires=REQUIRED_PACKAGES,
-python_requires='>=2.7,<3.0',
+python_requires=python_requires,
 test_suite='nose.collector',
 tests_require=REQUIRED_TEST_PACKAGES,
 extras_require={
diff --git a/sdks/python/tox.ini b/sdks/python/tox.ini
index 857e7b0e571..add9bbfb2cc 100644
--- a/sdks/python/tox.ini
+++ b/sdks/python/tox.ini
@@ -17,8 +17,7 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added 

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83748=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83748
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:35
Start Date: 23/Mar/18 18:35
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176829348
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
-]
-
 REQUIRED_TEST_PACKAGES = [
+'nose>=1.3.7',
 
 Review comment:
   Sounds good.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83748)
Time Spent: 12h  (was: 11h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 12h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83747
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:35
Start Date: 23/Mar/18 18:35
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176829272
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   That is fair to not change the version in this PR. We should consider 
upgrading it.
   
   It would be nice to have a single source of truth. Although I do not see how 
pip-compile can update the Dockerfile. It is worth filing a JIRA for that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83747)
Time Spent: 11h 50m  (was: 11h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83744
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:31
Start Date: 23/Mar/18 18:31
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176827405
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   +1 on a single __requirements.txt__ file.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83744)
Time Spent: 11h 40m  (was: 11.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83742
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:29
Start Date: 23/Mar/18 18:29
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176827577
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
-]
-
 REQUIRED_TEST_PACKAGES = [
+'nose>=1.3.7',
 
 Review comment:
   I also wanted to merge requirements. Now only setup.py has the nose 
requirement.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83742)
Time Spent: 11h 20m  (was: 11h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83743
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:29
Start Date: 23/Mar/18 18:29
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175250556
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Does it make sense to raise __REQUIRED_PIP_VERSION = '7.0.0'__ to something 
a closer to the current 9.0.3 or do we leave that alone for now?  
Alternatively, should the process do __pip install --upgrade pip__?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83743)
Time Spent: 11.5h  (was: 11h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83741=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83741
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:28
Start Date: 23/Mar/18 18:28
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176827405
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   +1 on a single __requirement.txt__ file.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83741)
Time Spent: 11h 10m  (was: 11h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83737=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83737
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:23
Start Date: 23/Mar/18 18:23
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176825065
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   This is the minimum version required by `setup.py` 
(`REQUIRED_CYTHON_VERSION`). I didn't want to increase the version beyond that 
and risk breaking things in this PR.
   
   Side note: `sdks/python/container/Dockerfile` specifies `0.27.2`. *shrug*
   I wish all these pinned versions were synced to a single requirements.txt 
file. (see [pip-compile](https://pypi.python.org/pypi/pip-tools))


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83737)
Time Spent: 11h  (was: 10h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83739=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83739
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:23
Start Date: 23/Mar/18 18:23
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176823318
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
 
 Review comment:
   No bleeding clue. Since I don't have ssh access I can't really debug it.
   Honestly I'm glad I got it working.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83739)
Time Spent: 11h  (was: 10h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83738=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83738
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 18:23
Start Date: 23/Mar/18 18:23
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176823498
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
-]
-
 REQUIRED_TEST_PACKAGES = [
+'nose>=1.3.7',
 
 Review comment:
   tox.ini already required 1.3.7. I thought this would be a no-op.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83738)
Time Spent: 11h  (was: 10h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 11h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83713
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 17:39
Start Date: 23/Mar/18 17:39
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176811130
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
+[testenv:py27-cython2]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-  cython==0.25.2
-whitelist_externals=
-  find
-  time
+  cython==0.26.1
 
 Review comment:
   Curious why this version? Latest is 0.28.1.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83713)
Time Spent: 10h 40m  (was: 10.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83714=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83714
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 17:39
Start Date: 23/Mar/18 17:39
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176811576
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
-]
-
 REQUIRED_TEST_PACKAGES = [
+'nose>=1.3.7',
 
 Review comment:
   Why are we increasing the required version for nose?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83714)
Time Spent: 10h 50m  (was: 10h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83712=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83712
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 17:39
Start Date: 23/Mar/18 17:39
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176811196
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,107 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython2,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip installation of apache-beam tarball.
+extras = test
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
   python --version
   pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+# This environment will fail in Jenkins if named "py27-cython".
 
 Review comment:
   For my education, do you know why it fails?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83712)
Time Spent: 10.5h  (was: 10h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83708
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 17:32
Start Date: 23/Mar/18 17:32
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-375743166
 
 
   Ready for review and merging if Python checks pass.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83708)
Time Spent: 10h 20m  (was: 10h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83469
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 05:24
Start Date: 23/Mar/18 05:24
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176641575
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   @udim Is it required to run `python setup.py test`. If not we can drop it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83469)
Time Spent: 10h 10m  (was: 10h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83468
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 05:20
Start Date: 23/Mar/18 05:20
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176641300
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   @cclauss Are you referring to a specific issue? `REQUIRED_PIP_VERSION` is 
just the minimum version we require, we are not force downgrading to that 
version. If there is a known issue in compatibility we can consider raising the 
minimum supported version


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83468)
Time Spent: 10h  (was: 9h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 10h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83436=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83436
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175934686
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
 
 Review comment:
   Removed. It was convenient for me but I realized it adds 15s to the run.
   You can always pass `--recreate` on the command line.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83436)
Time Spent: 9h 40m  (was: 9.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83435
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175908871
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   @aaltay setup_requires= lists packages required to run setup.py. I couldn't 
see why nose is required for that.
   Specifying nose in the tests_require= keyword arg is sufficient.
   
   @cclauss I would like to leave it alone in this PR, and I don't have any 
objection to raising the required version.
   Note that creating a new virtualenv installs (at least for me) a version of 
pip that's newer than the system installed version (9.0.2 vs 9.0.1).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83435)
Time Spent: 9.5h  (was: 9h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83438=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83438
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175935020
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
-  python --version
 
 Review comment:
   done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83438)
Time Spent: 9h 40m  (was: 9.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83434=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83434
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175926483
 
 

 ##
 File path: sdks/python/build.gradle
 ##
 @@ -64,7 +64,7 @@ build.dependsOn buildPython
 task lint (dependsOn: 'setupTest') {
   doLast {
 exec {
-  commandLine 'tox', '-e', 'lint', '-c', 'tox.ini'
+  commandLine 'tox', '-e', 'py27-lint', '-c', 'tox.ini'
 
 Review comment:
   done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83434)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83432=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83432
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175903622
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
 
 Review comment:
   These are not packages, but options for pip when installing the dist 
tarball. I reworded the comment above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83432)
Time Spent: 9h 10m  (was: 9h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83437=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83437
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176620089
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -27,44 +28,78 @@ select = E3
 
 # Shared environment options.
 [testenv]
-# Set [] options for pip install, e.g., pip install apache-beam[test].
+# Set [] options for pip installation of apache-beam tarball.
 extras = test
-# Always recreate the virtual environment.
-recreate = True
-# Pass these environment variables to the test environment.
-passenv = TRAVIS*
 # Don't warn that these commands aren't installed.
 whitelist_externals =
   find
   time
 
 [testenv:py27]
 commands =
+  python --version
+  pip --version
   {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
-  python setup.py test
+  #python setup.py test
   {toxinidir}/run_tox_cleanup.sh
 
 [testenv:py27-cython]
 
 Review comment:
   Please ignore. Testing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83437)
Time Spent: 9h 40m  (was: 9.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83433
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175935645
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
-  python --version
-  pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+[testenv:py27-cython]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
   cython==0.25.2
-whitelist_externals=
-  find
-  time
 commands =
-  python --version
-  pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  # Clean up all previous cython generated files.
-  - find apache_beam -type f -name '*.c' -delete
-  - find apache_beam -type f -name '*.so' -delete
-  - find target/build -type f -name '*.c' -delete
-  - find target/build -type f -name '*.so' -delete
-  time pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-  # Clean up all cython generated files. Ignore if deletion fails.
-  - find apache_beam -type f -name '*.c' -delete
-  - find apache_beam -type f -name '*.so' -delete
-  - find target/build -type f -name '*.c' -delete
-  - find target/build -type f -name '*.so' -delete
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27gcp]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-whitelist_externals=find
+[testenv:py27-gcp]
+extras = test,gcp
 commands =
-  python --version
-  pip --version
-  pip install -e .[test,gcp]
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:lint_py2]
-deps=
-  nose==1.3.7
+[testenv:py27-lint]
+deps =
   pycodestyle==2.3.1
   pylint==1.7.2
   future==0.16.0
   isort==4.2.15
   flake8==3.5.0
-whitelist_externals=time
 commands =
   python --version
-  pip --version
-  time pip install -e .[test]
   time {toxinidir}/run_pylint.sh
-passenv = TRAVIS*
 
-[testenv:lint_py3]
-deps=
-  nose==1.3.7
+[testenv:py3-lint]
+deps =
   pycodestyle==2.3.1
   pylint==1.7.2
   future==0.16.0
   isort==4.2.15
   flake8==3.5.0
-whitelist_externals=time
+setenv =
+BEAM_EXPERIMENTAL_PY3=1
 commands =
-  time pip install -e .[test]
+  python --version
   time {toxinidir}/run_mini_py3lint.sh
-passenv = TRAVIS*
-
 
 [testenv:docs]
-deps=
-  nose==1.3.7
-  grpcio-tools==1.3.5
+extras = docs
+deps =
   Sphinx==1.6.5
   sphinx_rtd_theme==0.2.4
-whitelist_externals=time
 commands =
   python --version
   pip --version
   time pip install -e .[test,gcp,docs]
 
 Review comment:
   done


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above 

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83439
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 23/Mar/18 01:45
Start Date: 23/Mar/18 01:45
Worklog Time Spent: 10m 
  Work Description: udim commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175934778
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
 
 Review comment:
   Removed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83439)
Time Spent: 9h 50m  (was: 9h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83239=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83239
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 16:56
Start Date: 22/Mar/18 16:56
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-375378623
 
 
   @udim Thank you. Let me know when it is ready for review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83239)
Time Spent: 9h  (was: 8h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83238=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83238
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 16:55
Start Date: 22/Mar/18 16:55
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-375378069
 
 
   @aaltay the changes you're seeing in the second commit are my attempts at 
debugging Jenkins runs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83238)
Time Spent: 8h 50m  (was: 8h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83090=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83090
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 06:28
Start Date: 22/Mar/18 06:28
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176317015
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   The virtualenv does bring a pip but it can be upgraded by pip 
(bootstrapping).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83090)
Time Spent: 8h 40m  (was: 8.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83088=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83088
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 06:14
Start Date: 22/Mar/18 06:14
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176317015
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Python packaging has changed a lot in the past three years especially in 
relation to Python 3 requirements https://pip.pypa.io/en/stable/news/
   
   The virtualenv does bring a pip but it can be upgraded by pip 
(bootstrapping).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83088)
Time Spent: 8.5h  (was: 8h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83086
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 06:07
Start Date: 22/Mar/18 06:07
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176316206
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Python packaging has changed a lot in the past three years especially in 
relation to Python 3 requirements https://pip.pypa.io/en/stable/news  We want 
to avoid compatibility issues in the migration to 
https://github.com/pypa/warehouse


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83086)
Time Spent: 8h 20m  (was: 8h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83064=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83064
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 04:26
Start Date: 22/Mar/18 04:26
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176305597
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,7 +17,8 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
+#envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
+envlist = py27-cython{,2,3}
 
 Review comment:
   left over comment?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83064)
Time Spent: 8h 10m  (was: 8h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83063=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83063
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 04:26
Start Date: 22/Mar/18 04:26
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176305583
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -27,44 +28,78 @@ select = E3
 
 # Shared environment options.
 [testenv]
-# Set [] options for pip install, e.g., pip install apache-beam[test].
+# Set [] options for pip installation of apache-beam tarball.
 extras = test
-# Always recreate the virtual environment.
-recreate = True
-# Pass these environment variables to the test environment.
-passenv = TRAVIS*
 # Don't warn that these commands aren't installed.
 whitelist_externals =
   find
   time
 
 [testenv:py27]
 commands =
+  python --version
+  pip --version
   {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
-  python setup.py test
+  #python setup.py test
   {toxinidir}/run_tox_cleanup.sh
 
 [testenv:py27-cython]
 
 Review comment:
   What is py27-cython, py27-cython2, py27-cython3 ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83063)
Time Spent: 8h  (was: 7h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83062
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 04:26
Start Date: 22/Mar/18 04:26
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176305503
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -27,44 +28,78 @@ select = E3
 
 # Shared environment options.
 [testenv]
-# Set [] options for pip install, e.g., pip install apache-beam[test].
+# Set [] options for pip installation of apache-beam tarball.
 extras = test
-# Always recreate the virtual environment.
-recreate = True
-# Pass these environment variables to the test environment.
-passenv = TRAVIS*
 # Don't warn that these commands aren't installed.
 whitelist_externals =
   find
   time
 
 [testenv:py27]
 commands =
+  python --version
+  pip --version
   {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
-  python setup.py test
+  #python setup.py test
   {toxinidir}/run_tox_cleanup.sh
 
 [testenv:py27-cython]
+deps =
+  nose==1.3.7
+  grpcio-tools==1.3.5
+  cython==0.25.2
+commands =
+  python --version
+  pip --version
+  time {toxinidir}/run_pylint.sh
+
+[testenv:py27-cython2]
+# cython tests are only expected to work in linux (2.x and 3.x)
+# If we want to add other platforms in the future, it should be:
+# `platform = linux2|darwin|...`
+# See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
+platform = linux2
+deps =
+  nose==1.3.7
+  grpcio-tools==1.3.5
+  cython==0.25.2
+commands =
+  python --version
+  pip --version
+  {toxinidir}/run_tox_cleanup.sh
+  python apache_beam/examples/complete/autocomplete_test.py
+  #python setup.py test
+  {toxinidir}/run_tox_cleanup.sh
+# TODO: remove if unneeded
+passenv = TRAVIS*
+
+[testenv:py27-cython3]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
 deps =
-  cython==0.26.1
+  cython==0.25.2
 commands =
+  python --version
+  pip --version
   {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
-  python setup.py test
+  #python setup.py test
   {toxinidir}/run_tox_cleanup.sh
+# TODO: remove if unneeded
+passenv = TRAVIS*
 
 [testenv:py27-gcp]
 extras = test,gcp
 commands =
+  python --version
+  pip --version
   {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
-  python setup.py test
+  #python setup.py test
 
 Review comment:
   left over?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83062)
Time Spent: 8h  (was: 7h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=83061=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83061
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 22/Mar/18 04:21
Start Date: 22/Mar/18 04:21
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r176305244
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Is there a reason for us to raise the required version?
   
   > Alternatively, should the process do pip install --upgrade pip?
   Which process? As far as I know, virtualenv brings a new version of pip with 
it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 83061)
Time Spent: 7h 50m  (was: 7h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81753=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81753
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 19/Mar/18 07:00
Start Date: 19/Mar/18 07:00
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-374117989
 
 
   Retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81753)
Time Spent: 7h 40m  (was: 7.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81752=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81752
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 19/Mar/18 07:00
Start Date: 19/Mar/18 07:00
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-374117989
 
 
   Retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81752)
Time Spent: 7.5h  (was: 7h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81695=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81695
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 18/Mar/18 21:31
Start Date: 18/Mar/18 21:31
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-374047926
 
 
   Retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81695)
Time Spent: 7h 20m  (was: 7h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81693
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 18/Mar/18 21:15
Start Date: 18/Mar/18 21:15
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-374047926
 
 
   Retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81693)
Time Spent: 7h 10m  (was: 7h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81539=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81539
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 09:17
Start Date: 17/Mar/18 09:17
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175250556
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Does it make sense to raise __REQUIRED_PIP_VERSION = '7.0.0'__ to something 
a closer to the current 9.0.2 or do we leave that alone for now?  
Alternatively, should the process do __pip install --upgrade pip__?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81539)
Time Spent: 7h  (was: 6h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81537
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 09:12
Start Date: 17/Mar/18 09:12
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175250556
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Does it make sense to raise __REQUIRED_PIP_VERSION = '7.0.0'__ to something 
a closer to the current 9.0.2 or do we leave that alone for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81537)
Time Spent: 6h 50m  (was: 6h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81532=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81532
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 08:45
Start Date: 17/Mar/18 08:45
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-370922880
 
 
   In general, I have been focused on 
https://issues.apache.org/jira/browse/BEAM-1251


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81532)
Time Spent: 6h 40m  (was: 6.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81531=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81531
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 08:36
Start Date: 17/Mar/18 08:36
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-370915151
 
 
   ~~Merge #4807 or similar __before__ merging this PR.~~
   
   Once #4561 and ~~#4806~~ (or similar) are merged then __python2 -m flake8 . 
--select=F821__ should pass but __python3 -m flake8 . --select=F821__ will 
require additional work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81531)
Time Spent: 6.5h  (was: 6h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81530
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 08:31
Start Date: 17/Mar/18 08:31
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-370915151
 
 
   ~~Merge #4807 or similar __before__ merging this PR.~~
   
   Once #4561 and #4806 (or similar) are merged then __python2 -m flake8 . 
--select=F821__ should pass but __python3 -m flake8 . --select=F821__ will 
require additional work.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81530)
Time Spent: 6h 20m  (was: 6h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81518
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 07:21
Start Date: 17/Mar/18 07:21
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175250556
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Does it make sense to raise __REQUIRED_PIP_VERSION = '7.0.0'__ to something 
a closer to the current 9.0.1 or do we leave that alone for now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81518)
Time Spent: 6h 10m  (was: 6h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81512
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 05:50
Start Date: 17/Mar/18 05:50
Worklog Time Spent: 10m 
  Work Description: cclauss commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175249040
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
-  python --version
 
 Review comment:
   +1 on this request.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81512)
Time Spent: 6h  (was: 5h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81508
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 05:24
Start Date: 17/Mar/18 05:24
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #4801: [BEAM-3738] Enable 
py3 lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-373895697
 
 
   +1 to using an environment variable, much better than falsely claiming
   3.0-3.4 support.
   
   
   On Fri, Mar 16, 2018 at 5:23 PM Ahmet Altay 
   wrote:
   
   > Closed #4801 .
   >
   > —
   > You are receiving this because you are subscribed to this thread.
   > Reply to this email directly, view it on GitHub
   > , or mute the
   > thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81508)
Time Spent: 5h 50m  (was: 5h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81455=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81455
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 00:23
Start Date: 17/Mar/18 00:23
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #4801: [BEAM-3738] Enable 
py3 lint
URL: https://github.com/apache/beam/pull/4801
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/setup.py b/sdks/python/setup.py
index 5ed623b2293..5ef1683e81b 100644
--- a/sdks/python/setup.py
+++ b/sdks/python/setup.py
@@ -204,7 +204,6 @@ def generate_common_urns():
 ]),
 setup_requires=REQUIRED_SETUP_PACKAGES,
 install_requires=REQUIRED_PACKAGES,
-python_requires='>=2.7,<3.0',
 test_suite='nose.collector',
 tests_require=REQUIRED_TEST_PACKAGES,
 extras_require={
diff --git a/sdks/python/tox.ini b/sdks/python/tox.ini
index 857e7b0e571..7d4152602f8 100644
--- a/sdks/python/tox.ini
+++ b/sdks/python/tox.ini
@@ -17,8 +17,7 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27gcp,py27cython,lint_py2,lint_py3,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
@@ -103,7 +102,6 @@ whitelist_externals=time
 commands =
   python --version
   pip --version
-  time pip install -e .[test]
   time {toxinidir}/run_pylint.sh
 passenv = TRAVIS*
 
@@ -116,12 +114,13 @@ deps=
   isort==4.2.15
   flake8==3.5.0
 whitelist_externals=time
+basepython=python3
 commands =
-  time pip install -e .[test]
+  python --version
+  pip --version
   time {toxinidir}/run_mini_py3lint.sh
 passenv = TRAVIS*
 
-
 [testenv:docs]
 deps=
   nose==1.3.7


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81455)
Time Spent: 5h 40m  (was: 5.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81454=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81454
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 17/Mar/18 00:23
Start Date: 17/Mar/18 00:23
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #4801: [BEAM-3738] Enable py3 
lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-373878311
 
 
   Thank you @udim. Closing this PR in favor of #4877.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81454)
Time Spent: 5.5h  (was: 5h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81383
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175218594
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
 
 Review comment:
   How are these extra packages installed? Is it a tox feature?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81383)
Time Spent: 5h 20m  (was: 5h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81381
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175214899
 
 

 ##
 File path: sdks/python/setup.py
 ##
 @@ -113,11 +113,8 @@ def get_version():
 'futures>=3.1.1,<4.0.0',
 ]
 
-REQUIRED_SETUP_PACKAGES = [
-'nose>=1.0',
 
 Review comment:
   Is not this required by setup, because test_suite references it?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81381)
Time Spent: 5h 10m  (was: 5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81379
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175218288
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
-  python --version
 
 Review comment:
   Printing python and pip version was very useful in debugging issues. Can we 
keep them?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81379)
Time Spent: 5h  (was: 4h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81380=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81380
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175198814
 
 

 ##
 File path: sdks/python/build.gradle
 ##
 @@ -64,7 +64,7 @@ build.dependsOn buildPython
 task lint (dependsOn: 'setupTest') {
   doLast {
 exec {
-  commandLine 'tox', '-e', 'lint', '-c', 'tox.ini'
+  commandLine 'tox', '-e', 'py27-lint', '-c', 'tox.ini'
 
 Review comment:
   Should we also run the py3 lint?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81380)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81385=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81385
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175215623
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
 
 Review comment:
   I believe we do not need this anymore.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81385)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81384
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175215562
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
 
 Review comment:
   Is there a need for this? Would not affect local development?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81384)
Time Spent: 5h 20m  (was: 5h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81382
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:28
Start Date: 16/Mar/18 21:28
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4877: 
[BEAM-3738] Enable py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#discussion_r175218495
 
 

 ##
 File path: sdks/python/tox.ini
 ##
 @@ -17,142 +17,101 @@
 
 [tox]
 # new environments will be excluded by default unless explicitly added to 
envlist.
-# TODO (after BEAM-3671) add lint_py3 back in.
-envlist = py27,py27gcp,py27cython,lint_py2,docs
+envlist = py27,py27-{gcp,cython,lint},py3-lint,docs
 toxworkdir = {toxinidir}/target/.tox
 
 [pycodestyle]
 # Disable all errors and warnings except for the ones related to blank lines.
 # pylint does not check the number of blank lines.
 select = E3
 
+# Shared environment options.
+[testenv]
+# Set [] options for pip install, e.g., pip install apache-beam[test].
+extras = test
+# Always recreate the virtual environment.
+recreate = True
+# Pass these environment variables to the test environment.
+passenv = TRAVIS*
+# Don't warn that these commands aren't installed.
+whitelist_externals =
+  find
+  time
+
 [testenv:py27]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
-whitelist_externals=find
 commands =
-  python --version
-  pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27cython]
+[testenv:py27-cython]
 # cython tests are only expected to work in linux (2.x and 3.x)
 # If we want to add other platforms in the future, it should be:
 # `platform = linux2|darwin|...`
 # See https://docs.python.org/2/library/sys.html#sys.platform for platform 
codes
 platform = linux2
-# autocomplete_test depends on nose when invoked directly.
 deps =
-  nose==1.3.7
-  grpcio-tools==1.3.5
   cython==0.25.2
-whitelist_externals=
-  find
-  time
 commands =
-  python --version
-  pip --version
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
-  # Clean up all previous cython generated files.
-  - find apache_beam -type f -name '*.c' -delete
-  - find apache_beam -type f -name '*.so' -delete
-  - find target/build -type f -name '*.c' -delete
-  - find target/build -type f -name '*.so' -delete
-  time pip install -e .[test]
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-  # Clean up all cython generated files. Ignore if deletion fails.
-  - find apache_beam -type f -name '*.c' -delete
-  - find apache_beam -type f -name '*.so' -delete
-  - find target/build -type f -name '*.c' -delete
-  - find target/build -type f -name '*.so' -delete
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:py27gcp]
-# autocomplete_test depends on nose when invoked directly.
-deps =
-  nose==1.3.7
-whitelist_externals=find
+[testenv:py27-gcp]
+extras = test,gcp
 commands =
-  python --version
-  pip --version
-  pip install -e .[test,gcp]
-  # Clean up all previous python generated files.
-  - find apache_beam -type f -name '*.pyc' -delete
+  {toxinidir}/run_tox_cleanup.sh
   python apache_beam/examples/complete/autocomplete_test.py
   python setup.py test
-passenv = TRAVIS*
+  {toxinidir}/run_tox_cleanup.sh
 
-[testenv:lint_py2]
-deps=
-  nose==1.3.7
+[testenv:py27-lint]
+deps =
   pycodestyle==2.3.1
   pylint==1.7.2
   future==0.16.0
   isort==4.2.15
   flake8==3.5.0
-whitelist_externals=time
 commands =
   python --version
-  pip --version
-  time pip install -e .[test]
   time {toxinidir}/run_pylint.sh
-passenv = TRAVIS*
 
-[testenv:lint_py3]
-deps=
-  nose==1.3.7
+[testenv:py3-lint]
+deps =
   pycodestyle==2.3.1
   pylint==1.7.2
   future==0.16.0
   isort==4.2.15
   flake8==3.5.0
-whitelist_externals=time
+setenv =
+BEAM_EXPERIMENTAL_PY3=1
 commands =
-  time pip install -e .[test]
+  python --version
   time {toxinidir}/run_mini_py3lint.sh
-passenv = TRAVIS*
-
 
 [testenv:docs]
-deps=
-  nose==1.3.7
-  grpcio-tools==1.3.5
+extras = docs
+deps =
   Sphinx==1.6.5
   sphinx_rtd_theme==0.2.4
-whitelist_externals=time
 commands =
   python --version
   pip --version
   time pip install -e .[test,gcp,docs]
 
 Review comment:
   Should we use `extras` instead of `[test,gcp,docs]`?


This is an automated message from the Apache Git Service.
To respond to the 

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81372=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81372
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:10
Start Date: 16/Mar/18 21:10
Worklog Time Spent: 10m 
  Work Description: holdenk commented on issue #4877: [BEAM-3738] Enable 
py3 lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-373845974
 
 
   Love the refactor, thanks @udim!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81372)
Time Spent: 4h 50m  (was: 4h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81369=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81369
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 21:04
Start Date: 16/Mar/18 21:04
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-373844621
 
 
   Missing license was fixed in #4881, retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81369)
Time Spent: 4h 40m  (was: 4.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81309=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81309
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 18:31
Start Date: 16/Mar/18 18:31
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-373805387
 
 
   run python postcommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81309)
Time Spent: 4.5h  (was: 4h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81308=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81308
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 18:26
Start Date: 16/Mar/18 18:26
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4877: [BEAM-3738] Enable py3 
lint and cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877#issuecomment-373804100
 
 
   R: @holdenk @cclauss @aaltay 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81308)
Time Spent: 4h 20m  (was: 4h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81307=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81307
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 18:25
Start Date: 16/Mar/18 18:25
Worklog Time Spent: 10m 
  Work Description: udim commented on issue #4801: [BEAM-3738] Enable py3 
lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-373803657
 
 
   I've created #4877, which should include the changes here as well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81307)
Time Spent: 4h 10m  (was: 4h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=81277=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81277
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 16/Mar/18 17:48
Start Date: 16/Mar/18 17:48
Worklog Time Spent: 10m 
  Work Description: udim opened a new pull request #4877: [BEAM-3738] 
Cleanup tox.ini.
URL: https://github.com/apache/beam/pull/4877
 
 
   The important change here is renaming the lint environments such that Tox 
will magically use the requested Python version.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81277)
Time Spent: 4h  (was: 3h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=80465=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80465
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 14/Mar/18 18:55
Start Date: 14/Mar/18 18:55
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #4801: [BEAM-3738] Enable py3 
lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-373136737
 
 
   I have not had a chance to work on this after the my previous comment. I 
like the idea of using an environment variable. 
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80465)
Time Spent: 3h 50m  (was: 3h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=80439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80439
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 14/Mar/18 18:19
Start Date: 14/Mar/18 18:19
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-373121221
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80439)
Time Spent: 3h 40m  (was: 3.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=80434=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80434
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 14/Mar/18 18:08
Start Date: 14/Mar/18 18:08
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-373121221
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80434)
Time Spent: 3.5h  (was: 3h 20m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=80433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80433
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 14/Mar/18 18:08
Start Date: 14/Mar/18 18:08
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-371944285
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80433)
Time Spent: 3h 20m  (was: 3h 10m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=80014=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80014
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 13/Mar/18 18:51
Start Date: 13/Mar/18 18:51
Worklog Time Spent: 10m 
  Work Description: holdenk commented on issue #4801: [BEAM-3738] Enable 
py3 lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-372778226
 
 
   @aaltay what are your thoughts?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80014)
Time Spent: 3h 10m  (was: 3h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=79019=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-79019
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 09/Mar/18 21:03
Start Date: 09/Mar/18 21:03
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-370974099
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 79019)
Time Spent: 2h 50m  (was: 2h 40m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=79020=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-79020
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 09/Mar/18 21:03
Start Date: 09/Mar/18 21:03
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4798: [BEAM-3738] Add more 
flake8 tests to run_pylint.sh
URL: https://github.com/apache/beam/pull/4798#issuecomment-371944285
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 79020)
Time Spent: 3h  (was: 2h 50m)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3738) Enable Py3 linting in Jenkins

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3738?focusedWorklogId=79011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-79011
 ]

ASF GitHub Bot logged work on BEAM-3738:


Author: ASF GitHub Bot
Created on: 09/Mar/18 20:24
Start Date: 09/Mar/18 20:24
Worklog Time Spent: 10m 
  Work Description: holdenk commented on issue #4801: [BEAM-3738] Enable 
py3 lint
URL: https://github.com/apache/beam/pull/4801#issuecomment-371229849
 
 
   So @cclauss I don't think that limiting to 3.4.3 is great since someone 
might read it and then just try to use 3.4.3. Personally I'd use an env 
variable to (e.g. `LINTING`)  and in linting have a different requirement. We 
also probably want folks to be able to run tox on their own machines since its 
in the guide.
   
   But I like the direction of selectively unblocking setup.py for 
testing/linting so that we more easily track our progress and avoid regressions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 79011)
Time Spent: 2h 40m  (was: 2.5h)

> Enable Py3 linting in Jenkins
> -
>
> Key: BEAM-3738
> URL: https://issues.apache.org/jira/browse/BEAM-3738
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, testing
>Reporter: holdenk
>Assignee: Ahmet Altay
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> After BEAM-3671 is finished enable linting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)