[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929356#comment-16929356 ] Valentyn Tymofieiev commented on BEAM-8229: --- Let's continue the conversation here. [~chadrik] You can take a look at https://github.com/apache/beam/pull/9223 for pointers to PostCommit entrypoints. Python 3.5 EOL is a year from now, so I think Beam should still support it, without very strong reasons not to. [~udim] I am not familiar with pytest, but suspected it may have some solutions for this problem. I thought that you might know if this is the case. > Python postcommit 3.7 tests are failing with syntax error > - > > Key: BEAM-8229 > URL: https://issues.apache.org/jira/browse/BEAM-8229 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Ahmet Altay >Assignee: Udi Meiri >Priority: Blocker > > Root cause: https://github.com/apache/beam/pull/9098 > 20:12:14 Traceback (most recent call last): > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/loader.py", > line 418, in loadTestsFromName > 20:12:14 addr.filename, addr.module) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 47, in importFromPath > 20:12:14 return self.importFromDir(dir_path, fqname) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 94, in importFromDir > 20:12:14 mod = load_module(part_fqname, fh, filename, desc) > 20:12:14 SyntaxError: invalid syntax (external_test_py37.py, line 46) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929346#comment-16929346 ] Chad Dombrova commented on BEAM-8229: - This is getting a bit difficult to communicate as the conversation is spread across 2 JIRA tickets and 2 github PRs. Reposting this from the others: Pre-commit does run the python 3.x tests, but it runs many of the tests on python 3.5. This PR introduced tests that can only run on python 3.6 or higher, and I had to do some extra work to the beam test framework to make that possible: https://github.com/apache/beam/pull/9098/commits/0c31f7c8b1c108f7df4cbedfaf9da0ce30092ce0 It appears that same work to properly exclude/include tests at the major + minor version needs to be done for post-commit tests. I didn't do that simply because I didn't know it could be a problem. I assumed everything went through tox. I'm currently looking for the post-commit entry point to see what else needs to be filtered. Note that the _other_ solution to this is dropping python 3.5 support, but I figured that'd be a much bigger conversation. > Python postcommit 3.7 tests are failing with syntax error > - > > Key: BEAM-8229 > URL: https://issues.apache.org/jira/browse/BEAM-8229 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Ahmet Altay >Assignee: Udi Meiri >Priority: Blocker > > Root cause: https://github.com/apache/beam/pull/9098 > 20:12:14 Traceback (most recent call last): > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/loader.py", > line 418, in loadTestsFromName > 20:12:14 addr.filename, addr.module) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 47, in importFromPath > 20:12:14 return self.importFromDir(dir_path, fqname) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 94, in importFromDir > 20:12:14 mod = load_module(part_fqname, fh, filename, desc) > 20:12:14 SyntaxError: invalid syntax (external_test_py37.py, line 46) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929343#comment-16929343 ] Udi Meiri commented on BEAM-8229: - What feature were you thinking of /wrt pytest? > Python postcommit 3.7 tests are failing with syntax error > - > > Key: BEAM-8229 > URL: https://issues.apache.org/jira/browse/BEAM-8229 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Ahmet Altay >Assignee: Udi Meiri >Priority: Blocker > > Root cause: https://github.com/apache/beam/pull/9098 > 20:12:14 Traceback (most recent call last): > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/loader.py", > line 418, in loadTestsFromName > 20:12:14 addr.filename, addr.module) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 47, in importFromPath > 20:12:14 return self.importFromDir(dir_path, fqname) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 94, in importFromDir > 20:12:14 mod = load_module(part_fqname, fh, filename, desc) > 20:12:14 SyntaxError: invalid syntax (external_test_py37.py, line 46) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929339#comment-16929339 ] Valentyn Tymofieiev commented on BEAM-8229: --- The rootcause of this failure is that we run a Python 3-only test in Python 2. To be excluded from execution on Python 2, test needs to end with _py3, not _py37. However, this may not be sufficient for the purposes of the failing test in case, it uses py37 syntax that is invalid on Python 35 (we can check if it also fails in Python 3.5 postcommits). If so, this is may be the first such test, as we don't have a per-minor version exclusion mechanism yet. We could improve the rules of how py3 tests are excluded, or, perhaps wait until we switch to py_test, which may make these type of configurations easier. cc: [~chadrik] [~udim] [~markliu] who may have opinions here. > Python postcommit 3.7 tests are failing with syntax error > - > > Key: BEAM-8229 > URL: https://issues.apache.org/jira/browse/BEAM-8229 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Ahmet Altay >Assignee: Udi Meiri >Priority: Blocker > > Root cause: https://github.com/apache/beam/pull/9098 > 20:12:14 Traceback (most recent call last): > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/loader.py", > line 418, in loadTestsFromName > 20:12:14 addr.filename, addr.module) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 47, in importFromPath > 20:12:14 return self.importFromDir(dir_path, fqname) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 94, in importFromDir > 20:12:14 mod = load_module(part_fqname, fh, filename, desc) > 20:12:14 SyntaxError: invalid syntax (external_test_py37.py, line 46) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929332#comment-16929332 ] Ahmet Altay commented on BEAM-8229: --- This is actually failing in py2 postcommits. Should not py2 post commit skip files with py37 in the name? > Python postcommit 3.7 tests are failing with syntax error > - > > Key: BEAM-8229 > URL: https://issues.apache.org/jira/browse/BEAM-8229 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Ahmet Altay >Assignee: Ahmet Altay >Priority: Blocker > > Root cause: https://github.com/apache/beam/pull/9098 > 20:12:14 Traceback (most recent call last): > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/loader.py", > line 418, in loadTestsFromName > 20:12:14 addr.filename, addr.module) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 47, in importFromPath > 20:12:14 return self.importFromDir(dir_path, fqname) > 20:12:14 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python2_PR/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/nose/importer.py", > line 94, in importFromDir > 20:12:14 mod = load_module(part_fqname, fh, filename, desc) > 20:12:14 SyntaxError: invalid syntax (external_test_py37.py, line 46) -- This message was sent by Atlassian Jira (v8.3.2#803003)
