[
https://issues.apache.org/jira/browse/BEAM-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092010#comment-17092010
]
Valentyn Tymofieiev commented on BEAM-9819:
-------------------------------------------
Latest httplib2 did not work:
{noformat}
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/base_api.py",
line 715, in _RunMethod
17:37:32 http_request, client=self.client)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 908, in InitializeUpload
17:37:32 return self.StreamInChunks()
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 1020, in StreamInChunks
17:37:32 additional_headers=additional_headers)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 957, in __StreamMedia
17:37:32 response = send_func(self.stream.tell())
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 943, in CallSendChunk
17:37:32 start, additional_headers=additional_headers)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 1120, in __SendChunk
17:37:32 return self.__SendMediaRequest(request, end)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/transfer.py",
line 1033, in __SendMediaRequest
17:37:32 retries=self.num_retries, check_response_func=CheckResponse)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/http_wrapper.py",
line 356, in MakeRequest
17:37:32 max_retry_wait, total_wait_sec))
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/http_wrapper.py",
line 304, in HandleExceptionsAndRebuildHttpConnections
17:37:32 raise retry_args.exc
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/http_wrapper.py",
line 346, in MakeRequest
17:37:32 check_response_func=check_response_func)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/apitools/base/py/http_wrapper.py",
line 396, in _MakeRequestNoRetry
17:37:32 redirections=redirections, connection_type=connection_type)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/oauth2client/transport.py",
line 169, in new_request
17:37:32 redirections, connection_type)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/oauth2client/transport.py",
line 169, in new_request
17:37:32 redirections, connection_type)
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/httplib2/__init__.py",
line 1991, in request
17:37:32 cachekey,
17:37:32 File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/httplib2/__init__.py",
line 1690, in _request
17:37:32 content,
17:37:32 httplib2.RedirectMissingLocation: Redirected but the response is
missing a Location: header.
{noformat}
https://github.com/httplib2/httplib2/issues/156 looks related. Workaround is to
stick with httplib2<1.16.0.
The fix may be required in apitools library, which is deprecated. Long term,
we should switch to google-api-python-client.
> Extend acceptable httplib2 version range.
> -----------------------------------------
>
> Key: BEAM-9819
> URL: https://issues.apache.org/jira/browse/BEAM-9819
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> A Beam Python 3 user reported an inconvenience when migrating their Python 2
> to Python 3 pipeline due to a bug in httlib2 dependency, where httplib2 on
> Python3 doesn’t respect NO_PROXY environment variable. This bug was fixed in
> 0.13.1 [1]. Looking at the changelog of httplib2[2], there were more
> Python3-specific fixes in recent versions.
> In the past we restricted httplib2 version due to a conflict with
> googledatastore[2]. We have since then removed[3] a dependency on
> googledatastore, and I don't see other reasons to restrict httplib2 to
> 0.12.0.
> [1] https://github.com/httplib2/httplib2/pull/140
> [2]
> https://github.com/apache/beam/commit/3b2c90156ddb67f4daddf275172c0b2d4eb1eaf6
> [3]
> https://github.com/apache/beam/pull/11175/files#diff-e9d0ab71f74dc10309a29b697ee99330L202
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
