+1 to pinning to exact versions, to be sure that our releases do not break when newer versions of dependencies are released.
On Fri, Dec 15, 2017 at 2:44 PM Ahmet Altay <[email protected]> wrote: > On Fri, Dec 15, 2017 at 2:42 PM, Chamikara Jayalath <[email protected]> > wrote: > >> +1 for automating the process of checking for possible version bumps. >> >> Also, what do you think about pinning dependencies to exact versions >> (instead of ranges) after cutting a release branch ? This should improve >> the stability of released SDKs (but not a prefect solution since transitive >> dependencies can still change). >> > > This is a reasonable suggestion. The issue with that is, by being less > flexible we will prevent users from using latest versions of dependencies. > On the other hand it will prevent breaking of already released versions. > > >> >> Thanks, >> Cham >> >> On Fri, Dec 15, 2017 at 2:19 PM Ahmet Altay <[email protected]> wrote: >> >>> On Fri, Dec 15, 2017 at 2:02 PM, Robert Bradshaw <[email protected]> >>> wrote: >>> >>>> On Fri, Dec 15, 2017 at 1:51 PM, Ahmet Altay <[email protected]> wrote: >>>> > >>>> > On Fri, Dec 15, 2017 at 1:38 PM, Robert Bradshaw <[email protected] >>>> > >>>> > wrote: >>>> >> >>>> >> I am also in favor of pinning as an immediate fix, bumping the bound >>>> >> otherwise. >>>> >> >>>> >> Regarding putting an upper bound to avoid being broken, the last two >>>> >> breaks have been due to just having an (unneeded) upper bound (which >>>> >> held us back to broken/incompatible releases in relationship to other >>>> >> dependencies). We should try to trust semantic versioning when >>>> >> possible, and when not we must regularly audit. >>>> > >>>> > +1 to this, especially the auditing part. We also had breaks because >>>> we >>>> > trusted semantic versioning. So far our semi-official policy was to >>>> trust a >>>> > package until they prove it otherwise. I will argue that grpc here is >>>> making >>>> > a breaking change in a minor version increment by changing the way >>>> they are >>>> > depending on a major package. >>>> >>>> A minor version bump should be allowed to require a minor version bump >>>> in its dependencies. >>>> >>>> > We have done a good job of auditing and updating those pinned (or >>>> upper >>>> > bounded) dependencies, and probably we are behind in some of those. >>>> > >>>> > I wonder if we can automate some of this? If we can get a report, that >>>> > audits our dependencies, warns us about new releases and potential >>>> conflicts >>>> > it would be much easier to keep things up to date. >>>> >>>> Big +1, it should be easy to set up a nightly that relaxes some of the >>>> requirements and sees what (if anything) breaks. Not breaking is >>>> likely a signal that we should relax ours. >>>> >>> >>> Filed https://issues.apache.org/jira/browse/BEAM-3363 to track this. I >>> think it would be awesome if we can tackle this as part of a better >>> infrastructure for testing work. >>> >>> >>>> >>>> >> On Fri, Dec 15, 2017 at 1:33 PM, Chamikara Jayalath (JIRA) >>>> >> <[email protected]> wrote: >>>> >> > >>>> >> > [ >>>> >> > >>>> https://issues.apache.org/jira/browse/BEAM-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293276#comment-16293276 >>>> >> > ] >>>> >> > >>>> >> > Chamikara Jayalath commented on BEAM-3357: >>>> >> > ------------------------------------------ >>>> >> > >>>> >> > You mean we could bump up the upper bound ? I think we should keep >>>> some >>>> >> > upper bound in case we get badly broken by a future protobuf >>>> release. >>>> >> > >>>> >> >> Python SDK head fails to run tests due to >>>> >> >> Requirement.parse('protobuf<=3.4.0,>=3.2.0') >>>> >> >> >>>> >> >> >>>> -------------------------------------------------------------------------------------- >>>> >> >> >>>> >> >> Key: BEAM-3357 >>>> >> >> URL: >>>> https://issues.apache.org/jira/browse/BEAM-3357 >>>> >> >> Project: Beam >>>> >> >> Issue Type: Bug >>>> >> >> Components: sdk-py-core >>>> >> >> Reporter: Chamikara Jayalath >>>> >> >> Priority: Critical >>>> >> >> >>>> >> >> Error is: >>>> >> >> running build_ext >>>> >> >> Traceback (most recent call last): >>>> >> >> File "setup.py", line 202, in <module> >>>> >> >> 'test': generate_protos_first(test), >>>> >> >> File >>>> >> >> >>>> "/Users/chamikara/testing/test_py_12_14_2017_2/env_proto_3.4/lib/python2.7/site-packages/setuptools/__init__.py", >>>> >> >> line 129, in setup >>>> >> >> return distutils.core.setup(**attrs) >>>> >> >> File >>>> >> >> >>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", >>>> >> >> line 151, in setup >>>> >> >> dist.run_commands() >>>> >> >> File >>>> >> >> >>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", >>>> >> >> line 953, in run_commands >>>> >> >> self.run_command(cmd) >>>> >> >> File >>>> >> >> >>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", >>>> >> >> line 972, in run_command >>>> >> >> cmd_obj.run() >>>> >> >> File "setup.py", line 142, in run >>>> >> >> super(cmd, self).run() >>>> >> >> File >>>> >> >> >>>> "/Users/chamikara/testing/test_py_12_14_2017_2/env_proto_3.4/lib/python2.7/site-packages/setuptools/command/test.py", >>>> >> >> line 225, in run >>>> >> >> with self.project_on_sys_path(): >>>> >> >> File >>>> >> >> >>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", >>>> >> >> line 17, in __enter__ >>>> >> >> return self.gen.next() >>>> >> >> File >>>> >> >> >>>> "/Users/chamikara/testing/test_py_12_14_2017_2/env_proto_3.4/lib/python2.7/site-packages/setuptools/command/test.py", >>>> >> >> line 164, in project_on_sys_path >>>> >> >> require('%s==%s' % (ei_cmd.egg_name, ei_cmd.egg_version)) >>>> >> >> File >>>> >> >> >>>> "/Users/chamikara/testing/test_py_12_14_2017_2/env_proto_3.4/lib/python2.7/site-packages/pkg_resources/__init__.py", >>>> >> >> line 984, in require >>>> >> >> needed = self.resolve(parse_requirements(requirements)) >>>> >> >> File >>>> >> >> >>>> "/Users/chamikara/testing/test_py_12_14_2017_2/env_proto_3.4/lib/python2.7/site-packages/pkg_resources/__init__.py", >>>> >> >> line 875, in resolve >>>> >> >> raise VersionConflict(dist, req).with_context(dependent_req) >>>> >> >> pkg_resources.ContextualVersionConflict: (protobuf 3.5.0.post1 >>>> >> >> >>>> (/Users/chamikara/testing/test_py_12_14_2017_2/beam/sdks/python/.eggs/protobuf-3.5.0.post1-py2.7.egg), >>>> >> >> Requirement.parse('protobuf<=3.4.0,>=3.2.0'), >>>> set(['apache-beam'])) >>>> >> >> Seems like grpcio did a release today which is breaking us: >>>> >> >> https://pypi.python.org/pypi/grpcio/1.8.1 >>>> >> >> We have to either bump our protobuf dependency or reduce the upper >>>> >> >> bound of grpcio dependency to previous release (1.7.3). >>>> >> > >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > This message was sent by Atlassian JIRA >>>> >> > (v6.4.14#64029) >>>> > >>>> > >>>> >>>
smime.p7s
Description: S/MIME Cryptographic Signature
