xhochy commented on a change in pull request #2268:
URL: https://github.com/apache/iceberg/pull/2268#discussion_r582679313
##########
File path: python/setup.py
##########
@@ -15,8 +15,28 @@
# specific language governing permissions and limitations
# under the License.
+import sys
+
from setuptools import setup
+DEPENDENCIES = ['botocore==1.20.14',
+ 'boto3==1.17.14',
+ 'fastavro==1.3.2',
+ 'hmsclient==0.1.1',
+ 'mmh3==3.0.0',
+ 'pyparsing==2.4.7',
+ 'python-dateutil==2.8.1',
+ 'pytz==2021.1',
+ 'requests==2.25.1',
+ 'retrying==1.3.3',
+ 'pyarrow==2.0.0'
+ ]
+
Review comment:
Yes, reducing this down to these and chaging the pins to e.g.
`pyarrow>=2,<3' (you can be stricter for other projects but `pyarrow` is mostly
releasing on the major level). In some cases I use a strict pinning (all
packages pinned) for the tests and a looser pinning in the package itself.
In `pyarrow` we have quite open pins but we spent a lot of time on fixing
the breakages directly. A different approach I also quite like and works for a
lot of things is have the pins as strict as suggested but have a bot like
dependendabot running that automatically makes PRs and updates the pins. Then
we see in these bot PRs whether the new versions break things.
(Also, when I finally get time to contribute here (:sob:), I would add
nightly tests that also test against `pyarrow` nightlies to get earlier
notifications about breakage).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
