Hi all, I volunteer to start work on making `requests` an optional dependency. It would be great if this could wait until the next release as Alan suggests to make sure there is enough time to ensure the transition goes smoothly.
Ville On Tue, May 28, 2019, 22:24 Alan Gates <[email protected]> wrote: > Again, I don't think this has to be fixed for your first incubator > release. It will need to be fixed before you do a second release. And if > you were a TLP you couldn't release with this. But for now, as a learning > exercise and to get a release out ASAP, I'm ok without fixing it first. > Especially since there are similar issues to deal with on the JavaScript > side. > > Alan. > > On Tue, May 28, 2019 at 11:54 AM Maxime Beauchemin < > [email protected]> wrote: > > > @community, it'd be great if someone could volunteer to do this (making > > the python library "requests" an optional dependency). It appears it's > used > > in its simplest form in a few places (requests.get) which should be easy > to > > replace with urrlib2. The Druid connector uses a more advanced feature > > (basic auth), but that could be made just an optional dependency, as > we're > > deprecating the Druid connector, to be replaced by the SQLAlchemy Druid > > connector/dialect. There it's just a matter of moving `pydruid` and > > `requests` to the "extra_requires" section of our setup.py, and doing > late > > imports. > > > > If not it may take me a moment to get to do this on top of everything > else > > needed for the ASF official release. > > > > Max > > > > On Sat, May 25, 2019 at 12:33 AM Bolke de Bruin <[email protected]> > wrote: > > > > > https://issues.apache.org/jira/browse/LEGAL-192 > > > > > > Details why. It comes down to that you can use LGPL, but only if it’s > an > > > optional dependency. This is policy of the ASF in order not to limit > > > downstream usage of its own products. Ie. A Non optional dependency > would > > > require downstream usage of Superset to abide by the LGPL (“reverse > > > engineering”) without a way out. > > > > > > As our use of the requests modules is fairly limited I suggest to > > > replicate the functionality that is required. Another option is to fork > > > requests and kick out the chardet dependency which is only used in one > > > place and probably not relevant to Superset. > > > > > > Cheers > > > Bolke > > > > > > Verstuurd vanaf mijn iPad > > > > > > > Op 23 mei 2019 om 01:37 heeft Maxime Beauchemin < > > > [email protected]> het volgende geschreven: > > > > > > > > Related, about requests/chardet > > > > https://github.com/kennethreitz/requests/issues/3389 > > > > > > > > For the source code release, do [unbundled] deps need to be cleared? > > From > > > > my understanding we only needed to clear the code we ship. > > > > > > > > If that's the case we've got work to do on the JS side. > > > > > > > > Max > > > > > > > >> On Wed, May 22, 2019 at 4:11 PM Bolke de Bruin <[email protected]> > > > wrote: > > > >> > > > >> Oef chardet is pulled in by requests. The usage of chardet (ie > > > triggered) > > > >> is unlikely as it is only used when the encoding is not set in > > headers. > > > >> > > > >> You could ask the maintainer of chardet to release under another > > > license. > > > >> This can be tough as their might be several people to contact that > > need > > > to > > > >> agree with relicensing. Or do a PR to make the usage of chardet > > > optional in > > > >> requests. Or use urllib and maybe create a wrapper that mimics > > requests. > > > >> > > > >> B. > > > >> > > > >> > > > >> Verstuurd vanaf mijn iPad > > > >> > > > >>> Op 23 mei 2019 om 00:03 heeft Alan Gates <[email protected]> > het > > > >> volgende geschreven: > > > >>> > > > >>> +1 with caveats, see below. I looked at the LICENSE, NOTICE, and > > > >>> DISCLAIMER files, checked for any binary files (executables, > there's > > > >> plenty > > > >>> of image files in the distribution), and looked over the licenses > of > > > the > > > >>> dependencies. > > > >>> > > > >>> More information on the dependencies: > > > >>> I found https://pypi.org/project/pip-licenses/ which explains how > to > > > >> check > > > >>> licenses, very useful. > > > >>> > > > >>> The licenses of modules that will be pulled in when a system is > > > compiled > > > >> or > > > >>> run matter, as the system won't run without them. So it isn't ok > to > > > >> have a > > > >>> GPL licensed library that's necessarily pulled in at > compile/runtime, > > > as > > > >> to > > > >>> run the product you'll still be pulling in the GPL which will > > basically > > > >>> turn the whole thing GPL. (Optional or contrib components are > > > different, > > > >>> as users can choose not to run with them if they aren't ok with the > > > >> license > > > >>> of the optional component.) > > > >>> > > > >>> Running the above on the modules in setup.py, I see that the vast > > > >> majority > > > >>> are BSD, MIT, Apache, or PSFL, all of which are fine. The ones > that > > > >> aren't > > > >>> in that category are: > > > >>> certifi MPL-2.0: This is ok, as it's binary > > > >>> chardet LGPL Not Ok > > > >>> click UNKNOWN > > > >>> jsonschema UNKNOWN > > > >>> python-dateutil Dual License > > > >>> python-dotenv UNKNOWN > > > >>> python-geohash UNKNOWN > > > >>> python3-openid UNKNOWN > > > >>> > > > >>> The MPL one is fine since it's included in binary form. The > unknown > > > and > > > >>> dual license need some digging to determine what they are. > chardet, > > > the > > > >>> LGPL one, is not ok. > > > >>> > > > >>> Since this is an incubating release I am still voting +1, with the > > > caveat > > > >>> that the unknown licenses need to be figured out before the next > > > release, > > > >>> and the LGPL dependency will have to be removed. Right now I think > > > >> getting > > > >>> a release out is more important than fixing these issues. > > > >>> > > > >>> Alan. > > > >>> > > > >>> On Wed, May 22, 2019 at 2:01 PM Maxime Beauchemin < > > > >>> [email protected]> wrote: > > > >>> > > > >>>> Oh actually the commands above just shows the dep tree. > > > >>>> > > > >>>> For deps in python there's > > > >>>> https://github.com/dhatim/python-license-check > > > >>>> > > > >>>> On the JS side I did some work here to attempt building the > LICENSE > > > file > > > >>>> dynamically as the dep tree evolves > > > >>>> https://github.com/apache/incubator-superset/pull/5801 > > > >>>> > > > >>>> I thought validating the licenses of deps wasn't necessary for > > source > > > >>>> releases though. We may want to start the conversation on > > convenience > > > >>>> releases. To me having solid docker images (or just dockerfiles if > > > >> images > > > >>>> are troublesome) (that are lean and optimized to build fast) would > > be > > > >>>> ideal, especially if they are used in CI. > > > >>>> > > > >>>> Max > > > >>>> > > > >>>> On Wed, May 22, 2019 at 1:52 PM Maxime Beauchemin < > > > >>>> [email protected]> wrote: > > > >>>> > > > >>>>> Python: > > > >>>>> pip install pipdeptree && pipdeptree > > > >>>>> > > > >>>>> NPM: > > > >>>>> cd superset/assets && npm ls > > > >>>>> > > > >>>>>> On Wed, May 22, 2019 at 11:09 AM Alan Gates < > [email protected] > > > > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Yes, I checked, it works now. I just haven't yet because I'm > > still > > > >>>>>> looking > > > >>>>>> at all the dependencies it pulls in. Maven makes this super > easy > > to > > > >> do, > > > >>>>>> but I need to learn enough about python setuptools to figure out > > how > > > >> to > > > >>>>>> check the licenses on those modules. > > > >>>>>> > > > >>>>>> Alan. > > > >>>>>> > > > >>>>>> On Wed, May 22, 2019 at 10:56 AM Bolke de Bruin < > > [email protected]> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> Is the signature now verifiable? Otherwise it won’t pass the > IPMC > > > ... > > > >>>>>>> > > > >>>>>>> Verstuurd vanaf mijn iPad > > > >>>>>>> > > > >>>>>>>>> Op 22 mei 2019 om 19:26 heeft Maxime Beauchemin < > > > >>>>>>>> [email protected]> het volgende geschreven: > > > >>>>>>>> > > > >>>>>>>> Oops, changing thread title this time around > > > >>>>>>>> > > > >>>>>>>> Vote passes! > > > >>>>>>>> > > > >>>>>>>> +3 binding votes (Max, Jeff & Abhishek) > > > >>>>>>>> +1 non-binding vote (Ville) > > > >>>>>>>> > > > >>>>>>>> No neutral or negative votes. > > > >>>>>>>> > > > >>>>>>>> On Tue, May 21, 2019 at 12:31 AM Jeff Feng > > > >>>>>> <[email protected] > > > >>>>>>>> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> +1 binding > > > >>>>>>>>> > > > >>>>>>>>> On Mon, May 20, 2019 at 3:54 PM Maxime Beauchemin < > > > >>>>>>>>> [email protected]> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> @Alan, looks like I messed up the signature somehow. I got > > > tangled > > > >>>>>> into > > > >>>>>>>>>> adding a new entry (moving from my gmail to my apache.org > > > >>>> address), > > > >>>>>>>>>> deleting the old one and my svn kungfu is beyond rusty... > > > >>>>>>>>>> > > > >>>>>>>>>> Oh I think I just forgot to run "svn commit" (maybe i ran > "svn > > > >>>>>> update" > > > >>>>>>>>>> instead?), so you should just have to import that new KEYS > > file > > > >>>> and > > > >>>>>> it > > > >>>>>>>>>> should work. > > > >>>>>>>>>> > > > >>>>>>>>>> Sorry about the confusion. All of this is pretty > error-prone, > > > >>>>>>> especially > > > >>>>>>>>>> the [few] first time[s] around. > > > >>>>>>>>>> > > > >>>>>>>>>> Max > > > >>>>>>>>>> > > > >>>>>>>>>> On Mon, May 20, 2019 at 11:29 AM Abhishek Sharma < > > > >>>>>>>>>> [email protected]> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>> > > > >>>>>>>>>>> +1 binding. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Newly built docker image > > > >>>>>>>>>>> < > > > >>>> > > > >> > > > > > > https://cloud.docker.com/u/abhioncbr/repository/docker/abhioncbr/docker-superset > > > >>>>>>>>>>> working fine. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thanks > > > >>>>>>>>>>> Abhishek > > > >>>>>>>>>>> > > > >>>>>>>>>>> On Mon, May 20, 2019 at 2:03 PM Alan Gates < > > > [email protected] > > > >>>>> > > > >>>>>>>>> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>>> Max, when I check the signature (gpg --verify ) it tells > me: > > > >>>>>>>>>>>> gpg: Signature made Sat May 18 15:36:55 2019 PDT > > > >>>>>>>>>>>> gpg: using RSA key > > > >>>>>>>>>>> 8CA186C4568E92301E5F2491A3B3BE2CCC1BB7E4 > > > >>>>>>>>>>>> gpg: Can't check signature: No public key > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> I imported the KEYS file referenced in your message, but > it > > > >>>>>> doesn't > > > >>>>>>>>>>> appear > > > >>>>>>>>>>>> to contain that key. I think you need to either generate > a > > > new > > > >>>>>>>>>> signature > > > >>>>>>>>>>>> with the key in the file and upload that .asc file to the > > dist > > > >>>>>> site > > > >>>>>>>>> (no > > > >>>>>>>>>>>> need to rerole the release itself) or place the key you > used > > > >>>> into > > > >>>>>> the > > > >>>>>>>>>>> KEYS > > > >>>>>>>>>>>> file. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Alan. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On Sat, May 18, 2019 at 4:01 PM Maxime Beauchemin < > > > >>>>>>>>>>>> [email protected]> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> Dear all, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> The source release 0.33.0 RC1 for Apache Superset is > baked > > > and > > > >>>>>>>>>>> available > > > >>>>>>>>>>>>> at: > > > >>>>>>>>>>>>> > https://dist.apache.org/repos/dist/dev/incubator/superset/ > > , > > > >>>>>> public > > > >>>>>>>>>>>>> keys are available > > > >>>>>>>>>>>>> at > > > >>>> > https://dist.apache.org/repos/dist/release/incubator/superset/KEYS > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> We're now attempting to use 0.33 as the base for the > first > > > >>>>>> release > > > >>>>>>>>> as > > > >>>>>>>>>>>>> opposed to 0.32 in previous attempts. Many > license-related > > > >>>> issues > > > >>>>>>>>> had > > > >>>>>>>>>>>> been > > > >>>>>>>>>>>>> solved by the process shipping visualizations as plugins, > > and > > > >>>>>> that > > > >>>>>>>>>>>>> migration wasn't completed on 0.32. This is the third ASF > > > >>>> release > > > >>>>>>>>>>>> candidate > > > >>>>>>>>>>>>> of Superset *We're still ironing out our release process, > > so > > > >>>>>> please > > > >>>>>>>>>>> bear > > > >>>>>>>>>>>>> with us and help if you can*. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> As I went along, I documented the process in > > > [yet-to-be-merged] > > > >>>>>>>>>>>>> RELEASING/README.md in the repo, latest edits here > > > >>>>>>>>>>>>> https://github.com/apache/incubator-superset/pull/7539 . > > As > > > >>>> part > > > >>>>>>>>> of > > > >>>>>>>>>>>>> `RELEASING/`, we ship docker files to help package and > test > > > >>>>>>>>> releases. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> For context the `0.33` release branch was cut at SHA > > > 51068f007, > > > >>>>>>>>> that > > > >>>>>>>>>>> was > > > >>>>>>>>>>>>> merged on master on Apr 17th. From that common ancestor, > > the > > > >>>>>>>>>> following > > > >>>>>>>>>>>> list > > > >>>>>>>>>>>>> of commit was added as cherry-picks. The SHAs in the list > > > >>>> bellow > > > >>>>>>>>>>>> reference > > > >>>>>>>>>>>>> the cherries on the release branch, PR number are > available > > > to > > > >>>>>> get > > > >>>>>>>>>> more > > > >>>>>>>>>>>>> details. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> 7591a709 (tag: 0.33.0rc1, apache/release--0.33) 0.33.0rc1 > > > >>>>>>>>>>>>> b7ffdb8b Improve instructions > > > >>>>>>>>>>>>> 4bbd68c6 Change babytux to open image in birth dashboard > > > >>>>>>>>>>>>> eaa679e8 remove unused LICENSE entries > > > >>>>>>>>>>>>> 42d50f9d Add Roboto font to LICENSE, remove glyphicons > > files > > > >>>>>>>>>>>>> 5ae2836b Address COPYRIGHT + LICENSE issues > > > >>>>>>>>>>>>> ea807f20 [WiP] Improvements related to ASF release > process > > > >>>>>>>>>>>>> c57ef5dc 0.31.0rc1.dev1 > > > >>>>>>>>>>>>> 51068f00 Adding permission for > > can_only_access_owned_queries > > > >>>>>>>>> (#7234) > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> > > > >>>>>>>>> *Jeff Feng* > > > >>>>>>>>> Product Lead > > > >>>>>>>>> m: (949)-610-5108 > > > >>>>>>>>> twitter: @jtfeng > > > >>>> > > > >> > > > > > >
