Dealing with False Positive during Livecheck
Hi, Recently I took upon a journey to try to update ports which are tagged as 'nomaintainer' and noticed that some of the ports give out false positives during livecheck. This is due to the reason that upstream decided to tag a newer version without actually updating the code / tarball. Also the upstream seem to be inactive since the last changes were made >5 years ago. Few of the ports that I have noticed till now are: 1. aacgain 2. chemtool I will be updating the list and probably put it up on wiki, so that in future we don't spend time looking for the updated tarballs. Feel free to add ports which you might have also encountered. P.S: I didn't check the archives to see if this topic was discussed previously. -- Jackson Isaac
"Accidentally" added 35 and 36 subports to "backport" python modules
Hi, I though it would be about time to add 35 and 36 to all python submodules. https://github.com/macports/macports-ports/pull/1561 I kept in mind that some might not work immediately, but I forgot that some are simply not designed to be used with python 3.6, like many modules for backporting functionality from newer version of python to older ones. I thought that opening a PR would remind maintainers, but the number of ports was above the limit to send any notifications out. I would like to apologise for adding 35 and 36 to ports that were not meant to be used with python 3.5 and/or 3.6. If you know of some ports like this: can you please either point me to them or revert my too-fast change? Some of the ports with "backport" in name are easy to spot, but even then it's not clear from which version the functionality is being backported. Thank you, Mojca
Re: Releasing 2.4.3
On 2018-04-10 19:39, Ryan Schmidt wrote: > Do you want to merge this performance enhancement too: [...] > And these bugfixes: I assigned this to the MacPorts 2.4.4 milestone, so we do not forget about them: https://trac.macports.org/ticket/56267 Rainer
Re: Releasing 2.4.3
On 2018-04-10 21:06, Ryan Schmidt wrote: > I'm not going to merge them, because I don't know how, and don't have the > energy to attempt to learn how to do so, given the zillion other things I'm > trying to do right now. This is even documented: https://trac.macports.org/wiki/WorkingWithGit#cherrypick > I proposed a list of commits to backport for 2.4.3 back in December. You've > merged those now; thank you, but it's now 4 months later. I don't want > another 4 months to go by before we bring the other critical fixes we've > already committed to master to our users. The use of MacPorts git and svn on > older systems spring to mind as pretty important fixes for older systems. I do not want to wait months for a release either. I think that the long time between releases is mainly caused by our release process. I cannot sign package installers. As long as we insist on having them for each and every release, I am not in the position to cut a full release anyway. So, what are we going to do? I already proposed not to wait for .pkg installers and just release to selfupdate whenever the release is ready. Rainer
Re: Releasing 2.4.3
On Apr 10, 2018, at 13:47, Rainer Müller wrote: > On 2018-04-10 20:26, Ryan Schmidt wrote: >> >> On Apr 10, 2018, at 12:54, Rainer Müller wrote: >>> >>> At some point we should make a cut and not add many changes in the last >>> minute... >> >> Well I can't force you to merge them. > > I cannot stop you either... I am just afraid that by merging changes in > the last minute we are likely to introduce problems we did not think of > before. Not all changes in backported commits can be applied cleanly as > master and release-2.4 diverged, so they might need to be modified. I'm not going to merge them, because I don't know how, and don't have the energy to attempt to learn how to do so, given the zillion other things I'm trying to do right now. I proposed a list of commits to backport for 2.4.3 back in December. You've merged those now; thank you, but it's now 4 months later. I don't want another 4 months to go by before we bring the other critical fixes we've already committed to master to our users. The use of MacPorts git and svn on older systems spring to mind as pretty important fixes for older systems. > The decision whether to merge them or not would be by Joshua now, as he > is going to tag the release.
Re: Releasing 2.4.3
On 2018-04-10, at 12:02 PM, Ryan Schmidt wrote: > > On Apr 10, 2018, at 13:55, Ken Cunningham wrote: > Mixing libc++ and libstdc++ results in a link failure. Mixing old libstdc++ > and new libstdc++ did not previously result in a link failure; it resulted in > a runtime crash which was difficult to diagnose and attribute to this > problem. That was back before gcc5. I don't know things have changed. > Mixing the old libstdc++ and the new libstdc++ usually results in a std::c++11__::something link failure, from what I have seen. I have never seen a runtime crash, although I haven't been doing this for years and years like some of you. Anyway, less desirable option, to be sure. Best, Ken
Re: Releasing 2.4.3
On Apr 10, 2018, at 13:55, Ken Cunningham wrote: > On 2018-04-10, at 11:47 AM, Rainer Müller wrote: >>> >>> I don't want to release 2.4.3 with the libstdc++ ABI issue unanswered. I >>> believe it will cause problems. >> >> Then we should revert this change for 2.4.3 and we have one more reason >> to go for a 2.4.4 release. >> >> Rainer > > > I would have like to set libgcc to ABI4 mode, at least on PPC, to fix this > forever, but Jeremy felt this was undesirable. Perhaps we might still > consider doing that, though, at least on PPC. > > Two other easy options: > > 1. add the required *ABI=0 flag by default to all cxx builds on PPC. This > would be completely fine I believe. More desirable. And then after MacPorts 2.4.3 release, revbump any ports that were built with *ABI=1, and add to them an invocation of macports_version and vercmp to ensure they're being rebuilt by 2.4.3 or later. > 2. wait for builds to fail to link (they will, on 10.6 to 10.8 and PPC if the > linking doesn't work, as we have all seen) then add the cxx11 PortGroup as we > do now when we see this failure. Less desirable. Mixing libc++ and libstdc++ results in a link failure. Mixing old libstdc++ and new libstdc++ did not previously result in a link failure; it resulted in a runtime crash which was difficult to diagnose and attribute to this problem. That was back before gcc5. I don't know things have changed.
Re: Releasing 2.4.3
On 2018-04-10 20:26, Ryan Schmidt wrote: > > On Apr 10, 2018, at 12:54, Rainer Müller wrote: >> >> At some point we should make a cut and not add many changes in the last >> minute... > > Well I can't force you to merge them. I cannot stop you either... I am just afraid that by merging changes in the last minute we are likely to introduce problems we did not think of before. Not all changes in backported commits can be applied cleanly as master and release-2.4 diverged, so they might need to be modified. The decision whether to merge them or not would be by Joshua now, as he is going to tag the release. >> I do not want to delay 2.4.3 any longer to finally release the fix for >> the broken trace mode on High Sierra. > > I don't want to release 2.4.3 with the libstdc++ ABI issue unanswered. I > believe it will cause problems. Then we should revert this change for 2.4.3 and we have one more reason to go for a 2.4.4 release. Rainer
Re: Releasing 2.4.3
On Apr 10, 2018, at 12:54, Rainer Müller wrote: > > On 2018-04-10 19:39, Ryan Schmidt wrote: >> >> On Apr 10, 2018, at 12:07, Ryan Schmidt wrote: >> >>> On Apr 9, 2018, at 17:15, Rainer Müller wrote: >>> If nobody objects, I am going to tag a new release on Tuesday and prepare tarballs. >>> >>> Please merge my other two commits from this ticket: >>> >>> https://trac.macports.org/ticket/55492 >>> >>> My first commit, which you merged, was buggy and needed fixing. >> >> Do you want to merge this performance enhancement too: > > At some point we should make a cut and not add many changes in the last > minute... Well I can't force you to merge them. > I do not want to delay 2.4.3 any longer to finally release the fix for > the broken trace mode on High Sierra. I don't want to release 2.4.3 with the libstdc++ ABI issue unanswered. I believe it will cause problems. > I would save reviewing and backporting of these fixes for a 2.4.4 soon > after.
Re: CI system for PR builds
On 7 April 2018 at 15:45, db wrote: > On 7 Apr 2018, at 14:37, Ryan Schmidt wrote: > > Is buildbot running on your basement??? Yes (not mine). > Streamlining this process could have been something for GSoC. Until the autumn of 2016 we had no concept of pull requests at all and we had nearly no influence on what exactly ran on the buildbot infrastructure. Until the late summer of 2017 we had no checking of pull requests at all. We in fact had a GSoC student in 2017 who set up automated testing of pull requests using Travis. We were discussing alternatives (adjusting the buildbot) to achieve that goal before GSOC, but the student was not interested in working on that, also considering all the security-related issues which would be causing headaches that he was not willing to deal with. Despite that, It is too late for a GSoC project in 2018 (we did discuss this idea a month or two months back when we had slightly less insight into how this could work properly in the first place; that was before Rainer knew what he wrote in one of these emails; and the student would have to be self-motivated to figure out the solutions himself since most of us would not be able to help without spending a lot of time ourselves). If you want to have this functionality as a product of GSoC, we cannot get it before autumn 2019 and even then if and only if: - we actually find a student willing to work on that, - the student successfully finishes the project. > Testing and reproducibility, doesn't seem to me as user to be a prime concern > in MP. We still have a long way to go before we can achieve 100% binary reproducibility. However, regarding other points: - We only have 4 developers with mostly full-time jobs (unrelated to MacPorts) who work on MacPorts infrastructure in their free time. Apart from the infrastructure those same developers are almost the only ones developing MacPorts base and maintaining more than 1000 ports. It is unfair to expect magic powers from them. - Making sure that 20k ports keep being up-to-date and working on almost everything from 10.6 to 10.13 is anything but trivial. HomeBrew "solved" the problem by archiving lots of working formulas and only keeping the most popular ones and leaving it up to others to provide private taps (leaving a lot of users quite unhappy, some of them switching to another "scientific" cross-platform package manager). - I don't think that our #1 problem are new commits. OK, some committers take somewhat less care before committing. In my (personal) opinion a much bigger problem is: * a huge number of existing broken or outdated ports (or both) * our inability to efficiently address 2400 known bugs (no patch), 400 submissions (with code), 300 other patches from Trac, ... see https://trac.macports.org/wiki/Tickets * that we currently have no overview which ports exactly are broken and on which platforms We now have a very details plan to address the latest point (getting a clear overview of which ports are broken), with a pretty high priority. What we need right now is mostly *code* (hardware would also help) as well as time & effort to help us reduce the number of all those Trac tickets, rather than pointing fingers to what MacPorts volunteers are not doing. > That streamlined process is what keeps new and updated portfiles in my local > repo… I have no clue what you wanted to say with this. Mojca
Re: Releasing 2.4.3
On 2018-04-10 19:39, Ryan Schmidt wrote: > > On Apr 10, 2018, at 12:07, Ryan Schmidt wrote: > >> On Apr 9, 2018, at 17:15, Rainer Müller wrote: >> >>> If nobody objects, I am going to tag a new release on Tuesday and >>> prepare tarballs. >> >> Please merge my other two commits from this ticket: >> >> https://trac.macports.org/ticket/55492 >> >> My first commit, which you merged, was buggy and needed fixing. > > Do you want to merge this performance enhancement too: At some point we should make a cut and not add many changes in the last minute... I do not want to delay 2.4.3 any longer to finally release the fix for the broken trace mode on High Sierra. I would save reviewing and backporting of these fixes for a 2.4.4 soon after. Rainer
Re: Releasing 2.4.3
On Apr 10, 2018, at 12:44, Rainer Müller wrote: > Building package installers is the most time-consuming operation in our > release process. Therefore I was asking whether it is actually worth to > wait for them to appear before releasing to selfupdate. I am rebuilding our installer creation process to hopefully make this much easier for MacPorts 2.5 and later.
Re: Releasing 2.4.3
I am not comfortable releasing a version of MacPorts that includes a MacPorts gcc in the default compiler list (which 2.4.3 now does) without having answered the new-vs-old ABI question satisfactorily. I believe releasing that will lead to the libstdc++ mismatch crashes that were the reason why we previously strongly prohibited the use of MacPorts gcc for C++ code: https://lists.macports.org/pipermail/macports-dev/2018-March/037829.html I had hoped Jeremy H would weigh in, but I don't think he has yet. Or did we reach a consensus on how we would handle this, and I've forgotten?
Re: Releasing 2.4.3
On 2018-04-10 19:07, Ryan Schmidt wrote: > > On Apr 9, 2018, at 17:15, Rainer Müller wrote: > >> If nobody objects, I am going to tag a new release on Tuesday and >> prepare tarballs. > > Please merge my other two commits from this ticket: > > https://trac.macports.org/ticket/55492 > > My first commit, which you merged, was buggy and needed fixing. Thanks for noticing. I cherry-picked the other two commits now. https://github.com/macports/macports-base/commit/47d219119281a5f574ee6298c9e731293bca464e https://github.com/macports/macports-base/commit/a98fe3b89fba99f8bb873663f85a04831718583b >> Should I wait with publishing the update over selfupdate until we also >> have installer packages ready? > > Sure? New releases could be made available via selfupdate immediately. Old installers will already always selfupdate to the latest version automatically, so the the result is the same anyway when using on older installer. Building package installers is the most time-consuming operation in our release process. Therefore I was asking whether it is actually worth to wait for them to appear before releasing to selfupdate. However, for 2.4.3, Joshua will tag the release instead, as he is also going to build installers right away. Rainer
Re: Releasing 2.4.3
On Apr 10, 2018, at 12:07, Ryan Schmidt wrote: > On Apr 9, 2018, at 17:15, Rainer Müller wrote: > >> If nobody objects, I am going to tag a new release on Tuesday and >> prepare tarballs. > > Please merge my other two commits from this ticket: > > https://trac.macports.org/ticket/55492 > > My first commit, which you merged, was buggy and needed fixing. Do you want to merge this performance enhancement too: macports_util: Optimize lshift and lunshift https://github.com/macports/macports-base/commit/cf998981c078c9804f1f8e0ce245796278876474 And these bugfixes: port: Add missing -N flag to usage https://github.com/macports/macports-base/commit/80d65609e11fc0b32f0f5f86e0696d9e244f7707 port: Remove misleading [privopts] from usage https://github.com/macports/macports-base/commit/f166b4809f88ddbb7fe77a495d5159fc87f764a4 reclaim: Split distfile into filename and disttag https://github.com/macports/macports-base/commit/3ad97625cc57d26e333dcdf06dc0d932f002056a Update copyright year https://github.com/macports/macports-base/commit/fa20ccb51e64fb2773125a1e5bd392a2fe322c90 Update Apple developer site URL in readme https://github.com/macports/macports-base/commit/9f7ffde631117650b96641595f626512409a3cbb Add link to Guide to readme https://github.com/macports/macports-base/commit/a3e250224da7667b4970f381878d5cd9c7fc6df2 HACKING: Add links to the MacPorts Guide https://github.com/macports/macports-base/commit/d7fd81a85065c29092e71743716004ba0eb578b2 reclaim.tcl: don't ask for partial removal https://github.com/macports/macports-base/commit/3578400bf5f593c92b1bc412a45776fe55c0db78 port mirror: Respect special license NoMirror https://github.com/macports/macports-base/commit/fae17c2c7b97f2c6b8d9ab527ef6776b93353a00 tests: Avoid bogus error message during test runs https://github.com/macports/macports-base/commit/4c11f3bea113e305776751e98d5236e34175a972 tests: Fix test failure due to missing SVN command https://github.com/macports/macports-base/commit/ea77eb50b06f009629afb95321f0d5cd932c070b tests: Remove non-existing constraint from test https://github.com/macports/macports-base/commit/0745d2d4efcbe082e13828f31c2c594c2be0a0a4 tests: Avoid using network in port1.0 unit tests https://github.com/macports/macports-base/commit/521e0143893f89c35f084205f9230fbd24e30165 The three commits for using MacPorts svn on newer macOS versions to fix fetch failures: https://trac.macports.org/ticket/55933 The two commits for using MacPorts git on newer macOS versions to fix fetch failures: https://trac.macports.org/ticket/55989 Fix trace mode when using ccache-devel: https://trac.macports.org/ticket/55680 Add size to default checksums: https://github.com/macports/macports-base/commit/95a153e6a4bff8b99502806960c916a2b5d20504 Correct default value of distpath in docs https://github.com/macports/macports-base/commit/b7e7fe52d7e6450181232ee635597fff56722c69 Don't allow non-integer epoch or revision https://github.com/macports/macports-base/commit/387cd8ca7c7a54a38ea24fc1d698444720dfa8eb Remove lint check for non-integer revision, epoch https://github.com/macports/macports-base/commit/dcf6a81c69556b0c03b508122f176f01f9d8da1e
Re: Releasing 2.4.3
On Apr 9, 2018, at 17:15, Rainer Müller wrote: > If nobody objects, I am going to tag a new release on Tuesday and > prepare tarballs. Please merge my other two commits from this ticket: https://trac.macports.org/ticket/55492 My first commit, which you merged, was buggy and needed fixing. > Should I wait with publishing the update over selfupdate until we also > have installer packages ready? Sure?
Re: Releasing 2.4.3
On Apr 9, 2018, at 22:43, Eric A. Borisch wrote: > I'd still like to see the opportunistic (via libarchive / bsdtar) compression > (pull request #45) integrated at some point; with modern machines and their > soldered-down storage, it's all the more useful. Sure, but this thread is about 2.4.3, a bugfix release. New features won't go into a bugfix release. > Shall I open a ticket in trac as a feature request for 2.5? A ticket already exists for that feature: https://trac.macports.org/ticket/36560
