Dear Marcus, Thank you very much for the extensive list of fresh ideas :)
Just a few general explanations. One general rule is that the suggested ideas should be something that a mentor is able to implement himself in cca. 3 weeks. There might be exceptions if you meet a really brilliant student (but the program should also be able to work for students with basic programming skills, who never worked on any big project before, who have never seen or heard about Tcl before, who have never been using MacPorts before). One of the super important goals of the program is to introduce students to opensource and the project in particular, and keep them engaged with the project. If the tasks are too challenging for the students, they would likely drop out of the program and never return. If we give them something that they can comprehend, they are more likely to stick with the project. I'm not too familiar with the base development, so in many cases I'm unable to judge how much effort a certain task would be, I just wanted to point out that we should not ask students to solve problems which we are not able to solve ourselves. If the student gets stuck at some point, we definitely need to be able to help them continue. I hope that some others will provide further feedback about the suggested projects for the base. Also, in the past we were often stuck with the task being to complex, then maybe implemented 60-80%, never rigorously tested, and consequently never merged. This is bad and demotivating for both the student and the project. I would say that ideally the first official merge of the student's code should not be later than after 4 weeks, so that it can get proper testing etc. On Thu, 7 Feb 2019 at 14:12, Marcus Calhoun-Lopez wrote: > > [1] https://trac.macports.org/wiki/SummerOfCode#Projects > > 1. Update Orphaned Ports > I originally became interested in MacPorts development because I wanted > to install certain software, and the MacPorts version was (slightly) out of > date. Same for me. Maybe we should stress that out more clearly on the website. > Fixing and/or updating ports is an excellent introduction to the > MacPorts systems (phases, variants, PortGroups, etc.). > It forces you to read the documentation (but only as an absolute last > resort) and examine commit histories. > This would not be the entire GSOC project, but it could be used as part > of the application process and/or a supplement to the main project. Agreed. Actually, I would list this under "ideas for ports" section. For some super complex software, or for stuff like "setting up the whole ecosystem to allow reasonable packaging of npm or ruby packages" or some other less trivial piece of software that could be a full-time work for the summer. > 2. Tweak qt5 PG > Currently, the qt5 PG defines three procedures to declare dependencies > on Qt components > > (https://github.com/macports/macports-ports/blob/000ec3f12e02aa6c3159bd1608cd8607eb9e18f0/_resources/port1.0/group/qt5-1.0.tcl#L572). > These should almost certainly be `options`. I agree that there's a lot in Qt5 that could be done, and it could certainly fill the whole summer. Or KDE. > 3. Make blacklisting MacPorts compilers easier. > Currently, the compiler_blacklist_versions PG makes blacklisting ranges > of compilers very easy, but it only works for Xcode compilers. > It would be nice, for example, to be able to have something like > `compiler.blacklist-append {macports-clang < 6.0}`. > See https://trac.macports.org/ticket/56093. Agreed. Maybe not enough work for the full summer(?), but it could be one part of the task. > 4. Add support for x86_64h architecture. > x86_64h is a lot like x86_64, except it allows compiler to take > advantage of the Haswell microarchitecture. > Unlike other architectures in MacPorts, x86_64h and x86_64 libraries can > be linked together. > As a side note, GCC does not support `-arch x86_64h`, but Xcode and > MacPort Clang do. > See https://github.com/openssl/openssl/pull/6497 for a discussion on why > OpenSSL did NOT want to add support for x86_64h. This is completely out of my own expertise, so I cannot comment about it. (I've never seen anyone explicitly requesting this, but I also have absolutely no idea how much performance improvement this could bring.) > 5. Allow for multiple runs of each phase. > There are times when it is convenient to run configure/build/destroot > more than once. > For example: > *) cargo depends on the port cargo-stage1, whose only purpose is to help > build cargo. > Instead, one could run configure/build and use the resulting binary > without installing another port. > *) fluidsynth depends on a subproject, which has no use beyond aiding > the build process. > The subproject does not inherit the MacPorts settings. > It would be nice to configure/build the subproject properly (from a > MacPorts point of view). > See https://github.com/macports/macports-ports/pull/3583. > *) 75% of the muniversal PG is to get multiple runs of the > configure/build/destroot phases. This would be super useful for cross-compilers. For MinGW32-w64 we now ship three versions of gcc (two of them are bootstrapping versions that need to be thrown away) and in fact we have a cyclic dependency. However in that case one first needs to (a) build and activate a working bootstrapping compiler, (b) install some other packages [crt, newlib. etc.], (c) build and activate the compiler again, (d) install more packages, (e) build and activate the final version of the compiler. I suspect this use case is more complicated than the one you described above. Again, I have absolutely no idea how much work is needed and whether this can realistically be done by the student. > 6. Allow a variant to more elegantly become “undefaulted.” > When it was decided that support for the FLTK toolkit should no longer > be the default in octave, the (previously default) variant fltk was renamed > fltk_toolkit. > There is another port (I cannot remember which one) that hacks together > another solution to avoid renaming a previously default variant. > Is there a better solution? This would be an awesome feature. I don't know how much work is needed / how difficult it is to implement. > 7. Find a way to make mass revision increases easier. > There are times when upgrading a library means increasing the revision > of ALL of its dependencies (e.g. readline, icu, and libgcc). > Finding all of these ports can be tricky and error prone. > -) Some ports do not have an explicit dependency. > -) Some ports do not explicitly set a revision variable. > -) If there are sub-ports, then there might be multiple revision > variables. > Personally, I have a script that partially automates this process, but > it is far from ideal. > Is there a better way? This would be a super useful feature, but do you know how you would implement it yourself in a way that is not so far from ideal (assuming you had sufficient amount of time)? > 8. Find a solution to the longstanding issue of std::locale being broken in > GCC. > See https://trac.macports.org/ticket/15653 This is completely outside of my expertise, so I cannot comment on it. > 9. Prevent `port reclaim` from removing build dependencies. > See > https://lists.macports.org/pipermail/macports-users/2019-January/thread.html#46344 This sounds doable by the student. (Maybe not on the super top of our priority list, but still useful.) Btw: Homebrew now runs the cleanup by default after each port installation. I would say that for MacPorts this is somewhat too dangerous, in particular on the plenty of cases where we don't provide the binaries. What I would also like to see is that after uninstalling a particular port or upgrading a port that now requires different dependencies, MacPorts would tell you that you can probably remove some other unrequested dependencies. I don't know how difficult that would be to implement. > 10. Improve the finding of dependencies. > Finding dependencies can sometimes be difficult: > *) Read the project documentation, which is sometimes incomplete. > *) Read the code, but dependencies can be quite well hidden within the > build system or the code itself. > *) Run the port command in trace mode, which is excellent, but not > 100%. > *) Wait for the bug reports. > Can this be improved? > See https://trac.macports.org/wiki/SummerOfCode#dependencies-gen. The project description on the wiki sounds ok to me. (I wouldn't know how to mentor or implement it myself, but that's a different topic.) Clemens apparently had a very clear idea how to do it (it's outside of my expertise). > 11.Give Portfile better access to CFLAGS, CXXFLAGS, etc. > Currently, CFLAGS and the like are set late in the process (arch flags > added, -pipe added, etc.). > Of course, there are good reasons for this. > Unfortunately, this means that there is no Portfile access to these > values. > This causes some ugly workarounds such as duplication of code from the > base or `configure.cmd printenv` and `configure.post_args {>> > Makefile.macports.inc}`. > Is there a way to give the Portfile access to most of these values? > See > https://lists.macports.org/pipermail/macports-dev/2018-November/thread.html#39694 > See > https://github.com/macports/macports-base/blob/5e17a692f991a55a7f450e40ce47ab6fe0f2da57/src/port1.0/portconfigure.tcl#L1367 [unable to comment] > 12. Simplify compilers and mpi PGs with recent base changes. > Recently, significant changes were made in the base code concerning > compiler selection. > Is there a way to use these changes to simplify the PortGroups that > manage compiler selection via variants? > See https://github.com/macports/macports-base/pull/88 > See https://trac.macports.org/ticket/55139 I cannot judge the complexity and suitability of this one either. > 13. Buildbot & Continuous Integration Improvements > I am afraid these are well outside of my knowledge area, but here are a > few issues that have arisen lately. > *) > https://lists.macports.org/pipermail/macports-dev/2018-November/039651.html How is this related to our CI? It's just that the user was surprised to see that his commit did not end up in his MacPorts installation, but that has more to do with our release cycles that our CI. > *) > https://lists.macports.org/pipermail/macports-dev/2018-April/thread.html#38023 > *) > https://lists.macports.org/pipermail/macports-dev/2018-July/thread.html#39148 This is one of the most burning issue in our CI. The idea could be addressed (improving our CI infrastructure to process PRs on buildbot all the way down to 10.6 or even 10.5/10.4), but it would have to be a brilliant student. It's a bad idea to ask a student to do something for which you have no idea how to implement yourself, unless you get someone outstanding. Zero King was an example of a super skilled student (we knew him from before GSOC) and we tried to ask him to implement testing of pull requests on our own infrastructure instead of working on Travis, but he was afraid of the zillion (mostly security-related) issues related with that, and went with Travis (for which we are really thankful, even if not perfect, as it still simplified our PR processing a lot). It could be that someone would step up and set up virtualisation of multiple OS versions in a few days, then do some minor fixes in the buildbot configuration to fire up those VMs with libvirt; something that might take 2 weeks to finish. I would put this idea on the page, albeit with a warning sign that only skilled students should attempt to do it. > *) > https://lists.macports.org/pipermail/macports-dev/2019-February/040070.html The student cannot be responsible for fixing the lack of disk space on the machine in Ryan's cellar :) OK, Ryan did recently suggest some ideas for auto-cleanup, but I'm not even sure that we know outselves what would be the best solution. I'm not sure if it's a good idea to ask a student to work on this. Mojca
