There is the resent distribution of the nb-javac distribution thread leveraging compiled binaries there as well. Would the profiler components fit in the same model? So make a project for the profiler, cause the builds, have a binary release component then pull this in?
On Sat, Jan 16, 2021 at 10:22 AM Eric Bresie <[email protected]> wrote: > On Sat, Jan 16, 2021 at 9:58 AM Jan Lahoda <[email protected]> wrote: > >> On Sat, Jan 16, 2021 at 2:48 PM Lars Bruun-Hansen <[email protected] >> > >> wrote: >> >> > > What would it take for the CDDL code to be relicensed to AL2 and >> > incorporated? >> > >> > The C code involved here [1] is indeed already AL2 licensed, meaning >> > it has already been donated from Oracle and is now incorporated into >> > the NetBeans Git repo. However, no binaries are ever build from this >> > code because a proper build pipeline is missing. Instead the NetBeans >> > distribution (still) uses binaries build long, long time ago from the >> > Oracle-Hg repo and thus those binaries are CDDL licensed. >> > >> > >> > > So is one possibility to break out the bin zip into multiple zips to >> > allow more independence between bin files? >> > Theoretically, yes. Say if one wanted to avoid re-building the binary >> > for MacOS just because there's a need for a re-build of the Windows >> > binary. But as I mention such separation would mean quite a lot of >> > changes to the Java code which utilizes the binary package. I would >> > >> >> I don't think it would be so bad. The zip with the binaries is unpacked by >> the build process, and (AFAIK) only this unpacked version is used in the >> product. It would be trivial to have another AL2-licensed zip, which would >> be unpacked over the unpacked CDDL licensed zip, overwriting same-named >> files. >> >> >> > question if it is worth the effort compared to biting to bullet once >> > and for all by making sure that these binaries can be auto-build from >> > the code in the Apache NetBeans git. >> > >> >> Well - if someone is up to this task, I surely don't have anything >> against. >> But please be prepared this means compiling the code on many platforms, >> some of which might be considered marginal. >> >> Regarding "auto-build" - I am not sure what exactly is the idea there. The >> standard NetBeans builds are generally meant to be platform neutral, not >> sure what is the proposed way to achieve that. Is there a way to >> cross-compiling a library for e.g. MacOS X on any other platform? >> > > So recently while working a given PR, when checked in to github I noticed > it performed some pulls, build, and sanity checks which seemed to have some > platform specific checks. So is something similar to that in the build > phased needed then? > > Where does that type of automated build activity get define? > > Are these platform (is) specific components something that would be better > broken out all together to allow independent builds? Or is it still pretty > Netbeans centric enough that it doesn’t make sense to separate it? > > Given the recent Netbeans Java editor VSCode integration is this another > case of breaking it down and handled similar to that? > > Assume there maybe some flavor of platform specific build flag or platform > specific build environment builds in use during build? > > So have platform specific build earlier in the mix and then higher level > (Java based) build after that! > > Where are the platform specific aspects contained in the codebase? Assume > this is the “native” code mentioned earlier? > > >> > >> > There is at least one positive example where setting up such build >> > pipeline for C/C++ code has actually been done in Apache NetBeans >> > universe, namely the Windows launcher. See [2]. Something similar >> > needs to be done for the Profile Launcher binary. >> > >> >> Note that this is optional, and requires non-trivial set-up. At very >> least, >> it needs MinGW, and I am not clear if this is tested outside of the Linux >> environment. Overall, I suspect most NetBeans builds done by developers >> don't actually build the launchers. Also, compared to the profiler >> binaries, this is very simple: the only target is Windows. >> >> Overall, my suggestion would be to create a new zip for the AL2-licensed >> profiler binaries, and unpack it over the unpacked content of the existing >> CDDL-licensed archive. Should be reasonably easy to setup. >> > > Where is the build for these defined? > > Jan >> >> > /Lars >> > >> > [1] >> > >> https://github.com/apache/netbeans/tree/master/profiler/lib.profiler/native >> > [2] https://github.com/apache/netbeans/pull/1073 >> > >> > On Sat, Jan 16, 2021 at 2:03 PM Eric Bresie <[email protected]> wrote: >> > > >> > > So is one possibility to break out the bin zip into multiple zips to >> > allow more independence between bin files? >> > > >> > > What would it take for the CDDL code to be relicensed to AL2 and >> > incorporated? >> > > >> > > Not knowing enough about said components but could the in some way be >> > extracted out in some way an published someplace (maven central, etc.) >> and >> > made a compile time dependency in Netbeans which is pulled from >> repository >> > at build time (maybe runtime)? >> > > >> > > Eric Bresie >> > > [email protected] (mailto:[email protected]) >> > > >> > > > On January 13, 2021 at 4:49:21 AM CST, Ernie Rael < >> [email protected] >> > (mailto:[email protected])> wrote: >> > > > Lars, maybe I'm just lucky in not knowing all the build >> complexities. I >> > > > have no direct experience with CI at all even in small projects. >> > > > >> > > > My simplistic view is that somewhere there is a binary and that it >> > could >> > > > be replaced with a new binary file. And. if needed, stick another >> > > > license file (or reference to a license file) somewhere. I haven't >> > heard >> > > > it said that the same binary used for 32 and 64 bit archs; so I >> guess >> > > > that's not a problem. >> > > > >> > > > 1) and 2) are nice and, from what you say, easy. Considering that >> the >> > > > bits have been in use for over two years, it doesn't seem required; >> but >> > > > hey, it's easy so why not. >> > > > >> > > > The problem is back to your point 3). I'll never understand the >> process >> > > > well enough to follow the discussion in any meaningful way (and I'm >> > > > quite content with that); and in my ignorance I will never believe >> that >> > > > some binary can't just be replaced without requiring all this other >> > > > stuff to happen. Worst case, the dll could be replaced during >> > > > installation, or even downloaded through the plugin manager couldn't >> > it? >> > > > >> > > > I guess there's a legitimate argument that things can't be fixed >> > > > piecemeal, because of support costs. Perhaps some investment in >> > > > "piecemeal fix" tools, would have great dividends considering all >> the >> > > > things that can't be done because they don't fit into the right way >> to >> > > > do it. >> > > > >> > > > With the cut off for the next release at hand, I've brought the >> issue >> > > > up. I'll shut up about it now... At least until it's time for the >> next >> > > > release ;-) >> > > > >> > > > -ernie >> > > > >> > > > On 1/12/2021 11:44 PM, Lars Bruun-Hansen wrote: >> > > > > Ernie, my comments were indeed very specific to the case at hand: >> the >> > > > > fix to the native binary for the Profiler Launch for Windows 64 >> bit >> > > > > and how to get that fix into the NetBeans distribution. >> > > > > >> > > > > It just happens to be exactly the same challenge in the other >> cases >> > I mention. >> > > > > >> > > > > I wrote my piece because I looked into if I could contribute for >> this >> > > > > specific problem (NETBEANS-1428, PR-2021). >> > > > > Let's recap: >> > > > > >> > > > > 1. Pedro's PR-2021 needs to be further reviewed and then - if >> > approved - merged. >> > > > > 2. Then a new binary needs to be build for Windows 64 bit, the >> file >> > > > > profilerinterface.dll, based on the new Apache code, i.e. the code >> > > > > which was just merged. It is not as such difficult to build the >> new >> > > > > DLL locally, I can do it. Pedro has already done it (#1). No >> sweat. >> > > > > 3. The new binary then needs to become part of >> > > > > https://netbeans.osuosl.org/binaries/ >> <hash>-profiler-external-binaries-8.2.zip >> > ( >> > >> https://netbeans.osuosl.org/binaries/%3Chash%3E-profiler-external-binaries-8.2.zip >> > ) >> > > > > (#2) in order to be picked up by the NetBeans build pipeline. >> > However, >> > > > > we cannot just replace that single DLL in that ZIP because then >> the >> > > > > ZIP file would become a mix of CDDL licensed binaries and AL2 >> > licensed >> > > > > binaries and from two different source code repos. Ouch!. So >> there's >> > > > > some complexity there. As a quick fix we want to avoid rebuilding >> for >> > > > > all those other platforms and only re-build for Windows, perhaps >> even >> > > > > only for Windows 64bit, right? We can maybe change the Java code >> for >> > > > > lib.profiler so that there's a external binaries ZIP for Windows >> and >> > > > > another for the rest of the platforms but that is a somewhat >> involved >> > > > > task and it adds complexity. Again, this specific design, the >> common >> > > > > ZIP, is shared with the other use cases I mention and the issue >> about >> > > > > integrating a new binary is exactly similar. >> > > > > >> > > > > This is why I struggle to see an easy quick fix for the problem at >> > > > > hand, integrating the new binary for the Profiler Launcher solely >> for >> > > > > Windows 64 bit. You very quickly end up in a situation where you >> need >> > > > > to rebuild binaries for MacOS and Linux too. >> > > > > >> > > > > I understand your frustration but I think this explains why people >> > > > > keep "hijacking" or digressing to the bigger picture as you >> mention >> > > > > and then the specific issue stalls. How annoying. The reason is >> it is >> > > > > just not that easy to see a path to a quick fix. From my analysis >> my >> > > > > conclusion is that we might as well tackle the bigger picture, >> i.e. >> > > > > creating an ASF build pipeline for those binaries. And I'm >> normally >> > > > > all for quick fixes :-) >> > > > > >> > > > > >> > > > > Peace. >> > > > > >> > > > > /Lars >> > > > > >> > > > > Ref #1: https://github.com/pedro-w/netbeans/releases/tag/v2 >> > > > > Ref #2: Currently >> > > > > >> > >> https://netbeans.osuosl.org/binaries/45225DCFC94A9782419E95C70957822A0C44612D-profiler-external-binaries-8.2.zip >> > > > > as per the file >> > > > > >> > >> https://github.com/apache/netbeans/blob/master/profiler/lib.profiler/external/binaries-list >> > . >> > > > > >> > > > > On Wed, Jan 13, 2021 at 3:00 AM Ernie Rael <[email protected] >> > (mailto:[email protected])> wrote: >> > > > > > This thread is about /Profiling on 64bit Windows/. With hopes to >> > find a >> > > > > > short term solution; quick and dirty if nothing more general is >> > available. >> > > > > > >> > > > > > This subject has been avoided for years often time by hijacking >> and >> > > > > > starting to talk about the big picture issue. Which of course is >> > > > > > important and includes important issues and information. Some of >> > this >> > > > > > may have been in various discussions over the past years. (I >> don't >> > know) >> > > > > > Maybe there's an issue open about it somewhere, where you could >> > > > > > contribute; or start a new thread. >> > > > > > >> > > > > > Where relevant, it would be great to reference those big picture >> > here. >> > > > > > But not avoid the issue at hand. >> > > > > > >> > > > > > Apologies (and thanks for the space) for blowing off steam. >> > > > > > -ernie >> > > > > > >> > > > > > On 1/12/2021 12:13 AM, Lars Bruun-Hansen wrote: >> > > > > > > I think we as a community need to fix the "native builds" >> issue >> > very >> > > > > > > soon. PRs are piling up. >> > > > > > > >> > > > > > > This current email thread is not the only one. Over the past 2 >> > years >> > > > > > > fixes have been made (PRs merged) to native binaries code in >> the >> > areas >> > > > > > > of >> > > > > > > >> > > > > > > * Native Launchers >> > > > > > > * NBI (installers) >> > > > > > > * Profiler launcher >> > > > > > > >> > > > > > > The problem is the same in all cases: The´source code is fixed >> > but >> > > > > > > there's never a new binary being build hence nothing really >> > happens. >> > > > > > > >> > > > > > > These binaries were never re-build in an Apache NetBeans >> > setting. They >> > > > > > > were simply copied from their old Oracle-era location to >> > > > > > > https://netbeans.osuosl.org/binaries from where they are >> picked >> > up by >> > > > > > > the Apache NetBeans build process but labelled as something >> > which has >> > > > > > > a CDDL license. This is legally correct but is a limbo state: >> The >> > > > > > > source code is (now) owned by ASF but for reason of lack of >> time >> > we >> > > > > > > are still using the pre-ASF binaries. I can 100% understand >> why >> > such >> > > > > > > "shortcut" was made and salute those who made it happen. >> > > > > > > >> > > > > > > However, obviously it is sub-optimal that parts of the >> NetBeans >> > > > > > > Platform and IDE distribution effectively cannot be bug-fixed. >> > > > > > > >> > > > > > > The right solution IMHO is that those parts of the Apache >> > NetBeans >> > > > > > > universe would be a first-class citizen in the general Jenkins >> > build >> > > > > > > pipeline for the Platform and IDE distribution. By >> first-class I >> > mean >> > > > > > > that the binary is automatically re-build whenever the source >> > code >> > > > > > > changes. This would require Jenkins build slaves for Linux, >> > MacOS and >> > > > > > > Windows (which I understand that ASF indeed has) and it would >> > probably >> > > > > > > require quite a bit of setup work to get right. Apache >> NetBeans >> > is a >> > > > > > > Java-oriented community so knowledge about building C/C++ >> > binaries is >> > > > > > > limited which is probably why nobody so far is stepping >> forward. >> > As >> > > > > > > Ernie suggests there may be a short term solution too which is >> > more >> > > > > > > quick-and-dirty i.e. someone would re-build locally, verify >> the >> > > > > > > binaries and then upload as one-shot fix. However it is not >> > always >> > > > > > > that simple. The binaries are distributed as one ZIP. If one >> > decides >> > > > > > > to only re-build the binary for the Windows platform then this >> > current >> > > > > > > packaging breaks down because then you have one binary in the >> ZIP >> > > > > > > which is ASF and the others which are CCDL and no longer build >> > from >> > > > > > > the same revision in the VCS. It would be a mess. Therefore, >> in >> > many >> > > > > > > ways, I'm not sure the quick-and-dirty solution really exists. >> > > > > > > >> > > > > > > Is this a correct status ? >> > > > > > > >> > > > > > > Lars >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > On Tue, Jan 12, 2021 at 2:07 AM Eirik Bakke < >> [email protected] >> > (mailto:[email protected])> wrote: >> > > > > > > > Build issues aside, I think the code in the PR actually >> needs >> > at least a cursory review from someone else than the original author. >> > Others, like myself, have passed by and tested the fix or left other >> > comments, but I'm not sure anyone sat down and reviewed it all in one >> > sitting. >> > > > > > > > >> > > > > > > > ( https://github.com/apache/netbeans/pull/2021 ) >> > > > > > > > >> > > > > > > > -- Eirik >> > > > > > > > >> > > > > > > > -----Original Message----- >> > > > > > > > From: Ernie Rael <[email protected] (mailto: >> > [email protected])> >> > > > > > > > Sent: Monday, January 11, 2021 12:34 PM >> > > > > > > > To: [email protected] (mailto:[email protected] >> ) >> > > > > > > > Subject: Re: Profiling on 64bit Windows >> > > > > > > > >> > > > > > > > On 1/11/2021 6:53 AM, Geertjan Wielenga wrote: >> > > > > > > > > What needs to be done? Can you provide a PR? >> > > > > > > > Sadly no. I suspect that someone who knows their way around >> a >> > distribution would have that knowledge. I could be wrong, but I suspect >> > it's simple. >> > > > > > > > >> > > > > > > > If it is simple and it hasn't been done in two years, I'd >> > guess it's more a lack of will or willingness to just check in the >> binaries. >> > > > > > > > >> > > > > > > > -ernie >> > > > > > > > >> > > > > > > > >> > > > > > > > > Gj >> > > > > > > > > >> > > > > > > > > On Mon, 11 Jan 2021 at 15:33, Ernie Rael < >> [email protected] >> > (mailto:[email protected])> wrote: >> > > > > > > > > >> > > > > > > > > > Hi all, >> > > > > > > > > > >> > > > > > > > > > There's been a fix for the 64 bit windows profiling >> > problem for over >> > > > > > > > > > two years. >> > > > > > > > > > >> > > > > > > > > > >> > https://github.com/pedro-w/netbeans/releases/tag/v0.1-alpha >> > > > > > > > > > https://issues.apache.org/jira/browse/NETBEANS-1428 >> > > > > > > > > > >> > > > > > > > > > AFAICT, it hasn't been integrated. There's some >> discussion >> > every few >> > > > > > > > > > months or so on how to get the binaries to be >> > automatically built as >> > > > > > > > > > part of the general build. An answer hasn't been found, >> so >> > the >> > > > > > > > > > problem remains. These binaries have been around and in >> > use for quite a while. >> > > > > > > > > > >> > > > > > > > > > This issue seems stuck in the /Perfectionism is the >> enemy >> > of >> > > > > > > > > > progress/ trap. >> > > > > > > > > > >> > > > > > > > > > Can't the binaries be checked in and included, while the >> > > > > > > > > > investigation for a solution proceeds? >> > > > > > > > > > >> > > > > > > > > > -ernie >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > --------------------------------------------------------------------- >> > > > > > > > > > To unsubscribe, e-mail: >> > [email protected] (mailto: >> > [email protected]) >> > > > > > > > > > For additional commands, e-mail: >> > [email protected] (mailto:[email protected]) >> > > > > > > > > > >> > > > > > > > > > For further information about the NetBeans mailing >> lists, >> > visit: >> > > > > > > > > > >> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> --- >> > -- > Eric Bresie > [email protected] > -- Eric Bresie [email protected]
