Re: Maven indexing on NB 18
On 12.06.23 23:28, Michael Bien wrote: On 12.06.23 20:51, Ernie Rael wrote: Using a newish large memory, multi-core laptop. The fan made itself known. Indexing took ~15 minutes. multi threading is off by default in NB 18 and has to be enabled in the settings. sorry, sometimes I forget what was merged when. MT is a NB 19 feature too, testable in nightly builds. -mbien there are two kinds of remote repo indexing tasks: - initial index creation when you start with a fresh NB config/cache. this takes about 300s on my system with MT enabled (dl not included) - incremental index update. I hope to get this < 70s for NB 19. This is a combination of an index extraction (of the delta index) and a merge. in NB 18 the initial index creation takes almost as much time as the update and can be easily confused, but this is a coincidence since those are two different problems. benchmarks above used master and had the two PRs applied: https://github.com/apache/maven-indexer/pull/318 https://github.com/apache/maven-indexer/pull/317 Some time in future, when most users used NB 19 for a while. We could import/move the more compact index of NB 19+ into future NB releases on first launch. This step is already prepared in NB 19 which will ship with lucene backwards compatibility codecs. Over a weeks worth of logs, only one indexing of central. There's a little "2nd thought" bouncing that seems to go on immediately after the indexing. should be harmless, I changed the log line from FINE to INFO for a few releases, will probably change it back in NB 20 or so. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.10 s. this is likely a update check and no update was found. the rest you posted doesn't seem to be related to remote repo indexing. -mbien - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven indexing on NB 18
On 12.06.23 20:51, Ernie Rael wrote: Using a newish large memory, multi-core laptop. The fan made itself known. Indexing took ~15 minutes. multi threading is off by default in NB 18 and has to be enabled in the settings. there are two kinds of remote repo indexing tasks: - initial index creation when you start with a fresh NB config/cache. this takes about 300s on my system with MT enabled (dl not included) - incremental index update. I hope to get this < 70s for NB 19. This is a combination of an index extraction (of the delta index) and a merge. in NB 18 the initial index creation takes almost as much time as the update and can be easily confused, but this is a coincidence since those are two different problems. benchmarks above used master and had the two PRs applied: https://github.com/apache/maven-indexer/pull/318 https://github.com/apache/maven-indexer/pull/317 Some time in future, when most users used NB 19 for a while. We could import/move the more compact index of NB 19+ into future NB releases on first launch. This step is already prepared in NB 19 which will ship with lucene backwards compatibility codecs. Over a weeks worth of logs, only one indexing of central. There's a little "2nd thought" bouncing that seems to go on immediately after the indexing. should be harmless, I changed the log line from FINE to INFO for a few releases, will probably change it back in NB 20 or so. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.10 s. this is likely a update check and no update was found. the rest you posted doesn't seem to be related to remote repo indexing. -mbien - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven indexing on NB 18
thanks for investigating, this sounds like a plausible explanation. NB 19 will disable remote repo indexing in out-of-space situations and won't try it again until enabled by the user (checkbox in maven indexing settings). index footprint is also almost cut in half and can be further decreased by enabling filters in the settings. Some users have also restricted tmp space, which is why extraction will go directly into the cache folder. this is all testable in dev builds, best regards, michael On 12.06.23 21:32, Peter Hull wrote: I found only three log files all from Friday but it showed I was running out of disk space, hence the indexing was never done. (My laptop has a small system drive and a bigger data drive, it’s a constant struggle to keep free space on the former). This might be the cause of my problem. On Mon, 12 Jun 2023 at 10:56, Michael Bien wrote: On 12.06.23 09:51, Peter Hull wrote: > On Sun, 11 Jun 2023 at 19:05, Tamás Cservenák wrote: >> The central index last publish happened on 5th June 2023. > Thanks everyone for the comments. > > I've not been monitoring it specifically but I had the feeling that > indexer was running more frequently, that's why I asked if there was a > log so I could confirm it (or not!) I forgot to answer this - sorry, check for: INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of [repo_name] took [time]s logger is enabled by default, time includes download. you can also see the last update timestamp in: [netbeans_config_folder]/18/config/Preferences/org/netbeans/modules/maven/repositories/[repo_name].properties which is set on every index update. > I believe it was indexing rather than just an update check, because it > makes my laptop fans spin up for a few minutes. > I have only recently upgraded to NB18 so for example if I had done > that on 4th June I would have had an indexer run on that day and again > the following day, is that correct? This shouldn't happen due to the min update interval which is 7 days by default. But I can take another look at the code. > And also, if I had closed NB > before the indexing had finished, would it start again from scratch, > or does it work incrementally? It would start again from scratch if it is cancelled by the user (e.g due to shutdown). > > I'll keep an eye on it and let you know if it seems to be running more > frequently than updates to the nexus-maven-repository-index.properties > would suggest. great, thanks! you could also give a NB 19 dev build a try if you want since it has many maven-indexer improvements already merged: https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/lastSuccessfulBuild/artifact/nbbuild/ best regards, -mbien > > Pete > > - > To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org > For additional commands, e-mail: dev-h...@netbeans.apache.org > > For further information about the NetBeans mailing lists, visit: > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists > > >
Re: Maven indexing on NB 18
Using a newish large memory, multi-core laptop. The fan made itself known. Indexing took ~15 minutes. Over a weeks worth of logs, only one indexing of central. There's a little "2nd thought" bouncing that seems to go on immediately after the indexing. An aside While taking a look at this I ran into something messy. Not repository, but source indexing relating. log filled with huge repeating message (4K each) https://github.com/apache/netbeans/issues/6071 WARNING: package com.apple.eio not in java.desktop --- >Log Session: Thursday, June 8, 2023 at 2:34:34 PM Pacific Daylight Time >System Info: Product Version = Apache NetBeans IDE 18 Operating System = Linux version 6.2.6-76060206-generic running on amd64 Java; VM; Vendor = 17.0.6; Java HotSpot(TM) 64-Bit Server VM 17.0.6+9-LTS-190; Oracle Corporation Runtime = Java(TM) SE Runtime Environment 17.0.6+9-LTS-190 ... INFO [org.apache.lucene.search.MultiTermQuery_patched]: patched MultiTermQuery.hashCode() initialized. (LUCENE-10431) INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of maven-snapshots took 0.99 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: could not (re-)index maven-snapshots org.apache.maven.wagon.ResourceDoesNotExistException: resource missing at https://oss.sonatype.org/content/repositories/snapshots/.index/nexus-maven-repository-index.properties, status: 404 Not Found at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1191) ... [catch] at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.lambda$spawnIndexLoadedRepo$4(NexusRepositoryIndexerImpl.java:668) ... one exception ... 18 lines of test application logging [maven-local-indexing] [1;34mINFO[m org.eclipse.aether.internal.impl.DefaultArtifactResolver - Artifact org.sonatype.nexus.buildsupport:nexus-buildsupport-shiro:pom:2.15.1-02 is present in the local repository, but cached from a remote repository ID that is unavailable in current build context, verifying that is downloadable from [maven-snapshots (https://oss.sonatype.org/content/repositories/snapshots/, default, releases+snapshots), central (https://repo.maven.apache.org/maven2/, default, releases+snapshots)] ... 231 messages like the one above, sigh INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of local took 18.73 s. ... 597 lines of seemingly unrelated random stuff INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 897.69 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.10 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.04 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.02 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.02 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.05 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central took 0.03 s. INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of central
Re: Contrib Repo
On Mon, 12 Jun 2023 at 11:26, Antonio wrote: > - CDDL binaries can indeed be used in Apache Projects. In fact we do use > many CDDL binaries in NetBeans. So can GPL+CPE binaries, and we currently ship with at least one. It currently requires permission on a case by case basis, to ensure all of the source is covered by the CPE, but it's doable. Although for those reasons, we would obviously pick CDDL where we can! :-) Best wishes, Neil - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Contrib Repo
Correction: Since the code is GPLv2+CPE or CDDL (at our discretion, you're expected to choose one): - It cannot be integrated in an Apache Repository (i.e. the source cannot be merged). - CDDL binaries can indeed be used in Apache Projects. In fact we do use many CDDL binaries in NetBeans. Apologies for the typo, Antonio On 12/6/23 11:56, Antonio wrote: Since the code is GPLv2+CPE or CDDL, it cannot be used in Apache projects (i.e., it cannot be integrated in an Apache repository), but can live as a NetBeans plugin(s) elsewhere. - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Will there be an 18-u1?
Hi, We have three PRs that have merged recently with Cherry Pick labels. https://github.com/apache/netbeans/issues?q=label%3A%22Cherry+Pick%22+is%3Aclosed+milestone%3ANB19 There are also one or two other potential fixes, (eg. maybe https://github.com/apache/netbeans/pull/6049 ), that come to mind. Do we need a NetBeans 18 update release? What would be in it? Is anyone interested in leading release management for an NB18 update? Anyone happy to handle cherry picking the PRs on to release180? There was mention of a VSCode update soon too? Best wishes, Neil - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven indexing on NB 18
On 12.06.23 09:51, Peter Hull wrote: On Sun, 11 Jun 2023 at 19:05, Tamás Cservenák wrote: The central index last publish happened on 5th June 2023. Thanks everyone for the comments. I've not been monitoring it specifically but I had the feeling that indexer was running more frequently, that's why I asked if there was a log so I could confirm it (or not!) I forgot to answer this - sorry, check for: INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Indexing of [repo_name] took [time]s logger is enabled by default, time includes download. you can also see the last update timestamp in: [netbeans_config_folder]/18/config/Preferences/org/netbeans/modules/maven/repositories/[repo_name].properties which is set on every index update. I believe it was indexing rather than just an update check, because it makes my laptop fans spin up for a few minutes. I have only recently upgraded to NB18 so for example if I had done that on 4th June I would have had an indexer run on that day and again the following day, is that correct? This shouldn't happen due to the min update interval which is 7 days by default. But I can take another look at the code. And also, if I had closed NB before the indexing had finished, would it start again from scratch, or does it work incrementally? It would start again from scratch if it is cancelled by the user (e.g due to shutdown). I'll keep an eye on it and let you know if it seems to be running more frequently than updates to the nexus-maven-repository-index.properties would suggest. great, thanks! you could also give a NB 19 dev build a try if you want since it has many maven-indexer improvements already merged: https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/lastSuccessfulBuild/artifact/nbbuild/ best regards, -mbien Pete - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Contrib Repo
Hi, Of course you may fork that source code either totally or partially and modify and enhance it as you see fit. That's what open source is all about. You may then create a NetBeans plugin and request for it to be published in the NetBeans portal, for instance. Things you should keep in mind: 1- You must respect the source code license (i.e., you cannot change the GPLv2+CPE/CDDL license to Apache License, for instance, unless the original authors, whoever they are, give you permission to do so). 2- You must keep the copyright notices (you should add yours to your modifications too). Since the code is GPLv2+CPE or CDDL, it cannot be used in Apache projects (i.e., it cannot be integrated in an Apache repository), but can live as a NetBeans plugin(s) elsewhere. HTH, Antonio On 5/6/23 21:29, Chris wrote: Do I need to create a new plugin for this and this will by my "fork" of this or how should we do this? Should all of them treated as 3rd party plugins? - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Contrib Repo
I there, That is actually a very good question. I was wondering to "resurrect" the Velocity and SFSBrowser plugins. On the code side, I am able to manage, but I still don't know how to proceed when it comes to commit the updated version as well as all the topics concerning IP from Oracle. It would be a good idea to have some information about that topic, since the contrib project is a mine of cool stuff that deserves to be reactivated by the community. Cheers, JM On Sun, Jun 11, 2023 at 2:13 PM Eric Bresie wrote: > I'm no expert on this but... > > I tried some time back to take the nbpython contribution to integrate it > into the Apache Netbeans work... The general input I got at the time was > that the nbPython contribution was to be included in coming donations from > Oracle which required some pre-donation checks to ensure no Oracle IP was > present. Some of this may have involved removal of some code (which might > require further updates). > > I believe the contribution paperwork signed when becoming a committer more > or less gave ownership/rights to Oracle so it was dependent on that. > However I believe in some cases the original contributors could have some > say in it if the given work was abandoned or no longer supported. > > There was a call out to see which of these contributions the community > wanted to have donated. With some being identified as candidates, others > being viewed as deprecated/not worth maintaining (i.e., requiring too much > work to bring it in line), Beyond that, I'm not sure any further donation > work against these is happening anymore. > > I think part of the work when doing so might involve > (1) updating licenses where applicable. Some of the contributions may have > existing licenses which did allow usages but still had to be in compliance > with Oracle licensing where applicable. > (2) managing the independent repo with contributions, > (3) delivering releasable artifacts (nbm, plugin portal, maven repo > publications, etc.) > (4) update for Netbeans compatibility (i.e. new NB versions builds with > applicable API compatibility, etc.) > > I believe part of this contribution fork also included some "mavenizing" of > some of the contributions. > > Once again...not an expert but that is my understanding of it. > > Eric Bresie > ebre...@gmail.com > > > On Mon, Jun 5, 2023 at 2:29 PM Chris wrote: > > > Hey, > > > > we have this wonderful repo here with all contribs since sun/oracle i > > think: https://github.com/timboudreau/netbeans-contrib so what is the > > correct procedure, If I want to extend or fix a package let's say, like > > this: > > > > > https://github.com/timboudreau/netbeans-contrib/tree/master/quickfilechooser > > > > Do I need to create a new plugin for this and this will by my "fork" of > > this or how should we do this? Should all of them treated as 3rd party > > plugins? > > > > > > Cheers > > > > Chris > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org > > For additional commands, e-mail: dev-h...@netbeans.apache.org > > > > For further information about the NetBeans mailing lists, visit: > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists > > > > > > > > >
Re: Maven indexing on NB 18
On Sun, 11 Jun 2023 at 19:05, Tamás Cservenák wrote: > The central index last publish happened on 5th June 2023. Thanks everyone for the comments. I've not been monitoring it specifically but I had the feeling that indexer was running more frequently, that's why I asked if there was a log so I could confirm it (or not!) I believe it was indexing rather than just an update check, because it makes my laptop fans spin up for a few minutes. I have only recently upgraded to NB18 so for example if I had done that on 4th June I would have had an indexer run on that day and again the following day, is that correct? And also, if I had closed NB before the indexing had finished, would it start again from scratch, or does it work incrementally? I'll keep an eye on it and let you know if it seems to be running more frequently than updates to the nexus-maven-repository-index.properties would suggest. Pete - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
