Re: [DISCUSS] Modello release
>correctly even if the encoding changes, because that will lead to the file
>being overwritten.
Once again: decoding does **not** guarantee you get **invalid** string when
decoding fails.
The replacement string might look like a regular string, and it might even
collide with the input string.
Here's an example:
1. Suppose someone writes out.txt file a Cyrillic letter ф in UTF-8
encoding.
jshell> "ф".getBytes("UTF-8")
$1 ==> byte[2] { -47, -124 }
2. Suppose there's an encoding UTF-Z that replaces unmappable bytes with ?
(ASCII "?")
Suppose UTF-Z treats all "negative bytes" as undecodable.
3. Suppose someone attempts to write "??" ( two ASCII question marks) using
UTF-Z encoding to the file out.txt
The expected file contents should be like 63, 63.
"CachingWriter" would try to decode file contents using UTF-Z, and it would
end up with "??" (two ASCII question marks because the file contains two
negative bytes).
4. The result of decoding would artificially match to the desired string
contents (remember, that at step 3 we wanted to store "??"),
so the caching writer would **skip** file overwrite even though the file
must be updated.
Note: the example does not work for UTF-8 because UTF-8 uses a special
"illegal" character for unmappable chars.
However, there's no guarantee every encoding would use illegal chars for
undecodable cases.
Vladimir
Re: [DISCUSS] Modello release
Le mar. 15 févr. 2022 à 08:19, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > >the result string will be corrupted when read > > The javadoc does not guarantee the string will be "corrupted" :-/ > > The "default replacement string" is quite a vague definition, > so there might be an encoding where "default replacement string" > means regular ASCII question mark "?". > Yes, so the string will definitely not be the same, so the current code is working correctly even if the encoding changes, because that will lead to the file being overwritten. > > All in all, String->byte is way easier to reason about. > Agreed. > > WDYT on having something like that in Maven core? > Sure, would you mean raising a JIRA issue on MSHARED ? https://issues.apache.org/jira/projects/MSHARED > > Vladimir > -- Guillaume Nodet
Re: [DISCUSS] Modello release
>the result string will be corrupted when read The javadoc does not guarantee the string will be "corrupted" :-/ The "default replacement string" is quite a vague definition, so there might be an encoding where "default replacement string" means regular ASCII question mark "?". All in all, String->byte is way easier to reason about. WDYT on having something like that in Maven core? Vladimir
Re: [DISCUSS] Modello release
I slightly disagree with your analysis. If the file has been written with a different encoding, the result string will be corrupted when read and unmappable chars replaced as indicated by [1], but then, the consequence is that the two strings will differ and the file will be overwritten anyway. So I don't think any failure can happen and the result should be the same, whether we do the comparison on the bytes or on the string. That said, I agree this will avoid an unnecessary decoding of the existing file and an optimization by checking the file length as you said. I've raised a PR for that [2]. Cheers, Guillaume Nodet [1] https://docs.oracle.com/javase/7/docs/api/java/lang/String.html#String(byte[],%20java.nio.charset.Charset) [2] https://github.com/codehaus-plexus/modello/pull/186 Le lun. 14 févr. 2022 à 08:41, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > I believe the added CachingWriter is might become a cause of silent > failures. > > What CachingWriter does is an attempt to read the file and decode it with > the provided encoding. > Apparently, the decoding might fail since the file might be written in > another encoding or it might be corrupted. > > A better approach would be to convert the created String into `byte[]`, and > then compare the bytes with file contents. > Then you never really need to decode the file > > Could someone with commit privileges just fix it? > The fix is just to remove readString and writeString methods, and use > Arrays.equals to compare the contents. > Then, you can compare file contents and avoid reading the file if you know > the length differs (it optimizes the "file differs" case). > > > > I remember I faced "Maven recompiling everything" issue multiple times, so > I wonder if Maven itself has a solution for that. > I understand tracking inputs and outputs might take a while to implement, > so > what if Maven had a solid API for generating files that skips overwriting > if the contents are the same? > > Sample issues on the top of my head: > https://issues.apache.org/jira/browse/MRRESOURCES-91 > https://github.com/freemarker/fmpp/issues/11 > https://github.com/julianhyde/hydromatic-resource/pull/4 > > Why does every plugin have to reinvent a half-broken caching wheel? > > Vladimir > -- Guillaume Nodet
Re: [DISCUSS] Modello release
I believe the added CachingWriter is might become a cause of silent failures. What CachingWriter does is an attempt to read the file and decode it with the provided encoding. Apparently, the decoding might fail since the file might be written in another encoding or it might be corrupted. A better approach would be to convert the created String into `byte[]`, and then compare the bytes with file contents. Then you never really need to decode the file Could someone with commit privileges just fix it? The fix is just to remove readString and writeString methods, and use Arrays.equals to compare the contents. Then, you can compare file contents and avoid reading the file if you know the length differs (it optimizes the "file differs" case). I remember I faced "Maven recompiling everything" issue multiple times, so I wonder if Maven itself has a solution for that. I understand tracking inputs and outputs might take a while to implement, so what if Maven had a solid API for generating files that skips overwriting if the contents are the same? Sample issues on the top of my head: https://issues.apache.org/jira/browse/MRRESOURCES-91 https://github.com/freemarker/fmpp/issues/11 https://github.com/julianhyde/hydromatic-resource/pull/4 Why does every plugin have to reinvent a half-broken caching wheel? Vladimir
Re: [DISCUSS] Modello release
great, the new report is really nice thank you Olivier Le lundi 14 février 2022, 03:38:00 CET Olivier Lamy a écrit : > normally there should be a draft one generated by release-drafter and you > only have to associate the tag and publish it. > Dependabot updates are under the label dependencies. > some PRs didn't have any label associated so I added some to have a correct > category. > anyway I have regenerated it > https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 > After the usual release process next time Hervé just go there > https://github.com/codehaus-plexus/modello/releases. > The top one is the draft for the current release. just edit the title or > not (feel free to add manual content), associate to a tag, publish it and > voila! > > On Mon, 14 Feb 2022 at 01:48, Hervé BOUTEMY wrote: > > I did it, but I'm not convinced by the result > > > > if someone knows how to do better, please help :) > > > > Regards, > > > > Hervé > > > > Le dimanche 13 février 2022, 15:54:08 CET Slawomir Jaranowski a écrit : > > > Hi, > > > > > > Hervé can you add release notes? > > > You should have a option "Auto-generate release notes." on tag > > > https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 > > > > > > > > > niedz., 13 lut 2022 o 15:42 Hervé BOUTEMY > > > > > > napisał(a): > > > > Modello 2.0.0 released > > > > > > > > https://codehaus-plexus.github.io/modello/ > > > > > > > > Regards, > > > > > > > > Hervé > > > > > > > > Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : > > > > > I was looking at why the maven builds are a bit slow and found out > > > > one > > > > > > > of > > > > > the culprit is modello which overwrites its generated files even if > > > > > there > > > > > are no changes: that cascades to recompiling the module, checking > > > > > the > > > > > > > > style > > > > > > > > > again, making a new archive and then recompiling the dependant > > > > modules > > > > > > > aso... > > > > > That was fixed a few months ago with > > > > > > > > > > https://github.com/codehaus-plexus/modello/pull/116 > > > > > > > > > > Would it be possible to release modello to be able to incorporate > > > > > the > > > > > fix > > > > > and continue the work on optimization the build ? > > > > > > > > > > I'm willing to help in any way if that can actually speed up > > > > things... > > > > > > > Cheers, > > > > > Guillaume > > > > > > > > - > > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Modello release
normally there should be a draft one generated by release-drafter and you only have to associate the tag and publish it. Dependabot updates are under the label dependencies. some PRs didn't have any label associated so I added some to have a correct category. anyway I have regenerated it https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 After the usual release process next time Hervé just go there https://github.com/codehaus-plexus/modello/releases. The top one is the draft for the current release. just edit the title or not (feel free to add manual content), associate to a tag, publish it and voila! On Mon, 14 Feb 2022 at 01:48, Hervé BOUTEMY wrote: > I did it, but I'm not convinced by the result > > if someone knows how to do better, please help :) > > Regards, > > Hervé > > Le dimanche 13 février 2022, 15:54:08 CET Slawomir Jaranowski a écrit : > > Hi, > > > > Hervé can you add release notes? > > You should have a option "Auto-generate release notes." on tag > > https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 > > > > > > niedz., 13 lut 2022 o 15:42 Hervé BOUTEMY > > > > napisał(a): > > > Modello 2.0.0 released > > > > > > https://codehaus-plexus.github.io/modello/ > > > > > > Regards, > > > > > > Hervé > > > > > > Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : > > > > I was looking at why the maven builds are a bit slow and found out > one > > > > of > > > > the culprit is modello which overwrites its generated files even if > > > > there > > > > are no changes: that cascades to recompiling the module, checking the > > > > > > style > > > > > > > again, making a new archive and then recompiling the dependant > modules > > > > aso... > > > > That was fixed a few months ago with > > > > > > > > https://github.com/codehaus-plexus/modello/pull/116 > > > > > > > > Would it be possible to release modello to be able to incorporate the > > > > fix > > > > and continue the work on optimization the build ? > > > > > > > > I'm willing to help in any way if that can actually speed up > things... > > > > > > > > Cheers, > > > > Guillaume > > > > > > - > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: [DISCUSS] Modello release
Frankly, it is very hard to filter human changes from the dependabot noise. Am 2022-02-13 um 16:48 schrieb Hervé BOUTEMY: I did it, but I'm not convinced by the result if someone knows how to do better, please help :) Regards, Hervé Le dimanche 13 février 2022, 15:54:08 CET Slawomir Jaranowski a écrit : Hi, Hervé can you add release notes? You should have a option "Auto-generate release notes." on tag https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 niedz., 13 lut 2022 o 15:42 Hervé BOUTEMY napisał(a): Modello 2.0.0 released https://codehaus-plexus.github.io/modello/ Regards, Hervé Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : I was looking at why the maven builds are a bit slow and found out one of the culprit is modello which overwrites its generated files even if there are no changes: that cascades to recompiling the module, checking the style again, making a new archive and then recompiling the dependant modules aso... That was fixed a few months ago with https://github.com/codehaus-plexus/modello/pull/116 Would it be possible to release modello to be able to incorporate the fix and continue the work on optimization the build ? I'm willing to help in any way if that can actually speed up things... Cheers, Guillaume - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Modello release
I did it, but I'm not convinced by the result if someone knows how to do better, please help :) Regards, Hervé Le dimanche 13 février 2022, 15:54:08 CET Slawomir Jaranowski a écrit : > Hi, > > Hervé can you add release notes? > You should have a option "Auto-generate release notes." on tag > https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 > > > niedz., 13 lut 2022 o 15:42 Hervé BOUTEMY > > napisał(a): > > Modello 2.0.0 released > > > > https://codehaus-plexus.github.io/modello/ > > > > Regards, > > > > Hervé > > > > Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : > > > I was looking at why the maven builds are a bit slow and found out one > > > of > > > the culprit is modello which overwrites its generated files even if > > > there > > > are no changes: that cascades to recompiling the module, checking the > > > > style > > > > > again, making a new archive and then recompiling the dependant modules > > > aso... > > > That was fixed a few months ago with > > > > > > https://github.com/codehaus-plexus/modello/pull/116 > > > > > > Would it be possible to release modello to be able to incorporate the > > > fix > > > and continue the work on optimization the build ? > > > > > > I'm willing to help in any way if that can actually speed up things... > > > > > > Cheers, > > > Guillaume > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Modello release
Hi, Hervé can you add release notes? You should have a option "Auto-generate release notes." on tag https://github.com/codehaus-plexus/modello/releases/tag/modello-2.0.0 niedz., 13 lut 2022 o 15:42 Hervé BOUTEMY napisał(a): > Modello 2.0.0 released > > https://codehaus-plexus.github.io/modello/ > > Regards, > > Hervé > > Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : > > I was looking at why the maven builds are a bit slow and found out one of > > the culprit is modello which overwrites its generated files even if there > > are no changes: that cascades to recompiling the module, checking the > style > > again, making a new archive and then recompiling the dependant modules > > aso... > > That was fixed a few months ago with > > https://github.com/codehaus-plexus/modello/pull/116 > > Would it be possible to release modello to be able to incorporate the fix > > and continue the work on optimization the build ? > > > > I'm willing to help in any way if that can actually speed up things... > > > > Cheers, > > Guillaume > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > -- Sławomir Jaranowski
Re: [DISCUSS] Modello release
Modello 2.0.0 released https://codehaus-plexus.github.io/modello/ Regards, Hervé Le lundi 31 janvier 2022, 08:30:08 CET Guillaume Nodet a écrit : > I was looking at why the maven builds are a bit slow and found out one of > the culprit is modello which overwrites its generated files even if there > are no changes: that cascades to recompiling the module, checking the style > again, making a new archive and then recompiling the dependant modules > aso... > That was fixed a few months ago with > https://github.com/codehaus-plexus/modello/pull/116 > Would it be possible to release modello to be able to incorporate the fix > and continue the work on optimization the build ? > > I'm willing to help in any way if that can actually speed up things... > > Cheers, > Guillaume - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Modello release
On 31. 01. 22 8:30, Guillaume Nodet wrote: > I was looking at why the maven builds are a bit slow and found out one of > the culprit is modello which overwrites its generated files even if there > are no changes: that cascades to recompiling the module, checking the style > again, making a new archive and then recompiling the dependant modules > aso... > That was fixed a few months ago with > https://github.com/codehaus-plexus/modello/pull/116 The linked PR has a comment that is not answered. Is the code correct with respect to encoding? > Would it be possible to release modello to be able to incorporate the fix > and continue the work on optimization the build ? > > I'm willing to help in any way if that can actually speed up things... -- VH - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Modello release
Fwiw, the following branch allows rebuilding the maven-core tree without any change at all, i.e. if you do a subsequent build, none of the output files or jars is rewritten. https://github.com/gnodet/maven/commits/stale-compiler There are still a few PR that needs to be merged and released in order to leverage that: * https://github.com/codehaus-plexus/modello/pull/116 * https://github.com/codehaus-plexus/plexus-containers/pull/46 * https://github.com/eclipse/sisu.mojos/pull/4 Subsequent builds using `mvn install -DskipTests` go from 29 seconds down to 16s (and 5s when using a hot mvnd) Le lun. 31 janv. 2022 à 08:30, Guillaume Nodet a écrit : > I was looking at why the maven builds are a bit slow and found out one of > the culprit is modello which overwrites its generated files even if there > are no changes: that cascades to recompiling the module, checking the style > again, making a new archive and then recompiling the dependant modules > aso... > That was fixed a few months ago with > https://github.com/codehaus-plexus/modello/pull/116 > Would it be possible to release modello to be able to incorporate the fix > and continue the work on optimization the build ? > > I'm willing to help in any way if that can actually speed up things... > > Cheers, > Guillaume > -- Guillaume Nodet
Re: [DISCUSS] Modello release
Awesome ! Thx Le mar. 1 févr. 2022 à 10:25, Olivier Lamy a écrit : > On Mon, 31 Jan 2022 at 19:08, Guillaume Nodet wrote: > > > Btw, the same thing applies to animal-sniffer with the inclusion of > > https://github.com/mojohaus/animal-sniffer/pull/165 > > > fyi animal-sniffer has just been released ;) > > > > > > > > Le lun. 31 janv. 2022 à 08:30, Guillaume Nodet a > > écrit : > > > > > I was looking at why the maven builds are a bit slow and found out one > of > > > the culprit is modello which overwrites its generated files even if > there > > > are no changes: that cascades to recompiling the module, checking the > > style > > > again, making a new archive and then recompiling the dependant modules > > > aso... > > > That was fixed a few months ago with > > > https://github.com/codehaus-plexus/modello/pull/116 > > > Would it be possible to release modello to be able to incorporate the > fix > > > and continue the work on optimization the build ? > > > > > > I'm willing to help in any way if that can actually speed up things... > > > > > > Cheers, > > > Guillaume > > > > > > > > > -- > > > > Guillaume Nodet > > > -- Guillaume Nodet
Re: [DISCUSS] Modello release
On Mon, 31 Jan 2022 at 19:08, Guillaume Nodet wrote: > Btw, the same thing applies to animal-sniffer with the inclusion of > https://github.com/mojohaus/animal-sniffer/pull/165 fyi animal-sniffer has just been released ;) > > > Le lun. 31 janv. 2022 à 08:30, Guillaume Nodet a > écrit : > > > I was looking at why the maven builds are a bit slow and found out one of > > the culprit is modello which overwrites its generated files even if there > > are no changes: that cascades to recompiling the module, checking the > style > > again, making a new archive and then recompiling the dependant modules > > aso... > > That was fixed a few months ago with > > https://github.com/codehaus-plexus/modello/pull/116 > > Would it be possible to release modello to be able to incorporate the fix > > and continue the work on optimization the build ? > > > > I'm willing to help in any way if that can actually speed up things... > > > > Cheers, > > Guillaume > > > > > -- > > Guillaume Nodet >
Re: [DISCUSS] Modello release
Howdy, As I see, the current master of modello is 2.0.0 and requires Java 8 I can lend a hand if needed, though I have never released any plexus stuff yet... (but am a committer, so maybe it is time for myself to set up my local env?) T On Mon, Jan 31, 2022 at 8:30 AM Guillaume Nodet wrote: > I was looking at why the maven builds are a bit slow and found out one of > the culprit is modello which overwrites its generated files even if there > are no changes: that cascades to recompiling the module, checking the style > again, making a new archive and then recompiling the dependant modules > aso... > That was fixed a few months ago with > https://github.com/codehaus-plexus/modello/pull/116 > Would it be possible to release modello to be able to incorporate the fix > and continue the work on optimization the build ? > > I'm willing to help in any way if that can actually speed up things... > > Cheers, > Guillaume >
Re: [DISCUSS] Modello release
Btw, the same thing applies to animal-sniffer with the inclusion of https://github.com/mojohaus/animal-sniffer/pull/165 Le lun. 31 janv. 2022 à 08:30, Guillaume Nodet a écrit : > I was looking at why the maven builds are a bit slow and found out one of > the culprit is modello which overwrites its generated files even if there > are no changes: that cascades to recompiling the module, checking the style > again, making a new archive and then recompiling the dependant modules > aso... > That was fixed a few months ago with > https://github.com/codehaus-plexus/modello/pull/116 > Would it be possible to release modello to be able to incorporate the fix > and continue the work on optimization the build ? > > I'm willing to help in any way if that can actually speed up things... > > Cheers, > Guillaume > -- Guillaume Nodet
[DISCUSS] Modello release
I was looking at why the maven builds are a bit slow and found out one of the culprit is modello which overwrites its generated files even if there are no changes: that cascades to recompiling the module, checking the style again, making a new archive and then recompiling the dependant modules aso... That was fixed a few months ago with https://github.com/codehaus-plexus/modello/pull/116 Would it be possible to release modello to be able to incorporate the fix and continue the work on optimization the build ? I'm willing to help in any way if that can actually speed up things... Cheers, Guillaume
