Re: Release Ant 1.10.13?
Thanks for the update. We have some workarounds in the Groovy codebase too. I'll try to tidy them up too once this has settled. Thanks again, Paul. On Mon, May 15, 2023 at 10:08 PM Jaikiran Pai wrote: > Hello Paul, > > On 12/12/22 5:30 am, Paul King wrote: > > Do you know if there is an issue with the "allow" class approach if > > multiple projects adopt that technique? E.g. if Netbeans or Groovy > > also have an allow class, will that cause a split package violation or > > since it isn't really referenced except for those early JDKs, that we > > should be okay? I will eventually try this out myself if searching > > doesn't help, but just wondering if someone has already checked this. > > The use of a "allow" class as a workaround to older versions of JDK > considering this value as a classname for -Djava.security.manager system > property, was always a brittle one. As such, Oracle JDK in its upcoming > October CPU release is going to introduce a change which will treat the > values "allow" and "disallow" specially (by ignoring them and not > considering them as a classname) for the java.security.manager system > property. This will be available in Oracle's 11, 8 and 7 releases and is > being tracked in https://bugs.openjdk.org/browse/JDK-8301118. Hopefully > other vendors too will bring in this change in their releases. > > What that will then mean is that, applications/users will no longer have > to first detect the version of Java before deciding whether or not they > can set the value "allow" for the java.security.manager system property > (if at all they want to set that value). > > As a related note, after Ant 1.10.13 was released we have received > reports that the "allow" workaround we introduced, has its own set of > issues. It was always a temporary change in Ant to allow for this > version of Ant to work against recent releases of Java. I'm in the > process of undoing this "allow" workaround and then completing skipping > setting of SecurityManager against recent versions of Java, in Ant. > > -Jaikiran > > >
Re: Release Ant 1.10.13?
Hello Paul, On 12/12/22 5:30 am, Paul King wrote: Do you know if there is an issue with the "allow" class approach if multiple projects adopt that technique? E.g. if Netbeans or Groovy also have an allow class, will that cause a split package violation or since it isn't really referenced except for those early JDKs, that we should be okay? I will eventually try this out myself if searching doesn't help, but just wondering if someone has already checked this. The use of a "allow" class as a workaround to older versions of JDK considering this value as a classname for -Djava.security.manager system property, was always a brittle one. As such, Oracle JDK in its upcoming October CPU release is going to introduce a change which will treat the values "allow" and "disallow" specially (by ignoring them and not considering them as a classname) for the java.security.manager system property. This will be available in Oracle's 11, 8 and 7 releases and is being tracked in https://bugs.openjdk.org/browse/JDK-8301118. Hopefully other vendors too will bring in this change in their releases. What that will then mean is that, applications/users will no longer have to first detect the version of Java before deciding whether or not they can set the value "allow" for the java.security.manager system property (if at all they want to set that value). As a related note, after Ant 1.10.13 was released we have received reports that the "allow" workaround we introduced, has its own set of issues. It was always a temporary change in Ant to allow for this version of Ant to work against recent releases of Java. I'm in the process of undoing this "allow" workaround and then completing skipping setting of SecurityManager against recent versions of Java, in Ant. -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
On 2022-12-11, Jaikiran Pai wrote: > On 26/11/22 11:33 pm, Stefan Bodewig wrote: >> On 2022-11-19, Stefan Bodewig wrote: >> Finding A solution was a nice puzzle, but that doesn't mean we have to >> use it. > I had a look at that commit and the PR discussion where the initial > fix was made. I don't have enough knowledge of this area to provide > relevant inputs, but from what you have explained here and in the PR > and looking at the code it looks fine to me. You (and the original > contributor) note that there are still issues that this change won't > solve, but I think that is OK for now. I say that because the current > state/fix addresses an actual issue that was reported[1]. Yes, I'm fine with keeping the code the way it is in master right now. Thanks Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
Do you know if there is an issue with the "allow" class approach if multiple projects adopt that technique? E.g. if Netbeans or Groovy also have an allow class, will that cause a split package violation or since it isn't really referenced except for those early JDKs, that we should be okay? I will eventually try this out myself if searching doesn't help, but just wondering if someone has already checked this. On Sun, Dec 11, 2022 at 11:38 PM Jaikiran Pai wrote: > > Hello Stefan, > > On 18/11/22 2:40 pm, Stefan Bodewig wrote: > > On 2022-11-16, Jaikiran Pai wrote: > > > >> Users can still use the current released Ant version against these > >> recent Java versions, but they additionally have to set a system > >> property while launching Ant to allow setting the security > >> manager. Ant's mainline code has changes where it does the necessary > >> work (to the extent possible) to set this property internally without > >> forcing the users to do that. So releasing this version of Ant should > >> help projects building against these recent versions. > > I guess you've seen Earl Hood's response, I haven't looked into it > > myself, yet. > > Yes, that response helped me decide which way to go. Before that, I was > still inclined to try and get this working by doing necessary changes to > the launcher scripts - but that was going nowhere and it introduced the > unnecessary dual launch of Java (once to identify the version and then > the real launch). ewh's response gave me the confidence that using > "allow" as a class (on Java versions where it isn't recognized as a > predefined value) would be the better of the 2 approaches. > > I found some time this weekend to get this implemented in Ant and have > now committed > https://github.com/apache/ant/commit/82c70f3202d5aec4d99fa3b6314ba4a6c338cd94. > Tests against JDK 8, 11, 17, 19 and latest 20 EA all came back fine > against Linux and Windows. > > -Jaikiran > > > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
Hello Stefan, On 26/11/22 11:33 pm, Stefan Bodewig wrote: On 2022-11-19, Stefan Bodewig wrote: while running all tests locally I realized https://github.com/apache/ant/pull/194 introduces a bug when tars have an encoding set. You can see this with UntarTest failing. A multibyte encoding like UTF16 may contain NUL bytes inside of the "normal" part of the name. I'll have to think about a way to solve this, but we should not release Ant with this regression. https://github.com/apache/ant/commit/e04fbe7ffa4f549bdc00bdfce688fb587120eedd fixes tthe problem, at least for our test. It makes parsing tar archives a bit slower, but that likely won't matter much for single-byte encodings (tar isn't meant to be used with multi-byte encodings). Also I don't think reading tars is what a normal build does for the majority of its time. Anyway, I'd appreciate a review of the code I've written there. What I wanted to do is to ask the encoding how it would represent a NUL and look search for that when finding the string terminator - as opposed to looking for a single NUL byte. Our testcase used UTF-16 BE with a BOM, so encoding "\0" ends up with two bytes BOM + 0x00 0x00 - while I really only wanted to 0x00 0x00. Next attempt was to encode an empty string to see whether there is a common prefix, but an empty string is encoded as 0 bytes, no BOM. :-) So finally I compared "X" to "X\0" and stripped what seems to be "X". I'm pretty sure this breaks for certain encodings, but not worse than it has worked before. And then I sprinkled some memoization on top. TBM I could also live with reverting the whole commit, saying "don't use multi-byte encodings in tars" and be done. The original test we had worked by accident, if the old test had used UTF16-LE instead and a 49 character file name it would have failed as we'd try to decode a byte array with an odd number of bytes. Finding A solution was a nice puzzle, but that doesn't mean we have to use it. I had a look at that commit and the PR discussion where the initial fix was made. I don't have enough knowledge of this area to provide relevant inputs, but from what you have explained here and in the PR and looking at the code it looks fine to me. You (and the original contributor) note that there are still issues that this change won't solve, but I think that is OK for now. I say that because the current state/fix addresses an actual issue that was reported[1]. I've verified that the current code in master with your changes does indeed allow extracting that .tar.gz fine (unlike Ant 1.10.12). Plus our testsuite continues to pass. So that's a good thing. [1] https://github.com/ibmruntimes/Semeru-Runtimes/issues/15 -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
Hello Stefan, On 18/11/22 2:40 pm, Stefan Bodewig wrote: On 2022-11-16, Jaikiran Pai wrote: Users can still use the current released Ant version against these recent Java versions, but they additionally have to set a system property while launching Ant to allow setting the security manager. Ant's mainline code has changes where it does the necessary work (to the extent possible) to set this property internally without forcing the users to do that. So releasing this version of Ant should help projects building against these recent versions. I guess you've seen Earl Hood's response, I haven't looked into it myself, yet. Yes, that response helped me decide which way to go. Before that, I was still inclined to try and get this working by doing necessary changes to the launcher scripts - but that was going nowhere and it introduced the unnecessary dual launch of Java (once to identify the version and then the real launch). ewh's response gave me the confidence that using "allow" as a class (on Java versions where it isn't recognized as a predefined value) would be the better of the 2 approaches. I found some time this weekend to get this implemented in Ant and have now committed https://github.com/apache/ant/commit/82c70f3202d5aec4d99fa3b6314ba4a6c338cd94. Tests against JDK 8, 11, 17, 19 and latest 20 EA all came back fine against Linux and Windows. -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
On 2022-11-19, Stefan Bodewig wrote: > while running all tests locally I realized > https://github.com/apache/ant/pull/194 introduces a bug when tars have > an encoding set. You can see this with UntarTest failing. A multibyte > encoding like UTF16 may contain NUL bytes inside of the "normal" part of > the name. I'll have to think about a way to solve this, but we should > not release Ant with this regression. https://github.com/apache/ant/commit/e04fbe7ffa4f549bdc00bdfce688fb587120eedd fixes tthe problem, at least for our test. It makes parsing tar archives a bit slower, but that likely won't matter much for single-byte encodings (tar isn't meant to be used with multi-byte encodings). Also I don't think reading tars is what a normal build does for the majority of its time. Anyway, I'd appreciate a review of the code I've written there. What I wanted to do is to ask the encoding how it would represent a NUL and look search for that when finding the string terminator - as opposed to looking for a single NUL byte. Our testcase used UTF-16 BE with a BOM, so encoding "\0" ends up with two bytes BOM + 0x00 0x00 - while I really only wanted to 0x00 0x00. Next attempt was to encode an empty string to see whether there is a common prefix, but an empty string is encoded as 0 bytes, no BOM. :-) So finally I compared "X" to "X\0" and stripped what seems to be "X". I'm pretty sure this breaks for certain encodings, but not worse than it has worked before. And then I sprinkled some memoization on top. TBM I could also live with reverting the whole commit, saying "don't use multi-byte encodings in tars" and be done. The original test we had worked by accident, if the old test had used UTF16-LE instead and a 49 character file name it would have failed as we'd try to decode a byte array with an odd number of bytes. Finding A solution was a nice puzzle, but that doesn't mean we have to use it. Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
Hello Stefan, On 19/11/22 10:41 pm, Stefan Bodewig wrote: On 2022-11-18, Stefan Bodewig wrote: On my personal TODO list there are a few minor things like Java 20 removing another target option that we may want to tell users about. Nothing of this is critical AFAIR and shouldn't hold up a release if I don't get to them soon enough. I'm through with this BUT while running all tests locally I realized https://github.com/apache/ant/pull/194 introduces a bug when tars have an encoding set. You can see this with UntarTest failing. A multibyte encoding like UTF16 may contain NUL bytes inside of the "normal" part of the name. I'll have to think about a way to solve this, but we should not release Ant with this regression. Please take your time, I won't rush the release. I'm still experimenting with the security manager changes. -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
On 2022-11-18, Stefan Bodewig wrote: > On my personal TODO list there are a few minor things like Java 20 > removing another target option that we may want to tell users > about. Nothing of this is critical AFAIR and shouldn't hold up a release > if I don't get to them soon enough. I'm through with this BUT while running all tests locally I realized https://github.com/apache/ant/pull/194 introduces a bug when tars have an encoding set. You can see this with UntarTest failing. A multibyte encoding like UTF16 may contain NUL bytes inside of the "normal" part of the name. I'll have to think about a way to solve this, but we should not release Ant with this regression. Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Release Ant 1.10.13?
On 2022-11-16, Jaikiran Pai wrote: > Users can still use the current released Ant version against these > recent Java versions, but they additionally have to set a system > property while launching Ant to allow setting the security > manager. Ant's mainline code has changes where it does the necessary > work (to the extent possible) to set this property internally without > forcing the users to do that. So releasing this version of Ant should > help projects building against these recent versions. I guess you've seen Earl Hood's response, I haven't looked into it myself, yet. On my personal TODO list there are a few minor things like Java 20 removing another target option that we may want to tell users about. Nothing of this is critical AFAIR and shouldn't hold up a release if I don't get to them soon enough. +1 on releasing a new version soonish. Stefan - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Release Ant 1.10.13?
Hello everyone, We last released Ant more than a year back. During that period we have had some good amount of bug fixes and changes that have gone in. Plus, Java 18 and beyond was released in the meantime where setting the security manager isn't allowed by default (Ant sets it). Users can still use the current released Ant version against these recent Java versions, but they additionally have to set a system property while launching Ant to allow setting the security manager. Ant's mainline code has changes where it does the necessary work (to the extent possible) to set this property internally without forcing the users to do that. So releasing this version of Ant should help projects building against these recent versions. The way we detect Java versions within Ant's startup script isn't straightforward, especially for Windows. We had a brief discussion[1] about it in the past and I'll be focusing on this area to see if I can improve that situation in this coming week before the release. [1] https://lists.apache.org/thread/53jo1sy8ly2645j68m7s2g2mb4f10dy5 -Jaikiran - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
