Hi all, yes, now I am certain that updating the other 9 places to set the time-zone of the SimpleDateFormat will help.
Here's a great article on exactly this topic: https://www.codeproject.com/Tips/1190426/When-Parsing-Formatting-Dates-in-Java-Make-Sure-Yo And now I'm hopefully really gonna call it a night. Chris Am 24.03.20, 21:50 schrieb "Carlos Rovira" <[email protected]>: Thanks for the huge effort Chris, I think it's being tough these days. Let's get rest and see if tomorrow we can get over step 7 :) Carlos El mar., 24 mar. 2020 a las 21:37, Christofer Dutz (< [email protected]>) escribió: > HI Alex, > > I think we're getting closer ... > > The maven reproducible build mechanism and the release plugin all work > with a standard time format: > > <project.build.outputTimestamp>2020-03-24T19:00:04Z</project.build.outputTimestamp></properties> > The format is: > yyyy-MM-dd'T'HH:mm:ss'Z' > So it actually doesn't contain time-zone information. > > Also did I do a little more searching in the compiler code and sort of > identical code was found in 9 places. > > I will try tomorrow if it helps setting the time-zone to UTC for every > place where the getSWFMetadataDate and getSWFMetadataDateFormat are used in > a SimpleDateFormat. I would assume if no timezone information is provided, > the SimpleDateFormat assumes it's the VMs default one. Now I could imagine > that if you call getTime this sort of is influenced by this. > > Now after a second full day of working on this I just want to go to bed ;-) > > Good night to you all, > > Chris > > > > Am 24.03.20, 20:57 schrieb "Alex Harui" <[email protected]>: > > Ok, I think I have a better idea of what code you are referring to. > > I can look into it with more detail later, but I recall thinking that > there could be a problem during the weeks the world is not synced on > "daylight savings time". > > Over on the US west coast, I was typing in dates like: "02/25/20 14:51 > -0800". The CI server is somewhere in the US, IIRC. And IIRC, I had to > use ""02/25/20 14:51 -0700" instead (or maybe the other way around) during > the period of time the US was changing from "PST" to "PDT" or back. > > So try adding or subtracting a zone from the date you are feeding into > Jenkins and see what happens. > > HTH, > -Alex > > > On 3/24/20, 12:34 PM, "Christofer Dutz" <[email protected]> > wrote: > > Hi Alex, > > Well it seems as if the "mod" attribute in the catalog.xml is > generated from the zipfile entry > xmlWriter.writeAttribute(ATTR_MOD, > String.valueOf(file.getLastModified())); > > And in the SWCWriter there's this code that cuts off the timezone, > you even wrote a comment on it > > if (metadataDate != null) > { > // TODO: Perhaps parsing without modification and then > serializing with a default timezone is the more solid approach. > // strip off timezone. Zip format doesn't store > timezone > // and the goal is to have the same date and time > regardless > // of which timezone the build machine is using. > int c = metadataDate.lastIndexOf(' '); > if(c != -1) { > metadataDate = metadataDate.substring(0, c); > } > c = metadataFormat.lastIndexOf(' '); > if(c != -1) { > metadataFormat = metadataFormat.substring(0, c); > } > try { > SimpleDateFormat sdf = new > SimpleDateFormat(metadataFormat); > sdf.setTimeZone(TimeZone.getTimeZone("UTC")); > fileDate = > sdf.parse(metadataDate).getTime(); > } catch (ParseException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } catch (IllegalArgumentException e1) { > e1.printStackTrace(); > } > } > > So in general it doesn’t matter what timezone is in the string > input, it was cut off anyway. > I added the line: > > sdf.setTimeZone(TimeZone.getTimeZone("UTC")); > > But it seems I still have the same issue with the con > > re-inspecting the code I noticed, it's probably just the timestamp > used in the zip file metadata ... not that used in the catalog.xml. > Will have another look at why I'm having this 36 offset. ... I > just did a little math and it's exactly one hour off ... > > The difference of all tags is: > 3600000 milliseconds, which is exactly one hour. So the timestamps > on the CI server are exactly one hour ahead of the builds we do in Europe. > > This is currently the last issue preventing us from finishing the > steps for the typedefs. > > Do you have any idea? ... Anything with the US being Summer-Time > and us still on Winter-Time? > > Perhaps you could try executing this command in the typedefs: > > ant -f releasesteps.xml Release_Step_007 -Drelease.version=0.9.7 > -DskipTests=true > > Would be interesting if this succeeds for you. > > Well actually the library.swf in the swc seems to have a "net" > Object in the locally built version and a "http" Object instead on the CI > Server-Version. > > Currently I'm trying to fix the timestamp version and address the > other problem after that's solved. > > Chris > > > > Am 24.03.20, 20:08 schrieb "Alex Harui" <[email protected] > >: > > Good to see progress. > > Re timestamps: I’m not quite sure what you are running into. > > The builds were taking a time that included a timezone. > Here's some output from recent builds: > > [java] -metadata.date=02/25/20 14:51 +0000 > [java] -metadata.dateFormat=MM/dd/yy HH:mm Z > > So I'm not sure how the timezone got lost on what you are > seeing. The RM was supposed to type in something like "02/25/20 14:51 > +0000". The prompts on the Jenkins job were supposed to hint that. > > HTH, > -Alex > > On 3/24/20, 7:47 AM, "Carlos Rovira" <[email protected]> > wrote: > > Hi Alex, > I finally said Chris to try it myself, so going with that > now :) > > El mar., 24 mar. 2020 a las 15:40, Christofer Dutz (< > [email protected]>) escribió: > > > Hi Alex, > > > > I added a call to set a fixed timezone to the > simpledateformat in the > > SWCWriter ... could you please review this? > > > > Chris > > > > > > > > Am 24.03.20, 15:26 schrieb "Christofer Dutz" < > [email protected]>: > > > > Hi all, > > > > so today we made progress ... not only did we fix > the version skipping > > we observed yesterday. For this the commands done > manually in the 001a step > > had to be changed. > > > > Now we managed to get the compiler part > reproducible, but not so much > > the typedef part. > > We are observing that the same timestamp is passed > in to the build > > when building the release on the ci server and when > building the release > > locally. However the timestamps have a strange constant > offset 36 in two > > digits of the timestamp in the catalog.xml > > > > 1585054125000 on the CI server > > 1585057725000 locally > > > > I am assuming that perhaps there is time-zone > information leaking into > > the parsing of the date as we are using a date-format > without time-zone > > information. > > > > I would assume that Alex, perhaps you could have a > look at what's > > happening in the SWCWriter constructor, where the > parsing of the timestamp > > is implemented. > > > > Thanks, > > > > Chris > > > > > > Am 24.03.20, 09:36 schrieb "Christofer Dutz" < > > [email protected]>: > > > > Hi Alex, > > > > I had another look ... I think the version skips > one is in step > > 001a ... > > There the job checks out develop, applies the > update to the new > > version of the build tools and jburg types, but then > this is pushed and > > then again to the release branch (hereby overwriting the > old > > release-branch) ... I'll try out a way with cherry > picking the change from > > develop into the release branch. Unfortunately I haven't > been able to find > > where the text in the emails comes from ... > > > > I'm talking about the order of the constants in > the class > > ProblemID. I forced that to be sorted now. That's why we > have to re-release > > the build tools. > > > > Chris > > > > Am 24.03.20, 08:14 schrieb "Alex Harui" > <[email protected] > > >: > > > > Re: credentials: Feel free to disable the > credential store. > > I did not have to both login and go through 2fa to Git > Push. There is > > some long password/token/credential/hash-like thing you > get when you sign > > up with Apache Git that seems to work when you use it as > a password. > > > > Re: the utils steps. I just looked and it > appears that > > compiler-jburg-types and compiler-build-tools are no > longer > > child/submodules of the root pom.xml in > royale-compiler. So now I get how > > why the development version won't get bumped up twice. > I think the way it > > was working may be that the updated dev version never > got pushed to > > develop. If you are satisfied with that setup, that's > fine with me. So > > then it might make more sense to get rid of 1a and > create a whole new set > > of steps for each of compiler-jburg-types and > compiler-build-tools. If I > > understand correctly, those new steps would run mvn > release:branch, mvn > > release:prepare and mvn release:perform in > compiler-jburg-types or > > compiler-build-tools, stopping if there are interim > pushes that need to be > > done, then start their own vote emails by copying other > CI steps. > > > > I use a Mac and thought I got all binaries > to match from the > > . I'm not sure what properties you are referring to. > > > > I'm done for tonight. Will catch up in my > morning. > > > > -Alex > > > > On 3/23/20, 11:15 PM, "Christofer Dutz" < > > [email protected]> wrote: > > > > Good morning all, > > > > well I didn't disable anythig as I > didn't want to break > > anything. So I thought before simply changing something > I'd ask first. > > So I would consider it a "bug" that it's > there and when we > > continue today, we'll try to find out and disable the > thing. > > However I would assume then the RM will > not only have to > > do his usual login but also the 2fa thing at every step > too, correct? > > > > And, just a kindly meant question ... > would it be possible > > to reply at the top instead of inline? I sort of have > missed several > > Of your Inline comments in the past as > at least in my mail > > client it's hard to spot what's mine and what's not. > > (I could start counting spaces) > > > > (I first missed this question of yours > ... but discovered > > it after counting spaces ;-) ) > > > > Well to release the build tool with > maven you would simply > > do the: > > mvn release:prepare > > mvn release:perform > > for each of them, then adjust the main > pom and do the mvn > > release:branch ... then prepare and perform after that. > > > > Currently also jburg types and the > build-tools are sort of > > released in parallel. With maven you would simply > release only the module > > that's needed to be released. In our case build-tools > would have been > > enough. > > > > Speaking of build-tools ... are you > possibly using Windows > > as OS on your system? Just asking because the > build-tools seem to have > > always generated a sorted set of properties on windows > and an unsorted one > > on Mac (Seems to be differences in the way the > filesystem works) ... in > > that case it would have been impossible to have a > release manager not using > > Java 8 on a Windows machine. (My changes recently never > touched this) > > > > Chris > > > > > > Am 23.03.20, 22:59 schrieb "Alex Harui" > > <[email protected]>: > > > > > > > > On 3/23/20, 2:29 PM, "Christofer > Dutz" < > > [email protected]> wrote: > > > > Hi Alex, > > > > sorry for the confusion ,... I > meant the 1a ... > > the one if you build the utils too. > > > > And regarding the credentials: > Something must have > > changes as I have never seen that gui thingy pop up > before and as soon as > > you login, it saves them in the password manager thingy > of windows. > > > > Did you try to disable the password > manager? > > > > It is true that I took those two > out of the main > > maven reactor, but that's actually a good practice. In > plc4x we setup a > > dedicated git repo for releasing these build tools. I > have never seen any > > good coming from the old setup (I know ... I did it, but > I learnt a lot). > > > > I am not opposed to moving those two > projects to a > > different repo. It would require that we have a truly > separate release > > vote for them, so more work in that regard, and the Ant > build would have to > > be adjusted to download those jars. I'm not sure now is > the time to make > > that change, but maybe. Let's see what others think. > > > > Regarding the banches ... we did > a git status > > after step 001a (the utils step) and indeed it was done > on develop. As the > > maven release isn't doing any selecting and switching of > branches it must > > have always been this way. If not the utils release > versions would only be > > updated in the release branch and not develop. I > couldn't find any cherry > > picking of commits in the process. > > > > Well in the end the steps are > way more complicated > > than I would be doing on my local machine. Especially > this tolling around > > checking out branches and tagging and pushing is pretty > complicated, in my > > opinion. Especially as most the stuff is fully automated. > > > > The Maven Release plugin > automatically pushes and > > signs, so yes, the CI steps are more complicated because > they have to > > continue on, but if there is a better way to stop and > continue, let us know > > what that is. But I was mainly addressing the > branch/version issue for > > those two "utils" projects. What would be the correct > set of Maven Release > > commands to manage that locally without updating the > develop version > > twice? Or maybe as you mention above, there really is > no good way and a > > separate repo is essentially the only way. > > > > I think we managed to get the > steps 1-4 (including > > the utils stuff) working again, while keeping the > cleanup I did a while > > ago. > > > > I hope tomorrow it'll be simper > to adjust the > > typedefs and the framework part. > > > > One thing we did notice however > that even after > > addressing the memory issues, the build sometimes simply > stuck and the > > build wouldn't continue. > > In these cases simply killing > the job (and the > > processes on the CI server) didn't really help much ... > we then had to > > reboot the machine in order to continue. I think we > rebooted the thing 5-6 > > times today. > > > > My experience was that the Jenkins > slave would > > occasionally disconnect and the job would fail right > away. Rebooting > > didn’t help. I didn't see it get stuck so unfortunately > you are seeing > > something new that I haven't seen. Did you search the > internet to see if > > anyone else reported something similar with Jenkins? > > > > Signing off for today, > > > > Chris > > > > > > Am 23.03.20, 21:38 schrieb "Alex > Harui" > > <[email protected]>: > > > > Re: credentials: Does that > mean that the git > > config is set to use the credential manager? If so, > maybe that should be > > turned off? When I did the release, I had to type my > password many times. > > Not sure what Piotr's experience was. > > > > Re: Branches: I don't > remember what the > > steps are. I didn't think there was a 1b, I don't see > it in my emails, > > just a 1a that did both compiler-jburg-types and > compiler-build-tools. > > Assuming 1a and 1b now reference these two projects, the > thing to keep in > > mind is that they are optional projects and so 1a (and > probably 1b) won't > > be run for every release. So, IMO, the branch should be > made for the main > > projects in royale-compiler. > > > > But if the changes you made > effectively change > > the way the poms are run (when there was a -utils > profile for those two > > projects, the main pom still got loaded and that may not > be true now), then > > the set of steps may need to test for existence of the > branch or something > > like that, not sure. > > > > In the end, the steps are > just what you would > > run on your local computer to fill the nexus staging > repo, but broken into > > discrete steps at each point Maven would normally push > or sign. If you > > haven’t, it might be worth just running Maven locally to > fill a staging > > repo with and without the jburg-types and build-tools > projects to see how > > to control when Maven makes branches and updates > versions. Then come back > > and break that down into steps on the CI server. > > > > HTH, > > -Alex > > > > On 3/23/20, 12:30 PM, > "Christofer Dutz" < > > [email protected]> wrote: > > > > Another thing we just > discovered. > > > > The current setup seems > to mess up the > > branches: > > - 001 creates the branch > and updates > > develop rel = 0.9.7-SNAPSHOT, develop = 0.9.8-SNAPSHOT > > - 001b updates DEVELOP > and not the release > > branch > > - 002 pushes develop to > the release-branch > > hereby bumping the release branch to the next version > rel = 0.9.8-SNAPSHOT, > > develop = 0.9.8-SNAPSHOT > > > > If the 001b changes > would have been done > > in the release branch, develop would still be using the > old version. > > > > So I would propose to > change the order of > > 001 and 001b to "release" the build tools before > branching. > > > > And I would propose to > fix 002 to work on > > the right branch. As I could see in maven central you > have been releasing > > only even version number so I assume this problem exists > for quite some > > time now. > > > > Chris > > > > Am 23.03.20, 20:12 > schrieb "Carlos Rovira" > > <[email protected]>: > > > > Hi, > > we just saw that > windows stores a > > personal access token in Windows > > Crendetials, so the > RM is responsible > > to remove it when finish all the > > operations. > > > > > > El lun., 23 mar. > 2020 a las 19:44, > > Alex Harui (<[email protected]>) > > escribió: > > > > > > > > > > > On 3/23/20, 11:32 > AM, "Christofer > > Dutz" <[email protected]> > > > wrote: > > > > > > Hi Alex, > > > > > > I did check > and I didn't > > directly find any .git .ssh or whatsoever > > > directories ... do > you have an Idea > > where that would be saved on windows? > > > The commits > are authorized by > > Carlos and it's his RDP connection, > > > that's why I'm > asking if there is > > any RDP magic going on. I didn't see him > > > entering anything > anywhere and he > > said he didn't do it before. > > > > > > I don't know for > sure, but here's a > > few links: > > > > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F46878457%2Fadding-git-credentials-on-windows&data=02%7C01%7Caharui%40adobe.com%7C8f87803163ef4c03a9c808d7d02a6da1%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206752966088690&sdata=oEh1Qw%2FT7ru8fAlZm7%2BBV9sR94hwN7Q2jTwXFdXYCLQ%3D&reserved=0 > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-scm.com%2Fbook%2Fen%2Fv2%2FGit-Tools-Credential-Storage&data=02%7C01%7Caharui%40adobe.com%7C8f87803163ef4c03a9c808d7d02a6da1%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206752966088690&sdata=6dUMmS2JOhrYmqIiDbxrV0HUaRpzxVip9v4VDQtOJfM%3D&reserved=0 > > > > > > I'm wondering if > Carlos can remember > > if he did.anything like that back > > > when he wrote this: > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread.html%2F8ae38cea0c736418b432b2353f967161c4f8448261a3bdce390e8c46%2540%253Cdev.royale.apache.org%253E&data=02%7C01%7Caharui%40adobe.com%7C8f87803163ef4c03a9c808d7d02a6da1%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206752966088690&sdata=VmN%2FQQbDVvR2LRzBxwlAnwCS7pWH3IFKDXj5Je9w25Q%3D&reserved=0 > > > > > > As I replied back > then, we shouldn't > > have GPG signing on the CI Server, > > > and hopefully no > other credentials > > got added either. > > > > > > HTH, > > > -Alex > > > > > > PS: I'm > purposefully not looking > > myself so as not to accidentally boot > > > someone off the > RDP connection. > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Carlos Rovira > > > > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C8f87803163ef4c03a9c808d7d02a6da1%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206752966098683&sdata=HJm%2FFSmKHKlD435iNQYouiMKu4wsv26j%2FCIP%2BjT6SIQ%3D&reserved=0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Carlos Rovira > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C8f87803163ef4c03a9c808d7d02a6da1%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206752966098683&sdata=HJm%2FFSmKHKlD435iNQYouiMKu4wsv26j%2FCIP%2BjT6SIQ%3D&reserved=0 > > > > > > > > > -- Carlos Rovira http://about.me/carlosrovira
