I'm not quite sure whether the problem you want to solve is that the timestamp of ZipFileEntru is not matching or the timestamps in the catalog.xml are not matching. I could be wrong, but I believe ZipFileEntry problem is different from what is described in the linked article. IIRC, the issue is that the Zip format does not store timezone and the problem to solve is to pick a timezone such that the month/day/year/hour/minute/second computes to be the same on the machines building the zip files. And due to differences in when clocks are shifted forward in the spring, a timezone like -0800 on the CI server may need to be -0700 or -0900 on some computers elsewhere in the world.
Of course, I could be wrong. HTH, -Alex On 3/24/20, 2:12 PM, "Christofer Dutz" <[email protected]> wrote: Hi all, yes, now I am certain that updating the other 9 places to set the time-zone of the SimpleDateFormat will help. Here's a great article on exactly this topic: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.codeproject.com%2FTips%2F1190426%2FWhen-Parsing-Formatting-Dates-in-Java-Make-Sure-Yo&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395069878&sdata=lVb8SNiL%2FFJXj%2F8IgDgUw1pj8FAeQ2cRR3B%2FMQD1iwE%3D&reserved=0 And now I'm hopefully really gonna call it a night. Chris Am 24.03.20, 21:50 schrieb "Carlos Rovira" <[email protected]>: Thanks for the huge effort Chris, I think it's being tough these days. Let's get rest and see if tomorrow we can get over step 7 :) Carlos El mar., 24 mar. 2020 a las 21:37, Christofer Dutz (< [email protected]>) escribió: > HI Alex, > > I think we're getting closer ... > > The maven reproducible build mechanism and the release plugin all work > with a standard time format: > > <project.build.outputTimestamp>2020-03-24T19:00:04Z</project.build.outputTimestamp></properties> > The format is: > yyyy-MM-dd'T'HH:mm:ss'Z' > So it actually doesn't contain time-zone information. > > Also did I do a little more searching in the compiler code and sort of > identical code was found in 9 places. > > I will try tomorrow if it helps setting the time-zone to UTC for every > place where the getSWFMetadataDate and getSWFMetadataDateFormat are used in > a SimpleDateFormat. I would assume if no timezone information is provided, > the SimpleDateFormat assumes it's the VMs default one. Now I could imagine > that if you call getTime this sort of is influenced by this. > > Now after a second full day of working on this I just want to go to bed ;-) > > Good night to you all, > > Chris > > > > Am 24.03.20, 20:57 schrieb "Alex Harui" <[email protected]>: > > Ok, I think I have a better idea of what code you are referring to. > > I can look into it with more detail later, but I recall thinking that > there could be a problem during the weeks the world is not synced on > "daylight savings time". > > Over on the US west coast, I was typing in dates like: "02/25/20 14:51 > -0800". The CI server is somewhere in the US, IIRC. And IIRC, I had to > use ""02/25/20 14:51 -0700" instead (or maybe the other way around) during > the period of time the US was changing from "PST" to "PDT" or back. > > So try adding or subtracting a zone from the date you are feeding into > Jenkins and see what happens. > > HTH, > -Alex > > > On 3/24/20, 12:34 PM, "Christofer Dutz" <[email protected]> > wrote: > > Hi Alex, > > Well it seems as if the "mod" attribute in the catalog.xml is > generated from the zipfile entry > xmlWriter.writeAttribute(ATTR_MOD, > String.valueOf(file.getLastModified())); > > And in the SWCWriter there's this code that cuts off the timezone, > you even wrote a comment on it > > if (metadataDate != null) > { > // TODO: Perhaps parsing without modification and then > serializing with a default timezone is the more solid approach. > // strip off timezone. Zip format doesn't store > timezone > // and the goal is to have the same date and time > regardless > // of which timezone the build machine is using. > int c = metadataDate.lastIndexOf(' '); > if(c != -1) { > metadataDate = metadataDate.substring(0, c); > } > c = metadataFormat.lastIndexOf(' '); > if(c != -1) { > metadataFormat = metadataFormat.substring(0, c); > } > try { > SimpleDateFormat sdf = new > SimpleDateFormat(metadataFormat); > sdf.setTimeZone(TimeZone.getTimeZone("UTC")); > fileDate = > sdf.parse(metadataDate).getTime(); > } catch (ParseException e) { > // TODO Auto-generated catch block > e.printStackTrace(); > } catch (IllegalArgumentException e1) { > e1.printStackTrace(); > } > } > > So in general it doesn’t matter what timezone is in the string > input, it was cut off anyway. > I added the line: > > sdf.setTimeZone(TimeZone.getTimeZone("UTC")); > > But it seems I still have the same issue with the con > > re-inspecting the code I noticed, it's probably just the timestamp > used in the zip file metadata ... not that used in the catalog.xml. > Will have another look at why I'm having this 36 offset. ... I > just did a little math and it's exactly one hour off ... > > The difference of all tags is: > 3600000 milliseconds, which is exactly one hour. So the timestamps > on the CI server are exactly one hour ahead of the builds we do in Europe. > > This is currently the last issue preventing us from finishing the > steps for the typedefs. > > Do you have any idea? ... Anything with the US being Summer-Time > and us still on Winter-Time? > > Perhaps you could try executing this command in the typedefs: > > ant -f releasesteps.xml Release_Step_007 -Drelease.version=0.9.7 > -DskipTests=true > > Would be interesting if this succeeds for you. > > Well actually the library.swf in the swc seems to have a "net" > Object in the locally built version and a "http" Object instead on the CI > Server-Version. > > Currently I'm trying to fix the timestamp version and address the > other problem after that's solved. > > Chris > > > > Am 24.03.20, 20:08 schrieb "Alex Harui" <[email protected] > >: > > Good to see progress. > > Re timestamps: I’m not quite sure what you are running into. > > The builds were taking a time that included a timezone. > Here's some output from recent builds: > > [java] -metadata.date=02/25/20 14:51 +0000 > [java] -metadata.dateFormat=MM/dd/yy HH:mm Z > > So I'm not sure how the timezone got lost on what you are > seeing. The RM was supposed to type in something like "02/25/20 14:51 > +0000". The prompts on the Jenkins job were supposed to hint that. > > HTH, > -Alex > > On 3/24/20, 7:47 AM, "Carlos Rovira" <[email protected]> > wrote: > > Hi Alex, > I finally said Chris to try it myself, so going with that > now :) > > El mar., 24 mar. 2020 a las 15:40, Christofer Dutz (< > [email protected]>) escribió: > > > Hi Alex, > > > > I added a call to set a fixed timezone to the > simpledateformat in the > > SWCWriter ... could you please review this? > > > > Chris > > > > > > > > Am 24.03.20, 15:26 schrieb "Christofer Dutz" < > [email protected]>: > > > > Hi all, > > > > so today we made progress ... not only did we fix > the version skipping > > we observed yesterday. For this the commands done > manually in the 001a step > > had to be changed. > > > > Now we managed to get the compiler part > reproducible, but not so much > > the typedef part. > > We are observing that the same timestamp is passed > in to the build > > when building the release on the ci server and when > building the release > > locally. However the timestamps have a strange constant > offset 36 in two > > digits of the timestamp in the catalog.xml > > > > 1585054125000 on the CI server > > 1585057725000 locally > > > > I am assuming that perhaps there is time-zone > information leaking into > > the parsing of the date as we are using a date-format > without time-zone > > information. > > > > I would assume that Alex, perhaps you could have a > look at what's > > happening in the SWCWriter constructor, where the > parsing of the timestamp > > is implemented. > > > > Thanks, > > > > Chris > > > > > > Am 24.03.20, 09:36 schrieb "Christofer Dutz" < > > [email protected]>: > > > > Hi Alex, > > > > I had another look ... I think the version skips > one is in step > > 001a ... > > There the job checks out develop, applies the > update to the new > > version of the build tools and jburg types, but then > this is pushed and > > then again to the release branch (hereby overwriting the > old > > release-branch) ... I'll try out a way with cherry > picking the change from > > develop into the release branch. Unfortunately I haven't > been able to find > > where the text in the emails comes from ... > > > > I'm talking about the order of the constants in > the class > > ProblemID. I forced that to be sorted now. That's why we > have to re-release > > the build tools. > > > > Chris > > > > Am 24.03.20, 08:14 schrieb "Alex Harui" > <[email protected] > > >: > > > > Re: credentials: Feel free to disable the > credential store. > > I did not have to both login and go through 2fa to Git > Push. There is > > some long password/token/credential/hash-like thing you > get when you sign > > up with Apache Git that seems to work when you use it as > a password. > > > > Re: the utils steps. I just looked and it > appears that > > compiler-jburg-types and compiler-build-tools are no > longer > > child/submodules of the root pom.xml in > royale-compiler. So now I get how > > why the development version won't get bumped up twice. > I think the way it > > was working may be that the updated dev version never > got pushed to > > develop. If you are satisfied with that setup, that's > fine with me. So > > then it might make more sense to get rid of 1a and > create a whole new set > > of steps for each of compiler-jburg-types and > compiler-build-tools. If I > > understand correctly, those new steps would run mvn > release:branch, mvn > > release:prepare and mvn release:perform in > compiler-jburg-types or > > compiler-build-tools, stopping if there are interim > pushes that need to be > > done, then start their own vote emails by copying other > CI steps. > > > > I use a Mac and thought I got all binaries > to match from the > > . I'm not sure what properties you are referring to. > > > > I'm done for tonight. Will catch up in my > morning. > > > > -Alex > > > > On 3/23/20, 11:15 PM, "Christofer Dutz" < > > [email protected]> wrote: > > > > Good morning all, > > > > well I didn't disable anythig as I > didn't want to break > > anything. So I thought before simply changing something > I'd ask first. > > So I would consider it a "bug" that it's > there and when we > > continue today, we'll try to find out and disable the > thing. > > However I would assume then the RM will > not only have to > > do his usual login but also the 2fa thing at every step > too, correct? > > > > And, just a kindly meant question ... > would it be possible > > to reply at the top instead of inline? I sort of have > missed several > > Of your Inline comments in the past as > at least in my mail > > client it's hard to spot what's mine and what's not. > > (I could start counting spaces) > > > > (I first missed this question of yours > ... but discovered > > it after counting spaces ;-) ) > > > > Well to release the build tool with > maven you would simply > > do the: > > mvn release:prepare > > mvn release:perform > > for each of them, then adjust the main > pom and do the mvn > > release:branch ... then prepare and perform after that. > > > > Currently also jburg types and the > build-tools are sort of > > released in parallel. With maven you would simply > release only the module > > that's needed to be released. In our case build-tools > would have been > > enough. > > > > Speaking of build-tools ... are you > possibly using Windows > > as OS on your system? Just asking because the > build-tools seem to have > > always generated a sorted set of properties on windows > and an unsorted one > > on Mac (Seems to be differences in the way the > filesystem works) ... in > > that case it would have been impossible to have a > release manager not using > > Java 8 on a Windows machine. (My changes recently never > touched this) > > > > Chris > > > > > > Am 23.03.20, 22:59 schrieb "Alex Harui" > > <[email protected]>: > > > > > > > > On 3/23/20, 2:29 PM, "Christofer > Dutz" < > > [email protected]> wrote: > > > > Hi Alex, > > > > sorry for the confusion ,... I > meant the 1a ... > > the one if you build the utils too. > > > > And regarding the credentials: > Something must have > > changes as I have never seen that gui thingy pop up > before and as soon as > > you login, it saves them in the password manager thingy > of windows. > > > > Did you try to disable the password > manager? > > > > It is true that I took those two > out of the main > > maven reactor, but that's actually a good practice. In > plc4x we setup a > > dedicated git repo for releasing these build tools. I > have never seen any > > good coming from the old setup (I know ... I did it, but > I learnt a lot). > > > > I am not opposed to moving those two > projects to a > > different repo. It would require that we have a truly > separate release > > vote for them, so more work in that regard, and the Ant > build would have to > > be adjusted to download those jars. I'm not sure now is > the time to make > > that change, but maybe. Let's see what others think. > > > > Regarding the banches ... we did > a git status > > after step 001a (the utils step) and indeed it was done > on develop. As the > > maven release isn't doing any selecting and switching of > branches it must > > have always been this way. If not the utils release > versions would only be > > updated in the release branch and not develop. I > couldn't find any cherry > > picking of commits in the process. > > > > Well in the end the steps are > way more complicated > > than I would be doing on my local machine. Especially > this tolling around > > checking out branches and tagging and pushing is pretty > complicated, in my > > opinion. Especially as most the stuff is fully automated. > > > > The Maven Release plugin > automatically pushes and > > signs, so yes, the CI steps are more complicated because > they have to > > continue on, but if there is a better way to stop and > continue, let us know > > what that is. But I was mainly addressing the > branch/version issue for > > those two "utils" projects. What would be the correct > set of Maven Release > > commands to manage that locally without updating the > develop version > > twice? Or maybe as you mention above, there really is > no good way and a > > separate repo is essentially the only way. > > > > I think we managed to get the > steps 1-4 (including > > the utils stuff) working again, while keeping the > cleanup I did a while > > ago. > > > > I hope tomorrow it'll be simper > to adjust the > > typedefs and the framework part. > > > > One thing we did notice however > that even after > > addressing the memory issues, the build sometimes simply > stuck and the > > build wouldn't continue. > > In these cases simply killing > the job (and the > > processes on the CI server) didn't really help much ... > we then had to > > reboot the machine in order to continue. I think we > rebooted the thing 5-6 > > times today. > > > > My experience was that the Jenkins > slave would > > occasionally disconnect and the job would fail right > away. Rebooting > > didn’t help. I didn't see it get stuck so unfortunately > you are seeing > > something new that I haven't seen. Did you search the > internet to see if > > anyone else reported something similar with Jenkins? > > > > Signing off for today, > > > > Chris > > > > > > Am 23.03.20, 21:38 schrieb "Alex > Harui" > > <[email protected]>: > > > > Re: credentials: Does that > mean that the git > > config is set to use the credential manager? If so, > maybe that should be > > turned off? When I did the release, I had to type my > password many times. > > Not sure what Piotr's experience was. > > > > Re: Branches: I don't > remember what the > > steps are. I didn't think there was a 1b, I don't see > it in my emails, > > just a 1a that did both compiler-jburg-types and > compiler-build-tools. > > Assuming 1a and 1b now reference these two projects, the > thing to keep in > > mind is that they are optional projects and so 1a (and > probably 1b) won't > > be run for every release. So, IMO, the branch should be > made for the main > > projects in royale-compiler. > > > > But if the changes you made > effectively change > > the way the poms are run (when there was a -utils > profile for those two > > projects, the main pom still got loaded and that may not > be true now), then > > the set of steps may need to test for existence of the > branch or something > > like that, not sure. > > > > In the end, the steps are > just what you would > > run on your local computer to fill the nexus staging > repo, but broken into > > discrete steps at each point Maven would normally push > or sign. If you > > haven’t, it might be worth just running Maven locally to > fill a staging > > repo with and without the jburg-types and build-tools > projects to see how > > to control when Maven makes branches and updates > versions. Then come back > > and break that down into steps on the CI server. > > > > HTH, > > -Alex > > > > On 3/23/20, 12:30 PM, > "Christofer Dutz" < > > [email protected]> wrote: > > > > Another thing we just > discovered. > > > > The current setup seems > to mess up the > > branches: > > - 001 creates the branch > and updates > > develop rel = 0.9.7-SNAPSHOT, develop = 0.9.8-SNAPSHOT > > - 001b updates DEVELOP > and not the release > > branch > > - 002 pushes develop to > the release-branch > > hereby bumping the release branch to the next version > rel = 0.9.8-SNAPSHOT, > > develop = 0.9.8-SNAPSHOT > > > > If the 001b changes > would have been done > > in the release branch, develop would still be using the > old version. > > > > So I would propose to > change the order of > > 001 and 001b to "release" the build tools before > branching. > > > > And I would propose to > fix 002 to work on > > the right branch. As I could see in maven central you > have been releasing > > only even version number so I assume this problem exists > for quite some > > time now. > > > > Chris > > > > Am 23.03.20, 20:12 > schrieb "Carlos Rovira" > > <[email protected]>: > > > > Hi, > > we just saw that > windows stores a > > personal access token in Windows > > Crendetials, so the > RM is responsible > > to remove it when finish all the > > operations. > > > > > > El lun., 23 mar. > 2020 a las 19:44, > > Alex Harui (<[email protected]>) > > escribió: > > > > > > > > > > > On 3/23/20, 11:32 > AM, "Christofer > > Dutz" <[email protected]> > > > wrote: > > > > > > Hi Alex, > > > > > > I did check > and I didn't > > directly find any .git .ssh or whatsoever > > > directories ... do > you have an Idea > > where that would be saved on windows? > > > The commits > are authorized by > > Carlos and it's his RDP connection, > > > that's why I'm > asking if there is > > any RDP magic going on. I didn't see him > > > entering anything > anywhere and he > > said he didn't do it before. > > > > > > I don't know for > sure, but here's a > > few links: > > > > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F46878457%2Fadding-git-credentials-on-windows&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395069878&sdata=alNWQOSo6ka45q7eY7lMLin3EGbKprF89Y5EGQGhkoc%3D&reserved=0 > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-scm.com%2Fbook%2Fen%2Fv2%2FGit-Tools-Credential-Storage&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395079868&sdata=qcHRtHyPDhFJBSsSKzp8KuzYgScS8OXJI1AH1NRw0n8%3D&reserved=0 > > > > > > I'm wondering if > Carlos can remember > > if he did.anything like that back > > > when he wrote this: > > > > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread.html%2F8ae38cea0c736418b432b2353f967161c4f8448261a3bdce390e8c46%2540%253Cdev.royale.apache.org%253E&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395079868&sdata=oVrqNg7LN%2FXFlQb8ziQyhoG6tiz%2B%2FFasO8LpqOd47w8%3D&reserved=0 > > > > > > As I replied back > then, we shouldn't > > have GPG signing on the CI Server, > > > and hopefully no > other credentials > > got added either. > > > > > > HTH, > > > -Alex > > > > > > PS: I'm > purposefully not looking > > myself so as not to accidentally boot > > > someone off the > RDP connection. > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Carlos Rovira > > > > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395079868&sdata=CZoLjFTiKZTp56iMlAaMKgdnKRepk37hsUll0r15DOQ%3D&reserved=0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Carlos Rovira > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395079868&sdata=CZoLjFTiKZTp56iMlAaMKgdnKRepk37hsUll0r15DOQ%3D&reserved=0 > > > > > > > > > -- Carlos Rovira https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C3d1ea5455a3d4bed6e1208d7d038085a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206811395079868&sdata=CZoLjFTiKZTp56iMlAaMKgdnKRepk37hsUll0r15DOQ%3D&reserved=0
