Hi Alex,
Well it seems as if the "mod" attribute in the catalog.xml is generated from
the zipfile entry
xmlWriter.writeAttribute(ATTR_MOD,
String.valueOf(file.getLastModified()));
And in the SWCWriter there's this code that cuts off the timezone, you even
wrote a comment on it
if (metadataDate != null)
{
// TODO: Perhaps parsing without modification and then serializing
with a default timezone is the more solid approach.
// strip off timezone. Zip format doesn't store timezone
// and the goal is to have the same date and time regardless
// of which timezone the build machine is using.
int c = metadataDate.lastIndexOf(' ');
if(c != -1) {
metadataDate = metadataDate.substring(0, c);
}
c = metadataFormat.lastIndexOf(' ');
if(c != -1) {
metadataFormat = metadataFormat.substring(0, c);
}
try {
SimpleDateFormat sdf = new
SimpleDateFormat(metadataFormat);
sdf.setTimeZone(TimeZone.getTimeZone("UTC"));
fileDate = sdf.parse(metadataDate).getTime();
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalArgumentException e1) {
e1.printStackTrace();
}
}
So in general it doesn’t matter what timezone is in the string input, it was
cut off anyway.
I added the line:
sdf.setTimeZone(TimeZone.getTimeZone("UTC"));
But it seems I still have the same issue with the con
re-inspecting the code I noticed, it's probably just the timestamp used in the
zip file metadata ... not that used in the catalog.xml.
Will have another look at why I'm having this 36 offset. ... I just did a
little math and it's exactly one hour off ...
The difference of all tags is:
3600000 milliseconds, which is exactly one hour. So the timestamps on the CI
server are exactly one hour ahead of the builds we do in Europe.
This is currently the last issue preventing us from finishing the steps for the
typedefs.
Do you have any idea? ... Anything with the US being Summer-Time and us still
on Winter-Time?
Perhaps you could try executing this command in the typedefs:
ant -f releasesteps.xml Release_Step_007 -Drelease.version=0.9.7
-DskipTests=true
Would be interesting if this succeeds for you.
Well actually the library.swf in the swc seems to have a "net" Object in the
locally built version and a "http" Object instead on the CI Server-Version.
Currently I'm trying to fix the timestamp version and address the other problem
after that's solved.
Chris
Am 24.03.20, 20:08 schrieb "Alex Harui" <[email protected]>:
Good to see progress.
Re timestamps: I’m not quite sure what you are running into.
The builds were taking a time that included a timezone. Here's some output
from recent builds:
[java] -metadata.date=02/25/20 14:51 +0000
[java] -metadata.dateFormat=MM/dd/yy HH:mm Z
So I'm not sure how the timezone got lost on what you are seeing. The RM
was supposed to type in something like "02/25/20 14:51 +0000". The prompts on
the Jenkins job were supposed to hint that.
HTH,
-Alex
On 3/24/20, 7:47 AM, "Carlos Rovira" <[email protected]> wrote:
Hi Alex,
I finally said Chris to try it myself, so going with that now :)
El mar., 24 mar. 2020 a las 15:40, Christofer Dutz (<
[email protected]>) escribió:
> Hi Alex,
>
> I added a call to set a fixed timezone to the simpledateformat in the
> SWCWriter ... could you please review this?
>
> Chris
>
>
>
> Am 24.03.20, 15:26 schrieb "Christofer Dutz"
<[email protected]>:
>
> Hi all,
>
> so today we made progress ... not only did we fix the version
skipping
> we observed yesterday. For this the commands done manually in the
001a step
> had to be changed.
>
> Now we managed to get the compiler part reproducible, but not so
much
> the typedef part.
> We are observing that the same timestamp is passed in to the build
> when building the release on the ci server and when building the
release
> locally. However the timestamps have a strange constant offset 36 in
two
> digits of the timestamp in the catalog.xml
>
> 1585054125000 on the CI server
> 1585057725000 locally
>
> I am assuming that perhaps there is time-zone information leaking
into
> the parsing of the date as we are using a date-format without
time-zone
> information.
>
> I would assume that Alex, perhaps you could have a look at what's
> happening in the SWCWriter constructor, where the parsing of the
timestamp
> is implemented.
>
> Thanks,
>
> Chris
>
>
> Am 24.03.20, 09:36 schrieb "Christofer Dutz" <
> [email protected]>:
>
> Hi Alex,
>
> I had another look ... I think the version skips one is in
step
> 001a ...
> There the job checks out develop, applies the update to the
new
> version of the build tools and jburg types, but then this is pushed
and
> then again to the release branch (hereby overwriting the old
> release-branch) ... I'll try out a way with cherry picking the change
from
> develop into the release branch. Unfortunately I haven't been able to
find
> where the text in the emails comes from ...
>
> I'm talking about the order of the constants in the class
> ProblemID. I forced that to be sorted now. That's why we have to
re-release
> the build tools.
>
> Chris
>
> Am 24.03.20, 08:14 schrieb "Alex Harui"
<[email protected]
> >:
>
> Re: credentials: Feel free to disable the credential
store.
> I did not have to both login and go through 2fa to Git Push. There
is
> some long password/token/credential/hash-like thing you get when you
sign
> up with Apache Git that seems to work when you use it as a password.
>
> Re: the utils steps. I just looked and it appears that
> compiler-jburg-types and compiler-build-tools are no longer
> child/submodules of the root pom.xml in royale-compiler. So now I
get how
> why the development version won't get bumped up twice. I think the
way it
> was working may be that the updated dev version never got pushed to
> develop. If you are satisfied with that setup, that's fine with me.
So
> then it might make more sense to get rid of 1a and create a whole new
set
> of steps for each of compiler-jburg-types and compiler-build-tools.
If I
> understand correctly, those new steps would run mvn release:branch,
mvn
> release:prepare and mvn release:perform in compiler-jburg-types or
> compiler-build-tools, stopping if there are interim pushes that need
to be
> done, then start their own vote emails by copying other CI steps.
>
> I use a Mac and thought I got all binaries to match from
the
> . I'm not sure what properties you are referring to.
>
> I'm done for tonight. Will catch up in my morning.
>
> -Alex
>
> On 3/23/20, 11:15 PM, "Christofer Dutz" <
> [email protected]> wrote:
>
> Good morning all,
>
> well I didn't disable anythig as I didn't want to
break
> anything. So I thought before simply changing something I'd ask first.
> So I would consider it a "bug" that it's there and
when we
> continue today, we'll try to find out and disable the thing.
> However I would assume then the RM will not only have
to
> do his usual login but also the 2fa thing at every step too, correct?
>
> And, just a kindly meant question ... would it be
possible
> to reply at the top instead of inline? I sort of have missed several
> Of your Inline comments in the past as at least in my
mail
> client it's hard to spot what's mine and what's not.
> (I could start counting spaces)
>
> (I first missed this question of yours ... but
discovered
> it after counting spaces ;-) )
>
> Well to release the build tool with maven you would
simply
> do the:
> mvn release:prepare
> mvn release:perform
> for each of them, then adjust the main pom and do the
mvn
> release:branch ... then prepare and perform after that.
>
> Currently also jburg types and the build-tools are
sort of
> released in parallel. With maven you would simply release only the
module
> that's needed to be released. In our case build-tools would have been
> enough.
>
> Speaking of build-tools ... are you possibly using
Windows
> as OS on your system? Just asking because the build-tools seem to have
> always generated a sorted set of properties on windows and an
unsorted one
> on Mac (Seems to be differences in the way the filesystem works) ...
in
> that case it would have been impossible to have a release manager not
using
> Java 8 on a Windows machine. (My changes recently never touched this)
>
> Chris
>
>
> Am 23.03.20, 22:59 schrieb "Alex Harui"
> <[email protected]>:
>
>
>
> On 3/23/20, 2:29 PM, "Christofer Dutz" <
> [email protected]> wrote:
>
> Hi Alex,
>
> sorry for the confusion ,... I meant the 1a
...
> the one if you build the utils too.
>
> And regarding the credentials: Something must
have
> changes as I have never seen that gui thingy pop up before and as
soon as
> you login, it saves them in the password manager thingy of windows.
>
> Did you try to disable the password manager?
>
> It is true that I took those two out of the
main
> maven reactor, but that's actually a good practice. In plc4x we setup
a
> dedicated git repo for releasing these build tools. I have never seen
any
> good coming from the old setup (I know ... I did it, but I learnt a
lot).
>
> I am not opposed to moving those two projects to a
> different repo. It would require that we have a truly separate
release
> vote for them, so more work in that regard, and the Ant build would
have to
> be adjusted to download those jars. I'm not sure now is the time to
make
> that change, but maybe. Let's see what others think.
>
> Regarding the banches ... we did a git status
> after step 001a (the utils step) and indeed it was done on develop.
As the
> maven release isn't doing any selecting and switching of branches it
must
> have always been this way. If not the utils release versions would
only be
> updated in the release branch and not develop. I couldn't find any
cherry
> picking of commits in the process.
>
> Well in the end the steps are way more
complicated
> than I would be doing on my local machine. Especially this tolling
around
> checking out branches and tagging and pushing is pretty complicated,
in my
> opinion. Especially as most the stuff is fully automated.
>
> The Maven Release plugin automatically pushes and
> signs, so yes, the CI steps are more complicated because they have to
> continue on, but if there is a better way to stop and continue, let
us know
> what that is. But I was mainly addressing the branch/version issue
for
> those two "utils" projects. What would be the correct set of Maven
Release
> commands to manage that locally without updating the develop version
> twice? Or maybe as you mention above, there really is no good way
and a
> separate repo is essentially the only way.
>
> I think we managed to get the steps 1-4
(including
> the utils stuff) working again, while keeping the cleanup I did a
while
> ago.
>
> I hope tomorrow it'll be simper to adjust the
> typedefs and the framework part.
>
> One thing we did notice however that even
after
> addressing the memory issues, the build sometimes simply stuck and the
> build wouldn't continue.
> In these cases simply killing the job (and the
> processes on the CI server) didn't really help much ... we then had to
> reboot the machine in order to continue. I think we rebooted the
thing 5-6
> times today.
>
> My experience was that the Jenkins slave would
> occasionally disconnect and the job would fail right away. Rebooting
> didn’t help. I didn't see it get stuck so unfortunately you are
seeing
> something new that I haven't seen. Did you search the internet to
see if
> anyone else reported something similar with Jenkins?
>
> Signing off for today,
>
> Chris
>
>
> Am 23.03.20, 21:38 schrieb "Alex Harui"
> <[email protected]>:
>
> Re: credentials: Does that mean that the
git
> config is set to use the credential manager? If so, maybe that
should be
> turned off? When I did the release, I had to type my password many
times.
> Not sure what Piotr's experience was.
>
> Re: Branches: I don't remember what the
> steps are. I didn't think there was a 1b, I don't see it in my
emails,
> just a 1a that did both compiler-jburg-types and compiler-build-tools.
> Assuming 1a and 1b now reference these two projects, the thing to
keep in
> mind is that they are optional projects and so 1a (and probably 1b)
won't
> be run for every release. So, IMO, the branch should be made for the
main
> projects in royale-compiler.
>
> But if the changes you made effectively
change
> the way the poms are run (when there was a -utils profile for those
two
> projects, the main pom still got loaded and that may not be true
now), then
> the set of steps may need to test for existence of the branch or
something
> like that, not sure.
>
> In the end, the steps are just what you
would
> run on your local computer to fill the nexus staging repo, but broken
into
> discrete steps at each point Maven would normally push or sign. If
you
> haven’t, it might be worth just running Maven locally to fill a
staging
> repo with and without the jburg-types and build-tools projects to see
how
> to control when Maven makes branches and updates versions. Then come
back
> and break that down into steps on the CI server.
>
> HTH,
> -Alex
>
> On 3/23/20, 12:30 PM, "Christofer Dutz" <
> [email protected]> wrote:
>
> Another thing we just discovered.
>
> The current setup seems to mess up the
> branches:
> - 001 creates the branch and updates
> develop rel = 0.9.7-SNAPSHOT, develop = 0.9.8-SNAPSHOT
> - 001b updates DEVELOP and not the
release
> branch
> - 002 pushes develop to the
release-branch
> hereby bumping the release branch to the next version rel =
0.9.8-SNAPSHOT,
> develop = 0.9.8-SNAPSHOT
>
> If the 001b changes would have been
done
> in the release branch, develop would still be using the old version.
>
> So I would propose to change the
order of
> 001 and 001b to "release" the build tools before branching.
>
> And I would propose to fix 002 to
work on
> the right branch. As I could see in maven central you have been
releasing
> only even version number so I assume this problem exists for quite
some
> time now.
>
> Chris
>
> Am 23.03.20, 20:12 schrieb "Carlos
Rovira"
> <[email protected]>:
>
> Hi,
> we just saw that windows stores a
> personal access token in Windows
> Crendetials, so the RM is
responsible
> to remove it when finish all the
> operations.
>
>
> El lun., 23 mar. 2020 a las 19:44,
> Alex Harui (<[email protected]>)
> escribió:
>
> >
> >
> > On 3/23/20, 11:32 AM,
"Christofer
> Dutz" <[email protected]>
> > wrote:
> >
> > Hi Alex,
> >
> > I did check and I didn't
> directly find any .git .ssh or whatsoever
> > directories ... do you have an
Idea
> where that would be saved on windows?
> > The commits are authorized
by
> Carlos and it's his RDP connection,
> > that's why I'm asking if there
is
> any RDP magic going on. I didn't see him
> > entering anything anywhere and
he
> said he didn't do it before.
> >
> > I don't know for sure, but
here's a
> few links:
> >
> >
> >
>
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F46878457%2Fadding-git-credentials-on-windows&data=02%7C01%7Caharui%40adobe.com%7C1599b62f96cf480e8f0608d7d002388d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206580278180034&sdata=wixiuEWeJIEwbTk8UZIh7vzB3KpxjwssAx2zkYtUN1k%3D&reserved=0
> >
> >
>
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-scm.com%2Fbook%2Fen%2Fv2%2FGit-Tools-Credential-Storage&data=02%7C01%7Caharui%40adobe.com%7C1599b62f96cf480e8f0608d7d002388d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206580278180034&sdata=2LFP5g40Cc7bPcgGMLzwd6bj2asuJd0%2BCJvRjYh9rYQ%3D&reserved=0
> >
> > I'm wondering if Carlos can
remember
> if he did.anything like that back
> > when he wrote this:
> >
> >
>
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread.html%2F8ae38cea0c736418b432b2353f967161c4f8448261a3bdce390e8c46%2540%253Cdev.royale.apache.org%253E&data=02%7C01%7Caharui%40adobe.com%7C1599b62f96cf480e8f0608d7d002388d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206580278180034&sdata=M6eP12UqOyguJ87vtnGnvaJM4rIpxe9IsrDBRN8z4S8%3D&reserved=0
> >
> > As I replied back then, we
shouldn't
> have GPG signing on the CI Server,
> > and hopefully no other
credentials
> got added either.
> >
> > HTH,
> > -Alex
> >
> > PS: I'm purposefully not looking
> myself so as not to accidentally boot
> > someone off the RDP connection.
> >
> >
> >
> >
> >
> >
> >
>
> --
> Carlos Rovira
>
>
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C1599b62f96cf480e8f0608d7d002388d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206580278180034&sdata=W4ij818dadNgTDcz6JN0FM4oz5Scb5BDdrMdGjFYs%2BU%3D&reserved=0
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
--
Carlos Rovira
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C1599b62f96cf480e8f0608d7d002388d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637206580278180034&sdata=W4ij818dadNgTDcz6JN0FM4oz5Scb5BDdrMdGjFYs%2BU%3D&reserved=0
