That has nothing to do with the Ant build of release artifacts. That is just
using Ant to run command line commands to download and verify Maven release
artifacts. I'm pretty sure you can just look at the steps and type them in
manually at the command line and you'll get the same results.
AFAICT, the problem is that the reproducible binaries generated by Maven aren't
reproducible. It could be an issue when the US and Europe have different
daylight savings time settings. When we saw it work, we were not at one of
those transition points. So maybe it is best to just wait until Sunday. But
it did work for me in the US and Piotr in EU.
Proposals for better ways to have an RM verify artifacts built on a remote
machine are welcome. Testing reproducible binaries was the only way I could
think of and implement.
-Alex
On 3/25/20, 11:13 AM, "Carlos Rovira" <carlosrov...@apache.org> wrote:
Hi Alex,
after run CI Server 007 we get this email:
From the royale-typedefs repo:
1. Run ant -f releasesteps.xml Release_Step_007 -Drelease.version=0.9.7
-DskipTests=true
This will download the artifacts then unzip and compile the source artifact.
2. Validate that the compiled artifacts match the downloaded artifacts.
3. If they do, then run ant -f releasesteps.xml Release_Step_007_Sign
-Drelease.version=0.9.7
This will PGP sign the source ZIP and compiled JARs
4. Then run ant -f releasesteps.xml Release_Step_007_Upload
-Drelease.version=0.9.7
This will upload the signed artifacts to Maven Release Staging. Do not
"Close" the staging repository until the other repos have been added.
This is what is failing from sub step 007-1, so as part of the step, I'm
downloading with ant and building, signing and then uploading (Or steps
should allow me to do that, but is failing)
A part from that I don't think we really want a solution where of breaking
steps event more. What we're requesting is remove steps.
El mié., 25 mar. 2020 a las 18:54, Alex Harui (<aha...@adobe.com.invalid>)
escribió:
> I'm sorry, I must have missed something. Where was Ant involved in any of
> the problems reported so far? AFAICT, it was all Maven plugins driving
our
> compiler and we couldn't get reproducible binaries.
>
> I suggested in one reply that Chris fills a staging repo using his local
> machine just to make sure we know what Maven commands need to be run to
> generate all of the Maven artifacts. And that would validate the claim
> that it is really doable in two or three steps. A partial verification of
> validity is to check that the same set of files was produced as in 0.9.6
> (there might be some new SWCs though).
>
> Then someone else (maybe Carlos) would download all of the artifacts, run
> the same steps, download the new artifacts and verify that the binary
> match. Then we can work on splitting those steps into chunks for the CI
> server.
>
> -Alex
>
>
> On 3/25/20, 10:13 AM, "Carlos Rovira" <carlosrov...@apache.org> wrote:
>
> We imposed ourselves a need to release building with two build
systems.
> That's the biggest issue and that's a requirement of this PMC.
> That's what makes the need of the actual CI steps and the complex and
> intricate process.
>
> For the rest of Apache a release is:
>
> "For an *Apache* project, that means any publication outside the
> development community, defined as individuals actively participating
in
> development or following the dev list. More narrowly, an official
> *Apache
> release* is one which has been endorsed as an "act of the Foundation"
> by a
> PMC."
>
> That means that we can release with ANT, with Maven, or any other
build
> systems as part of the process, and even try what we're doing of
> trying to
> build with n build systems and try to check all is ok with the
> artifacts
> generated by all that build systems and check all of them...but if
> that's
> what we want, I personally will left this wagon, and wish you the best
> of
> lucks in the process.
>
> Chris and I proposed to just use Maven *for release* (so please, I
> expect
> no body starts to say we are throwing away ANT from the project for
> building, please....we are just discussing release here).
>
> What do you think about to try to release just with Maven? If can do
> in the
> next day or two and get the release published...will that work for
> you? I
> think we don't need anything more, and I'm sure our users will be
> happy too.
>
> And yes, people wanting then to build with Ant or with Maven will be
> able
> to do so... :-)
>
> Thanks
>
> Carlos
>
>
> [1]
>
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache.org%2Fdev%2Frelease-publishing.html&data=02%7C01%7Caharui%40adobe.com%7Ccaac200c25a74888e76008d7d0e83611%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637207568075948274&sdata=dsxwqItfcgSIUYZMYzgMziBunHmiBgubwQfiHAsaovA%3D&reserved=0
>
>
>
> El mié., 25 mar. 2020 a las 17:46, Christofer Dutz (<
> christofer.d...@c-ware.de>) escribió:
>
> > Hi Yiashay,
> >
> > I agree ... but I didn't create Maven ... haven't even provided a
> single
> > PR to it ;-)
> > I think my only noticeable contribution was that they changed their
> > website to forbid the usage of "maven-{something}-plugin" to Apache
> Maven
> > Core modules (It initially said Apache Maven Plugins which would
have
> > applied for maven-flex-plugin and maven-plc4x-plugin)
> >
> > I think you should focus not on the process, but the bases a process
> > should cover.
> >
> > Focus on what you want to be provided or ensured by any process, no
> matter
> > which tool it is based on.
> >
> > That's what I created the google doc for.
> >
> > If you decide to loosen the requirement on sticking 100% to the
> existing
> > process I am more than willing to help.
> > However if you stick to the current process, it doesn't make any
> sense for
> > me to do so.
> >
> > Looking forward to a productive discussion,
> >
> > Chris
> >
> >
> > Am 25.03.20, 17:36 schrieb "Yishay Weiss" <yishayj...@hotmail.com>:
> >
> > I think the focus should be to agree on the requirements.
> Whether an
> > RM uses the tool created by Chris or Alex can be a matter of choice.
> >
> > From: Carlos Rovira<mailto:carlosrov...@apache.org>
> > Sent: Wednesday, March 25, 2020 5:18 PM
> > To: Apache Royale Development<mailto:dev@royale.apache.org>
> > Subject: Re: Releasing: Finally giving up
> >
> > Hi,
> >
> > first of all, many thanks for all the time invested by Chris
this
> > days. We
> > almost didn't have any normal life this latest 2'5 days, but as
> well
> > many
> > other work (spliced in many hours in the several previous days)
> was
> > prepared to start this release, so for a person out side this
> project,
> > I
> > think we all should thanks a lot his dedications. The good
> output is
> > that
> > process was streamlined, removing unneeded params to enter and
> fixing
> > many
> > bugs over the place. That only can be performed by a good
> specialist in
> > build systems like Chris, I just could be a companion to execute
> steps
> > and
> > try to be trained in the process.
> >
> > About me: I must to say that I tried hard to make this work, but
> I must
> > assume I don't have the skills to do it (what is normal, since
> if an
> > expert
> > in build stuff like Chris can't do it, I don't think any other
> one
> > could do
> > it, and less me). The good point is that I know I have a great a
> deep
> > knowledge about the overall process, and while in November I
> could
> > envision, something of what I could see this days, now I have a
> clear
> > knowledge of all of it (taking into account that we just go
> through
> > compiler and typedefs and not framework, that could be, due to
> size, a
> > much
> > bigger challenge...)
> >
> > About the process: I must say that the initial intention was so
> good,
> > and I
> > think we all want the premise offered thanks to Alex. The
> reality I
> > think
> > is defeating the premise, and we should all see the reality with
> clear
> > sight and without worrying about egos but just looking and what
> the
> > project
> > need.
> >
> > Since the CI process involves so many steps, manual commands in
> CI
> > steps,
> > and...manual commands in local machine. One of the most annoying
> > things is
> > the CI server hanging lots of time (the Java agent exits and
that
> > means you
> > must reboot in order the system working all again without
> problems).
> > The
> > final problem is that you as a RM will fail some times in doing
> what
> > the
> > process needs (indicated in emails), and although you are
> trained a
> > lot in
> > how to do the project you will found stuck in some situations
> where you
> > simple don't know what to do.
> >
> > (For example I forgot or did a bad step that makes me rollover
> things
> > in
> > succeed steps, so I had to roll back versions in the compiler,
> but
> > since I
> > was doing a tools release too, I didn't have into account that I
> need
> > to
> > roll back versions in tools too. It's clear as Chris notice
> that, but
> > not
> > for me at all. So we all should know with this example that the
> > process is
> > not automatic at all. If just involve 2-3 steps, that would be
> > manageable,
> > but we are talking of 13 steps with 32 manual steps (until step
> 7),
> > plus
> > maybe other 30 or so commands until step 13?)...hope you
> understand the
> > challenge here and that is nothing easy for any of us.
> >
> > What I mean, and my advice, from someone that loves this project
> and
> > want
> > it to make it shine: We should try to simple focus in what's
> important.
> > Chris is offering a way that ensures release done simple and
> easily. My
> > take is that we can't afford to go this way more time, since the
> > project
> > will see his existence threatened. We're here many years here,
> and this
> > actual problem is a big one. It's clear that we have a problem
> if we
> > can
> > release every month (or two month) with a process that is easy
> for
> > everyone
> > and requires not a great training to do so, and will require
> some few
> > commands and operations to be done. We simply can't release, and
> we can
> > rely in a process that is so fragile, to make every change we do
> in the
> > future will break it.
> >
> > The fact is that we didn't get so many people jumping to Royale
> and the
> > ones with an eye on this project can finally gone if we continue
> > trying to
> > force things that we don't need. We can release just with maven
> in few
> > steps, that's a fact, and we don't really need to make things in
> the
> > actual
> > way. If we do is because we're forcing it, but nobody cares, and
> my
> > vision
> > is that will cost the project.
> >
> > We simplify the build Maven process to make it more reliable,
> that
> > break
> > the release process in November. The build process has priority
> over
> > the
> > release process, since lots of users depend on them (royale
> devs, and
> > users). The release process is important in the project context
> to
> > offer
> > the bits to the public as a tag or point in time.
> >
> > We simply can't put the release process to the service of the
> build
> > process
> > and make all of that stop in time to not break a fragile release
> > process.
> > That's not an option for an open source project. The release
> process
> > is at
> > the service of the rest (people, builds, code....)
> >
> > I'd want people here to rethink all of this since:
> >
> >
> > - We can't afford this project not to release easily monthly
> (or
> > every 2
> > months) for anyone here (and I mean all people here that
> wants to
> > be an RM).
> > - We can't afford so many hours invested to release, or to
> maintain
> > a
> > process that is very, very, very difficult to operate,
> fragile and
> > easy to
> > break.
> >
> >
> > The motivations are:
> >
> >
> > 1. Although the process was designed and done with very good
> > intentions,
> > the final product does not match the premise. "To make the
> Release
> > process
> > affordable and reliable for each RM"
> > 2. The CI server steps needs you to work on you local
> machine, what
> > defeats the premise to work just in the CI server.
> > 3. The actual process involve 13 steps instead of something
> more
> > manageable like 2-3
> > 4. The actual process involve many manual commands in the CI
> server
> > splitter through the 13 steps (1 to 5 commands for each step
> > depending on
> > the step)
> > 5. The actual process involve many manual commands in your
> local
> > machine
> > splittled every 3 steps. So you need to use your local
> machine what
> > defeats
> > entirely the CI steps to not need the RM to configure his
> machine.
> > This
> > happens in steps 3 and 7 (and fo sure later in frameworks
> part).
> > 6. The CI server use to hang the Jenkins java agent process
> that
> > involves a reboot, what make all the process highly
> cumbersome. You
> > can
> > easily get to reboot CI server 1-2 times while doing the
> first 3
> > steps.
> > 7. If you fail in some step (because you simply fail since
> you're
> > human,
> > or the hangs and things happening make you loose the track of
> what
> > you're
> > doing and make something that provoke the fail) you need to
> > rollback, and
> > know how to operate to restore al the process, crossing
> fingers to
> > expect
> > all goes well, rolling back versions, removing release
branch,
> > removing
> > tags,...)
> > 8. In the end, If you finally get all the process to work,
> since is
> > very, very fragile, is very possible that some updates in the
> > future make
> > it fail again, and the fix will required more and more hours
> of
> > experimented people, patience, and testing.
> >
> > About time. For people trained in the process, (and I think I
> get very
> > trained this days). Steps 1 to 6-7 is about 1h and 30' to 2h. I
> think
> > getting to step 13 and taking into account that framework is
> much more
> > code
> > and time to build, I expect the global process is about 2h or
> maybe
> > more.
> > So we are talking about 3'5 to 4h, of all this complicated
> process with
> > lots of opportunities to fail in some of the ways already
> commented.
> >
> > We have the opportunity now: Chris wanted to use the project for
> PCL4X
> > to
> > build IoT apps. I think that's a huge opportunity for Royale.
> He's
> > giving
> > up to use whatever other frontend tech (React, Angular,
Vue,...).
> >
> > So all this is making Royale to continue moving backwards
> instead to
> > conquer new positions. I have clear that this is not the way to
> go,
> > and if
> > the rest of the team wants to continue forcing all of this. I
> just can
> > do
> > anything but start thinking in other options, a fork or
> something that
> > ensure we are not loosing more for the years to come.
> >
> > I still hope all of you try to think well on all of this. Remove
> egos,
> > since is not about you and me, is about just Apache Royale, and
> nobody
> > out
> > there cares who or what did this or that. Is just a matter of
> what we
> > want
> > for Royale, since people will just see what the project can do
> for him.
> >
> > Hope this long email could make you think, and hope you all will
> > comment to
> > have clear our positions and have clear what we can expect in
> this
> > problematic releasing problem, since is what will make us get
> more
> > traction
> > or in reverse, end loosing more users and maybe die in the way.
> >
> > Thanks
> >
> > Carlos
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > El mié., 25 mar. 2020 a las 12:39, Christofer Dutz (<
> > christofer.d...@c-ware.de>) escribió:
> >
> > > Hi all,
> > >
> > > after 3-4 days of some times 10-16 hours of working on getting
> the
> > > “process” running, I’m finally giving up.
> > > We managed to fix a lot of issues in the way the steps were
> setup and
> > > managed to get to step 7 however there’s no getting past that.
> > >
> > > The problem are the reproducible builds.
> > >
> > > While for the java part I think we have all bases covered,
> however I
> > can’t
> > > see however how I can fix the compiler to actually produce
> > reproducible
> > > builds.
> > >
> > > We are passing in the timestamp (slightly different format,
> however
> > still
> > > correct as we also adjusted the format-string). When building
> > locally, the
> > > timestamp is exactly as the timestamp string tells the
> compiler. On
> > the CI
> > > however there’s this 60 minute offset. I have explicitly set
> the
> > time-zone
> > > wherever I found time-zone relevant code to UTC (Which is the
> > general best
> > > practice for date stuff). However the offset didn’t change. So
> I am
> > > completely out of ideas what could be causing this. If it
> really is
> > the
> > > Summertime in the US, well I don’t know how to fix it (perhaps
> setup
> > a rule
> > > to not release in this time).
> > >
> > > Besides that, there are still differences in the library SWF:
> > >
> > > * As I mentioned yesterday, the node typedef contains a
> “net”
> > object
> > > definition in the local build and a “http” one on the CI
server
> > version.
> > > * In other modules it’s simply the order of items
> generated. I
> > guess
> > > it’s a similar issue as the one I fixed in the build-tools,
> but I
> > have no
> > > idea how to fix it in the compiler
> > >
> > > So in the end I’m giving up … I think the “process” is now way
> more
> > stable
> > > than before, but it’s still a really fragile construct. Today
> Greg,
> > Carlos
> > > and I did it together via Zoom and still we had to re-do some
> steps
> > because
> > > one little preparation was missed.
> > >
> > > I would strongly doubt that the reproducible builds really
> built
> > globally
> > > reproducible builds in the past … perhaps it worked in a
narrow
> > corridor of
> > > time-zones and OS/Java versions (I did use java 1.8), but I
> simply
> > can’t
> > > believe it ever was 100% reproducible.
> > >
> > > So as long as this reproducibility is a requirement, I guess
> we’re
> > done. I
> > > do understand what it’s used for in this context, however I
> think
> > it’s just
> > > not working.
> > >
> > > The other option would be to simply release with maven, which
> would
> > have
> > > been a 1-2 hour experience for all modules.
> > >
> > > I do agree that if the steps work and all preconditions would
> be
> > checked
> > > (which they currently are not) it is very easy to break the
> > “process”. And
> > > it involves running 13 jobs … but as far as I got it also the
> > execution of
> > > 32 manual commands for everything till step 7
> > > 4 + 6 + 3 + 3 + 3 + 4 + 2 + 3 + 3 = 32
> > >
> > > So as long as we’re forced to stick to this process … I guess
> > someone else
> > > will have to do it.
> > >
> > > Please feel free to revert any changes I did in the last few
> months.
> > >
> > > Signing off,
> > > Chris
> > >
> > > PS: You really should make this ApacheRoyaleCI guy a committer
> > considering
> > > the amount of emails he’s writing ;-)
> > >
> > >
> > >
> >
> > --
> > Carlos Rovira
> >
>
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7Ccaac200c25a74888e76008d7d0e83611%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637207568075948274&sdata=yfDv5ocD5wFYvpuHrY2OUyw0tUpc3NZzbzbkaBF35aI%3D&reserved=0
> >
> >
> >
> >
>
> --
> Carlos Rovira
>
>
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7Ccaac200c25a74888e76008d7d0e83611%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637207568075948274&sdata=yfDv5ocD5wFYvpuHrY2OUyw0tUpc3NZzbzbkaBF35aI%3D&reserved=0
>
>
>
--
Carlos Rovira
https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7Ccaac200c25a74888e76008d7d0e83611%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637207568075948274&sdata=yfDv5ocD5wFYvpuHrY2OUyw0tUpc3NZzbzbkaBF35aI%3D&reserved=0
