Hello All,
I'm going to go into a fair bit of detail relative to previous updates, so
that everyone has a clearer idea of exactly where we are at, the current
state, and what still needs to be done.
*Server Configuration Changes*
First off, the new Jenkins instance has some differences from the old one:
- It is, of course, running on a newer version of Jenkins, with security
updates installed, so some things may look a little different.
- Worker nodes now get spun up (and down) dynamically (via Amazon
SpotFleet). Ideally, this should give decent build performance without
having a large number of resources idling when there are no builds in
progress. This does result in a couple of compromises:
- If there are no jobs running for a while, all the worker nodes will
get shut down, and will take a little bit of time to start up
when the next
job is started
- It makes copying nightly artifacts back onto the master a little
more complicated, although this should be (mostly) working now.
At current,
artifacts get archived to the master node by Jenkins, then copied to the
location we use to host the nightly builds via a nightly cron
job (so they
will not show up immediately). I hope to change this to just ssh from the
worker node to the hosting location on master if I can figure
out how. One
benefit of the new configuration is that artifacts are now also linked
directly from the nightly build that produced them.
- Configuration can not be assumed to be persistent between builds.
This will likely require the release jobs to be reconfigured somewhat.
To help deal with this, some configuration which used to just be assumed to
be on the build server is now managed by Jenkins (which is overall,
probably a good thing). This currently includes: Java versions/installers,
Git identity, and the Maven settings.xml, and will eventually include keys
for GitHub, SourceForge, and GeoTools.org, once those get added.
*Job Status:*
I have been focusing on getting the main jobs for GeoTools, GeoWebCache,
and GeoServer working.
- geoserver-<ver>, geoserver-<ver>-nightly and geoserver-<ver>-live-docs
are all working (except for geoserver-master-live-docs, but this appears to
be caused by a legitimate error in the docs on master
- geowebcache-<ver> jobs are currently working (and deploying to maven).
geowebcache-<ver>-nightly jobs are not, as they assume they will have
access to the results of geowebcache-<ver>, which is no longer true. They
will need to be reconfigured. I'm currently looking into this.
- geotools-<ver> and geotools-<ver>-docs builds are building
successfully, but failing when they try to copy javadocs/docs
(respectively) to geotools.org. We need to aquire the key to geotools.org
and add it to jenkins before we can get these jobs working (or disable doc
uploads until we have the key).
- Most *-java11 jobs are working, since they generally don't need to
copy artifacts anywhere (java 11 has been configured in jenkins)
I haven't looked at any of the other jobs yet
- Cite and Online jobs aren't going to work until PostGIS is configured
on the worker nodes
- Release jobs require github and sourceforge keys, and may need some
other modifications
- I haven't looked at the geofence/geogig/geoscript jobs, and don't know
what is needed to get them working.
Mailing list notifications also aren't working yet - a bit of configuration
still needs to be done on the build server before those will be working
(and its probably best to get most of the jobs working first anyways, to
avoid excess noise).
*How you can help*
- If you are a PSC member and would like to be able to
start/stop/configure jobs on the new jenkins server, please create a new
Jenkins account by visiting build.geoserver.org* please click "sign up"
in the top right corner*. You will initially have the same permissions
as if you were not logged in - after creating an account *please email
me with your account name* and I will elevate your permissions.
All of following points will only be applicable to members who currently
have a Jenkins account.
- If you are familiar with the geofence/geogig/geoscript jobs, please
take a look at the recent failures to see what is going wrong there (and
maybe see if you can fix it)
- If you are familiar with geowebcache, any modifications to the
geowebcache-<ver>-nightly jobs so that they don't rely on the corresponding
geowebcache-<ver> job being built in the same workspace would be
appreciated (although if you do make changes, please contact me so that we
aren't both changing the same job at the same time)
The following points will require some coordination between various project
administrators:
- If you have access to the geotools.org ssh key, please contact me so
we can add it to Jenkins.
- If you are an administrator for both the GeoTools and GeoServer
projects in GitHub, try and create a separate CI account with push
permissions to those projects and generate an ssh key for it, so that the
release jobs will be able to push artifacts to GitHub
- If you are an administrator for both the GeoTools and GeoServer
projects in SourceForge, try and create a separate CI account with push
permissions to those projects and generate an ssh key for it, so that the
release jobs will be able to push artifacts to SourceForge
Thanks,
Torben
On Thu, May 23, 2019 at 11:36 AM Andrea Aime <[email protected]>
wrote:
> Hi Torben,
> please let us know where you're at at the end of your day. I'm not sure I
> can help tomorrow but
> if worst come to worst I could give it a try during the weekend
>
> Cheers
> Andrea
>
> On Thu, May 23, 2019 at 7:08 PM Torben Barsballe <
> [email protected]> wrote:
>
>> The main thing blocking functionality at this point is adding keys for
>> all the things Jenkins needs to talk to.
>> A lot of the original keys used before were set up by Justin using his
>> accounts back when we were still using Ares; given that there was a
>> security breach I'd like us to generate new keys, and ideally use a shared
>> / bot account for them. If someone is able to create such generic accounts
>> and generate ssh keys for them, that would help a lot.
>>
>> Keys/accounts needed:
>>
>> - github (For releases; commit access to GeoTools, GeoServer)
>> - sourceforge (For releases, push access to GeoTools, GeoServer)
>>
>> I'll also need a copy of the key that is used to push docs to
>> geotools.org in order to get the geotools builds working.
>>
>> The artifactory key is already set up, and I am currently working on
>> setting up a key so the worker nodes can talk to master to push nightly
>> builds (again).
>> If there are any other keys I'm forgetting about, that would be good to
>> know as well.
>>
>> Thanks,
>> Torben
>>
>> On Wed, May 22, 2019 at 11:24 PM Andrea Aime <
>> [email protected]> wrote:
>>
>>> Hi Torben,
>>> thanks!
>>>
>>> Is there anything we can do to help getting the rest of the
>>> functionality up and running?
>>>
>>> Cheers
>>> Andrea
>>>
>>>
>>> On Wed, May 22, 2019 at 10:23 PM Torben Barsballe <
>>> [email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Wed, May 22, 2019 at 9:13 AM Torben Barsballe <
>>>> [email protected]> wrote:
>>>>
>>>>>
>>>>> On Tue, May 21, 2019 at 11:47 PM Andrea Aime <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi Torben,
>>>>>> thanks for the update! Nice to see it online.
>>>>>> It seems that nightly build results are not accessible though, even
>>>>>> going without https, e.g.: http://build.geoserver.org/geoserver
>>>>>> won't respond.
>>>>>>
>>>>>
>>>>> This was working yesterday, but isn't working for me now either; I'll
>>>>> look into what might have changed.
>>>>>
>>>>> I found out the issue - there was an error with how redirects were
>>>> handled when the trailing slash was missing from the URL (e.g.
>>>> http://build.geoserver.org/geoserver vs
>>>> http://build.geoserver.org/geoserver/ - It should be fixed now.
>>>>
>>>> Torben
>>>>
>>>>
>>>>
>>>>> Torben
>>>>>
>>>>>
>>>>
>>>
>>> --
>>>
>>> Regards, Andrea Aime == GeoServer Professional Services from the
>>> experts! Visit http://goo.gl/it488V for more information. == Ing.
>>> Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito
>>> 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob:
>>> +39 339 8844549 http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>> ------------------------------------------------------- *Con
>>> riferimento alla normativa sul trattamento dei dati personali (Reg. UE
>>> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
>>> precisa che ogni circostanza inerente alla presente email (il suo
>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
>>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>>> This email is intended only for the person or entity to which it is
>>> addressed and may contain information that is privileged, confidential or
>>> otherwise protected from disclosure. We remind that - as provided by
>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
>>> e-mail or the information herein by anyone other than the intended
>>> recipient is prohibited. If you have received this email by mistake, please
>>> notify us immediately by telephone or e-mail.*
>>>
>>
>
> --
>
> Regards, Andrea Aime == GeoServer Professional Services from the experts!
> Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime
> @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054
> Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339
> 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it
> ------------------------------------------------------- *Con riferimento
> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
> circostanza inerente alla presente email (il suo contenuto, gli eventuali
> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
> sarei comunque grato se potesse darmene notizia. This email is intended
> only for the person or entity to which it is addressed and may contain
> information that is privileged, confidential or otherwise protected from
> disclosure. We remind that - as provided by European Regulation 2016/679
> “GDPR” - copying, dissemination or use of this e-mail or the information
> herein by anyone other than the intended recipient is prohibited. If you
> have received this email by mistake, please notify us immediately by
> telephone or e-mail.*
>
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
