Thank you, Felix and Shane.
+1 for packaging except signing in Jenkins. That sounds perfect for the
next releases (2.2.3, 2.3.3, 3.0.0).
BTW, thanks to Shane, the followings are recovered from today.
1. For all live branches (master to branch-2.2), Jenkins Packaging jobs are
recovered to build and packaging. Only there exist signing errors.
- Master:
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-package/2181/console
- branch-2.4:
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.4-package/95/console
- branch-2.3:
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.3-package/230/console
- branch-2.2:
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.2-package/240/console
2. Daily snapshot maven publishing is recovered. This is helpful for
partial module testing.
-
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-maven-snapshots/
Thank you all. Everything becomes healthier during 2.2.3 release.
This will reduce the burden of 2.3.3/3.0.0 releases. :D
Bests,
Dongjoon.
On Sun, Jan 6, 2019 at 11:42 AM Felix Cheung <felixcheun...@hotmail.com>
wrote:
> Awesome Shane!
>
>
> ------------------------------
> *From:* shane knapp <skn...@berkeley.edu>
> *Sent:* Sunday, January 6, 2019 11:38 AM
> *To:* Felix Cheung
> *Cc:* Dongjoon Hyun; Wenchen Fan; dev
> *Subject:* Re: Spark Packaging Jenkins
>
> noted. i like the idea of building (but not signing) the release and will
> update the job(s) this week.
>
> On Sun, Jan 6, 2019 at 11:22 AM Felix Cheung <felixcheun...@hotmail.com>
> wrote:
>
>> https://spark.apache.org/release-process.html
>>
>> Look for do-release-docker.sh script
>>
>>
>> ------------------------------
>> *From:* Felix Cheung <felixcheun...@hotmail.com>
>> *Sent:* Sunday, January 6, 2019 11:17 AM
>> *To:* Dongjoon Hyun; Wenchen Fan
>> *Cc:* dev; shane knapp
>> *Subject:* Re: Spark Packaging Jenkins
>>
>> The release process doc should have been updated on this - as mentioned
>> we do not use Jenkins for release signing (take this offline if further
>> discussion is needed)
>>
>> The release build on Jenkins can still be useful for pre-validating the
>> release build process (without actually signing it)
>>
>>
>> ------------------------------
>> *From:* Dongjoon Hyun <dongjoon.h...@gmail.com>
>> *Sent:* Saturday, January 5, 2019 9:46 PM
>> *To:* Wenchen Fan
>> *Cc:* dev; shane knapp
>> *Subject:* Re: Spark Packaging Jenkins
>>
>> Thank you, Wenchen.
>>
>> I see. I'll update the doc and proceed to the next step manually as you
>> advise. And it seems that we can stop the outdated Jenkins jobs, too.
>>
>> Bests,
>> Dongjoon.
>>
>> On Sat, Jan 5, 2019 at 20:15 Wenchen Fan <cloud0...@gmail.com> wrote:
>>
>>> IIRC there was a change to the release process: we stop using the shared
>>> gpg key on Jenkins, but use the personal key of the release manager. I'm
>>> not sure Jenkins can help testing package anymore.
>>>
>>> BTW release manager needs to run the packaging script by himself. If
>>> there is a problem, the release manager will find it out sooner or later.
>>>
>>>
>>>
>>> On Sun, Jan 6, 2019 at 6:34 AM Dongjoon Hyun <dongjoon.h...@gmail.com>
>>> wrote:
>>>
>>>> Hi, All.
>>>>
>>>> It turns out that `gpg signing` is the next huddle in Spark Packaging
>>>> Jenkins.
>>>> Since 2.4.0 release, is there something changed in our Jenkins machine?
>>>>
>>>> gpg: skipped
>>>> "/home/jenkins/workspace/spark-master-package/spark-utils/new-release-scripts/jenkins/jenkins-credentials-JEtz0nyn/gpg.tmp":
>>>> No secret key
>>>> gpg: signing failed: No secret key
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Fri, Jan 4, 2019 at 11:52 AM shane knapp <skn...@berkeley.edu>
>>>> wrote:
>>>>
>>>>> https://issues.apache.org/jira/browse/SPARK-26537
>>>>>
>>>>> On Fri, Jan 4, 2019 at 11:31 AM shane knapp <skn...@berkeley.edu>
>>>>> wrote:
>>>>>
>>>>>> this may push in to early next week... these builds were set up
>>>>>> before my time, and i'm currently unraveling how they all work before
>>>>>> pushing a commit to fix stuff.
>>>>>>
>>>>>> nothing like some code archaeology to make my friday more exciting!
>>>>>> :)
>>>>>>
>>>>>> shane
>>>>>>
>>>>>> On Fri, Jan 4, 2019 at 11:08 AM Dongjoon Hyun <
>>>>>> dongjoon.h...@gmail.com> wrote:
>>>>>>
>>>>>>> Thank you, Shane!
>>>>>>>
>>>>>>> Bests,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>> On Fri, Jan 4, 2019 at 10:50 AM shane knapp <skn...@berkeley.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> yeah, i'll get on that today. thanks for the heads up.
>>>>>>>>
>>>>>>>> On Fri, Jan 4, 2019 at 10:46 AM Dongjoon Hyun <
>>>>>>>> dongjoon.h...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi, All
>>>>>>>>>
>>>>>>>>> As a part of release process, we need to check
>>>>>>>>> Packaging/Compile/Test Jenkins status.
>>>>>>>>>
>>>>>>>>> http://spark.apache.org/release-process.html
>>>>>>>>>
>>>>>>>>> 1. Spark Packaging:
>>>>>>>>> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/
>>>>>>>>> 2. Spark QA Compile:
>>>>>>>>> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/
>>>>>>>>> 3. Spark QA Test:
>>>>>>>>> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/
>>>>>>>>>
>>>>>>>>> Currently, (2) and (3) are working because it uses GitHub (
>>>>>>>>> https://github.com/apache/spark.git).
>>>>>>>>> But, (1) seems to be broken because it's looking for old repo(
>>>>>>>>> https://git-wip-us.apache.org/repos/asf/spark.git/info/refs)
>>>>>>>>> instead of new GitBox.
>>>>>>>>>
>>>>>>>>> Can we fix this in this week?
>>>>>>>>>
>>>>>>>>> Bests,
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Shane Knapp
>>>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>>>>>> https://rise.cs.berkeley.edu
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Shane Knapp
>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>>>> https://rise.cs.berkeley.edu
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Shane Knapp
>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>>> https://rise.cs.berkeley.edu
>>>>>
>>>>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
