7;m not holding my breath, however... only time will tell.
On Tue, Mar 19, 2019 at 7:18 AM Imran Rashid wrote:
> seems wedged again?
>
> sorry for the bad news Shane, thanks for all the work on fixing it
>
> On Mon, Mar 18, 2019 at 4:02 PM shane knapp wrote:
>
>> ok, i d
again,
shane
On Thu, Mar 21, 2019 at 12:54 PM shane knapp wrote:
> i tweaked some apache settings (MaxClients increased to fix an error i
> found buried in the logs, and added 'retry' and 'acquire' to the reverse
> proxy settings to hopefully combat the dreaded 502
quick update:
since kicking httpd on the jenkins master "fixes" the GUI hanging, i set up
a cron job to restart httpd 4 times per day.
this is not the final solution, but will definitely help over the weekend
as i'm heading out of town.
shane
On Fri, Mar 22, 2019 at 9:50 AM sha
(I cc'ed Shane as well).
>
> PS: I roughly heard that Shane's busy for some work stuff .. but it's kind
> of important in my perspective.
>
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
wrote:
>
>> +1 on doing this in 3.0.
>>
>>
>> On Mon, Mar 25, 2019 at 9:31 PM, Felix Cheung
>> wrote:
>>
>>> I’m +1 if 3.0
>>>
>>>
>>> --
>>> *From:* Sean Owen
>>> *Sent:
ulos
and the PR from hell for throwing me in
the deep end of k8s. :)
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
>
>
> If there was, it looks inevitable to upgrade Jenkins\s Python from 3.4 to
> 3.5.
>
> this is inevitable. 3.4s final release was 10 days ago (
https://www.python.org/dev/peps/pep-0429/) so we're basically EOL.
looks like the same for 3.5... https://www.python.org/dev/peps/pep-0478/
let's pick a python version and start testing.
On Thu, Mar 28, 2019 at 7:52 PM shane knapp wrote:
>
>> If there was, it looks inevitable to upgrade Jenkins\s Python from 3.4 to
>> 3.5.
>>
&g
>
>>
>>
>> --
>> *From:* shane knapp
>> *Sent:* Thursday, March 28, 2019 7:54 PM
>> *To:* Hyukjin Kwon
>> *Cc:* Bryan Cutler; dev; Felix Cheung
>> *Subject:* Re: Upgrading minimal PyArrow version to 0.12.x [SPARK-27276]
gt; In any case I think in the next release it will be great to get more
> Python 3.x release test coverage.
>
>
>
> ------
> *From:* shane knapp
> *Sent:* Friday, March 29, 2019 4:46 PM
> *To:* Bryan Cutler
> *Cc:* Felix Cheung; Hyukjin Kwon; de
ade... however, i'll be out wednesday through friday this week and
don't want to make a massive change before disappearing for a few days.
so: how does early next week sound for the python upgrade? :)
shane
On Mon, Apr 1, 2019 at 8:58 AM shane knapp wrote:
> i'd much p
will be fine, as a point of
> reference, pyarrow CI uses 2.7 and 3.6.
>
> On Mon, Apr 1, 2019 at 3:09 PM shane knapp wrote:
>
>> well now! color me completely surprised... i decided to whip up a fresh
>> python3.6.8 conda environment this morning to "see if things just work
the host serving the reverse proxy to jenkins is wedged and is being
rebooted now. jenkins is still up and happily building, so no jenkins
restart required.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
github.com/apache/spark/pull/24266
question: which other branches should i be applying this change to?
definitely master and 2.4... also 2.3? 2.2?
thanks in advance,
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
thon 3.4; would this mean it's now just
> tested vs 3.6? that's not out of the question, but can the older
> branches continue to test on older versions or is that super complex?
>
> On Wed, Apr 10, 2019 at 1:37 PM shane knapp wrote:
> >
> > details here (see most r
ct if something broke for 3.4 which is still 'supported'
> in 2.3.x, 2.4.x? For 3.x, no problem, let's raise the minimum anyway.
> This could be fine if we accept that small risk and it keeps the
> testing env from getting crazy; I don't actually want to test vs N
> Py
n Mon, Apr 15, 2019 at 11:49 AM shane knapp wrote:
> 1) i absolutely do not want to test against more than two python
> versions. consider my foot to have been put down on that. :)
>
> 2) i'll start testing against 2.3 and 2.4 now (last week was a bit crazy,
> so i didn'
this afternoon about the upgrade.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
ake more than an hour and i really expect things to be
back up and running pretty quickly. i will send updates as needed.
shane
1-- this will be for 2.3/2.4 only, and tests against pandas 0.19.2 and
pyarrow 0.8.0. master tests against pandas 0.23.2 and pyarrow 0.12.1
--
Shane Knapp
UC Berkele
i will actually be doing this now!
On Thu, Apr 18, 2019 at 2:57 PM shane knapp wrote:
> well, upon us on monday. :)
>
> firstly, an important note: if you have an open PR, please check to see
> if you need to rebase my changes on it before testing.
>
> monday @ 11am PST,
and this is done.
welcome to the brave new world of python3.6!
On Fri, Apr 19, 2019 at 9:34 AM shane knapp wrote:
> i will actually be doing this now!
>
>
>
> On Thu, Apr 18, 2019 at 2:57 PM shane knapp wrote:
>
>> well, upon us on monday. :)
>>
>> firstly
pact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==
>> But my bug isn't fixed?
>> ==
>>
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
demos (schedule TBD).
our booth is right behind pepperdata's, on the left as you enter the expo
floor.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
jenkins is back up, and PRB jobs should start trickling in.
On Fri, Apr 26, 2019 at 12:20 PM shane knapp wrote:
> if this doesn't fix things, i will reboot the server.
>
> shane (who is also dealing w/a major sewer backup @ his house)
> --
> Shane Knapp
> UC Berkeley
if this doesn't fix things, i will reboot the server.
shane (who is also dealing w/a major sewer backup @ his house)
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
d for both lab and spark integration tests, but will
NOT BE READY for the next couple of weeks.
i repeat: even though there will be GPUs, they will not be ready for use
yet. ;)
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
machines are down, gpus are about to go in. i expect these workers to back
up and building in ~30min.
On Fri, May 17, 2019 at 1:47 PM shane knapp wrote:
> we're installing some new GPUs for builds to use for tests... the
> following workers will be offline for the next coup
amp-jenkins-staging-worker-02 and ubuntu-testing are back up.
-01 is being a little reluctant to boot and we're investigating.
On Fri, May 17, 2019 at 2:08 PM shane knapp wrote:
> machines are down, gpus are about to go in. i expect these workers to
> back up and building in ~30
all workers are now up, online and ready to build!
On Fri, May 17, 2019 at 2:55 PM shane knapp wrote:
> amp-jenkins-staging-worker-02 and ubuntu-testing are back up.
>
> -01 is being a little reluctant to boot and we're investigating.
>
> On Fri, May 17, 2019 at 2:08 P
actually, amp-jenkins-staging-worker-01 is seriously unhappy and just
crashed. we will investigate more on monday.
:(
shane
On Fri, May 17, 2019 at 3:19 PM shane knapp wrote:
> all workers are now up, online and ready to build!
>
> On Fri, May 17, 2019 at 2:55 PM shane knapp wrote:
t;>>>>> Remember that JIRAs stay searchable and can be reopened, so
>>>>>>>>>>>> it's not
>>>>>>>>>>>> like we lose much information.
>>>>>>>>>>>>
>>>>>>>>>>>> I'd close anything that hasn't had activity in 2 years (?), as
>>>>>>>>>>>> a start.
>>>>>>>>>>>> I like the idea of closing things that only affect an EOL
>>>>>>>>>>>> release,
>>>>>>>>>>>> but, many items aren't marked, so may need to cast the net
>>>>>>>>>>>> wider.
>>>>>>>>>>>>
>>>>>>>>>>>> I think only then does it make sense to look at bothering to
>>>>>>>>>>>> reproduce
>>>>>>>>>>>> or evaluate the 1000s that will still remain.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon <
>>>>>>>>>>>> gurwls...@gmail.com> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Hi all,
>>>>>>>>>>>> >
>>>>>>>>>>>> > I would like to propose to resolve all JIRAs that affects EOL
>>>>>>>>>>>> releases - 2.2 and below. and affected version
>>>>>>>>>>>> > not specified. I was rather against this way and considered
>>>>>>>>>>>> this as last resort in roughly 3 years ago
>>>>>>>>>>>> > when we discussed. Now I think we should go ahead with this.
>>>>>>>>>>>> See below.
>>>>>>>>>>>> >
>>>>>>>>>>>> > I have been talking care of this for so long time almost
>>>>>>>>>>>> every day those 3 years. The number of JIRAs
>>>>>>>>>>>> > keeps increasing and it does never go down. Now the number is
>>>>>>>>>>>> going over 2500 JIRAs.
>>>>>>>>>>>> > Did you guys know? in JIRA, we can only go through page by
>>>>>>>>>>>> page up to 1000 items. So, currently we're even
>>>>>>>>>>>> > having difficulties to go through every JIRA. We should
>>>>>>>>>>>> manually filter out and check each.
>>>>>>>>>>>> > The number is going over the manageable size.
>>>>>>>>>>>> >
>>>>>>>>>>>> > I am not suggesting this without anything actually trying.
>>>>>>>>>>>> This is what we have tried within my visibility:
>>>>>>>>>>>> >
>>>>>>>>>>>> > 1. In roughly 3 years ago, Sean tried to gather committers
>>>>>>>>>>>> and even non-committers people to sort
>>>>>>>>>>>> > out this number. At that time, we were only able to keep
>>>>>>>>>>>> this number as is. After we lost this momentum,
>>>>>>>>>>>> > it kept increasing back.
>>>>>>>>>>>> > 2. At least I scanned _all_ the previous JIRAs at least
>>>>>>>>>>>> more than two times and resolved them. Roughly
>>>>>>>>>>>> > once a year. The rest of them are mostly obsolete but not
>>>>>>>>>>>> enough information to investigate further.
>>>>>>>>>>>> > 3. I strictly stick to "Contributing to JIRA Maintenance"
>>>>>>>>>>>> https://spark.apache.org/contributing.html and
>>>>>>>>>>>> > resolve JIRAs.
>>>>>>>>>>>> > 4. Promoting other people to comment on JIRA or actively
>>>>>>>>>>>> resolve them.
>>>>>>>>>>>> >
>>>>>>>>>>>> > One of the facts I realised is the increasing number of
>>>>>>>>>>>> committers doesn't virtually help this much (although
>>>>>>>>>>>> > it might be helpful if somebody active in JIRA becomes a
>>>>>>>>>>>> committer.)
>>>>>>>>>>>> >
>>>>>>>>>>>> > One of the important thing I should note is that, it's now
>>>>>>>>>>>> almost pretty difficult to reproduce and test the
>>>>>>>>>>>> > issues found in EOL releases. We should git clone, checkout,
>>>>>>>>>>>> build and test. And then, see if that issue
>>>>>>>>>>>> > still exists in upstream, and fix. This is non-trivial
>>>>>>>>>>>> overhead.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Therefore, I would like to propose resolving _all_ the JIRAs
>>>>>>>>>>>> that targets EOL releases - 2.2 and below.
>>>>>>>>>>>> > Please let me know if anyone has some concerns or objections.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
x27;t be expanding.
that being said, i will be cracking a bottle of champagne when i can delete
all of the ansible and anaconda configs for python2.x. :)
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
;
>>>>> On Thu, May 30, 2019 at 2:18 AM Felix Cheung <
>>>>> felixcheun...@hotmail.com> wrote:
>>>>>
>>>>>> We don’t usually reference a future release on website
>>>>>>
>>>>>> > Spark websit
mins bring
everything back up on sunday, august 4th. if they run in to issues, i will
jump in first thing monday, august 5th.
as the time approaches, i will send reminders and updates.
thanks,
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
scuss before
>>>> making a change. Does anyone else have thoughts on this?
>>>>
>>>> Regards,
>>>> Bryan
>>>>
>>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
year ago, 0.24.0 in
> January and 0.24.2 in March.
>
> On Fri, Jun 14, 2019 at 9:27 AM shane knapp wrote:
>
>> just to everyone knows, our python 3.6 testing infra is currently on
>> 0.24.2...
>>
>> On Fri, Jun 14, 2019 at 9:16 AM Dongjoon Hyun
>> wrote:
>
ative, so it's pretty unlikely that we
> will add something that would break 0.23.X.
>
> On Fri, Jun 14, 2019 at 10:10 AM shane knapp wrote:
>
>> ah, ok... should we downgrade the testing env on jenkins then? any
>> specific version?
>>
>> shane, who
On Fri, Jun 14, 2019 at 11:36 AM Felix Cheung
> wrote:
>
>> How about pyArrow?
>>
>> --
>> *From:* Holden Karau
>> *Sent:* Friday, June 14, 2019 11:06:15 AM
>> *To:* Felix Cheung
>> *Cc:* Bryan Cutler; Dongjoon
i will do it later this week. also, in the future, please file jiras for
stuff like this rather than pinging me on the list. ;)
On Wed, Jun 19, 2019 at 1:39 PM Xiao Li wrote:
> That sounds good to me!
>
> @shane knapp Could you help this? Or Dongjoon can
> do it by himself sinc
i'm aware and will be looking in to this later today.
see:
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/6043/console
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
ah, ok. thanks for letting me know. :)
On Mon, Jun 24, 2019 at 9:39 AM Sean Owen wrote:
> (We have two PRs to patch it up anyway already)
>
> On Mon, Jun 24, 2019 at 11:39 AM shane knapp wrote:
> >
> > i'm aware and will be looking in to this later today
>>> aarch64 server, so it will be better to have the ARM CI to make sure the
>>> spark is compatible for AArch64 platforms.
>>> >>
>>> >> I’m from OpenLab team(https://openlabtesting.org/ ,a community to do
>>> open source project testing. And
...or via VM as you mentioned earlier. :)
shane (who will file a JIRA tomorrow)
On Tue, Jun 25, 2019 at 6:44 PM shane knapp wrote:
> i'd much prefer that we keep the test/build infrastructure in one place.
>
> we don't have ARM hardware, but there's a slim possibility
is a lot of builds.
as a temporary work-around, i will see if i can find a cached version of
pycodestyle.py somewhere and put that in each worker's PATH.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
ok, it seems to be back now, but github is experiencing major outages right
now:
https://www.githubstatus.com/
shane
On Mon, Jul 22, 2019 at 9:01 AM shane knapp wrote:
> hey everyone!
>
> i was debugging some SSL cert issues this morning, and noticed that once i
> got that resol
On Fri, Jun 14, 2019 at 9:13 AM shane knapp wrote:
> the campus colo will be performing some electrical maintenance, which
> means that they'll be powering off the entire building.
>
> since the jenkins cluster is located in that colo, we are most definitely
> affected. :
here's the latest timetable:
* all machines powered off some time tomorrow (friday) night ~9pm
* sunday morning, all machines will be powered back up
* if any stragglers fail to come back, we will investigate monday morning
On Tue, Jul 30, 2019 at 11:30 AM shane knapp wrote:
> On Fri
o could any CI maintainers help/tell us for how to split them and
>> the details about different CI jobs does? Such as PR title contains [SQL],
>> [INFRA], [ML], [DOC], [CORE], [PYTHON], [k8s], [DSTREAMS], [MLlib],
>> [SCHEDULER], [SS],[YARN], [BUIILD] and etc..I found each of them
R I'm not able to see
> if "Test build" is being processed. But I can see this after a few minutes
> or hours later. Is it possible to see if "Test Build" is being processed
> after PR is created right away?
>
> Thank you,
> Younggyu Chun
>
--
Shane Kn
r the inconvenience...
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
quick update:
it's been 4 hours, the colo is still down, and i haven't gotten any news
yet as to when they're planning on getting power restored.
once i hear something i will let everyone know what's up.
On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp wrote:
> the berkeley
5, 2019 at 2:07 PM Shane Knapp wrote:
> quick update:
>
> it's been 4 hours, the colo is still down, and i haven't gotten any news
> yet as to when they're planning on getting power restored.
>
> once i hear something i will let everyone know what's up.
&g
it's back up! some of the workers didn't come back cleanly, so i'll have
to hit up the colo tomorrow and persuade them in person.
On Thu, Aug 15, 2019 at 6:45 PM Wenchen Fan wrote:
> Thanks for tracking it Shane!
>
> On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp wrote:
a couple of workers needed a bit more time to finish booting up, so no need
for my excursion tomorrow. :)
builds be building, things look happy.
On Thu, Aug 15, 2019 at 6:46 PM Shane Knapp wrote:
> it's back up! some of the workers didn't come back cleanly, so i'll have
&g
it's that time of the year again when jenkins decides to take it's time
serving pages.
i will post any updates/etc when i get them.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
i kicked apache, and the number of active connections dropped from ~500 to
~100... things are much snappier now.
On Fri, Aug 16, 2019 at 1:05 PM Shane Knapp wrote:
> it's that time of the year again when jenkins decides to take it's time
> serving pages.
>
> i will post
anks to your many many contributions,
>>>>>>> Apache Spark master branch starts to pass on JDK11 as of today.
>>>>>>> (with `hadoop-3.2` profile: Apache Hadoop 3.2 and Hive 2.3.6)
>>>>>>>
>>>>>>>
>>>>>>> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-3.2-jdk-11/326/
>>>>>>> (JDK11 is used for building and testing.)
>>>>>>>
>>>>>>> We already verified all UTs (including PySpark/SparkR) before.
>>>>>>>
>>>>>>> Please feel free to use JDK11 in order to build/test/run `master`
>>>>>>> branch and
>>>>>>> share your experience including any issues. It will help Apache
>>>>>>> Spark 3.0.0 release.
>>>>>>>
>>>>>>> For the follow-ups, please follow
>>>>>>> https://issues.apache.org/jira/browse/SPARK-24417 .
>>>>>>> The next step is `how to support JDK8/JDK11 together in a single
>>>>>>> artifact`.
>>>>>>>
>>>>>>> Bests,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> [image: Databricks Summit - Watch the talks]
>>>>> <https://databricks.com/sparkaisummit/north-america>
>>>>>
>>>>
>>>
>>> --
>>> ---
>>> Takeshi Yamamuro
>>>
>>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
w to
> the new ASF repo.
>
> Report any problems here!
>
> Sean
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
JDK 11 at least, and why we could
> probably collapse some too.
>
> Comments welcome on the JIRA, as I'm sure I'm missing a thing or two.
i will definitely be adding my thoughts, but i most likely won't be
able to get to this until after the labor day holiday (i'm busy
writing
l the best,
>
> Matei and the Spark PMC
>
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
EOM
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
it's back up and building.
On Mon, Sep 16, 2019 at 11:02 AM Shane Knapp wrote:
>
> EOM
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
--
Shane Knapp
UC Berkeley EECS Research / RISELab
to reply late, we had a
> Mid-Autumn holiday:)
>
> If you hope to integrate ARM CI to amplab jenkins, we can offer the arm
> instance, and then the ARM job will run together with other x86 jobs, so
> maybe there is a guideline to do this? @shane knapp
> would you help us?
>
>
atsoever. ¯\_(ツ)_/¯
what i CAN do first thing tomorrow is pause builds, wipe out the
ivy/sbt caches and SparkPullRequestBuilder* dirs on all workers and
see if that helps.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technica
whether it helps.
>
> On Tue, Sep 17, 2019 at 1:08 PM Shane Knapp wrote:
>>
>> > Note that they're different workers (amp-jenkins-worker-0x) with same
>> > workspace /home/jenkins/workspace/SparkPullRequestBuilder@6.
>> >
>> the '@6' re
her builds that i manually cancel.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
this is done and jenkins is building again!
On Tue, Sep 17, 2019 at 10:14 AM Shane Knapp wrote:
>
> i'm going to clean up the spark workspaces on the jenkins workers and
> clear out ivy and maven caches.
>
> this means no new builds will be started as of right now, and curren
the terminal font setup, but there are no erroneous control
characters popping up, and -e shows a $ at EOL)
i'm confused that this is happening. anyone have any ideas?
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berk
t; That's super weird; can you just delete ~/.m2 and let it download the
> internet again? or at least blow away the downloaded Kafka dir?
> Turning it on and off, so to speak, often works.
>
> On Tue, Sep 17, 2019 at 2:41 PM Shane Knapp wrote:
> >
> > a bunch of t
-- not good!
i rmed that file and hopefully it will repopulate w/o issue. if not,
i'll kill builds on that worker and wipe all local caches. again.
On Tue, Sep 17, 2019 at 12:53 PM Shane Knapp wrote:
>
> that's what i literally just did! i wiped the .m2, .ivy and
> per-exec
> ah, i found this sucker on amp-jenkins-worker-02:
s/02/06
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
in to work on
monday.
i apologize for any inconvenience that this will cause... and i hope
that this is the final piece of the power puzzle that our colo team
has to deal with.
thanks in advance,
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
htt
power switchover is happening now. more updates to come once machines
come back up.
On Mon, Sep 23, 2019 at 3:16 PM Shane Knapp wrote:
>
> the main transformer for our colo is experiencing major issues, and
> campus be performing emergency work on it starting tomorrow morning
> (tue
aand that was quick! everything is back up and building
On Tue, Sep 24, 2019 at 9:39 AM Shane Knapp wrote:
>
> power switchover is happening now. more updates to come once machines
> come back up.
>
> On Mon, Sep 23, 2019 at 3:16 PM Shane Knapp wrote:
> >
> > t
hanks,
shane
On Tue, Sep 24, 2019 at 9:50 AM Shane Knapp wrote:
>
> aand that was quick! everything is back up and building
>
> On Tue, Sep 24, 2019 at 9:39 AM Shane Knapp wrote:
> >
> > power switchover is happening now. more updates to come once machines
> >
.
>
>
> Could you please check on this?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
>
> On Wed, Sep 25, 2019 at 2:48 AM Shane Knapp wrote:
>>
>> quick update from our colo admin: they are going to keep the colo on
>> generator power until monday morning and
9 at 6:07 AM Jungtaek Lim wrote:
>>
>> Awesome, thanks for the quick update!
>>
>> On Wed, Sep 25, 2019 at 6:04 AM Shane Knapp wrote:
>>>
>>> no worries. since we deprecated the packaging builds i put that
>>> worker back in to the rotation...
it was wedged and i had to perform a quick restart. sorry about the
interruption of service!
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
-
To unsubscribe e-mail
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-maven-snapshots/
https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-2.4-maven-snapshots/
i created dry-run test builds and everything looked great. please
file a JIRA is anything published by these jobs looks fishy.
shane
--
Shane
nt of time
for these builds to run... do we need to increase it? can
someone(s) here figure out what's taking so long and refactor some of
the tests?
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.be
just chatted w/sean privately and i'm going to up the test timeouts to
480mins (8 hours).
i still don't like this but at least it should hopefully get things green again.
On Mon, Oct 7, 2019 at 11:31 AM Shane Knapp wrote:
>
> https://amplab.cs.berkeley.edu/jenkins/job/spark-m
;t think that they needed load balancers. :\
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
service will be restored (i HOPE) by friday morning.
shane (who doesn't ever want to check this list's archives and count
how many times we've had power issues)
On Tue, Oct 8, 2019 at 12:50 PM Shane Knapp wrote:
>
> here in the lovely bay area, we are currently experiencing som
jenkins is going down now.
On Tue, Oct 8, 2019 at 4:21 PM Shane Knapp wrote:
>
> quick update:
>
> we are definitely going to have our power shut off starting early
> tomorrow morning (by 4am PDT oct 9th), and expect at least 48 hours
> before it is restored.
>
> i will
vers including
jenkins) until monday.
more updates as they come. thanks for your patience!
On Tue, Oct 8, 2019 at 7:32 PM Shane Knapp wrote:
>
> jenkins is going down now.
>
> On Tue, Oct 8, 2019 at 4:21 PM Shane Knapp wrote:
> >
> > quick update:
> >
> >
another quick update:
campus lost power ~1130pm, and is closed for the entirety of today.
no word on power restoration, campus status, etc etc.
updates as they come. :\
On Wed, Oct 9, 2019 at 2:34 PM Shane Knapp wrote:
>
> quick update:
>
> campus is losing power @ 8pm. this
enkins to come back up, manually run things ourselves and commit?
>>>>
>>>> Tom
>>>>
>>>> -
>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>>
&g
ust run tests locally and it’ll be a
>> slower process but I don’t think we need to halt all merging.
>>
>> On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp wrote:
>>>
>>> if we do get power back before the weekend, i can have my sysadmin
>>> head down to
上午9:11写道：
>>>
>>> Awesome, thanks Shane :)
>>>
>>> In the meantime I think committers can just run tests locally and it’ll be
>>> a slower process but I don’t think we need to halt all merging.
>>>
>>> On Thu, Oct 10, 2019 at 9:07 AM
this means any open PRs will need to be retriggered.
please don't do this until *after* you get the notice that jenkins is
up.
thanks for your patience,
shane
On Thu, Oct 10, 2019 at 9:05 AM Shane Knapp wrote:
>
> another quick update:
>
> campus lost power ~1130pm, and is clos
thanks jon!
On Fri, Oct 11, 2019 at 3:25 PM Jon Kuroda wrote:
>
> Jenkins is back up and building. Build queue purged, builders are building.
> Fiat Lux --Jon
>
> On Fri, Oct 11, 2019 at 1:58 PM Shane Knapp wrote:
>>
>> finally, some good news! power was just re
Specifically, can we start to deprecate JDK8u81 and older at 3.0.
>
> Currently, Apache Spark testing infra are testing only with jdk1.8.0_191 and
> above.
>
> Bests,
> Dongjoon.
--
Shane Knapp
UC Berkeley EECS Research / RISELab St
eally have any more information for either of these events.
the situation regarding the power shutoff is fluid and changing, and
depending on the weather and winds, could potentially not happen at
all, or impact us until monday.
once i hear more i will update this message.
shane
--
Shane Knapp
UC
ds, so if your PR or
'retest this please' comment doesn't seem to catch and trigger a build
after ~15 mins, feel free (if you're whitelisted or an admin) to
request another test with 'test this please'.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab
ll of the workers will shut down
> and power back on automatically when power is restored.
>
ok, it looks like the colo will have power until monday morning, and
it will be shut down from 8am to noon to perform some maintenance.
this means jenkins will be up all weekend, but down monday mornin
it to return in
~3.5 hours.
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
i will need to restart jenkins -- the worker's ENV vars got borked when
they came back up.
this is happening NOW.
shane
On Mon, Oct 28, 2019 at 10:37 AM Shane Knapp wrote:
> we're back up and building!
>
> On Mon, Oct 28, 2019 at 8:35 AM Shane Knapp wrote:
>
>>
we're back up and building!
On Mon, Oct 28, 2019 at 8:35 AM Shane Knapp wrote:
> ok, it looks like the colo will have power until monday morning, and
>> it will be shut down from 8am to noon to perform some maintenance.
>>
>> this means jenkins will be up all weeken
n.
>>>>
>>>> Specifically:
>>>>
>>>>- Python 3.4 has been retired this year.
>>>> - Python 3.5 is already in the "security fixes only" mode and
>>>>should be retired in the middle of 2020.
>>>>
>>>> Continued support of these two blocks adoption of many new Python
>>>> features (PEP 468) and it is hard to justify beyond 2020.
>>>>
>>>> Should these two be deprecated in 3.0.0 as well?
>>>>
>>>> --
>>>> Best regards,
>>>> Maciej
>>>>
>>>>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
2019 at 5:45 AM Takeshi Yamamuro
> wrote:
>
>> +1, too.
>>
>> On Tue, Oct 29, 2019 at 4:16 PM Holden Karau
>> wrote:
>>
>>> +1 to deprecating but not yet removing support for 3.6
>>>
>>> On Tue, Oct 29, 2019 at 3:47 AM Shane Knapp w
1 - 100 of 774 matches
Mail list logo
