Jenkins upgraded

2024-04-04 Thread Luca Toscano
Hi folks! I have just upgraded Jenkins to 2.452, everything seems to work fine but please let me know if you see anything out of the ordinary. The upgrade was due to some security issues, and I also took the chance to upgrade some plugins for the same reason. Luca

Re: [Discuss] Bigtop release 3.3

2024-03-21 Thread Luca Toscano
Hi Masatake, Not sure if you saw https://issues.apache.org/jira/browse/BIGTOP-4058 but we could take some time to check also the experimental smoke test pipelines for branch-3.3, lemme know what you think :) Luca On Tue, Mar 19, 2024 at 6:50 AM Masatake Iwasaki wrote: > > I'm updating CI jobs

Re: Installed plugin for pipelines

2024-02-27 Thread Luca Toscano
On Mon, Jan 15, 2024 at 12:06 PM Luca Toscano wrote: > > As promised: https://issues.apache.org/jira/browse/BIGTOP-4058 > > I started adding some jobs to show the idea, feedback welcome :) > > Luca > > On Thu, Jan 11, 2024 at 3:56 PM Luca Toscano wrote: > > >

Re: Installed plugin for pipelines

2024-01-15 Thread Luca Toscano
As promised: https://issues.apache.org/jira/browse/BIGTOP-4058 I started adding some jobs to show the idea, feedback welcome :) Luca On Thu, Jan 11, 2024 at 3:56 PM Luca Toscano wrote: > > Nevermind I fixed the apt issue, but the current problem is that the > COMPONENTS of the s

[jira] [Created] (BIGTOP-4058) Replace Jenkins' Matrix reloaded plugin

2024-01-14 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-4058: Summary: Replace Jenkins' Matrix reloaded plugin Key: BIGTOP-4058 URL: https://issues.apache.org/jira/browse/BIGTOP-4058 Project: Bigtop Issue Type

Re: Installed plugin for pipelines

2024-01-11 Thread Luca Toscano
a task and work on it, hopefully there is an easy solution :) On Thu, Jan 11, 2024 at 12:11 PM Luca Toscano wrote: > > I have also created: https://ci.bigtop.apache.org/job/pipeline-smoke-test/ > > It currently doesn't work since there seems to be an error when apt > runs in

Re: Installed plugin for pipelines

2024-01-11 Thread Luca Toscano
On Wed, Jan 10, 2024 at 9:44 PM Luca Toscano wrote: > > Hi folks, > > I created https://ci.bigtop.apache.org/job/test-elukey as an example > of how Jenkins pipelines can be used to replace the Matrix Reloaded > plugin. The prototype is limited for the moment to: > - It uses scri

Re: Installed plugin for pipelines

2024-01-10 Thread Luca Toscano
m. Do you think that this process is viable? If so we could use pipelines as experimental step during the next release, to see if they can be adopted. Let me know your thoughts and if I am missing something big (probably happening, apologies in advance). Thanks! Luca On Sun, Jan 7, 2024 at 3:

Installed plugin for pipelines

2024-01-07 Thread Luca Toscano
Hi folks, just letting you know that I have installed the Pipelines plugin in the Jenkins' master, to test if we can define our jobs in there and avoid the Matrix reloaded plugin. It would be also very nice to have all pipelines checked out in the BigTop's repo eventually. I'll open a jira to

[jira] [Created] (BIGTOP-4041) Move httpd on ci.bigtop.apache.org to mpm worker/event

2023-11-10 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-4041: Summary: Move httpd on ci.bigtop.apache.org to mpm worker/event Key: BIGTOP-4041 URL: https://issues.apache.org/jira/browse/BIGTOP-4041 Project: Bigtop

Re: Ongoing maintenance for ci.bigtop.apache.org

2023-11-09 Thread Luca Toscano
On Thu, Nov 9, 2023 at 5:20 PM Luca Toscano wrote: > > Hi folks, > > working on https://issues.apache.org/jira/browse/BIGTOP-4038, so > ci.bigtop.apache.org may not be working for a couple of hours. > > Let me know if this is a problem or not. > > Thanks! > > Luca

Ongoing maintenance for ci.bigtop.apache.org

2023-11-09 Thread Luca Toscano
Hi folks, working on https://issues.apache.org/jira/browse/BIGTOP-4038, so ci.bigtop.apache.org may not be working for a couple of hours. Let me know if this is a problem or not. Thanks! Luca

Replace certbot with mod_md for ci.bigtop.apache.org

2023-11-06 Thread Luca Toscano
Hi folks! I added an idea in https://issues.apache.org/jira/browse/BIGTOP-4038 about how to automate the let's-encrypt cert renewal for ci.bigtop.apache.org. Let me know what you think and if you like the idea! Luca

Re: Issues for ci.bigtop.apache.org

2023-11-06 Thread Luca Toscano
Issue fixed! Apologies for the trouble :) The Jenkins master node has been upgraded to the latest kernel and packages, I'll do the same during the next few days with the worker nodes. Luca On Sun, Nov 5, 2023 at 9:24 AM Luca Toscano wrote: > > Created https://issues.apache.org/jira/

Re: Issues for ci.bigtop.apache.org

2023-11-05 Thread Luca Toscano
Created https://issues.apache.org/jira/browse/BIGTOP-4039 to track the issue, still unable to make Jenkins start :( Luca On Sat, Nov 4, 2023 at 7:39 PM Luca Toscano wrote: > > Hi folks, > > I am working on the host behind ci.bigtop.apache.org, after a reboot + > os package

[jira] [Created] (BIGTOP-4039) Jenkins not starting for ci.bigtop.apache.org after reboot

2023-11-05 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-4039: Summary: Jenkins not starting for ci.bigtop.apache.org after reboot Key: BIGTOP-4039 URL: https://issues.apache.org/jira/browse/BIGTOP-4039 Project: Bigtop

Issues for ci.bigtop.apache.org

2023-11-04 Thread Luca Toscano
Hi folks, I am working on the host behind ci.bigtop.apache.org, after a reboot + os package upgrade Jenkins refuses to start. I'll keep this list posted, sorry for the inconvenience! Luca

[jira] [Created] (BIGTOP-4038) Replace certbot with apache httpd's mod_md

2023-11-04 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-4038: Summary: Replace certbot with apache httpd's mod_md Key: BIGTOP-4038 URL: https://issues.apache.org/jira/browse/BIGTOP-4038 Project: Bigtop Issue Type

Jenkins upgraded for security fixes

2023-10-22 Thread Luca Toscano
Hi everybody, Jenkins running behind ci.bigtop.apache.org has been upgraded to apply security fixes. If you see anything strange in jobs during the next few days please let me know. Cheers, Luca

Re: [ANNOUNCE] New Bigtop PMC member: Luca Toscano

2023-10-04 Thread Luca Toscano
Thanks a lot for all the messages, I really appreciated the warm welcome :) Luca On Tue, Sep 26, 2023, 19:48 Olaf Flebbe wrote: > On behalf of the Apache Bigtop PMC, I am pleased to announce that > Luca Toscano (elukey) has accepted the invitation to join the > Bigtop Project M

Re: Arm workers need to be updated

2023-06-28 Thread Luca Toscano
It seems that the two Arm nodes are working properly: > > https://ci.bigtop.apache.org/computer/docker-slave-arm-4/log > https://ci.bigtop.apache.org/computer/docker-slave-arm-5/log > > Thanks. > > BRs, > Yuqi > > > > Luca Toscano 于2023年6月27日周二 18:05写道: > >

Re: Arm workers need to be updated

2023-06-27 Thread Luca Toscano
VA_HOME was mistakenly set to /home/jenkins/jdk/. > Could you please try to "source /home/jenkins/.bashrc" before launching the > agent? > > BRs, > Yuqi > > Luca Toscano 于2023年6月27日周二 15:06写道: > > > Hi Yuqi! > > > > I tried to re-launch the

Renewed TLS cert for ci.bigtop.apache.org

2023-06-27 Thread Luca Toscano
Hi folks, I followed https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide#BigtopCISetupGuide-Renewingthecert and renewed the certificate since it expired. Luca

Re: Arm workers need to be updated

2023-06-27 Thread Luca Toscano
Yuqi > > On Sun, 25 Jun 2023 at 19:00, Yuqi Gu wrote: > > > Hi Luca, > > > > Sorry for the late reply. > > Let me try to set the default JVM version to 11 as you mentioned. > > I will ping you once JDK 11 has been properly configured on the Arm nodes. >

Re: Arm workers need to be updated

2023-06-25 Thread Luca Toscano
Hi folks! Sorry to ping you again, but the ARM workers are still down. Anybody with access credentials that can fix them? Thanks :) Luca On Fri, Jun 16, 2023 at 5:50 PM Luca Toscano wrote: > > Hi folks, > > sending an email in here for visibility. After the Jenkins upg

Arm workers need to be updated

2023-06-16 Thread Luca Toscano
Hi folks, sending an email in here for visibility. After the Jenkins upgrade I see the following errors on the Arm workers: Exception in thread "main" java.lang.UnsupportedClassVersionError: hudson/remoting/Launcher has been compiled by a more recent version of the Java Runtime (class file

Re: [DISCUSSION] Propose to remove Sqoop

2022-08-11 Thread Luca Toscano
+1! Maybe in the future Gobblin could be a valid alternative in Bigtop (IIRC it recently graduated as a full Apache project). The Wikimedia foundation replaced sqoop with Gobblin and so far we are very happy about the move. Luca On Mon, Aug 8, 2022 at 11:39 AM Yuqi Gu wrote: > > Hi folks, > >

Re: Preparation status of the 3.0.1 release

2022-03-08 Thread Luca Toscano
Seki wrote: > > Thanks a lot, Luca! I'll start the release builds after testing and merging > BIGTOP-3649. > (I may need a few days to set up an EC2 instance for Nexus and configure CI > jobs to leverage it after merging) > > Kengo Seki apache.org> > > > On Mon, Mar

Re: Preparation status of the 3.0.1 release

2022-03-07 Thread Luca Toscano
Hi everybody, Infra answered and unblocked the IPs. More details in https://issues.apache.org/jira/browse/INFRA-22953 Luca On Thu, Mar 3, 2022 at 11:36 AM Luca Toscano wrote: > > Hi everybody, > > Still no answers from Infra, I sent another email this morning, I'll > update t

[jira] [Created] (BIGTOP-3650) Modernize deb packages

2022-03-07 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3650: Summary: Modernize deb packages Key: BIGTOP-3650 URL: https://issues.apache.org/jira/browse/BIGTOP-3650 Project: Bigtop Issue Type: Improvement Affects

Re: Preparation status of the 3.0.1 release

2022-03-03 Thread Luca Toscano
Hi everybody, Still no answers from Infra, I sent another email this morning, I'll update the list as soon as I receive something :) Luca On Tue, Mar 1, 2022 at 2:25 PM Kengo Seki wrote: > > Hi everyone, > > As discussed in [1], I'd like to publish the 3.0.1 release based on > branch-3.0 soon,

Re: Release during the next weeks?

2022-02-22 Thread Luca Toscano
606 be in 3.1, it is not yet resolved. > > After Bigtop 3.0.1, we can release Bigtop 3.1.0 even without BIGTOP-3606. > > If it takes more time in Hadoop side, we can address the Hadoop 3.2.3 in > > Bigtop 3.1.1. > > > > Thanks, > > Masatake Iwasaki > >

Release during the next weeks?

2022-02-21 Thread Luca Toscano
Hi everybody, I am wondering if it would be good to release 3.1 during the coming weeks to address the log4j vulnerabilities. Thoughts? No idea how tough the process is, but I'll help if needed! Luca

Re: Followup on yesterday

2022-02-20 Thread Luca Toscano
ctly. > > However this will only work for maven, not handling gradle or ivy builds > AFAIK. > > Best > Olaf > > > > Am 19.02.2022 um 14:23 schrieb Luca Toscano : > > > > Hi everybody, > > > > I have one doubt related to Nexus, namely when it is used.

[jira] [Created] (BIGTOP-3644) Improve dowload artifact resilency in Gradle/Maven/etc..

2022-02-20 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3644: Summary: Improve dowload artifact resilency in Gradle/Maven/etc.. Key: BIGTOP-3644 URL: https://issues.apache.org/jira/browse/BIGTOP-3644 Project: Bigtop

Re: Followup on yesterday

2022-02-19 Thread Luca Toscano
-11/lastBuild/consoleFull On Thu, Feb 17, 2022 at 9:48 PM Olaf Flebbe wrote: > > Hi everyone, > > Yesterday Luca Toscano and me had a call to look into improving the situation > of artifact downloads by caching . > > I was a bit surprised that the „nexus“ code is still in

Re: Jenkins upgrade scheduled for Feb 18th 9:00 CET

2022-02-18 Thread Luca Toscano
Upgraded Jenkins to 2.334 and upgraded the Pipeline Groovy plugin (both for security issues). Everything looks good, lemme know if you notice anything strange! Luca On Thu, Feb 17, 2022 at 9:48 AM Luca Toscano wrote: > > Hi everybody, > > I am planning to upgrade Jenkins tomorr

Jenkins upgrade scheduled for Feb 18th 9:00 CET

2022-02-17 Thread Luca Toscano
Hi everybody, I am planning to upgrade Jenkins tomorrow (Feb 18th) at around 9:00 CET if nobody opposes it. I will stop the current package builds (mostly trunk + old backlog queue when the ppc node was down), since they will be restarted on Saturday (part of the weekly builds). Let me know if

Re: Network failures while maven runs

2022-02-01 Thread Luca Toscano
suggestions :) Thanks, Luca On Mon, Jan 31, 2022 at 2:21 PM Konstantin Boudnik wrote: > > Hey Luca. > > Did you notice if this is happening with some specific repo server? > > Tahnks! > Cos > > On Sat, Jan 29, 2022 at 08:31AM, Luca Toscano wrote: > > Hi ever

Re: Network failures while maven runs

2022-02-01 Thread Luca Toscano
Hi Olaf, On Sat, Jan 29, 2022 at 9:18 AM Olaf Flebbe wrote: > > Hi Luca, > > I think you are referring to random failures of maven downloads? Exactly yes, maven download tasks and also random connect timeouts when downloading poms. > > My next proposal would be to use docker volumes (not bind

Re: docker worker ppc-2 down

2022-01-28 Thread Luca Toscano
Hi Amir, Thanks a lot, everything works now! Luca On Fri, Jan 28, 2022 at 7:16 PM MrAsanjar wrote: > > Hi team, > it is running now. Please next time email me directly to asan...@apache.com > or amir.san...@ibm.com > > On Fri, Jan 28, 2022 at 4:32 AM Luca Toscano wrote:

Network failures while maven runs

2022-01-28 Thread Luca Toscano
Hi everybody, I have been seeing build failures for trunk packages due to maven network failures (mostly connection timeouts). Is there a workaround for this? For example retrying x times etc.. I am asking mostly because of ignorance, I am wondering what we do when cutting a release for example

[jira] [Created] (BIGTOP-3636) Remove slave terminology from the project

2022-01-28 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3636: Summary: Remove slave terminology from the project Key: BIGTOP-3636 URL: https://issues.apache.org/jira/browse/BIGTOP-3636 Project: Bigtop Issue Type

docker worker ppc-2 down

2022-01-28 Thread Luca Toscano
Hi everybody, In a previous email I mentioned the fact that the docker ppc-2 worker has been down for a while. It was mentioned that Amir is the point of contact for the node, so I am sending a more visible email seeking for help :) Luca

Re: Docker arm-5 node not reachable by Jenkins

2022-01-23 Thread Luca Toscano
a, > > > > Amir is taking care of the PowerPC server. > > It still seems to be unavailable. Would you take a look, Amir? > > > > Kengo Seki > > > > On Sat, Jan 15, 2022 at 7:39 PM Luca Toscano > > wrote: > > > > > > https://ci.bigtop

[jira] [Created] (BIGTOP-3633) Update build.gradle and our docs

2022-01-22 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3633: Summary: Update build.gradle and our docs Key: BIGTOP-3633 URL: https://issues.apache.org/jira/browse/BIGTOP-3633 Project: Bigtop Issue Type: Improvement

Re: ci.bigtop.apache.org currently under upgrade

2022-01-17 Thread Luca Toscano
Jenkins up and running, there are a couple of things left to do after the upgrade but nothing blocking (more details in the task). Luca On Mon, Jan 17, 2022 at 5:56 PM Luca Toscano wrote: > > Hi everybody, > > I reached out to Olaf and Kengo to upgrade Jenkins and

ci.bigtop.apache.org currently under upgrade

2022-01-17 Thread Luca Toscano
Hi everybody, I reached out to Olaf and Kengo to upgrade Jenkins and some plugins, and then executed the plan (https://issues.apache.org/jira/browse/BIGTOP-3611) some minutes ago. There is a problem with a plugging (sshslave afaics) that is not able to launch the agent on the various workers. It

Re: Docker arm-5 node not reachable by Jenkins

2022-01-15 Thread Luca Toscano
https://ci.bigtop.apache.org/computer/docker%2Dslave%2Dppc%2D2/ seems down as well, not sure who to contact :) Marking both nodes as temporarily down in Jenkins.. Luca On Sat, Jan 15, 2022 at 11:37 AM Luca Toscano wrote: > > Hi folks, > > I see again the same proble

Re: Docker arm-5 node not reachable by Jenkins

2022-01-15 Thread Luca Toscano
Hi folks, I see again the same problem in https://ci.bigtop.apache.org/computer/docker-slave-arm-5/ :( Luca On Wed, Jan 5, 2022 at 10:14 AM Luca Toscano wrote: > > Hi to both, > > Thanks a lot for the fix, I see the host up and running now :) > > Luca > > On Wed, Jan 5,

Re: Broken package builds for Trunk

2022-01-14 Thread Luca Toscano
Corretto 17 is used (that is IIUC the root cause of the problem). I am going to try it during the next few hours if nobody opposes it. If you think that there is a different root cause please let me know :) Luca On Tue, Jan 11, 2022 at 9:07 AM Luca Toscano wrote: > > Hi everybody, > &g

[jira] [Created] (BIGTOP-3631) Gradlew failing while starting on new docker workers

2022-01-14 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3631: Summary: Gradlew failing while starting on new docker workers Key: BIGTOP-3631 URL: https://issues.apache.org/jira/browse/BIGTOP-3631 Project: Bigtop Issue

[jira] [Created] (BIGTOP-3630) Some repositories yield to HTTP 403

2022-01-11 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3630: Summary: Some repositories yield to HTTP 403 Key: BIGTOP-3630 URL: https://issues.apache.org/jira/browse/BIGTOP-3630 Project: Bigtop Issue Type: Bug

Broken package builds for Trunk

2022-01-11 Thread Luca Toscano
Hi everybody, I checked the trunk packages build matrix and I noticed that a lot of packages are failing to build. An example is: https://ci.bigtop.apache.org/job/Bigtop-trunk-packages/762/COMPONENTS=kibana,OS=debian-10/console [...] Using 4 worker leases. Starting Build

[jira] [Created] (BIGTOP-3629) Drop Debian 9 support - second attempt

2022-01-08 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3629: Summary: Drop Debian 9 support - second attempt Key: BIGTOP-3629 URL: https://issues.apache.org/jira/browse/BIGTOP-3629 Project: Bigtop Issue Type

Changes to the Docker provisioner

2022-01-08 Thread Luca Toscano
Hi everybody, I filed a code review for the Docker provisioner in https://github.com/apache/bigtop/pull/851, Masatake has already reviewed it but since a lot of people use it I wanted to bring it to your attention. The main issue that I found is that on recent OSes like Debian 11, mounting

Re: Docker arm-5 node not reachable by Jenkins

2022-01-05 Thread Luca Toscano
gt; Yuqi > > Jun HE 于2022年1月4日周二 09:43写道: > > > Hi Luca, > > > > Sorry for the late response. Yuqi and I are responsible for these Arm CI > > nodes. I'll check with this and update you later. > > > > Regards, > > > > Jun > >

[jira] [Created] (BIGTOP-3626) Patch log4j version of ycsb

2021-12-28 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3626: Summary: Patch log4j version of ycsb Key: BIGTOP-3626 URL: https://issues.apache.org/jira/browse/BIGTOP-3626 Project: Bigtop Issue Type: Improvement

[jira] [Created] (BIGTOP-3625) Fix Livy's build failure

2021-12-28 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3625: Summary: Fix Livy's build failure Key: BIGTOP-3625 URL: https://issues.apache.org/jira/browse/BIGTOP-3625 Project: Bigtop Issue Type: Bug Affects

Docker arm-5 node not reachable by Jenkins

2021-12-28 Thread Luca Toscano
Hi everybody, I've set https://ci.bigtop.apache.org/computer/docker-slave-arm-5/ temporarily offline in the Jenkins UI, it seems not reachable via ssh from the Jenkin's perspective. What is the procedure to follow for these nodes? Luca

[jira] [Created] (BIGTOP-3624) Bump Alluxio's log4j dependencies to 2.17.0

2021-12-22 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3624: Summary: Bump Alluxio's log4j dependencies to 2.17.0 Key: BIGTOP-3624 URL: https://issues.apache.org/jira/browse/BIGTOP-3624 Project: Bigtop Issue Type

Re: Some ideas about Jenkins

2021-12-20 Thread Luca Toscano
Hi everybody, cleanups done, the master is now using ~240G of disk space (24% of the root partition). Luca On Sun, Dec 19, 2021 at 9:16 AM Luca Toscano wrote: > > Hi Kengo, > > Thanks for the detailed explanation! I'll try to do some cleanup > today/tomorrow, I am 100% in favor

[jira] [Created] (BIGTOP-3622) Remove usage of $(PWD) in Debian rules files

2021-12-20 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3622: Summary: Remove usage of $(PWD) in Debian rules files Key: BIGTOP-3622 URL: https://issues.apache.org/jira/browse/BIGTOP-3622 Project: Bigtop Issue Type

[jira] [Created] (BIGTOP-3621) Review Oozie 4.x and 5.x configs for CVE-2021-44228

2021-12-20 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3621: Summary: Review Oozie 4.x and 5.x configs for CVE-2021-44228 Key: BIGTOP-3621 URL: https://issues.apache.org/jira/browse/BIGTOP-3621 Project: Bigtop Issue

Re: Some ideas about Jenkins

2021-12-19 Thread Luca Toscano
8/h. > If we replace it with m6a.xlarge (4vCPUs, 16GiB mem) in the new > account, it costs only $0.1728/h, so we can save $97.344/mo. > The price of gp3 EBS volume is $0.08/mo for 1GB, so I'm estimating we > can add 1.2TB extra capacity to the master node. > > Kengo Seki > > On Sat

Re: ci.bigtop.apache.org's TLS cert expired

2021-12-18 Thread Luca Toscano
hat document is up-to-date and should work ;) > (Only docker container name may be different, IIRC) > > Kengo Seki > > On Sat, Dec 18, 2021 at 7:12 PM Luca Toscano wrote: > > > > Hi everybody, > > > > the TLS cert for ci.bigtop.apache.org is expired, I found

ci.bigtop.apache.org's TLS cert expired

2021-12-18 Thread Luca Toscano
Hi everybody, the TLS cert for ci.bigtop.apache.org is expired, I found https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide#BigtopCISetupGuide-Renewingthecert but before executing it I'd like to get some confirmation that the guide is up-to-date :) Luca

Re: Some ideas about Jenkins

2021-12-18 Thread Luca Toscano
o work on the issues above in this weekend, > would you share the result after that? > > Kengo Seki > > On Thu, Dec 16, 2021 at 1:30 AM Olaf Flebbe wrote: > > > > hi luca > > > > we can do the update together in the next days (evening local time in eu). > > &g

Re: Some ideas about Jenkins

2021-12-15 Thread Luca Toscano
there will be more people watching. The upgrade itself shouldn't last long (10/15 mins if everything goes fine). Thanks in advance! Luca On Sat, Dec 11, 2021 at 10:01 AM Luca Toscano wrote: > > Hi everybody, > > I opened a couple of Jiras for Jenkins: > - https://issues.apache.

[jira] [Created] (BIGTOP-3614) Docker provisioner fails to start a cluster on Debian 10/11 images

2021-12-13 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3614: Summary: Docker provisioner fails to start a cluster on Debian 10/11 images Key: BIGTOP-3614 URL: https://issues.apache.org/jira/browse/BIGTOP-3614 Project: Bigtop

[jira] [Created] (BIGTOP-3613) Review log4j configurations for CVE-2021-44228

2021-12-11 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3613: Summary: Review log4j configurations for CVE-2021-44228 Key: BIGTOP-3613 URL: https://issues.apache.org/jira/browse/BIGTOP-3613 Project: Bigtop Issue Type

Re: Power (ppc64le) CI/CD upgrade

2021-12-11 Thread Luca Toscano
new node you need to do a „docker login“ on the jenkins slave > account once. IIUC this node has been setup only a few days before > > olaf > > > Am 11.12.2021 um 09:17 schrieb Luca Toscano : > > > > Hi! > > > > I am very new to the project so I don't ha

Some ideas about Jenkins

2021-12-11 Thread Luca Toscano
Hi everybody, I opened a couple of Jiras for Jenkins: - https://issues.apache.org/jira/browse/BIGTOP-3611 - Upgrade Jenkins to the latest upstream - https://issues.apache.org/jira/browse/BIGTOP-3612 - Add a backup for Jenkins' /home/jenkins dir I didn't find anything open for these topics,

[jira] [Created] (BIGTOP-3612) Move Jenkins master's /home/jenkins to a dedicated EBS volume

2021-12-11 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3612: Summary: Move Jenkins master's /home/jenkins to a dedicated EBS volume Key: BIGTOP-3612 URL: https://issues.apache.org/jira/browse/BIGTOP-3612 Project: Bigtop

[jira] [Created] (BIGTOP-3611) Upgrade Jenkins to latest upstream

2021-12-11 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3611: Summary: Upgrade Jenkins to latest upstream Key: BIGTOP-3611 URL: https://issues.apache.org/jira/browse/BIGTOP-3611 Project: Bigtop Issue Type: Improvement

Re: Set up CI for Debian 11 Bullseye

2021-12-11 Thread Luca Toscano
was done :) Thanks! Luca On Mon, Dec 6, 2021 at 9:57 PM Luca Toscano wrote: > > Update: I have deleted (via the UI) the failed builds that I tried in > these days for Trunk packages, and I freed ~100G. Is there any > specific reason to keep more than 2/3 builds for each project

Re: Power (ppc64le) CI/CD upgrade

2021-12-11 Thread Luca Toscano
Hi! I am very new to the project so I don't have a lot of context on the ppc64 node, but today I tried to rebuild the docker images and all the jobs for ppc64 failed: https://ci.bigtop.apache.org/job/Docker-Puppet-Trunk/34/ It is weird since, from the logs, it seems that the node works but it

[jira] [Created] (BIGTOP-3610) Hadoop's debian packaging fails due to Zookeeper jars

2021-12-09 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3610: Summary: Hadoop's debian packaging fails due to Zookeeper jars Key: BIGTOP-3610 URL: https://issues.apache.org/jira/browse/BIGTOP-3610 Project: Bigtop Issue

Re: Set up CI for Debian 11 Bullseye

2021-12-06 Thread Luca Toscano
the rest. Let me know what you think! Luca Luca On Sun, Dec 5, 2021 at 10:02 AM Luca Toscano wrote: > > Thanks a lot Olaf, totally missed the credentials for AWS, I was able > to get the new hostnames and ssh in. The master's root partition seems > to be full afaics, and under /home/

Re: Set up CI for Debian 11 Bullseye

2021-12-05 Thread Luca Toscano
tate=running > > Best > Olaf > > > Am 04.12.2021 um 18:23 schrieb Luca Toscano : > > > > Hi :) > > > > https://ci.bigtop.apache.org/log/all seems to indicate that we are > > again in trouble, docker-slave-02 seems full. I got the credentials to > >

Re: Set up CI for Debian 11 Bullseye

2021-12-04 Thread Luca Toscano
;> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe >> <mailto:o...@oflebbe.de>>: > >> > >> I am working with luca to get him access .. > >> > >> Olaf > >> > >>> Am 27.11.2021 um 18:29 schrieb Luca Toscano >>> <mailt

[jira] [Created] (BIGTOP-3609) Hive package build failure for CentOS 7

2021-12-04 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3609: Summary: Hive package build failure for CentOS 7 Key: BIGTOP-3609 URL: https://issues.apache.org/jira/browse/BIGTOP-3609 Project: Bigtop Issue Type: Bug

[jira] [Created] (BIGTOP-3608) HBase zookeeper build failure

2021-12-04 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3608: Summary: HBase zookeeper build failure Key: BIGTOP-3608 URL: https://issues.apache.org/jira/browse/BIGTOP-3608 Project: Bigtop Issue Type: Bug Affects

Re: Set up CI for Debian 11 Bullseye

2021-11-27 Thread Luca Toscano
m with at least this repo. So comment it out for now if > it resolves the isse. Or remove the whole proxy stuff altoghether. We had > stability issues with the internet connectivity at that time. The proxy > resolved that mostly. > > Olaf > > > > Am 26.11.2021 um 19:06 schrieb

Re: Set up CI for Debian 11 Bullseye

2021-11-26 Thread Luca Toscano
f308a66ae/build.gradle#L462 > > <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> > > Olaf > > > > Am 26.11.2021 um 14:23 schrieb Luca Toscano : > > > > Hi Olaf, > > > > I restarted the jobs a

Re: Set up CI for Debian 11 Bullseye

2021-11-26 Thread Luca Toscano
start/ > <https://ci.bigtop.apache.org/job/nexus-restart/> > > If this network is not present you might get this kind of error > So try to restart nexus. > > Best > Olaf > > > Am 26.11.2021 um 09:04 schrieb Luca Toscano : > > > > Update: > > > > - Fou

Re: Set up CI for Debian 11 Bullseye

2021-11-26 Thread Luca Toscano
be appreciated :) Luca On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano wrote: > > Update: I added debian-11 support, if nobody disagrees I'd kick off a > package build. I can also enable periodic execution but I am not sure > what I should put in the related time interval field (I

Re: Set up CI for Debian 11 Bullseye

2021-11-25 Thread Luca Toscano
Update: I added debian-11 support, if nobody disagrees I'd kick off a package build. I can also enable periodic execution but I am not sure what I should put in the related time interval field (I didn't find it in the docs). Luca On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano wrote: > > H

Re: [DISCUSS] Release plan of Bigtop 3.1

2021-11-23 Thread Luca Toscano
Hi Masatake, one thing that I'd be worried about from an admin/operator point of view is a potential double upgrade for Bigtop 3.1, namely Zookeper first and then Hadoop (plus other services if needed). In my case, we would probably keep relying on Debian's zookeeper packages (3.4) since we don't

Re: Set up CI for Debian 11 Bullseye

2021-11-21 Thread Luca Toscano
oblems etc..). Luca On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe wrote: > > Hi Luca, > > I gave you super powers. > > Best > Olaf > > > Am 19.11.2021 um 08:15 schrieb Luca Toscano : > > > > Hi everybody! > > > > I am working on https://issues

Set up CI for Debian 11 Bullseye

2021-11-18 Thread Luca Toscano
Hi everybody! I am working on https://issues.apache.org/jira/browse/BIGTOP-3600 and IIUC I'd need to add the CI jobs now, but of course I don't have permissions. If anybody has some time to help I'd be happy to learn the process and help out :) Thanks! Luca

[jira] [Created] (BIGTOP-3600) Add support for Debian 11 Bullseye

2021-11-14 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3600: Summary: Add support for Debian 11 Bullseye Key: BIGTOP-3600 URL: https://issues.apache.org/jira/browse/BIGTOP-3600 Project: Bigtop Issue Type: New Feature

Re: [DISCUSS] New Features post Bigtop 3.0

2021-11-06 Thread Luca Toscano
Hi Evans! On Tue, Nov 2, 2021 at 5:35 PM Evans Ye wrote: > > Hi folks, > > With Bigtop 3.0 been released, I think it's time to discuss what's new as > our next steps. Of course the open source ver. of unified compatible Hadoop > Distro. is still our core product going forward. But the

Re: [VOTE] Release Bigtop version 3.0.0

2021-10-20 Thread Luca Toscano
This is really great, so happy to see 3.0 in RC state! We will probably not be able to test a full upgrade of our testing environment for this vote, but we'll surely report back any issue that we find when we decide to upgrade. Since this is a hadoop 2 -> hadoop 3 upgrade for Bigtop, is there any

Re: 3.0.0 release branch cutting notification

2021-08-26 Thread Luca Toscano
Hi Arnaud, I'd be interested in Bullseye support as well. IIRC it was already discussed, and the plan was to cut another release after 3.0.0 to avoid waiting (at the time it wasn't clear when Debian would have released). I think it is fine to postpone Bullseye support and release 3.0 asap, maybe

[jira] [Created] (BIGTOP-3566) Add support for the Namenode Observer role in puppet

2021-07-02 Thread Luca Toscano (Jira)
Luca Toscano created BIGTOP-3566: Summary: Add support for the Namenode Observer role in puppet Key: BIGTOP-3566 URL: https://issues.apache.org/jira/browse/BIGTOP-3566 Project: Bigtop Issue

Re: Interesting blog post from Linkedin

2021-07-02 Thread Luca Toscano
/me apologizes, I just learned that Masatake's answers ended up in my gmail spam folder :( Going to answer those thanks :) On Fri, Jul 2, 2021 at 7:59 AM Luca Toscano wrote: > > Ping on this, I'd be curious to know people's opinion and experiences > (it is also fine to not answer

Re: Interesting blog post from Linkedin

2021-07-01 Thread Luca Toscano
Ping on this, I'd be curious to know people's opinion and experiences (it is also fine to not answer of course :) Lica On Sat, Jun 12, 2021 at 7:40 PM Roman Shaposhnik wrote: > > On Fri, Jun 11, 2021 at 9:53 AM Luca Toscano wrote: > > > Hi everybody, &g

Re: Powered By Bigtop

2021-07-01 Thread Luca Toscano
On Thu, Jul 1, 2021 at 1:12 AM Roman Shaposhnik wrote: > > On Mon, Jun 28, 2021 at 10:24 PM Evans Ye wrote: > > > > Sure. That's a good suggestion. But to me it's hard to gather the > > information by solely googling it. Even though there are evidences showing > > they're using Bigtop, we can't

Interesting blog post from Linkedin

2021-06-11 Thread Luca Toscano
Hi everybody, https://engineering.linkedin.com/blog/2021/the-exabyte-club--linkedin-s-journey-of-scaling-the-hadoop-distr is a nice blog post reading. There are some interesting follow ups in my opinion: - Fair vs Non-Fair locking for the HDFS Namenode. IIUC this seems to be a code change rather

  1   2   >