Hi, > > I don't understand the background: we currently have Lucene node, so why > a second one? > > Our randomized testing regime means the more bandwidth, the better. Are > you seriously arguing against increasing testing frequency?
Of course not! When reading your original e-mail, I was under the impression that Infra wants to turn off our lucene-slave node! The second question was, if there is anything against using the other nodes for running Lucene tests? My answer to that is: - I talked with Infra several times about this and the general problem is: As the other nodes are shared by all projects, it would be not good to block other nodes with running Lucene/Solr tests 24/7 in a loop. This is what the current lucene node is doing. It triggers builds by a "fake" schedule so it ensures something is running all the time. This already led to confusion at infra, because this is untypical. Lucene is "the" user of randomized testing and it is not yet used at other places to this extent. Because of this, Infra does not allow "scheduled" builds with high frequencies and disables them from time to time. I figured out with them, that all jobs running on the lucene node are excluded by that limitation, so we can use it 100% for randomization. - You can easily run Lucene jobs on other nodes, just not with a 24/7 schedule ๐ The only thing you have to do is: (a) invoke "ant iv-bootstrap" once per node (see the manual job on our node to trigger this); (b) ideally place a lucene.build.properties file on the node's home directory with node-specific config. Problem is that Lucene's automatic CPU assignment can be tuned for jenkins nodes (you can use all cpus, but on the other hand if a node has multiple executors, go down). As this is job-unspecific, it's better to deploy that as config file on the node's home directy. The lucene node has a lucene.build.properties, the same for Policeman Jenkins. > There has been discussion of automatically testing patches on JIRA issues > (SOLR-10912). As far as Iโm concerned, we wonโt be able to consider that > until/unless we have more Jenkins bandwidth. I think that's fine, because it's not running 24/7, only once per JIRA patch upload. > Also, it would be nice to be able to run regular performance benchmarks, but > without more bandwidth, that would not be a prudent use of the currenly > limited resources. That again has the same problems as described above, a separate node would be good! > > BTW, the Policeman server now has new hardware like NVME RAID drives > and faster CPU. See it as donation to the project. It is not linked to ASF > because of custom build behavior regarding JDK randomization, that's not > possible to setup at ASF. > > Thanks Uwe for running your Jenkins. Itโs great to have regular testing on > all > the platforms you support. Thanks! About donating nodes: Maybe we can get a second sponsor for another node running Lucene jobs in our usual 24/7 loop. The difference to Policeman's machine would be that it can be included into ASF's Jenkins infrastructure, as it would be a non-random-JVM slave and would only run Linux. So we just need to buy/rent the hardware and install the OS (e.g., Ubuntu 16.04) on it, everything else would be handled by ASF. The node would get the "lucene" label and would be part of the round. FYI, the Policeman Jenkins node is a rented datacenter node, hosted at Hetzner in Germany. I updated it recently ago to this one (perviously I was at another hoster, but this one is way better because of better network, hardware, native IPv6,...): https://www.hetzner.com/dedicated-rootserver/px61-nvme For monthly 59,00 EUR you get 2x512 GB NVMe drives and a - ok, well enough - CPU with 8 hypercores (that runs main Jenkins and Slave 3 VMs at same time). I am very happy with that one. Maybe some Solr-related company like my own sponsoring SD DataSolutions GmbH could sponsor the second one? I can take over the initial installation and some admin tasks. The machine gets a single IPv4 address by default and a whole IPv6 /64 network. Policeman Jenkins mainly uses the IPv6 network, only for the not-so-equipped countries without IPv6 in every household I added extra IPv4 addresses for the virtual machines on it ๐. What's interesting: Apache's Europe webservers are running in same datacenter and all IPv6 Apache webserver visitors use this server (also when coming from the US; I think it's the only one at ASF with IPv6 - which is incredible lame...). Here is the traceroute from Policeman Jenkins to ASF webserver (somehow Apache missed to add the reverse IPv6 lookup): root@serv1 ~ # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 90:1b:0e:c4:3f:e9 inet addr:88.99.242.108 Bcast:88.99.242.127 Mask:255.255.255.192 inet6 addr: 2a01:4f8:10b:2ab::2/64 Scope:Global inet6 addr: fe80::921b:eff:fec4:3fe9/64 Scope:Link [...] root@serv1 ~ # traceroute6 lucene.apache.org traceroute to lucene.apache.org (2a01:4f8:130:2192::2), 30 hops max, 80 byte packets 1 2a01:4f8::a:10:b (2a01:4f8::a:10:b) 0.345 ms 0.333 ms 0.329 ms 2 core21.fsn1.hetzner.com (2a01:4f8:0:3::65) 0.302 ms 0.307 ms core21.fsn1.hetzner.com (2a01:4f8:0:3::5d) 0.221 ms 3 ex9k1.rz13.hetzner.de (2a01:4f8:0:3::21a) 3.797 ms ex9k1.rz13.hetzner.de (2a01:4f8:0:3::21e) 3.898 ms ex9k1.rz13.hetzner.de (2a01:4f8:0:3::21a) 3.797 ms 4 2a01:4f8:130:2192::2 (2a01:4f8:130:2192::2) 0.237 ms 0.243 ms 0.236 ms root@serv1 ~ # host 2a01:4f8:10b:2ab::2 2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.a.2.0.b.0.1.0.8.f.4.0.1.0.a.2.ip6.arpa domain name pointer serv1.sd-datasolutions.de. root@serv1 ~ # host 2a01:4f8:10b:2ab::10 0.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.a.2.0.b.0.1.0.8.f.4.0.1.0.a.2.ip6.arpa domain name pointer serv1-vm1.sd-datasolutions.de. root@serv1 ~ # host 2a01:4f8:10b:2ab::11 1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.a.2.0.b.0.1.0.8.f.4.0.1.0.a.2.ip6.arpa domain name pointer serv1-vm2.sd-datasolutions.de. root@serv1 ~ # host 2a01:4f8:10b:2ab::12 2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.a.2.0.b.0.1.0.8.f.4.0.1.0.a.2.ip6.arpa domain name pointer serv1-vm3.sd-datasolutions.de. root@serv1 ~ # host 2a01:4f8:10b:2ab::22a01:4f8:130:2192::2 Host 2a01:4f8:10b:2ab::22a01:4f8:130:2192::2 not found: 3(NXDOMAIN) <==== !?!?!?!?! โน But, as you see, same datacenter ๐ Uwe --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
