Dear all,
I have to bring up this old item again. According to the Apache release
processes, I'd need to archive the current 9.0 release from the site.
this means that our 9.0 NBM-s are going to be removed from there as
well. The files are going to be available on archive.apache.org which is
not mirrored. Only the most recent release are supposed to be out on the
release and their mirrors.
This means we need to find some place to our 9.0 nbms. I also won't mind
if we could move away storing our nbms, "officially" (there are no
official binary releases, yet) on some other place than Apache dist
infrastructure. It's a bit weird to sign and upload 650+ binaries with
each and every release. So let's go back to square one.
1. We can change the redirection to the Apache archive site. That is
not mirrored, and initially we would put a real traffic on there.
2. We can serve the nbm-s from our netbeans-vm.
Checked the stats we get ~25000 requests a month on updates.xml.gz
if we'd transfer all the update center (~120 Mb, for NetBeans 10.0)
all the modules with each request for updates.xml.gz our transfer
would be ~3 Tb. So we would definitely fit into the 3-5 Tb/per month
usage limit. That would be actually much less, as this is a worst
case scenario.
3. Search for a free third party solution.
I'd try Bintray, we can get 10 Gb space and CDN, so it is pretty
reliable, the only concern is the 1 Tb of network limit per month.
Unfortunately I only can guess that the 3 Tb is really exaggerated.
All the log files I've found do not include the actual download
statistics of our nbm-s
4. Something else?
I can't archive the current 9.0 till this thing is decided.
On 4/10/18 11:39 PM, Antonio wrote:
Hi all,
In order to understand this long thread (about NETBEANS-330 [1]) I
tried to summarize it. Please review and send corrections & questions
as appropriate.
Kind regards,
Antonio
== Objectives
Host the NetBeans 9.0 Update Center on Apache infrastructure.
== Constraints/facts/options
- We cannot host it on a website due to bandwidth requirements of 3-5
Tb/month.
- We must use the Apache Mirror system and their "closer.cgi,
closer.lua" cgi scripts [2] to select the closest mirror to the user.
This script can either redirect directly to the closest binary file
(ready to download), or return a JSON response with a list of closest
mirrors.
- Distributing through the Apache Mirror system requires a proper
release, with voting, approval and signing, etc.
- Geertjan says that prior to the final release we could do an rc
release to have this feature tested.
- The UC catalog xml file can be hosted in the mirror system. For this
to work the catalog xml file must contain relative paths, so when aged
releases are moved away from the mirror system and into the archive
things keep working.
- We can HTTP redirect to the proper catalog.xml file...
a) ... From our website, redirecting to the Apache mirror system or to
the archive with an .htaccess file (under our control).
b) ... Idem, by asking infra to modify our server configuration (more
performant but requires Infra tickets).
c) ... Using a script of ours hosted at the "VM" (a web server ours
currently hosting selenium), that may also track/log some statistics.
- The NetBeans UC module will make a request for the XML file to our
website, from there will be redirected to the mirror/archive. Later on
it will build up proper URLs (using some prefix and the relative paths
inside the catalog xml file) for the final NBM downloads.
- Mirror download statistics may be currently downloaded from
https://www.apache.org/dyn/stats/netbeans.log
- Web server statistics are available through a very detailed request
to infra. They say they're not in the "counting business" :-)
- Jan is working in NBM generation (with NOTICE & LICENSE, etc.).
Jenkins now generates these artifacts:
https://builds.apache.org/view/Incubator%20Projects/job/incubator-netbeans-linux/lastSuccessfulBuild/artifact/nbbuild/nbms/
== Open issues & questions
1- It seems we won't be able to host some OSGi bundles in the mirror
system due to licensing issues?
2- Mirrors are not as reliable as a CDN: files can be corrupt, or
whatever. The UC module should verify the integrity of the downloaded
files, etc.
3- We may want to select a small set of NBM files for the first run,
to workaround previous licensing problems.
[1]
https://issues.apache.org/jira/browse/NETBEANS-330
[2]
https://reference.apache.org/pmc/mirror_scripts
On 05/04/18 14:38, Geertjan Wielenga wrote:
Hi all,
We need to nail down this one and I think the key blocker is that
there are
different ideas about what this is about:
https://issues.apache.org/jira/browse/NETBEANS-330
The above is not about the Plugin Portal.
If I understand it correctly, this is about where the NBMs (which
ones? how
many? do we know?) and the related XML file (a.k.a. update center)
will be
hosted.
AFAIK, the XML file and the NBMs could be put onto our Apache
NetBeans VM
just like Synergy:
http://netbeans-vm.apache.org/synergy
The key question remains, which NBMs are we talking about here,
applicable
to the 9.0 release, I think.
Thanks,
Gj
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
