There is no rsync access to central. But the crawling is doing the
equivalent amount of damage.
There is no upside to using rsync over a repository manager.
On 29-Sep-08, at 10:51 AM, Daniel Kulp wrote:
One thing I keep thinking about doing is creating a public mirror
that is
synced from central (it's a public mirror, thus, they would allow
that), but
provide rsync acess on some sort of paid agreement. Maybe $5/month
or
possibly just a ontime $100 setup fee or similar. Basically,
enough to
cover the bandwidth/hosting charges plus deter "everyone and their
mother"
from just rsyncing away. Is that something that people would have
interest
in?
If I only had the time to get it setup... :-(
Dan
On Monday 29 September 2008 10:21:54 am Beyer,Nathan wrote:
What would you suggest then? Anything that requires customized maven
installs or modifying 'settings.xml' post install is not feasible
in our
environment - development is too distributed.
In the long-run I believe the rsync approach does reduce bandwith,
but more
importantly, the concurrent access to the central repo via HTTP is
close to
nil.
Additionally, as I mentioned, the repository managers are NOT
stable and
require too much configuration and setup. These are not acceptable
options.
The repository managers aren't providing any other value beyond
central
repo caching for us.
If you're going to cut off anonymous rsync access, you might as
well just
kill anonymous central repo access too, as that's the only way
you'll be
able to force people into use repository managers.
I would suggest more granular rsync access, so that requests can be
more
targeted.
-Nathan
-----Original Message-----
From: Jason van Zyl [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 3:51 PM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote:
I disagree. 10gb or even 20gb isn't that much data, and rsync isn't
pulling that same amount down every time it runs. We're doing it and
it's working quite well. It's much more stable and reliable than any
other current mirroring practices. The internal DNS modification
makes user setup easy, since there isn't any. The use of mirror
settings per device is a non-starter for large, disparate
organizations. All of the various caching servers just aren't stable
enough yet, in my opinion.
It is possible to get blocked by the central repo - we were
contacted about our significant usage and told we were on the verge
of being blacklisted, which is what lead us to rsync the mirror.
There is no way you could use less bandwidth rsyncing then using a
repository manager. If everyone rsynced and we allowed that against
central we would get destroyed. We only allow mirrors to rsync, not
users and mirrors will probably also stop providing rsync access
because the first hit is just too high now if everyone did it.
-Nathan
-----Original Message-----
From: Wayne Fay [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 11:11 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
IIRC Central is well over 10gb at this point (possibly 20gb) and a
given organization will really only use at the most 1gb of it, so
rsync'ing it is just a bad idea unless you are setting up an actual
external mirror that will be available to the community.
They are already using Artifactory, and I certainly hope/assume they
are caching the results. This would limit their use of Central to
one
access per artifact (GAV) plus some hits by people not using their
Artifactory instance.
I would generally doubt they are actually blocked by Central, but
rather this is an intermittent failure that will eventually resolve
itself.
Wayne
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>:
It's possible that from the central repo's perspective, all traffic
from your company may seem like it's coming from one IP address
because of NAT.
Using an internal mirror can help alleviate things. The most non-
invasive mirror would be to rsync the central repo periodically and
then modify internal DNS to point 'repo1.maven.org' to an internal
IP address. You can save a lot of bandwidth and time this way.
-Nathan
-----Original Message-----
From: 陈思淼 [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 10:47 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
we didn't do that kind of thing. we have a company-level
artifactory
repository.someone didn't follow the rule but most of us are good
citizen,
and follow the maven RULE,
Is maven block strategy to block IP too strict?
Can I do anything to Fix it Up?
2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
It is possible to get blocked if you are acting as a bad citizen
(downloading the entire Central repo using wget, for example).
Have
you (or someone else at your company) attempted to do this from
your
IP address?
If not, the repo is probably just busy, or you had some random
Internet connection failure. Try again. "Normal" Maven usage of
the
repo will not get you blocked.
Wayne
On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]>
wrote:
This's log from artifactory.
2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:
259{10}) -
repo1:
Error in getting information for 'org/apache/maven
/maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
(org.apache.commons.httpclient.ConnectionPoolTimeoutException:
Timeout
waiting
for connection).
we company only have one outlet IP address ,someone may download
Maven
from
apache and didn't set the Mirror of central in the conf/
setting.xml. so
they
download the pom directly from central? Is that the reason why
the
central
repo block our IP address?
----------------------------------------------------------------------
CONFIDENTIALITY NOTICE This message and any included attachments
are from Cerner Corporation and are intended only for the
addressee. The information contained in this message is
confidential and may constitute inside or non-public information
under international, federal, or state securities laws.
Unauthorized forwarding, printing, copying, distribution, or use of
such information is strictly prohibited and may be unlawful. If you
are not the addressee, please promptly delete this message and
notify the sender of the delivery error by e-mail or you may call
Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1)
(816)221-1024.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
----------------------------------------------------------------------
CONFIDENTIALITY NOTICE This message and any included attachments
are from
Cerner Corporation and are intended only for the addressee. The
information
contained in this message is confidential and may constitute inside
or
non-public information under international, federal, or state
securities
laws. Unauthorized forwarding, printing, copying, distribution, or
use of
such information is strictly prohibited and may be unlawful. If you
are not
the addressee, please promptly delete this message and notify the
sender of
the delivery error by e-mail or you may call Cerner's corporate
offices in
Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
--
Daniel Kulp
[EMAIL PROTECTED]
http://www.dankulp.com/blog
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
believe nothing, no matter where you read it,
or who has said it,
not even if i have said it,
unless it agrees with your own reason
and your own common sense.
-- Buddha
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
