Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-19 Thread Olblak 
Hey, 

My two cents here, regarding update-center
I wanted to work on it quite a long time ago, in short that service is running 
in a deprecated cloud account, and I was thinking of refactoring it before 
moving it on Azure.
I created the ticket INFRA-1656 
 to track the work but 
unfortunately, I had to de-prioritized it, so if you have some thoughts, feel 
free to add them there as well.



---
gpg --keyserver keys.gnupg.net --recv-key 52210D3D
---


On Fri, Nov 15, 2019, at 12:49 PM, Rick wrote:
> What I'm talking about is not the internal mirror. The mirror is public. I 
> just set up one mirror on GitHub which is 
> https://github.com/jenkins-zh/update-center-mirror/tree/master/tsinghua.
> 
> On Fri, Nov 15, 2019 at 7:46 PM Oliver Gondža  wrote:
>> I suspect few people would bother to report (accurate) numbers there. 
>>  Setting up internal mirror is the part that needs to be done and this is 
>>  the kind of bonus task that always gets outprioritized. IMO.
>> 
>>  On 15/11/2019 12.14, Rick wrote:
>>  > Hi Björn,
>>  > 
>>  > It's true that those download statistics cannot be count. What about 
>>  > building a third party statistic site. And we can publish this data 
>>  > besides the official one as a reference. What do you think about this?
>>  > 
>>  > Best,
>>  > Rick
>>  > 
>>  > On Fri, Nov 15, 2019 at 6:52 PM 'Björn Pedersen' via Jenkins Developers 
>>  > mailto:jenkinsci-dev@googlegroups.com>> 
>>  > wrote:
>>  > 
>>  > Hi,
>>  > 
>>  > So the idea is to omit the round-trip to the main update center on
>>  > each download. Community-wise this has one downside: all those
>>  > downloads then do not show up in the download statistics.
>>  > But the same is true for all fully private mirrors behind firewalls
>>  > that serve updatecenter.json with internal urls.
>>  > 
>>  > 
>>  > Björn
>>  > 
>>  > Am Donnerstag, 14. November 2019 13:45:11 UTC+1 schrieb Rick:
>>  > 
>>  > Hi Baptiste,
>>  > 
>>  > Thanks for your response. Jenkins download the plugin by the URL
>>  > which comes from update-center.json. For example this plugin
>>  > 
>> http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi.
>>  > 
>>  > The problem is that it’s too slow for me. I hope jenkins can
>>  > download it from
>>  > 
>> https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi.
>>  > 
>>  > 
>>  > -- 
>>  > You received this message because you are subscribed to the Google
>>  > Groups "Jenkins Developers" group.
>>  > To unsubscribe from this group and stop receiving emails from it,
>>  > send an email to jenkinsci-dev+unsubscr...@googlegroups.com 
>> 
>>  > > >.
>>  > To view this discussion on the web visit
>>  > 
>> https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com
>>  > 
>> .
>>  > 
>>  > 
>>  > 
>>  > -- 
>>  > Zhao Xiaojie (Rick)
>>  > Blog: https://github.com/LinuxSuRen
>>  > Twitter: https://twitter.com/suren69811254
>>  > 
>>  > -- 
>>  > You received this message because you are subscribed to the Google 
>>  > Groups "Jenkins Developers" group.
>>  > To unsubscribe from this group and stop receiving emails from it, send 
>>  > an email to jenkinsci-dev+unsubscr...@googlegroups.com 
>>  
>>  > > >.
>>  > To view this discussion on the web visit 
>>  > 
>> https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF%3Ddk69exkvWN0Tv6rsdDa0cbcCLDSHaf8KxTsqKdgdRg%40mail.gmail.com
>>  
>>  > 
>> .
>> 
>> 
>>  -- 
>>  oliver
>> 
>>  -- 
>>  You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>>  To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-dev+unsubscr...@googlegroups.com 
>> .
>>  To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/2e072c53-8421-a371-f9d5-4e53e5c88cd3%40gmail.com.
> 
> 
> -- 
> Zhao Xiaojie (Rick)
> Blog: https://github.com/LinuxSuRen
> Twitter: https://twitter.com/suren69811254
> 
> 

> --
>  You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
>  To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>  To view this discussion on

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-15 Thread Rick 
What I'm talking about is not the internal mirror. The mirror is public. I
just set up one mirror on GitHub which is
https://github.com/jenkins-zh/update-center-mirror/tree/master/tsinghua.

On Fri, Nov 15, 2019 at 7:46 PM Oliver Gondža  wrote:

> I suspect few people would bother to report (accurate) numbers there.
> Setting up internal mirror is the part that needs to be done and this is
> the kind of bonus task that always gets outprioritized. IMO.
>
> On 15/11/2019 12.14, Rick wrote:
> > Hi Björn,
> >
> > It's true that those download statistics cannot be count. What about
> > building a third party statistic site. And we can publish this data
> > besides the official one as a reference. What do you think about this?
> >
> > Best,
> > Rick
> >
> > On Fri, Nov 15, 2019 at 6:52 PM 'Björn Pedersen' via Jenkins Developers
> > mailto:jenkinsci-dev@googlegroups.com>>
>
> > wrote:
> >
> > Hi,
> >
> > So the idea is to omit the round-trip to the main update center on
> > each download. Community-wise this has one downside: all those
> > downloads then do not show up in the download statistics.
> > But the same is true for all fully private mirrors behind firewalls
> > that serve updatecenter.json  with internal urls.
> >
> >
> > Björn
> >
> > Am Donnerstag, 14. November 2019 13:45:11 UTC+1 schrieb Rick:
> >
> > Hi Baptiste,
> >
> > Thanks for your response. Jenkins download the plugin by the URL
> > which comes from update-center.json. For example this plugin
> >
> http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi
> .
> >
> > The problem is that it’s too slow for me. I hope jenkins can
> > download it from
> >
> https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi
> .
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Jenkins Developers" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to jenkinsci-dev+unsubscr...@googlegroups.com
> > .
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com
> > <
> https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com?utm_medium=email_source=footer
> >.
> >
> >
> >
> > --
> > Zhao Xiaojie (Rick)
> > Blog: https://github.com/LinuxSuRen
> > Twitter: https://twitter.com/suren69811254
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Jenkins Developers" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to jenkinsci-dev+unsubscr...@googlegroups.com
> > .
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF%3Ddk69exkvWN0Tv6rsdDa0cbcCLDSHaf8KxTsqKdgdRg%40mail.gmail.com
> > <
> https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF%3Ddk69exkvWN0Tv6rsdDa0cbcCLDSHaf8KxTsqKdgdRg%40mail.gmail.com?utm_medium=email_source=footer
> >.
>
>
> --
> oliver
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/2e072c53-8421-a371-f9d5-4e53e5c88cd3%40gmail.com
> .
>


-- 
Zhao Xiaojie (Rick)
Blog: https://github.com/LinuxSuRen
Twitter: https://twitter.com/suren69811254

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTGUKxgoWERq%2B1Cyx9bSUdiNDRwSskLc32h-FBt2W7%3DB%3DQ%40mail.gmail.com.

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-15 Thread Oliver Gondža 
I suspect few people would bother to report (accurate) numbers there. 
Setting up internal mirror is the part that needs to be done and this is 
the kind of bonus task that always gets outprioritized. IMO.


On 15/11/2019 12.14, Rick wrote:

Hi Björn,

It's true that those download statistics cannot be count. What about 
building a third party statistic site. And we can publish this data 
besides the official one as a reference. What do you think about this?


Best,
Rick

On Fri, Nov 15, 2019 at 6:52 PM 'Björn Pedersen' via Jenkins Developers 
mailto:jenkinsci-dev@googlegroups.com>> 
wrote:


Hi,

So the idea is to omit the round-trip to the main update center on
each download. Community-wise this has one downside: all those
downloads then do not show up in the download statistics.
But the same is true for all fully private mirrors behind firewalls
that serve updatecenter.json  with internal urls.


Björn

Am Donnerstag, 14. November 2019 13:45:11 UTC+1 schrieb Rick:

Hi Baptiste,

Thanks for your response. Jenkins download the plugin by the URL
which comes from update-center.json. For example this plugin

http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi.

The problem is that it’s too slow for me. I hope jenkins can
download it from

https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi.


-- 
You received this message because you are subscribed to the Google

Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to jenkinsci-dev+unsubscr...@googlegroups.com
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com

.



--
Zhao Xiaojie (Rick)
Blog: https://github.com/LinuxSuRen
Twitter: https://twitter.com/suren69811254

--
You received this message because you are subscribed to the Google 
Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jenkinsci-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF%3Ddk69exkvWN0Tv6rsdDa0cbcCLDSHaf8KxTsqKdgdRg%40mail.gmail.com 
.



--
oliver

--
You received this message because you are subscribed to the Google Groups "Jenkins 
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/2e072c53-8421-a371-f9d5-4e53e5c88cd3%40gmail.com.

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-15 Thread Rick 
Hi Björn,

It's true that those download statistics cannot be count. What about
building a third party statistic site. And we can publish this data besides
the official one as a reference. What do you think about this?

Best,
Rick

On Fri, Nov 15, 2019 at 6:52 PM 'Björn Pedersen' via Jenkins Developers <
jenkinsci-dev@googlegroups.com> wrote:

> Hi,
>
> So the idea is to omit the round-trip to the main update center on each
> download. Community-wise this has one downside: all those downloads then do
> not show up in the download statistics.
> But the same is true for all fully private mirrors behind firewalls that
> serve updatecenter.json  with internal urls.
>
>
> Björn
>
> Am Donnerstag, 14. November 2019 13:45:11 UTC+1 schrieb Rick:
>>
>> Hi Baptiste,
>>
>> Thanks for your response. Jenkins download the plugin by the URL which
>> comes from update-center.json. For example this plugin
>> http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi
>> .
>>
>> The problem is that it’s too slow for me. I hope jenkins can download it
>> from
>> https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi
>> .
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com
> 
> .
>


-- 
Zhao Xiaojie (Rick)
Blog: https://github.com/LinuxSuRen
Twitter: https://twitter.com/suren69811254

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF%3Ddk69exkvWN0Tv6rsdDa0cbcCLDSHaf8KxTsqKdgdRg%40mail.gmail.com.

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-15 Thread 'Björn Pedersen' via Jenkins Developers 
Hi, 

So the idea is to omit the round-trip to the main update center on each 
download. Community-wise this has one downside: all those downloads then do 
not show up in the download statistics. 
But the same is true for all fully private mirrors behind firewalls that 
serve updatecenter.json  with internal urls.


Björn

Am Donnerstag, 14. November 2019 13:45:11 UTC+1 schrieb Rick:
>
> Hi Baptiste,
>
> Thanks for your response. Jenkins download the plugin by the URL which 
> comes from update-center.json. For example this plugin 
> http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi
> .
>
> The problem is that it’s too slow for me. I hope jenkins can download it 
> from 
> https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/4e01b094-d310-448c-9706-6ed526a36e52%40googlegroups.com.

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-14 Thread Xiaojie Zhao (Rick) 
Hi Baptiste,

Thanks for your response. Jenkins download the plugin by the URL which comes 
from update-center.json. For example this plugin 
http://updates.jenkins-ci.org/download/plugins/AnchorChain/1.0/AnchorChain.hpi 
.

The problem is that it’s too slow for me. I hope jenkins can download it from 
https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins/AnchorChain/1.0/AnchorChain.hpi
 
.

> On 14 Nov 2019, at 20:39, Baptiste Mathus  wrote:
> 
> FWIW

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/25B553BC-4244-4E8F-852C-716DCA4DD709%40gmail.com.

Re: speed your Jenkins by an adapter of Jenkins update center

2019-11-14 Thread Baptiste Mathus 
Hey Rick,

FWIW I've read twice the Why, and am still unsure what problem this would
be trying to solve. Care to maybe rephrase?

Thank you!

Le mer. 13 nov. 2019 à 12:07, Rick  a écrit :

> Hi all,
>
> An adapter of Jenkins update center.
>
> ## Why
>
> Jenkins downloads the plugins by parsing the JSON file which comes from a 
> update center.
> One thing that you might already noticed is about the speed of downloading.
> Accessing a global storage for everyone could be slow although
> there're many [mirror sites](http://mirrors.jenkins-ci.org/status.html).
>
> If you deep into the file 
> [update-center.json](https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json).
> Almost every mirror file is base on 
> `http://updates.jenkins-ci.org/download/plugins` 
> .
> So, the result is that these mirror sites can only speed up the process of 
> downloading file update-center.json.
> Jenkins can download the `.hpi` from your target mirror site.
>
> ## Background
>
> It's might not be a good idea to change the `update-center.json` file 
> directly. keeping sync all files
> should be simple without other logic.
>
> Second, you need to provide a certificate file if you changed the 
> `update-center.json`. Because Jenkins
> will validate the file before parsing it. It's necessary due to the safety 
> reason.
>
> ## Design
>
> * Create an adapter to replace the base URL
> * Provide a certificate file
>
> ## How to
>
> Here's a prototype implement which added into 
> [localization-zh-cn-plugin](https://github.com/jenkinsci/localization-zh-cn-plugin/pull/115).
> You Just need to take three steps if you want to use a real mirror of update 
> center:
>
> * install localization-zh-cn-plugin 1.0.10
> * use the new certificate file
> * change the update center URL
>
> ## Feedback
>
> Please don't hesitate to tell us your thoughts.
>
>
> You can find the codes from https://github.com/jenkins-zh/mirror-adapter.
>
>
> Best regards,
>
> Rick
>
> --
> Zhao Xiaojie (Rick)
> Blog: https://github.com/LinuxSuRen
> Twitter: https://twitter.com/suren69811254
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF0UDz17%3Dcd95cPN9Om_FmyZY0OK1VPNVO%2BFASykkMnpQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPyTVp2Awy2DGP7hGRHgMkKfJaDSimH379FQ%3DAk7eXDrb1HRPg%40mail.gmail.com.

speed your Jenkins by an adapter of Jenkins update center

2019-11-13 Thread Rick 
Hi all,

An adapter of Jenkins update center.

## Why

Jenkins downloads the plugins by parsing the JSON file which comes
from a update center.
One thing that you might already noticed is about the speed of downloading.
Accessing a global storage for everyone could be slow although
there're many [mirror sites](http://mirrors.jenkins-ci.org/status.html).

If you deep into the file
[update-center.json](https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json).
Almost every mirror file is base on
`http://updates.jenkins-ci.org/download/plugins`.
So, the result is that these mirror sites can only speed up the
process of downloading file update-center.json.
Jenkins can download the `.hpi` from your target mirror site.

## Background

It's might not be a good idea to change the `update-center.json` file
directly. keeping sync all files
should be simple without other logic.

Second, you need to provide a certificate file if you changed the
`update-center.json`. Because Jenkins
will validate the file before parsing it. It's necessary due to the
safety reason.

## Design

* Create an adapter to replace the base URL
* Provide a certificate file

## How to

Here's a prototype implement which added into
[localization-zh-cn-plugin](https://github.com/jenkinsci/localization-zh-cn-plugin/pull/115).
You Just need to take three steps if you want to use a real mirror of
update center:

* install localization-zh-cn-plugin 1.0.10
* use the new certificate file
* change the update center URL

## Feedback

Please don't hesitate to tell us your thoughts.


You can find the codes from https://github.com/jenkins-zh/mirror-adapter.


Best regards,

Rick

-- 
Zhao Xiaojie (Rick)
Blog: https://github.com/LinuxSuRen
Twitter: https://twitter.com/suren69811254

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAMM7nTF0UDz17%3Dcd95cPN9Om_FmyZY0OK1VPNVO%2BFASykkMnpQ%40mail.gmail.com.