Re: [Essentials] (client) Error Telemetry logging: JEP in early draft review

[Jesse Glick]

Seems reasonable.

The implementation separates JSON records with a newline. This is not
apparent from the JEP; it would be nice to mention it, so that readers
do not wonder whether you are using some kind of streaming JSON parser
(which is unnecessary when each line is a well-formed JSON document).

`JsonFormatter.formatException` can be replaced with
`Functions.printThrowable`, which formats chained exceptions more
readably. The JEP proposal for a future `stacktrace` field neglects to
mention that this will not work as is for exceptions with causes
and/or suppressed exceptions. In general `Throwable.printStackTrace`
can be overridden in nontrivial ways, sometimes including information
not present in the `message` nor stack trace (though less so after
JENKINS-46140), so I would not recommend bothering with anything but a
plain printed string.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1mbH2e86BymBfSBcxu4%2BhM0TewBS7XyBreok8AFg9yOA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Need help designing a fix for JENKINS-50504

[Jesse Glick]

On Wed, Apr 4, 2018 at 4:26 AM, Oleg Nenashev  wrote:
> WorkspaceList#inUse should be reacquired by Pipeline for sure when it
> reconnects to a new agent. I would guess it happens even now (or not?)

No, currently a lock is acquired only when a `node` (or `ws`) body is
started. I made a note in JENKINS-41854 about this.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1k446ET852Ot3g%3D1cOJAcPE8YFPsc%3DrVoKby3EcA%2BMEQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Using available pods in a Kubernetes deployment as slaves via Kubernetes plugin

[Karthik Duddu]

Hi all,

At my company, we use Jenkins to run builds with lots of parallel tasks, 
where the slaves for each task are provisioned from a private Kubernetes 
cluster. We have a very specific problem with these provisioned slaves: 
we'd like to reduce the time overhead of a Kubernetes slave to match that 
of a physical slave (or get as close as possible). Since our slave 
container itself has  a non-trivial start-up time (after provisioning, but 
before registering with the Jenkins master), we're thinking of maintaining 
a Kubernetes deployment of 'ready' slaves that register themselves with the 
master, and then are removed from the deployment when they're assigned a 
job; the rest of the lifecycle remains the same (that is, the slaves are 
still used only once). This ensures that we have a continuous supply of 
ready slaves, and we can also use pool size auto-scaling to keep up with 
load.

We've tried this out internally by modified the Kubernetes plugin a little 
to be able to support this system, and are reasonably satisfied with the 
results. I have a couple of questions with regard to this:

1. Is there a better way to reduce overhead? In our case, overhead 
essentially comprises of provisioning request time + pod scheduling time + 
container setup + slave connect-back.

2. Does this use-case fall within the realm of the Kubernetes plugin, or is 
it better off developed as a plugin dependent on this one?

Looking forward to feedback from y'all!

Thanks and regards,

Karthik Duddu

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/5d7300fd-a59c-40db-b591-8e3a533b6ca8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Essentials] (client) Error Telemetry logging: JEP in early draft review

[Baptiste Mathus]

Hello everyone,

So as a followup on the recent discussions around telemetry logging for
Essentials, I've drafted a JEP on this matter I would love to get feedback
on:

https://github.com/batmat/jep/tree/JENKINS-49805/jep/

Thanks!

-- Baptiste

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS4bikJoV3vEGdiMbMSzbgAwL1_LMZYKcUZhWBxrcj8qPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[DISCUSS] Change Jenkins default logging format

[Baptiste Mathus]

Hello everyone,

Having worked on more things related to Jenkins logging recently, I've had
the opportunity to remember my past pain when I was operating a Jenkins
instance and sending logs to an ELK cluster.
Compared to almost everything else in the infrastructure, the logstash
rules for Jenkins logs were unnecessarily complex.

The main pain-points, for me at least, had been the two-lines (sigh) per
log default, and also the localized date format (or log level...).
Even now, so many years after reading those, I still struggle daily to make
sure I'm reading the right line/date associated to the message I'm reading
on the line above.

I would like to propose we change the current logging format behavior to a
more readable and more operation-friendly one.
This would result in something close to the following format:

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to
org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log
initialized)
[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from
winstone.Logger logInternal)
[WARNING][2018-04-04 12:40:49] Empty contextPath (from
org.eclipse.jetty.server.handler.ContextHandler setContextPath)
[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from
org.eclipse.jetty.server.Server doStart)

Instead of the usual:

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog
Apr 04, 2018 12:36:41 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler
setContextPath
WARNING: Empty contextPath
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.z-SNAPSHOT


WDYT?

If this looks interesting to people, I'm ready to file the associated JEP
for it and possibly work on its implementation in the future.

Obviously, we would need some way to revert to the "legacy" format, at
least for some time for users to adapt. But that is not something I'm
particularly worried about.

-- Baptiste

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS4rN67DpzH5FoOg7YBU8jxFArjzq0ev%2BqRjowObu_pKvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[Oleg Nenashev]

Hi Baptiste,

Anyone can easily change the logging format in Jenkins by passing custom 
J.U.L properties file.
So it should not be a problem to change logging format even if you do not 
add a custom logging appender for ELK.

Changing a default logging format may cause regressions in existing 
monitoring systems. I would rather be conservative and keep the default 
logging format in Jenkins unless there is a strong justification to do 
that. New subprojects like Jenkins Essentials and Jenkins X could easily 
change the default format, because they have no compatibility commitment so 
far.

Best regards,
Oleg


On Wednesday, April 4, 2018 at 3:18:00 PM UTC+2, Baptiste Mathus wrote:
>
> Hello everyone,
>
> Having worked on more things related to Jenkins logging recently, I've had 
> the opportunity to remember my past pain when I was operating a Jenkins 
> instance and sending logs to an ELK cluster.
> Compared to almost everything else in the infrastructure, the logstash 
> rules for Jenkins logs were unnecessarily complex.
>
> The main pain-points, for me at least, had been the two-lines (sigh) per 
> log default, and also the localized date format (or log level...). 
> Even now, so many years after reading those, I still struggle daily to 
> make sure I'm reading the right line/date associated to the message I'm 
> reading on the line above.
>
> I would like to propose we change the current logging format behavior to a 
> more readable and more operation-friendly one.
> This would result in something close to the following format:
>
> [   INFO][2018-04-04 12:40:49] Logging initialized @180ms to 
> org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log 
> initialized)
> [   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from 
> winstone.Logger logInternal)
> [WARNING][2018-04-04 12:40:49] Empty contextPath (from 
> org.eclipse.jetty.server.handler.ContextHandler setContextPath)
> [   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from 
> org.eclipse.jetty.server.Server doStart)
>
> Instead of the usual:
>
> Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized
> INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog
> Apr 04, 2018 12:36:41 PM winstone.Logger logInternal
> INFO: Beginning extraction from war file
> Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler 
> setContextPath
> WARNING: Empty contextPath
> Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart
> INFO: jetty-9.4.z-SNAPSHOT
>
>
> WDYT?
>
> If this looks interesting to people, I'm ready to file the associated JEP 
> for it and possibly work on its implementation in the future.
>
> Obviously, we would need some way to revert to the "legacy" format, at 
> least for some time for users to adapt. But that is not something I'm 
> particularly worried about.
>
> -- Baptiste
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/a7cbc69c-e9d1-4c37-8a96-b35b1e920200%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Trademark Sublicensing Request: DevOps World - Jenkins World

[Kohsuke Kawaguchi]

I will be travelling on April 11th, so I asked Tyler to drive this part of
the meeting. My regrets for missing the meeting.

As a board member, given that Jenkins World is an approved name, from the
technical perspective of approving this trademark request, here is my
absentee +1 vote.

On Fri, Mar 30, 2018 at 9:53 AM Kohsuke Kawaguchi 
wrote:

> On behalf of CloudBees, I’d like to request the permission to use the
> following name that uses ‘Jenkins’ in it:
>
>- DevOps World - Jenkins World
>
> This is the new name of the event that we've been calling "Jenkins World", 
> which
> is a previously approved mark
> , and
> needless to say, many of you have been there in the past several years. I
> think all the basic considerations led to the approval of Jenkins World
> still applies. I’ve put this up as an agenda for April 11th project
> meeting to be able to get a decision there. If you have any concerns or
> objections, please chime in on this thread prior to the meeting, so that we
> can address them.
>
> Now, related but separately from the trademark sub-licensing request, I
> think this event name change naturally raises the question of "what does
> this change mean?", so let me describe how I see it as positive change.
>
> Over the years, Jenkins has become more and more widespread and mainstream
> in every corner of the software development world. At the same time, the
> problems solved by Jenkins is increasingly seen not as an isolated problem
> of build/test/deploy/etc, but as a part of a bigger problem space of
> Continuous Delivery and DevOps.
>
> The content of the past few Jenkins Worlds reflect this shift. There were
> many deep Jenkins talks that were very popular, like this
>  or that , but there are also
> other great sessions that discussed this broader problem space, like this
>  or that .
>
> So in order for the name to represent the substance more accurately, as
> the event producer, CloudBees decided that it would like to rename the
> event to “DevOps World - Jenkins World.” This way, the event can appeal to
> both those who are deep in Jenkins and those who are approaching this more
> broadly. It will help Jenkins be seen as the key piece of this landscape
> for the latter audience. While the word “DevOps” might feel ambiguous and
> fuzzy for those of us deep in technology, the event team has done various
> research to conclude that it is used more and to refer to this broad area
> than other terms, thereby making it the best choice. There are some other
> conferences that follow similar "AX - BX" naming patterns, like this one
> ,
> presumably for similar reasons.
>
> In terms of how the event will be actually produced, I have heard that it
> will not be that different from Jenkins World 2017. There will be one
> ticket, one event venue, one set of keynotes, and one exhibit space. There
> will be session tracks who are marked as Jenkins, DevOps, etc. And of
> course there will be marked community space. A few weeks back, Alyssa,
> Mark, Liam, Tyler, myself and a few others spent some time brainstorming
> what other interesting things we can do this year.
>
> I think Jenkins World remains one of those cases where the interest of the
> community and the interest of the event producer (CloudBees and other
> sponsors) are well aligned. The developer community benefits from having a
> get together and a place where we can interact with users. The user
> community benefits from the learning. And CloudBees and other sponsors
> benefit from being in front of those people. I think this name change will
> help us draw an even bigger crowd and in their eyes signify the key role
> Jenkins plays in this space, which I think will only make this event even
> more valuable.
> --
> Kohsuke Kawaguchi
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4xe3PgrVMUrq7ydc%3D-u0J93HXBHh%2B4iV6sVSeHjSG3OTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[Daniel Beck]

> On 4. Apr 2018, at 18:12, Oleg Nenashev  wrote:
> 
> Changing a default logging format may cause regressions in existing 
> monitoring systems. I would rather be conservative and keep the default 
> logging format in Jenkins unless there is a strong justification to do that. 

I expect that a change in logging format would surface quickly enough to not 
cause major headaches. Users with elaborate monitoring are probably also more 
likely to use LTS and we can cover this change in an upgrade guide.

The format is annoying enough that we should change it to something sane.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/879DFDED-BDED-4BC9-AB75-CD633C1C6A2F%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[R. Tyler Croy]

(replies inline)

On Wed, 04 Apr 2018, Oleg Nenashev wrote:

> Hi Baptiste,
> 
> Anyone can easily change the logging format in Jenkins by passing custom 
> J.U.L properties file.
> So it should not be a problem to change logging format even if you do not 
> add a custom logging appender for ELK.
> 
> Changing a default logging format may cause regressions in existing 
> monitoring systems. I would rather be conservative and keep the default 
> logging format in Jenkins unless there is a strong justification to do 
> that. New subprojects like Jenkins Essentials and Jenkins X could easily 
> change the default format, because they have no compatibility commitment so 
> far.


Thanks for sharing your thoughts Oleg, it makes me wonder: how could we
quantify the positive/negative impact of this (arguably) "bad default."

I'm reminded of some of the pre-Jenkins 2 discussions about security defaults
for example. I spent some time trying to discover Jenkins-log-parsing projects,
tools, and their communities to which we could reach out. Unfortunately I
wasn't able to find anything big past a couple of blog posts :-/

>From my personal experience as an Operator, compact and easily parseable
(sed/awk/perl) one-line logs are HUGELY valuable and worth any temporary
heartburn some people may experience with their existing hacks.



Cheers

- R. Tyler Croy

--
 Code: 
  Chatter: 
 xmpp: rty...@jabber.org

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
--

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/20180404172432.wlr4c5f35tuvnijj%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: PGP signature

Re: [DISCUSS] Change Jenkins default logging format

[Arnaud Héritier]

I think I already claimed my hate for this logging format many times.
For sure I understand Oleg's concern with the impact of changing the
default settings.
Like always for such kind of change when we don't really know how it could
impact our users I would prefer to introduce the change non activated by
default in 2.x and wait a 3.x for the change of the default value but if we
have to wait for 500 weeklies before the 3.x . I'm a bit septic about
my approach


On Wed, Apr 4, 2018 at 7:45 PM, Carlos Sanchez  wrote:

> I'm +1 to change the format. 2 lines is really annoying
>
> On Wed, Apr 4, 2018 at 7:24 PM, R. Tyler Croy  wrote:
>
>> (replies inline)
>>
>> On Wed, 04 Apr 2018, Oleg Nenashev wrote:
>>
>> > Hi Baptiste,
>> >
>> > Anyone can easily change the logging format in Jenkins by passing custom
>> > J.U.L properties file.
>> > So it should not be a problem to change logging format even if you do
>> not
>> > add a custom logging appender for ELK.
>> >
>> > Changing a default logging format may cause regressions in existing
>> > monitoring systems. I would rather be conservative and keep the default
>> > logging format in Jenkins unless there is a strong justification to do
>> > that. New subprojects like Jenkins Essentials and Jenkins X could easily
>> > change the default format, because they have no compatibility
>> commitment so
>> > far.
>>
>>
>> Thanks for sharing your thoughts Oleg, it makes me wonder: how could we
>> quantify the positive/negative impact of this (arguably) "bad default."
>>
>> I'm reminded of some of the pre-Jenkins 2 discussions about security
>> defaults
>> for example. I spent some time trying to discover Jenkins-log-parsing
>> projects,
>> tools, and their communities to which we could reach out. Unfortunately I
>> wasn't able to find anything big past a couple of blog posts :-/
>>
>> From my personal experience as an Operator, compact and easily parseable
>> (sed/awk/perl) one-line logs are HUGELY valuable and worth any temporary
>> heartburn some people may experience with their existing hacks.
>>
>>
>>
>> Cheers
>>
>> - R. Tyler Croy
>>
>> --
>>  Code: 
>>   Chatter: 
>>  xmpp: rty...@jabber.org
>>
>>   % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
>> --
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/jenkinsci-dev/20180404172432.wlr4c5f35tuvnijj%
>> 40blackberry.coupleofllamas.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/CALHFn6NsdOsMRtD%3DacAATvOiY_Wo15gxu6Fwh5iRCs3ogqMmfQ%
> 40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
-
Arnaud Héritier
http://aheritier.net
Mail/GTalk: aheritier AT gmail DOT com
Twitter/Skype : aheritier

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAFNCU-_ck5Tc0gbvBqDDMqnUnBtvnHP3cRUaapfcvJJ6jCmDyg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[Carlos Sanchez]

I'm +1 to change the format. 2 lines is really annoying

On Wed, Apr 4, 2018 at 7:24 PM, R. Tyler Croy  wrote:

> (replies inline)
>
> On Wed, 04 Apr 2018, Oleg Nenashev wrote:
>
> > Hi Baptiste,
> >
> > Anyone can easily change the logging format in Jenkins by passing custom
> > J.U.L properties file.
> > So it should not be a problem to change logging format even if you do not
> > add a custom logging appender for ELK.
> >
> > Changing a default logging format may cause regressions in existing
> > monitoring systems. I would rather be conservative and keep the default
> > logging format in Jenkins unless there is a strong justification to do
> > that. New subprojects like Jenkins Essentials and Jenkins X could easily
> > change the default format, because they have no compatibility commitment
> so
> > far.
>
>
> Thanks for sharing your thoughts Oleg, it makes me wonder: how could we
> quantify the positive/negative impact of this (arguably) "bad default."
>
> I'm reminded of some of the pre-Jenkins 2 discussions about security
> defaults
> for example. I spent some time trying to discover Jenkins-log-parsing
> projects,
> tools, and their communities to which we could reach out. Unfortunately I
> wasn't able to find anything big past a couple of blog posts :-/
>
> From my personal experience as an Operator, compact and easily parseable
> (sed/awk/perl) one-line logs are HUGELY valuable and worth any temporary
> heartburn some people may experience with their existing hacks.
>
>
>
> Cheers
>
> - R. Tyler Croy
>
> --
>  Code: 
>   Chatter: 
>  xmpp: rty...@jabber.org
>
>   % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
> --
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/20180404172432.wlr4c5f35tuvnijj%40blackberry.
> coupleofllamas.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CALHFn6NsdOsMRtD%3DacAATvOiY_Wo15gxu6Fwh5iRCs3ogqMmfQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[Baptiste Mathus]

2018-04-04 18:12 GMT+02:00 Oleg Nenashev :

> Hi Baptiste,
>
> Anyone can easily change the logging format in Jenkins by passing custom
> J.U.L properties file.
>

Yes. But in practice, I'm convinced virtually nobody does this.


> So it should not be a problem to change logging format even if you do not
> add a custom logging appender for ELK.
>
> Changing a default logging format may cause regressions in existing
> monitoring systems. I would rather be conservative and keep the default
> logging format in Jenkins unless there is a strong justification to do
> that. New subprojects like Jenkins Essentials and Jenkins X could easily
> change the default format, because they have no compatibility commitment so
> far.
>

FTR, I did consider this strategy about keeping the current behavior, and
introduce some sysprop to enable the new (and much much better, IMO)
behavior.
But this is IMO not the way we provide the best possible Jenkins experience
to users, especially newcomers.

And I don't think we should consider Jenkins Essentials to be the only
place where we make the OOTB experience as smooth as possible.

Do you really see a big issue if we offer a sysprop to revert to the
current behavior, *but* change the default to something more sensible?
I fail to see how hard it would be for people having built monitoring about
Jenkins logs to just temporarily use the sysprop while they adapt their
logstash or whatever parsing rules.
I can even probably provide it for logstash at least.

And again, this is IMO for the good of the project and its users, so even
if there's some (small) disruption this is probably worth it.

Thanks a bunch for your feedback in any case.

-- Baptiste

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS48-5JTQKOSjt9dtFRimET-%3DH9JKGNh0OoWnKwq6QS0Gg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [DISCUSS] Change Jenkins default logging format

[Daniel Beck]

> On 4. Apr 2018, at 21:20, Oleg Nenashev  wrote:
> 
>   • We enable the new logging format in Essentials + immediately get 
> benefit from it (e.g. via Logstash/fluentd)
>   • We offer a feature flag in standard Jenkins. Start from opt-in, then 
> maybe opt-out at some point
>   • For installations passing through the installation wizard, we enable 
> the feature by default (like we do with security defaults now)
> 

I'm conflicted about these. On the surface, this provides a compelling path 
forward with everyone getting what they want. However,

1. If we plan to allow opt-out in the end anyway, why start with opt-in? It's 
not a new subsystem that needs to be proven first. Good defaults are important, 
and letting those in the know who care deeply enable it for themselves might 
just delay the general implementation (perhaps indefinitely).
2. For this to be a setup wizard type distinction, it almost certainly needs to 
be promoted to a UI option (I think it would be the first wizard setting to not 
have one). I'm not positive this is a good idea.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/E4201E29-D362-4A14-BCFB-45D7E9FD9445%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Ouch, that's a shame. It looked like an interesting project, I hope my
writing to you didn't trigger that.

You say "stress builds and also to burn in our build agents" -- can you
elaborate on that? It sounds like you are trying to warm up a cache or
something, but I'm not sure what that means in the context of builds.

On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:

> Hi Kohsuke,
>
> Thanks for getting in touch. I've had to remove that repo temporarily at
> the request of my former employer. It looks like we are trying to solve the
> same problem. I wanted to use jenkinless to stress builds and also to burn
> in our build agents. It's more of a convenience tool. Running the slave
> using docker in docker worked really well. I might try and turn this into
> an executable that can be stored in the repo alongside code.
>
> Tom
>
>
> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  wrote:
>
>> Hi, Thomas,
>>
>> Tyler passed me a link to your project
>> https://github.com/tomwillfixit/jenkinless, which is in a similar space
>> with my project of the day called Jenkinsfile Runner.
>>
>> I haven't studied your project carefully yet, but I already see some
>> interesting ingredients like memcached that I have no idea what you use it
>> for :-)   I'd love to hear from you the philosophy & use cases that led to
>> it. And I'd also love to hear what you think of Jenkinsfile Runner. I think
>> we have similar interests here, are there any opportunity to collaborate?
>>
>> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi  wrote:
>>
>>> And of course I forgot to have the link to the project!
>>> https://github.com/kohsuke/jenkinsfile-runner
>>>
>>>
>>> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi 
>>> wrote:
>>>
 Jenkinsfile Runner is an experiment to package Jenkins pipeline
 execution as a command line tool. The intend use cases include:

- Use Jenkins in Function-as-a-Service context
- Assist editing Jenkinsfile locally
- Integration test shared libraries

 Over the past year, I've done some deep-dive 1:1 conversations with
 some Jenkins users and I felt something like this might move the needle for
 them in an important way.

 I'd love to hear any reactions on your side. Could something like this
 be important for you, does it miss any key points for you? If you mentally
 picture a perfect version of this, what would that do, and how would you
 use?

 Let me know!

 --
 Kohsuke Kawaguchi

>>> --
>>> Kohsuke Kawaguchi
>>>
>> --
>> Kohsuke Kawaguchi
>>
> --
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4w8VgdCKyp2Gz9ZXGjP1AN88Qb-NvWR3dCyP-p8dVkDsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: JEP: Modification of "Accepted" JEPs

[Liam Newman]

Hello all,
I've attempted to synthesize the key points made here into a PR for JEP 1:
https://github.com/jenkinsci/jep/pull/76

The scope of this change expanded a bit, but I think the result is clearer.
Please feel free to comment (or even better to commit edits directly). 

Thanks, 
Liam


On Monday, March 26, 2018 at 1:25:34 PM UTC-7, Liam Newman wrote:
>
> Andrew,
>
> That's an interesting point as well. The process is still relatively new, 
> so it's not surprising that there's a learning curve and a need for more 
> examples. JEPs with tighter focus will definitely move through the process 
> more quickly, since they will require less discussion, design-time, 
> implementation time, and consensus building.   JEP-200 was a good example 
> of that tight focus and it still took some time.
>
> One way that Tyler has been addressing the scope considerations in 
> relation the Jenkins Essentials is splitting the overall project into 
> multiple JEPs.  JEP-300 covers the overall goals and high-level design, and 
> delegates the internal design of those components to sub-JEPs that are 
> being filed as work gets rolling.   I don't know if that means JEP-300 will 
> be accepted sooner or will remain as draft, but it looks like that will be 
> the case.  
>
> It would be nice to have more JEPs filed to for a base of examples we can 
> point people to, but I suspect we'll have to wait for that to grow 
> organically over time.
>
> -L. 
>
>
> On Sat, Mar 24, 2018 at 7:48 AM Andrew Bayer  
> wrote:
>
>> I think it’s more a cultural thing re comfortableness of followup JEPs - 
>> we need to have precedents and examples so that people will feel like oh, 
>> it’s ok that this stuff didn’t get into that JEP, we can just do a new JEP 
>> with it. Leaving proposals open for too long in order to make sure every 
>> possible tangentially related matter gets in is a path to stagnation. We’re 
>> far better off with more JEPs of potentially smaller size/scope, 
>> potentially amending earlier JEPs, than a small number of bloated ones, IMO.
>>
>> A.
>>
>> On Sat, Mar 24, 2018 at 9:34 AM Liam Newman  wrote:
>>
>>> This is a ton of great feedback, thanks!  
>>>
>>> Ewelina,
>>>
>>> JEPs have a number of purposes, but they are definitely _not_ meant to 
>>> stifle innovation.  At minimum though, they are meant to provide a medium 
>>> for building consensus on the design and implementation of major 
>>> features/processes of the Jenkins (and related) project.  
>>>
>>> Without JEP, contributors such as yourself, might do months of work only 
>>> to have that work rejected or asked to be redesigned.  From the other side, 
>>> the Project might get random contributors who ride in and want to have drop 
>>> in some massive features without having discussed and reviewed with the 
>>> rest of the project. 
>>>
>>> I think the main misunderstanding (due to lack of clarity in JEP-1) is 
>>> the idea that a JEP must be "Accepted" in order for contributors to have 
>>> confidence in the value and validity of their work. That is absolutely not 
>>> the case.
>>>
>>> "Accepted" means that that Reviewer and Sponsor have looked at the JEP 
>>> and agreed that _scoping and design_ are complete and have the consensus of 
>>> the community, and that what remains is completing the (already well 
>>> underway) implementation.  "Accepted" is a description of the technical 
>>> state of the proposed component/feature/process.  "Accepted" is _not_ (and 
>>> should not be viewed or used as) a "vote of confidence".  
>>>
>>> Conversely, "Draft" is not a commentary on the likelihood that the JEP 
>>> will eventually be "Accepted".  That can only determined by the Sponsors 
>>> and the Reviewers based on discussion and feedback on the mailing list or 
>>> other forums. "Draft" is _not_ (and should not be viewed as) an indicator 
>>> of any lack of confidence in a proposal. 
>>>
>>> > Now when I see a requirement 
>>> *> "all 'significant changes' to a JEP should be completed before it is 
>>> Accepted"*
>>> > it makes me worry that if I end up working on next JEP, for another 
>>> project, 
>>> > I will be very careful and it will take ages to put it into "Accepted" 
>>> (maybe it's
>>> > not a problem). And then, like with JCasC, we learn while we implement 
>>> it, 
>>> > we discover issues and things that we can't do the way we want to do. 
>>> > Do we have to discover all of that before "Accepting" JEP?
>>>
>>> In short, yes, but as you might gather from my response above, that is 
>>> not a bad thing. 
>>>
>>> In the case of JEP-201, there has been no commentary it indicate that it 
>>> lacks support, nor any doubt about the value of the work being done.  I 
>>> think that the lack of clarity about the meaning of "Accepted" extends to 
>>> the reviewers, so JEP-201 was marked as "Accepted" before the design was 
>>> sufficiently complete.   But I also personally have no doubt that once the 
>>> 

Re: JEP: Modification of "Accepted" JEPs

[Joseph P]

Reading the updated JEP 1 sounds like PR 59 for JEP 201 would pass as minor 
changes which provides clarification.
Unless I am misinterpreting the intend of both pull requests 

Den onsdag den 4. april 2018 kl. 08.42.21 UTC+2 skrev Liam Newman:
>
> Hello all,
> I've attempted to synthesize the key points made here into a PR for JEP 1:
> https://github.com/jenkinsci/jep/pull/76
>
> The scope of this change expanded a bit, but I think the result is clearer.
> Please feel free to comment (or even better to commit edits directly). 
>
> Thanks, 
> Liam
>
>
> On Monday, March 26, 2018 at 1:25:34 PM UTC-7, Liam Newman wrote:
>>
>> Andrew,
>>
>> That's an interesting point as well. The process is still relatively new, 
>> so it's not surprising that there's a learning curve and a need for more 
>> examples. JEPs with tighter focus will definitely move through the process 
>> more quickly, since they will require less discussion, design-time, 
>> implementation time, and consensus building.   JEP-200 was a good example 
>> of that tight focus and it still took some time.
>>
>> One way that Tyler has been addressing the scope considerations in 
>> relation the Jenkins Essentials is splitting the overall project into 
>> multiple JEPs.  JEP-300 covers the overall goals and high-level design, and 
>> delegates the internal design of those components to sub-JEPs that are 
>> being filed as work gets rolling.   I don't know if that means JEP-300 will 
>> be accepted sooner or will remain as draft, but it looks like that will be 
>> the case.  
>>
>> It would be nice to have more JEPs filed to for a base of examples we can 
>> point people to, but I suspect we'll have to wait for that to grow 
>> organically over time.
>>
>> -L. 
>>
>>
>> On Sat, Mar 24, 2018 at 7:48 AM Andrew Bayer > > wrote:
>>
>>> I think it’s more a cultural thing re comfortableness of followup JEPs - 
>>> we need to have precedents and examples so that people will feel like oh, 
>>> it’s ok that this stuff didn’t get into that JEP, we can just do a new JEP 
>>> with it. Leaving proposals open for too long in order to make sure every 
>>> possible tangentially related matter gets in is a path to stagnation. We’re 
>>> far better off with more JEPs of potentially smaller size/scope, 
>>> potentially amending earlier JEPs, than a small number of bloated ones, IMO.
>>>
>>> A.
>>>
>>> On Sat, Mar 24, 2018 at 9:34 AM Liam Newman >> > wrote:
>>>
 This is a ton of great feedback, thanks!  

 Ewelina,

 JEPs have a number of purposes, but they are definitely _not_ meant to 
 stifle innovation.  At minimum though, they are meant to provide a medium 
 for building consensus on the design and implementation of major 
 features/processes of the Jenkins (and related) project.  

 Without JEP, contributors such as yourself, might do months of work 
 only to have that work rejected or asked to be redesigned.  From the other 
 side, the Project might get random contributors who ride in and want to 
 have drop in some massive features without having discussed and reviewed 
 with the rest of the project. 

 I think the main misunderstanding (due to lack of clarity in JEP-1) is 
 the idea that a JEP must be "Accepted" in order for contributors to have 
 confidence in the value and validity of their work. That is absolutely not 
 the case.

 "Accepted" means that that Reviewer and Sponsor have looked at the JEP 
 and agreed that _scoping and design_ are complete and have the consensus 
 of 
 the community, and that what remains is completing the (already well 
 underway) implementation.  "Accepted" is a description of the technical 
 state of the proposed component/feature/process.  "Accepted" is _not_ (and 
 should not be viewed or used as) a "vote of confidence".  

 Conversely, "Draft" is not a commentary on the likelihood that the JEP 
 will eventually be "Accepted".  That can only determined by the Sponsors 
 and the Reviewers based on discussion and feedback on the mailing list or 
 other forums. "Draft" is _not_ (and should not be viewed as) an indicator 
 of any lack of confidence in a proposal. 

 > Now when I see a requirement 
 *> "all 'significant changes' to a JEP should be completed before it is 
 Accepted"*
 > it makes me worry that if I end up working on next JEP, for another 
 project, 
 > I will be very careful and it will take ages to put it into 
 "Accepted" (maybe it's
 > not a problem). And then, like with JCasC, we learn while we 
 implement it, 
 > we discover issues and things that we can't do the way we want to do. 
 > Do we have to discover all of that before "Accepting" JEP?

 In short, yes, but as you might gather from my response above, that is 
 not a bad thing. 

 In the case of JEP-201, 

Re: Install extra Software with plugin installation

[Oleg Nenashev]

Hi Johann,

Apparently not enough.
Could you please provide some examples?

Thanks in advance,
Oleg

On Wednesday, March 28, 2018 at 9:23:18 AM UTC+2, Johann wrote:
>
> Hello guys, 
>
> I am wondering if its possible to install external software automatically 
> while installing a plugin?
>
> Its not much to ask here, so I hope thats enough and you understand my 
> question
>
> Best Regards,
>
> Johann
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/67e11a6e-262b-4564-9754-e8f4e46bf159%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Need help designing a fix for JENKINS-50504

[Oleg Nenashev]

This issue seems to be Pipeline-specific (actually DueableTask-Specific). 
Standard Freestyle jobs should abort immediately on the agent 
disconnection, but Pipeline jobs may recover and continue using the 
workspace.

However, it seems ugly to use the new channel in the "in use" map, because 
> the job is still technically running under the old channel.


No, it should be running under the new channel. Old channel gets disposed, 
and Remoting 3.14+ adds some diagnostics for these cases (e.g. JENKINS-45294 
). Now it causes some 
issues in Durable task which does not always recreate FilePath and 
underlying Workspace (JENKINS-41854 
 and other similar 
issues with "Channel is closing or closed").

WorkspaceList#inUse should be reacquired by Pipeline for sure when it 
reconnects to a new agent. I would guess it happens even now (or not?), but 
clearly there is a potential of race conditions between recovered jobs and 
new submissions.

The proposed patch may help, although workspace management is not really 
the strongest part of the Jenkins core. I would rather suggest redesigning 
it so that workspaces can be tracked independently on the node state (the 
proposed change does the same for a single cache). Some better UI/ 
workspace release features could be added as an added value.

BR, Oleg
 

On Monday, April 2, 2018 at 10:08:28 PM UTC+2, m...@basilcrow.com wrote:
>
> Hi Ivan,
>
> Thanks for your reply. I'm not exactly sure how my proposed workaround 
> would necessarily cause concurrency issues. Doesn't that depend on how it's 
> implemented? I agree that it's strange that the agent wasn't disconnected 
> and still keeps the old connection to the master, even though new jobs use 
> a new connection. Doesn't this violate the invariant implied by the 
> implementation of WorkspaceList#inUse, which is that the entries in the map 
> always represent the latest channel for a given node? This definitely seems 
> like a core bug to me. I don't believe I should need to tune my TCP stack, 
> because pipeline claims to be resilient to network outages. If the master 
> logs "SEVERE: I/O error in channel jenkins-node" and "INFO: Attempting to 
> reconnect jenkins-node", then why do jobs continue running on the old 
> connection, violating the invariant in WorkspaceList#inUse?
>
> Thanks,
> Basil
>
> On Sunday, April 1, 2018 at 6:47:02 AM UTC-7, Ivan Fernandez Calvo wrote:
>>
>> The pruposed workaround could cause concurrence issues, I think the the 
>> main issue why the agent is not disconnected and keep the old connection is 
>> the most important thing. Did you checked the open connection from the 
>> Agent to the master with netstat? It should be two connections the old one 
>> an an new one, Has the  agent more than one slave.jar process running? Are 
>> your agents VM or baremetal? Did you tune your tcp stack with proper values 
>> to keepalive?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/66d428d6-c7c1-48fb-ab8a-4b7b7236a1eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Fixing bug where git project is built twice when changes are pushed

[thomas]

My project is suffering from a problem where jobs sometimes build the same 
commit twice. This seems to happen when two or more pushes are made within 
a short time (by one or more developers).

I think the problem matches one or both of these:

https://issues.jenkins-ci.org/browse/JENKINS-17346
https://issues.jenkins-ci.org/browse/JENKINS-30350

My job is a multi-configuration job configured for pre-build branch merging 
as describe here: 
https://plugins.jenkins.io/git#GitPlugin-UsingGit,Jenkinsandpre-buildbranchmerging

My full job config.xml can be found here: https://pastebin.com/BqBUQEpe

I could use some help figuring out why this happens. Here is what I have 
done so far. I've checked out bitbucket-plugin to start from the trigger. 
Here, I've drilled down to BitBucketTrigger.onPost() which appears to be 
where the interesting stuff is happening.

public void onPost(String triggeredByUser, final String payload) {
final String pushBy = triggeredByUser;
getDescriptor().queue.execute(new Runnable() {
private boolean runPolling() {
try {
StreamTaskListener listener = new StreamTaskListener(
getLogFile());
try {
PrintStream logger = listener.getLogger();
long start = System.currentTimeMillis();
logger.println("Started on "+ DateFormat.
getDateTimeInstance().format(new Date()));
boolean result = SCMTriggerItem.SCMTriggerItems.
asSCMTriggerItem(job).poll(listener).hasChanges();
logger.println("Done. Took "+ Util.getTimeSpanString
(System.currentTimeMillis()-start));
if(result)
logger.println("Changes found");
else
logger.println("No changes");
return result;
} catch (Error e) {
e.printStackTrace(listener.error("Failed to record 
SCM polling"));
LOGGER.log(Level.SEVERE,"Failed to record SCM 
polling",e);
throw e;
} catch (RuntimeException e) {
e.printStackTrace(listener.error("Failed to record 
SCM polling"));
LOGGER.log(Level.SEVERE,"Failed to record SCM 
polling",e);
throw e;
} finally {
listener.close();
}
} catch (IOException e) {
LOGGER.log(Level.SEVERE,"Failed to record SCM polling",e
);
}
return false;
}

public void run() {
if (runPolling()) {
String name = " #"+job.getNextBuildNumber();
BitBucketPushCause cause;
try {
cause = new BitBucketPushCause(getLogFile(), pushBy
);
} catch (IOException e) {
LOGGER.log(Level.WARNING, "Failed to parse the 
polling log",e);
cause = new BitBucketPushCause(pushBy);
}
ParameterizedJobMixIn pJob = new ParameterizedJobMixIn() 
{
@Override protected Job asJob() {
return job;
}
};
BitBucketPayload bitBucketPayload = new BitBucketPayload
(payload);
pJob.scheduleBuild2(5, new CauseAction(cause), 
bitBucketPayload);
if (pJob.scheduleBuild(cause)) {
LOGGER.info("SCM changes detected in "+ job.getName
()+". Triggering "+ name);
} else {
LOGGER.info("SCM changes detected in "+ job.getName
()+". Job is already in the queue");
}
}
}

});
}

The things my novice eyes notice in the function above is that it is a 
Runnable that is added to a getDescriptor().queue. Does this mean that only 
one of these can be processed at a time, or is it conceivable that two 
quick git pushes in succession can cause two of these runnables to execute 
at the same time.

The other thing I notice is that if pJob.scheduleBuild(cause) returns 
something that evaluates to false then we log a message that the job is 
already in the queue. Is a job in the queue when it is executing or does 
being in the queue mean that it is scheduled to be executed but hasn't 
startet yet?

Finally, I have a question about this statement from the function above:

   boolean result = 
SCMTriggerItem.SCMTriggerItems.asSCMTriggerItem(job).poll(listener).hasChanges();

hasChanges() relative to what? What happens if a job has been scheduled and 
it has started executing, but it has not decided yet what git sha or indeed 
even what git branch to build? Does that leave a window