That was it! Adding proxy_buffering off helped.
For reference, here is my SSL vhost definition:
server {
listen 443 ssl;
server_name jenkins.my-domain;
ssl_certificate ssl/my-domain.crt;
ssl_certificate_key ssl/my-domain.key;
ssl_dhparam ssl/dhparam-2048.pem;
add_header Strict-Transport-Security "max-age=31536000;
includeSubDomains";
add_header X-Frame-Options SAMEORIGIN;
access_log /var/log/nginx/jenkins.my-domain.access.log;
error_log /var/log/nginx/jenkins.my-domain.error.log;
location / {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect http:// https://;
proxy_pass http://jenkins;
# Required for new HTTP-based CLI
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
# workaround for
https://issues.jenkins-ci.org/browse/JENKINS-45651
# not used for this installation
#add_header 'X-SSH-Endpoint' 'jenkins.my-domain:22' always;
}
}
On Friday, October 27, 2017 at 6:56:55 AM UTC+9, Devin Nusbaum wrote:
>
> Some comments on https://issues.jenkins-ci.org/browse/JENKINS-43666 suggest
> that proxy_buffering off; is needed for SSL connections. (I was also
> testing nginx reverse proxies in the past and have that setting in my SSL
> config; might have forgotten to update the SSL docs with my findings) Try
> setting that option, and if it works I can update the SSL section of the
> wiki page.
>
> On Oct 26, 2017, at 5:43 PM, 'Tomasz Chmielewski' via Jenkins Users <
> [email protected] <javascript:>> wrote:
>
> nginx vhost is almost the exact copy of the vhost on
> https://wiki.jenkins.io/display/JENKINS/Jenkins+behind+an+NGinX+reverse+proxy:
>
> upstream jenkins {
> server 127.0.0.1:8080 fail_timeout=0;
> }
>
> server {
>
> listen 80;
> server_name jenkins.my-domain;
>
> add_header X-Frame-Options SAMEORIGIN;
> include /etc/nginx/release.conf;
>
> access_log /var/log/nginx/redirects-access.log vhosts;
> error_log /var/log/nginx/redirects-error.log;
>
> rewrite ^ https://$host$request_uri? permanent;
> }
>
>
> server {
>
> listen 443 ssl;
>
> server_name jenkins.my-domain;
>
> ssl_certificate ssl/my-domain.crt;
> ssl_certificate_key ssl/my-domain.key;
> ssl_dhparam ssl/dhparam-2048.pem;
> add_header Strict-Transport-Security "max-age=31536000;
> includeSubDomains";
> add_header X-Frame-Options SAMEORIGIN;
>
> access_log /var/log/nginx/jenkins.my-domain.access.log;
> error_log /var/log/nginx/jenkins.my-domain.error.log;
>
> location / {
> proxy_set_header Host $host:$server_port;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For
> $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_redirect http:// https://;
> proxy_pass http://jenkins;
> # Required for new HTTP-based CLI
> proxy_http_version 1.1;
> proxy_request_buffering off;
> # workaround for
> https://issues.jenkins-ci.org/browse/JENKINS-45651
> add_header 'X-SSH-Endpoint' 'jenkins.my-domain:22' always;
> }
> }
>
>
> So either I'm blind, or the documentation is somehow wrong?
>
> And indeed, I can see "java.io.IOException: HTTP full-duplex channel
> timeout" in jenkins log.
>
> This one indeed works:
>
> java -jar jenkins-cli.jar -s http://localhost:8080 <https://jenkins-url/>
> -auth
> user:pass help offline-node
>
> But since I need to execute it from remote, I'd rather connect to
> https://jenkins.my-domain
>
>
> On Friday, October 27, 2017 at 6:26:39 AM UTC+9, Devin Nusbaum wrote:
>>
>> Make sure to follow
>> https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Nginx if
>> Nginx is configured as a a reverse proxy.
>>
>> Notably proxy_http_version 1.1; and proxy_request_buffering off; are
>> required for your version of Jenkins. (If your Jenkins logs at the time you
>> try to connect via CLI have errors that say something to the effect
>> of “Full-duplex channel timeout” then I expect those settings to fix it.)
>>
>> On Oct 26, 2017, at 5:18 PM, 'Tomasz Chmielewski' via Jenkins Users <
>> jenkins...@googlegroups. <http://googlegroups.com/>com
>> <http://googlegroups.com/>> wrote:
>>
>> Except... it doesn't seem to work.
>>
>> $ java -jar jenkins-cli.jar -s https://jenkins-url -auth user:pass help
>> offline-node
>> $ echo $?
>> 255
>>
>> In nginx log:
>>
>> 10.11.0.8 - user [26/Oct/2017:21:11:51 +0000] "GET / HTTP/1.1" 200 150393
>> "-" "Java/1.8.0_131"
>> 10.11.0.8 - user [26/Oct/2017:21:11:52 +0000] "GET
>> /crumbIssuer/api/xml/?xpath=concat(//crumbRequestField,\x22:\x22,//crumb)
>> HTTP/1.1" 404 335 "-" "Java/1.8.0_131"
>> 10.11.0.8 - user [26/Oct/2017:21:12:07 +0000] "POST /cli?remoting=false
>> HTTP/1.1" 200 11 "-" "Java/1.8.0_131"
>> 10.11.0.8 - user [26/Oct/2017:21:12:07 +0000] "POST /cli?remoting=false
>> HTTP/1.1" 500 13912 "-" "Java/1.8.0_131"
>>
>> How do I debug this?
>>
>>
>>
>> On Friday, October 27, 2017 at 6:07:03 AM UTC+9, Tomasz Chmielewski wrote:
>>>
>>> Got it, thanks:
>>>
>>> https://wiki.jenkins.io/display/JENKINS/Jenkins+CLI
>>>
>>> On Friday, October 27, 2017 at 5:57:18 AM UTC+9, Robert Hales wrote:
>>>>
>>>> You have to use the Jenkins CLI. I guess that can be a bit confusing.
>>>> It isn't a script available to run at the command line. Jenkins has their
>>>> own CLI. If you google for it, you will find the details pretty easily.
>>>>
>>>> On Thursday, October 26, 2017 at 2:55:07 PM UTC-6, Tomasz Chmielewski
>>>> wrote:
>>>>>
>>>>> Hmm, where do I find "offline-node" command?
>>>>>
>>>>> root@jenkins:~# dpkg -L jenkins
>>>>> /.
>>>>> /usr
>>>>> /usr/share
>>>>> /usr/share/doc
>>>>> /usr/share/doc/jenkins
>>>>> /usr/share/doc/jenkins/changelog.gz
>>>>> /usr/share/doc/jenkins/copyright
>>>>> /usr/share/jenkins
>>>>> /usr/share/jenkins/jenkins.war
>>>>> /etc
>>>>> /etc/logrotate.d
>>>>> /etc/logrotate.d/jenkins
>>>>> /etc/default
>>>>> /etc/default/jenkins
>>>>> /etc/init.d
>>>>> /etc/init.d/jenkins
>>>>> /var
>>>>> /var/cache
>>>>> /var/cache/jenkins
>>>>> /var/lib
>>>>> /var/lib/jenkins
>>>>> /var/log
>>>>> /var/log/jenkins
>>>>>
>>>>> root@jenkins:~# find / -name offline-node
>>>>>
>>>>> root@jenkins:~#
>>>>>
>>>>> root@jenkins:~# dpkg -l | grep jenkins
>>>>> ii jenkins 2.73.2 (...)
>>>>>
>>>>>
>>>>>
>>>>> On Friday, October 27, 2017 at 12:21:17 AM UTC+9, Robert Hales wrote:
>>>>>>
>>>>>> In the CLI, use the 'offline-node' command. Another useful command in
>>>>>> what it looks like you want to do might be "wait-offline-node".
>>>>>>
>>>>>> You could also create a groovy script to do it and run that from the
>>>>>> REST API.
>>>>>>
>>>>>> On Thursday, October 26, 2017 at 3:35:29 AM UTC-6, Tomasz Chmielewski
>>>>>> wrote:
>>>>>>>
>>>>>>> Is there a CLI/scripted way to stop scheduling any new builds on a
>>>>>>> given node?
>>>>>>>
>>>>>>> Basically, any builds currently running on a given node should
>>>>>>> continue to run until they are finished -- and no new builds should be
>>>>>>> started.
>>>>>>>
>>>>>>> Think of "retiring" a node, and replacing it with a new one -- but
>>>>>>> allowing any existing jobs to finish gracefully.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Tomasz Chmielewski
>>>>>>> https://lxadm.com
>>>>>>>
>>>>>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/a19302b1-6ed1-44bb-b65b-28868a64708b%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jenkinsci-users/a19302b1-6ed1-44bb-b65b-28868a64708b%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/476c0ec2-753f-45bd-944b-2f9dcf60deae%40googlegroups.com
>
> <https://groups.google.com/d/msgid/jenkinsci-users/476c0ec2-753f-45bd-944b-2f9dcf60deae%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/cb454333-dddf-4186-ac97-66550f856454%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.