PSA for openshift users - dockerhub will be down for scheduled outage on August 25th

[Clayton Coleman]

Please see https://status.docker.com/ for times.

Remember, if you have autoscaling nodes that need to pull new apps, or have
pods that run with PullAlways, or push builds to the docker hub, while the
hub is down those operations will fail.

Mitigations could include:

1. Disable autoscaling for the duration
2. Use the image mirroring and transparent proxying feature of the
openshift integrated registry (switch the resolutionPolicy for your image
streams to Local on 3.9 or later) to automatically mirror remote images and
serve them from the local registry
3. Disable PullAlways from any deployed workloads so you can leverage
cached local images (if a pod tries to restart while the registry is down
and pull always is set, the new container won’t be started).
4. Push to a different registry than dockerhub, like the integrated
registry or quay.io
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Openshift centralized logging - add custom container logfiles

[Aleksandar Lazic]

Hi.

Am 16.08.2018 um 16:27 schrieb Rich Megginson:
> On 08/16/2018 05:42 AM, Aleksandar Lazic wrote:
>> Am 16.08.2018 um 12:48 schrieb Aleksandar Kostadinov:
>>> Might be real nice to allow pod to request sockets created where different 
>>> log
>>> streams can be sent to central logging without extra containers in the pod.
>> You can run socklog/fluentbit/... in the background to handle the logging and
>> your app logs to this socket.
>
> So you would need to configure your app to log to a socket instead of a log 
> file?
> Where does socklog write the logs?  Who reads from that destination?

Socklog writes to stdout by default.
In my setup is the haproxy configured to write to the unix socket but he can
also listen to udp socket.
In any case the output is written to stdout

http://smarden.org/socklog/

I have describe the setup in two blog posts
https://www.me2digital.com/blog/2017/05/syslog-in-a-container-world/
https://www.me2digital.com/blog/2017/09/syslog-receiver/

Another possible tool is https://fluentbit.io/ as it can use more input sources.
https://fluentbit.io/documentation/0.13/input/

For example you can use tail if it's not possible to change easily the logging
setup of the app.
https://fluentbit.io/documentation/0.13/input/tail.html

In the past was the rsyslog hard to setup for openshift with normal privileges
from the rhel image, that was the reason for me to build this solution, imho.
The https://www.rsyslog.com/doc/v8-stable/configuration/modules/omstdout.html is
documented to not use it in real deployments

Best Regards
Aleks

>> Something similar as I have done it in my haproxy image.
>>
>> https://gitlab.com/aleks001/haproxy18-centos/blob/master/containerfiles/container-entrypoint.sh#L92-93
>>
>>
>> ###
>> ...
>> echo "starting socklog"
>> /usr/local/bin/socklog unix /tmp/haproxy_syslog &
>> ...
>> ###
>>
>> Regards
>> Aleks
>>> Jeff Cantrill wrote on 08/15/18 16:50:
 The recommended options with the current log stack are either to 
 reconfigure
 your log to send to stdout or add a sidecar container that is capable of
 tailing the log in question which would write it to stdout and ultimately
 read by fluentd.

 On Wed, Aug 15, 2018 at 2:47 AM, Leo David >>> > wrote:

  Hi Everyone,
  I have logging with fluentd / elasticsearch at cluster level running
  fine,  everything works as expected.
  I have an issue though...
  What would it be the procedure to add some custom log files from
  different containers ( logs that are not shown in stdout ) to be
  delivered to elasticseach as well ?
  I two different clusters ( 3.7 and 3.9 ) up and running,  and i know
  that in 3.7 docker logging driver is configured with journald whilst
  in 3.9 is json-file.
  Any thoughts on this ?
  Thanks a lot !

  --     Best regards, Leo David

 -- 
 -- 
 Jeff Cantrill
 Senior Software Engineer, Red Hat Engineering
 OpenShift Logging
 Red Hat, Inc.
 *Office*: 703-748-4420 | 866-546-8970 ext. 8162420
 jcant...@redhat.com 
 http://www.redhat.com


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Automating overriding of openshift-sync Jenkins plugin configuration

[Gabe Montero]

On Thu, Aug 16, 2018 at 2:37 PM Andrew Feller  wrote:

> Yeah, I think it's this one
> .
> Depending on the level of effort, I might be able to potentially contribute
> on a fix for it.  I didn't know if anyone has a workaround other than
> mounting a Groovy script for Jenkins to load.
>

Yeah a groovy init script is the best workaround until the referenced RFE
happens.


> On Thu, Aug 16, 2018 at 1:41 PM Ben Parees  wrote:
>
>>
>>
>> On Thu, Aug 16, 2018 at 10:52 AM, Andrew Feller 
>> wrote:
>>
>>> Is anyone familiar with overriding the openshift-sync plugin namespace
>>> property?
>>>
>>> I'm working to condense the Jenkins instances we have running with a
>>> central Jenkins running in OpenShift pulling BuildConfigs from remote
>>> namespaces.  However, I rather not manually configure Jenkins
>>> openshift-sync plugin on the namespaces it should be monitoring and hoping
>>> there is a better way than crafting a Groovy script to be loaded by Jenkins
>>> on startup to override this.
>>>
>>
>> Pretty sure this has come up before and might even exist as an RFE
>> (namely the ability to set the list of namespaces to watch via an env var)
>> but i don't think anything has been implemented yet.  Gabe would know for
>> sure.
>>
>>
>>
>>>
>>> Thanks!
>>> --
>>>
>>> [image: BandwidthMaroon.png]
>>>
>>> Andy Feller  •  Sr DevOps Engineer
>>>
>>> 900 Main Campus Drive, Suite 500, Raleigh, NC 27606
>>> 
>>>
>>>
>>> e: afel...@bandwidth.com
>>>
>>> ___
>>> users mailing list
>>> users@lists.openshift.redhat.com
>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>>>
>>>
>>
>>
>> --
>> Ben Parees | OpenShift
>>
>>
>
> --
>
> [image: BandwidthMaroon.png]
>
> Andy Feller  •  Sr DevOps Engineer
>
> 900 Main Campus Drive, Suite 500, Raleigh, NC 27606
> 
>
>
> e: afel...@bandwidth.com
>
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Automating overriding of openshift-sync Jenkins plugin configuration

[Andrew Feller]

Yeah, I think it's this one
.
Depending on the level of effort, I might be able to potentially contribute
on a fix for it.  I didn't know if anyone has a workaround other than
mounting a Groovy script for Jenkins to load.

On Thu, Aug 16, 2018 at 1:41 PM Ben Parees  wrote:

>
>
> On Thu, Aug 16, 2018 at 10:52 AM, Andrew Feller 
> wrote:
>
>> Is anyone familiar with overriding the openshift-sync plugin namespace
>> property?
>>
>> I'm working to condense the Jenkins instances we have running with a
>> central Jenkins running in OpenShift pulling BuildConfigs from remote
>> namespaces.  However, I rather not manually configure Jenkins
>> openshift-sync plugin on the namespaces it should be monitoring and hoping
>> there is a better way than crafting a Groovy script to be loaded by Jenkins
>> on startup to override this.
>>
>
> Pretty sure this has come up before and might even exist as an RFE (namely
> the ability to set the list of namespaces to watch via an env var) but i
> don't think anything has been implemented yet.  Gabe would know for sure.
>
>
>
>>
>> Thanks!
>> --
>>
>> [image: BandwidthMaroon.png]
>>
>> Andy Feller  •  Sr DevOps Engineer
>>
>> 900 Main Campus Drive, Suite 500, Raleigh, NC 27606
>>
>>
>> e: afel...@bandwidth.com
>>
>> ___
>> users mailing list
>> users@lists.openshift.redhat.com
>> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>>
>>
>
>
> --
> Ben Parees | OpenShift
>
>

-- 

[image: BandwidthMaroon.png]

Andy Feller  •  Sr DevOps Engineer

900 Main Campus Drive, Suite 500, Raleigh, NC 27606


e: afel...@bandwidth.com
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Automating overriding of openshift-sync Jenkins plugin configuration

[Ben Parees]

On Thu, Aug 16, 2018 at 10:52 AM, Andrew Feller 
wrote:

> Is anyone familiar with overriding the openshift-sync plugin namespace
> property?
>
> I'm working to condense the Jenkins instances we have running with a
> central Jenkins running in OpenShift pulling BuildConfigs from remote
> namespaces.  However, I rather not manually configure Jenkins
> openshift-sync plugin on the namespaces it should be monitoring and hoping
> there is a better way than crafting a Groovy script to be loaded by Jenkins
> on startup to override this.
>

Pretty sure this has come up before and might even exist as an RFE (namely
the ability to set the list of namespaces to watch via an env var) but i
don't think anything has been implemented yet.  Gabe would know for sure.



>
> Thanks!
> --
>
> [image: BandwidthMaroon.png]
>
> Andy Feller  •  Sr DevOps Engineer
>
> 900 Main Campus Drive, Suite 500, Raleigh, NC 27606
>
>
> e: afel...@bandwidth.com
>
> ___
> users mailing list
> users@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>
>


-- 
Ben Parees | OpenShift
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Automating overriding of openshift-sync Jenkins plugin configuration

[Andrew Feller]

Is anyone familiar with overriding the openshift-sync plugin namespace
property?

I'm working to condense the Jenkins instances we have running with a
central Jenkins running in OpenShift pulling BuildConfigs from remote
namespaces.  However, I rather not manually configure Jenkins
openshift-sync plugin on the namespaces it should be monitoring and hoping
there is a better way than crafting a Groovy script to be loaded by Jenkins
on startup to override this.

Thanks!
-- 

[image: BandwidthMaroon.png]

Andy Feller  •  Sr DevOps Engineer

900 Main Campus Drive, Suite 500, Raleigh, NC 27606


e: afel...@bandwidth.com
___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Logging / Kibana export Logs

[Rich Megginson]

You could expose Elasticsearch externally 
https://docs.okd.io/latest/install_config/aggregate_logging.html



|openshift_logging_es_allow_external|

Set to |true| to expose Elasticsearch as a reencrypt route. Set to 
|false| by default.




Except that username/password and token auth is currently broken due to 
the oauth proxy.


On 08/16/2018 07:16 AM, Tobias Brunner wrote:

Hi,

Does anyone have an idea how logs could be exported from the OpenShift
integrated logging for further analysis? Constraints: We can't give the
users access to the logging namespace and therefore also not to the
Elasticsearch Pod as this would allow the user to bypass access control
(Searchguard).

Thanks,
Tobias

___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users



___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Openshift centralized logging - add custom container logfiles

[Rich Megginson]

On 08/16/2018 05:42 AM, Aleksandar Lazic wrote:

Am 16.08.2018 um 12:48 schrieb Aleksandar Kostadinov:

Might be real nice to allow pod to request sockets created where different log
streams can be sent to central logging without extra containers in the pod.

You can run socklog/fluentbit/... in the background to handle the logging and
your app logs to this socket.


So you would need to configure your app to log to a socket instead of a 
log file?

Where does socklog write the logs?  Who reads from that destination?


Something similar as I have done it in my haproxy image.

https://gitlab.com/aleks001/haproxy18-centos/blob/master/containerfiles/container-entrypoint.sh#L92-93

###
...
echo "starting socklog"
/usr/local/bin/socklog unix /tmp/haproxy_syslog &
...
###

Regards
Aleks

Jeff Cantrill wrote on 08/15/18 16:50:

The recommended options with the current log stack are either to reconfigure
your log to send to stdout or add a sidecar container that is capable of
tailing the log in question which would write it to stdout and ultimately
read by fluentd.

On Wed, Aug 15, 2018 at 2:47 AM, Leo David mailto:leoa...@gmail.com>> wrote:

     Hi Everyone,
     I have logging with fluentd / elasticsearch at cluster level running
     fine,  everything works as expected.
     I have an issue though...
     What would it be the procedure to add some custom log files from
     different containers ( logs that are not shown in stdout ) to be
     delivered to elasticseach as well ?
     I two different clusters ( 3.7 and 3.9 ) up and running,  and i know
     that in 3.7 docker logging driver is configured with journald whilst
     in 3.9 is json-file.
     Any thoughts on this ?
     Thanks a lot !

     --     Best regards, Leo David

     ___
     users mailing list
     users@lists.openshift.redhat.com
     
     http://lists.openshift.redhat.com/openshiftmm/listinfo/users
     




--
--
Jeff Cantrill
Senior Software Engineer, Red Hat Engineering
OpenShift Logging
Red Hat, Inc.
*Office*: 703-748-4420 | 866-546-8970 ext. 8162420
jcant...@redhat.com 
http://www.redhat.com


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users



___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Openshift centralized logging - add custom container logfiles

[Aleksandar Lazic]

Am 16.08.2018 um 12:48 schrieb Aleksandar Kostadinov:
> Might be real nice to allow pod to request sockets created where different log
> streams can be sent to central logging without extra containers in the pod.

You can run socklog/fluentbit/... in the background to handle the logging and
your app logs to this socket.
Something similar as I have done it in my haproxy image.

https://gitlab.com/aleks001/haproxy18-centos/blob/master/containerfiles/container-entrypoint.sh#L92-93

###
...
echo "starting socklog"
/usr/local/bin/socklog unix /tmp/haproxy_syslog &
...
###

Regards
Aleks
> Jeff Cantrill wrote on 08/15/18 16:50:
>> The recommended options with the current log stack are either to reconfigure
>> your log to send to stdout or add a sidecar container that is capable of
>> tailing the log in question which would write it to stdout and ultimately
>> read by fluentd.
>>
>> On Wed, Aug 15, 2018 at 2:47 AM, Leo David > > wrote:
>>
>>     Hi Everyone,
>>     I have logging with fluentd / elasticsearch at cluster level running
>>     fine,  everything works as expected.
>>     I have an issue though...
>>     What would it be the procedure to add some custom log files from
>>     different containers ( logs that are not shown in stdout ) to be
>>     delivered to elasticseach as well ?
>>     I two different clusters ( 3.7 and 3.9 ) up and running,  and i know
>>     that in 3.7 docker logging driver is configured with journald whilst
>>     in 3.9 is json-file.
>>     Any thoughts on this ?
>>     Thanks a lot !
>>
>>     --     Best regards, Leo David
>>
>>     ___
>>     users mailing list
>>     users@lists.openshift.redhat.com
>>     
>>     http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>>     
>>
>>
>>
>>
>> -- 
>> -- 
>> Jeff Cantrill
>> Senior Software Engineer, Red Hat Engineering
>> OpenShift Logging
>> Red Hat, Inc.
>> *Office*: 703-748-4420 | 866-546-8970 ext. 8162420
>> jcant...@redhat.com 
>> http://www.redhat.com
>>
>>
>> ___
>> users mailing list
>> users@lists.openshift.redhat.com
>> http://lists.openshift.redhat.com/openshiftmm/listinfo/users
>>
>
> ___
> users mailing list
> users@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Re: Openshift centralized logging - add custom container logfiles

[Aleksandar Kostadinov]

Might be real nice to allow pod to request sockets created where 
different log streams can be sent to central logging without extra 
containers in the pod.


Jeff Cantrill wrote on 08/15/18 16:50:
The recommended options with the current log stack are either to 
reconfigure your log to send to stdout or add a sidecar container that 
is capable of tailing the log in question which would write it to stdout 
and ultimately read by fluentd.


On Wed, Aug 15, 2018 at 2:47 AM, Leo David > wrote:


Hi Everyone,
I have logging with fluentd / elasticsearch at cluster level running
fine,  everything works as expected.
I have an issue though...
What would it be the procedure to add some custom log files from
different containers ( logs that are not shown in stdout ) to be
delivered to elasticseach as well ?
I two different clusters ( 3.7 and 3.9 ) up and running,  and i know
that in 3.7 docker logging driver is configured with journald whilst
in 3.9 is json-file.
Any thoughts on this ?
Thanks a lot !

-- 
Best regards, Leo David


___
users mailing list
users@lists.openshift.redhat.com

http://lists.openshift.redhat.com/openshiftmm/listinfo/users





--
--
Jeff Cantrill
Senior Software Engineer, Red Hat Engineering
OpenShift Logging
Red Hat, Inc.
*Office*: 703-748-4420 | 866-546-8970 ext. 8162420
jcant...@redhat.com 
http://www.redhat.com


___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users



___
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users