Re: [kubernetes-users] Current status of GPU support in Kubernetes

['Vishnu Kannan' via Kubernetes user discussion and Q]

Clusters are not ready to go just yet. please sign up via this link -
https://goo.gl/forms/HR0Upm9w30DW8aIU2

On Oct 25, 2017 8:11 AM, "Itamar O" <itamar...@gmail.com> wrote:

> I already took that cluster down, so I'll reproduce it later and run
> `kubectl describe no`.
> What user guide are you referring to? I was under the impression that
> using GKE with the Google optimized images gives me ready-to-go-machines.
> The user guide I'm familiar with talks about installing GPU drivers on GCE
> instances that I spin up on my own, not GKE-managed instances.
>
> On Wed, Oct 25, 2017 at 5:25 PM 'Vishnu Kannan' via Kubernetes user
> discussion and Q <kubernetes-users@googlegroups.com> wrote:
>
>> Can you post output of 'kubectl describe no'? Did you follow the user
>> guide to install drivers?
>>
>> On Oct 25, 2017 7:23 AM, "Itamar O" <itamar...@gmail.com> wrote:
>>
>>> Vishnu,
>>> I have an alpha cluster on GKE with GPU (project is whitelisted, running
>>> 1.7.8), but I am unable to schedule workloads that require GPUs:
>>> [image: unnamed.png]
>>>
>>> example YAML for reproducing this:
>>> apiVersion: apps/v1beta1
>>> kind: Deployment
>>> metadata:
>>>   name: my-tf
>>>   labels:
>>> app: tf
>>> spec:
>>>   selector:
>>> matchLabels:
>>>   app: tf
>>>   replicas: 1
>>>   template:
>>> metadata:
>>>   labels:
>>> app: tf
>>> spec:
>>>   containers:
>>>   - image: tensorflow/tensorflow:latest-gpu
>>>     name: my-tf
>>> ports:
>>> - containerPort: 
>>> resources:
>>>   limits:
>>> *alpha.kubernetes.io/nvidia-gpu
>>> <http://alpha.kubernetes.io/nvidia-gpu>: 1*
>>>
>>> Am I missing something?
>>> Thanks.
>>>
>>> On Mon, Oct 23, 2017 at 11:00 PM 'Vishnu Kannan' via Kubernetes user
>>> discussion and Q <kubernetes-users@googlegroups.com> wrote:
>>>
>>>> Hey there, if you are trying out alpha experience for GPUs on GKE,
>>>> please sign up via this form <https://goo.gl/forms/HR0Upm9w30DW8aIU2>
>>>> and request access to the doc listed in the form. If you have issues with
>>>> GKE after going through the user guide, please PM me and I'll help you out.
>>>>
>>>> On Mon, Oct 23, 2017 at 12:48 AM, <kt...@cogent.co.jp> wrote:
>>>>
>>>>> Hi Vishnu,
>>>>>
>>>>> I am also trying to run a GPU cluster on GKE. And also hitting
>>>>> allocatable GPU = 0 problem. Can you please guide me?
>>>>>
>>>>> On Tuesday, October 17, 2017 at 6:36:20 AM UTC+9, Vishnu Kannan wrote:
>>>>> > GPU support in kubernetes is moving out of tree using a new
>>>>> extension mechanism called device plugins. Nvidia has published a GPU
>>>>> device plugin recently that is expected to work with kubernetes v1.8.
>>>>> >
>>>>> >
>>>>> > If you are on GCP, please reach out to me and I can share an
>>>>> official alpha user guide for GPUs with k8s.
>>>>> >
>>>>> >
>>>>> > On Mon, Oct 16, 2017 at 12:54 AM,  <burtc...@gmail.com> wrote:
>>>>> >
>>>>> >
>>>>> > Hi Vu Pham,
>>>>> >
>>>>> > My allocatable GPU number is 0.
>>>>> >
>>>>> > Can you advise how you overcome the issue?
>>>>> >
>>>>> > Thanks, Burt
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> >
>>>>> > You received this message because you are subscribed to the Google
>>>>> Groups "Kubernetes user discussion and Q" group.
>>>>> >
>>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>>> send an email to kubernetes-use...@googlegroups.com.
>>>>> >
>>>>> > To post to this group, send email to kubernet...@googlegroups.com.
>>>>> >
>>>>> > Visit this group at https://groups.google.com/group/kubernetes-users
>>>>> .
>>>>> >
>>>>> > For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Current status of GPU support in Kubernetes

['Vishnu Kannan' via Kubernetes user discussion and Q]

Hey there, if you are trying out alpha experience for GPUs on GKE, please sign
up via this form  and request
access to the doc listed in the form. If you have issues with GKE after
going through the user guide, please PM me and I'll help you out.

On Mon, Oct 23, 2017 at 12:48 AM,  wrote:

> Hi Vishnu,
>
> I am also trying to run a GPU cluster on GKE. And also hitting allocatable
> GPU = 0 problem. Can you please guide me?
>
> On Tuesday, October 17, 2017 at 6:36:20 AM UTC+9, Vishnu Kannan wrote:
> > GPU support in kubernetes is moving out of tree using a new extension
> mechanism called device plugins. Nvidia has published a GPU device plugin
> recently that is expected to work with kubernetes v1.8.
> >
> >
> > If you are on GCP, please reach out to me and I can share an official
> alpha user guide for GPUs with k8s.
> >
> >
> > On Mon, Oct 16, 2017 at 12:54 AM,   wrote:
> >
> >
> > Hi Vu Pham,
> >
> > My allocatable GPU number is 0.
> >
> > Can you advise how you overcome the issue?
> >
> > Thanks, Burt
> >
> >
> >
> >
> >
> > --
> >
> > You received this message because you are subscribed to the Google
> Groups "Kubernetes user discussion and Q" group.
> >
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to kubernetes-use...@googlegroups.com.
> >
> > To post to this group, send email to kubernet...@googlegroups.com.
> >
> > Visit this group at https://groups.google.com/group/kubernetes-users.
> >
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Current status of GPU support in Kubernetes

['Vishnu Kannan' via Kubernetes user discussion and Q]

GPU support in kubernetes is moving out of tree using a new extension
mechanism called device plugins
.
Nvidia has published a GPU device plugin recently
that is expected to work with
kubernetes v1.8.

If you are on GCP, please reach out to me and I can share an official alpha
user guide for GPUs with k8s.

On Mon, Oct 16, 2017 at 12:54 AM,  wrote:

>
> Hi Vu Pham,
> My allocatable GPU number is 0.
> Can you advise how you overcome the issue?
> Thanks, Burt
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] is kubernetes aware of the performance difference between say a nehalem and a broadwell core?

['Vishnu Kannan' via Kubernetes user discussion and Q]

No. Kubernetes is not aware of perf differences between CPU or RAM hardware.

Can you describe why you are looking for such a solution? What's your
specific use case?

On Sep 20, 2017 4:04 PM, "z3ro"  wrote:

> is kubernetes aware of the performance difference between say a nehalem
> and a broadwell core?
>
> and ram performance?
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] "Docker-Aware" APM for GKE / Kubernetes environments ?

['Vishnu Kannan' via Kubernetes user discussion and Q]

+Filip

On Mon, Jul 24, 2017 at 2:38 AM, Traiano Welcome  wrote:

> Hi All
>
> Does anyone have a list of APM (Application Performance Monitoring)
> solutions along the lines of Appdynamics/New-Relic and Dynatrace that
> integrate well with GKE?
>
> We've evaluated both Dynatrace and Appdynamics for monitoring dockerized
> applications running on GKE, however neither of them actually function on
> GKE (despite their documentation claiming otherwise).
>
> Alternatively, would anyone have a suggestion for implementing a bespoke
> monitoring solution for APM, possibly assembled from multiple open source
> components to handle a large array of application technologies running in a
> dockerized environment on GKE?
>
>
> Thanks in advance,
> Traiano
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Re: SIG Architecture

['Vishnu Kannan' via Kubernetes user discussion and Q]

+1

On Thu, Jun 22, 2017 at 1:49 PM, 'Eric Tune' via Kubernetes
developer/contributor discussion  wrote:

> +1.
>
>
> On Thu, Jun 22, 2017 at 1:39 PM, Andy Goldstein  wrote:
>
>> +1 from me too
>>
>> On Thu, Jun 22, 2017 at 4:37 PM, Clayton Coleman 
>> wrote:
>>
>>> +1 - there have been several sig discussions recently about having a
>>> more streamlined way to seek consensus on broad reaching technical changes,
>>> and this seems like a natural (possibly overdue) sig.
>>>
>>> On Jun 22, 2017, at 3:13 PM, 'Brian Grant' via Kubernetes
>>> developer/contributor discussion 
>>> wrote:
>>>
>>> At the leadership summit a few weeks ago, I believe there was consensus
>>> that we should start an Architecture SIG. There were also discussions of
>>> Working Groups around extensibility and repo refactoring, but I'd like to
>>> fold that into SIG Architecture, since they are all related, and because
>>> I've been driving these areas, directly or indirectly, and there are only
>>> so many meetings I can attend.
>>>
>>> This is the proposal for such a SIG, as the first step in the SIG
>>> creation procedure:
>>> https://github.com/kubernetes/community/blob/master/sig-gove
>>> rnance.md#sig-creation-procecure
>>>
>>> Initial mission statement (thanks to Jaice):
>>>
>>> The SIG would be intended to guide the design principles of Kubernetes,
>>> as well as provide a consistent body of expertise necessary to ensure
>>> architectural consistency over time.
>>>
>>>
>>> The scope would cover the whole project -- issues that span/encompass
>>> all the components, how they fit together, how they interact, etc. But the
>>> SIG would not get involved with issues specific to a particular component
>>> or functional area, which would be the purview of some other SIG, except
>>> where they deviate from project-wide principles/conventions.
>>>
>>> The specific areas I propose are:
>>>
>>> Defining the scope of the Kubernetes project:
>>>
>>>- https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
>>>- Ecosystem examples
>>>
>>> 
>>>
>>> Documenting and evolving the system architecture:
>>>
>>>- https://github.com/kubernetes/community/blob/master/contribu
>>>tors/design-proposals/architecture.md
>>>
>>> 
>>>- Kubernetes Architecture presentation
>>>
>>> 
>>>- Kubernetes Architectural roadmap working document
>>>
>>> 
>>>
>>> Defining and driving necessary extensibility points
>>> 
>>> .
>>>
>>> Establishing and documenting design principles and conventions for the
>>> overall system and user-facing APIs:
>>>
>>>- https://github.com/kubernetes/community/blob/master/contribu
>>>tors/design-proposals/principles.md
>>>
>>> 
>>>- https://github.com/kubernetes/community/blob/master/contribu
>>>tors/devel/api-conventions.md
>>>Note that the API conventions aren't part of API machinery because
>>>that SIG is about the mechanisms for building the apiservers, APIs, and
>>>client libraries, not the principles/conventions driving the design of 
>>> the
>>>APIs and their contents.
>>>
>>> Driving improvement of overall code organization -- multiple repos, etc.
>>>
>>> Developing necessary review processes, such as the proposal and API
>>> review processes.
>>>
>>> Educating approvers/owners of other SIGs (e.g., by holding office hours).
>>>
>>>
>>> Suggested meeting day/time: Thursday @ 9am Pacific, before the community
>>> meeting, biweekly
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes developer/contributor discussion" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-dev+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/kubernetes-dev/CAKCBhs6Wmq%2BKgo8ayrnHvS-jL-MBFgv%3DoUj%
>>> 2BpS7-u_jFdM-QHQ%40mail.gmail.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this 

Re: [kubernetes-users] cos-stable-59 breaks kubelet launch

['Vishnu Kannan' via Kubernetes user discussion and Q]

On Fri, Jun 9, 2017 at 10:56 AM, Vishnu Kannan  wrote:

> +cos team
>
> On Fri, Jun 9, 2017 at 1:15 AM, Pavel  wrote:
>
>> As of cos-stable-58-9334-74-0, I was able to launch a kubelet on a single
>> compute instance with:
>>
>> --metadata-from-file \
>> "google-container-manifest=containers.yaml,user-data=cloudinit.yaml" \
>>
>> where cloudinit.yaml contains the following:
>>
>> #cloud-configs
>>
>> runcmd:
>> - systemctl start kubelet.service
>>
>>
>> Once cos-stable-59 replaced -58 as cos-stable, this setup does not work.
>> The most relevant log enntry seems to be:
>>
>> "Usage of /usr/bin/kubelet: unknown flag: --config"
>>
>> Any suggestions on what to modify so that the service starts my container
>> in the newer Container OS release? Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Leverage local ssd for statefulset

['Vishnu Kannan' via Kubernetes user discussion and Q]

+msau

https://github.com/kubernetes/community/pull/306 is tackling this problem.
Expect an alpha feature that includes scheduling, fault tolerance and
provisioning by v1.8.

On Sun, May 7, 2017 at 12:50 AM, vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:

> Hello
> I would like to have your opinion on statefulset implemented on local
> volumes. What would be the best option today ?
>
> - a custom scheduler ?
> - a custom scheduler + an operator ?
> - an external mapping system ?
>
> I would like to dedicate local disks per pod.
> TX
>
> Vg
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Resource requests/limits on Pod-level

['Vishnu Kannan' via Kubernetes user discussion and Q]

As davidopp@ mentioned a pod is the unit of scheduling. So to find a
suitable node for a pod, the scheduler has to identify a node that sum(all
container requests) available.

It is also used by the node to enforce overall resource usage across all
containers. Not all memory is reclaimed when a container dies or restarts
and so we need an isolation sandbox that spans across all the containers.

It's not a "concept", but more of an implementation detail. Maybe we can
clarify that more explicitly until we introduce it as a concept in the
future?

On Fri, May 5, 2017 at 12:35 AM, 'David Oppenheimer' via Kubernetes user
discussion and Q  wrote:

>
>
> On Wed, May 3, 2017 at 12:10 PM, 'Ahmet Alp Balkan' via Kubernetes user
> discussion and Q  wrote:
>
>> I am afraid it still does not make sense to me.
>>
>> Why is there even a concept of "Pod-level request/limit" if it is not
>> used anywhere?
>>
>
> It is used by the scheduler. Pod is the atomic unit of scheduling.
>

>
>> As a user, this is confusing me. As far as I can tell, I can configure
>> limits on the Container and if I go beyond that my *Pod* will be killed
>> altogether. This part is clear. However I can't tell what a Pod-level
>> request/limit (just a sum of things which I can't configure directly) does
>> on my cluster today?
>>
>> On Wed, May 3, 2017 at 11:30 AM, 'David Oppenheimer' via Kubernetes user
>> discussion and Q  wrote:
>>
>>>
>>>
>>> On Wed, May 3, 2017 at 10:47 AM, 'Ahmet Alp Balkan' via Kubernetes user
>>> discussion and Q  wrote:
>>>
 Hello, I am trying to understand the Resource Limits/Requests for Pods
 and Containers
 
  document.
 In multiple places, the document implies that users can specify
 ResourceRequirements *at pod-level**. *(I don't mean
 pod.spec.containers.resources.) Most relevantly the doc says:

 A Pod resource request/limit for a particular resource type* is the
> sum *of the resource requests/limits of that type for each Container
> in the Pod.


>>> Request and limit are specified only at the per-container level. The
>>> system computes pod-level request and limit by adding up the request and
>>> limit of the containers that are inside the pod. But you can't specify it
>>> at the pod level yourself.
>>>
>>> Does that make sense?
>>>
>>>
>>>

 However I can’t find any examples or any fields on the API (kubectl
 explain pod.spec) to specify resource requirements on the pod level.

 Any ideas if this is possible at all? This particular document is
 particularly gives the strong impression that this feature exists today. I
 opened this docs issue
  to
 track this.

 --
 You received this message because you are subscribed to the Google
 Groups "Kubernetes user discussion and Q" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to kubernetes-users+unsubscr...@googlegroups.com.
 To post to this group, send email to kubernetes-users@googlegroups.com.
 Visit this group at https://groups.google.com/group/kubernetes-users.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion and Q" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google 

Re: [kubernetes-users] kubectl can't connect to cluster after creating daemonset that logs too much

['Vishnu Kannan' via Kubernetes user discussion and Q]

This is one of the reasons why the master node should ideally not be
running user pods.
We are working on making the system resilient to abusive pods.

On Wed, Mar 22, 2017 at 3:04 PM, Justin Seely 
wrote:

> I managed to fix it by logging onto the master to use kubectl to delete
> the daemon set. Thanks for the response though Brandon.
>
> On Wed, Mar 22, 2017 at 1:06 PM Brandon Philips <
> brandon.phil...@coreos.com> wrote:
>
>> Can you get logs from etcd to figure out what is wrong?
>>
>> On Wed, Mar 22, 2017 at 11:15 AM  wrote:
>>
>> Hi all,
>> I just created a daemonset on one of my Kubernetes clusters which is just
>> a busybox logging to stdout. Unfortunately I think it is logging too much
>> and I now can't connect to the cluster using kubectl. When I run a command
>> I get this error "Error from server: client: etcd cluster is unavailable or
>> misconfigured". Any help would be greatly appreciated.
>> Justin
>>
>> --
>>
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>>
>>
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Kubernetes user discussion and Q" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/kubernetes-users/r0naIJqrm5E/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] EmptyDir on local SSD?

['Vishnu Kannan' via Kubernetes user discussion and Q]

Richard,

Can you describe your workflow/use-case a bit more?
Are you having I/O starvation issues with accessing container images and/or
logging as well?

--Vish

On Tue, Mar 7, 2017 at 1:35 PM, Richard Musiol 
wrote:

> I have solved this with a wrapper now which does the cleanup on SIGTERM
> after the child has exited.
>
> Thanks for the suggestions, though!
>
> Cheers,
> Richard
>
>  schrieb am Sa., 4. März 2017 um 20:31 Uhr:
>
>> Maybe you can use a flexVolume plugin? (in bash like the lvm example)
>> (https://github.com/kubernetes/kubernetes/blob/master/examples/volumes/
>> flexvolume/README.md)
>> in the mount you can check if the ssd drive is mounted and then create a
>> new directory that you can remove when the unmount method is called.
>>
>>
>> On Friday, March 3, 2017 at 3:24:50 PM UTC-3, Richard Musiol wrote:
>> > Hi Michelle, Hi Tim,
>> >
>> >
>> > there is one local SSD per node, so yes, it is shared between many pods.
>> >
>> >
>> > Solution 1 is already a step forward, but I would really like to use a
>> local SSD. The use case is that I'm running Buildkite agents on that
>> cluster and I want to autoscale them. The limits of a normal disk were very
>> quickly reached with that kind of workload, switching to the local SSD
>> solved it.
>> >
>> >
>> > Solution 2 sounds involved, but I'm willing to be a guinea pig for this
>> because the workload is not production critical.
>> >
>> >
>> > -Richard
>> >
>> >
>> > 'Michelle Au' via Kubernetes user discussion and Q <
>> kubernet...@googlegroups.com> schrieb am Do., 2. März 2017 um 23:33 Uhr:
>> >
>> >
>> > Hi Richard,
>> >
>> >
>> > Are you sharing the local SSD between many pods, or just one pod per
>> SSD?
>> >
>> >
>> > If sharing is ok, then in the short term we could look into one of the
>> following approaches:
>> > 1. Ability to create a GKE cluster with kubelet installed on top of a
>> PD-SSD.  Then all EmptyDirs will use this PD.  It's not going to perform as
>> well as a local SSD though.
>> > 2. Use the alpha flex volumes interface with an lvm plugin, that can
>> carve out lvs out of a vg comprised of local SSDs. The vgs would have to be
>> precreated by some DaemonSet on each node before any normal pods start
>> running.  This approach would be meant as a short term solution for now and
>> require some extra management by the user/admin.  Flex itself is alpha and
>> going through lots of revision.
>> >
>> >
>> > If dedicated disks are required, then we don't have any short-term
>> solutions besides hostpath.  The long term solution is to expose disks as
>> LocalDisk PVs, and for the temporary use cases, have an "inline" option
>> where the PV gets created and destroyed with the pod.
>> >
>> >
>> >
>> > -Michelle
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Mar 2, 2017 at 9:33 AM, 'Tim Hockin' via Kubernetes user
>> discussion and Q  wrote:
>> >
>> > There isn't a clean way to express what you want today.  There are some
>> ideas about being able to express local storage as volumes, but that work
>> is a long pipeline for what feels.like a simple request.
>> >
>> >
>> > We already have an idea of "medium" in emptyDir.  What if we extended
>> that?  The question becomes how to express that multitudes of potential SSD
>> technologies, current and yet to be developed, without resorting to calling
>> them all the same?
>> >
>> >
>> > You could imagine a way to config kubelet to build a map of local
>> mountpoints as named "Local" media, and then allow users to request those.
>> It's imperfect in a lot of ways, but it might be tractable.
>> >
>> >
>> > @vishh @msau this comes up not FREQUENTLY but enough that maybe we want
>> to think of a short term goal here?
>> >
>> >
>> > Just thinking out loud...
>> >
>> >
>> >
>> >
>> > On Mar 2, 2017 9:21 AM, "Richard Musiol"  wrote:
>> >
>> > Hi,
>> >
>> >
>> > I would like to use GKE's local SSD feature to have fast temporary disk
>> space.
>> >
>> >
>> > The problem when using it with a "hostPath" volume as described on
>> https://cloud.google.com/container-engine/docs/local-ssd is that the
>> temporary files do not get removed when the pod gets deleted. Over time the
>> local SSD would fill up.
>> >
>> >
>> > The volume type "emptyDir" would do what I want, but I don't see how to
>> put it on the local SSD.
>> >
>> >
>> > Any ideas?
>> >
>> >
>> > Cheers,
>> > Richard
>> >
>> >
>> >
>> >
>> > --
>> >
>> > You received this message because you are subscribed to the Google
>> Groups "Kubernetes user discussion and Q" group.
>> >
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to kubernetes-use...@googlegroups.com.
>> >
>> > To post to this group, send email to kubernet...@googlegroups.com.
>> >
>> > Visit this group at https://groups.google.com/group/kubernetes-users.
>> >
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> >
>> >
>> >
>> 

Re: [kubernetes-users] EmptyDir on local SSD?

['Vishnu Kannan' via Kubernetes user discussion and Q]

Can you use a memory backed EmptyDir volume?

Solution `2` described by msau@ would work in the short term, but we'd much
rather invest in our existing plans for exposing local SSDs as Persistent
Volumes and get that feature to alpha by v1.7 for example.
More details on the existing plans/design here
.

--Vish

On Fri, Mar 3, 2017 at 10:42 AM, Richard Musiol 
wrote:

I would need a *Post*Stop hook. But yes, maybe the best solution right now
would be to wrap the agent into some helper that on SIGTERM first forwards
the signal to the agent and then does the cleanup before exiting itself.

'David Aronchick' via Kubernetes user discussion and Q  schrieb am Fr., 3. März 2017 um 19:36 Uhr:

Is a preStop hook enough as a temp solution?

e.g. /bin/sh rm -rf 

https://kubernetes.io/docs/user-guide/container-environment/#hook-details

On Fri, Mar 3, 2017 at 10:24 AM, Richard Musiol 
wrote:

Hi Michelle, Hi Tim,

there is one local SSD per node, so yes, it is shared between many pods.

Solution 1 is already a step forward, but I would really like to use a
local SSD. The use case is that I'm running Buildkite agents on that
cluster and I want to autoscale them. The limits of a normal disk were very
quickly reached with that kind of workload, switching to the local SSD
solved it.

Solution 2 sounds involved, but I'm willing to be a guinea pig for this
because the workload is not production critical.

-Richard

'Michelle Au' via Kubernetes user discussion and Q  schrieb am Do., 2. März 2017 um 23:33 Uhr:

Hi Richard,

Are you sharing the local SSD between many pods, or just one pod per SSD?

If sharing is ok, then in the short term we could look into one of the
following approaches:
1. Ability to create a GKE cluster with kubelet installed on top of a
PD-SSD.  Then all EmptyDirs will use this PD.  It's not going to perform as
well as a local SSD though.
2. Use the alpha flex volumes interface with an lvm plugin, that can carve
out lvs out of a vg comprised of local SSDs. The vgs would have to be
precreated by some DaemonSet on each node before any normal pods start
running.  This approach would be meant as a short term solution for now and
require some extra management by the user/admin.  Flex itself is alpha and
going through lots of revision.

If dedicated disks are required, then we don't have any short-term
solutions besides hostpath.  The long term solution is to expose disks as
LocalDisk PVs, and for the temporary use cases, have an "inline" option
where the PV gets created and destroyed with the pod.

-Michelle



On Thu, Mar 2, 2017 at 9:33 AM, 'Tim Hockin' via Kubernetes user discussion
and Q  wrote:

There isn't a clean way to express what you want today.  There are some
ideas about being able to express local storage as volumes, but that work
is a long pipeline for what feels.like a simple request.

We already have an idea of "medium" in emptyDir.  What if we extended
that?  The question becomes how to express that multitudes of potential SSD
technologies, current and yet to be developed, without resorting to calling
them all the same?

You could imagine a way to config kubelet to build a map of local
mountpoints as named "Local" media, and then allow users to request those.
It's imperfect in a lot of ways, but it might be tractable.

@vishh @msau this comes up not FREQUENTLY but enough that maybe we want to
think of a short term goal here?

Just thinking out loud...

On Mar 2, 2017 9:21 AM, "Richard Musiol"  wrote:

Hi,

I would like to use GKE's local SSD feature to have fast temporary disk
space.

The problem when using it with a "hostPath" volume as described on
https://cloud.google.com/container-engine/docs/local-ssd is that the
temporary files do not get removed when the pod gets deleted. Over time the
local SSD would fill up.

The volume type "emptyDir" would do what I want, but I don't see how to put
it on the local SSD.

Any ideas?

Cheers,
Richard

-- 
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.

Re: [kubernetes-users] kubelet failing with "eviction manager: unexpected err: failed GetNode: "

['Vishnu Kannan' via Kubernetes user discussion and Q]

+sig-cluster-lifecycle

On Fri, Feb 24, 2017 at 3:45 PM, Àbéjídé Àyodélé <abejideayod...@gmail.com>
wrote:

> Like I said I have a similar cluster that I setup by hand that works,
> trying to automate the cluster I setup by hand is where I am running into
> problems, also I am interested in knowing why I am running into this
> problem, so I can learn for the future
>
> Abejide Ayodele
> It always seems impossible until it's done. --Nelson Mandela
>
> On Fri, Feb 24, 2017 at 5:42 PM, 'Vishnu Kannan' via Kubernetes user
> discussion and Q <kubernetes-users@googlegroups.com> wrote:
>
>> It depends on how you are trying to setup the cluster. I'd recommend
>> starting with a known working solution like "kubeadm
>> <https://kubernetes.io/docs/getting-started-guides/kubeadm/>" to
>> bootstrap your cluster before customizing custer bringup.
>>
>> On Fri, Feb 24, 2017 at 3:39 PM, ayodele abejide <
>> abejideayod...@gmail.com> wrote:
>>
>>> Your node bootstrapping is failing.
>>>
>>>
>>> How do I debug this?
>>>
>>>
>>>
>>> On Friday, 24 February 2017 17:28:53 UTC-6, Vishnu Kannan wrote:
>>>>
>>>> Your node bootstrapping is failing. I'd recommend focussing on getting
>>>> your node registered and ignore the eviction manager error. The latter is a
>>>> red herring.
>>>>
>>>> On Fri, Feb 24, 2017 at 2:45 PM, ayodele abejide <abejide...@gmail.com>
>>>> wrote:
>>>>
>>>>> kubeclt get nodes returns No resources found. I do not know how to
>>>>> verify:
>>>>>
>>>>>
>>>>> Has the node successfully registered itself with the apiserver?
>>>>>
>>>>>
>>>>> Relevant code that is generating the failure is here
>>>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fkubernetes%2Fkubernetes%2Fblob%2F55bee3ad21f025b1416a4e1f10de753f484b66d3%2Fpkg%2Fkubelet%2Fserver%2Fstats%2Fsummary.go%23L80=D=1=AFQjCNEzlGPM6j6QAZEUHvbLqsKVNqo6CQ>
>>>>> .
>>>>>
>>>>>
>>>>> I sort of have browsed the code, but don't have enough context to get
>>>>> enough information out of it
>>>>>
>>>>>
>>>>> On Friday, 24 February 2017 16:25:49 UTC-6, Vishnu Kannan wrote:
>>>>>>
>>>>>> Kubelet is unable to retrieve the "Node" object that represents the
>>>>>> node its running on from the api server. Has the node successfully
>>>>>> registered itself with the apiserver?
>>>>>>
>>>>>> Relevant code that is generating the failure is here
>>>>>> <https://github.com/kubernetes/kubernetes/blob/55bee3ad21f025b1416a4e1f10de753f484b66d3/pkg/kubelet/server/stats/summary.go#L80>
>>>>>> .
>>>>>>
>>>>>> On Fri, Feb 24, 2017 at 2:19 PM, ayodele abejide <
>>>>>> abejide...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have searched the internet and found no answers that solve my
>>>>>>> problem.
>>>>>>>
>>>>>>> Background:
>>>>>>>
>>>>>>> I have a working cluster that I intended to automate it's creation
>>>>>>> via puppet and terraform, all seemed to have worked well except I find:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Feb 24 22:05:41 kube-worker02.mydomain.com kubelet[18956]: E0224 22:
>>>>>>> 05:41.775438 18956 eviction_manager.go:204] eviction manager:
>>>>>>> unexpected err: failed GetNode: node 'kube-worker02.mydomain.com'
>>>>>>> not found
>>>>>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 22:
>>>>>>> 05:51.725950 18956 kubelet.go:1155] Image garbage collection
>>>>>>> succeeded
>>>>>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:
>>>>>>> 05:51.733365 18956 container_manager_linux.go:728] CPUAccounting not
>>>>>>> enabled for pid: 6095
>>>>>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:
>>>>>>> 05:51.733381 18956 container_manager_linux.go:731] MemoryAccounting
>>>>>>> not enabled for pid: 6095
>&g

Re: [kubernetes-users] kubelet failing with "eviction manager: unexpected err: failed GetNode: "

['Vishnu Kannan' via Kubernetes user discussion and Q]

It depends on how you are trying to setup the cluster. I'd recommend
starting with a known working solution like "kubeadm
" to bootstrap
your cluster before customizing custer bringup.

On Fri, Feb 24, 2017 at 3:39 PM, ayodele abejide 
wrote:

> Your node bootstrapping is failing.
>
>
> How do I debug this?
>
>
>
> On Friday, 24 February 2017 17:28:53 UTC-6, Vishnu Kannan wrote:
>>
>> Your node bootstrapping is failing. I'd recommend focussing on getting
>> your node registered and ignore the eviction manager error. The latter is a
>> red herring.
>>
>> On Fri, Feb 24, 2017 at 2:45 PM, ayodele abejide 
>> wrote:
>>
>>> kubeclt get nodes returns No resources found. I do not know how to
>>> verify:
>>>
>>>
>>> Has the node successfully registered itself with the apiserver?
>>>
>>>
>>> Relevant code that is generating the failure is here
>>> 
>>> .
>>>
>>>
>>> I sort of have browsed the code, but don't have enough context to get
>>> enough information out of it
>>>
>>>
>>> On Friday, 24 February 2017 16:25:49 UTC-6, Vishnu Kannan wrote:

 Kubelet is unable to retrieve the "Node" object that represents the
 node its running on from the api server. Has the node successfully
 registered itself with the apiserver?

 Relevant code that is generating the failure is here
 
 .

 On Fri, Feb 24, 2017 at 2:19 PM, ayodele abejide 
 wrote:

> Hi,
>
> I have searched the internet and found no answers that solve my
> problem.
>
> Background:
>
> I have a working cluster that I intended to automate it's creation via
> puppet and terraform, all seemed to have worked well except I find:
>
>
>
> Feb 24 22:05:41 kube-worker02.mydomain.com kubelet[18956]: E0224 22:05
> :41.775438 18956 eviction_manager.go:204] eviction manager:
> unexpected err: failed GetNode: node 'kube-worker02.mydomain.com' not
> found
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 22:05
> :51.725950 18956 kubelet.go:1155] Image garbage collection succeeded
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05
> :51.733365 18956 container_manager_linux.go:728] CPUAccounting not
> enabled for pid: 6095
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05
> :51.733381 18956 container_manager_linux.go:731] MemoryAccounting not
> enabled for pid: 6095
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 22:05
> :51.733391 18956 container_manager_linux.go:434] Discovered runtime
> cgroups name: /system.slice/docker.service
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05
> :51.733478 18956 container_manager_linux.go:728] CPUAccounting not
> enabled for pid: 18956
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05
> :51.733489 18956 container_manager_linux.go:731] MemoryAccounting not
> enabled for pid: 18956
> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: E0224 22:05
> :51.776184 18956 eviction_manager.go:204] eviction manager:
> unexpected err: failed GetNode: node 'kube-worker02.mydomain.com' not
> found
>
> in the kubelet logs.
>
> What have I tried?
>
> I have disabled ssl, set authorization-mode=AlwaysAllow, confirmed
> that kube-proxy running on the same worker can reach the api-server, also
> confirmed that cadvisor is running:
>
> curl localhost:4194/api/v2.0/
>
> Supported request types: "appmetrics,attributes,events,
> machine,ps,spec,stats,storage,summary,version"
>
>
> I have also rebooted the worker a couple of times and restarted
> kubelet and kube-proxy so many times.
>
> I am at loss here and don't know what to try, any help will be
> appreciated.
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google
> Groups "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubernetes-use...@googlegroups.com.
> To post to this group, send email to kubernet...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

 --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion 

Re: [kubernetes-users] kubelet failing with "eviction manager: unexpected err: failed GetNode: "

['Vishnu Kannan' via Kubernetes user discussion and Q]

Your node bootstrapping is failing. I'd recommend focussing on getting your
node registered and ignore the eviction manager error. The latter is a red
herring.

On Fri, Feb 24, 2017 at 2:45 PM, ayodele abejide 
wrote:

> kubeclt get nodes returns No resources found. I do not know how to verify:
>
>
> Has the node successfully registered itself with the apiserver?
>
>
> Relevant code that is generating the failure is here
> 
> .
>
>
> I sort of have browsed the code, but don't have enough context to get
> enough information out of it
>
>
> On Friday, 24 February 2017 16:25:49 UTC-6, Vishnu Kannan wrote:
>>
>> Kubelet is unable to retrieve the "Node" object that represents the node
>> its running on from the api server. Has the node successfully registered
>> itself with the apiserver?
>>
>> Relevant code that is generating the failure is here
>> 
>> .
>>
>> On Fri, Feb 24, 2017 at 2:19 PM, ayodele abejide 
>> wrote:
>>
>>> Hi,
>>>
>>> I have searched the internet and found no answers that solve my problem.
>>>
>>> Background:
>>>
>>> I have a working cluster that I intended to automate it's creation via
>>> puppet and terraform, all seemed to have worked well except I find:
>>>
>>>
>>>
>>> Feb 24 22:05:41 kube-worker02.mydomain.com kubelet[18956]: E0224 22:05:
>>> 41.775438 18956 eviction_manager.go:204] eviction manager: unexpected
>>> err: failed GetNode: node 'kube-worker02.mydomain.com' not found
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 22:05:
>>> 51.725950 18956 kubelet.go:1155] Image garbage collection succeeded
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05:
>>> 51.733365 18956 container_manager_linux.go:728] CPUAccounting not
>>> enabled for pid: 6095
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05:
>>> 51.733381 18956 container_manager_linux.go:731] MemoryAccounting not
>>> enabled for pid: 6095
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 22:05:
>>> 51.733391 18956 container_manager_linux.go:434] Discovered runtime
>>> cgroups name: /system.slice/docker.service
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05:
>>> 51.733478 18956 container_manager_linux.go:728] CPUAccounting not
>>> enabled for pid: 18956
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 22:05:
>>> 51.733489 18956 container_manager_linux.go:731] MemoryAccounting not
>>> enabled for pid: 18956
>>> Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: E0224 22:05:
>>> 51.776184 18956 eviction_manager.go:204] eviction manager: unexpected
>>> err: failed GetNode: node 'kube-worker02.mydomain.com' not found
>>>
>>> in the kubelet logs.
>>>
>>> What have I tried?
>>>
>>> I have disabled ssl, set authorization-mode=AlwaysAllow, confirmed that
>>> kube-proxy running on the same worker can reach the api-server, also
>>> confirmed that cadvisor is running:
>>>
>>> curl localhost:4194/api/v2.0/
>>>
>>> Supported request types: "appmetrics,attributes,events,
>>> machine,ps,spec,stats,storage,summary,version"
>>>
>>>
>>> I have also rebooted the worker a couple of times and restarted kubelet
>>> and kube-proxy so many times.
>>>
>>> I am at loss here and don't know what to try, any help will be
>>> appreciated.
>>>
>>> Thanks!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion and Q" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-use...@googlegroups.com.
>>> To post to this group, send email to kubernet...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at 

Re: [kubernetes-users] Re: Proposing SIG-GPU

['Vishnu Kannan' via Kubernetes user discussion and Q]

I'd recommend posting a Design proposal before posting PRs.

On Fri, Feb 17, 2017 at 12:47 PM,  wrote:

> I'd be happy to lead any GPU discussion if this support is looking for an
> owner.
>
> I am preparing a PR that adds more advanced GPU scheduling features
> withNVML. I plan to submit follow-up PRs to cadvisor,
> node-problem-detector, etc as well for GPU enhancements.
>
> I added this as a topic to the Feb 22nd meeting agenda for general
> discussion/prioritization.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes developer/contributor discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/kubernetes-dev/93409983-78af-43c5-b752-6ed8ec153930%
> 40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Can k8s scheduler be confiugured on GKE?

['Vishnu Kannan' via Kubernetes user discussion and Q]

+davidopp@

On Thu, Feb 9, 2017 at 1:26 PM, Gareth  wrote:

> Hi,
>
> I'd like to take advantage of some of the new scheduler priority functions
> that emerged circa 1.5 (e.g. MostRequestedPriority)
>
> Is it currently possible to use these on Google Container Engine?
>
> I'm aware
> 
>  that
> we can't configure GKE's own scheduler, as it's "out of reach".  My second
> thought was to attempt to deploy a second scheduler
> , but a naive
> walk-through of the tutorial there did not yield a functioning scheduler.
> I've appealed to StackExchange
> 
> for help, but to no avail.
>
> Does GKE support configurable pod scheduling, or should I be looking at
> deploying my own k8s on GCE?
>
> ~ gareth
> [Apologies if this gets posted twice; first attempt seems to have failed.]
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] logs folder permission

['Vishnu Kannan' via Kubernetes user discussion and Q]

`/var/lib/docker/` is managed by docker and it is not intended to be
consumed outside of docker directly. IIUC, `docker logs` is the interface
that users are expected to use. Is running "fluentd" as root user an option?


On Fri, Jan 6, 2017 at 12:22 PM, Rodrigo Campos  wrote:

> It happens also if you run the docker image on your local PC, right?
>
> Probably you need to create it and chmod in your dockerfile, but remember
> that docker storage is not persistent and it gets erased every time the
> container is removed (on deploy, container crash, node drain, etc.)
>
>
> On Friday, January 6, 2017,  wrote:
>
>> Hello everyone. Everytime I create a new containers on Kubernetes, new
>> log folders in /var/lib/docker/containers get created. The permission is
>> automatically set to drwx--  . That doesn't allow me to correctly
>> collect the content and read it through FluentD. Any suggestion on how to
>> fix the situation?
>> Thank you so much in advance.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Autoscale volume and pods simultaneously

['Vishnu Kannan' via Kubernetes user discussion and Q]

Check out dynamic volumes provisioning here

.

On Thu, Jan 5, 2017 at 3:07 PM, Montassar Dridi 
wrote:

> Hello!!
>
> I'm using Kubernetes deployment with persistent volume to run my
> application, but when I try to add more replicas or autoscale, all the new
> pods try to connect to the same volume.
> How can I simultaneously auto create new volumes for each new pod., like
> statefulsets(petsets) are able to do it.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Re: New SIG On-Prem announcement

['Vishnu Kannan' via Kubernetes user discussion and Q]

On Tue, Jan 3, 2017 at 4:17 PM, 'David Oppenheimer' via Kubernetes
developer/contributor discussion  wrote:



On Mon, Jan 2, 2017 at 8:00 AM, Tomasz 'Zen' Napierala <
tnapier...@mirantis.com> wrote:

> Hi Kubernetes community!
>
> I hope everyone is doing great in the new year of 2017.
>
> On behalf of SIG On-Prem leads, I’m really happy to announce new SIG in
> our community: SIG On-Prem.
>
> The mission of this SIG is to bring together all members interested in
> running Kubernetes on premise / on bare metal or to put it more general:
> beyond cloud providers. If you are interested in motivations and more
> details, please read original SIG chapter proposal prepared by Joseph Jacks
> here: https://docs.google.com/document/d/1oYtW7fgSJsQDl-ln6ETvAQrN
> dne4w_0am_qHTxtd3Yw/edit I’m providing some excerpts for that document
> below.
>
> In this new SIG we would like to be:
> - A discussion forum for people deploying Kubernetes into existing
> environments, which has high correlation with on-prem.
> - A place for people to benefit from sharing experiences with generic
> setups not in a public cloud (on-prem VMs, etc.) as well as it could help
> surface limitations and problems users should expect along the way. Most
> documentation is targeted toward kube-up/kops which obviously don't apply
> but some newer tools do (kubeadm).
> - Used for improving support for workloads that require private/hybrid
> cloud that include bare metal. It could include things like exposing the
> heterogeneity, data-plane acceleration, enhanced low-level isolation
> (pinned/exclusive cores, etc.)
>

I would suggest using sig-node for this topic, to avoid fragmentation of
the discussions.

+1.



> - A focused area for improvements on:
> - How e2e tests pass on BM -- currently, there are many problems
> in this area. It would be also nice to have more tests dedicated to BM.
> - Storage and networking requirements in BM environments are very
> different when compared to public cloud APIs and services. What are some
> best practices and recommendations the community can rally around?
>
> We hope to be useful part of great Kubernetes community, but it’s up to
> you - members interested in running Kubernetes on premise. So everyone is
> welcome to join new SIG and share their experience and ideas.
>
> Our first meeting is planned for January 11th, 2017 @ 8:00 AM PST (11:00
> AM ET, 5:00 PM CET).
>
> Please find below all details about this new SIG:
> - Google Group: https://groups.google.com/forum/#!forum/kubernetes-sig-on-
> prem
> - Slack channel: https://kubernetes.slack.com/messages/sig-onprem/
> - Meeting Agenda/Minutes: https://docs.google.com/docume
> nt/d/1AHF1a8ni7iMOpUgDMcPKrLQCML5EMZUAwP4rro3P6sk/edit#
> - On premise efforts tracking document (input needed):
> https://docs.google.com/spreadsheets/d/1Ca9ZpGXM4PfycYv0Foi7
> Y4vmN4KVXrGYcJipbH8_xLY/edit#gid=1763270493
> - Zoom meeting space: https://zoom.us/my/k8s.sig.onprem
>
>
> Regards,
> --
> Tomasz 'Zen' Napierala
> Kubernetes Engineering - Poland
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes developer/contributor discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/kubernetes-dev/F7E2F06C-81F9-4CAA-9846-F5D519AC3013%40mirantis.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups
"Kubernetes developer/contributor discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to kubernetes-dev+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/kubernetes-dev/CAOU1bzepHCxyrKST6QBJdpgZw5bM1
QRUJG9Rrwd-pnPewX-sJQ%40mail.gmail.com

.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Re: Adding swap to Kubernetes nodes on the new GCI image

['Vishnu Kannan' via Kubernetes user discussion and Q]

On Sat, Oct 8, 2016 at 4:00 AM, Matthias Rampke  wrote:



On Sat, Oct 8, 2016, 00:15 ravi prasad l r  wrote:

> Matthias, how are disabling swap for the containers ?
>

We disabled and removed all swap on the nodes, and changed new installs not
to include any.



Kubelet seems to set MemorySwap (libcontainer config) to -1 by default here:
> https://github.com/kubernetes/kubernetes/blob/v1.4.0/pkg/
> kubelet/dockertools/docker_manager.go#L662
>
> which means  /sys/fs/cgroup/memory.memsw.limit_in_bytes is set to
> unlimited (mem + swap) inside the container.
>

Ah, that would explain why it's behaving different from our older container
system.


> Vish, isn't the Memory+swap should also be set to container.Resources.
> Limits.Memory().Value() above. Are am i missing something.
>

That would match my expectation in the absence of a separate swap limit.

Ideally, kubelet should be disabling swap by default since it cannot manage
swap.


/MR

> --
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Re: Adding swap to Kubernetes nodes on the new GCI image

['Vishnu Kannan' via Kubernetes user discussion and Q]

K8s does not recommend running with swap.

On Fri, Oct 7, 2016 at 8:51 AM, 'Aditya Kali' via Kubernetes user
discussion and Q  wrote:

> Swap is disabled by default in gci images. To enable it, run:
>
> $ sudo sysctl vm.disk_based_swap=1
> OR
> $ echo 1 > /proc/sys/vm/disk_based_swap
>
> After that, your instructions below should work.
>
> Moreover, instead of using /mnt/stateful_partition/ path, you can create
> your swapfile under /var. Its the same partition as
> /mnt/stateful_partition, but the path then won't be distro specific.
>
>
> On Thursday, October 6, 2016 at 1:33:58 AM UTC-7, June Rhodes wrote:
>>
>> I used to automatically create and mount a swapfile on Kubernetes nodes
>> at startup, however, this doesn't appear to be working anymore on the newer
>> GCI images.
>>
>> The startup script used to do:
>>
>> fallocate -l 2G /swapfile
>> chmod 600 /swapfile
>> mkswap /swapfile
>> swapon /swapfile
>>
>> This obviously no longer works because the root partition is now
>> read-only, but it doesn't even work if I try and place the swapfile under
>> the stateful partition:
>>
>> fallocate -l 2G /mnt/stateful_partition/swapfile
>> chmod 600 /mnt/stateful_partition/swapfile
>> mkswap /mnt/stateful_partition/swapfile
>> swapon /mnt/stateful_partition/swapfile
>>
>> Whenever I try and mount a swap file, I just get:
>>
>> # swapon /mnt/stateful_partition/swapfile
>> swapon: /mnt/stateful_partition/swapfile: swapon failed: Invalid argument
>>
>> Is there a recommended way of mounting a swapfile on Kubernetes nodes at
>> startup now?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] How to develop lxd runtime for kubernetes

['Vishnu Kannan' via Kubernetes user discussion and Q]

Related Kubernetes issue here
.

On Mon, Sep 26, 2016 at 10:38 AM, Jonathan Boulle <
jonathan.bou...@coreos.com> wrote:

> The future of integrating new container runtimes is the Container Runtime
> Interface, so you should start by looking at the proposal and (early
> version of the) API:
>
> https://github.com/kubernetes/kubernetes/blob/master/docs/
> proposals/container-runtime-interface-v1.md#redefine-
> container-runtime-interface
>
> https://github.com/kubernetes/kubernetes/blob/master/pkg/
> kubelet/api/v1alpha1/runtime/api.proto
>
> On 26 September 2016 at 19:30, Dilip Renkila 
> wrote:
>
>> Hi all,
>>
>> As lxd is a promising technology by delivering full OS level containers
>> rather than process containers like docker, i wanna know what are the
>> specifications required by kubernetes in order to integrate lxd as a
>> runtime or where can i found them
>>
>> Best Regards
>>
>> 'Dilip Renkila
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Monitor OOM inside pods on GKE

['Vishnu Kannan' via Kubernetes user discussion and Q]

Kubelet detects OOMs and surfaces it as part of container status. `kubectl
describe pods` should show OOM as the termination reason whenever a
container is OOM killed.
Kubelet also detects System OOMs automatically and surfaces them as node
events.

Does that meet your requirements?

On Wed, Aug 17, 2016 at 9:03 AM, Romain Vrignaud 
wrote:

> Hello,
>
> Today I have a hacky grok of my logging DaemonSet that parse kernel logs
> to detect OOM inside pods due to memory limits.
>
> This is a bit hacky and I would like to improve that. I think that
> cAdvisor already detect OOMs but I'm not sur what is the recommended way to
> monitor that. I'm using prometheus, so a prometheus exposed metric would be
> awesome.
>
> How do you guys manage to monitor oom ?
>
> Regards,
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.