I would follow https://github.com/mbohlool/, he'll probably be the one
making the repos.

On Fri, Oct 14, 2016 at 11:07 AM, Diego Rodríguez Rodríguez <
diegorgz...@gmail.com> wrote:

> It was just that, but I feel like I would need a library to rely on as
> soon as possible (I am doing everything by hand) because the actual ones
> are not mature enough. When do you plan to release it? I am very interested.
>
>
> On Friday, 14 October 2016, 'Daniel Smith' via Kubernetes user discussion
> and Q&A <kubernetes-users@googlegroups.com> wrote:
>
>> Glad you figured out what was going on (I was about to say, I don't know
>> python well but that code doesn't look like it could possibly work for a
>> streaming connection).
>>
>> We are working on generating a python client. I will make sure we try the
>> watch feature.
>>
>> On Fri, Oct 14, 2016 at 5:15 AM, Diego Rodríguez Rodríguez <
>> diegorgz...@gmail.com> wrote:
>>
>>> I have identified the problem. It has nothing to do with Kubernetes, it
>>> is about how Python's requests module handles this kind of connections.
>>> Further information can be found in:
>>>
>>> https://github.com/kennethreitz/requests/issues/2433   ---> I have
>>> tested it, and it does not work for me.
>>> https://cobe.io/blog/posts/kubernetes-watch-python/     ---> Talks
>>> about the problem and provides a link to the Python module they have
>>> created to wrap the k8s API calls
>>>
>>> The module is https://bitbucket.org/cobeio/kube/overview and it might
>>> be of help for someone who runs in that kind of troubles, for me does not
>>> work because as I could read in their docs, they do not support k8s jobs
>>> yet.
>>>
>>>
>>> On 14 October 2016 at 10:36, Diego Rodríguez Rodríguez <
>>> diegorgz...@gmail.com> wrote:
>>>
>>>> This is what I am doing already
>>>>
>>>> def get_job_state(job_id):
>>>>>     response = requests.get('{}{}/{}'.format('https://kubernetes',
>>>>>                                              ENDPOINT, job_id),
>>>>>                             headers={'Authorization': 'Bearer
>>>>> {}'.format(
>>>>>                                 TOKEN
>>>>>                             )},
>>>>>                             verify=CA_CERT_PATH)
>>>>>
>>>>>     if response.ok:
>>>>>         res_dict = json.loads(response.text)
>>>>>         resource_version = res_dict['metadata']['resourceVersion']
>>>>>         while not (res_dict['status'].get('succeeded', False) or
>>>>>                    res_dict['status'].get('failed', False)):
>>>>>             response = requests.get('{0}{1}/{2}?watch
>>>>> =true&resourceVersion={3}'
>>>>>                                     .format('https://kubernetes',
>>>>>                                             ENDPOINT, job_id,
>>>>>                                             resource_version),
>>>>>                                     headers={'Authorization': 'Bearer
>>>>> {}'.
>>>>>                                              format(
>>>>>                                                  TOKEN
>>>>>                                              )},
>>>>>                                     verify=CA_CERT_PATH)
>>>>>
>>>>>             if response.ok:
>>>>>                 res_dict = json.loads(response.text)
>>>>>             else:
>>>>>                 # handle this
>>>>>
>>>> But it is not working, it is doing just the same as when I was using it
>>>> without watch and RV.
>>>>
>>>>
>>>>
>>>> On 13 October 2016 at 19:13, 'Daniel Smith' via Kubernetes user
>>>> discussion and Q&A <kubernetes-users@googlegroups.com> wrote:
>>>>
>>>>> Watch definitely should do the thing you want.
>>>>>
>>>>> In a loop:
>>>>> 1. List. Note the returned RV. This gives you the current state of the
>>>>> system.
>>>>> 2. Watch, passing the RV.
>>>>> 3. Process events until the watch closes or times out.
>>>>>
>>>>> On Thu, Oct 13, 2016 at 7:43 AM, Diego Rodríguez Rodríguez <
>>>>> diegorgz...@gmail.com> wrote:
>>>>>
>>>>>> Thanks for the advice but I have already tried the `watch` parameter
>>>>>> and as far as I understand it does not work as expected. It gives me the
>>>>>> same output as if I did not use the `watch` parameter even if I provide 
>>>>>> the
>>>>>> first `resourceVersion` in order to be sure that it is comparing with the
>>>>>> first state of the job.
>>>>>>
>>>>>> I am making reference to http://kubernetes.io/docs/api-
>>>>>> reference/extensions/v1beta1/operations/ on the 'watch changes to an
>>>>>> object of kind Job' section.
>>>>>>
>>>>>> To be honest I have never used something like this `watch` parameter
>>>>>> before, so I may be misunderstanding it, but I am afraid I am not.
>>>>>>
>>>>>> On 13 October 2016 at 16:22, Rodrigo Campos <rodrig...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> You can probably watch via the API and get notified, but Haven't
>>>>>>> used it.
>>>>>>>
>>>>>>> But if you are building something not trivial, you may be better off
>>>>>>> creating a third party resource that handles this and creates kubernetes
>>>>>>> jobs under the hood. You will have more flexibility, etc.
>>>>>>>
>>>>>>>
>>>>>>> On Thursday, October 13, 2016, Diego Rodríguez Rodríguez <
>>>>>>> diegorgz...@gmail.com> wrote:
>>>>>>>
>>>>>>>> I have already created an issue
>>>>>>>> <https://github.com/kubernetes/kubernetes/issues/34715> in Github.
>>>>>>>>
>>>>>>>> Building on top of that, and closely related to
>>>>>>>> https://github.com/kubernetes/kubernetes/issues/34363 and to
>>>>>>>> https://github.com/kubernetes/kubernetes/pull/18827 I have the
>>>>>>>> following question. Since I am planning to use Kubernetes as a 
>>>>>>>> executing
>>>>>>>> backend for the workflow execution environment I am building, at some 
>>>>>>>> point
>>>>>>>> we will need to know the current state of a job. Well, this time has 
>>>>>>>> come,
>>>>>>>> and for now the only thing I am able to do is polling the API (
>>>>>>>> kubernetes.io/docs/api-reference/extensions/v1beta1/operations/).
>>>>>>>> Taking into account that this is unacceptable, and I only did it for
>>>>>>>> testing purposes, what is the recommended way in Kubernetes for doing 
>>>>>>>> so?
>>>>>>>> (Keep in mind that I should do it from inside the cluster, no kubectl
>>>>>>>> command) If it is not possible right now, will it be possible soon? I 
>>>>>>>> am
>>>>>>>> working on a quite big project and since it is starting, and the first
>>>>>>>> problems we are facing are related to this issues, it is time to 
>>>>>>>> decide if
>>>>>>>> I continue with Kubernetes or I have to start looking for alternatives.
>>>>>>>>
>>>>>>>> Thanks for your commitment
>>>>>>>>
>>>>>>>> On Wednesday, 12 October 2016 23:54:16 UTC+2, Derek Carr wrote:
>>>>>>>>>
>>>>>>>>> The quota controller observes pod deletions, and queues the
>>>>>>>>> pertinent quotas for processing in response.  The replenishment 
>>>>>>>>> should be
>>>>>>>>> relatively quick now.
>>>>>>>>>
>>>>>>>>> We actually exercise this behavior in e2e runs.
>>>>>>>>>
>>>>>>>>>    1. What version of Kubernetes are you running?
>>>>>>>>>    2. How many quotas?
>>>>>>>>>    3. How many pods?
>>>>>>>>>
>>>>>>>>> My guess is that there is a conflict of some kind during the quota
>>>>>>>>> sync, and the 5 minutes you are seeing is a full re-sync interval for 
>>>>>>>>> the
>>>>>>>>> quota controller.
>>>>>>>>>
>>>>>>>>> It would be good to try to determine what the conflict is so we
>>>>>>>>> can fix.
>>>>>>>>>
>>>>>>>>> If you have a quick set of steps I can run to reproduce that would
>>>>>>>>> be great.
>>>>>>>>>
>>>>>>>>> I recommend opening an issue with the above details and cc
>>>>>>>>> @derekwaynecarr
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Derek
>>>>>>>>>
>>>>>>>>> On Wednesday, October 12, 2016 at 3:18:07 PM UTC-4, David Eads
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Quota reconcilation of pods was supposed to run at a faster
>>>>>>>>>> rate.  I'm not sure if 5 minutes is that faster rate.
>>>>>>>>>>
>>>>>>>>>> *Derek:* Do you recall where and how the fast pod replenishment
>>>>>>>>>> is?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wednesday, October 12, 2016 at 1:28:45 PM UTC-4, Daniel Smith
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Ah, you're blocked because the quota check reconciles slowly.
>>>>>>>>>>> The quick fix is probably to just get more quota.
>>>>>>>>>>>
>>>>>>>>>>> +David who may know of an already existing issue about this.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Oct 12, 2016 at 5:39 AM, Diego Rodríguez Rodríguez <
>>>>>>>>>>> diego...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I have already configured my cluster to test what you have
>>>>>>>>>>>> stated. What I have done so far is to create a ResourceQuota which 
>>>>>>>>>>>> takes
>>>>>>>>>>>> care that there will not be more than 4 pods running. Then I ask 
>>>>>>>>>>>> for say 20
>>>>>>>>>>>> jobs to be completed.
>>>>>>>>>>>>
>>>>>>>>>>>> What happens in reality is that the first 4 jobs are completed
>>>>>>>>>>>> and then, even though the pods are completed and therefore 
>>>>>>>>>>>> resources are
>>>>>>>>>>>> available, it takes some minutes (around 4-5 minutes) to have 
>>>>>>>>>>>> another 4
>>>>>>>>>>>> jobs being completed. Indeed, what you said is true, however it is 
>>>>>>>>>>>> no
>>>>>>>>>>>> practical because a delay of minutes can not be assumed.
>>>>>>>>>>>> The waiting jobs events look like this:
>>>>>>>>>>>> `FailedCreate    Error creating: pods
>>>>>>>>>>>> "d59fa9b2-6c6b-4dc5-b149-7f89b35421bf-10-" is forbidden:
>>>>>>>>>>>> Exceeded quota: compute-resources, requested: pods=1, used: pods=4,
>>>>>>>>>>>> limited: pods=4`
>>>>>>>>>>>> So, it fails because there are no resources but it is trying it
>>>>>>>>>>>> again only after some minutes. This behavior is far from the 
>>>>>>>>>>>> desired one,
>>>>>>>>>>>> which is relaying on Kubernetes for the execution of a set of 
>>>>>>>>>>>> tasks no
>>>>>>>>>>>> matter the resources available, just getting them executed as soon 
>>>>>>>>>>>> as
>>>>>>>>>>>> possible.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks
>>>>>>>>>>>>
>>>>>>>>>>>> On Monday, 10 October 2016 19:24:46 UTC+2, Daniel Smith wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> If the system lacks resources, Pods will remain "Pending"
>>>>>>>>>>>>> until resources become available. Cluster scalers may use pending 
>>>>>>>>>>>>> pods as a
>>>>>>>>>>>>> signal that the cluster size should be increased.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 10, 2016 at 5:58 AM, Diego Rodríguez Rodríguez <
>>>>>>>>>>>>> diego...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello, I have a doubt about how Kubernetes' jobs are handled.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a queue to submit certain amount of incoming tasks
>>>>>>>>>>>>>> (Celery and RabbitMQ take care of this). Each one of this tasks 
>>>>>>>>>>>>>> is, in
>>>>>>>>>>>>>> fact, a *workflow* which will be executed in a worker
>>>>>>>>>>>>>> (Celery worker with a DAG executor inside). Each step of the
>>>>>>>>>>>>>> *workflow* is a Docker image with an input file and an
>>>>>>>>>>>>>> output file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My question is, if I submit jobs from the workflow engine
>>>>>>>>>>>>>> directly to the Kubernetes API, what happens I at some point 
>>>>>>>>>>>>>> there are no
>>>>>>>>>>>>>> more resources? Will the remaining tasks be kept or will they be 
>>>>>>>>>>>>>> lost? My
>>>>>>>>>>>>>> goal is to treat Kubernetes' jobs as a black box to submit works 
>>>>>>>>>>>>>> to. This
>>>>>>>>>>>>>> works are of a very different and heterogeneous nature and I 
>>>>>>>>>>>>>> wouldn't need
>>>>>>>>>>>>>> to bother with what is inside them because they are dockerized 
>>>>>>>>>>>>>> and executed
>>>>>>>>>>>>>> by Kubernetes at some point.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To sum up, I already have the layer of Celery workers with a
>>>>>>>>>>>>>> DAG executor inside which knows the right order of the tasks and 
>>>>>>>>>>>>>> knows how
>>>>>>>>>>>>>> to manage everything concerning the *workflow*. These
>>>>>>>>>>>>>> components will submit jobs (through Kubernetes API) and will 
>>>>>>>>>>>>>> wait for them
>>>>>>>>>>>>>> to be executed and then continue with the remaining tasks asking 
>>>>>>>>>>>>>> Kubernetes
>>>>>>>>>>>>>> to run them until the *workflow* ends.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have read about a somehow related issue in Github:
>>>>>>>>>>>>>> https://github.com/kubernetes/kubernetes/pull/17787
>>>>>>>>>>>>>> https://github.com/kubernetes/kubernetes/pull/18827
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I couldn't determine if this is closed or it is coming in a
>>>>>>>>>>>>>> future release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks in advance!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>> Google Groups "Kubernetes user discussion and Q&A" group.
>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>>> it, send an email to kubernetes-use...@googlegroups.com.
>>>>>>>>>>>>>> To post to this group, send email to
>>>>>>>>>>>>>> kubernet...@googlegroups.com.
>>>>>>>>>>>>>> Visit this group at https://groups.google.com/grou
>>>>>>>>>>>>>> p/kubernetes-users.
>>>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> Google Groups "Kubernetes user discussion and Q&A" group.
>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>> it, send an email to kubernetes-use...@googlegroups.com.
>>>>>>>>>>>> To post to this group, send email to
>>>>>>>>>>>> kubernet...@googlegroups.com.
>>>>>>>>>>>> Visit this group at https://groups.google.com/grou
>>>>>>>>>>>> p/kubernetes-users.
>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Kubernetes user discussion and Q&A" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to kubernetes-users+unsubscr...@googlegroups.com.
>>>>>>>> To post to this group, send email to kubernetes-users@googlegroups.
>>>>>>>> com.
>>>>>>>> Visit this group at https://groups.google.com/grou
>>>>>>>> p/kubernetes-users.
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to a topic in
>>>>>>> the Google Groups "Kubernetes user discussion and Q&A" group.
>>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>>>>>> pic/kubernetes-users/qd5Ua2UqQNo/unsubscribe.
>>>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>>>> kubernetes-users+unsubscr...@googlegroups.com.
>>>>>>> To post to this group, send email to kubernetes-users@googlegroups.
>>>>>>> com.
>>>>>>> Visit this group at https://groups.google.com/group/kubernetes-users
>>>>>>> .
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Kubernetes user discussion and Q&A" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to kubernetes-users+unsubscr...@googlegroups.com.
>>>>>> To post to this group, send email to kubernetes-users@googlegroups.
>>>>>> com.
>>>>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to a topic in the
>>>>> Google Groups "Kubernetes user discussion and Q&A" group.
>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>>>> pic/kubernetes-users/qd5Ua2UqQNo/unsubscribe.
>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>> kubernetes-users+unsubscr...@googlegroups.com.
>>>>> To post to this group, send email to kubernetes-users@googlegroups.com
>>>>> .
>>>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion and Q&A" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Kubernetes user discussion and Q&A" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/kubernetes-users/qd5Ua2UqQNo/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
  • Re: [kubernetes-... David Eads
    • Re: [kubern... Derek Carr
      • Re: [ku... Diego Rodríguez Rodríguez
        • Re:... Rodrigo Campos
        • Re:... Diego Rodríguez Rodríguez
        • Re:... 'Daniel Smith' via Kubernetes user discussion and Q&A
        • Re:... Diego Rodríguez Rodríguez
        • Re:... Diego Rodríguez Rodríguez
        • Re:... 'Daniel Smith' via Kubernetes user discussion and Q&A
        • Re:... Diego Rodríguez Rodríguez
        • Re:... 'Daniel Smith' via Kubernetes user discussion and Q&A
        • Re:... ravi prasad l r

Reply via email to