Jake/Chris,
Thanks for the info.
When I try to install thrift v0.9.3 from source, I get an error as follows
while running `make check`:
...
...
[junit] Running org.apache.thrift.protocol.TestTProtocolUtil
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
0.062 sec
[junit] Running org.apache.thrift.protocol.TestTSimpleJSONProtocol
[junit] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
0.046 sec
BUILD FAILED
/tmp/thrift-0.9.3/lib/java/build.xml:202: Test
org.apache.thrift.protocol.TestTSimpleJSONProtocol failed
Total time: 17 seconds
make[3]: *** [check-local] Error 1
make[3]: Leaving directory `/tmp/thrift-0.9.3/lib/java'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory `/tmp/thrift-0.9.3/lib/java'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/tmp/thrift-0.9.3/lib'
make: *** [check-recursive] Error 1
--
κρισhναν
On Thu, Mar 17, 2016 at 7:32 PM, Chris Bannister <[email protected]>
wrote:
> I've used the latest thrift to generate go code, and then manually created
> executor config which works and is able to launch jobs.
>
> On Thu, 17 Mar 2016, 1:55 p.m. Jake Farrell, <[email protected]> wrote:
>
>> Hi Krish
>> We are using Thrift with go for all our api calls to Aurora, would
>> recommend you use Thrift 0.9.3 to interact with the api.
>>
>> happy to help answer any questions you might have
>>
>> -Jake
>>
>> On Thu, Mar 17, 2016 at 9:43 AM, Krish <[email protected]> wrote:
>>
>>> Thanks, Bill.
>>>
>>> Well I have started my foray into the the thrift API today. And I think
>>> I am stuck with some thrift configs.
>>>
>>> Does it matter if I use thrift v0.9.0 on the client side to talk with
>>> aurora using thrift 0.9.1? Are they compatible? I couldn't find any
>>> changelog or compatibility statement on the thrift project site.
>>>
>>>
>>> Since Aurora v0.12 uses thrift version 0.9.1, and the debian repos have
>>> 0.9.0, I had to compile the thrift compiler v0.9.1 from source. However,
>>> when I try to generate golang code, I think I hit a compiler bug:
>>> krish@krish:/tmp
>>> > thrift --gen go api.thrift
>>> ./gen-go//api/ttypes.go:2623:6: missing ',' in composite literal
>>> ./gen-go//api/ttypes.go:2624:19: expected '==', found '='
>>> WARNING - Running 'gofmt -w ./gen-go//api/ttypes.go' failed.
>>>
>>> I can modify the golang code by hand, but I would like to play it safe
>>> and use the working compiler from the debian repos.
>>>
>>> Also, when I use thrift v0.9.0, and try to integrate code into a test
>>> golang app, it fails to find "thriftlib/api" package. Anyone faced a
>>> similar error and gone past it?
>>> I have already done a `go get
>>> git.apache.org/thrift.git/lib/go/thrift/...`
>>> <http://git.apache.org/thrift.git/lib/go/thrift/...>
>>>
>>>
>>>
>>>
>>> --
>>> κρισhναν
>>>
>>> On Wed, Mar 16, 2016 at 10:30 PM, Bill Farner <[email protected]>
>>> wrote:
>>>
>>>> Regarding documentation - Maxim is correct that there isn't much in the
>>>> way of independent/holistic docs for the thrift API. There is, however,
>>>> scant javadoc-style documentation within the IDL spec itself:
>>>> https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache/aurora/gen/api.thrift
>>>>
>>>> If you are looking to use the thrift API directly, the most difficult
>>>> API method will be defining the ExecutorConfig.data value when calling
>>>> createJob. Please don't hesitate to ask for assistance if you get to that
>>>> point!
>>>>
>>>> On Wed, Mar 16, 2016 at 9:19 AM, Maxim Khutornenko <[email protected]>
>>>> wrote:
>>>>
>>>>> 1. All APIs require thrift inputs of the structs specified, and return
>>>>>> thrift values only in Response.result field.
>>>>>
>>>>> Correct. There is also 'details' field that may have additional
>>>>> messages (of error or informational nature)
>>>>>
>>>>> 2. Is there a set of examples in the documentation to help understand
>>>>>> Thrift API better?
>>>>>
>>>>> The thrift API is largely undocumented. There is an effort to bring up
>>>>> a fully supported REST API that will presumably get documented and become
>>>>> much easier to use. It's mostly in flux now.
>>>>>
>>>>> 3. createJob(JobDescription desc, Lock lock):
>>>>>
>>>>> This is the API to use when you a brand new service or adhoc (batch)
>>>>> job created. The JobDescription is populated from the .aurora config. You
>>>>> may want to trace "aurora job create" client command implementation to see
>>>>> how it happens.
>>>>>
>>>>> 4. What is the Lock object? I see that some APIs require locking and
>>>>>> some don't. For example, createJob needs a Lock object as parameter, & I
>>>>>> am
>>>>>> assuming that it is required so that one does not create multiple jobs
>>>>>> with
>>>>>> the same JobKey.
>>>>>
>>>>> Ignore this object as it's an echo of the old client updater. It's now
>>>>> deprecated and will be removed soon. You can pass NULL for now.
>>>>>
>>>>> 5. addInstances(AddInstancesConfig cfg, Lock lock):
>>>>>
>>>>> Another echo of the client updater but this time it's got a second
>>>>> life. Check out its new signature and comments in the api.thrift. It's
>>>>> essentially a "scale-out" API that can add instances to the existing job
>>>>> without changing the underlying task assumptions.
>>>>>
>>>>> 6. getPendingResult(TaskQuery taskquery):
>>>>>
>>>>> It's actually 'getPendingReason' and is currently used exclusively by
>>>>> the UI to get the reason for a task PENDING state.
>>>>>
>>>>> 7. setQuota & getQuota for setting user level quotas.
>>>>>
>>>>> This is to set role-level quota. Currently only required for tasks
>>>>> with 'production=True'. Search through our docs for more details.
>>>>>
>>>>> 8. killTasks to kill all running instances of a job in the cluster.
>>>>>
>>>>> It's quite versatile and can be used to kill some or all instances of
>>>>> the job.
>>>>>
>>>>> 9. startJobUpdate(JobUpdateRequest request, string message):
>>>>>
>>>>> Your observations are correct. This is the main API to change your
>>>>> service job in any way (including adding, removing or modifying
>>>>> instances).
>>>>>
>>>>> An aurora scheduling question is if I start a job with 5 instances,
>>>>>> and there are resources available to run only 4 of them, does the entire
>>>>>> job block, or only the 5th instance of the job blocks?
>>>>>
>>>>> Scheduler will try to schedule as many instances as it can. Those that
>>>>> will not find resources will remain in PENDING state until more resources
>>>>> are available. In your particular example only the 5th will remain
>>>>> PENDING.
>>>>>
>>>>>
>>>>> On Wed, Mar 16, 2016 at 5:54 AM, Krish <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I was going through the Aurora Thrift API to determine how to add new
>>>>>> jobs.
>>>>>> I am using aurora v0.12 released last month and have upgraded to
>>>>>> mesos v0.25 accordingly.
>>>>>>
>>>>>> Below is a summary of my (very limited) understanding of some APIs, &
>>>>>> would help it if someone can point out flaws in my understanding:
>>>>>>
>>>>>> 1. All APIs require thrift inputs of the structs specified, and
>>>>>> return thrift values only in Response.result field.
>>>>>>
>>>>>> 2. Is there a set of examples in the documentation to help
>>>>>> understand Thrift API better?
>>>>>>
>>>>>> 3. createJob(JobDescription desc, Lock lock):
>>>>>> This is basically the API to replace the Aurora DSL/.aurora files
>>>>>> for job configuration.
>>>>>>
>>>>>> 4. What is the Lock object? I see that some APIs require locking
>>>>>> and some don't. For example, createJob needs a Lock object as
>>>>>> parameter, &
>>>>>> I am assuming that it is required so that one does not create
>>>>>> multiple jobs
>>>>>> with the same JobKey.
>>>>>>
>>>>>> 5. addInstances(AddInstancesConfig cfg, Lock lock):
>>>>>> By the naming convention, it seems this is used to increase the
>>>>>> number of instances of a job. It will not result in stopping of
>>>>>> current
>>>>>> instances of the job.
>>>>>>
>>>>>> My second explanation for this API: Since it uses a set of
>>>>>> instanceIds, this is used for adding already running job in slaves to
>>>>>> the
>>>>>> internal data structures of Aurora to track the job.
>>>>>>
>>>>>> 6. getPendingResult(TaskQuery taskquery):
>>>>>> Return the reason (in string) about why the job is PENDING. For
>>>>>> example: insufficient CPU.
>>>>>>
>>>>>> 7. setQuota & getQuota for setting user level quotas.
>>>>>>
>>>>>> 8. killTasks to kill all running instances of a job in the
>>>>>> cluster.
>>>>>>
>>>>>> 9. startJobUpdate(JobUpdateRequest request, string message):
>>>>>> Used for updating jobs with the new TaskConfig specified. Can be
>>>>>> used if resource requirement changes. For example: If I wanted aurora
>>>>>> to
>>>>>> update the version of container used for a job using
>>>>>> TaskConfig.Container
>>>>>> attribute.
>>>>>>
>>>>>>
>>>>>> An aurora scheduling question is if I start a job with 5 instances,
>>>>>> and there are resources available to run only 4 of them, does the entire
>>>>>> job block, or only the 5th instance of the job blocks?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> --
>>>>>> κρισhναν
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
