What if we make annotations a dictionary?
-r
On Aug 19, 2019, at 9:54 AM, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote:
>> I second the idea of recording the request/transaction id as part of the
>> activation record. We don’t currently do that, but we do have a caused-by
>> annotation for composite activations. I suggest another annotation which is
>> the transaction id.
>
> There are few other cases where I felt a need to record some "meta"
> ids in activation
>
> 1. While integrating with Lambda or other such compute stack I would
> like to save the responseId from the backend system in activation so
> as to enable fetching logs later based on the responseId from previous
> call.
> 2. In Mesos or k8s setup one may want to record the Mesos taskId, k8s
> podId for correlating with some system logs later
>
> Should we consider it as yet another annotation or add new sub
> document `meta` and make it a property there?
>
> If we make it an annotation then it becomes tricky to index this
> property in db due to the document structure. For couchdb you can do
> it by extracting the key when generating the view but for other db's
> we typically have to add a computed field at time of persistence.
>
> Another option would be to introduce a new sub structure say `meta`
> where we store such ids as flat keys
>
> "meta" : {
> "transactionId" : "xxx",
> "podId" : "ow_xxx"
> }
>
> Such an structure would be easier to index compared to one based on
> annotations
>
> Chetan Mehrotra
>
> {
> "activationId": "e40c9340aea340f58c9340aea320f5e9",
> "annotations": [
> {
> "key": "causedBy",
> "value": "sequence"
> },
> {
> "key": "kind",
> "value": "nodejs:10"
> },
> {
> "key": "timeout",
> "value": false
> },
> {
> "key": "limits",
> "value": {
> "concurrency": 200,
> "logs": 10,
> "memory": 256,
> "timeout": 60000
> }
> }
> ],
> "cause": "f3380aaeca7d4791b80aaeca7d5791f6",
> "duration": 5015,
> "end": 1562344247086,
> "entityType": "activation",
> "response": { ... },
> "start": 1562344242071,
>
> }
>
>> On Mon, Aug 19, 2019 at 5:34 AM Rodric Rabbah <rod...@gmail.com> wrote:
>>
>> I second the idea of recording the request/transaction id as part of the
>> activation record. We don’t currently do that, but we do have a caused-by
>> annotation for composite activations. I suggest another annotation which is
>> the transaction id.
>>
>> Chetan’s pr is straightforward and I think achieves the goal of linking
>> parents to children for external tracing.
>>
>> -r
>>
>>> On Aug 19, 2019, at 5:02 AM, Erez Hadad <er...@il.ibm.com> wrote:
>>>
>>> Hi folks,
>>>
>>> My two cents: recall my "shared context" presentation about using a
>>> semi-transparent execution context that is shared across multiple
>>> consequent invocations. Similar to the transaction id being discussed.
>>> https://cwiki.apache.org/confluence/download/attachments/74689638/Shared%20Context%20in%20OpenWhisk%20Invocations.pptx?api=v2
>>>
>>> Specifically, you may want to consider the implementation options in slide
>>> 11:
>>> 1. One particular option of interest is to embed the shared context id ( =
>>> transaction id) in the activation id. When the new activation id is
>>> generated, the transaction id can be taken from the caller's activation id
>>> and embedded in the new activation id. This way the activation id remains
>>> unique per invocation but the transaction id can be extracted from it if
>>> needed, manually or via a client library (e.g., extending the OW js
>>> library). Kind of a minimal change.
>>> 2. Another possibly valuable option is to have the transaction id used as
>>> a key for retrieving data (e.g., key/value) from a 3rd-party fast service,
>>> such as redis. This can be left open for the user of the transaction id or
>>> provide a reference implementation in the client library. Such data can
>>> also be pre-fetched before the invocation starts (e.g., from CouchDB /
>>> Redis), but again, at the cost of a higher code change.
>>>
>>> Regards,
>>> -- Erez
>>>
>>>
>>>
>>>
>>>
>>>
>>> From: Rodric Rabbah <rod...@gmail.com>
>>> To: dev@openwhisk.apache.org
>>> Cc: tyson.nor...@gmail.com
>>> Date: 17/08/2019 03:36
>>> Subject: [EXTERNAL] Re: Passing TransactionId as part of action
>>> invocation
>>>
>>>
>>>
>>> Of course if the transaction id is the same as the activation id (of the
>>> composite action) the solutions are comparable.
>>>
>>> The downside of using transaction id is that we have no APIs to query by
>>> it today, and it?s not recorded in the activation metadata. So while it?s
>>> useful for external tracing (no objections to doing it), I think from an
>>> openwhisk API point of view we still have a gap.
>>>
>>> -r
>>>
>>> On Aug 16, 2019, at 4:58 PM, Chetan Mehrotra <chetan.mehro...@gmail.com>
>>> wrote:
>>>
>>>>> I think if OW SDK, and sequences/compositions, propagate X-Request-Id
>>>>> header (using the existing transaction id/X-Request-Id), the parent is
>>> not
>>>>> needed?
>>>>
>>>> Thats what I counting on as we just need to correlate calls made for a
>>>> given flow corelated with time to get a sequence of flow.
>>>>
>>>>> - expose the transaction id to runtime container
>>>>> - propagate the transaction id in requests initiated from runtime
>>>>> container/controller
>>>>
>>>> Makes sense. Would open a ticket capturing these changes then
>>>> Chetan Mehrotra
>>>>
>>>>> On Fri, Aug 16, 2019 at 7:44 AM Tyson Norris <tysonnor...@gmail.com>
>>> wrote:
>>>>>
>>>>> I think if OW SDK, and sequences/compositions, propagate X-Request-Id
>>>>> header (using the existing transaction id/X-Request-Id), the parent is
>>> not
>>>>> needed? i.e. there may be 2 parts to this effort:
>>>>> - expose the transaction id to runtime container
>>>>> - propagate the transaction id in requests initiated from runtime
>>>>> container/controller
>>>>>
>>>>>
>>>>>
>>>>>> On Thu, Aug 15, 2019 at 10:38 AM Rodric Rabbah <rod...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> In general yes but I think generally do you need the transaction id or
>>> the
>>>>>> parent id for an activation?
>>>>>>
>>>>>> This issue is relevant -
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_openwhisk_issues_3083&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Oo9B0p_tCCWIIum5GpjjqA&m=RVvqHH3EKTdA9cNFz3n2dPeXV_wFUOU01I5vxvAqt8Q&s=eZzttWaoeE0WFe_MndFO1O6K1wA9hZbc-BKbRZa6kDM&e=
>>> .
>>>>>> I also recall in the early days of the composer, we wanted a way to
>>> query
>>>>>> parent/child activations but this requires new couch views and we
>>> didn't
>>>>>> pursue it.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 15, 2019 at 1:20 PM Chetan Mehrotra
>>> <chetan.mehro...@gmail.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Currently we pass the `activation_id` as part of `/run` call to any
>>>>>>> action runtime [1]. Would it be fine to also pass the `TransactionId`
>>>>>>> such that it can be accessed by action code?
>>>>>>>
>>>>>>> One usecase of this would be to enable tracing a sequence/composition
>>>>>>> by linking all activations which are part of same transaction in
>>>>>>> epsagon [2]
>>>>>>>
>>>>>>> Chetan Mehrotra
>>>>>>> [1]
>>>>>>>
>>>>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_openwhisk_blob_master_docs_actions-2Dnew.md-23activation&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Oo9B0p_tCCWIIum5GpjjqA&m=RVvqHH3EKTdA9cNFz3n2dPeXV_wFUOU01I5vxvAqt8Q&s=-vjUf19hoPEdgOTjnbs7jjHFqsJhWIRJjQSQYIme8xw&e=
>>>
>>>>>>> [2]
>>>>>>>
>>>>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__epsagon.com_blog_epsagon-2Dmakes-2Dtroubleshooting-2Dapache-2Dopenwhisk-2Da-2Dsnap_&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Oo9B0p_tCCWIIum5GpjjqA&m=RVvqHH3EKTdA9cNFz3n2dPeXV_wFUOU01I5vxvAqt8Q&s=IkGq6vR80pwlGlaXAkXHi_rV14IXnX0hc3smCWO8BM8&e=
>>>
>>>>>>>
>>>>>>
>>>
>>>
>>>
>>>
