Luca,
I have a setup of the petstore sample from aws:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-from-example.html
<https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-from-example.html>
The only way that I can reproduce your issue is if:
a. the region is incorrect
b. the resource is wrong
If I have those correct, then I get the api response I expect
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Fri Jun 18 14:45:57 EDT 2021'
Key: 'lineageStartDate'
Value: 'Fri Jun 18 14:45:57 EDT 2021'
Key: 'fileSize'
Value: '184'
FlowFile Attribute Map Content
Key: 'Access-Control-Allow-Origin'
Value: '*'
Key: 'Connection'
Value: 'keep-alive'
Key: 'Content-Length'
Value: '184'
Key: 'Content-Type'
Value: 'application/json'
Key: 'Date'
Value: 'Fri, 18 Jun 2021 18:45:57 GMT'
Key: 'X-Amzn-Trace-Id'
Value: '
Key: 'aws.gateway.api.endpoint'
Value: 'execute-api'
Key: 'aws.gateway.api.resource'
Value: '/pets'
Key: 'aws.gateway.api.status.code'
Value: '200'
Key: 'aws.gateway.api.status.message'
Value: 'OK'
--------------------------------------------------
[
{
"id": 1,
"type": "dog",
"price": 249.99
},
{
"id": 2,
"type": "cat",
"price": 124.99
},
{
"id": 3,
"type": "fish",
"price": 0.99
}
]
> On Jun 18, 2021, at 13:00, Luca Giovannini <[email protected]>
> wrote:
>
>
> Thank you for your help, Otto!
>
> In the meantime I thought to get around the obstacle by trying to reach the
> API via a python script using the AWS SDK (to be then run by NiFi in an
> ExecuteScript processor), but that proved to be not at all straightforward
> and I am still struggling. The closest thing I found to a short example on
> how to do this is the following:
> https://gist.github.com/bjinwright/8c0d1e3a65017e1479d61e7dbbffb79f
> <https://gist.github.com/bjinwright/8c0d1e3a65017e1479d61e7dbbffb79f>
> I will try to adapt it to my case and hope it works.
>
> But this leaves me with a big question:
> I HAVE to be doing something wrong myself with the InvokeAWSGatewayApi… how
> can it be that nobody else has had my problem?
> It seems to me that I am using it for such a simple request! Nobody in the
> NiFi community has ever used it to access to a ‘execute-api’ resource in the
> ‘eu-west-1’ region? Really?
> Or am I not understanding the error that I am getting?
>
> Sorry, I am a little frustrated by this… any insight will be really
> appreciated! 😊
>
> Thank you all very much,
> Luca
>
>
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/ <http://www.linkedin.com/in/lucagio/>
> T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
> Dedagroup Public Services Srl – Sede di Casalecchio di Reno, Via del Lavoro 67
> www.dedagroup.it/public-services <http://www.dedagroup.it/public-services>
>
> <image001.jpg> <http://www.dedagroup.it/home>
>
> Da: Otto Fowler <[email protected]>
> Inviato: mercoledì 16 giugno 2021 21:37
> A: [email protected]
> Oggetto: Re: How to access to an Amazon API?
>
> I hope to be able to try it tomorrow.
>
>
> On Jun 16, 2021, at 09:01, Luca Giovannini <[email protected]
> <mailto:[email protected]>> wrote:
>
>
> Thank you Otto,
>
> please let me know if you need any additional information to set up the
> processor with my configuration!
>
> Just an additional element: I cloned the flow in a different machine with
> NiFi 1.11.4 on it and the same exact problem shows up (same error in logs,
> same absence of the raw API call in the provenance data).
>
> Thank you again for the help you might be able to provide me…I am really
> stuck here! ☹
>
> Luca
>
>
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Flucagio%2F&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240328254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ddCKIhrWdvGZ91f4ZZM4B%2FM1WUfHLe6hXLN9%2Bik0kgc%3D&reserved=0>
> T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
> Dedagroup Public Services Srl – Sede di Casalecchio di Reno, Via del Lavoro 67
> www.dedagroup.it/public-services
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fpublic-services&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240338212%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=U%2FHdfYaeuKonyAi2w2fPuqwHAUUu7bq8y4Ug2M4eAFE%3D&reserved=0>
>
> <image001.jpg>
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fhome&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240338212%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Pf6yTHHxPqi1BSK%2BNi7GhsOtzr5q5%2Bp0k1xFDiBbk30%3D&reserved=0>
>
> Da: Otto Fowler <[email protected] <mailto:[email protected]>>
> Inviato: martedì 15 giugno 2021 17:04
> A: [email protected] <mailto:[email protected]>
> Oggetto: Re: How to access to an Amazon API?
>
> That error means that we passed info into AWS api ( asked the region to find
> the resource ) and it said nope, and tried to guess.
> I would have to try to setup the processor with your configuration ( enough
> to try to parse, ok if it fails later because of cred or whatever ).
>
>
>
>
> On Jun 15, 2021, at 06:26, Luca Giovannini <[email protected]
> <mailto:[email protected]>> wrote:
>
>
> Thank you Otto,
>
> I can confirm that the resource API is correct: I tested it successfully with
> Postman by doing a GET on the request URL
> (https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/prod/test
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fzsslnu1awf.execute-api.eu-west-1.amazonaws.com%2Fprod%2Ftest&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240338212%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=73TmFt24FwsmDA4XItR3rNsRAzRWeUKkkjJ6oBNfBIU%3D&reserved=0>)
> and providing AccessKey, SecretKey and AWS Region in the “Authorization” tab
> of the Postman client.
>
> I also checked log/nifi-app.log and the only informative line that I found is:
> 2021-06-15 11:59:24,046 INFO [Timer-Driven Process Thread-8]
> c.a.i.DefaultServiceEndpointBuilder {execute-api, eu-west-1} was not found in
> region metadata, trying to construct an endpoint using the standard pattern
> for this region: 'execute-api.eu-west-1.amazonaws.com
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexecute-api.eu-west-1.amazonaws.com%2F&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240348170%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Jk2lb2fQKW68rWqZE58%2FGucY40jNN%2By%2BnvqXp798EAY%3D&reserved=0>'.
> Which seems to indicate that the processor is not using the endpoint I
> provided as parameter
> (https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/)
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fzsslnu1awf.execute-api.eu-west-1.amazonaws.com%2F)&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240348170%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZC1MuVZ6Lu7W7CZ6gd%2BXN%2F6zHeibpB55%2BHNACJfD1D0%3D&reserved=0>...
>
> I also checked the provenance data (both from the “Provenance” button of the
> output FlowFile listing and from the “Data Provenance” element of the main
> menu) for records of the call, but I could not find any provenance data at
> all for the processor InvokeAWSGatewayApi under scrutiny (my flow right now
> is made up just by that processor alone and a LogMessage one).
>
> Is this new information shedding any more light on the problem for you?
>
> Thank you very much,
> Luca
>
>
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Flucagio%2F&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240358127%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cQZBEfUBxCc7XdZGT1BPIwgYHV%2FrbrMgJVLMoV782Kw%3D&reserved=0>
> T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
> Dedagroup Public Services Srl – Sede di Casalecchio di Reno, Via del Lavoro 67
> www.dedagroup.it/public-services
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fpublic-services&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240358127%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UgF5004XwN0429Bxs399PFcXss9Ww9Ml%2BjZ1F0adwA0%3D&reserved=0>
>
> <image001.jpg>
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fhome&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240368082%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pP2kDfigIs0dqOClSeEgK3mAv0LjlneGxN2o0ctPCCo%3D&reserved=0>
>
> Da: Otto Fowler <[email protected] <mailto:[email protected]>>
> Inviato: lunedì 14 giugno 2021 19:48
> A: [email protected] <mailto:[email protected]>
> Oggetto: Re: How to access to an Amazon API?
>
> **ATTENZIONE** Questo messaggio proviene da un ACCOUNT ESTERNO, presta
> attenzione ad eventuali link o allegati al suo interno.
>
> You should also confirm the resource api you are trying to call is correct.
>
> On Jun 14, 2021, at 12:04, Luca Giovannini <[email protected]
> <mailto:[email protected]>> wrote:
>
>
> Hello everybody!
>
> I have no experience with Amazon services but I now have to access to an API
> hosted on AWS.
> I have been given the following parameters:
> HTTP Method GET
> Amazon Region eu-west-1
> Access Key ID **** (hidden
> but known)
> Secret Access Key **** (hidden but
> known)
> Amazon Gateway Api ResourceName /test
> Amazon Gateway Api Endpoint
> https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/xxxx
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fzsslnu1awf.execute-api.eu-west-1.amazonaws.com%2Fxxxx&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240368082%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TZNsR3PNIVFBWadZK1uv1CVXEIlqkx%2F5N6Q9gxlTFvc%3D&reserved=0>
> (anonymized)
>
> I have used the “InvokeAWSGatewayApi” and set the previous 6 parameters (and
> only those), but I got {"message": "Internal server error"} as an answer.
> Maybe I did not format the ResourceName correctly (I see here [*] that the
> ResourceName is usually much longer)?
> I am using NiFi 1.12.1.
>
> Thank you very much for your help!
>
> Luca Giovannini
>
> [*]
> https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.aws.amazon.com%2Fgeneral%2Flatest%2Fgr%2Faws-arns-and-namespaces.html&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240378035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=seQM5n6oijNQhTfBMMeNTswOhinxv3Azgnni9BVFH2A%3D&reserved=0>
>
>
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Flucagio%2F&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240378035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gDAGbyDNyFCQRjorOTM4c%2FwtAzPtAvYWBWwAVAqxgks%3D&reserved=0>
> T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
> Dedagroup Public Services Srl – Sede di Casalecchio di Reno, Via del Lavoro 67
> www.dedagroup.it/public-services
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fpublic-services&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240387996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xHsjcIVIN5sBAEEzSpLhQ3ONUHuro9EWMQFFFt%2BPySM%3D&reserved=0>
>
> <image001.jpg>
> <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dedagroup.it%2Fhome&data=04%7C01%7CLuca.Giovannini%40dedagroup.it%7Ccae5d160c8fa4e0f7e1208d930fe1be2%7Cbbf156d433fa4fee86f62cfcb1359ef0%7C0%7C0%7C637594690240387996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cOqqo8xf8BOWAV6Urz1%2BeQbi9l7IgdHsyexHc7ncDWY%3D&reserved=0>
>
> Le informazioni contenute in questo messaggio di posta elettronica sono
> riservate e confidenziali e ne e' vietata la diffusione in qualsiasi modo o
> forma. Qualora Lei non fosse la persona destinataria del presente messaggio,
> La invitiamo a non diffonderlo e ad eliminarlo, dandone gentilmente
> comunicazione al mittente.
>
> The information included in this e-mail and any attachments are confidential
> and may also be privileged. If you are not the correct recipient, you are
> kindly requested to notify the sender immediately, to cancel it and not to
> disclose the contents to any other person.
