Re: Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-20 Thread Eron Wright
It is not surprising to see fidelity issues with the YARN proxy. I suggest opening a ticket on Flink side to update the cancel-with-savepoint API to take the target directory as a query string parameter (of course, backwards compatibility should be maintained). On Wed, Sep 20, 2017 at 1:55 AM,

Re: Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-20 Thread Nico Kruber
Hi Emily, I'm not familiar with the details of the REST API either but if this is a problem with the proxy, maybe it is already interpreting the encoded URL and passes it on un-encoded - have you tried encoding the path again? That is, encoding the percent-signs: http://

Re: Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-19 Thread Emily McMahon
Thanks Eron & Fabian. The issue was hitting a yarn proxy url vs the node itself. For example this worked http:// {ip}:37716/jobs/1a0fd176ec8aabb9b8464fa481f755f0/cancel-with-savepoint/target-directory/s3%3A%2F%2F%2Fremit-flink But this did not http://

Re: Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-19 Thread Eron Wright
Good news, it can be done if you carefully encode the target directory with percent-encoding, as per: https://tools.ietf.org/html/rfc3986#section-2.1 For example, given the directory `s3:///savepoint-bucket/my-awesome-job`, which encodes to `s3%3A%2F%2F%2Fsavepoint-bucket%2Fmy-awesome-job`, I was

Re: Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-19 Thread Fabian Hueske
Hi Emily, thanks for reaching out. I'm not familiar with the details of the Rest API but Ufuk (in CC) might be able to help you. Best, Fabian 2017-09-19 10:23 GMT+02:00 Emily McMahon : > I've tried every combination I can think of to pass an s3 path as the > target

Rest API cancel-with-savepoint: 404s when passing path as target-directory

2017-09-19 Thread Emily McMahon
I've tried every combination I can think of to pass an s3 path as the target directory (url encode, include trailing slash, etc) I can successfully pass a local path as the target directory (ie /jobs/$jobID/cancel-with-savepoint/target-directory/tmp) so I don't think there's a problem with the