[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-17 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-801122674 Very nice! Thanks a lot for your efforts! This is an automated message from the Apache Git Service. To respond

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-11 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-796665243 I was concerned that this would happen to you. A new hire in our company is facing the same issue. Looks like Azure is having some issues with cryptocurrency mining on their

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-794477315 It failed again: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8955=logs=9401bf33-03c4-5a24-83fe-e51d75db73ef=72901ab2-7cd0-57be-82b1-bca51de20fba

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-794286699 https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8955=results This is an automated message from the

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-794285369 Here's how to setup azure: https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-794233139 Have you setup your CI with the password as well, and verified the change? This is an automated message from

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793577861 > Why the s3 access key is need to be added? Because some S3 end to end tests work against the real S3 service, not some mock or "fake" service.

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793576430 Sorry, I meant ``` SECRET_GLUE_SCHEMA_ACCESS_KEY: $[variables.IT_CASE_GLUE_SCHEMA_ACCESS_KEY] SECRET_GLUE_SCHEMA_SECRET_KEY:

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793560044 The `-z` approach should work: https://github.com/apache/flink/blob/master/flink-end-to-end-tests/test-scripts/common_s3.sh#L25 but it's worth a try. Can you also

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-09 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793518186 Oh, I didn't notice that the main CI isn't working either. The condition looks fine. Maybe the problem is that we are not setting the environment variables here:

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-08 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793459808 My commit needs to be included into the PR so that it ends up in master. If this environment variable is not available (for example here during CI verification, or with

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-08 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-793006791 There are no secrets setup for the main CI (which runs the pull request validation). We can not do this, because people could steal our credentials by opening a pull request

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-08 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-792650616 https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8949=results This is an automated message from the

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-06 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-791962050 https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8944=results This is an automated message from the

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-05 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-791455142 Also note that you need to include this commit if you want to make the secrets forwarding work:

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-05 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-791348220 Looks like the test failed: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8940=logs=9401bf33-03c4-5a24-83fe-e51d75db73ef=72901ab2-7cd0-57be-82b1-bca51de20fba

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-04 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-791200406 Both CIs are green, however, the e2e test didn't execute on my CI. I'll quickly try to fix that. This is an

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-04 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-790878966 I pushed your rebased branch also again: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8938=results

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-04 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-790878304 yes, rebasing to the latest master should fix the issue! This is an automated message from the Apache Git

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-03 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-790381549 Thanks for your review @dannycranmer! Before we can merge this PR, we need to make sure that CI is passing without credentials (this is the case in the PR CI

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-03 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-790375946 I pushed this PR to my personal azure, where the CI credentials are provided: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8929=results

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-03-03 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-790375151 Looks like the e2e test is failing. Can you write the e2e test in a way that it only executes if the credentials are available? (personal Azure accounts won't have

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-01-27 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-768331002 Thanks a lot for the clarification This is an automated message from the Apache Git Service. To respond to the

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-01-25 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766724268 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-01-25 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-767029789 You don't need to open a PR for every fix, you can just keep (force) pushing to the branch. Can you close the old PR? I'll review the PR tomorrow.

[GitHub] [flink] rmetzger commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration

2021-01-25 Thread GitBox
rmetzger commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766724268 What's the relationship of this PR to https://github.com/apache/flink/pull/14490 ? This is an automated