[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1006625360 Damn, you are right. Then my explanation doesn't make much sense. So we have a fix now, which we don't understand? Nice. @zentol is back next week, he might have an idea. I'm not the biggest fan of my "dependencyManagement" approach, as this is prone to breaking as soon as we bump our dependencies. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1006612141 Thanks. I've tried this with the latest maven version, and it still failed. We are using an oudated Maven, because it supports "double shading" Flink dependencies: https://lists.apache.org/thread/ph8zjbf21vy3kv52l6trx5h7vbymncvg -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1006405358 Ah, nice catch. I've pushed another commit in my PR to see if that fixes the issue. As far as I understand the original issue, the grpc dependency is defined as a dependency range. The range contains only "1.41.1", and for some reason maven fails to resolve that range. By putting the 1.41.1 version explicitly into the dependency management, we disable the dependency resolution mechanism. Maven will see that we have transitive grpc dependencies, and will just take the version defined in the dependency management. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1006372738 It seems that with this change, compiling Flink will work: https://github.com/apache/flink/pull/18275/commits/6e568b86e5ce14fe03c6a74f91bb686ed393f0be Now, the license check is failing. Not sure why. Is it related to this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1006053132 Thanks a lot for giving it a try. This is very annoying. Maybe its a bug in the highly outdated maven version we are using? I've created myself a fork of this PR to debug this issue (I don't know what timezone you are in @galenwarren, but I'll soon go to bed ;) ) ... if you have time and energy to look into the issue, this is what I would try (by pushing commits): - log what maven is trying to access on maven central. I've done that already in my fork: https://github.com/apache/flink/pull/18275 - try wiping the Azure maven cache (by changing the cache keys in the pipeline definition) - try using the latest maven version (just to see if its a bug in maven) (we can't use the latest maven version for Flink CI, but it will at least help us understand the problem better) - try some of the suggestions from SO? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-1005986537 Have you tried what I [suggested here](https://github.com/apache/flink/pull/15599#issuecomment-827326084)? (adding that `-U` flag to maven?) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-831048950 Note: There is some related activity in contributing another GCS FS implementation: https://issues.apache.org/jira/browse/FLINK-19481?focusedCommentId=17338198=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17338198 Feel free to add to the discussion in the Jira ticket! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-827326084 Yeah, I'm happy to help, I also think I know what's causing this, but I don't know how to resolve it I pushed this PR onto - my Personal CI, where it passed without problems. - onto Flink's CI, where it passed without problems. - I manually restarted this PRs CI, and it broke again. The only difference between all these runs is that the broken PR CI run is downloading data from the maven cache on Azure (e.g. the local .m2 directory contains stuff in the failure case). @galenwarren Could you do me a favor and add a "-U" argument here: https://github.com/apache/flink/blob/master/tools/ci/maven-utils.sh#L107. This COULD resolve the issue. > This can be confirmed from the maven log, where no "Downloading xxx" can be found until the failure. The reason for the missing log statements is that we've disabled the download messages from maven: https://github.com/apache/flink/blob/master/tools/ci/maven-utils.sh#L103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
rmetzger commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-826593416 I can not explain what's happening here with Azure. I deployed this PR do my personal Azure account, and it was able to download the dependency. Here's the run: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=9061=logs=9401bf33-03c4-5a24-83fe-e51d75db73ef=3965ec8d-ff2e-54b6-c0a6-70172f3e6f55 I'll further investigate the issue ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
