liangyu-1 commented on PR #6935: URL: https://github.com/apache/hadoop/pull/6935#issuecomment-2241869801
> > But in my scenario, I am using flink's fileSystem API, and I have read the source code of the flink API that it used FileSystem.create(Path f) , which means that if I want to use the CreateFlag.IGNORE_CLIENT_LOCALITY in hadoop, I have to change the source code of flink filesystem API and rebuild the whole flink project. > > I think this will also happens in most computation engines because most engines directly uses function FileSystem.create(Path f) . This will cause too many extra work. > > Got it. But I am sorry to disagree your opinion. There is one flexible interface however upstream system do not invoke it, thus we should push the upstream system to update. Another side, if config as this PR do, this will affect whole side run at this Client which could not be expected. In one word, suggest to proposal and submit PR at Flink side. Thanks again. @Hexiaoqiao , This does not only happens in Flink, but also other engines like Spark etc. If I only sunmit a PR at FLINK side, the other engines' API (like SPARK, SPARK Structured stream) will not be able to use this feature and we need to rebuilt the whole computation project whenever we choose to use a new computation engine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
