[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-1018109938 > I think that the partitions that the reducer is trying to read are not finalized by the shuffle server. This log statement would have shown that. I still don't know how the r

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-1017349727 Thanks @otterc, changed code as you suggested, https://github.com/pan3793/spark/commit/1c14eb9c4d2710c6a906781d80246e8adc1628ff, and have sent logs to you via email, please tak

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-1015001194 > To clarify, these logs are with a version of spark/shuffle service without modifications ? Or were there any code changes made to them ? Thx. Oops, I forgot link the co

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-1010826471 @otterc I reproduced this issue today and sent a email to you with logs and spark confs you requested [here](https://github.com/apache/spark/pull/35076#issuecomment-1010458523)

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-1002955536 https://github.com/apache/spark/pull/32287#discussion_r625773529 NOT sure if it's related, but I do hang here several times. @Ngone51 -- This is an automated message from

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-997928493 Sounds like there is something wrong earlier than I dealt with, I will investigate in this direction, and use this change as a workaround until the issue has been addressed to u

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-997398321 After the change, my 1T TPCDS passed one round, but failed in the second round, I'm not sure if it's related or just another issue. I'm a newbie in this area, I'll appreciate it

[GitHub] [spark] pan3793 commented on pull request #34934: [SPARK-37675][CORE][SHUFFLE] Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

pan3793 commented on pull request #34934: URL: https://github.com/apache/spark/pull/34934#issuecomment-997186091 @mridulm @otterc It's ready for review now, please take a look when you have time. -- This is an automated message from the Apache Git Service. To respond to the message, plea