[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733725927 thank all for patch review! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733725405 @cloud-fan yes, I have a JIRA account named 'duripeng' This is an automated message from the Apache Git

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-07 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-723525322 Thank you for the reply @cloud-fan > 1. Is it only to fix `FileAlreadyExistsException`? AFAIK the builtin Spark file source doesn't support concurrent writes, not

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-09-30 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-701193725 @HyukjinKwon Can you give further review or help to find a member for further review, thx : ) This is an

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-09-19 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-695199796 gentle ping @xuanyuanking @tgravescs for further review, thx This is an automated message from the Apache Git

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-09-03 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-686516518 Thanks @Ngone51 , gentle ping @cloud-fan @xuanyuanking for further review This is an automated message from

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-08-29 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-683275178 gentle ping @Ngone51 for further review, or involving other committer to review this patch, thanks :) This is

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-08-20 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-677463648 @Ngone51 Sorry for late update, please have a further review, thanks :) This is an automated message from the

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-08-03 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-667951102 Gentle ping @Ngone51 for further review, thanks This is an automated message from the Apache Git Service. To

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-30 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-666098220 Gentle ping @Ngone51 for further review, thanks :) This is an automated message from the Apache Git Service.

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-17 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-660008806 gentle ping @dongjoon-hyun @cloud-fan @xuanyuanking to review this pr :) This is an automated message from the

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-16 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-659792693 > You can push a new commit(like two commits, the second one revert the first one) to trigger jenkins job. thanks for your advice, will try it :)

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-16 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-659410866 gentle ping @cloud-fan @dongjoon-hyun @SparkQA to retest this pr This is an automated message from the Apache

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-13 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-657538307 retest this please @cloud-fan @dongjoon-hyun This is an automated message from the Apache Git Service. To

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-08 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-655893223 Thanks for your reply @turboFei Gentle ping @dongjoon-hyun @cloud-fan @xuanyuanking to review this pr, thank you

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-07 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-654632940 @turboFei Thanks for your comments. Actually I think there is no partition-explosion cost. Commit task output dir is generated by `FileFormatDataWriter.newOutputWriter`, no

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-07-05 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-653932931 gentle ping @cloud-fan @dongjoon-hyun @xuanyuanking @turboFei @LuciferYang @advancedxy plz have a review, thx