milenkovicm commented on code in PR #1541:
URL:
https://github.com/apache/datafusion-ballista/pull/1541#discussion_r3036643171
##########
ballista/core/proto/ballista.proto:
##########
@@ -458,15 +454,15 @@ message TaskKilled {
message ShuffleWritePartition {
uint64 partition_id = 1;
+ string path = 2;
uint64 num_batches = 3;
uint64 num_rows = 4;
uint64 num_bytes = 5;
- optional uint64 file_id = 6;
- bool is_sort_shuffle = 7;
-
- // reserved after removing path
- reserved 2;
- reserved "path";
+ // Remote shuffle service fields (empty when using local shuffle).
+ string remote_shuffle_app_id = 6;
Review Comment:
Is this unique per session context ?
##########
ballista/core/proto/ballista.proto:
##########
@@ -458,15 +454,15 @@ message TaskKilled {
message ShuffleWritePartition {
uint64 partition_id = 1;
+ string path = 2;
uint64 num_batches = 3;
uint64 num_rows = 4;
uint64 num_bytes = 5;
- optional uint64 file_id = 6;
- bool is_sort_shuffle = 7;
-
- // reserved after removing path
- reserved 2;
- reserved "path";
+ // Remote shuffle service fields (empty when using local shuffle).
+ string remote_shuffle_app_id = 6;
+ int32 remote_shuffle_id = 7;
+ string remote_shuffle_server_host = 8;
Review Comment:
Can this be part of external UniffleConfig, similar to BallistaConfig but
registered when Uniffle extension is registered?
##########
ballista/core/proto/ballista.proto:
##########
@@ -458,15 +454,15 @@ message TaskKilled {
message ShuffleWritePartition {
uint64 partition_id = 1;
+ string path = 2;
uint64 num_batches = 3;
uint64 num_rows = 4;
uint64 num_bytes = 5;
- optional uint64 file_id = 6;
- bool is_sort_shuffle = 7;
-
- // reserved after removing path
- reserved 2;
- reserved "path";
+ // Remote shuffle service fields (empty when using local shuffle).
+ string remote_shuffle_app_id = 6;
+ int32 remote_shuffle_id = 7;
Review Comment:
Is this unique per execution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
