Re: Question on implementing Celeborn client,

[orpl]

Is there some way to use Celeborn API to check if CommitFiles succeeds in step 6? Currently we are testing with TPC-DS 10TB data, and some heavy query (query 24) occasionally fails with:

 Caused by: java.io.IOException: Premature EOF from inputStream

We are speculating that this error occurs because we miss the check in step 6.

It looks like this is due to a bug in our code that rssShuffleClient.cleanup()
is called before rssShuffleClient.mapperEnd().

Thanks,

--- Sungwoo