vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-848056191
@danny0405 please take a look. Once CI passes, we can land.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-848040019
@danny0405 we left this hanging a bit. Let me re-review this and get it
landing in some form.
--
This is an automated message from the Apache Git Service.
To respond to
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-838080706
Please let me know once you have tried the configs.
```
public class FileSystemViewStorageConfig extends DefaultHoodieConfig {
// Property Names
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-838049296
@danny0405 if your concern is memory usage alone and not the time taken to
spin up the timeline server for each write task, then worth exploring if you
want to use
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-837953122
Longer term, I think we should think about running the timeline server with
the job manager, rather than task manager IMO.
I was trying to understand how incremental
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-837816065
> each task manager is, sending RPC calls to the timeline server running
locally?
each write task is sending a RPC call to itself? Could you confirm that.
>we
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-837465319
>each write task write out these records buffer (grouping by file group id)
using a write client there.
Understood. So each task manager is, sending RPC calls to the
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834595116
>each writer task takes a AbstractWriteClient, and there may be multiple
write tasks in one process.
@danny0405 please help me understand this better. By one process,
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-834116492
>This change is introducing another instance which will double the overhead.
What benefits do you see by doing this ?
Would it though? I mean, its just an object
