[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-24 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-663646201 Hi Balaji, I think I've narrowed down my issue somewhat for my MOR table. I started again with a fresh table and the initial commits make sense, but after a time I notice It's

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-23 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-663189699 What do you mean by "runs serially with ingestion"? My understanding was that inline compaction happened in the same flow as writing so an inline compaction would simply slow down

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-23 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-663153261 @bvaradar I think the issue I'm facing is due to configuration, but I can't pinpoint what it is. I'm ending up with an extremely large number of files fo a single partition

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-23 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-663002064 Only 1 compaction.inflight now This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-23 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-663000848 I updated to master @ 743ef322b88d90be9775b889f6381925cdda5f35 and then applied the patch you linked above. The first batch that ran had several "RunCompactionActionExecutor"

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-22 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-662626520 1. I'm trying this now. 2. The stack trace is the one I provided above. This is an automated message from the

[GitHub] [hudi] ssomuah commented on issue #1852: [SUPPORT]

2020-07-21 Thread GitBox
ssomuah commented on issue #1852: URL: https://github.com/apache/hudi/issues/1852#issuecomment-661970919 I don't see any exceptions in the driver logs or executor logs. I see these two warnings in driver logs ``` 20/07/21 13:12:28 WARN IncrementalTimelineSyncFileSystemView: