bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-586108720
Ok, after reading https://github.com/apache/incubator-hudi/issues/800 (which
is similar), I think this is a memory issue (even though GC seems to be a
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-586071719
Ok, thanks for this.
I have run the jobs again. First, insert 4m records, then upsert 3m of them,
then upsert 4m, then upsert 4m. The two jobs
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-586071719
Ok, thanks for this.
I have run the jobs again. First, insert 4m records, then upsert 3m of them,
then upsert 4m, then upsert 4m. The two jobs
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-586071719
Ok, thanks for this.
I have run the jobs again. First, insert 4m records, then upsert 3m of them,
then upsert 4m, then upsert 4m. The two jobs
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585512613
@vinothchandar Thanks for taking the time to reply!
Let me describe the simplest example of this problem on a tiny COW data set:
Create a data
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585512613
@vinothchandar Thanks for taking the time to reply!
Let me describe the simplest example of this problem on a tiny COW data set:
Create a data
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585512613
@vinothchandar Thanks for taking the time to reply!
Let me describe the simplest example of this problem on a tiny COW data set:
Create a data
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585512613
@vinothchandar Thanks for taking the time to reply!
Let me describe the simplest example of this problem on a tiny COW data set:
Create a data
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585512613
@vinothchandar Thanks for taking the time to reply!
Let me describe the simplest example of this problem on a tiny COW data set:
Create a data
bwu2 edited a comment on issue #1328: Hudi upsert hangs
URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-585471970
An update on this: the job did eventually complete. But took 2.4 hours
(other similar jobs ran more than 9 hours before we had to kill them). This
10 matches
Mail list logo
