joshhamann commented on issue #10822:
URL: https://github.com/apache/hudi/issues/10822#issuecomment-1981391484
Here is my configuration:
{'hoodie.table.name': 'analytics_events',
'hoodie.datasource.write.recordkey.field': 'event_uuid',
'hoodie.datasource.write.partitionpath.field':
'event_create_time_date_partition', 'hoodie.datasource.write.table.name':
'analytics_events', 'hoodie.datasource.write.operation': 'upsert',
'hoodie.datasource.write.precombine.field': 'event_create_time',
'hoodie.datasource.write.keygenerator.class':
'org.apache.hudi.keygen.SimpleKeyGenerator',
'hoodie.parquet.compression.codec': 'snappy'}
I was under the impression that by not using global bloom, this upsert
process would happen within the particular partition in question, and
therefore, if the volume stays fairly across partitions (in this case, days),
we shouldn't see job duration increasing. Is that not the case?
Let me know if I can provide any more information. Thanks for your help on
this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
