[GitHub] [hudi] rubenssoto commented on issue #1902: [SUPPORT] Hudi dont put the same day in the same file

Wed, 05 Aug 2020 08:05:50 -0700 


rubenssoto commented on issue #1902:
URL: https://github.com/apache/hudi/issues/1902#issuecomment-669246957


   Hi bvaradar, how are you?
   
   I had the same behavior
   <img width="1248" alt="Captura de Tela 2020-08-05 às 12 03 03" 
src="https://user-images.githubusercontent.com/36298331/89429232-b50b7980-d713-11ea-93b3-ca5fdbc8c36f.png";>
   
   My hudi configs
   
   hudi_options = {
               'hoodie.table.name': table_name,
               'hoodie.datasource.write.recordkey.field':'created_date_brt,id'
               'hoodie.datasource.write.table.name': table_name,
               'hoodie.datasource.write.operation': hudi_config.write_operation,
               'hoodie.datasource.write.hive_style_partitioning': 'true',
               'hoodie.combine.before.insert': hudi_config.write_operation in 
['insert','bulkinsert'] if 'true' else 'false',
               'hoodie.combine.before.upsert': hudi_config.write_operation == 
'upsert' if 'true' else 'false',
               'hoodie.datasource.write.precombine.field': 
hudi_config.precombined_column,
               'hoodie.datasource.write.keygenerator.class': 
'org.apache.hudi.keygen.ComplexKeyGenerator',
               'hoodie.parquet.small.file.limit': 943718400,
               'hoodie.parquet.max.file.size': 1073741824,
               'hoodie.parquet.block.size': 1073741824,
               'hoodie.copyonwrite.record.size.estimate': 
hudi_config.record_size_estimate,
               'hoodie.cleaner.commits.retained': 1,
               'hoodie.datasource.hive_sync.enable': 'true',
               'hoodie.datasource.hive_sync.table': table_name,
               'hoodie.datasource.hive_sync.database': 'datalake_raw',
               'hoodie.datasource.hive_sync.partition_fields': 'partitionpath',
               'hoodie.datasource.hive_sync.partition_extractor_class': 
'org.apache.hudi.hive.MultiPartKeysValueExtractor',
               'hoodie.datasource.hive_sync.jdbcurl': 
'jdbc:hive2://ec2-52-12-56-39.us-west-2.compute.amazonaws.com:10000'
           }
   
   
   Did I something wrong?
   
   Thank you
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to