maheshguptags commented on issue #8195:
URL: https://github.com/apache/hudi/issues/8195#issuecomment-1480701240

   Hi @danny0405,
   
   Thank you for joining me on the call. I tried your suggestion and deployed 
the code in k8s cluster and it is working fine.
   it seems it was an issue with the local IDE run.
   
   below is the configuration that worked for me.
   ```
   `table.type' = 'COPY_ON_WRITE',
   'write.operation'='insert',
   'hoodie.datasource.write.recordkey.field' = 'x,y',
   'read.streaming.enabled'='true',
   'read.start-commit'='earliest',
   'hoodie.clean.max.commits'='3',
   'hoodie.cleaner.commits.retained' = '3',
   'hoodie.clean.async'='true',
   'hoodie.cleaner.policy' = 'KEEP_LATEST_COMMITS',
   'hoodie.parquet.small.file.limit'='104857600',
   'hoodie.parquet.max.file.size'= '536870912',
   'hoodie.parquet.compression.codec'='snappy',
   'clustering.schedule.enabled'='true', 
   'clustering.async.enabled'='true', 
   'hoodie.clustering.inline'= 'true', 
   'hoodie.clustering.inline.max.commits'= '2',
   'hoodie.clustering.plan.strategy.max.bytes.per.group'= '107374182400',
   'hoodie.clustering.plan.strategy.max.num.groups'= '1',
   'write.tasks'='2'
   ```
   
   
   **DAG Plan** 
   
![image](https://user-images.githubusercontent.com/115445723/227127466-2f6cf2bb-9f19-4215-89a2-4a7d5dcd1a52.png)
   
   **timeline**
   <img width="1236" alt="image" 
src="https://user-images.githubusercontent.com/115445723/227127542-bd678fb1-15e0-4d91-8fa6-b71f3577095e.png";>
   
   Thank you very much for all the help. Really appreciate!!! 
   
   Thanks 
   - Mahesh  
   
   
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to