dm-tran commented on issue #2020: URL: https://github.com/apache/hudi/issues/2020#issuecomment-680789158
@bvaradar The workflow of my applications is the following one: 1. Initialize a Hudi table using a spark batch and "bulk insert". 2. Launch a spark structured streaming application that consumes messages from Kafka and saves them to Hudi, using "upsert". Yesterday, I used property "hoodie.consistency.check.enabled=true" for step 2, but I forgot to use it for step 1. Sorry about that. I used this property for both steps today. > Given, that you are able to reproduce very easily and I have not seen this issue reported by anyone, Would you be able to provide us a self-contained code to reproduce this. Actually, I have been successfully running a dozen of structured streaming applications for several weeks. I got this "java.io.FileNotFoundException" for the first time a few days ago, when launching a structured streaming application for a new data source. Providing a self-contained code to reproduce this error isn't easy. It might be related to the input data or workload. > If not, can you turn on INFO level logging and catch the logs till you hit the exception and attach them. Sure, I have been running the structured streaming application from the start for several hours, with INFO level logging. So far, it works fine. I will attach the logs if the exception is raised. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
