[GitHub] [incubator-hudi] vinothchandar commented on issue #1384: [SUPPORT] Hudi datastore missing updates for many records

[GitBox]

vinothchandar commented on issue #1384: [SUPPORT] Hudi datastore missing 
updates for many records
URL: https://github.com/apache/incubator-hudi/issues/1384#issuecomment-596226463
 
 
   Hmmm the datasource does fail the commit if there are such errors..
   
   ```
    } else {
         log.error(s"$operation failed with $errorCount errors :")
         if (log.isTraceEnabled) {
           log.trace("Printing out the top 100 errors")
           writeStatuses.rdd.filter(ws => ws.hasErrors)
             .take(100)
             .foreach(ws => {
               log.trace("Global error :", ws.getGlobalError)
               if (ws.getErrors.size() > 0) {
                 ws.getErrors.foreach(kt =>
                   log.trace(s"Error for key: ${kt._1}", kt._2))
               }
             })
         }
         false
       }
   ``` 
   
   In any case, having some information on workload,  MOR vs COW and % of 
missing records would help debug more.. Did you also have the issue on 0.4.7? 
or only after you upgraded to 0.5.1? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services