zhaizhenhua opened a new issue, #7338:
URL: https://github.com/apache/hudi/issues/7338

   **_Tips before filing an issue_**
   
   - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
   yes
   - Join the mailing list to engage in conversations and get faster support at 
[email protected].
   
   - If you have triaged this as a bug, then file an 
[issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
   
   **Describe the problem you faced**
   
   Now we use Flink sql+hudi+presto as the solution of streaming and batching. 
Hudi is used for storage, Flink sql is used for processing, and presto is used 
as the query engine. Considering the data volume, stream processing only deals 
with the data of the current day, but the history will change. At this time, we 
need to repair the historical data by batch processing in the early morning 
every day. When we tested and verified according to this expectation, we found 
that when stream processing is running, batch processing can no longer be run 
(at this time, stream processing will report an error). We cannot directly load 
data only through stream processing. The status of historical data will change 
(order status). Therefore, batch processing is required for regular data 
repair. The final expectation of this solution is the same processing engine, 
the same code and the same target table. It is not so much a problem as a 
solution. At present, there are few online introductions in this
  regard. No suitable method was found. So ask for help here.
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   1.Start a stream processing task, write data to partition table A, and only 
write today's data.
   
   2.Use the same finksql code to start in batch mode and write data to 
partition table A. Reload data for the past 30 days.
   
   3.After the batch task is submitted successfully, when it starts to write 
data to the hudi table, the stream processing task starts to report an error 
and restarts continuously. The error message is as follows:
   Exception:Ignoring the request to fail job because the job is already 
failing. The ignored failure cause is
   org.apache.hudi.exception.HoodieException: Executor executes action [handle 
write metadata event for instant 20221125180423588] error
   
   **Expected behavior**
   
   A clear and concise description of what you expected to happen.
   
   **Environment Description**
   
   * Hudi version :
   0.13
   * Spark version :
   
   * Hive version :
   2.1.1
   * Hadoop version :
   3.0.0
   * Storage (HDFS/S3/GCS..) :
   HDFS
   * Running on Docker? (yes/no) :
   NO
   
   **Additional context**
   
   Add any other context about the problem here.
   
   **Stacktrace**
   
   ```Add the stacktrace of the error.```
   
   Exception:Ignoring the request to fail job because the job is already 
failing. The ignored failure cause is
   org.apache.hudi.exception.HoodieException: Executor executes action [handle 
write metadata event for instant 20221125180423588] error


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to