pengzhiwei2018 commented on pull request #2485: URL: https://github.com/apache/hudi/pull/2485#issuecomment-781777083
> @pengzhiwei2018 > > no, for example, today with spark structured streaming in a regular parquet if my tableA as a source to my streaming, if I reprocess/recreate tableA spark streaming will process all new files of my reprocessed tableA > > if for any reason I need to recreate my tableA, what will happen to my streams? > > I dont know if I make myself clear Hi @rubenssoto. If the table has recreated, the offset of the stream source should be reset(e.g. use another checkpoint directory or delete the old checkpoint directory). Otherwise, the old offset may not match the new recreated table and we cannot read data correctly. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
