HeartSaVioR commented on issue #26821: [SPARK-20656][CORE]Support Incremental parsing of event logs in SHS URL: https://github.com/apache/spark/pull/26821#issuecomment-563502972 Thanks for cc.ing me, @dongjoon-hyun . I'll take a look. Btw, I think we have another JIRA issue for supporting incremental parsing [SPARK-28870](https://issues.apache.org/jira/browse/SPARK-28870) which has broader goal - run with any implementation of KVStore. At first glance, this patch could cover SPARK-29261 and with SPARK-29111 it may resolve SPARK-28870 altogether - though we struggled on the details previously so I need some time to go through deeply. @shahidki31 I guess you've been following through the previous discussions/efforts @Ngone51 and me, and @vanzin, @squito have been made. (#25577 and #25943, and relevant google docs in relevant JIRA issues/PRs) If not, it would worth to go through, as we've discussed in details.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
