As a workaround, I am wondering if you can do this for now. Let's say you can records deleted from commit c1 to now.
- Do a snapshot query at commit c1 - Do a snapshot query at latest commit - Diff the two It will inefficient, since it has to scan all the data twice, but may work functionally? On Thu, Mar 26, 2020 at 1:47 PM Vinoth Chandar <[email protected]> wrote: > Currently, soft deletes will show up in the incremental stream, while hard > deletes will not.. > > We are debating how to add this features, since it has come up few times > recently.. > > May be this can be a good discuss thread for that? :) > > On Thu, Mar 26, 2020 at 1:42 PM Joaquim S <[email protected]> wrote: > >> Folks, >> >> I am looking at DMS integration and have a question. >> >> It is clear that the incremental queries only show incrementals ( :) ) . I >> need to extract the deletes too from a specific commit (ideally sparksql >> or >> hive). >> >> What am I missing, is there another way to query the timeline to get the >> records that were deleted after a specific commit? >> >> Thank you! >> >
