You can just add the target partitioning filter to your MERGE or UPDATE
condition, e.g.
MERGE INTO target
USING source
ON target.key = source.key AND target.year = year(current_date())
...
Best,
Burak
On Thu, Oct 31, 2019, 10:15 PM ayan guha wrote:
>
> Hi
>
> we have a scenario where we have
If the dataset contains a column like changed_at/created_at you can use this as
watermark and filter out rows that have changed_at/created_at before the
watermark.
Best Regards
Roland Johann
Software Developer/Data Engineer
phenetic GmbH
Lütticher Straße 10, 50674 Köln, Germany
Mobil: +49
should not a where clause on the partition field help with that? I am
obviously missing something in the question.
Regards,
Gourav
On Thu, Oct 31, 2019 at 9:15 PM ayan guha wrote:
>
> Hi
>
> we have a scenario where we have a large table ie 5-6B records. The table
> is repository of data from
Hi
we have a scenario where we have a large table ie 5-6B records. The table
is repository of data from past N years. It is possible that some updates
take place on the data and thus er are using Delta table.
As part of the business process we know updates can happen only within M
years of past
