Hey folks, I actively using observe method on my spark jobs and noticed interesting behavior: Here is an example of working and non working code: https://gist.github.com/Coola4kov/8aeeb05abd39794f8362a3cf1c66519c
In a few words, if I'm joining dataframe after some filter rules and it became empty, observations configured on the first dataframe never return any results, unless some action called on the empty dataframe specifically before join. Looks like a bug to me, I will appreciate any advice on how to fix this behavior.
