I understand your point. Even if null elements are present, it would consider as not_null. I assumed this operator would be similar in function to Pandas isnull.
import pandas as pd pdf = pd.DataFrame([[1, 2, 3, 4, None], [11, 12, 13, None, 15]]) pdf.isnull() For instance in this one, element-wise check is done. With Regards, Vibhatha Abeykoon On Wed, Nov 18, 2020 at 2:28 PM Micah Kornfield <[email protected]> wrote: > It looks right. Could you clarify why you think it might not be expected > behavior? The arrays that are being constructed are of two different > types. One is an integer array (first example). The rest on List<Int>. > None of the lists in the examples are null (but they do contain null > elements). > > On Wed, Nov 18, 2020 at 11:19 AM Vibhatha Abeykoon <[email protected]> > wrote: > >> Hello, >> >> I am looking into the is_null compute and observed the following. >> >> import pyarrow as pa >> arw_ar = pa.array([1, 2, 3, 4, None]) >> arw_ar_1 = pa.array([[1, 2, 3, 4, None], [11, 12, 13, None, 15]]) >> arw_ar_2 = pa.array([[1, 2, 3, 4], [1, 2, 3, 4]]) >> arw_ar_3 = pa.array([[None, None, None, None, None], [11, 12, 13, None, >> 15]]) >> >> # Case 1 with random None value in a 1D array >> >> print(arw_ar.is_null()) >> >> # [ >> # false, >> # false, >> # false, >> # false, >> # true >> # ] >> >> # Case 2 with random None value in a 2-D array >> >> print(arw_ar_1.is_null()) >> >> # [ >> # false, >> # false >> # ] >> >> # Case 3 without random None value in a 2-D array >> print(arw_ar_2.is_null()) >> >> # [ >> # false, >> # false >> # ] >> >> # Case 4 with None value in a 2-D array >> print(arw_ar_3.is_null()) >> # [ >> # false, >> # false >> # ] >> >> Is this an expected behavior? >> >> >> With Regards, >> Vibhatha Abeykoon, >> >>
