carloea2 commented on code in PR #8488: URL: https://github.com/apache/texera/pull/8488#discussion_r3993441924
########## amber/src/main/python/core/models/table.py: ########## @@ -25,6 +26,20 @@ class Table(pandas.DataFrame): + @staticmethod + def empty_of(schema) -> pandas.DataFrame: + """ + The declared columns with no rows under them. + + ``from_tuple_likes`` reads the column names off the tuples it is given, + so with none to read it produces a frame of no columns at all, and an + operator naming any of its own columns raises KeyError. A port that + carried no rows still has a schema, and this is what it looks like as a + table. Building it through Arrow gives each column the dtype it would + have had with rows in it. + """ + return pa.Table.from_pylist([], schema=schema.as_arrow_schema()).to_pandas() Review Comment: Could this return a Table rather than a plain DataFrame? A TableOperator UDF using `yield from table.as_tuples()` now raises AttributeError when its input is empty and has a declared schema. The same UDF still works with populated input and with the previous empty-input path. I reproduced this against 2004373 with three local pytest cases: those two controls pass, while the declared empty input fails. Wrapping the frame in Table would preserve the existing UDF API while keeping the columns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
