[ https://issues.apache.org/jira/browse/BEAM-14431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550097#comment-17550097 ]
Danny McCormick commented on BEAM-14431: ---------------------------------------- This issue has been migrated to https://github.com/apache/beam/issues/21659 > Handle nulls using SnowflakeIO > ------------------------------ > > Key: BEAM-14431 > URL: https://issues.apache.org/jira/browse/BEAM-14431 > Project: Beam > Issue Type: Bug > Components: io-java-snowflake > Affects Versions: 2.38.0 > Reporter: Varun > Priority: P2 > Labels: snowflake > Attachments: csv_mapper_data.png, snowflake_data.png > > > When SnowflakeIO is used within python beam code using ReadFromSnowflake [1] > ; the returned data isn't able to list the nulls from snowflake side. > The nulls from snowflake becomes b'N' ; which is a wrong value and gives > incorrect value for the column. In case of strings the actual value can also > be 'N'. So it becomes tough to handle it in csv_mapper. > To mitigate this we are using query instead of table and using ifnull / > coleasce within the select query for String columns. For non-string columns > we are converting back b'N' to null or empty as per requirement. > > [1] : > [https://beam.apache.org/releases/pydoc/2.33.0/apache_beam.io.snowflake.html#apache_beam.io.snowflake.ReadFromSnowflake] > > -- This message was sent by Atlassian Jira (v8.20.7#820007)