[
https://issues.apache.org/jira/browse/BEAM-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865187#comment-16865187
]
Steven Ensslen commented on BEAM-5510:
--------------------------------------
Workaround is now
{color:#569cd6}class{color}
{color:#4ec9b0}ConvertNonStringsToStringsFn{color}{color:#d4d4d4}({color}{color:#4ec9b0}beam{color}{color:#d4d4d4}.{color}{color:#4ec9b0}DoFn{color}{color:#d4d4d4}):{color}
{color:#ce9178}"""{color}
{color:#ce9178} Workaround for
https://issues.apache.org/jira/browse/BEAM-5510{color}
{color:#ce9178} """{color}
{color:#569cd6}def{color}
{color:#dcdcaa}process{color}{color:#d4d4d4}({color}{color:#9cdcfe}self{color}{color:#d4d4d4},
{color}{color:#9cdcfe}record{color}{color:#d4d4d4}):{color}
{color:#c586c0}for{color}{color:#d4d4d4} key
{color}{color:#569cd6}in{color}{color:#d4d4d4} record:{color}
{color:#c586c0}try{color}{color:#d4d4d4}:{color}
{color:#c586c0}if{color}{color:#d4d4d4}
({color}{color:#4ec9b0}type{color}{color:#d4d4d4}(record[key])
{color}{color:#d4d4d4}=={color}{color:#d4d4d4} date):{color}
{color:#d4d4d4} record[key] {color}{color:#d4d4d4}={color}{color:#d4d4d4}
record[key].strftime({color}{color:#ce9178}'%Y-%m-{color}{color:#569cd6}%d{color}{color:#ce9178}'{color}{color:#d4d4d4}){color}
{color:#c586c0}elif{color}{color:#d4d4d4}
({color}{color:#4ec9b0}type{color}{color:#d4d4d4}(record[key])
{color}{color:#d4d4d4}=={color}{color:#d4d4d4} time):{color}
{color:#d4d4d4} record[key] {color}{color:#d4d4d4}={color}{color:#d4d4d4}
record[key].strftime({color}{color:#ce9178}'%H:%M:%S'{color}{color:#d4d4d4}){color}
{color:#c586c0}elif{color}
{color:#dcdcaa}hasattr{color}{color:#d4d4d4}(record[key],
{color}{color:#ce9178}"strftime"{color}{color:#d4d4d4}):{color}
{color:#d4d4d4} record[key] {color}{color:#d4d4d4}={color}{color:#d4d4d4}
record[key].strftime(DATE_FORMAT){color}
{color:#c586c0}elif{color}{color:#d4d4d4}
({color}{color:#4ec9b0}type{color}{color:#d4d4d4}(record[key])
{color}{color:#d4d4d4}!={color}
{color:#4ec9b0}str{color}{color:#d4d4d4}):{color}
{color:#d4d4d4} record[key] {color}{color:#d4d4d4}={color}
{color:#4ec9b0}str{color}{color:#d4d4d4}(record[key]){color}
{color:#c586c0}except{color} {color:#4ec9b0}ValueError{color}
{color:#c586c0}as{color}{color:#d4d4d4} e:{color}
{color:#d4d4d4} logging.error({color}{color:#ce9178}"failed to convert
'{color}{color:#569cd6}{d}{color}{color:#ce9178}', error
{color}{color:#569cd6}{e}{color}{color:#ce9178}"{color}{color:#d4d4d4}.format({color}{color:#9cdcfe}d{color}{color:#d4d4d4}={color}{color:#d4d4d4}record[key],{color}{color:#9cdcfe}e{color}{color:#d4d4d4}={color}{color:#d4d4d4}e)){color}
{color:#d4d4d4} record[key] {color}{color:#d4d4d4}={color}
{color:#4ec9b0}str{color}{color:#d4d4d4}(record[key]){color}
{color:#c586c0}yield{color}{color:#d4d4d4} record{color}
> Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery
> ---------------------------------------------------------------------------
>
> Key: BEAM-5510
> URL: https://issues.apache.org/jira/browse/BEAM-5510
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.6.0
> Reporter: Pascal Gula
> Priority: Major
>
> When trying to write some row in BigQuery that include a python datetime
> object, the marshaling used to save a row in BigQuery is impossible.
> {code:java}
> File
> "/home/pascal/Wks/GitHub/PEAT-AI/Albatros/venv/local/lib/python2.7/site-packages/apache_beam/internal/gcp/json_value.py",
> line 124, in to_json_value
> raise TypeError('Cannot convert %s to a JSON value.' % repr(obj))
> TypeError: Cannot convert datetime.datetime(2018, 9, 25, 18, 57, 18, 108579)
> to a JSON value. [while running 'save/WriteToBigQuery']
> {code}
> However, this is something perfectly feasible, as `google-cloud-python`
> supports it since this issue has been solved:
> [https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2957]
> thanks to this pull request:
> [https://github.com/GoogleCloudPlatform/google-cloud-python/pull/3426/files]
> As similar approach could be taken for the `json_value.py` helper.
> Is there any workaround that can be applied to solve this issue?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
