[
https://issues.apache.org/jira/browse/BEAM-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847201#comment-16847201
]
Kishan Kumar commented on BEAM-6846:
------------------------------------
h4. [Kenneth
Knowles|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=kenn] I
tries to replace the TableRow object with Java Classes so while Reading the
Data From BigQuery I used SerializableFunction.
h4. {{private static <T> T mapRecordToObject(GenericRecord record, T object) \{
Assert.notNull(record, "record must not be null"); Assert.notNull(object,
"object must not be null"); final Schema schema =
ReflectData.get().getSchema(object.getClass());
Assert.isTrue(schema.getFields().equals(record.getSchema().getFields()),
"Schema fields didn’t match"); record.getSchema().getFields().forEach(d ->
PropertyAccessorFactory.forDirectFieldAccess(object).setPropertyValue(d.name(),
record.get(d.name()) == null ? record.get(d.name()) :
record.get(d.name()).toString())); return object; } PCollection<MyType> data =
pipeline.apply( BigQueryIO .read(new SerializableFunction<SchemaAndRecord,
MyType>() \{ public MyType apply(SchemaAndRecord schemaAndRecord) { return
mapRecordToObject(schemaAndRecord.getRecord(), new MyType()); } })
.from("mydataset:mytable"));}}
while reading numeric data type value from Big Query the numeric column
resultant is java.nio.HeapByteBuffer[pos=0 lim=16 cap=16].
So it would also be a blocker for me and if I am missing something or there is
any method through which I can resolve the issue then let me know.
h4.
> Beam Performance Degradation on Dataflow Runner
> -----------------------------------------------
>
> Key: BEAM-6846
> URL: https://issues.apache.org/jira/browse/BEAM-6846
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.7.0, 2.8.0
> Reporter: Kishan Kumar
> Priority: Major
>
> After Upgrading From 2.5.0 Version the jobs of Version 2.7.0 are taking a
> long time compared to 2.5.0.
> System Lag is also got introduced the Error Getting Generated:
> "Workflow failed. Causes: The Dataflow job appears to be stuck because no
> worker activity has been seen in the last 1h."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
