Chris Larsen created BEAM-8029:
----------------------------------
Summary: Using BigQueryIO.read with DIRECT_READ causes Illegal
Mutation
Key: BEAM-8029
URL: https://issues.apache.org/jira/browse/BEAM-8029
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.14.0
Reporter: Chris Larsen
Code to read from BigQuery that is causing the issue:
{code:java}
pipeline
.apply(BigQueryIO
.read(SchemaAndRecord::getRecord)
.from(options.getTableRef())
.withMethod(Method.DIRECT_READ)
.withCoder(AvroCoder.of(schema)))
{code}
If we remove .withMethod(Method.DIRECT_READ) then there is no issue.
The error is:
{code:java}
org.apache.beam.sdk.util.IllegalMutationException: PTransform
BigQueryIO.TypedRead/Read(BigQueryStorageTableSource) mutated value
{"device_id": "rpi-rpi0-thermostat", "temperature_c": 20.0, "temperature_f":
52.0, "sample_time": 1564412307969368, "humidity": 74.3} after it was output
(new value was {"device_id": "rpi-rpi0-thermostat", "temperature_c": 20.0,
"temperature_f": 52.0, "sample_time": 1564412360458615, "humidity": 74.7}).
Values must not be mutated in any way after being output.
at
org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit
(ImmutabilityCheckingBundleFactory.java:134)
at org.apache.beam.runners.direct.EvaluationContext.commitBundles
(EvaluationContext.java:210)
at org.apache.beam.runners.direct.EvaluationContext.handleResult
(EvaluationContext.java:151)
at
org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult
(QuiescenceDriver.java:262)
at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle
(DirectTransformExecutor.java:189)
at org.apache.beam.runners.direct.DirectTransformExecutor.run
(DirectTransformExecutor.java:126)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.beam.sdk.util.IllegalMutationException: Value
{"device_id": "rpi-rpi0-thermostat", "temperature_c": 20.0, "temperature_f":
52.0, "sample_time": 1564412307969368, "humidity": 74.3} mutated illegally, new
value was {"device_id": "rpi-rpi0-thermostat", "temperature_c": 20.0,
"temperature_f": 52.0, "sample_time": 1564412360458615, "humidity": 74.7}.
Encoding was
AiZycGktcnBpMC10aGVybW9zdGF0AgAAAAAAADRAAgAAAAAAAEpAArDVsP7jtMcFAjMzMzMzk1JA,
now
AiZycGktcnBpMC10aGVybW9zdGF0AgAAAAAAADRAAgAAAAAAAEpAAu6FuLDktMcFAs3MzMzMrFJA.
at
org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.illegalMutation
(MutationDetectors.java:153)
at
org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodifiedThrowingCheckedExceptions
(MutationDetectors.java:148)
at
org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.verifyUnmodified
(MutationDetectors.java:123)
at
org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.commit
(ImmutabilityCheckingBundleFactory.java:124)
at org.apache.beam.runners.direct.EvaluationContext.commitBundles
(EvaluationContext.java:210)
at org.apache.beam.runners.direct.EvaluationContext.handleResult
(EvaluationContext.java:151)
at
org.apache.beam.runners.direct.QuiescenceDriver$TimerIterableCompletionCallback.handleResult
(QuiescenceDriver.java:262)
at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle
(DirectTransformExecutor.java:189)
at org.apache.beam.runners.direct.DirectTransformExecutor.run
(DirectTransformExecutor.java:126)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
