[
https://issues.apache.org/jira/browse/BEAM-7568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz reassigned BEAM-7568:
---
Assignee: Steve Niemitz
> Java dataflow harness re-encodes value state cells even if they
[
https://issues.apache.org/jira/browse/BEAM-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116879#comment-17116879
]
Steve Niemitz commented on BEAM-9383:
-
{quote}Steve, why would two workers jars be staged at the same
[
https://issues.apache.org/jira/browse/BEAM-10076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz updated BEAM-10076:
-
Description:
The work item status page incorrectly renders its table due to an incorrectly
Steve Niemitz created BEAM-10076:
Summary: Dataflow worker status page incorrectly displays work
item statuses
Key: BEAM-10076
URL: https://issues.apache.org/jira/browse/BEAM-10076
Project: Beam
[
https://issues.apache.org/jira/browse/BEAM-10076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz updated BEAM-10076:
-
Description:
The work item status page incorrectly renders its table due to an incorrectly
[
https://issues.apache.org/jira/browse/BEAM-10076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz updated BEAM-10076:
-
Attachment: image-2020-05-25-17-13-49-465.png
> Dataflow worker status page incorrectly displays
[
https://issues.apache.org/jira/browse/BEAM-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116185#comment-17116185
]
Steve Niemitz commented on BEAM-9383:
-
There's actually another much more serious bug here. If I set
[
https://issues.apache.org/jira/browse/BEAM-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116181#comment-17116181
]
Steve Niemitz commented on BEAM-9383:
-
I was just validating the 2.22 branch and this seems to have
Steve Niemitz created BEAM-10075:
Summary: Allow users to tune the grouping table size in batch
dataflow pipelines
Key: BEAM-10075
URL: https://issues.apache.org/jira/browse/BEAM-10075
Project: Beam
Steve Niemitz created BEAM-9931:
---
Summary: Support custom avro DatumReaders in AvroIO
Key: BEAM-9931
URL: https://issues.apache.org/jira/browse/BEAM-9931
Project: Beam
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/BEAM-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz resolved BEAM-9795.
-
Fix Version/s: 2.22.0
Resolution: Fixed
> Support custom avro DatumWriters when writing to
[
https://issues.apache.org/jira/browse/BEAM-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz updated BEAM-9795:
Status: Open (was: Triage Needed)
> Support custom avro DatumWriters when writing to BigQuery
>
[
https://issues.apache.org/jira/browse/BEAM-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on BEAM-9795 started by Steve Niemitz.
---
> Support custom avro DatumWriters when writing to BigQuery
>
Steve Niemitz created BEAM-9795:
---
Summary: Support custom avro DatumWriters when writing to BigQuery
Key: BEAM-9795
URL: https://issues.apache.org/jira/browse/BEAM-9795
Project: Beam
Issue
[
https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066643#comment-17066643
]
Steve Niemitz commented on BEAM-9557:
-
ah ok, yeah I agree!
> Error setting processing time timers
[
https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066280#comment-17066280
]
Steve Niemitz commented on BEAM-9557:
-
The default case will work as you mentioned, but if the
[
https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065220#comment-17065220
]
Steve Niemitz commented on BEAM-9557:
-
I had actually tried that (for other reasons) in the past. The
[
https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065090#comment-17065090
]
Steve Niemitz commented on BEAM-9557:
-
imo this should be a blocker for 2.20. It breaks most use of
Steve Niemitz created BEAM-9557:
---
Summary: Error setting processing time timers near end-of-window
Key: BEAM-9557
URL: https://issues.apache.org/jira/browse/BEAM-9557
Project: Beam
Issue Type:
Steve Niemitz created BEAM-9308:
---
Summary: Optimize state cleanup at end-of-window
Key: BEAM-9308
URL: https://issues.apache.org/jira/browse/BEAM-9308
Project: Beam
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/BEAM-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz resolved BEAM-2879.
-
Fix Version/s: 2.17.0
Resolution: Fixed
> Implement and use an Avro coder rather than the
[
https://issues.apache.org/jira/browse/BEAM-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027016#comment-17027016
]
Steve Niemitz commented on BEAM-2879:
-
oh, yeah, I think this was actually in 2.17
> Implement and
[
https://issues.apache.org/jira/browse/BEAM-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014335#comment-17014335
]
Steve Niemitz commented on BEAM-1589:
-
this is really cool! Both of these features are going to be
Steve Niemitz created BEAM-8809:
---
Summary: AvroWriteRequest should have a public constructor
Key: BEAM-8809
URL: https://issues.apache.org/jira/browse/BEAM-8809
Project: Beam
Issue Type:
[
https://issues.apache.org/jira/browse/BEAM-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz reassigned BEAM-2879:
---
Assignee: Steve Niemitz
> Implement and use an Avro coder rather than the JSON one for
[
https://issues.apache.org/jira/browse/BEAM-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900133#comment-16900133
]
Steve Niemitz commented on BEAM-2680:
-
Sharding would delay the inevitable more, but the issue of an
[
https://issues.apache.org/jira/browse/BEAM-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899655#comment-16899655
]
Steve Niemitz commented on BEAM-2680:
-
I ran into this today. We have a long-running streaming
Steve Niemitz created BEAM-7745:
---
Summary: StreamingSideInputDoFnRunner/StreamingSideInputFetcher
have suboptimal state access pattern during normal operation
Key: BEAM-7745
URL:
[
https://issues.apache.org/jira/browse/BEAM-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz closed BEAM-7718.
---
Resolution: Done
Fix Version/s: 2.14.0
> PubsubIO to use gRPC API instead of JSON REST API
>
[
https://issues.apache.org/jira/browse/BEAM-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883837#comment-16883837
]
Steve Niemitz commented on BEAM-7718:
-
Looks like there was a change very recently
[
https://issues.apache.org/jira/browse/BEAM-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883084#comment-16883084
]
Steve Niemitz commented on BEAM-7718:
-
Additionally, the gRPC implementation seems efficient enough
[
https://issues.apache.org/jira/browse/BEAM-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883006#comment-16883006
]
Steve Niemitz commented on BEAM-7718:
-
One thing I did notice using the gRPC client is that there are
[
https://issues.apache.org/jira/browse/BEAM-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882998#comment-16882998
]
Steve Niemitz commented on BEAM-7717:
-
[
https://issues.apache.org/jira/browse/BEAM-7716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882993#comment-16882993
]
Steve Niemitz commented on BEAM-7716:
-
Hacking up PubsubMessagePayloadOnlyCoder to always use a nested
[
https://issues.apache.org/jira/browse/BEAM-7716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882640#comment-16882640
]
Steve Niemitz edited comment on BEAM-7716 at 7/11/19 4:14 AM:
--
I did a little
[
https://issues.apache.org/jira/browse/BEAM-7716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882640#comment-16882640
]
Steve Niemitz commented on BEAM-7716:
-
I did a little debugging tonight and tracked this down a
Steve Niemitz created BEAM-7684:
---
Summary: WindmillStateCache grossly misunderestimates object size
in its weighting function
Key: BEAM-7684
URL: https://issues.apache.org/jira/browse/BEAM-7684
[
https://issues.apache.org/jira/browse/BEAM-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876198#comment-16876198
]
Steve Niemitz edited comment on BEAM-7547 at 7/1/19 1:50 PM:
-
Is it possible
[
https://issues.apache.org/jira/browse/BEAM-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876198#comment-16876198
]
Steve Niemitz commented on BEAM-7547:
-
Is it possible this was the cause of BEAM-7614?
>
[
https://issues.apache.org/jira/browse/BEAM-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869637#comment-16869637
]
Steve Niemitz commented on BEAM-7614:
-
oh, another odd thing I should mention. I originally found
Steve Niemitz created BEAM-7614:
---
Summary: Event-time timers seem to sometimes fire multiple times
on dataflow + streaming engine
Key: BEAM-7614
URL: https://issues.apache.org/jira/browse/BEAM-7614
[
https://issues.apache.org/jira/browse/BEAM-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Niemitz updated BEAM-7605:
Summary: Provide a way for user code to read dataflow runner stats (was:
Provide a way for user
Steve Niemitz created BEAM-7605:
---
Summary: Provide a way for user code to read dataflow runner stats
(for streaming at least)
Key: BEAM-7605
URL: https://issues.apache.org/jira/browse/BEAM-7605
Steve Niemitz created BEAM-7568:
---
Summary: Java dataflow harness re-encodes value state cells even
if they haven't changed
Key: BEAM-7568
URL: https://issues.apache.org/jira/browse/BEAM-7568
Project:
[
https://issues.apache.org/jira/browse/BEAM-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863122#comment-16863122
]
Steve Niemitz edited comment on BEAM-1589 at 6/13/19 3:02 PM:
--
has there been
[
https://issues.apache.org/jira/browse/BEAM-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863122#comment-16863122
]
Steve Niemitz commented on BEAM-1589:
-
has there been any progress on implementing this other than
Steve Niemitz created BEAM-7507:
---
Summary: StreamingDataflowWorker attempts to decode non-utf8
binary data as utf8
Key: BEAM-7507
URL: https://issues.apache.org/jira/browse/BEAM-7507
Project: Beam
Steve Niemitz created BEAM-7397:
---
Summary: Avoid String.format in state namespace construction
Key: BEAM-7397
URL: https://issues.apache.org/jira/browse/BEAM-7397
Project: Beam
Issue Type:
[
https://issues.apache.org/jira/browse/BEAM-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845091#comment-16845091
]
Steve Niemitz commented on BEAM-6813:
-
Confirmed! I finally got back to this, added an implementation
[
https://issues.apache.org/jira/browse/BEAM-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805506#comment-16805506
]
Steve Niemitz commented on BEAM-6813:
-
I believe this was using a 4 hour fixed window without any
Steve Niemitz created BEAM-6813:
---
Summary: Issues with state + timers in java Direct Runner
Key: BEAM-6813
URL: https://issues.apache.org/jira/browse/BEAM-6813
Project: Beam
Issue Type: Bug
Steve Niemitz created BEAM-6757:
---
Summary: Random "IllegalStateException: TimestampCombiner moved
element from X to Y" seen in running topology
Key: BEAM-6757
URL: https://issues.apache.org/jira/browse/BEAM-6757
[
https://issues.apache.org/jira/browse/BEAM-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726108#comment-16726108
]
Steve Niemitz commented on BEAM-6249:
-
I can give it a try, but it'll take me a little while to get it
[
https://issues.apache.org/jira/browse/BEAM-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726086#comment-16726086
]
Steve Niemitz commented on BEAM-6249:
-
2018-12-17_08_21_19-13033399877532513432 is an example of one
[
https://issues.apache.org/jira/browse/BEAM-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725569#comment-16725569
]
Steve Niemitz commented on BEAM-5710:
-
Ah amazing thank you! I also had reported this in BEAM-6249 so
[
https://issues.apache.org/jira/browse/BEAM-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723198#comment-16723198
]
Steve Niemitz commented on BEAM-6249:
-
I should mention that I'm building the beam libs myself and not
Steve Niemitz created BEAM-6249:
---
Summary: Vendored gRPC doesn't seem to work with dataflow
Key: BEAM-6249
URL: https://issues.apache.org/jira/browse/BEAM-6249
Project: Beam
Issue Type: Bug
[
https://issues.apache.org/jira/browse/BEAM-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722518#comment-16722518
]
Steve Niemitz commented on BEAM-5710:
-
sorry I haven't pinged this for awhile.
2.8.0 did fix this,
[
https://issues.apache.org/jira/browse/BEAM-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700479#comment-16700479
]
Steve Niemitz edited comment on BEAM-6102 at 11/27/18 2:58 PM:
---
Where did
[
https://issues.apache.org/jira/browse/BEAM-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700479#comment-16700479
]
Steve Niemitz commented on BEAM-6102:
-
Where did you pull that class file from? I was looking at the
[
https://issues.apache.org/jira/browse/BEAM-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650620#comment-16650620
]
Steve Niemitz commented on BEAM-5710:
-
Sure, I can test out on 2.8.0. Currently we're working around
[
https://issues.apache.org/jira/browse/BEAM-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650454#comment-16650454
]
Steve Niemitz commented on BEAM-5710:
-
So, quick update here. From what I can tell, the issue seems
62 matches
Mail list logo
