[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17336797#comment-17336797
]
Flink Jira Bot commented on FLINK-5891:
---
This issue was labeled "stale-major" 7 ago and has not received any updates so
it is being deprioritized. If this ticket is actually Major, please raise the
priority and ask a committer to assign you the issue or revive the public
discussion.
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Library / Graph Processing (Gelly)
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>Priority: Major
> Labels: stale-major
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater>
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17328824#comment-17328824
]
Flink Jira Bot commented on FLINK-5891:
---
This major issue is unassigned and itself and all of its Sub-Tasks have not
been updated for 30 days. So, it has been labeled "stale-major". If this ticket
is indeed "major", please either assign yourself or give an update. Afterwards,
please remove the label. In 7 days the issue will be deprioritized.
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Library / Graph Processing (Gelly)
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>Priority: Major
> Labels: stale-major
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater>
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883142#comment-15883142
]
Greg Hogan commented on FLINK-5891:
---
The trouble with object reuse is that it is a global flag which is simplest to
leave unset when composing code or using library methods. Better would be to
mark object reuse enabled or disabled with annotations on each user defined
{{Function}}.
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Gelly
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882017#comment-15882017
]
Xingcan Cui commented on FLINK-5891:
Thanks for your explanation, [~greghogan]. I'm afraid my PR on
https://issues.apache.org/jira/browse/FLINK-1526 gets the same problem as I
also store values with non-primitive types (anyhow the primitive types will not
be affected, right?) from the received messages. I saw the following code in
Flink's ML lib. To avoid the reference problem, it makes a deep copy of each
{{StreamRecord element}}.
{code:title=AbstractCEPBasePatternOperator.java | borderStyle=solid}
...
// we have to buffer the elements until we receive the proper watermark
if (getExecutionConfig().isObjectReuseEnabled()) {
// copy the StreamRecord so that it cannot be changed
priorityQueue.offer(new
StreamRecord(inputSerializer.copy(element.getValue()),
element.getTimestamp()));
} else {
priorityQueue.offer(element);
}
updatePriorityQueue(priorityQueue);
...
{code}
So, what's your suggestions on fixing this? I'd like to work on it (and surely
also the PR of Flink-1526).
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Gelly
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880459#comment-15880459
]
Greg Hogan commented on FLINK-5891:
---
[~xccui] even when wrapping in another object both references will point to the
same object, and Flink will eventually overwrite the value with a later
deserialization. Elsewhere we have made use of {{CopyableValue}} but this
restricts the permitted types. One can also leave object reuse disabled or use
immutable types (effectively disabling object reuse).
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Gelly
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5891) ConnectedComponents is broken when object reuse enabled
[
https://issues.apache.org/jira/browse/FLINK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879742#comment-15879742
]
Xingcan Cui commented on FLINK-5891:
Hi Greg, may I ask what does "when object reuse enabled" mean here? Is it
necessary to make a copy of the attribute min before storing it?
> ConnectedComponents is broken when object reuse enabled
> ---
>
> Key: FLINK-5891
> URL: https://issues.apache.org/jira/browse/FLINK-5891
> Project: Flink
> Issue Type: Bug
> Components: Gelly
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>
> {{org.apache.flink.graph.library.ConnectedComponents.CCUpdater#updateVertex}}
> is storing a value from its iterator.
> {{GSAConnectedComponents}} does not have this limitation.
> {code}
> public static final class CCUpdater
> extends GatherFunction {
> @Override
> public void updateVertex(Vertex vertex,
> MessageIterator messages) throws Exception {
> VV current = vertex.getValue();
> VV min = current;
> for (VV msg : messages) {
> if (msg.compareTo(min) < 0) {
> min = msg;
> }
> }
> if (!min.equals(current)) {
> setNewVertexValue(min);
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
