[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-987: - Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-987: - Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-987: - Priority: Minor (was: Major) > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Priority: Minor > Labels: auto-deprioritized-major > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-987: - Labels: auto-deprioritized-major (was: stale-major) > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Priority: Major > Labels: auto-deprioritized-major > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-987: - Labels: stale-major (was: ) > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Runtime / Task >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Priority: Major > Labels: stale-major > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske updated FLINK-987: Fix Version/s: (was: 0.10.0) 1.0.0 > Extend TypeSerializers and -Comparators to work directly on Memory Segments > --- > > Key: FLINK-987 > URL: https://issues.apache.org/jira/browse/FLINK-987 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: 0.6-incubating >Reporter: Stephan Ewen >Assignee: Aljoscha Krettek > Fix For: 1.0.0 > > > As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to > change the way that the TypeSerialzers/Comparators and > DataInputViews/DataOutputViews work. > The goal is to allow more flexibility in the construction on the binary > representation of data types, and to allow partial deserialization of > individual fields. Both is currently prohibited by the fact that the > abstraction of the memory (into which the data goes) is a stream abstraction > ({{DataInputView}}, {{DataOutputView}}). > An idea is to offer a random-access buffer like view for construction and > random-access deserialization, as well as various methods to copy elements in > a binary fashion between such buffers and streams. > A possible set of methods for the {{TypeSerializer}} could be: > {code} > long serialize(T record, TargetBuffer buffer); > > T deserialize(T reuse, SourceBuffer source); > > void ensureBufferSufficientlyFilled(SourceBuffer source); > > X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); > > int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); > > void copy(DataInputView in, TargetBuffer buffer); > > void copy(SourceBuffer buffer,, DataOutputView out); > > void copy(DataInputView source, DataOutputView target); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
[ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ufuk Celebi updated FLINK-987: -- Fix Version/s: (was: 0.9) 0.10 Extend TypeSerializers and -Comparators to work directly on Memory Segments --- Key: FLINK-987 URL: https://issues.apache.org/jira/browse/FLINK-987 Project: Flink Issue Type: Improvement Components: Local Runtime Affects Versions: 0.6-incubating Reporter: Stephan Ewen Assignee: Aljoscha Krettek Fix For: 0.10 As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to change the way that the TypeSerialzers/Comparators and DataInputViews/DataOutputViews work. The goal is to allow more flexibility in the construction on the binary representation of data types, and to allow partial deserialization of individual fields. Both is currently prohibited by the fact that the abstraction of the memory (into which the data goes) is a stream abstraction ({{DataInputView}}, {{DataOutputView}}). An idea is to offer a random-access buffer like view for construction and random-access deserialization, as well as various methods to copy elements in a binary fashion between such buffers and streams. A possible set of methods for the {{TypeSerializer}} could be: {code} long serialize(T record, TargetBuffer buffer); T deserialize(T reuse, SourceBuffer source); void ensureBufferSufficientlyFilled(SourceBuffer source); X X deserializeField(X reuse, int logicalPos, SourceBuffer buffer); int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer); void copy(DataInputView in, TargetBuffer buffer); void copy(SourceBuffer buffer,, DataOutputView out); void copy(DataInputView source, DataOutputView target); {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)