[
https://issues.apache.org/jira/browse/FLINK-8172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272358#comment-16272358
]
ASF GitHub Bot commented on FLINK-8172:
---------------------------------------
GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/5104
[FLINK-8172][network] Write directly to memorySegment in RecordSerializer
This increases throughput of network stack by factor of 2, because
previously method getMemorySegment() was called twice per record and it is a
synchronized method on recycleLock, while RecordSerializer is sole owner of the
Buffer at this point, so synchronisation is not needed.
## Verifying this change
This change is already covered by existing tests in `flink-runtime`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (**YES** /
no / don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (**not applicable** / docs /
JavaDocs / not documented)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink f8172
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5104.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5104
----
commit d807b9246a2f68e0ea8991047954396adc4703c2
Author: Piotr Nowojski <[email protected]>
Date: 2017-11-28T15:49:37Z
[FLINK-8172][network] Write to memorySegment directly in RecordSerializer
This increases throughput of network stack by factor of 2, because
previously
method getMemorySegment() was called twice per record and it is a
synchronized
method on recycleLock, while RecordSerializer is sole owner of the Buffer at
this point, so synchronisation is not needed.
commit e88506d15885b0d7e60fdc9f930725e83e959fbb
Author: Piotr Nowojski <[email protected]>
Date: 2017-11-29T15:33:06Z
[hotfix][network] Drop redundant this reference usages
----
> Remove unnecessary synchronisation in RecordSerializer
> ------------------------------------------------------
>
> Key: FLINK-8172
> URL: https://issues.apache.org/jira/browse/FLINK-8172
> Project: Flink
> Issue Type: Improvement
> Components: Network
> Affects Versions: 1.4.0, 1.3.2
> Reporter: Piotr Nowojski
> Assignee: Piotr Nowojski
> Fix For: 1.5.0
>
>
> While writing the records, RecordSerializer is the only owner of the `Buffer`
> into which data are written. Yet we are synchronisation twice per record
> while accessing MemorySegment. Removing this synchronisation speeds up the
> Network throughput in point to point benchmark by a factor of two (from
> ~12500records/ms up to 23000 records/ms).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
