GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/5134
[FLINK-8220] Implement set of network benchmarks
This PR implements sets of network benchmarks that are intended to simulate
various workloads for network stack. Benchmarks will be executed by
`flink-benchmarks` project. We want to keep the code of the benchmarks here,
since components they are using are not part of public api and we don't wont
them braking unknowingly on changes in the flink code.
## Verifying this change
This change adds two tests ensuring that benchmarks are compiling and
executing as expected without an exceptions.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes /
**no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (**not applicable** / docs /
JavaDocs / not documented)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink network-benchmarks
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5134.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5134
----
commit 5151d74980fd5995c718173ea03db5049ed0ada2
Author: Piotr Nowojski <[email protected]>
Date: 2017-11-28T15:49:37Z
[FLINK-8172][network] Write to memorySegment directly in RecordSerializer
This increases throughput of network stack by factor of 2, because
previously
method getMemorySegment() was called twice per record and it is a
synchronized
method on recycleLock, while RecordSerializer is sole owner of the Buffer at
this point, so synchronisation is not needed.
commit 42a6a0daefdb9e9b1185a8195c28eccf055a83ce
Author: Piotr Nowojski <[email protected]>
Date: 2017-11-28T07:41:40Z
[hotfix][test] Add timeout for joining with CheckedThread
commit b777e4f44344f12d0196c77515aac0815e7f31c5
Author: Piotr Nowojski <[email protected]>
Date: 2017-11-28T07:42:19Z
[hotfix][util] Added suppressExceptions for lambda functions
commit 28e25a2bd0af704af1fa1b084be6b0cfa05d9928
Author: Piotr Nowojski <[email protected]>
Date: 2017-12-07T09:03:32Z
[FLINK-XXX][network-benchmarks] Define network benchmarks in Flink project
commit 4f88c6522e176bc20ae53c67136e53dc7970c9e0
Author: Nico Kruber <[email protected]>
Date: 2017-12-07T09:03:49Z
[FLINK-XXX][network-benchmarks] Define latency network benchmarks in Flink
project
----
---
