[jira] [Commented] (FLINK-8734) SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378218#comment-16378218
 ] 

ASF GitHub Bot commented on FLINK-8734:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5550


> SpillableSubpartition does not update byte statistics when spilled by a 
> SpillableSubpartitionView
> -
>
> Key: FLINK-8734
> URL: https://issues.apache.org/jira/browse/FLINK-8734
> Project: Flink
>  Issue Type: Sub-task
>  Components: Network
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
> Fix For: 1.5.0
>
>
> With FLINK-8583, {{SpillableSubpartition}} does not update byte statistics 
> anymore when spilled by a {{SpillableSubpartitionView}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8734) SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373126#comment-16373126
 ] 

ASF GitHub Bot commented on FLINK-8734:
---

Github user NicoK commented on the issue:

https://github.com/apache/flink/pull/5550
  
Since the tests go through various different scenarios, that's the natural 
place to also verify statistics which should be aligned with the real world 
despite the overhead during changes.

The test failure indeed is interesting but unrelated as proposed - I 
created [FLINK-8750](https://issues.apache.org/jira/browse/FLINK-8750) for this.


> SpillableSubpartition does not update byte statistics when spilled by a 
> SpillableSubpartitionView
> -
>
> Key: FLINK-8734
> URL: https://issues.apache.org/jira/browse/FLINK-8734
> Project: Flink
>  Issue Type: Sub-task
>  Components: Network
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
> Fix For: 1.5.0
>
>
> With FLINK-8583, {{SpillableSubpartition}} does not update byte statistics 
> anymore when spilled by a {{SpillableSubpartitionView}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8734) SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371709#comment-16371709
 ] 

ASF GitHub Bot commented on FLINK-8734:
---

GitHub user NicoK opened a pull request:

https://github.com/apache/flink/pull/5550

[FLINK-8734][network] fix partition bytes counting and re-enable in tests

## What is the purpose of the change

With [FLINK-8583], `SpillableSubpartition` does not update byte statistics 
anymore when spilled by a `SpillableSubpartitionView`.

## Brief change log

- let `SpillableSubpartitionView` update its parent's statistics when 
spilling buffers
- re-enable byte counting verifications in unit tests

## Verifying this change

This change re-added checks to the following tests:
- `PipelinedSubpartitionTest`
- `SpillableSubpartitionTest`
- `SubpartitionTestBase`

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): **no**
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: **no**
  - The serializers: **no**
  - The runtime per-record code paths (performance sensitive): **no**
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
  - The S3 file system connector: **no**

## Documentation

  - Does this pull request introduce a new feature? **no**
  - If yes, how is the feature documented? **not applicable**


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NicoK/flink flink-8734

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5550.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5550


commit 0bab9a053d8e1c7810c5f71cdb0c40b3f41a288a
Author: Nico Kruber 
Date:   2018-02-20T17:05:54Z

[FLINK-8734][network] fix partition bytes counting and re-enable in tests




> SpillableSubpartition does not update byte statistics when spilled by a 
> SpillableSubpartitionView
> -
>
> Key: FLINK-8734
> URL: https://issues.apache.org/jira/browse/FLINK-8734
> Project: Flink
>  Issue Type: Sub-task
>  Components: Network
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
> Fix For: 1.5.0
>
>
> With FLINK-8583, {{SpillableSubpartition}} does not update byte statistics 
> anymore when spilled by a {{SpillableSubpartitionView}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)