[jira] [Comment Edited] (FLINK-8322) support getting number of existing timers in TimerService
[
https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304285#comment-16304285
]
Bowen Li edited comment on FLINK-8322 at 12/27/17 7:24 AM:
---
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {{O[n]}}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
Well, we can do a tradeoff between storage and time, and use a map
{{Map}} to keep track of the number of timers for a given
namespace. [~StephanEwen] [~aljoscha] What do you think?
was (Author: phoenixjiangnan):
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {{O[n]}}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
> support getting number of existing timers in TimerService
> -
>
> Key: FLINK-8322
> URL: https://issues.apache.org/jira/browse/FLINK-8322
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.4.0
>Reporter: Bowen Li
>Assignee: Bowen Li
>
> We have the use cases where we want to use timers as scheduled threads - e.g.
> add a timer to wake up x hours later and do something (reap old data usualy)
> only if there's no existing timers, basically we only want at most 1 timer
> exists for the key all the time
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (FLINK-8322) support getting number of existing timers in TimerService
[ https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-8322: Fix Version/s: (was: 1.5.0) > support getting number of existing timers in TimerService > - > > Key: FLINK-8322 > URL: https://issues.apache.org/jira/browse/FLINK-8322 > Project: Flink > Issue Type: Improvement > Components: DataStream API >Affects Versions: 1.4.0 >Reporter: Bowen Li >Assignee: Bowen Li > > We have the use cases where we want to use timers as scheduled threads - e.g. > add a timer to wake up x hours later and do something (reap old data usualy) > only if there's no existing timers, basically we only want at most 1 timer > exists for the key all the time -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (FLINK-8322) support getting number of existing timers in TimerService
[
https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304285#comment-16304285
]
Bowen Li edited comment on FLINK-8322 at 12/27/17 7:17 AM:
---
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {{O[n]}}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
was (Author: phoenixjiangnan):
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {{O(n)}}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
> support getting number of existing timers in TimerService
> -
>
> Key: FLINK-8322
> URL: https://issues.apache.org/jira/browse/FLINK-8322
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.4.0
>Reporter: Bowen Li
>Assignee: Bowen Li
>
> We have the use cases where we want to use timers as scheduled threads - e.g.
> add a timer to wake up x hours later and do something (reap old data usualy)
> only if there's no existing timers, basically we only want at most 1 timer
> exists for the key all the time
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8322) support getting number of existing timers in TimerService
[
https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304285#comment-16304285
]
Bowen Li commented on FLINK-8322:
-
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {O(n)}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
> support getting number of existing timers in TimerService
> -
>
> Key: FLINK-8322
> URL: https://issues.apache.org/jira/browse/FLINK-8322
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.4.0
>Reporter: Bowen Li
>Assignee: Bowen Li
> Fix For: 1.5.0
>
>
> We have the use cases where we want to use timers as scheduled threads - e.g.
> add a timer to wake up x hours later and do something (reap old data usualy)
> only if there's no existing timers, basically we only want at most 1 timer
> exists for the key all the time
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (FLINK-8322) support getting number of existing timers in TimerService
[
https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304285#comment-16304285
]
Bowen Li edited comment on FLINK-8322 at 12/27/17 7:16 AM:
---
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {{O(n)}}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
was (Author: phoenixjiangnan):
This might be a followup of [this email
thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Why-TimerService-interface-in-ProcessFunction-doesn-t-have-deleteEventTimeTimer-td12719.html]
According to my research, this can be done by exposing
{{numProcessingTimeTimers(N namespace)}} and {{numEventTimeTimers(N
namespace)}}. But their performance is both {O(n)}, which seems to be a bit
inefficient and thus falls to the same situation with {{delete()}} that we may
not want to expose to users.
[~StephanEwen] [~aljoscha] Any suggestions?
> support getting number of existing timers in TimerService
> -
>
> Key: FLINK-8322
> URL: https://issues.apache.org/jira/browse/FLINK-8322
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
>Affects Versions: 1.4.0
>Reporter: Bowen Li
>Assignee: Bowen Li
> Fix For: 1.5.0
>
>
> We have the use cases where we want to use timers as scheduled threads - e.g.
> add a timer to wake up x hours later and do something (reap old data usualy)
> only if there's no existing timers, basically we only want at most 1 timer
> exists for the key all the time
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (FLINK-8322) support getting number of existing timers in TimerService
[ https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-8322: Description: We have the use cases where we want to use timers as scheduled threads - e.g. add a timer to wake up x hours later and do something (reap old data usualy) only if there's no existing timers, basically we only want at most 1 timer exists for the key all the time (was: We have the use cases where we want to use timers as scheduled threads - e.g. add a timer to wake up x hours later and do something (reap old data usualy) only if there's no existing timers ) > support getting number of existing timers in TimerService > - > > Key: FLINK-8322 > URL: https://issues.apache.org/jira/browse/FLINK-8322 > Project: Flink > Issue Type: Improvement > Components: DataStream API >Affects Versions: 1.4.0 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.5.0 > > > We have the use cases where we want to use timers as scheduled threads - e.g. > add a timer to wake up x hours later and do something (reap old data usualy) > only if there's no existing timers, basically we only want at most 1 timer > exists for the key all the time -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-8322) support getting existing timers and number of existing timers in TimerService
[ https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-8322: Description: We have the use cases where we want to use timers as scheduled threads - e.g. add a timer to wake up x hours later and do something (reap old data usualy) only if there's no existing timers > support getting existing timers and number of existing timers in TimerService > - > > Key: FLINK-8322 > URL: https://issues.apache.org/jira/browse/FLINK-8322 > Project: Flink > Issue Type: Improvement > Components: DataStream API >Affects Versions: 1.4.0 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.5.0 > > > We have the use cases where we want to use timers as scheduled threads - e.g. > add a timer to wake up x hours later and do something (reap old data usualy) > only if there's no existing timers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-8322) support getting number of existing timers in TimerService
[ https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-8322: Summary: support getting number of existing timers in TimerService (was: support getting existing timers and number of existing timers in TimerService) > support getting number of existing timers in TimerService > - > > Key: FLINK-8322 > URL: https://issues.apache.org/jira/browse/FLINK-8322 > Project: Flink > Issue Type: Improvement > Components: DataStream API >Affects Versions: 1.4.0 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.5.0 > > > We have the use cases where we want to use timers as scheduled threads - e.g. > add a timer to wake up x hours later and do something (reap old data usualy) > only if there's no existing timers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-8322) support getting existing timers and number of existing timers in TimerService
[ https://issues.apache.org/jira/browse/FLINK-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-8322: Summary: support getting existing timers and number of existing timers in TimerService (was: support getting existing timers from TimerService) > support getting existing timers and number of existing timers in TimerService > - > > Key: FLINK-8322 > URL: https://issues.apache.org/jira/browse/FLINK-8322 > Project: Flink > Issue Type: Improvement > Components: DataStream API >Affects Versions: 1.4.0 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8322) support getting existing timers from TimerService
Bowen Li created FLINK-8322: --- Summary: support getting existing timers from TimerService Key: FLINK-8322 URL: https://issues.apache.org/jira/browse/FLINK-8322 Project: Flink Issue Type: Improvement Components: DataStream API Affects Versions: 1.4.0 Reporter: Bowen Li Assignee: Bowen Li Fix For: 1.5.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-7475) support update() in ListState
[
https://issues.apache.org/jira/browse/FLINK-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304215#comment-16304215
]
ASF GitHub Bot commented on FLINK-7475:
---
Github user bowenli86 commented on the issue:
https://github.com/apache/flink/pull/4963
@yunfan123 @aljoscha @StefanRRichter feedbacks are appreciated
> support update() in ListState
> -
>
> Key: FLINK-7475
> URL: https://issues.apache.org/jira/browse/FLINK-7475
> Project: Flink
> Issue Type: Improvement
> Components: Core, DataStream API, State Backends, Checkpointing
>Affects Versions: 1.4.0
>Reporter: yf
>Assignee: Bowen Li
> Fix For: 1.5.0
>
>
> If I want to update the list.
> I have to do two steps:
> listState.clear()
> for (Element e : myList) {
> listState.add(e);
> }
> Why not I update the state by:
> listState.update(myList) ?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4963: [FLINK-7475] [core][DataStream API] support update() in L...
Github user bowenli86 commented on the issue: https://github.com/apache/flink/pull/4963 @yunfan123 @aljoscha @StefanRRichter feedbacks are appreciated ---
[jira] [Updated] (FLINK-8321) Generate _SUCCESS (map-reduce style) when folder has been written！
[ https://issues.apache.org/jira/browse/FLINK-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8321: -- Description: There is no success file generation, and the downstream processing can't determine when the directory is finished > Generate _SUCCESS (map-reduce style) when folder has been written！ > -- > > Key: FLINK-8321 > URL: https://issues.apache.org/jira/browse/FLINK-8321 > Project: Flink > Issue Type: Bug > Components: filesystem-connector >Affects Versions: 1.4.0 > Environment: How to write the data that is processed in > each time window to each single HDFS directory correspondingly when using > DataStream API in streaming processing application, and generate _SUCCESS > (map-reduce style) when folder has been written！ >Reporter: yanxiaobin > > There is no success file generation, and the downstream processing can't > determine when the directory is finished -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-4534) Lack of synchronization in BucketingSink#restoreState()
[
https://issues.apache.org/jira/browse/FLINK-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304163#comment-16304163
]
Ted Yu commented on FLINK-4534:
---
lgtm
> Lack of synchronization in BucketingSink#restoreState()
> ---
>
> Key: FLINK-4534
> URL: https://issues.apache.org/jira/browse/FLINK-4534
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
>Reporter: Ted Yu
>Assignee: mingleizhang
>
> Iteration over state.bucketStates is protected by synchronization in other
> methods, except for the following in restoreState():
> {code}
> for (BucketState bucketState : state.bucketStates.values()) {
> {code}
> and following in close():
> {code}
> for (Map.Entry entry :
> state.bucketStates.entrySet()) {
> closeCurrentPartFile(entry.getValue());
> {code}
> w.r.t. bucketState.pendingFilesPerCheckpoint , there is similar issue
> starting line 752:
> {code}
> Set pastCheckpointIds =
> bucketState.pendingFilesPerCheckpoint.keySet();
> LOG.debug("Moving pending files to final location on restore.");
> for (Long pastCheckpointId : pastCheckpointIds) {
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8278) Scala examples in Metric documentation do not compile
[
https://issues.apache.org/jira/browse/FLINK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304139#comment-16304139
]
ASF GitHub Bot commented on FLINK-8278:
---
Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5177
Thanks for looking into this @fhueske. I'll close this PR.
> Scala examples in Metric documentation do not compile
> -
>
> Key: FLINK-8278
> URL: https://issues.apache.org/jira/browse/FLINK-8278
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0, 1.3.2, 1.5.0
>Reporter: Fabian Hueske
>Assignee: Xingcan Cui
> Fix For: 1.5.0, 1.4.1
>
>
> The Scala examples in the [Metrics
> documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html]
> do not compile.
> The line
> {code}
> @transient private var counter: Counter
> {code}
> needs to be extended to
> {code}
> @transient private var counter: Counter = _
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #5177: [FLINK-8278] [doc] Fix the private member init pro...
Github user xccui closed the pull request at: https://github.com/apache/flink/pull/5177 ---
[jira] [Commented] (FLINK-8278) Scala examples in Metric documentation do not compile
[
https://issues.apache.org/jira/browse/FLINK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304140#comment-16304140
]
ASF GitHub Bot commented on FLINK-8278:
---
Github user xccui closed the pull request at:
https://github.com/apache/flink/pull/5177
> Scala examples in Metric documentation do not compile
> -
>
> Key: FLINK-8278
> URL: https://issues.apache.org/jira/browse/FLINK-8278
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0, 1.3.2, 1.5.0
>Reporter: Fabian Hueske
>Assignee: Xingcan Cui
> Fix For: 1.5.0, 1.4.1
>
>
> The Scala examples in the [Metrics
> documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html]
> do not compile.
> The line
> {code}
> @transient private var counter: Counter
> {code}
> needs to be extended to
> {code}
> @transient private var counter: Counter = _
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #5177: [FLINK-8278] [doc] Fix the private member init problem fo...
Github user xccui commented on the issue: https://github.com/apache/flink/pull/5177 Thanks for looking into this @fhueske. I'll close this PR. ---
[jira] [Updated] (FLINK-7775) Remove unreferenced method PermanentBlobCache#getNumberOfCachedJobs
[
https://issues.apache.org/jira/browse/FLINK-7775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated FLINK-7775:
--
Description:
{code}
public int getNumberOfCachedJobs() {
return jobRefCounters.size();
}
{code}
The method of PermanentBlobCache is not used.
We should remove it.
was:
{code}
public int getNumberOfCachedJobs() {
return jobRefCounters.size();
}
{code}
The method of PermanentBlobCache is not used.
We should remove it.
> Remove unreferenced method PermanentBlobCache#getNumberOfCachedJobs
> ---
>
> Key: FLINK-7775
> URL: https://issues.apache.org/jira/browse/FLINK-7775
> Project: Flink
> Issue Type: Task
> Components: Local Runtime
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> public int getNumberOfCachedJobs() {
> return jobRefCounters.size();
> }
> {code}
> The method of PermanentBlobCache is not used.
> We should remove it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (FLINK-8146) Potential resource leak in PythonPlanBinder#unzipPythonLibrary
[
https://issues.apache.org/jira/browse/FLINK-8146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated FLINK-8146:
--
Description:
{code}
while (entry != null) {
...
}
zis.closeEntry();
{code}
Looking at the catch block inside the loop, it seems the intention is to close
zis upon getting exception.
zis.close() should be called outside the loop.
was:
{code}
while (entry != null) {
...
}
zis.closeEntry();
{code}
Looking at the catch block inside the loop, it seems the intention is to close
zis upon getting exception.
zis.close() should be called outside the loop.
> Potential resource leak in PythonPlanBinder#unzipPythonLibrary
> --
>
> Key: FLINK-8146
> URL: https://issues.apache.org/jira/browse/FLINK-8146
> Project: Flink
> Issue Type: Bug
> Components: Python API
>Reporter: Ted Yu
>
> {code}
> while (entry != null) {
> ...
> }
> zis.closeEntry();
> {code}
> Looking at the catch block inside the loop, it seems the intention is to
> close zis upon getting exception.
> zis.close() should be called outside the loop.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (FLINK-7917) The return of taskInformationOrBlobKey should be placed inside synchronized in ExecutionJobVertex
[
https://issues.apache.org/jira/browse/FLINK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated FLINK-7917:
--
Description:
Currently in ExecutionJobVertex#getTaskInformationOrBlobKey:
{code}
}
return taskInformationOrBlobKey;
{code}
The return should be placed inside synchronized block.
was:
Currently in ExecutionJobVertex#getTaskInformationOrBlobKey:
{code}
}
return taskInformationOrBlobKey;
{code}
The return should be placed inside synchronized block.
> The return of taskInformationOrBlobKey should be placed inside synchronized
> in ExecutionJobVertex
> -
>
> Key: FLINK-7917
> URL: https://issues.apache.org/jira/browse/FLINK-7917
> Project: Flink
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> Currently in ExecutionJobVertex#getTaskInformationOrBlobKey:
> {code}
> }
> return taskInformationOrBlobKey;
> {code}
> The return should be placed inside synchronized block.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8278) Scala examples in Metric documentation do not compile
[
https://issues.apache.org/jira/browse/FLINK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304000#comment-16304000
]
ASF GitHub Bot commented on FLINK-8278:
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5177
Hi @xccui, I forgot to close the PR while merging. Can you please close the
PR?
Thanks, Fabian
> Scala examples in Metric documentation do not compile
> -
>
> Key: FLINK-8278
> URL: https://issues.apache.org/jira/browse/FLINK-8278
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0, 1.3.2, 1.5.0
>Reporter: Fabian Hueske
>Assignee: Xingcan Cui
> Fix For: 1.5.0, 1.4.1
>
>
> The Scala examples in the [Metrics
> documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html]
> do not compile.
> The line
> {code}
> @transient private var counter: Counter
> {code}
> needs to be extended to
> {code}
> @transient private var counter: Counter = _
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #5177: [FLINK-8278] [doc] Fix the private member init problem fo...
Github user fhueske commented on the issue: https://github.com/apache/flink/pull/5177 Hi @xccui, I forgot to close the PR while merging. Can you please close the PR? Thanks, Fabian ---
[jira] [Closed] (FLINK-8278) Scala examples in Metric documentation do not compile
[
https://issues.apache.org/jira/browse/FLINK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske closed FLINK-8278.
Resolution: Fixed
Fix Version/s: 1.4.1
1.5.0
Fixed for 1.4.1 with e42182d40bbf0b0ce6721f91ed28d348ed36847d
Fixed for 1.5.0 with d74869f8a4642fec0064ae0d0a2911aa2dea3ce4
> Scala examples in Metric documentation do not compile
> -
>
> Key: FLINK-8278
> URL: https://issues.apache.org/jira/browse/FLINK-8278
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0, 1.3.2, 1.5.0
>Reporter: Fabian Hueske
>Assignee: Xingcan Cui
> Fix For: 1.5.0, 1.4.1
>
>
> The Scala examples in the [Metrics
> documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html]
> do not compile.
> The line
> {code}
> @transient private var counter: Counter
> {code}
> needs to be extended to
> {code}
> @transient private var counter: Counter = _
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-8278) Scala examples in Metric documentation do not compile
[
https://issues.apache.org/jira/browse/FLINK-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16303988#comment-16303988
]
ASF GitHub Bot commented on FLINK-8278:
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5177
thanks for the fix @xccui.
Merging
> Scala examples in Metric documentation do not compile
> -
>
> Key: FLINK-8278
> URL: https://issues.apache.org/jira/browse/FLINK-8278
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0, 1.3.2, 1.5.0
>Reporter: Fabian Hueske
>Assignee: Xingcan Cui
>
> The Scala examples in the [Metrics
> documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html]
> do not compile.
> The line
> {code}
> @transient private var counter: Counter
> {code}
> needs to be extended to
> {code}
> @transient private var counter: Counter = _
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #5177: [FLINK-8278] [doc] Fix the private member init problem fo...
Github user fhueske commented on the issue: https://github.com/apache/flink/pull/5177 thanks for the fix @xccui. Merging ---
[jira] [Created] (FLINK-8321) Generate _SUCCESS (map-reduce style) when folder has been written！
yanxiaobin created FLINK-8321: - Summary: Generate _SUCCESS (map-reduce style) when folder has been written！ Key: FLINK-8321 URL: https://issues.apache.org/jira/browse/FLINK-8321 Project: Flink Issue Type: Bug Components: filesystem-connector Affects Versions: 1.4.0 Environment: How to write the data that is processed in each time window to each single HDFS directory correspondingly when using DataStream API in streaming processing application, and generate _SUCCESS (map-reduce style) when folder has been written！ Reporter: yanxiaobin -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (FLINK-8301) Support Unicode in codegen for SQL && TableAPI
[ https://issues.apache.org/jira/browse/FLINK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sunjincheng closed FLINK-8301. -- Resolution: Fixed 1.4.0 Fixed in 47c067a5924e56abeb5a3d0769b6356f19a66b1e 1.5.0 Fixed in edf10c714fb1fe7a58e96a4c7006d0df48954b79 > Support Unicode in codegen for SQL && TableAPI > -- > > Key: FLINK-8301 > URL: https://issues.apache.org/jira/browse/FLINK-8301 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Affects Versions: 1.4.0, 1.5.0 >Reporter: Ruidong Li >Assignee: Ruidong Li > > The current code generation do not support Unicode, "\u0001" will be > generated to "\\u0001", function call like concat(str, "\u0001") will lead to > wrong result. > This issue intend to handle char/varchar literal correctly, some examples > followed as below. > literal: '\u0001abc'-> codegen: "\u0001abc" > literal: '\u0022\' -> codegen: "\"\\" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8320) Flink cluster does not work on Java 9
Steve Layland created FLINK-8320:
Summary: Flink cluster does not work on Java 9
Key: FLINK-8320
URL: https://issues.apache.org/jira/browse/FLINK-8320
Project: Flink
Issue Type: Bug
Affects Versions: 1.4.0
Environment: flink-1.4.0, mac os x, 10.13.1
Reporter: Steve Layland
Recently got a new macbook and figured it was a good time to install java 9 and
try it out. I didn't realize that Java 9 was such a breaking update (eg:
https://blog.codefx.org/java/java-9-migration-guide/) and took the Flink
documentation at face value and assumed that Java 7+ or higher would be fine.
Here's is what happens after starting a local cluster and attempting to run the
sample WordCount program under Java 9:
{noformat}
flink-1.4.0 $ export JAVA_HOME=$(/usr/libexec/java_home -v 9)
cru@lappy:flink-1.4.0 $ java -version
java version "9.0.1"
Java(TM) SE Runtime Environment (build 9.0.1+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode)
cru@lappy:flink-1.4.0 $ bin/start-cluster.sh
Starting cluster.
Starting jobmanager daemon on host lappy.local.
Starting taskmanager daemon on host lappy.local.
cru@lappy:flink-1.4.0 $ bin/flink run examples/streaming/WordCount.jar
Cluster configuration: Standalone cluster with JobManager at
localhost/127.0.0.1:6123
Using address localhost:6123 to connect to JobManager.
JobManager web interface address http://localhost:8081
Starting execution of program
Executing WordCount example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
Submitting job with JobID: ee054ffeb4784848143b76b7d51d99c1. Waiting for job
completion.
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The program
execution failed: Couldn't retrieve the JobExecutionResult from the JobManager.
at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:492)
at
org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)
at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:456)
at
org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:66)
at
org.apache.flink.streaming.examples.wordcount.WordCount.main(WordCount.java:89)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:525)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:417)
at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:396)
at
org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:802)
at org.apache.flink.client.CliFrontend.run(CliFrontend.java:282)
at
org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1054)
at org.apache.flink.client.CliFrontend$1.call(CliFrontend.java:1101)
at org.apache.flink.client.CliFrontend$1.call(CliFrontend.java:1098)
at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1098)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Couldn't
retrieve the JobExecutionResult from the JobManager.
at
org.apache.flink.runtime.client.JobClient.awaitJobResult(JobClient.java:300)
at
org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:387)
at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:481)
... 18 more
Caused by:
org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost
connection to the JobManager.
at
org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:219)
at
org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:104)
at
org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:71)
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at
[jira] [Created] (FLINK-8319) Savepoint restore failing in CEP
Shashank Agarwal created FLINK-8319:
---
Summary: Savepoint restore failing in CEP
Key: FLINK-8319
URL: https://issues.apache.org/jira/browse/FLINK-8319
Project: Flink
Issue Type: Bug
Components: CEP, State Backends, Checkpointing, YARN
Affects Versions: 1.4.0
Environment: Yarn Cluster
Reporter: Shashank Agarwal
Fix For: 1.5.0, 1.4.1
I have reported some bugs before also in 1.3.2 but this time error is different
while restoring savepoint or checkpoint.
https://issues.apache.org/jira/browse/FLINK-7760
{code}
java.lang.IllegalStateException: Could not initialize keyed state backend.
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:293)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:225)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.InvalidClassException:
org.apache.flink.cep.scala.pattern.Pattern$$anon$3; invalid descriptor for
field
at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:723)
at
java.io.ObjectInputStream.readClassDescriptor(ObjectInputStream.java:833)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1609)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at
org.apache.flink.cep.nfa.NFA$NFASerializer.deserializeCondition(NFA.java:1171)
at
org.apache.flink.cep.nfa.NFA$NFASerializer.deserializeStates(NFA.java:1129)
at org.apache.flink.cep.nfa.NFA$NFASerializer.deserialize(NFA.java:917)
at org.apache.flink.cep.nfa.NFA$NFASerializer.deserialize(NFA.java:820)
at
org.apache.flink.runtime.state.heap.StateTableByKeyGroupReaders$StateTableByKeyGroupReaderV2V3.readMappingsInKeyGroup(StateTableByKeyGroupReaders.java:133)
at
org.apache.flink.runtime.state.heap.HeapKeyedStateBackend.restorePartitionedState(HeapKeyedStateBackend.java:575)
at
org.apache.flink.runtime.state.heap.HeapKeyedStateBackend.restore(HeapKeyedStateBackend.java:446)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.createKeyedStateBackend(StreamTask.java:773)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:283)
... 6 more
Caused by: java.lang.IllegalArgumentException: illegal signature
at java.io.ObjectStreamField.(ObjectStreamField.java:122)
at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:721)
... 21 more
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (FLINK-8301) Support Unicode in codegen for SQL && TableAPI
[ https://issues.apache.org/jira/browse/FLINK-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sunjincheng updated FLINK-8301: --- Affects Version/s: 1.5.0 1.4.0 > Support Unicode in codegen for SQL && TableAPI > -- > > Key: FLINK-8301 > URL: https://issues.apache.org/jira/browse/FLINK-8301 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL >Affects Versions: 1.4.0, 1.5.0 >Reporter: Ruidong Li >Assignee: Ruidong Li > > The current code generation do not support Unicode, "\u0001" will be > generated to "\\u0001", function call like concat(str, "\u0001") will lead to > wrong result. > This issue intend to handle char/varchar literal correctly, some examples > followed as below. > literal: '\u0001abc'-> codegen: "\u0001abc" > literal: '\u0022\' -> codegen: "\"\\" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
