[jira] [Assigned] (FLINK-8426) Error in Generating Timestamp/Watermakr doc
[
https://issues.apache.org/jira/browse/FLINK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitrii Kniazev reassigned FLINK-8426:
--
Assignee: Dmitrii Kniazev
> Error in Generating Timestamp/Watermakr doc
> ---
>
> Key: FLINK-8426
> URL: https://issues.apache.org/jira/browse/FLINK-8426
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.4.0
>Reporter: Christophe Jolif
>Assignee: Dmitrii Kniazev
>Priority: Trivial
>
> In
> https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamps_watermarks.html
> {{public class BoundedOutOfOrdernessGenerator extends
> AssignerWithPeriodicWatermarks}}
> should be
> {{public class BoundedOutOfOrdernessGenerator implements
> AssignerWithPeriodicWatermarks}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (FLINK-4286) Have Kafka examples that use the Kafka 0.9 connector
[
https://issues.apache.org/jira/browse/FLINK-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073961#comment-16073961
]
Dmitrii Kniazev edited comment on FLINK-4286 at 7/4/17 8:04 PM:
Cool. I suppose, in this case we can close this issue with 'won't fix'
resolution.
was (Author: mylog00):
Cool. In this case we can close this issue with 'won't fix' resolution.
> Have Kafka examples that use the Kafka 0.9 connector
>
>
> Key: FLINK-4286
> URL: https://issues.apache.org/jira/browse/FLINK-4286
> Project: Flink
> Issue Type: Improvement
> Components: Examples
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Dmitrii Kniazev
>Priority: Minor
> Labels: starter
>
> The {{ReadFromKafka}} and {{WriteIntoKafka}} examples use the 0.8 connector,
> and the built example jar is named {{Kafka.jar}} under
> {{examples/streaming/}} in the distributed package.
> Since we have different connectors for different Kafka versions, it would be
> good to have examples for different versions, and package them as
> {{Kafka08.jar}} and {{Kafka09.jar}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-4286) Have Kafka examples that use the Kafka 0.9 connector
[
https://issues.apache.org/jira/browse/FLINK-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073961#comment-16073961
]
Dmitrii Kniazev commented on FLINK-4286:
Cool. In this case we can close this issue with 'won't fix' resolution.
> Have Kafka examples that use the Kafka 0.9 connector
>
>
> Key: FLINK-4286
> URL: https://issues.apache.org/jira/browse/FLINK-4286
> Project: Flink
> Issue Type: Improvement
> Components: Examples
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Dmitrii Kniazev
>Priority: Minor
> Labels: starter
>
> The {{ReadFromKafka}} and {{WriteIntoKafka}} examples use the 0.8 connector,
> and the built example jar is named {{Kafka.jar}} under
> {{examples/streaming/}} in the distributed package.
> Since we have different connectors for different Kafka versions, it would be
> good to have examples for different versions, and package them as
> {{Kafka08.jar}} and {{Kafka09.jar}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (FLINK-5476) Fail fast if trying to submit a job to a non-existing Flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitrii Kniazev reassigned FLINK-5476:
--
Assignee: (was: Dmytro Shkvyra)
> Fail fast if trying to submit a job to a non-existing Flink cluster
> ---
>
> Key: FLINK-5476
> URL: https://issues.apache.org/jira/browse/FLINK-5476
> Project: Flink
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Till Rohrmann
>Priority: Minor
>
> In case of entering the wrong job manager address when submitting a job via
> {{flink run}}, the {{JobClientActor}} waits per default {{60 s}} until a
> {{JobClientActorConnectionException}}, indicating that the {{JobManager}} is
> no longer reachable, is thrown. In order to fail fast in case of wrong
> connection information, we could change it such that it uses initially a much
> lower timeout and only increases the timeout if it had at least once
> successfully connected to a {{JobManager}} before.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5476) Fail fast if trying to submit a job to a non-existing Flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902680#comment-15902680
]
Dmitrii Kniazev commented on FLINK-5476:
Hi, Till. Sorry for delay. Yes, we can configure it, but it will more
complexity than uses existing solution based on cluster status message and will
require more changes at code.
> Fail fast if trying to submit a job to a non-existing Flink cluster
> ---
>
> Key: FLINK-5476
> URL: https://issues.apache.org/jira/browse/FLINK-5476
> Project: Flink
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Till Rohrmann
>Assignee: Dmitrii Kniazev
>Priority: Minor
>
> In case of entering the wrong job manager address when submitting a job via
> {{flink run}}, the {{JobClientActor}} waits per default {{60 s}} until a
> {{JobClientActorConnectionException}}, indicating that the {{JobManager}} is
> no longer reachable, is thrown. In order to fail fast in case of wrong
> connection information, we could change it such that it uses initially a much
> lower timeout and only increases the timeout if it had at least once
> successfully connected to a {{JobManager}} before.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5476) Fail fast if trying to submit a job to a non-existing Flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863837#comment-15863837
]
Dmitrii Kniazev commented on FLINK-5476:
Thank for response, Till. Solution based on changing timing will affect Yarn и
Mesos clients, that I would like to avoid.
Furthermore the initial request timing should be hardcoded, that is a bad idea
from my point of view.
I whould prefer cluster lookup {{StandaloneClusterClient#getClusterStatus()}}
because it could be customised by "akka.lookup.timeout" property (default value
10s).
Hence I created new issue to add the support {{GetClusterStatus}} for
standalone cluster.
> Fail fast if trying to submit a job to a non-existing Flink cluster
> ---
>
> Key: FLINK-5476
> URL: https://issues.apache.org/jira/browse/FLINK-5476
> Project: Flink
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Till Rohrmann
>Assignee: Dmitrii Kniazev
>Priority: Minor
>
> In case of entering the wrong job manager address when submitting a job via
> {{flink run}}, the {{JobClientActor}} waits per default {{60 s}} until a
> {{JobClientActorConnectionException}}, indicating that the {{JobManager}} is
> no longer reachable, is thrown. In order to fail fast in case of wrong
> connection information, we could change it such that it uses initially a much
> lower timeout and only increases the timeout if it had at least once
> successfully connected to a {{JobManager}} before.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5786) Add support GetClusterStatus message for standalong flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863735#comment-15863735
]
Dmitrii Kniazev commented on FLINK-5786:
Example of log with exception
{panel:title=Log}
2017-02-10 15:35:25,460 INFO
org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager
- Consolidated 1 TaskManagers
2017-02-10 15:36:16,588 ERROR akka.actor.OneForOneStrategy
- GetClusterStatus (of class
org.apache.flink.runtime.clusterframework.messages.GetClusterStatus)
scala.MatchError: GetClusterStatus (of class
org.apache.flink.runtime.clusterframework.messages.GetClusterStatus)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:248)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:246)
at
org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:290)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
at
org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
at
org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33)
at
org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
at
org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
at
org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:118)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-02-10 15:36:16,593 INFO org.apache.flink.runtime.jobmanager.JobManager
- Stopping JobManager
akka.tcp://flink@localhost:6123/user/jobmanager.
2017-02-10 15:36:16,599 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager akka://flink/user/taskmanager disconnects from
JobManager akka://flink/user/jobmanager: JobManager requested disconnect:
JobManager is shuttind down.
2017-02-10 15:36:16,599 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Disassociating from JobManager
2017-02-10 15:36:16,600 INFO org.apache.flink.runtime.blob.BlobServer
- Stopped BLOB server at 0.0.0.0:46780
2017-02-10 15:36:16,603 ERROR org.apache.flink.runtime.jobmanager.JobManager
- Actor akka://flink/user/jobmanager#-364585011 terminated,
stopping process...
2017-02-10 15:36:16,603 INFO org.apache.flink.runtime.blob.BlobCache
- Shutting down BlobCache
2017-02-10 15:36:16,605 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Trying to register at JobManager
akka.tcp://flink@localhost:6123/user/jobmanager (attempt 1, timeout: 500
milliseconds)
2017-02-10 15:36:16,708 INFO
org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Removing web
dashboard root cache directory
/tmp/flink-web-ad368bef-7394-4065-8c98-704fb94777b6
2017-02-10 15:36:16,714 INFO
org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Removing web
dashboard jar upload directory
/tmp/flink-web-b0f08882-207c-49cc-bfef-30badbfab011
2017-02-10 15:36:16,722 INFO
org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager
removed spill file directory /tmp/flink-io-387025c9-b52c-4b71-9122-8d8d96c5a8a6
{panel}
> Add support GetClusterStatus message for standalong flink cluster
> -
>
> Key: FLINK-5786
> URL: https://issues.apache.org/jira/browse/FLINK-5786
> Project: Flink
[jira] [Created] (FLINK-5786) Add support GetClusterStatus message for standalong flink cluster
Dmitrii Kniazev created FLINK-5786:
--
Summary: Add support GetClusterStatus message for standalong flink
cluster
Key: FLINK-5786
URL: https://issues.apache.org/jira/browse/FLINK-5786
Project: Flink
Issue Type: Bug
Affects Versions: 1.2.0, 1.3.0
Reporter: Dmitrii Kniazev
Assignee: Dmitrii Kniazev
Priority: Minor
Currently, the invoke of {{StandaloneClusterClient#getClusterStatus()}} causes
the failure of all Flink cluster, because {{JobManager}} has no handler for
{{GetClusterStatus}} message.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5476) Fail fast if trying to submit a job to a non-existing Flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861388#comment-15861388
]
Dmitrii Kniazev commented on FLINK-5476:
For solving this task i decide invoke method
StandaloneClusterClient#getClusterStatus()
inside StandaloneClusterClient#waitForClusterToBeReady() to check cluster
availability.
But it causes to fall of flink cluster (v1.3) with following error:
{panel:title=Log}
2017-02-10 15:35:25,460 INFO
org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager
- Consolidated 1 TaskManagers
2017-02-10 15:36:16,588 ERROR akka.actor.OneForOneStrategy
- GetClusterStatus (of class
org.apache.flink.runtime.clusterframework.messages.GetClusterStatus)
scala.MatchError: GetClusterStatus (of class
org.apache.flink.runtime.clusterframework.messages.GetClusterStatus)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:248)
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:246)
at
org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:290)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
at
org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
at
scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
at
org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33)
at
org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
at
org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
at
org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:118)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-02-10 15:36:16,593 INFO org.apache.flink.runtime.jobmanager.JobManager
- Stopping JobManager
akka.tcp://flink@localhost:6123/user/jobmanager.
2017-02-10 15:36:16,599 INFO org.apache.flink.runtime.taskmanager.TaskManager
- TaskManager akka://flink/user/taskmanager disconnects from
JobManager akka://flink/user/jobmanager: JobManager requested disconnect:
JobManager is shuttind down.
2017-02-10 15:36:16,599 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Disassociating from JobManager
2017-02-10 15:36:16,600 INFO org.apache.flink.runtime.blob.BlobServer
- Stopped BLOB server at 0.0.0.0:46780
2017-02-10 15:36:16,603 ERROR org.apache.flink.runtime.jobmanager.JobManager
- Actor akka://flink/user/jobmanager#-364585011 terminated,
stopping process...
2017-02-10 15:36:16,603 INFO org.apache.flink.runtime.blob.BlobCache
- Shutting down BlobCache
2017-02-10 15:36:16,605 INFO org.apache.flink.runtime.taskmanager.TaskManager
- Trying to register at JobManager
akka.tcp://flink@localhost:6123/user/jobmanager (attempt 1, timeout: 500
milliseconds)
2017-02-10 15:36:16,708 INFO
org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Removing web
dashboard root cache directory
/tmp/flink-web-ad368bef-7394-4065-8c98-704fb94777b6
2017-02-10 15:36:16,714 INFO
org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Removing web
dashboard jar upload directory
/tmp/flink-web-b0f08882-207c-49cc-bfef-30badbfab011
2017-02-10 15:36:16,722 INFO
org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager
removed spill file directory /tmp/flink-io-387025c9-b52c-4b71-9122-8d8d96c5a8a6
{panel}
I think it a bug. What do you think about it and about
[jira] [Assigned] (FLINK-5476) Fail fast if trying to submit a job to a non-existing Flink cluster
[
https://issues.apache.org/jira/browse/FLINK-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitrii Kniazev reassigned FLINK-5476:
--
Assignee: Dmitrii Kniazev
> Fail fast if trying to submit a job to a non-existing Flink cluster
> ---
>
> Key: FLINK-5476
> URL: https://issues.apache.org/jira/browse/FLINK-5476
> Project: Flink
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Till Rohrmann
>Assignee: Dmitrii Kniazev
>Priority: Minor
>
> In case of entering the wrong job manager address when submitting a job via
> {{flink run}}, the {{JobClientActor}} waits per default {{60 s}} until a
> {{JobClientActorConnectionException}}, indicating that the {{JobManager}} is
> no longer reachable, is thrown. In order to fail fast in case of wrong
> connection information, we could change it such that it uses initially a much
> lower timeout and only increases the timeout if it had at least once
> successfully connected to a {{JobManager}} before.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5686) Print meaningful message when JM is lost
[
https://issues.apache.org/jira/browse/FLINK-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856248#comment-15856248
]
Dmitrii Kniazev commented on FLINK-5686:
I propose this solution for task:
{code:title=JobClientActor.java|borderStyle=solid}
if (jobManager != null && msg.address() != null) {
// only print this message when we had been connected to a JobManager
before
logAndPrintMessage("New JobManager elected. Connecting to " +
msg.address());
} else {
logAndPrintMessage("JobManager not available. Client will reconnect
after it will re-elected");
}
{code}
If it good enough, i can create PR.
> Print meaningful message when JM is lost
>
>
> Key: FLINK-5686
> URL: https://issues.apache.org/jira/browse/FLINK-5686
> Project: Flink
> Issue Type: Improvement
> Components: Client
>Reporter: Ufuk Celebi
>Priority: Minor
>
> When a leading JM dies, the client prints:
> {code}
> New JobManager elected. Connecting to null
> {code}
> This is not helpful to the user. Instead it should inform the user that no
> JobManager is available currently and that the client will reconnect as soon
> as one is re-elected.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-5629) Unclosed RandomAccessFile in StaticFileServerHandler#respondAsLeader()
[
https://issues.apache.org/jira/browse/FLINK-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854275#comment-15854275
]
Dmitrii Kniazev commented on FLINK-5629:
This code has comment
{code:title=StaticFileServerHandler.java|borderStyle=solid}
// Don't need to close this manually. Netty's DefaultFileRegion will take care
of it.
final RandomAccessFile raf;
try {
raf = new RandomAccessFile(file, "r");
}
catch (FileNotFoundException e) {
sendError(ctx, NOT_FOUND);
return;
}
long fileLength = raf.length();
{code}
I think this bug may be closed
> Unclosed RandomAccessFile in StaticFileServerHandler#respondAsLeader()
> --
>
> Key: FLINK-5629
> URL: https://issues.apache.org/jira/browse/FLINK-5629
> Project: Flink
> Issue Type: Bug
> Components: Webfrontend
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> final RandomAccessFile raf;
> try {
> raf = new RandomAccessFile(file, "r");
> ...
> long fileLength = raf.length();
> {code}
> The RandomAccessFile should be closed upon return from method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
