What you see is that the effect of extremely little memory arriving at the memory manager.
Given that you are saying that restarting the machine helps (or using a downloaded version of Flink), the reason is probably that you do not configure the JVM memory (in which case Java grabs a bit of the system's free memory). That is also why the downloaded installation works - it always starts Java with a fix memory size. Try adding "-Xmx500m" or so to your "lauch" options in the IDE. On Sun, May 10, 2015 at 9:19 PM, Michele Bertoni < michele1.bert...@mail.polimi.it> wrote: > Hi, > thank you for your fast answer > > next time it happens i will try > > Michele > > > > Il giorno 10/mag/2015, alle ore 19:45, Fabian Hueske <fhue...@gmail.com> > ha scritto: > > Hi Michele, > > Flink manages its own memory using the MemoryManager. When a Flink worker > (TaskManager) starts, the MemoryManager allocates a certain share of the > JVM's free memory as MemorySegments: Default size of a MemorySegment is > 32KB. When a Flink program is executed, the MemoryManager hands a certain > number of MemorySegments to each processing operator such as HashJoin or > Sort. These operators have a lower bound of MemorySegments which they need > for operations, e.g., 33 MemorySegments for a HashJoin. > > To solve your problem, you should try to give more memory to the Flink > JVM. You can also configure a fixed amount of memory for the MemoryManager > via the taskmanager.memory.size configuration parameter [1]. > > Cheers, Fabian > > [1] > http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#jobmanager-amp-taskmanager > > > 2015-05-10 19:18 GMT+02:00 Michele Bertoni < > michele1.bert...@mail.polimi.it>: > >> Hi everybody, >> I am running a flink instance on my IDE >> sometimes (totally random) I start it i getting this error >> >> Exception in thread "main" >> org.apache.flink.runtime.client.JobExecutionException: Job execution failed. >> at >> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306) >> at >> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) >> at >> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) >> at >> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) >> at >> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37) >> at >> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30) >> at >> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) >> at >> org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30) >> at akka.actor.Actor$class.aroundReceive(Actor.scala:465) >> at >> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91) >> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) >> at akka.actor.ActorCell.invoke(ActorCell.scala:487) >> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) >> at akka.dispatch.Mailbox.run(Mailbox.scala:221) >> at akka.dispatch.Mailbox.exec(Mailbox.scala:231) >> at >> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) >> at >> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >> at >> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >> at >> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >> Caused by: java.lang.Exception: The data preparation for task 'CHAIN >> Join(Join at >> org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) >> -> Map (Map at >> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33)) >> -> Combine(Distinct at >> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))' >> , caused an error: Too few memory segments provided. Hash Join needs at >> least 33 memory segments. >> at >> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472) >> at >> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362) >> at >> org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217) >> at java.lang.Thread.run(Thread.java:722) >> Caused by: java.lang.IllegalArgumentException: Too few memory segments >> provided. Hash Join needs at least 33 memory segments. >> at >> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373) >> at >> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359) >> at >> org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48) >> at >> org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78) >> at >> org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148) >> at >> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466) >> ... 3 more >> >> >> >> >> when this happens to solve it i need either to restart the computer or >> run it from the downloaded flink instance instead of the IDE >> >> sometime the error is different: it says something like “you must start >> the memory manager with a positive value now is 0MB” but also this one of >> 33 memory segment is quite common >> >> it is quite annoying to restart everything and it happens quite often >> >> do you have any idea why it happens? >> >> >> >> thank you > > > >
