Thanks to all for the help..now let's hope for a fix ;)
On 14 Jul 2015 22:01, "Andra Lungu" wrote:
> Hello,
>
> Sorry for the delay. The bug is not in Gelly, but is, as hinted in the
> exception and as can be seen in the logs, in Flink's Runtime. Mihail may
> actually be on to something. The bug
Ok thank you, now I solved it!
The problem was in the env.readFile(myInputFormat, path)
now that path is actually a list of paths what should I pass it?
I solved in this way
env.readFile(new myDelimitedInputFormat(parser)(paths), paths.head)
where that paths.head gives to the read file a ur
Hello,
Sorry for the delay. The bug is not in Gelly, but is, as hinted in the
exception and as can be seen in the logs, in Flink's Runtime. Mihail may
actually be on to something. The bug is actually very similar to the one
described in FLINK-1916.
However, as can be seen in the discussion thread
Sure, here is a example [1] of using `partitionCustom()` method in Java API.
Scala API is
similar to Java API.
You should implement Partitioner interface. The interface has a method
called
partition with two parameters. The first parameter is key value of each record
and
the second parameter i
Hi Chiwan,
great thanks. Is there any example available?
Regards
Juan
On Wed, 2015-07-15 at 01:19 +0900, Chiwan Park wrote:
> Hi, If you use `partitionCustom()` method [1] with custom partitioner, you
> can guarantee the order of partition.
>
> Regards,
> Chiwan Park
>
> [1]
> https://ci.
For the approach that I outlined, you need to subclass of the file input
format.
In that subclass, you store the list of URIs (in a new variable), and
override the "createInputSplits()" method.
Stephan
On Tue, Jul 14, 2015 at 6:42 PM, Michele Bertoni <
michele1.bert...@mail.polimi.it> wrote:
>
Hi Stephan, I started working on this today, but I am having a problem
Can you be a little more detailed in the procedure?
actually I don’t understand how to give to the input format the list of URI
since it will try putting it in a Path variable
createinputsplit does not receive the path but ta
Hi, If you use `partitionCustom()` method [1] with custom partitioner, you can
guarantee the order of partition.
Regards,
Chiwan Park
[1]
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/api/java/DataSet.html#partitionCustom(org.apache.flink.api.common.functions.
Hi,
I am doing pure map computation with typical benchmarks like
BlackScholes and NBody.
I am using local configuration with multiple threads. It seems like,
inside the chuck (total size / numThreads) the order is correct. But the
ordering between chunks is not correct, giving an incorrect res
Hi everybody, this discussion started in an other thread about a problem in
union, but you said it was a different error then i am opening a new topic
I am doing the union of two dataset and I am getting this error
Exception in thread "main" org.apache.flink.api.common.InvalidProgramException
Hi,
looks very similar to this bug:
https://issues.apache.org/jira/browse/FLINK-1916
Best,
Mihail
On 14.07.2015 14:09, Andra Lungu wrote:
Hi Flavio,
Could you also show us a code snippet?
On Tue, Jul 14, 2015 at 2:06 PM, Flavio Pompermaier
mailto:pomperma...@okkam.it>> wrote:
Hi to al
Hi Flavio,
Could you also show us a code snippet?
On Tue, Jul 14, 2015 at 2:06 PM, Flavio Pompermaier
wrote:
> Hi to all,
>
> in my vertex centric iteration I get the following exception, am I doing
> something wrong or is it a bug of Gelly?
>
> starting iteration [1]: CoGroup (Messaging) (6/8
Hi to all,
in my vertex centric iteration I get the following exception, am I doing
something wrong or is it a bug of Gelly?
starting iteration [1]: CoGroup (Messaging) (6/8)
IterationHead(WorksetIteration (Vertex-centric iteration
(test.gelly.functions.VUpdateFunction@1814786f
| test.gelly.func
Hi Paul,
I don't think you need 10 GB of heap space for the JobManager. Usually 1 GB
are sufficient.
Since you have 3 nodes, I would start Flink with 3 task managers.
I think you can also launch such a cluster:
./flink-0.9.0/bin/yarn-session.sh -n 3 -jm 1024 -tm 13000
Regarding the memory you are
Hi Paul,
when you run your Flink cluster with YARN then we cannot give the full
amount of the allocated container memory to Flink. The reason is that YARN
itself needs some of the memory as well. Since YARN is quite strict with
containers which exceed their memory limit (the container is instantly
hello community,
i want run my flink app on a cluster (cloudera 5.4.4) with 3 nodes (one pc
has i7 8core with 16GB RAM). now i want submit my spark job on yarn (20GB
RAM).
my script to deploy the flink cluster on yarn:
export HADOOP_CONF_DIR=/etc/hadoop/conf/
./flink-0.9.0/bin/yarn-session.sh -n
16 matches
Mail list logo
