This does not look like a Flink problem, but like a maven plugin bug, or a
setup problem.
Can you search is that is a known error in maven/scala?
On Thu, Jul 30, 2015 at 3:53 PM, skaterQiang wrote:
> Hello guys, I have met a problem when use mvn to build flink on mac
> I use
> mvn install -D
I think in short: Spark never worried about types. It is just something
arbitrary.
Flink worries about types, for memory management.
Aljoscha's suggestion is a good one: have a PythonTypeInfo that is dynamic.
Till' also found a pretty nice way to connect Python and Java in his
Zeppelin-based dem
The Pull Request is basically ready. I would like to benchmark a bit before
merging it.
The on-heap Flink-managed memory classes are highly optimized to be JIT
friendly. Just want to make sure that we don't loose that.
I have worked a lot on streaming issues lately, so this is still in my
backlog
I think a Tuple0 cannot be implemented like the current tuples, at least
with respect to runtime serialization.
The system makes the assumption that it makes progress in consuming bytes
when deserializing values. If a Tuple= never consumes data from the byte
stream, this assumption is broken. It w
I agree with Gyula here.
Getting the API right is too important to "quick fix" it.
On Fri, Jul 31, 2015 at 10:06 PM, Gyula Fóra wrote:
> Hi Matthias,
>
> I think Aljoscha is preparing a nice PR that completely reworks the
> DataStream classes and the information they actually contain. I don't t
I don't get the discussion here, can you help me with what you mean by
"different iteration heads and tails" ?
An iteration does not have one parallel head and one parallel tail?
On Fri, Jul 31, 2015 at 6:52 PM, Gyula Fóra wrote:
> Maybe you can reuse some of the logic that is currently there o
uce the streams passed
> into
> > closeWith tail operators. With this terminology we can have many heads
> and
> > tails with varying parallelism.
> >
> > Stephan Ewen ezt írta (időpont: 2015. aug. 2., V,
> > 20:16):
> >
> > > I don't get the discus
le0 at runtime aswell. ha...ha... -.-
> >
> > Guess I now know what I'm working on next.
> >
> > On 02.08.2015 21:24, Matthias J. Sax wrote:
> >> Can you elaborate how and why Python used Tuple0? If it cannot be
> >> serialized similar to regular Tuples, w
e> wrote:
> Thanks for the advice about Tuple0.
>
> I personally don't see any advantage in having "flink-tuple" project. Do
> I miss anything about it? Furthermore, I am not sure if it is a good
> idea the have too many too small projects.
>
>
> On 08/03/2015 1
t; To me it seems intuitive that I get the feedback at the head they way I
> > specify it at the tail. But maybe that's also just me... :D
> >
> > On Mon, 3 Aug 2015 at 00:14 Stephan Ewen wrote:
> >
> > > This model strikes me as pretty complicated. Imagine t
Sounds fair to rename them...
On Mon, Aug 3, 2015 at 11:46 AM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Hi,
>
> I think that the log files are not named consistently (and it annoys me
> every single time).
>
> -> flink-mjsax-jobmanager-log
> -> flink-mjsax-jobmanager-out
Seen this a few times as well.
May be something with the latest "partitioned state" changes...
On Mon, Aug 3, 2015 at 5:48 PM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Hi,
>
> I just hit a failing test
> (https://travis-ci.org/apache/flink/jobs/73899795). It is know or new?
>
>
;> > Best regards,
> >> >> > Felix
> >> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <
> >> lungu.an...@gmail.com>:
> >> >> >
> >> >> >> Hi Stephan,
> >> >> >
schaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN
>
> -Original Message-
> From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of
> Stephan Ewen
> Sent: Friday, July 31, 2015 4:02 PM
> To: de
Yes, the build stability is super serious right now.
Here are the problems in question, and what we could do about this:
BarrierBuffer:
Barrier Buffer tests fail in Java 6 builds.
I have not found a way to diagnose that problem, yet, but if we cannot find
the issue today,
> Honestly I don't think the partitioned state changes have anything to do
> with the stability, only the reworked test case, which now test proper
> exactly-once which was missing before.
>
> Stephan Ewen ezt írta (időpont: 2015. aug. 4., K,
> 12:12):
>
> > Yes, the
ropping java 6 support?
>
>
> On 04.08.2015 12:21, Stephan Ewen wrote:
>
>> The "StateCheckpointedITCase" has not failed so far, which also test these
>> guarantees thoroughly.
>>
>> But we need to first rule out the BarrierBuffer. The problem is t
That is interesting, never seen that before.
Travis support would be best, I guess.
On Wed, Aug 5, 2015 at 10:04 AM, huangwei (G)
wrote:
> Thank you!
> I`ll try supp...@travis-ci.com.
> It`s just a line printing "Do you have a question?" and nothing more when
> I click the link.
>
> -邮件原件--
Hi all!
We just published a blog post about how streaming fault tolerance
mechanisms evolved, and what kind of performance Flink gets with its
checkpointing mechanism.
I think it is a pretty interesting read for people that are interested in
Flink or data streaming in general.
The blog post talk
I forgot the link ;-)
http://data-artisans.com/high-throughput-low-latency-and-exactly-once-stream-processing-with-apache-flink/
On Wed, Aug 5, 2015 at 4:11 PM, Stephan Ewen wrote:
> Hi all!
>
> We just published a blog post about how streaming fault tolerance
> mechanisms evolv
> On Wed, Aug 5, 2015 at 4:11 PM, Stephan Ewen wrote:
>
> > Hi all!
> >
> > We just published a blog post about how streaming fault tolerance
> > mechanisms evolved, and what kind of performance Flink gets with its
> > checkpointing mechanism.
> >
>
I think we are all eager for Robert's rework of the Kafka connector and
tests...
On Mon, Aug 3, 2015 at 12:38 AM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Hi,
>
> I hit a failing test KafkaITCase.bigRecordTestTopology
> https://travis-ci.org/mjsax/flink/jobs/73807115
>
> I know t
I have a pending fix that uses a linguist git attribute to mark the
directory with generated files as "vendored", which excludes it.
On Tue, Jul 28, 2015 at 9:25 PM, Stephan Ewen wrote:
> Good idea!
>
> On Mon, Jul 27, 2015 at 10:31 AM, Maximilian Michels
> wrote:
>
Very interesting addition, Sachin!
I cc-ed Nam-Luc Tran, who recently opened a pull request with a draft for
SSP iterations in Flink, which usually work closely together with a
parameter server.
I think that a parameter server is an orthogonal piece to Flink, and should
be decoupled from the runt
It would also help if you could paste the entire POM file of your project.
Maybe something is amiss with the dependencies, or the scopes?
On Sat, Aug 8, 2015 at 7:27 PM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Hi Huang,
>
> about Storm compatibility. Did you double check, that t
May be an issue with the embedded YARN mini cluster...
On Mon, Aug 10, 2015 at 8:37 PM, Stephan Ewen wrote:
> I think the YARN problem is as before, but with a longer timeout.
>
> Before, when after 60 seconds the expected output did not come, the tests
> aborted.
> The tim
I think the YARN problem is as before, but with a longer timeout.
Before, when after 60 seconds the expected output did not come, the tests
aborted.
The timeout is now 180 seconds, which is probably so long that the deadlock
detector (5 minutes no output) kicks in.
In any case, there is something
Hi!
You can think of it the following way: When you execute a join, the data
shuffled from node A to node B for the left side of the join will have a
separate TCP connection than the data shuffled from node A to node B for
the right side of the join.
That is currently important to avoid distribut
Hi Christian!
Sounds like a very cool proposal, and it seems that Flink and SPQR could
complement each other very well.
If you are interested to see this on top of Flink, let's have a chat to see
how to best get started with this - what would be the easiest points of
integration.
As said, Flink
y/experiments/Throughput$*.class
> >
> >
> >
> >
> >
> >
> >
> test-jar
> >
> >
> >
@ffbin: Would this be a JVM singleton that updates a static field every
millisecond?
It is hard to say at this point whether this eliminates a bottleneck, but I
think it is fine to try.
To evaluate this, you could use a streaming job that attaches a timestamp
to every record, and measure the diff
Same here as for Max, I am not familiar enough any more to make really good
comments.
Some generic comments, though:
- Check whether you really need a technique. Techniques that improve
corner cases, but make the code much more complex and make the behavior of
algorithms less robust are often b
That is a shading problem.
Are you including Guava in your dependencies, or are you only
"transitively" referring to it?
Have a look here:
https://cwiki.apache.org/confluence/display/FLINK/Hadoop+Versions+and+Dependency+Shading
On Thu, Aug 13, 2015 at 2:19 PM, Matthias J. Sax <
mj...@informatik.
+1 for proposal (2)
On Fri, Aug 14, 2015 at 10:28 AM, Fabian Hueske wrote:
> Hi Liang Chen,
>
> thanks for starting this discussion.
>
> I have no strong preference for one of your proposals. Both are fine with
> me.
>
> What would proposal 2 mean if we add examples for other API languages, for
Hi!
Nice idea.
It would be good to have this implemented as a kind of service that
components can "acquire".
A component that needs time can do something like "Clock clock =
ClockService.aquire()", which increments a reference count in the central
clock service and starts the thread if it has no
opic: What we could also do is to move the shared components
> (e.g. the testing data) to a distinct module (flink-examples/example-data).
> But since the generated scala job jars only contain the java data classes
> this won't change much at all.
>
> On Fri, Aug 14, 2015 at 10
> when you really want the current time.
>
>
> On Fri, Aug 14, 2015 at 2:03 AM, Stephan Ewen wrote:
>
> > Hi!
> >
> > Nice idea.
> >
> > It would be good to have this implemented as a kind of service that
> > components can "acquir
Hi!
I very much support that. A bit stricter rules in the style checkers lead
to more uniform and better readable code. We can have stricter rules both
in Java and Scala.
Note that the hardest part of adding the style checks is actually adjusting
all the existing code that violates the style.
Th
Hi all!
Henry raised the point about non.descriptive bug reports earlier. I would
like to bring this to everyone's mind again and add some additional
thoughts:
We are seeing a lot of issues reported right now, and a lot of pull
requests opened right now, for issues that are not really a problem.
puter Science, IIT Delhi
> m. +91-9871457685
>
> On Sun, Aug 16, 2015 at 7:36 PM, Stephan Ewen wrote:
>
> > Hi!
> >
> > I very much support that. A bit stricter rules in the style checkers lead
> > to more uniform and better readable code. We can have stricter rules bot
Looks like a rare race between the cleanup (two changes) and the test
validating both changes.
I'll push a fix to make the test more reliable.
On Sun, Aug 16, 2015 at 11:04 PM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Hi,
>
> I hit a failing test in flink-runtime. Not sure if it
Hi Pieter!
The pattern you need is common and we have code for such loops in the
works. It is being delayed right now due to work on fault-tolerance / high
availability, but we plan to resume with that
after those issues are done.
Greetings,
Stephan
On Tue, Aug 18, 2015 at 12:38 PM, Pieter-Jan
Hi!
The thinking is good. Although in this case, it is probably not beneficial
to replace the list by a heap or so, due to the rare usage of that function.
As you said, the cost depends on how often the "getMaxPartition()" is
called.
Current approach: seldom O(n) - (only on function call)
Heap a
Hi!
This looks like your IDE setup is not quite correct. The Scala library is
probably not in the classpath when you launch the program.
Please check if you correctly installed your Scala Plugin, of the Scala Lib
has been added to the build path in the project properties, ...
Greetings,
Stephan
Hi!
Can you explain what you mean by "BTW, my personal opinion: currently scala
be acknowledged by more and more developers, in future there may have some
different examples for scala." ?
It would be fine to have different Scala and Java examples, there is no
need to have them in sync (although t
Hi!
The beta status refers exactly to API stability and test coverage.
Full status APIs, like the batch API, are very stable. For example, we have
not made any breaking changes in the last two releases (except for rare
corner case flags). The test coverage is also pretty good.
Beta APIs (like st
Welcome!
On Fri, Aug 21, 2015 at 10:42 AM, Fabian Hueske wrote:
> Welcome on board Chesnay!
>
>
> 2015-08-20 21:25 GMT+02:00 Andra Lungu :
>
> > Congrats Chesnay! :)
> >
> > On Thu, Aug 20, 2015 at 8:51 PM, Márton Balassi <
> balassi.mar...@gmail.com>
> > wrote:
> >
> > > Welcome Chesnay!
> > >
Seems fair. This should not immediately fail, but also only give a
"MissingTypeInfo" so that type hints would work.
The error message is also wrong, btw: It should state that raw types are
not supported.
On Fri, Aug 21, 2015 at 1:02 PM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> H
Hi!
Usually, TaskManagers should automatically choose a network interface from
which they can communicate to the JobManager. They also use that network
interface to talk to each other, then.
If I understand correctly, what happened is that the TaskManager that ran
on the same machine as the JobMa
This is an interesting issue, because, quite frankly, the join hint you
passed simply reversed the sides of the join. The algorithm is still the
same and has the same minimum memory requirements.
The fact that it made a difference is quite curious. The only thing I can
imagine is that this hint ch
Pushed a fix for the StateCheckpointedITCase
On Mon, Aug 24, 2015 at 12:19 PM, Maximilian Michels wrote:
> +1 for labeling the JIRAs with "test-stability".
>
> On Sat, Aug 22, 2015 at 8:21 PM, Márton Balassi
> wrote:
>
> > +1 for Vasia's suggestion
> > On Aug 22, 2015 8:07 PM, "Vasiliki Kalavri
+1 for a "test-stability" label and labeling these issues as "critical"
On Mon, Aug 24, 2015 at 6:31 PM, Stephan Ewen wrote:
> Pushed a fix for the StateCheckpointedITCase
>
> On Mon, Aug 24, 2015 at 12:19 PM, Maximilian Michels
> wrote:
>
>> +1 fo
Looking into this. Seems like a wrong use of assertions in a parallel
thread, where errors are not propagated to the main JUNit thread.
On Mon, Aug 24, 2015 at 6:53 PM, Sachin Goel
wrote:
> Hi Max
> I have already created a Jira for this.
> https://issues.apache.org/jira/browse/FLINK-2528
> The
Test for a fix is pending...
On Tue, Aug 25, 2015 at 1:30 PM, Stephan Ewen wrote:
> Looking into this. Seems like a wrong use of assertions in a parallel
> thread, where errors are not propagated to the main JUNit thread.
>
> On Mon, Aug 24, 2015 at 6:53 PM, Sachin Goel
> wro
> > Hi all,
> >
> >
> > thank you so much for your comments!
> >
> > Most of them have been addressed already.
> > I would also like to add Fabian's suggestion on providing a library
> > method
> > for initializing vertices with unique IDs an
Hi all!
I have started to try and backport some of the bugfixes on the latest
master to the 0.9 branch, to be part of the 0.9.1 bugfixing release.
While doing this, I came across so many places where critical bugs were
fixed as parts of larger refactorings. It is very hard to backport these
bug f
We can also try and "rebase" a fork of the maser to the "0.9" branch, where
we select something like 70%-80% of the commits (all fixes and reworks) and
drop the API beaking ones.
Let me try this and see how feasible it is...
On Wed, Aug 26, 2015 at 9:52 AM, Ufuk Celebi wrote:
> I think you are
> maybe these are even it:
> https://cwiki.apache.org/confluence/display/FLINK/0.10+Release
>
> I added my breaking stuff there. And of course the whole Timestamp thing is
> a change, but it does not affect the normal source interface.
>
> On Wed, 26 Aug 2015 at 10:24 Stephan E
source contexts)
On Wed, Aug 26, 2015 at 10:46 AM, Stephan Ewen wrote:
> The timestamp thing is one of the biggest questions.
>
> The fixes that came as part of that pull request are crucial and hard to
> pull out of the change.
>
> On Wed, Aug 26, 2015 at 10:44 AM, Aljosch
see applying a
> subset of carefully selected commits from the master branch as an
> option. Only those commits should be cherry-picked which are required
> to fix the streaming issues.
>
> On Wed, Aug 26, 2015 at 10:50 AM, Stephan Ewen wrote:
> > @Aljoscha: Correct me if I am
That sounds like a very good compromise.
+1
On Wed, Aug 26, 2015 at 2:48 PM, Fabian Hueske wrote:
> I'm +1 for Robert's proposal as well.
>
> 2015-08-26 14:46 GMT+02:00 Ufuk Celebi :
>
> > +1
> >
> > I very much like Robert's suggestion. This way we can proceed with the
> > 0.9.1 release as pla
Quick update:
Ufuk and me worked on fixing the last issues for 0.9.1 today, bringing the
branch up to speed for the minor release.
Ufuk de facto stepped up as release manager and will create a release
candidate later today or early tomorrow. So we can get cracking with
testing the release candida
+/- 0
One minor issue:
- The binary distribution LICENSE file mentions a wrong version of
javakaffee (mentions 27, but we reference 36). If we redo the release
candidate, this should be fixed.
Performed the following checks:
- Checked LICENSE / NOTICE files
- Checked the README file
- Built
If you start the job via the "bin/flink" script, then simply use
"ExecutionEnvironment.getExecutionEnvironment()" rather then creating a
remote environment manually.
That way, hosts and ports are configured automatically.
On Thu, Aug 27, 2015 at 6:39 PM, Robert Metzger wrote:
> Hi,
>
> Which va
e and try to run the
> client program from our machine, using create remote environment- Flink
> JobManager is raising this exception
>
> On Thu, Aug 27, 2015 at 7:41 PM, Stephan Ewen wrote:
>
>> If you start the job via the "bin/flink" script, then simply use
>&
akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
> > > > at akka.actor.ActorCell.terminate(ActorCell.scala:369)
> > > > at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
> > > > at akka.actor.ActorCell.systemInvoke(ActorCell.scala:4
Hi!
I found quite a few tests that are not actually executed by Maven as part
of the builds. Some actually are in error.
ExecutionGraphRestartTest
TaskManagerLossFailsTasksTest
JobManagerRegistrationTest
...
All of those are Scala tests with WordSpecLike traits. Seems that this
configuration has
on.FaultHandling$class.terminate(FaultHandling.scala:172)
> > > > at akka.actor.ActorCell.terminate(ActorCell.scala:369)
> > > > at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
> > > > at akka.actor.ActorCe
+1
Performed the following checks:
- Checked LICENSE / NOTICE files
- Checked the README file
- Built against Hadoop 2.6.0
- Built against Scala 2.11
- Executed all tests
- Builds in IntelliJ
- Manual tests all work
On Fri, Aug 28, 2015 at 1:27 PM, Ufuk Celebi wrote:
> Dear community,
>
; On Sun, Aug 30, 2015 at 3:19 PM, Robert Metzger
> wrote:
>
> > The output of the YARN session should look like this:
> >
> > Flink JobManager is now running on quickstart.cloudera:39956
> > JobManager Web Interface:
> > http://quickstart.cloudera:8088/proxy/appli
uot;hostname"
> command.
> > It crashes.
> > It works only with "localhost" and works only locally
> > So what you suggest we will put there so the remote client could connect.
> > ?
> >
> > Best regards
> >
> > On Sun, Aug 30, 2015
with 0.0.0.0- basically this is
> a
> > > way in Java to make it being seen widely.
> > > We tried to put there the hostname - what you get from "hostname"
> > command.
> > > It crashes.
> > > It works only with "localhost" and works
; ale...@scalabill.it
> > > >
> > > > wrote:
> > > >
> > > > > Hello Stephan.
> > > > >
> > > > > We run this Linux machine on Amazon, which I predict, most of the
> > > people
> > > > > will do.
15 at 4:44 PM, Alexey Sapozhnikov
wrote:
> Fully understand.
> 1.My suggestion is to drop Akka and take something else, since this issue
> is really big
> 2.Not hostname not the endpoint are not working, clarifying the VPC topic
> now.
>
> On Sun, Aug 30, 2015 at 5:41 PM, Steph
gt; Thats why all app servers, for example JBoss or even Flume are using
> "0.0.0.0"
>
> On Sun, Aug 30, 2015 at 5:53 PM, Stephan Ewen wrote:
>
> > What you can do as a temporary workaround is to actually enter the IP
> > address for "jobmanager.rpc.address&qu
Welcome!
On Wed, Sep 2, 2015 at 2:08 PM, Till Rohrmann wrote:
> Congratulations Matthias! Welcome on board :-)
>
> On Wed, Sep 2, 2015 at 2:01 PM, Robert Metzger
> wrote:
>
> > Congrats Matthias!
> >
> > On Wed, Sep 2, 2015 at 1:34 PM, Timo Walther wrote:
> >
> > > Congratulations Matthias!
>
The BlobClassloader uses the App classloader as the parent. That should
make the classloading go first to the app classpath (TaskManager's
classpath) and then to the user-defined code.
Is that broken somehow?
On Sun, Sep 6, 2015 at 1:40 PM, gkrastev wrote:
> Hi,I'm facing the exact same issue
Effectively, the method with the blob-classloader / parent classloader is
exactly intended to do that: Start with the app class path and add some
jars in addition, if you need to.
On Mon, Sep 7, 2015 at 1:05 PM, Stephan Ewen wrote:
> The BlobClassloader uses the App classloader as the par
The JobConfig is a system level config. Would be nice to not expose them to
the user-level unless necessary.
What about using the ExecutionConfig, where you can add shared user-level
parameters?
On Mon, Sep 7, 2015 at 1:39 PM, Matthias J. Sax wrote:
> Thanks for the input.
>
> However, I doubt
I have a patch pending that should help with these timeout issues (and null
checks)...
On Mon, Sep 7, 2015 at 2:41 PM, Matthias J. Sax wrote:
> Please lock here:
>
> https://travis-ci.org/apache/flink/jobs/79086396
>
> > Failed tests:
> > KafkaITCase>KafkaTestBase.prepare:155 Test setup failed:
Hi all!
Some day back we talked about releasing an 0.10.0-milestone1 release. The
master has advanced quite a bit (especially due to high-availability code).
I cherry picked the important additions to the release-0.10.0-milestone1
branch (fixes and Kafka consumer/producer rework).
How about rele
@Gyula
Can you explain a bit what this KeyValue store would do more then the
partitioned key/value state?
On Tue, Sep 8, 2015 at 2:49 PM, Gábor Gévay wrote:
> Hello,
>
> As for use cases, in my old job at Ericsson we were building a
> streaming system that was processing data from telephone net
s the Web Client's error and results display for jobs.
> > Would be nice to fix it but IMHO it is not critical for the milestone
> > release.
> >
> > On Tue, Sep 8, 2015 at 1:00 PM, Ufuk Celebi wrote:
> > >
> > >> On 08 Sep 2015, at 12:01, Stephan
ffects the Web Client's error and results display for jobs.
> > > Would be nice to fix it but IMHO it is not critical for the milestone
> > > release.
> > >
> > > On Tue, Sep 8, 2015 at 1:00 PM, Ufuk Celebi wrote:
> > > >
> > > >&
Hi!
I think there are four ways to do this:
1) Create a new abstract class "SerializableBaseTaskHook" that extends
BaseTaskHook and implements java.io.Serializabe. Then write the object into
bytes and put it into the config.
2) Offer in the StormCompatibilityAPI a method "public void addHook(X
Did you do a "clean" together with the "install"? Then it should work.
The problem occurred when you switch between versions where the link is set
(0.9+) and versions prior to the link (< 0.9) ...
Stephan
On Thu, Sep 10, 2015 at 1:37 PM, Matthias J. Sax wrote:
> I could resolve this by manual
Anybody around to pick this one up?
> >>>>>
> >>>>> Cheers, Fabian
> >>>>>
> >>>>> 2015-09-09 12:00 GMT+02:00 Till Rohrmann :
> >>>>>
> >>>>>> I forgot to mention that there is also a bug wi
fix of FLINK-2637 (TypeInfo equals + hashcode).
> Once this is in, we do the first release candidate, OK?
>
> 2015-09-11 14:48 GMT+02:00 Stephan Ewen :
>
> > Fixes come up on a daily base. I think at some point we need to declare
> > freeze, or this thing will never be releas
h, here's the discussion I was looking for :-)
> > I think Stephan refers to ExecutionConfig.setGlobalJobParameters().
> >
> > 2015-09-15 0:25 GMT+02:00 Matthias J. Sax :
> >
> >> It might sound stupid. But how could such a configuration be set?
> >>
>
ion) and you
> want
> > > to
> > > > > > combine it with the last known avg temperature to produce (city,
> > > temp,
> > > > > pop)
> > > > > > triplets. The second stream is a pair of cities (city,city) and
> you
> &
Do you know between what classes the exception is? What is the original
class and what the cast target class?
Are they both the same, and not castable because of different copies being
loaded from different classloaders, or are they really different types?
On Wed, Sep 16, 2015 at 10:36 AM, Matthi
:56 PM, Matthias J. Sax wrote:
> > Not right now... The ClassCastException occurs deeply inside Storm. I
> > guess it is a Storm issue.
> >
> > I just instantiate a (Storm)LocalCluster and it fails internally.
> >
> >> LocalCluster cluster = new LocalCluster
Hi all!
The build time of Flink with all tests is nearing 1h on Travis for the
shortest run.
It is good that we do excessive testing, there are many mechanisms that
need that.
I have also seen that a lot of fixes that could be tested in a UnitTest
style are actually tested as a full Flink program
Good problem...
We were thinking for a while to make the input and output type serializers
available from the RuntimeContext.
That way you could call "T copy = serializer.copy(inValue)".
The "copy()" method on the copyable value is actually a good addition
nonetheless!
On Thu, Sep 17, 2015 at 1
Hi!
I just saw that all tests in the streaming API are declared as Unit tests,
even though the vast majority are integration tests (spawn mini clusters).
That leads to problems, because the streaming test mini clusters do not
properly clean up after themselves and unit tests reuse JVMs (for speed
This happened locally on your machine?
Can you dump the stack-trace and post it? "jps >
stacktrace.txt" or so...
On Mon, Sep 21, 2015 at 10:09 PM, Vasiliki Kalavri <
vasilikikala...@gmail.com> wrote:
> Hi squirrels,
>
> I've been meaning to merge a PR (#1520), but my local maven build gets
> st
I am actually very happy that it is not the
"BarrierBufferMassiveRandomTest", that would be hell to debug...
On Mon, Sep 21, 2015 at 10:51 PM, Stephan Ewen wrote:
> Ah, actually it is a different test. I think you got confused by the
> sysout log, because multiple parallel t
rio=5 tid=0x00007ff9d1805000 nid=0x2503
> runnable
>
> "GC task thread#4 (ParallelGC)" prio=5 tid=0x7ff9d1805800 nid=0x2703
> runnable
>
> "GC task thread#5 (ParallelGC)" prio=5 tid=0x7ff9d1806800 nid=0x2903
> runnable
>
> "GC task thread#
Sounds like a nice idea!
Do you want to make this a new maven project as part of the Flink
repository, or create a dedicated repository for that?
BTW: We are currently not mixing microbenchmarks with test execution. The
code for these benchmarks resides in the test scope of the projects (so it
is
The Record type will probably stay, some people requested to keep it.
We only discourage its further use in examples and tests.
The Record-based API will definitely go...
On Tue, Sep 22, 2015 at 4:38 PM, Greg Hogan wrote:
> Is org.apache.flink.types.Record to be deprecated or just the Record A
901 - 1000 of 2273 matches
Mail list logo
