Hi Robert, this might be a problem with Eclipse not having a strict separation between compiling the src/main and src/test code. The code that generates the TypeInformation is a macro. Macros are only usable if the code that uses them is compiled in a separate compilation step from the compilation step that copiles the macro itself.
If Eclipse cannot deal with this we might have to add all the Scala tests to the flink-tests package. There it would be ensured that they are compiled in a separate step. Hopefully? Cheers, Aljoscha On Sun, Sep 28, 2014 at 9:25 PM, Henry Saputra <[email protected]> wrote: > Hi Robert, I didn't have problem with IntelliJ idea last week, will try it > again with latest master. > > User can deactivate unwanted maven profile via idea's maven project option > (not remember exact name of that feature) > > On Sunday, September 28, 2014, Robert Metzger <[email protected]> wrote: > >> I've worked on a quite outdated version of Flink for a while now and >> rebased my code to the latest master on Friday. >> Back at home, I wanted to continue my work and found that it is very >> difficult to properly set up the latest eclipse for Flink. >> >> What I've done so far: >> - Downloaded Eclipse Luna SR1 (for linux) >> - Installed Scala IDE 4.0 nightly >> - add the -clean flag to the eclipse.ini, because eclipse is not starting >> anymore due to a bug >> - installed some missing m2e connectors >> - Set the Scala compiler to 2.10 (thats a new feature in Scala IDE) (See: >> http://scala-ide.org/blog/scala-installations.html) >> >> Now, all the tests in flink-scala/src/test are mostly about this: >> could not find implicit value for evidence parameter of type >> org.apache.flink.api.common.typeinfo.TypeInformation[(Array[String],)]; >> /flink-scala/src/test/scala/org/apache/flink/api/scala/runtime >> >> It seems that the flink-tests project is calling Scala examples for testing >> purposes. So if the flink-scala project is not building, its blocking >> flink-tests from building. >> >> Any ideas? >> >> If you are wondering why I'm not switching to IntelliJ: It is currently not >> possible to deactivate a maven build profile that is activated by default, >> so I'm not able to properly work on the "flink-yarn" project with IntelliJ >> (the respective issue has been closed at IntelliJ, so I guess the next >> release will contain the fix). >> In addition, I think its good if at least a few core developers make sure >> that Flink is also "developable" using Eclipse. >> >> >> >> >> On Sun, Sep 14, 2014 at 12:48 PM, Márton Balassi <[email protected] >> <javascript:;>> >> wrote: >> >> > Answer posted to "Example packages naming convention" thread as the issue >> > diverged from this topic. >> > >> > On Sun, Sep 14, 2014 at 11:14 AM, Kostas Tzoumas <[email protected]> >> > wrote: >> > >> > > Good catch, I suggest to use examples >> > > >> > > On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi < >> > [email protected]> >> > > wrote: >> > > >> > > > Pull request issued. One minor naming concern: >> > > > >> > > > As of today the scala examples are located at >> > > > the org.apache.flink.examples.scala package, while the java ones in >> > > > the org.apache.flink.example.java. I suggest using only one >> convention >> > > for >> > > > this either example or examples. >> > > > >> > > > Cheers, >> > > > >> > > > Marton >> > > > >> > > > On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi < >> > > [email protected]> >> > > > wrote: >> > > > >> > > > > Sorry for being a bit silent after already bidding on LR. The pull >> > > > request >> > > > > is coming soon. >> > > > > >> > > > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[email protected]> >> > > wrote: >> > > > > >> > > > >> I suppose that having the option between simple return type, and a >> > > > >> collector is the easiest to understand. >> > > > >> Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" < >> > [email protected] >> > > >: >> > > > >> >> > > > >> > So, should I change join and coGroup to have a simple return >> > value, >> > > no >> > > > >> > Option or Collection? Also what's happening with the relational >> > > > >> > examples and the LinearRegression examples? I'd like to make a >> > pull >> > > > >> > request before this weekend. >> > > > >> > >> > > > >> > I also added a test that checks whether the Scala API has the >> same >> > > > >> > methods as the Java API (ScalaAPICompletenessTest). >> > > > >> > >> > > > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek < >> > > > [email protected]> >> > > > >> > wrote: >> > > > >> > > Yes, there is already a Collector version, you can do: >> > > > >> > > >> > > > >> > > left.join(right).where("foo").equalTo("bar") { >> > > > >> > > (left, right, out: Collector[Page]) => >> > > > >> > > if (...) out.collect(...) >> > > > >> > > } >> > > > >> > > >> > > > >> > > I wasn't sure on what our Function2 variant should be. That's >> > why >> > > I >> > > > >> > > asked. There are some cases where you want to have the option >> > (pun >> > > > >> > > intended) of either returning something from a join or not. >> But >> > > you >> > > > >> > > could also use a filter after the join. It's a preference >> > thing, I >> > > > >> > > suppose. >> > > > >> > > >> > > > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen < >> [email protected] >> > > >> > > > >> wrote: >> > > > >> > >> I think it seems weird that normal joins need to go through >> > > option. >> > > > >> > >> >> > > > >> > >> The option variant is to allow filters in the join function. >> > > > >> Wouldn't a >> > > > >> > >> collector variant allow you to do the same, and would be >> > > function3 >> > > > ? >> > > > >> I >> > > > >> > know >> > > > >> > >> that option reads more functionally... >> > > > >> > >> >> > > > >> > >> >> > > > >> > >> >> > > > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" < >> > > > [email protected] >> > > > >> >: >> > > > >> > >>> >> > > > >> > >>> As already mentioned this is not possible because of type >> > > erasure. >> > > > >> We >> > > > >> > >>> can only have one join variant that takes a Function2. >> > > > >> > >>> >> > > > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen < >> > > [email protected]> >> > > > >> > wrote: >> > > > >> > >>> > It would be nice to have a join variant that directly >> > returns >> > > > the >> > > > >> > value >> > > > >> > >>> > rathern than an option. Why not have both (they are >> wrapped >> > as >> > > > >> > flatJoins >> > > > >> > >>> > anyway below, right?) >> > > > >> > >>> > >> > > > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske < >> > > > >> [email protected]> >> > > > >> > >> wrote: >> > > > >> > >>> > >> > > > >> > >>> >> Sweet! I'm lovin' this :-) >> > > > >> > >>> >> >> > > > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek < >> > > > [email protected] >> > > > >> >: >> > > > >> > >>> >> >> > > > >> > >>> >> > Also, you can use CaseClasses directly as the type for >> > CSV >> > > > >> input. >> > > > >> > So >> > > > >> > >>> >> > instead of reading it as tuples and then having a >> mapper >> > > that >> > > > >> > maps to >> > > > >> > >>> >> > your case classes you can use: >> > > > >> > >>> >> > >> > > > >> > >>> >> > env.readCsv[Edge](...) >> > > > >> > >>> >> > >> > > > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek < >> > > > >> > >> [email protected]> >> > > > >> > >>> >> > wrote: >> > > > >> > >>> >> > > I added support for specifying keys by name for >> > > > CaseClasses. >> > > > >> > Check >> > > > >> > >> out >> > > > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see >> it >> > > in >> > > > >> > action. >> > > > >> > >>> >> > > >> > > > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H >> > > examples. >> > > > >> > >>> >> > > >> > > > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek < >> > > > >> > >> [email protected] >> > > > >> > >>> >> > >> > > > >> > >>> >> > wrote: >> > > > >> > >>> >> > >> Yes, that would allow list comprehensions. It would >> be >> > > > >> > possible to >> > > > >> > >>> >> > >> have the Collection signature for join (and >> coGroup), >> > > > i.e.: >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O] >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> (T and O are the left and right input type, R is >> > result >> > > > >> type) >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> Then you can return collections and still return an >> > > > option, >> > > > >> as >> > > > >> > in: >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > >> ...) >> > > > >> Some(l) >> > > > >> > >> else >> > > > >> > >>> >> > None } >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> Because there is an implicit conversion from Options >> > to >> > > a >> > > > >> > >> Collection. >> > > > >> > >>> >> > >> This will always wrap the return value in a List >> with >> > > only >> > > > >> one >> > > > >> > >> value. >> > > > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not >> > > sure >> > > > >> > whether >> > > > >> > >> we >> > > > >> > >>> >> > >> want the overhead of always having to use an Option >> > even >> > > > >> though >> > > > >> > >> the >> > > > >> > >>> >> > >> join always returns a value. >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> What do you think? >> > > > >> > >>> >> > >> >> > > > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske < >> > > > >> > >> [email protected]> >> > > > >> > >>> >> > wrote: >> > > > >> > >>> >> > >>> Hmmm, tricky question... >> > > > >> > >>> >> > >>> How about the Option for Join as this is a >> tuple-wise >> > > > >> > operation >> > > > >> > >> and >> > > > >> > >>> >> the >> > > > >> > >>> >> > >>> Collection for Cogroup which is group-wise? >> > > > >> > >>> >> > >>> Could we in that case use list comprehensions in >> > > Cogroup >> > > > >> > >> functions? >> > > > >> > >>> >> > >>> >> > > > >> > >>> >> > >>> Or is that too much mixing? >> > > > >> > >>> >> > >>> >> > > > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek < >> > > > >> > [email protected] >> > > > >> > >>>: >> > > > >> > >>> >> > >>> >> > > > >> > >>> >> > >>>> I didn't look at the example either. >> > > > >> > >>> >> > >>>> >> > > > >> > >>> >> > >>>> Addings collections is easy, it's just that we can >> > > > either >> > > > >> > have >> > > > >> > >>> >> > >>>> Collections or the Option, not both. >> > > > >> > >>> >> > >>>> >> > > > >> > >>> >> > >>>> For the coding style I followed this: >> > > > >> > >>> >> > >>>> >> > > > >> > >>> >> > >> > > > >> > >> >> > > > >> > >> > > > >> >> > > > >> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide >> > > > >> > >>> >> , >> > > > >> > >>> >> > >>>> which itself is based on this: >> > > > >> > http://docs.scala-lang.org/style/ >> > > > >> > >> . >> > > > >> > >>> >> It >> > > > >> > >>> >> > >>>> is different from the Java Code Guidelines we have >> > in >> > > > >> place, >> > > > >> > >> yes. >> > > > >> > >>> >> > >>>> >> > > > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske < >> > > > >> > >> [email protected] >> > > > >> > >>> >> > >> > > > >> > >>> >> > >>>> wrote: >> > > > >> > >>> >> > >>>> > I haven't looked at the LineRank example in >> > detail, >> > > > but >> > > > >> if >> > > > >> > you >> > > > >> > >>> >> > think that >> > > > >> > >>> >> > >>>> > it adds something new to the examples >> collection, >> > we >> > > > can >> > > > >> > >> certainly >> > > > >> > >>> >> > port >> > > > >> > >>> >> > >>>> it >> > > > >> > >>> >> > >>>> > also to Java. >> > > > >> > >>> >> > >>>> > I think the Option and Collector return types >> are >> > > > >> > sufficient >> > > > >> > >> right >> > > > >> > >>> >> > now >> > > > >> > >>> >> > >>>> but >> > > > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-) >> > > > >> > >>> >> > >>>> > >> > > > >> > >>> >> > >>>> > Great that the Scala primitives are working! >> Also >> > > > thanks >> > > > >> > for >> > > > >> > >>> >> adding >> > > > >> > >>> >> > >>>> > genSequence and adapting my examples. >> > > > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala >> files >> > or >> > > > do >> > > > >> we >> > > > >> > >> have a >> > > > >> > >>> >> > >>>> different >> > > > >> > >>> >> > >>>> > there? >> > > > >> > >>> >> > >>>> > >> > > > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek < >> > > > >> > >> [email protected] >> > > > >> > >>> >> >: >> > > > >> > >>> >> > >>>> > >> > > > >> > >>> >> > >>>> >> What about the LineRank example? We had that in >> > > Scala >> > > > >> but >> > > > >> > >> never >> > > > >> > >>> >> > had a >> > > > >> > >>> >> > >>>> >> Java Example. >> > > > >> > >>> >> > >>>> >> >> > > > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha >> > Krettek < >> > > > >> > >>> >> > [email protected]> >> > > > >> > >>> >> > >>>> >> wrote: >> > > > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always >> just >> > > > >> copied >> > > > >> > the >> > > > >> > >> Java >> > > > >> > >>> >> > >>>> ITCase. >> > > > >> > >>> >> > >>>> >> > >> > > > >> > >>> >> > >>>> >> > The only examples that are missing now are >> > > > >> > >> LinearRegression and >> > > > >> > >>> >> > the >> > > > >> > >>> >> > >>>> >> > relational stuff. >> > > > >> > >>> >> > >>>> >> > >> > > > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian >> Hueske >> > < >> > > > >> > >>> >> > [email protected]> >> > > > >> > >>> >> > >>>> >> wrote: >> > > > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees >> > example. >> > > > >> > >>> >> > >>>> >> >> That was a preprocessing step for the >> > > > >> > >> TriangleEnumeration, and >> > > > >> > >>> >> > is now >> > > > >> > >>> >> > >>>> >> part >> > > > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example. >> > > > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. >> > As I >> > > > >> said >> > > > >> > >> before, >> > > > >> > >>> >> > I'd >> > > > >> > >>> >> > >>>> >> prefer to >> > > > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync. >> > > > >> > >>> >> > >>>> >> >> >> > > > >> > >>> >> > >>>> >> >> Cheers, Fabian >> > > > >> > >>> >> > >>>> >> >> >> > > > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek >> < >> > > > >> > >>> >> > [email protected]>: >> > > > >> > >>> >> > >>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again >> > > > fabian. >> > > > >> :D >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> Regarding the other stuff: >> > > > >> > >>> >> > >>>> >> >>> - There is a comment in DataSet.scala >> about >> > > > >> including >> > > > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the >> > > > >> > >> TypeInformation. >> > > > >> > >>> >> > >>>> >> >>> - I added generateSequence to >> > > > >> ExecutionEnvironment. >> > > > >> > >>> >> > >>>> >> >>> - It is possible to use Scala Primitives >> in >> > > > >> Array, I >> > > > >> > >> noticed >> > > > >> > >>> >> > it >> > > > >> > >>> >> > >>>> while >> > > > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an >> older >> > > > >> version >> > > > >> > of >> > > > >> > >> the >> > > > >> > >>> >> > code. >> > > > >> > >>> >> > >>>> >> >>> - Yes, using List and other Interfaces is >> > not >> > > > >> > possible, >> > > > >> > >> this >> > > > >> > >>> >> > is >> > > > >> > >>> >> > >>>> also >> > > > >> > >>> >> > >>>> >> >>> a restriction in the Java API. >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> What do you think about the interface of >> join >> > > and >> > > > >> > >> coGroup? >> > > > >> > >>> >> > Right >> > > > >> > >>> >> > >>>> now, >> > > > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an >> > > > Option >> > > > >> or >> > > > >> > the >> > > > >> > >>> >> > lambda >> > > > >> > >>> >> > >>>> with >> > > > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have >> > also >> > > > >> have a >> > > > >> > >> lambda >> > > > >> > >>> >> > that >> > > > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type >> erasure >> > > > this >> > > > >> has >> > > > >> > >> the >> > > > >> > >>> >> > same type >> > > > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't >> > use >> > > > it. >> > > > >> > >> There is >> > > > >> > >>> >> an >> > > > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a >> > > Collection, >> > > > >> so I >> > > > >> > >> could >> > > > >> > >>> >> > change >> > > > >> > >>> >> > >>>> it >> > > > >> > >>> >> > >>>> >> >>> without breaking the examples we have now. >> > What >> > > > do >> > > > >> you >> > > > >> > >> think? >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans, >> > > > >> > >>> >> ConnectedComponents, >> > > > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive, >> > > > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt, >> > > > >> > >>> >> > >>>> >> >>> PageRank >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> These are the examples people called dibs >> on: >> > > > >> > >>> >> > >>>> >> >>> - BatchGradientDescent (Márton) (Should >> be a >> > > > port >> > > > >> of >> > > > >> > >>> >> > >>>> LinearRegression >> > > > >> > >>> >> > >>>> >> >>> Example from Java) >> > > > >> > >>> >> > >>>> >> >>> - ComputeEdgeDegrees (Hermann) >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken): >> > > > >> > >>> >> > >>>> >> >>> - The relational Stuff >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan >> > Ewen < >> > > > >> > >>> >> > [email protected]> >> > > > >> > >>> >> > >>>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery >> > > > >> > >>> >> > >>>> >> >>> > >> > > > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha >> > > > >> Krettek < >> > > > >> > >>> >> > >>>> >> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> > wrote: >> > > > >> > >>> >> > >>>> >> >>> > >> > > > >> > >>> >> > >>>> >> >>> >> By the way, what was called >> > > > >> BatchGradientDescent in >> > > > >> > >> the >> > > > >> > >>> >> > Scala >> > > > >> > >>> >> > >>>> >> examples >> > > > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the >> > > > >> > LinearRegression >> > > > >> > >>> >> > Example from >> > > > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate >> examples >> > > > >> earlier. >> > > > >> > >>> >> > >>>> >> >>> >> >> > > > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. >> > Any >> > > > >> > thoughts >> > > > >> > >>> >> about >> > > > >> > >>> >> > >>>> removing >> > > > >> > >>> >> > >>>> >> >>> >> RelationalQuery? >> > > > >> > >>> >> > >>>> >> >>> >> >> > > > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, >> Aljoscha >> > > > >> Krettek >> > > > >> > < >> > > > >> > >>> >> > >>>> >> [email protected] >> > > > >> > >>> >> > >>>> >> >>> > >> > > > >> > >>> >> > >>>> >> >>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration >> > Examples, >> > > > >> thanks >> > > > >> > >>> >> Fabian. >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, >> > KMeans, >> > > > >> > >>> >> > ConnectedComponents, >> > > > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, >> TransitiveClosureNaive, >> > > > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> > These are the examples people called >> > dibs >> > > > on: >> > > > >> > >>> >> > >>>> >> >>> >> > - PageRank (Fabian) >> > > > >> > >>> >> > >>>> >> >>> >> > - BatchGradientDescent (Márton) >> > > > >> > >>> >> > >>>> >> >>> >> > - ComputeEdgeDegrees (Hermann) >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not >> > mistaken): >> > > > >> > >>> >> > >>>> >> >>> >> > - The relational Stuff >> > > > >> > >>> >> > >>>> >> >>> >> > - LinearRegression >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, >> > Aljoscha >> > > > >> > Krettek < >> > > > >> > >>> >> > >>>> >> >>> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a >> running >> > > > list >> > > > >> of >> > > > >> > >>> >> > >>>> ported/unported >> > > > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the >> > > java >> > > > >> > example >> > > > >> > >>> >> > package to >> > > > >> > >>> >> > >>>> >> >>> examples >> > > > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done. >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is >> > fine >> > > > as >> > > > >> it >> > > > >> > is. >> > > > >> > >>> >> Just >> > > > >> > >>> >> > >>>> because >> > > > >> > >>> >> > >>>> >> >>> Scala >> > > > >> > >>> >> > >>>> >> >>> >> >> enables functional programming >> doesn't >> > > mean >> > > > >> it's >> > > > >> > >> always >> > > > >> > >>> >> > the >> > > > >> > >>> >> > >>>> best >> > > > >> > >>> >> > >>>> >> >>> >> >> choice. :D >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, >> > KMeans, >> > > > >> > >>> >> > ConnectedComponents, >> > > > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, >> TransitiveClosureNaive >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called >> > dibs >> > > > on: >> > > > >> > >>> >> > >>>> >> >>> >> >> - TriangleEnumration and PageRank >> > > (Fabian) >> > > > >> > >>> >> > >>>> >> >>> >> >> - BatchGradientDescent (Márton) >> > > > >> > >>> >> > >>>> >> >>> >> >> - ComputeEdgeDegrees (Hermann) >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not >> > > mistaken): >> > > > >> > >>> >> > >>>> >> >>> >> >> - The relational Stuff >> > > > >> > >>> >> > >>>> >> >>> >> >> - LinearRegression >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> Cheers, >> > > > >> > >>> >> > >>>> >> >>> >> >> Aljoscha >> > > > >> > >>> >> > >>>> >> >>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, >> Kostas >> > > > >> Tzoumas >> > > > >> > < >> > > > >> > >>> >> > >>>> >> [email protected] >> > > > >> > >>> >> > >>>> >> >>> > >> > > > >> > >>> >> > >>>> >> >>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also >> added >> > a >> > > > >> > >> termination >> > > > >> > >>> >> > criterion >> > > > >> > >>> >> > >>>> >> in the >> > > > >> > >>> >> > >>>> >> >>> >> Java >> > > > >> > >>> >> > >>>> >> >>> >> >>> version: >> > > > >> > >>> >> > >>>> >> >>> >> >> > > > >> > >>> >> > >>>> >> > > > >> > >> >> > > https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example >> > > > >> > >>> >> > >>>> >> >>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination >> > > > >> criterion >> > > > >> > in >> > > > >> > >>> >> Scala >> > > > >> > >>> >> > more >> > > > >> > >>> >> > >>>> >> >>> >> functional? >> > > > >> > >>> >> > >>>> >> >>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package >> > name >> > > > is >> > > > >> > >>> >> > example.java but >> > > > >> > >>> >> > >>>> >> >>> >> examples.scala >> > > > >> > >>> >> > >>>> >> >>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>> Kostas >> > > > >> > >>> >> > >>>> >> >>> >> >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, >> Kostas >> > > > >> Tzoumas >> > > > >> > < >> > > > >> > >>> >> > >>>> >> [email protected] >> > > > >> > >>> >> > >>>> >> >>> > >> > > > >> > >>> >> > >>>> >> >>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and >> > > > >> PiEstimation >> > > > >> > >> (was not >> > > > >> > >>> >> > on >> > > > >> > >>> >> > >>>> your >> > > > >> > >>> >> > >>>> >> >>> list). >> > > > >> > >>> >> > >>>> >> >>> >> >>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the >> > relational >> > > > >> stuff >> > > > >> > I >> > > > >> > >> can >> > > > >> > >>> >> > take >> > > > >> > >>> >> > >>>> those >> > > > >> > >>> >> > >>>> >> as >> > > > >> > >>> >> > >>>> >> >>> >> well. >> > > > >> > >>> >> > >>>> >> >>> >> >>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the >> > > "RelationalQuery" >> > > > >> from >> > > > >> > >> both >> > > > >> > >>> >> > Scala and >> > > > >> > >>> >> > >>>> >> Java? >> > > > >> > >>> >> > >>>> >> >>> It >> > > > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of >> TPC-H >> > > Q3. >> > > > >> Does >> > > > >> > it >> > > > >> > >> add >> > > > >> > >>> >> > some >> > > > >> > >>> >> > >>>> >> teaching >> > > > >> > >>> >> > >>>> >> >>> >> value on >> > > > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3? >> > > > >> > >>> >> > >>>> >> >>> >> >>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>> Kostas >> > > > >> > >>> >> > >>>> >> >>> >> >>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, >> > > Aljoscha >> > > > >> > Krettek >> > > > >> > >> < >> > > > >> > >>> >> > >>>> >> >>> [email protected] >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an >> > > > ITCase. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, >> > > > KMeans, >> > > > >> > >>> >> > >>>> ConnectedComponents, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people >> called >> > > > dibs >> > > > >> on: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - TriangleEnumration and PageRank >> > > > >> (Fabian) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - BatchGradientDescent (Márton) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - ComputeEdgeDegrees (Hermann) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not >> > > > mistaken): >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - TransitiveClosure >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - The relational Stuff >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> - LinearRegression >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, >> > Kostas >> > > > >> > Tzoumas < >> > > > >> > >>> >> > >>>> >> >>> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > > > >> > >>> >> > >>>> >> >>> >> >> > > > >> > >>> >> > >>>> >> >>> >> > > > >> > >>> >> > >>>> >> >> > > > >> > >>> >> > >>>> >> > > > >> > >>> >> > >> > > > >> > >>> >> >> > > > >> > >> >> > > > >> > >> > > > >> >> > > > >> > > >> > >> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done? >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, >> > > > Aljoscha >> > > > >> > >> Krettek < >> > > > >> > >>> >> > >>>> >> >>> >> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents >> > > > Example >> > > > >> > from >> > > > >> > >>> >> Vasia. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, >> > > > Fabian >> > > > >> > >> Hueske < >> > > > >> > >>> >> > >>>> >> >>> [email protected] >> > > > >> > >>> >> > >>>> >> >>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks! >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 >> > > Aljoscha >> > > > >> > >> Krettek < >> > > > >> > >>> >> > >>>> >> >>> >> [email protected]>: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive >> decision. >> > > :D >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at >> KMeansData.java >> > > and >> > > > >> > >>> >> > KMeans.scala. I'm >> > > > >> > >>> >> > >>>> >> storing >> > > > >> > >>> >> > >>>> >> >>> >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object >> > > arrays >> > > > >> and >> > > > >> > >> then >> > > > >> > >>> >> > >>>> converting >> > > > >> > >>> >> > >>>> >> it to >> > > > >> > >>> >> > >>>> >> >>> >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala >> > objects. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to >> > > > >> equalTo to >> > > > >> > >> make >> > > > >> > >>> >> it >> > > > >> > >>> >> > >>>> >> consistent >> > > > >> > >>> >> > >>>> >> >>> >> with the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and >> coGroup). >> > > > There >> > > > >> is >> > > > >> > no >> > > > >> > >> need >> > > > >> > >>> >> > for a >> > > > >> > >>> >> > >>>> >> >>> keyword, >> > > > >> > >>> >> > >>>> >> >>> >> you >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> left.join(right).where(0).equalTo(1) { >> > > > >> > >> (le, >> > > > >> > >>> >> re) >> > > > >> > >>> >> > => >> > > > >> > >>> >> > >>>> new >> > > > >> > >>> >> > >>>> >> >>> >> MyResult(le, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> } >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 >> > PM, >> > > > >> Fabian >> > > > >> > >> Hueske >> > > > >> > >>> >> < >> > > > >> > >>> >> > >>>> >> >>> >> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet >> > issue, >> > > I >> > > > >> also >> > > > >> > >> found >> > > > >> > >>> >> an >> > > > >> > >>> >> > >>>> >> >>> inconsistency >> > > > >> > >>> >> > >>>> >> >>> >> with >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done >> > as: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> ds1.join(ds2).where(...).equalTo(...) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala >> > > this >> > > > >> is: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > ds1.join(d2).where(...).isEqualTo(...) >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be >> > renamed >> > > to >> > > > >> > >> equalTo(), >> > > > >> > >>> >> > IMO. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and >> > > coGroup?) >> > > > >> > lacks >> > > > >> > >> the >> > > > >> > >>> >> > with() >> > > > >> > >>> >> > >>>> >> method >> > > > >> > >>> >> > >>>> >> >>> >> because >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with" >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should >> > be >> > > > >> offer >> > > > >> > >>> >> something >> > > > >> > >>> >> > >>>> similar >> > > > >> > >>> >> > >>>> >> for >> > > > >> > >>> >> > >>>> >> >>> >> Scala >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, >> > right)? >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 >> > > > Stephan >> > > > >> > Ewen >> > > > >> > >> < >> > > > >> > >>> >> > >>>> >> [email protected] >> > > > >> > >>> >> > >>>> >> >>> >: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, >> > > Object[][] >> > > > >> would >> > > > >> > >> work >> > > > >> > >>> >> as >> > > > >> > >>> >> > well. >> > > > >> > >>> >> > >>>> >> That >> > > > >> > >>> >> > >>>> >> >>> is a >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a >> Tuple. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could >> > be >> > > > >> stored >> > > > >> > as >> > > > >> > >> Java >> > > > >> > >>> >> > or >> > > > >> > >>> >> > >>>> Scala >> > > > >> > >>> >> > >>>> >> >>> Tuples, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert >> > > > between >> > > > >> > the >> > > > >> > >> two. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at >> > 10:55 >> > > > AM, >> > > > >> > Fabian >> > > > >> > >>> >> > Hueske >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the >> same >> > > > >> > problem... >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings >> and >> > > > >> parsing >> > > > >> > >> them, >> > > > >> > >>> >> but >> > > > >> > >>> >> > >>>> using >> > > > >> > >>> >> > >>>> >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based >> on >> > a >> > > > >> > >>> >> FileInputFormat. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to >> parse >> > > the >> > > > >> > Strings >> > > > >> > >>> >> > >>>> manually... >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 >> > GMT+02:00 >> > > > >> > Aljoscha >> > > > >> > >>> >> Krettek >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[email protected]>: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. >> > Maybe >> > > we >> > > > >> > should >> > > > >> > >> just >> > > > >> > >>> >> > change >> > > > >> > >>> >> > >>>> >> all >> > > > >> > >>> >> > >>>> >> >>> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> input >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and >> use >> > > CSV >> > > > >> > input >> > > > >> > >>> >> > formats in >> > > > >> > >>> >> > >>>> all >> > > > >> > >>> >> > >>>> >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> What >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think? >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 >> at >> > > 7:46 >> > > > >> AM, >> > > > >> > >>> >> Aljoscha >> > > > >> > >>> >> > >>>> Krettek >> > > > >> > >>> >> > >>>> >> < >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [email protected]> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's >> unfortunate >> > > that >> > > > >> the >> > > > >> > >> data >> > > > >> > >>> >> > types are >> > > > >> > >>> >> > >>>> >> >>> >> incompatible. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what >> > you >> > > > >> > proposed: >> > > > >> > >>> >> move >> > > > >> > >>> >> > the >> > > > >> > >>> >> > >>>> >> data to >> > > > >> > >>> >> > >>>> >> >>> a >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> field >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the >> > > > >> > >>> >> > getDefaultEdgeDataSet() >> > > > >> > >>> >> > >>>> >> method in >> > > > >> > >>> >> > >>>> >> >>> >> Scala. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying >> > would >> > > > >> > >> duplicate the >> > > > >> > >>> >> > data >> > > > >> > >>> >> > >>>> and >> > > > >> > >>> >> > >>>> >> >>> make it >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> for >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync >> in >> > > the >> > > > >> Java >> > > > >> > and >> > > > >> > >>> >> Scala >> > > > >> > >>> >> > >>>> >> versions. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others >> > > think? >> > > > >> This >> > > > >> > >> will >> > > > >> > >>> >> > probably >> > > > >> > >>> >> > >>>> >> occur >> > > > >> > >>> >> > >>>> >> >>> in >> > > > >> > >>> >> > >>>> >> >>> >> all >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 >> > at >> > > > >> 10:04 >> > > > >> > PM, >> > > > >> > >>> >> > Vasiliki >> > > > >> > >>> >> > >>>> >> Kalavri >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > < >> > > > [email protected] >> > > > >> > >> > > > >> > >> wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the >> > > > >> Connected >> > > > >> > >>> >> Components >> > > > >> > >>> >> > >>>> >> example, >> > > > >> > >>> >> > >>>> >> >>> but >> > > > >> > >>> >> > >>>> >> >>> >> I am >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example >> > > input >> > > > >> data >> > > > >> > >> from >> > > > >> > >>> >> > >>>> >> java-examples. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the >> > > > >> > ConnectedComponentsData >> > > > >> > >>> >> class, >> > > > >> > >>> >> > the >> > > > >> > >>> >> > >>>> >> vertices >> > > > >> > >>> >> > >>>> >> >>> >> and >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> data >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the >> > > methods >> > > > >> > >>> >> > >>>> >> getDefaultVertexDataSet() >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and >> > > > >> getDefaultEdgeDataSet(), >> > > > >> > >> which >> > > > >> > >>> >> > take >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an >> > > > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment >> > > > >> > >>> >> > >>>> >> >>> as >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to >> > provide >> > > > >> public >> > > > >> > >> static >> > > > >> > >>> >> > fields >> > > > >> > >>> >> > >>>> >> (like >> > > > >> > >>> >> > >>>> >> >>> in >> > > > >> > >>> >> > >>>> >> >>> >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this >> > > > >> introduces a >> > > > >> > >>> >> > conversion >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from >> > > > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to >> > > > >> > >>> >> > >>>> >> >>> Scala >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to >> > > > >> scala.Long >> > > > >> > >> and I >> > > > >> > >>> >> > guess >> > > > >> > >>> >> > >>>> this >> > > > >> > >>> >> > >>>> >> is >> > > > >> > >>> >> > >>>> >> >>> an >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example >> (?). >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of >> > > > >> course, to >> > > > >> > >> copy >> > > > >> > >>> >> the >> > > > >> > >>> >> > >>>> example >> > > > >> > >>> >> > >>>> >> >>> data >> > > > >> > >>> >> > >>>> >> >>> >> in >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing >> > something >> > > > >> here? >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks! >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers, >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V. >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September >> 2014 >> > > > 15:52, >> > > > >> > >> Aljoscha >> > > > >> > >>> >> > >>>> Krettek < >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> [email protected] >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I >> updated >> > my >> > > > >> repo: >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >> > > > >> > >>> >> > >> > > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a >> > > working >> > > > >> > >> WordCount >> > > > >> > >>> >> > example. >> > > > >> > >>> >> > >>>> >> It's >> > > > >> > >>> >> > >>>> >> >>> >> pretty >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example >> > with >> > > > >> some >> > > > >> > >> fixups >> > > > >> > >>> >> > for the >> > > > >> > >>> >> > >>>> >> syntax >> > > > >> > >>> >> > >>>> >> >>> and >> > > > >> > >>> >> > >>>> >> >>> >> >>
