Re: how to get rid of duplicate rows group by in DataStream

2016-08-24 Thread subash basnet
gt;(val1.f0, val1.f1, val1.f2, val1.f3 + val2.f3); } } Regards, Subash Basnet On Mon, Aug 22, 2016 at 6:34 PM, Kostas Kloudas <k.klou...@data-artisans.com > wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (k.klou...@data-artis

how to get rid of duplicate rows group by in DataStream

2016-08-22 Thread subash basnet
istinct rows with final count: Needed Output: (1,3) (2,2).. What could be the way to achieve this. Regards, Subash Basnet

counting elements in datastream

2016-08-18 Thread subash basnet
Hello all, If anyone had idea, what could be the probable way to count the elements of a current instance of the datastream. Is it possible? DataStream<Tuple2<Point, Grid>> pointsWithGridCoordinates; Regards, Subash Basnet

Re: Sorting in datastream

2016-08-17 Thread subash basnet
ething infinite ;-) > That's why the operation does not exist on DataStream. > > Stephan > > > On Wed, Aug 17, 2016 at 6:22 PM, subash basnet <yasub...@gmail.com> wrote: > >> Hello all, >> >> I found the *sortPartition()* function in dataset for ordering the >

Sorting in datastream

2016-08-17 Thread subash basnet
ods to sort the elements in datastream. DataStream<Tuple2<Integer, String>> data; DataStream<Tuple2<Integer, String>> partitionedData =data.?? What could be the way to achieve sorting in datastream elements. Best Regards, Subash Basnet

counting elements in datastream

2016-08-17 Thread subash basnet
Hello all, There is *count()* function in database to count the number of elements in the dataset. But it's not there in case of datastream. What could be the way to count the number of elements in case of datastream. Best Regards, Subash Basnet

applying where after group by in dataset

2016-08-17 Thread subash basnet
Hello all, In the following dataset: DataSet<Tuple2<Point, Boolean>> distancePoints; I wanted to count the number of *distancePoints* where boolean value is either true of false. distancePoints.groupBy(1). didn't find how to apply there 'where' clause here. Best Regards, Subash Basnet

DataStreamUtils conversion problem, showing varied results for same code

2016-07-21 Thread subash basnet
lusteredPoints* have the same centroid id '-1' which should have been rather timestamp as shown in case 2. How could be the solution to this issue. Best Regards, Subash Basnet

Re: DataStreamUtils not working properly

2016-07-20 Thread subash basnet
all of them. And as exception occurs and program ends, how will I be able to use this *testCentroids* in operations that I need to perform in the later stage of the program. Regards, Subash Basnet On Wed, Jul 20, 2016 at 4:00 PM, Maximilian Michels <m...@apache.org> wrote: > Everyth

Re: DataStreamUtils not working properly

2016-07-20 Thread subash basnet
s[1] = value.getHigh(); columns[2] = value.getLow(); columns[3] = value.getOpen(); columns[4] = (double) value.getVolume(); return (new Tuple2<String, Double[]>(value.getId(), columns)); } }); Regards, Subash Basnet On Wed, Jul 20, 2016 at 12:20 PM, Maximilian Michels <m...@apache.org&

Understanding iteration error message

2016-07-20 Thread subash basnet
ot able to fully understand the error message. Could you explain it more in depth the error message in relation to above code. Best Regards, Subash Basnet

Re: DataStreamUtils not working properly

2016-07-19 Thread subash basnet
the collection of datastream via DataStreamUtils is giving empty result. Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 4:52 PM, Till Rohrmann <trohrm...@apache.org> wrote: > It depends if you have a log4j.properties file specified in your > classpath. If you see log output on the console, th

Re: DataStreamUtils not working properly

2016-07-19 Thread subash basnet
Regards, Subash Basnet On Tue, Jul 19, 2016 at 2:49 PM, Till Rohrmann <trohrm...@apache.org> wrote: > Have you checked your logs whether they contain some problems? In general > it is not recommended collecting the streaming result back to your client. > It might al

Re: DataStreamUtils not working properly

2016-07-19 Thread subash basnet
(Mon Jul 18 17:43:58 CEST 2016) (Mon Jul 18 17:42:59 CEST 2016) (Mon Jul 18 17:34:01 CEST 2016) (Mon Jul 18 17:52:00 CEST 2016) (Mon Jul 18 17:40:58 CEST 2016) for centroidId.print(), but no output for System.out.println(c); Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 10:48 AM, su

DataStreamUtils not working properly

2016-07-19 Thread subash basnet
119.535 119.54 119.445 119.455 152900.0 But the next *System.out.println(c) *within the for loop prints nothing. What could be the problem. My maven has following configuration for dataStreamUtils: org.apache.flink flink-streaming-contrib_2.10 ${flink.version} Best Regards, Subash Basnet

Unable to get the value of datatype in datastream

2016-07-18 Thread subash basnet
ion as intially the centroids is null, hence the execution never comes out of *flatMap1*. It would be great if you could suggest what could be the probable problem or solution to the case. Best Regards, Subash Basnet

Re: how to get rid of null pointer exception in collection in DataStream

2016-07-18 Thread subash basnet
Hello Robert, Yup thank you, I used array in place of collection to rid of that error. Best Regards, Subash Basnet On Thu, Jul 14, 2016 at 4:03 PM, Robert Metzger <rmetz...@apache.org> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for

how to get rid of null pointer exception in collection in DataStream

2016-07-13 Thread subash basnet
*centroids*.add(value); } } The instatation as below is not allowed. So it always throws null pointer exception. private Collection *centroids = * new Collection(); Best Regards, Subash Basnet

Streaming Exception error message Explanation

2016-07-06 Thread subash basnet
bstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.(Socket.java:434) at java.net.Socket.(Socket.java:244) at org.apache.flink.contrib.streaming.CollectSink.initializeConnection(CollectSink.java:69) Best Regards, Subash Basnet

Broadcast and read broadcast variable in DataStream

2016-05-16 Thread subash basnet
StreamExecutionEnvironment.java:1170) at org.apache.flink.contrib.streaming.DataStreamUtils$CallExecute.run(DataStreamUtils.java:80) Could you please suggest me possible cause and solution to this exception, as I am not able to see any other option beside to use global variable in absence of broadcast of variable in datastream. Best Regards, Subash Basnet

Re: Unable to understand datastream error message

2016-05-14 Thread subash basnet
am.java:75) at wikiedits.StockAnalysis.main(StockAnalysis.java:64) Best Regards, Subash Basnet On Sat, May 14, 2016 at 4:26 PM, subash basnet <yasub...@gmail.com> wrote: > Hello Aljoscha, > > Below is the shorted version of StockAnalysis class which is a datastream > adapation of th

Re: Unable to understand datastream error message

2016-05-14 Thread subash basnet
48286_rand=1488102128_source=stf_medium=email_campaign=ANNO_CLEANUP_ADD_content=001> > > Could you please post your code. > > On Sat, 7 May 2016 at 19:16 subash basnet <yasub...@gmail.com> wrote: > >> Hello all, >> >> I am getting the below error on execut

Unable to understand datastream error message

2016-05-07 Thread subash basnet
;, e); } } I am not able to understand what to infer from this error message so that I could solve it. Best Regards, Subash Basnet

Re: how to convert datastream to collection

2016-05-04 Thread subash basnet
as given in maven repository: org.apache.flink flink-streaming-contrib 0.10.2 Best Regards, Subash Basnet On Wed, May 4, 2016 at 12:12 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > I think you have to explicitly import contrib packages as a dependency in > Maven/SBT. >

Re: how to convert datastream to collection

2016-05-04 Thread subash basnet
Hello there, Thank you! But I couldn't find DataStreamUtils. Where is it located? I am using *1.0.0*, The given, org.apache.flink.*contrib*.streaming.DataStreamUtils doesn't work as there is no *contrib *package within org.apache.flink. Best Regards, Subash Basnet On Tue, May 3, 2016 at 4:35

Re: How to perform Broadcast and groupBy in DataStream like DataSet

2016-05-03 Thread subash basnet
MapFunctions<>{ private Collection *centroids*; public void open(Configuration parameters) throws Exception { this.*centroids* = getRuntimeContext().getBroadcastVariable("centroids"); } for (Centroid cent : *centroids*) { } } Best Regards, Subash Basnet On Tue, May 3, 2016 at

How to perform Broadcast and groupBy in DataStream like DataSet

2016-05-03 Thread subash basnet
ccumulator()) .map(new CentroidAverager()); DataStream newCentroids = points.map(new SelectNearestCenter()).??? Best Regards, Subash Basnet

first() function in DataStream

2016-05-02 Thread subash basnet
Hello all, In DataSet *first(n)* function can be called to get 'n' no. of elements in the DataSet, how could similar operations be done in DataStream to get 'n' no. of elements from the current DataStream. Best Regards, Subash Basnet

adding source not serializable exception in streaming implementation

2016-04-19 Thread subash basnet
utputStream.java:348) at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:300) at org.apache.flink.api.java.ClosureCleaner.ensureSerializable(ClosureCleaner.java:97) ... 6 more I have attached Stock.java which is just a model with getters and setters. Not sure what am I doing wrong. Best Regards, Subash Basnet packag

Getting elements from DataStream

2016-04-11 Thread subash basnet
Regards, Subash Basnet

Re: Unable to run the batch examples after running stream examples

2016-03-21 Thread subash basnet
h it's working when I used the original pom of batch, but I am confused why the modified one is not working. Best Regards, Subash Basnet On Mon, Mar 21, 2016 at 5:14 PM, Chesnay Schepler <ches...@apache.org> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eli

Unable to run the batch examples after running stream examples

2016-03-21 Thread subash basnet
ation.main(PiEstimation.java:63) Previously before running the Kafka examples, the batch examples were working fine, now not. Now only the streaming examples are working. Best Regards, Subash Basnet

Re: Unable to Read from Kafka [ zookeeper.connect error]

2016-03-19 Thread subash basnet
Hello Stefano, Thank you, it's working :). Regards, Subash Basnet On Wed, Mar 16, 2016 at 3:24 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Hi Subash, > > you just have to add the following parameter: --zookeeper.connect > localhost:2181 > Let us kn

Unable to Read from Kafka [ zookeeper.connect error]

2016-03-19 Thread subash basnet
eeper.connect', or what am I doing wrong? Best Regards, Subash Basnet

how to convert DataStream to DataSet

2016-03-15 Thread subash basnet
mOperator<Tuple1> to DataSet<Tuple1>* Best Regards, Subash Basnet

Re: Retrieve elements from the Dataset without using collect

2016-03-06 Thread subash basnet
Hello Konstantin, Yup thanks. Best Regards, Subash Basnet On Sun, Mar 6, 2016 at 7:20 PM, Konstantin Knauf < konstantin.kn...@tngtech.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (konstantin.kn...@tngtech.com

Retrieve elements from the Dataset without using collect

2016-03-06 Thread subash basnet
. And, what would be the way to retrieve individual elements of DataSet without using list via collect? Best Regards, Subash Basnet

Re: Multi-dimensional[more than 2] input for KMeans Clustering inApache flink

2016-03-01 Thread subash basnet
Hello Fabian, Thanks! Is KMeans only the clustering implementation currently existing in flink. Best Regards, Subash Basnet On Tue, Mar 1, 2016 at 5:22 PM, Fabian Hueske <fhue...@gmail.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible

Multi-dimensional[more than 2] input for KMeans Clustering in Apache flink

2016-03-01 Thread subash basnet
dimensional data as input implemented in flink? Best Regards, Subash Basnet

Re: How to convert List to flink DataSet

2016-02-12 Thread subash basnet
anner. Best Regards, Subash Basnet On Wed, Feb 10, 2016 at 6:33 PM, Fabian Hueske <fhue...@gmail.com> wrote: > I would try to do the outlier compuation with the DataSet API instead of > fetching the results to the client with collect(). > If you do that, you can directly use writeAsCsv be

Re: Master Thesis [Apache-flink paper references]

2016-02-12 Thread subash basnet
Hello Matthias, Thank you very much :) Best Regards, Subash Basnet On Fri, Feb 12, 2016 at 8:22 PM, Matthias J. Sax <mj...@apache.org> wrote: > You might want to check out the Stratosphere project web site: > http://stratosphere.eu/project/publications/ > > -Matthias > >

Re: How to convert List to flink DataSet

2016-02-10 Thread subash basnet
write via flink web submission client. Best Regards, Subash Basnet On Wed, Feb 10, 2016 at 1:58 PM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Subash, > > I would not fetch the data to the client, do the computation there, and > send it back, just for the purpose of writ

Re: How to convert List to flink DataSet

2016-02-10 Thread subash basnet
nters, points file are located. I am only able to print it to the console via *fElements.print();* Does it have something to do with *env.exectue("")*, which must be set somewhere in the previous case but not in my case. Best Regards, Subash Basnet On Tue, Feb 9, 2016 at

How to convert List to flink DataSet

2016-02-09 Thread subash basnet
nts, so that I could use fElements.writeAsCsv(outputPath, "\n"," "); Best Regards, Subash Basnet

Re: Not able to import MBoxParser in MailCount.java

2016-02-01 Thread subash basnet
ink-training/devSetup/handsOn.html#clone-and-build-the-flink-training-exercises-project > > Hope I helped. > > Best, > Stefano > > On Mon, Feb 1, 2016 at 12:03 PM, subash basnet <yasub...@gmail.com> wrote: > >> Hello, >> >> I copied the plugin configurat