[ANNOUNCE] Apache Bahir 2.3.3 Released

Bahir extensions in your application please visit our documentation page https://bahir.apache.org/docs/spark/overview/ The Apache Bahir PMC -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com

[ANNOUNCE] Apache Bahir 2.2.3 Released

Bahir extensions in your application please visit our documentation page https://bahir.apache.org/docs/spark/overview/ The Apache Bahir PMC -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com

[ANNOUNCE] Apache Bahir 2.3.1 Released

Bahir extensions in your application please visit our documentation page http://bahir.apache.org/docs/spark/overview/ The Apache Bahir PMC -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com

[ANNOUNCE] Apache Bahir 2.3.2 Released

Bahir extensions in your application please visit our documentation page http://bahir.apache.org/docs/spark/overview/ The Apache Bahir PMC -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com

[ANNOUNCE] Apache Bahir 2.3.0 Released

Bahir extensions in your application please visit our documentation page http://bahir.apache.org/docs/spark/overview/ The Apache Bahir PMC -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com

[ANNOUNCE] Apache Toree 0.3.0-incubating Released

page: https://toree.incubator.apache.org/docs/current/user/quick-start/ -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/ - To unsubscribe e-mail: user-unsubscr

[ANNOUNCE] Apache Bahir 2.2.2 Released

-- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

[ANNOUNCE] Apache Bahir 2.1.3 Released

-- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

[ANNOUNCE] Apache Toree 0.2.0-incubating Released

information about Apache Toree and go download the latest release go to: https://toree.incubator.apache.org/ For more information on how to use Apache Toree please visit our documentation page: https://toree.incubator.apache.org/docs/current/user/quick-start/ -- Luciano Resende http

[ANNOUNCE] Apache Bahir 2.2.1 Released

-- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

[ANNOUNCE] Apache Bahir 2.1.2 Released

-- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

[ANNOUNCE] Apache Bahir 2.2.0 Released

Apache Bahir provides extensions to multiple distributed analytic platforms, extending their reach with a diversity of streaming connectors and SQL data sources. The Apache Bahir community is pleased to announce the release of Apache Bahir 2.2.0 which provides the following extensions for Apache

[ANNOUNCE] Apache Bahir 2.1.1 Released

- ZeroMQ Streaming For more information about Apache Bahir and to download the latest release go to: http://bahir.apache.org The Apache Bahir streaming connectors are also available at: https://spark-packages.org/?q=bahir -- Luciano Resende http://twitter.com/lresende1975 http

Re: Old version of Spark [v1.2.0]

;> > >> > >> > >> > >> > >> Regards, > >> _ > >> Md. Rezaul Karim, BSc, MSc > >> PhD Researcher, INSIGHT Centre for Data Analytics > >> National University of Ireland, Galway > >

Re: Regarding Connection Problem

en you can see an example here https://github.com/lresende/bahir-iot-demo -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Create a Column expression from a String

Are you looking for UDFs? https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-sql-udfs.html On Sun, Nov 20, 2016 at 3:12 AM Stuart White wrote: > I'd like to allow for runtime-configured Column expressions in my > Spark SQL application. For

[ANNOUNCE] Apache SystemML 0.11.0-incubating released

he ASF. -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Akka Stream as the source for Spark Streaming. Please advice...

If you are interested in Akka streaming, it is being maintained in Apache Bahir. For Akka there isn't a structured streaming version yet, but we would be interested in collaborating in the structured streaming version for sure. On Thu, Nov 10, 2016 at 8:46 AM shyla deshpande

[ANNOUNCE] Apache Bahir 2.0.1

Apache Bahir and to download the release: http://bahir.apache.org -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Spark 1.2

ge drop down > > -- > Best Regards, > Ayan Guha > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: JSON Arrays and Spark

|-- _corrupt_record: string (nullable = true) > > Both are legit JSON structures… Do you think that #2 is a bug? > > jg > > > > > > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Scala Vs Python

low level apis written in java/scala. Take a look at Apache SystemML http://systemml.apache.org/ for more details. -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Spark to Kafka communication encrypted ?

.com> wrote: >> > I can't find in Spark 1.6.2's docs in how to turn encryption on for >> Spark to >> > Kafka communication ... I think that the Spark docs only tells you how >> to >> > turn on encryption for inter Spark node communications .. Am I wrong ? >> > >> > Thanks. >> > >> > -- >> > >> > -eric ho >> > >> >> - >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> >> >> >> > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

[ANNOUNCE] Apache Bahir 2.0.0

The Apache Bahir PMC is pleased to announce the release of Apache Bahir 2.0.0 which is our first major release and provides the following extensions for Apache Spark 2.0.0 : Akka Streaming MQTT Streaming and Structured Streaming Twitter Streaming ZeroMQ Streaming For more information about

Re: Help testing the Spark Extensions for the Apache Bahir 2.0.0 release

://lresende.blogspot.gr/2016/08/getting-started-with-apache-bahir-mqtt_4.html On Sun, Aug 7, 2016 at 11:24 AM, Sivakumaran S <siva.kuma...@me.com> wrote: > Hi, > > How can I help? > > regards, > > Sivakumaran S > > On 06-Aug-2016, at 6:18 PM, Luciano Resende <luckbr1..

Help testing the Spark Extensions for the Apache Bahir 2.0.0 release

Twitter DStream connector ZeroMQ DStream connector MQTT Structured Streaming Thanks in advance -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: 2.0.0 packages for twitter streaming, flume and other connectors

On Thursday, August 4, 2016, Sean Owen wrote: > You're looking for http://bahir.apache.org/ > > And we should have a release supporting Spark 2.0 in a few days... Thanks -- Luciano -- Sent from my Mobile device

Re: Integration tests for Spark Streaming

ark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

[ANNOUNCE] Apache SystemML 0.10.0-incubating released

The Apache SystemML team is pleased to announce the release of Apache SystemML version 0.10.0-incubating. Apache SystemML provides declarative large-scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic generation of hybrid runtime plans ranging from

Re: Suggestions on Lambda Architecture in Spark

This might be useful: https://spark-summit.org/east-2016/events/lambda-at-weather-scale/ On Monday, June 13, 2016, KhajaAsmath Mohammed wrote: > Hi, > > In my current project, we are planning to implement POC for lambda > architecture using spark streaming. My use case

Re: sbt for Spark build with Scala 2.11

SHOT. > Does this mean, the only option is to put all the required jars in the lib > folder (unmanaged dependencies)? > > Regards, > Raghava. > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Spark build failure with com.oracle:ojdbc6:jar:11.2.0.1.0

RA, however, I do see > a fix required to patch pom.xml to apply the correct groupId and version # > for ojdbc6 driver. > > > Thoughts? > > > > Get Oracle JDBC drivers and UCP from Oracle Maven ... > <https://blogs.oracle.com/dev2dev/entry/how_to_get_oracle_jdbc> &

Re: Spark build failure with com.oracle:ojdbc6:jar:11.2.0.1.0

] >>> >>> >>> [INFO] BUILD FAILURE >>> >>> [INFO] >>> ------------ >>> >>> [INF

Re: Spark support for Complex Event Processing (CEP)

ew?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Spark on Mobile platforms

l: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Setting up spark to run on two nodes

There is also sbin/star-all.sh and sbin/stop-all.sh which enables you to star/stop master and workers all together On Sunday, March 20, 2016, Akhil Das wrote: > You can simply execute the sbin/start-slaves.sh file to start up all slave > processes. Just make sure you

Re: Flume with Spark Streaming Sink

impact3.indigo.co.il): java.lang.NoClassDefFoundError: > org/apache/spark/streaming/flume/sink/SparkFlumeProtocol$Callback > at > org.apache.spark.streaming.flume.FlumePollingReceiver$$anonfun$onStart$1.apply(FlumePollingInputDStream.scala:84) > > > What deps am I

Re: Spark Integration Patterns

-- >>> View this message in context: Spark Integration Patterns >>> <http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Integration-Patterns-tp26354.html> >>> Sent from the Apache Spark User List mailing list archive >>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >>> >> >> -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

[ANNOUNCE] Apache SystemML 0.9.0-incubating released

The Apache SystemML team is pleased to announce the release of Apache SystemML version 0.9.0-incubating. This is the first release as an Apache project. Apache SystemML provides declarative large-scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic

Re: Imported CSV file content isn't identical to the original file

rrowed down to upgrade my spark to 1.6.0, to see if this >> issue is gone. >> >> — >> Cheers, >> Todd Leo >> >> >> ​ >> On Mon, Feb 8, 2016 at 2:12 PM Luciano Resende <luckbr1...@gmail.com> >> wrote: >> >>> I tried in both 1.5.

Re: Imported CSV file content isn't identical to the original file

-51.png] > > and even the boundary of the table wasn’t shown correctly. I also used the > other way to read csv file, by sc.textFile(...).map(_.split(" ")) and > sqlContext.createDataFrame, and the result is the same. Can someone point > me out where I did it wrong? > > — > BR, > Todd Leo > ​ > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: How to edit/delete a message posted in Apache Spark User List?

> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: pyspark + kafka + streaming = NoSuchMethodError

ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:259) > at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) > at py4j.commands.CallCommand.execute(CallCommand.java:79) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Thread.java:745) > > Am I missing something? > > Thanks in advance > Chris M. > > > > > > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: Rule Engine for Spark

t may help. > > On Wed, Nov 4, 2015 at 1:42 AM, Cassa L <lcas...@gmail.com> wrote: > >> Hi, >> Has anyone used rule engine with spark streaming? I have a case where >> data is streaming from Kafka and I need to apply some rules on it (instead >> of hard co

Re: SF Spark Office Hours Experiment - Friday Afternoon

gt; > Holden :) > -- > Cell : 425-233-8271 > Twitter: https://twitter.com/holdenkarau > +1, Very good initiative !!! Will try to attend some as time permits. -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: spark streaming 1.51. uses very old version of twitter4j

easily by just including the > following lines in your pom.xml. > > > >org.twitter4j > twitter4j-core >[4.0,) > >... > > > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: SparkR - calling as.vector() with rdd dataframe causes error

when I call a clustering > algorithm like amap::hcluster(), I get an error from as.vector() that the > rdd cannot be coerced into a vector. > > On Fri, Sep 18, 2015 at 12:33 PM, Luciano Resende <luckbr1...@gmail.com> > wrote: > >> I see the thread with all the responses

Re: SparkR - calling as.vector() with rdd dataframe causes error

be >> doing something else to coerce the object into a vector? >> >> Thanks, >> Ellen >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/SparkR-calling-as-vector-with-rdd-dataframe-causes-error-tp24717.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional >> commands, e-mail: user-h...@spark.apache.org >> >> > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/

Re: SparkR - calling as.vector() with rdd dataframe causes error

ata) : > no method for coercing this S4 class to a vector > > > I get the same error if I try to directly call as.vector(rdd) as well. > > Is there a reason why this works for localDF and not rdd? Should I be > doing something else to coerce the object into a vector? > > Thanks, > Ellen > > > -- Luciano Resende http://people.apache.org/~lresende http://twitter.com/lresende1975 http://lresende.blogspot.com/