subject:"Re\: Storm benchmarks"

Re: Storm benchmarks

2016-11-06 Thread Walid Aljoby

Dear Dominik,
Thank you for details. 
Regards,--Walid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "u...@storm.apache.org" <u...@storm.apache.org>; "dev@storm.apache.org" 
<dev@storm.apache.org>
 Sent: Friday, November 4, 2016 7:05 PM
 Subject: Re: Storm benchmarks

1- What do you mean "able to control message size"? Is it max-pending-spout 
parameter?

By using for example Kafka as your source of information of the benchmark 
topology, you may produce i.e. control the size of messages in terms of bytes 
length. Why would you want to do this? Because there is a relation between 
certain performance characteristics such as throughput and message size. 

Is there any published benchmark like this old-one here:

As far up to my knowledge, no. However, we at the Web Information Systems 
research group of the Delft University of Technology are currently in the 
process of benchmarking several streaming engines (including Storm) part of an 
empirical research. If you’d like to here more about the insight so far 
gathered, feel free to email me.  

On 4 Nov 2016, at 10:02, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Thank you Dominik. I have two more points, please.1- What do you mean "able to 
control message size"? Is it max-pending-spout parameter?2- Is there any 
published benchmark like this old-one here: 
https://github.com/stormprocessor/storm-benchmark/commit/22bd17a81020ceef71ed73168ac89d3f8eaf61e2
Best Regards,Walid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "u...@storm.apache.org" <u...@storm.apache.org>; "dev@storm.apache.org" 
<dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:53 PM
 Subject: Re: Storm benchmarks

Well, this depends onto the aspects of the measurements. 
You may for example define a topology consisting of a spout, transformation 
bolt and sink that receives byte arrays from Kafka, transforms them and 
outputs. The nice thing is that you’d be able to control for the size of the 
messages. 
In addition, if you care about the performance in conjunction to stateful 
operations such as aggregations, your topology might look alike the for example 
WordCount topology.
Regards,Dominik

On 4 Nov 2016, at 09:50, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Dominik,
Many thanks for details. Actually I am looking for a set typologies for my test.
Thank you again,--RegardsWalid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: u...@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "dev@storm.apache.org" <dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:41 PM
 Subject: Re: Storm benchmarks

Hi Walid,
You may benchmark Storm’s performance in terms of throughput and end-to-end 
latency for example. In addition, the investigation could also include 
variances in the configurational settings, such as the parallelism, message 
size, intra-worker and inter-worker buffer size which some of them have a 
profound effect onto the performance of Storm. 
There are already a few benchmarks of Storm’s performance such as:
https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/Streams-and-Storm-April-2014-Final.pdf
In addition, you may want to take a look at the academic paper Storm@Twitter 
and Twitter Heron: Stream processing at scale which describe among others 
certain performance aspects of Storm that might be helpful to you when 
designing the benchmark. 
Regards,Dominik 

On 4 Nov 2016, at 09:36, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Everyone,
Anyone please could tell what are the common benchmarks for testing Storm 
performance? 
Thank you,--Regards
WA

Re: Storm benchmarks

2016-11-04 Thread Roshan Naik

http://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/


-roshan


On 11/4/16, 7:13 AM, "Paul Poulosky" <ppoul...@yahoo-inc.com.INVALID>
wrote:

>IEEE Xplore Document - Benchmarking Streaming Computation Engines: Storm,
>Flink and Spark Streaming
>
>  
>|  
>|   |  
>IEEE Xplore Document - Benchmarking Streaming Computation Engines: Storm,
>Flink and Spark Streaming
> Streaming data processing has been gaining attention due to its
>application into a wide range of scenarios. To s...  |  |
>
>  |
>
> 
>
>
>Benchmarking Streaming Computation Engines at Yahoo!
>
>  
>|  
>|   
>|   
>|   ||
>
>   |
>
>  |
>|  
>||  
>Benchmarking Streaming Computation Engines at Yahoo!
> (Yahoo Storm Team in alphabetical order) Sanket Chintapalli, Derek
>Dagit, Bobby Evans, Reza Farivar, Tom Graves,...  |   |
>
>  |
>
>  |
>
> 
>
> 
>
>On Friday, November 4, 2016 6:05 AM, Dominik Safaric
><dominiksafa...@gmail.com> wrote:
> 
>
> 
>1- What do you mean "able to control message size"? Is it
>max-pending-spout parameter?
>
>By using for example Kafka as your source of information of the benchmark
>topology, you may produce i.e. control the size of messages in terms of
>bytes length. Why would you want to do this? Because there is a relation
>between certain performance characteristics such as throughput and
>message size. 
>
>Is there any published benchmark like this old-one here:
>
>As far up to my knowledge, no. However, we at the Web Information Systems
>research group of the Delft University of Technology are currently in the
>process of benchmarking several streaming engines (including Storm) part
>of an empirical research. If you¹d like to here more about the insight so
>far gathered, feel free to email me.
>
>On 4 Nov 2016, at 10:02, Walid Aljoby <walid_alj...@yahoo.com> wrote:
>Thank you Dominik. I have two more points, please.1- What do you mean
>"able to control message size"? Is it max-pending-spout parameter?2- Is
>there any published benchmark like this old-one here:
>https://github.com/stormprocessor/storm-benchmark/commit/22bd17a81020ceef7
>1ed73168ac89d3f8eaf61e2
>Best Regards,Walid
>
>  From: Dominik Safaric <dominiksafa...@gmail.com>
> To: Walid Aljoby <walid_alj...@yahoo.com>
>Cc: "u...@storm.apache.org" <u...@storm.apache.org>;
>"dev@storm.apache.org" <dev@storm.apache.org>
> Sent: Friday, November 4, 2016 4:53 PM
> Subject: Re: Storm benchmarks
>  
>Well, this depends onto the aspects of the measurements.
>You may for example define a topology consisting of a spout,
>transformation bolt and sink that receives byte arrays from Kafka,
>transforms them and outputs. The nice thing is that you¹d be able to
>control for the size of the messages.
>In addition, if you care about the performance in conjunction to stateful
>operations such as aggregations, your topology might look alike the for
>example WordCount topology.
>Regards,Dominik
>
>On 4 Nov 2016, at 09:50, Walid Aljoby <walid_alj...@yahoo.com> wrote:
>Hi Dominik,
>Many thanks for details. Actually I am looking for a set typologies for
>my test.
>Thank you again,--RegardsWalid
>
>  From: Dominik Safaric <dominiksafa...@gmail.com>
> To: u...@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com>
>Cc: "dev@storm.apache.org" <dev@storm.apache.org>
> Sent: Friday, November 4, 2016 4:41 PM
> Subject: Re: Storm benchmarks
>  
>Hi Walid,
>You may benchmark Storm¹s performance in terms of throughput and
>end-to-end latency for example. In addition, the investigation could also
>include variances in the configurational settings, such as the
>parallelism, message size, intra-worker and inter-worker buffer size
>which some of them have a profound effect onto the performance of Storm.
>There are already a few benchmarks of Storm¹s performance such as:
>https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/S
>treams-and-Storm-April-2014-Final.pdf
>In addition, you may want to take a look at the academic paper
>Storm@Twitter and Twitter Heron: Stream processing at scale which
>describe among others certain performance aspects of Storm that might be
>helpful to you when designing the benchmark.
>Regards,Dominik 
>
>On 4 Nov 2016, at 09:36, Walid Aljoby <walid_alj...@yahoo.com> wrote:
>Hi Everyone,
>Anyone please could tell what are the common benchmarks for testing Storm
>performance? 
>Thank you,--Regards
>WA
>
>
>
>   
>
>
>
>   
>
>
>
>

Re: Storm benchmarks

2016-11-04 Thread Paul Poulosky

IEEE Xplore Document - Benchmarking Streaming Computation Engines: Storm, Flink 
and Spark Streaming

  
|  
|   |  
IEEE Xplore Document - Benchmarking Streaming Computation Engines: Storm, Flink 
and Spark Streaming
 Streaming data processing has been gaining attention due to its application 
into a wide range of scenarios. To s...  |  |

  |

 


Benchmarking Streaming Computation Engines at Yahoo!

  
|  
|   
|   
|   ||

   |

  |
|  
||  
Benchmarking Streaming Computation Engines at Yahoo!
 (Yahoo Storm Team in alphabetical order) Sanket Chintapalli, Derek Dagit, 
Bobby Evans, Reza Farivar, Tom Graves,...  |   |

  |

  |

 

 

On Friday, November 4, 2016 6:05 AM, Dominik Safaric 
<dominiksafa...@gmail.com> wrote:
 

 
1- What do you mean "able to control message size"? Is it max-pending-spout 
parameter?

By using for example Kafka as your source of information of the benchmark 
topology, you may produce i.e. control the size of messages in terms of bytes 
length. Why would you want to do this? Because there is a relation between 
certain performance characteristics such as throughput and message size. 

Is there any published benchmark like this old-one here:

As far up to my knowledge, no. However, we at the Web Information Systems 
research group of the Delft University of Technology are currently in the 
process of benchmarking several streaming engines (including Storm) part of an 
empirical research. If you’d like to here more about the insight so far 
gathered, feel free to email me.  

On 4 Nov 2016, at 10:02, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Thank you Dominik. I have two more points, please.1- What do you mean "able to 
control message size"? Is it max-pending-spout parameter?2- Is there any 
published benchmark like this old-one here: 
https://github.com/stormprocessor/storm-benchmark/commit/22bd17a81020ceef71ed73168ac89d3f8eaf61e2
Best Regards,Walid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "u...@storm.apache.org" <u...@storm.apache.org>; "dev@storm.apache.org" 
<dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:53 PM
 Subject: Re: Storm benchmarks
  
Well, this depends onto the aspects of the measurements. 
You may for example define a topology consisting of a spout, transformation 
bolt and sink that receives byte arrays from Kafka, transforms them and 
outputs. The nice thing is that you’d be able to control for the size of the 
messages. 
In addition, if you care about the performance in conjunction to stateful 
operations such as aggregations, your topology might look alike the for example 
WordCount topology.
Regards,Dominik

On 4 Nov 2016, at 09:50, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Dominik,
Many thanks for details. Actually I am looking for a set typologies for my test.
Thank you again,--RegardsWalid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: u...@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "dev@storm.apache.org" <dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:41 PM
 Subject: Re: Storm benchmarks
  
Hi Walid,
You may benchmark Storm’s performance in terms of throughput and end-to-end 
latency for example. In addition, the investigation could also include 
variances in the configurational settings, such as the parallelism, message 
size, intra-worker and inter-worker buffer size which some of them have a 
profound effect onto the performance of Storm. 
There are already a few benchmarks of Storm’s performance such as:
https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/Streams-and-Storm-April-2014-Final.pdf
In addition, you may want to take a look at the academic paper Storm@Twitter 
and Twitter Heron: Stream processing at scale which describe among others 
certain performance aspects of Storm that might be helpful to you when 
designing the benchmark. 
Regards,Dominik 

On 4 Nov 2016, at 09:36, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Everyone,
Anyone please could tell what are the common benchmarks for testing Storm 
performance? 
Thank you,--Regards
WA

Re: Storm benchmarks

2016-11-04 Thread Walid Aljoby

Thank you Dominik. I have two more points, please.1- What do you mean "able to 
control message size"? Is it max-pending-spout parameter?2- Is there any 
published benchmark like this old-one here: 
https://github.com/stormprocessor/storm-benchmark/commit/22bd17a81020ceef71ed73168ac89d3f8eaf61e2
Best Regards,Walid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "u...@storm.apache.org" <u...@storm.apache.org>; "dev@storm.apache.org" 
<dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:53 PM
 Subject: Re: Storm benchmarks

Well, this depends onto the aspects of the measurements. 
You may for example define a topology consisting of a spout, transformation 
bolt and sink that receives byte arrays from Kafka, transforms them and 
outputs. The nice thing is that you’d be able to control for the size of the 
messages. 
In addition, if you care about the performance in conjunction to stateful 
operations such as aggregations, your topology might look alike the for example 
WordCount topology.
Regards,Dominik

On 4 Nov 2016, at 09:50, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Dominik,
Many thanks for details. Actually I am looking for a set typologies for my test.
Thank you again,--RegardsWalid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: u...@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "dev@storm.apache.org" <dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:41 PM
 Subject: Re: Storm benchmarks

Hi Walid,
You may benchmark Storm’s performance in terms of throughput and end-to-end 
latency for example. In addition, the investigation could also include 
variances in the configurational settings, such as the parallelism, message 
size, intra-worker and inter-worker buffer size which some of them have a 
profound effect onto the performance of Storm. 
There are already a few benchmarks of Storm’s performance such as:
https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/Streams-and-Storm-April-2014-Final.pdf
In addition, you may want to take a look at the academic paper Storm@Twitter 
and Twitter Heron: Stream processing at scale which describe among others 
certain performance aspects of Storm that might be helpful to you when 
designing the benchmark. 
Regards,Dominik 

On 4 Nov 2016, at 09:36, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Everyone,
Anyone please could tell what are the common benchmarks for testing Storm 
performance? 
Thank you,--Regards
WA

Re: Storm benchmarks

2016-11-04 Thread Walid Aljoby

Hi Dominik,
Many thanks for details. Actually I am looking for a set typologies for my test.
Thank you again,--RegardsWalid

  From: Dominik Safaric <dominiksafa...@gmail.com>
 To: u...@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com> 
Cc: "dev@storm.apache.org" <dev@storm.apache.org>
 Sent: Friday, November 4, 2016 4:41 PM
 Subject: Re: Storm benchmarks

Hi Walid,
You may benchmark Storm’s performance in terms of throughput and end-to-end 
latency for example. In addition, the investigation could also include 
variances in the configurational settings, such as the parallelism, message 
size, intra-worker and inter-worker buffer size which some of them have a 
profound effect onto the performance of Storm. 
There are already a few benchmarks of Storm’s performance such as:
https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/Streams-and-Storm-April-2014-Final.pdf
In addition, you may want to take a look at the academic paper Storm@Twitter 
and Twitter Heron: Stream processing at scale which describe among others 
certain performance aspects of Storm that might be helpful to you when 
designing the benchmark. 
Regards,Dominik 

On 4 Nov 2016, at 09:36, Walid Aljoby <walid_alj...@yahoo.com> wrote:
Hi Everyone,
Anyone please could tell what are the common benchmarks for testing Storm 
performance? 
Thank you,--Regards
WA

Re: Storm Benchmarks

2016-07-18 Thread Walid Aljoby

Thank you Anshu.
I tried that but it doesn't work. Same error: Exception in thread "main" 
java.lang.NoClassDefFoundError: backtype/storm/generated/AlreadyAliveException.
Regards
WA

  From: anshu shukla <anshushuk...@gmail.com>
 To: u...@storm.apache.org 
Cc: Walid Aljoby <walid_alj...@yahoo.com>; "dev@storm.apache.org" 
<dev@storm.apache.org>
 Sent: Tuesday, July 19, 2016 12:45 AM
 Subject: Re: Storm Benchmarks

This was suggested from someone in mailing list only . I have tested that 
already it works .
You can run the old code with the newer version by putting up this config in 
storm.yaml of client (machine where you submit the topologies from) - 
client.jartransformer.class: "org.apache.storm.hack.StormShadeTransformer"
On Mon, Jul 18, 2016 at 10:04 PM, Manu Zhang <owenzhang1...@gmail.com> wrote:

I don't think storm-benchmark can be run against storm 1.0.1 without code 
changes.Yes, both computation and communication parts are included. The metrics 
are collected by storm nimbus including end-to-end throughput and latency. I 
know nothing more than you on that micro-benchmark. Maybe +ptgo...@gmail.com 
can help here.
On Mon, Jul 18, 2016 at 9:10 PM Walid Aljoby <walid_alj...@yahoo.com> wrote:

Hi Manu Zhang,
Thank you for your prompt replies.
I am using Apache storm 1.0.1. So I don't whether it is feasible to change the 
version of storm inside pom.xml file of storm-benchmark to 1.0.1. ?Do the 
metrics you use include both computation and communication parts?Do you have an 
idea about the implementation of the Benchmarks shown here: Microbenchmarking 
Apache Storm 1.0 Performance - Hortonworks ? and How is it relative to your own 
metrics?
Sorry for all these questions,
Thank you--RegardsWalidFrom: Manu Zhang <owenzhang1...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com>; "u...@storm.apache.org" 
<u...@storm.apache.org>; "dev@storm.apache.org" <dev@storm.apache.org> 
 Sent: Monday, July 18, 2016 8:09 PM
 Subject: Re: Storm Benchmarks

Hey Walid,

storm-benchmark only supports storm 0.9.x for now. Which storm version do
you want to run against ?

Regards,
Manu Zhang

On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby <walid_alj...@yahoo.com.invalid>
wrote:

> Thank you.
> You mean that it could not be run with new version of Apache storm?
> Regards
> Walid
>
>      From: darion.yaphet <fly...@163.com>
>  To: dev@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com>
> Cc: User <u...@storm.apache.org>
>  Sent: Monday, July 18, 2016 6:35 PM
>  Subject: Re:Storm Benchmarks
>
> It seems not found backtype.storm.generated.AlreadyAliveException
> .Because AlreadyAliveException have move to org.apache.storm.generated
> At 2016-07-18 18:06:36, "Walid Aljoby" <walid_alj...@yahoo.com.INVALID>
> wrote:
> >Hi Everyone,
> >Anyone used to run Storm Benchmarks in this site:
> https://github.com/roshannaik/storm-benchmark-sol ?The compilation was
> successful, but I got into error through run time?
> >
> >.0.1/bin/storm -jar
> ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf
> ./conf/sol.yaml -c topology.workers=2 storm.benchmark.tools.Runner
> storm.benchmark.benchmarks.SOL Running:
> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -client -Ddaemon.name=
> -Dstorm.options=topology.workers%3D2
> -Dstorm.home=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1
> -Dstorm.log.dir=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/logs
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> -Dstorm.conf.file=sol.yaml -cp
> /home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/objenesis-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/asm-5.0.3.jar:./target/storm-benchmark-0.1.0-jar-with-dependencies.jar:/home/jiach

Re: Storm Benchmarks

2016-07-18 Thread Manu Zhang

I don't think storm-benchmark can be run against storm 1.0.1 without code
changes.
Yes, both computation and communication parts are included. The metrics are
collected by storm nimbus including end-to-end throughput and latency.
I know nothing more than you on that micro-benchmark. Maybe +
ptgo...@gmail.com <+ptgo...@gmail.com> can help here.

On Mon, Jul 18, 2016 at 9:10 PM Walid Aljoby <walid_alj...@yahoo.com> wrote:

> Hi Manu Zhang,
>
> Thank you for your prompt replies.
>
> I am using Apache storm 1.0.1. So I don't whether it is feasible to
> change the version of storm inside pom.xml file of storm-benchmark to
> 1.0.1. ?
> Do the metrics you use include both computation and communication parts?
> Do you have an idea about the implementation of the Benchmarks shown here: 
> Microbenchmarking
> Apache Storm 1.0 Performance - Hortonworks
> <http://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/> ?
> and How is it relative to your own metrics?
>
> Sorry for all these questions,
>
> Thank you
> --
> Regards
> Walid
> --
> *From:* Manu Zhang <owenzhang1...@gmail.com>
> *To:* Walid Aljoby <walid_alj...@yahoo.com>; "u...@storm.apache.org" <
> u...@storm.apache.org>; "dev@storm.apache.org" <dev@storm.apache.org>
> *Sent:* Monday, July 18, 2016 8:09 PM
> *Subject:* Re: Storm Benchmarks
>
> Hey Walid,
>
> storm-benchmark only supports storm 0.9.x for now. Which storm version do
> you want to run against ?
>
> Regards,
> Manu Zhang
>
>
> On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby <
> walid_alj...@yahoo.com.invalid>
> wrote:
>
> > Thank you.
> > You mean that it could not be run with new version of Apache storm?
> > Regards
> > Walid
> >
> >  From: darion.yaphet <fly...@163.com>
> >  To: dev@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com>
> > Cc: User <u...@storm.apache.org>
> >  Sent: Monday, July 18, 2016 6:35 PM
> >  Subject: Re:Storm Benchmarks
> >
> > It seems not found backtype.storm.generated.AlreadyAliveException
> > .Because AlreadyAliveException have move to
> org.apache.storm.generated
> > At 2016-07-18 18:06:36, "Walid Aljoby" <walid_alj...@yahoo.com.INVALID>
> > wrote:
> > >Hi Everyone,
> > >Anyone used to run Storm Benchmarks in this site:
> > https://github.com/roshannaik/storm-benchmark-sol ?The compilation was
> > successful, but I got into error through run time?
> > >
> > >.0.1/bin/storm -jar
> > ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf
> > ./conf/sol.yaml -c topology.workers=2 storm.benchmark.tools.Runner
> > storm.benchmark.benchmarks.SOL Running:
> > /usr/lib/jvm/java-7-openjdk-amd64/bin/java -client -Ddaemon.name=
> > -Dstorm.options=topology.workers%3D2
> > -Dstorm.home=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1
> >
> -Dstorm.log.dir=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/logs
> > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> > -Dstorm.conf.file=sol.yaml -cp
> >
> /home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/objenesis-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/asm-5.0.3.jar:./target/storm-benchmark-0.1.0-jar-with-dependencies.jar:/home/jiachen/Desktop/Storm-Performance/storm-benchmark/conf:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/bin
> > -Dstorm.jar=./target/storm-benchmark-0.1.0-jar-with-dependencies.jar
> > storm.benchmark.tools.Runner storm.benchmark.benchmarks.SOLException in
> > thread "main" java.lang.NoClassDefFoundError:
> > backtype

Re: Storm Benchmarks

2016-07-18 Thread Walid Aljoby

Hi Manu Zhang,
Thank you for your prompt replies.
I am using Apache storm 1.0.1. So I don't whether it is feasible to change the 
version of storm inside pom.xml file of storm-benchmark to 1.0.1. ?Do the 
metrics you use include both computation and communication parts?Do you have an 
idea about the implementation of the Benchmarks shown here: Microbenchmarking 
Apache Storm 1.0 Performance - Hortonworks ? and How is it relative to your own 
metrics?
Sorry for all these questions,
Thank you--RegardsWalid
 

  From: Manu Zhang <owenzhang1...@gmail.com>
 To: Walid Aljoby <walid_alj...@yahoo.com>; "u...@storm.apache.org" 
<u...@storm.apache.org>; "dev@storm.apache.org" <dev@storm.apache.org> 
 Sent: Monday, July 18, 2016 8:09 PM
 Subject: Re: Storm Benchmarks
   
Hey Walid,

storm-benchmark only supports storm 0.9.x for now. Which storm version do
you want to run against ?

Regards,
Manu Zhang


On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby <walid_alj...@yahoo.com.invalid>
wrote:

> Thank you.
> You mean that it could not be run with new version of Apache storm?
> Regards
> Walid
>
>      From: darion.yaphet <fly...@163.com>
>  To: dev@storm.apache.org; Walid Aljoby <walid_alj...@yahoo.com>
> Cc: User <u...@storm.apache.org>
>  Sent: Monday, July 18, 2016 6:35 PM
>  Subject: Re:Storm Benchmarks
>
> It seems not found backtype.storm.generated.AlreadyAliveException
> .Because AlreadyAliveException have move to org.apache.storm.generated
> At 2016-07-18 18:06:36, "Walid Aljoby" <walid_alj...@yahoo.com.INVALID>
> wrote:
> >Hi Everyone,
> >Anyone used to run Storm Benchmarks in this site:
> https://github.com/roshannaik/storm-benchmark-sol ?The compilation was
> successful, but I got into error through run time?
> >
> >.0.1/bin/storm -jar
> ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf
> ./conf/sol.yaml -c topology.workers=2 storm.benchmark.tools.Runner
> storm.benchmark.benchmarks.SOL Running:
> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -client -Ddaemon.name=
> -Dstorm.options=topology.workers%3D2
> -Dstorm.home=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1
> -Dstorm.log.dir=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/logs
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> -Dstorm.conf.file=sol.yaml -cp
> /home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/objenesis-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/asm-5.0.3.jar:./target/storm-benchmark-0.1.0-jar-with-dependencies.jar:/home/jiachen/Desktop/Storm-Performance/storm-benchmark/conf:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/bin
> -Dstorm.jar=./target/storm-benchmark-0.1.0-jar-with-dependencies.jar
> storm.benchmark.tools.Runner storm.benchmark.benchmarks.SOLException in
> thread "main" java.lang.NoClassDefFoundError:
> backtype/storm/generated/AlreadyAliveException at
> java.lang.Class.getDeclaredMethods0(Native Method) at
> java.lang.Class.privateGetDeclaredMethods(Class.java:2625) at
> java.lang.Class.getMethod0(Class.java:2866) at
> java.lang.Class.getMethod(Class.java:1676) at
> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)Caused
> by: java.lang.ClassNotFoundException:
> backtype.storm.generated.AlreadyAliveException at
> java.net.URLClassLoader$1.run(URLClassLoader.java:366) at
> java.net.URLClassLoader$1.run(URLClassLoader.java:355) at
> java.security.AccessController.doPrivileged(Native Method) at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:425) at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more
> >
> >Thank You
> >--
> >Regards,
> >Walid
> >
>
>
>

Re: Storm Benchmarks

2016-07-18 Thread Manu Zhang

Hey Walid,

storm-benchmark only supports storm 0.9.x for now. Which storm version do
you want to run against ?

Regards,
Manu Zhang


On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby 
wrote:

> Thank you.
> You mean that it could not be run with new version of Apache storm?
> Regards
> Walid
>
>   From: darion.yaphet 
>  To: dev@storm.apache.org; Walid Aljoby 
> Cc: User 
>  Sent: Monday, July 18, 2016 6:35 PM
>  Subject: Re:Storm Benchmarks
>
> It seems not found backtype.storm.generated.AlreadyAliveException
> .Because AlreadyAliveException have move to org.apache.storm.generated
> At 2016-07-18 18:06:36, "Walid Aljoby" 
> wrote:
> >Hi Everyone,
> >Anyone used to run Storm Benchmarks in this site:
> https://github.com/roshannaik/storm-benchmark-sol ?The compilation was
> successful, but I got into error through run time?
> >
> >.0.1/bin/storm -jar
> ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf
> ./conf/sol.yaml -c topology.workers=2 storm.benchmark.tools.Runner
> storm.benchmark.benchmarks.SOL Running:
> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -client -Ddaemon.name=
> -Dstorm.options=topology.workers%3D2
> -Dstorm.home=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1
> -Dstorm.log.dir=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/logs
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> -Dstorm.conf.file=sol.yaml -cp
> /home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/objenesis-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/asm-5.0.3.jar:./target/storm-benchmark-0.1.0-jar-with-dependencies.jar:/home/jiachen/Desktop/Storm-Performance/storm-benchmark/conf:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/bin
> -Dstorm.jar=./target/storm-benchmark-0.1.0-jar-with-dependencies.jar
> storm.benchmark.tools.Runner storm.benchmark.benchmarks.SOLException in
> thread "main" java.lang.NoClassDefFoundError:
> backtype/storm/generated/AlreadyAliveException at
> java.lang.Class.getDeclaredMethods0(Native Method) at
> java.lang.Class.privateGetDeclaredMethods(Class.java:2625) at
> java.lang.Class.getMethod0(Class.java:2866) at
> java.lang.Class.getMethod(Class.java:1676) at
> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)Caused
> by: java.lang.ClassNotFoundException:
> backtype.storm.generated.AlreadyAliveException at
> java.net.URLClassLoader$1.run(URLClassLoader.java:366) at
> java.net.URLClassLoader$1.run(URLClassLoader.java:355) at
> java.security.AccessController.doPrivileged(Native Method) at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:425) at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more
> >
> >Thank You
> >--
> >Regards,
> >Walid
> >
>
>
>

Re: Storm Benchmarks

2016-07-18 Thread Walid Aljoby

Thank you.
You mean that it could not be run with new version of Apache storm?
Regards
Walid

  From: darion.yaphet 
 To: dev@storm.apache.org; Walid Aljoby  
Cc: User 
 Sent: Monday, July 18, 2016 6:35 PM
 Subject: Re:Storm Benchmarks
   
It seems not found backtype.storm.generated.AlreadyAliveException .Because 
AlreadyAliveException have move to org.apache.storm.generated
At 2016-07-18 18:06:36, "Walid Aljoby"  wrote:
>Hi Everyone,
>Anyone used to run Storm Benchmarks in this site: 
>https://github.com/roshannaik/storm-benchmark-sol ?The compilation was 
>successful, but I got into error through run time?
>
>.0.1/bin/storm -jar ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar 
>-conf ./conf/sol.yaml -c topology.workers=2 storm.benchmark.tools.Runner 
>storm.benchmark.benchmarks.SOL Running: 
>/usr/lib/jvm/java-7-openjdk-amd64/bin/java -client -Ddaemon.name= 
>-Dstorm.options=topology.workers%3D2 
>-Dstorm.home=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1 
>-Dstorm.log.dir=/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/logs 
>-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib 
>-Dstorm.conf.file=sol.yaml -cp 
>/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/objenesis-2.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/lib/asm-5.0.3.jar:./target/storm-benchmark-0.1.0-jar-with-dependencies.jar:/home/jiachen/Desktop/Storm-Performance/storm-benchmark/conf:/home/jiachen/Desktop/Single-Storm/apache-storm-1.0.1/bin
> -Dstorm.jar=./target/storm-benchmark-0.1.0-jar-with-dependencies.jar 
>storm.benchmark.tools.Runner storm.benchmark.benchmarks.SOLException in thread 
>"main" java.lang.NoClassDefFoundError: 
>backtype/storm/generated/AlreadyAliveException at 
>java.lang.Class.getDeclaredMethods0(Native Method) at 
>java.lang.Class.privateGetDeclaredMethods(Class.java:2625) at 
>java.lang.Class.getMethod0(Class.java:2866) at 
>java.lang.Class.getMethod(Class.java:1676) at 
>sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at 
>sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)Caused 
>by: java.lang.ClassNotFoundException: 
>backtype.storm.generated.AlreadyAliveException at 
>java.net.URLClassLoader$1.run(URLClassLoader.java:366) at 
>java.net.URLClassLoader$1.run(URLClassLoader.java:355) at 
>java.security.AccessController.doPrivileged(Native Method) at 
>java.net.URLClassLoader.findClass(URLClassLoader.java:354) at 
>java.lang.ClassLoader.loadClass(ClassLoader.java:425) at 
>sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at 
>java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more
>
>Thank You
>--
>Regards,
>Walid 
>

Re: Storm benchmarks

Re: Storm benchmarks

Re: Storm benchmarks

Re: Storm benchmarks

Re: Storm benchmarks

Re: Storm Benchmarks

Re: Storm Benchmarks

Re: Storm Benchmarks

Re: Storm Benchmarks

Re: Storm Benchmarks

10 matches

Site Navigation

Mail list logo

Footer information