Sorry, wrong Link. Correct one at [1]. 1. https://gist.github.com/cdwijayarathna/8928949
Thank You. On Tue, Feb 11, 2014 at 9:14 AM, Chamila Wijayarathna <[email protected]>wrote: > Hi Mohan, > I checked output by CEP which I forwarded to BAM [1]. This is for 8.59 > minute and it has 95 requests per 1 minute. > Since at this time there are less crowd in office, this can be increased > for about 10 times larger than this value. > But yesterday I got the error when I tested it at about 11.00 p.m. which > is a time with very low traffic. > Do you think 1000 per min is a traffic that BAM's cassandra can't bear? > And also when I test this in my local machine at the same time, I don't > receive this error. > > Thank You. > > > 1. https://gist.github.com/gists > > > On Tue, Feb 11, 2014 at 7:57 AM, Mohanadarshan Vivekanandalingam < > [email protected]> wrote: > >> Hi Chamila, >> >> On Mon, Feb 10, 2014 at 11:19 PM, Chamila Wijayarathna >> <[email protected]>wrote: >> >>> Hi, >>> I again ran BAM with my custom producer sending events to it. For more >>> than 2 hours it didn't gave me above mentioned exception. Then I started >>> CEP and sent netflow records to BAM through CEP. >>> After about 5 minutes it began to give above mentioned error >>> continuously.(nohup.out[1]). >>> I also noticed that at several places I got message saying " INFO >>> {org.apache.cassandra.net.MessagingService} - 1732 MUTATION messages >>> dropped in last 5000ms >>> " with number of messages increasing. In morning I discussed with Mohan >>> and in several resources at internet I read this error can occur because of >>> high message frequency to cassandra. I have done the performance tuning >>> recommendations mentioned in [2] also. >>> And also I didn't receive openjdk exception on BAM this time.Cassandra >>> Can someone please give me some clue on this? >>> >> >> Then, I think as Prabath mentioned you have to scale the BAM where you >> need a fully distributed BAM cluster.. Do you have any clear idea about how >> many events are coming to BAM?? >> >> Thanks, >> Mohan >> >> >> >> >>> 1. https://gist.github.com/cdwijayarathna/8920593 >>> 2. >>> http://docs.wso2.org/display/BAM240/Performance+Tuning+Recommendations<http://www.google.com/url?q=http%3A%2F%2Fdocs.wso2.org%2Fdisplay%2FBAM240%2FPerformance%2BTuning%2BRecommendations&sa=D&sntz=1&usg=AFQjCNFEiVzw4i1-kHAke3YxSC7WHw2qOg> >>> >>> Thank you. >>> >>> >>> On Mon, Feb 10, 2014 at 10:56 PM, Chamila Wijayarathna <[email protected] >>> > wrote: >>> >>>> Hi Mohan, >>>> I'll check that. >>>> Thank you >>>> >>>> >>>> On Mon, Feb 10, 2014 at 9:30 PM, Mohanadarshan Vivekanandalingam < >>>> [email protected]> wrote: >>>> >>>>> Hi Chamila, >>>>> >>>>> I can see below exception before the timeoutException, where openjdk >>>>> is used but we normally recommend to use oracle jdk.. Please see link >>>>> [1].. >>>>> I am doubt whether below issue leads to unstable situation.. Can you >>>>> please >>>>> use the recommended jdk and see.. >>>>> >>>>> BAM team, any more input on this?? >>>>> >>>>> java.io.IOException: Cannot run program >>>>> "/usr/lib/jvm/java-6-openjdk/bin/java" (in directory >>>>> "/home/wso2/product/wso2bam-2.4.0"): java.io.IOException: error=12, >>>>> Cannot allocate memory >>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:488) >>>>> at java.lang.Runtime.exec(Runtime.java:610) >>>>> >>>>> at java.lang.Runtime.exec(Runtime.java:448) >>>>> at >>>>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:317) >>>>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:129) >>>>> at >>>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:62) >>>>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1351) >>>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1126) >>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:934) >>>>> >>>>> at >>>>> org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:201) >>>>> at >>>>> org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:187) >>>>> at >>>>> org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.executeHiveQuery(HiveExecutorServiceImpl.java:569) >>>>> at >>>>> org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:282) >>>>> >>>>> >>>>> >>>>> >>>>> at >>>>> org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:189) >>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>>>> >>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:679) >>>>> >>>>> Caused by: java.io.IOException: java.io.IOException: error=12, Cannot >>>>> allocate memory >>>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:164) >>>>> >>>>> at java.lang.ProcessImpl.start(ProcessImpl.java:81) >>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:470) >>>>> ... 18 more >>>>> >>>>> >>>>> [1] http://docs.wso2.org/display/BAM240/Installation+Prerequisites >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Mohan >>>>> >>>>> >>>>> >>>>> On Mon, Feb 10, 2014 at 7:47 PM, Chamila Wijayarathna < >>>>> [email protected]> wrote: >>>>> >>>>>> I ran BAM without sending events to it and then it don't give any >>>>>> exeption. >>>>>> >>>>>> >>>>>> On Mon, Feb 10, 2014 at 6:53 PM, Chamila Wijayarathna < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello all, >>>>>>> I have a BAM instance running on a remote server with port offset of >>>>>>> 1. >>>>>>> I'm sending a stream of wso2 events to it using an external >>>>>>> publisher[1]. >>>>>>> After about 1.5 hours, I am getting the following error. >>>>>>> >>>>>>> *[2014-02-10 18:12:08,721] ERROR >>>>>>> {org.wso2.carbon.databridge.datasink.cassandra.subscriber.BAMEventSubscriber} >>>>>>> - Error processing event. * >>>>>>> *me.prettyprint.hector.api.exceptions.HTimedOutException: >>>>>>> TimedOutException()* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)* >>>>>>> * at >>>>>>> org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector.commit(CassandraConnector.java:185)* >>>>>>> * at >>>>>>> org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector.insertEventList(CassandraConnector.java:676)* >>>>>>> * at >>>>>>> org.wso2.carbon.databridge.datasink.cassandra.subscriber.BAMEventSubscriber.receive(BAMEventSubscriber.java:75)* >>>>>>> * at >>>>>>> org.wso2.carbon.databridge.core.internal.queue.QueueWorker.run(QueueWorker.java:80)* >>>>>>> * at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)* >>>>>>> * at >>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)* >>>>>>> * at java.util.concurrent.FutureTask.run(FutureTask.java:166)* >>>>>>> * at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)* >>>>>>> * at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)* >>>>>>> * at java.lang.Thread.run(Thread.java:679)* >>>>>>> *Caused by: TimedOutException()* >>>>>>> * at >>>>>>> org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20269)* >>>>>>> * at >>>>>>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)* >>>>>>> * at >>>>>>> org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922)* >>>>>>> * at >>>>>>> org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)* >>>>>>> * at >>>>>>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)* >>>>>>> * ... 12 more* >>>>>>> >>>>>>> Full command line output can be found at [2]. Events I am sending >>>>>>> has been written to cassandra properly. I have a hive script which run >>>>>>> at >>>>>>> each minute on BAM too. >>>>>>> What is the reason for this error? How can I solve it? >>>>>>> >>>>>>> >>>>>>> 1. https://gist.github.com/cdwijayarathna/8915500 >>>>>>> 2. https://gist.github.com/cdwijayarathna/8915782 >>>>>>> >>>>>>> Thank You! >>>>>>> >>>>>>> -- >>>>>>> *Chamila Wijayarathna* >>>>>>> Engineering Intern, >>>>>>> WSO2 Inc. >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Chamila Wijayarathna* >>>>>> Engineering Intern, >>>>>> WSO2 Inc. >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *V. Mohanadarshan* >>>>> *Software Engineer,* >>>>> *Data Technologies Team,* >>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>> *lean.enterprise.middleware.* >>>>> >>>>> email: [email protected] >>>>> phone:(+94) 771117673 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Chamila Wijayarathna* >>>> Engineering Intern, >>>> WSO2 Inc. >>>> >>>> >>> >>> >>> -- >>> *Chamila Wijayarathna* >>> Engineering Intern, >>> WSO2 Inc. >>> >>> >> >> >> -- >> *V. Mohanadarshan* >> *Software Engineer,* >> *Data Technologies Team,* >> *WSO2, Inc. http://wso2.com <http://wso2.com> * >> *lean.enterprise.middleware.* >> >> email: [email protected] >> phone:(+94) 771117673 >> > > > > -- > *Chamila Wijayarathna* > Engineering Intern, > WSO2 Inc. > > -- *Chamila Wijayarathna* Engineering Intern, WSO2 Inc.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
