On 9 February 2014 18:32, Nirmal Fernando <[email protected]> wrote: > I debugged with Chamila, and seems like in his remote server, normal users > are not allowed to have a ulimit >32768. That's why even we set to 100000, > it showed as 32768 for both BAM and CEP. Then in order to validate whether > my above argument is correct, I've asked him to login as root user and set > ulimit manually in a terminal. And then we could see that the ulimit got > set correctly. > > Then we've started servers as root user and got the correct limit values. > There might be a setting to change the limit put upon normal user, which > needs to be Googled :) >
Cool... :-) > > > On Sun, Feb 9, 2014 at 6:23 PM, Chamila Wijayarathna <[email protected]>wrote: > >> Hi, >> I ran "ulimit -n 100000" as root user and then ran BAM as root user. >> Now it shows as "Max open files 100000 100000 >> files" in /proc/<pid>/limits file. >> >> Thank You. >> >> >> On Sun, Feb 9, 2014 at 6:15 PM, Lasantha Fernando <[email protected]>wrote: >> >>> Hi Chamila, >>> >>> Does the output of `cat /proc/<pid>/limits` show the ulimit properly set >>> for BAM as well? If it was set properly for BAM, it is strange that it >>> doesn't set properly for CEP...? >>> >>> You might have stumbled upon across something like the article in [1]. >>> If not, can you go through and see if the user-level and system-wide >>> maximum file descriptor limits are set properly. Also are you running both >>> BAM and CEP as the same user? >>> >>> [1] >>> http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/ >>> >>> Thanks, >>> Lasantha >>> >>> >>> On 9 February 2014 17:46, Chamila Wijayarathna <[email protected]> wrote: >>> >>>> When I opened CEP wso2server.sh file it has "ulimit -n 100000 " line >>>> on it which I uncommented. >>>> >>>> >>>> On Sun, Feb 9, 2014 at 5:41 PM, Nirmal Fernando <[email protected]>wrote: >>>> >>>>> It's not set it seems. You should see something like: >>>>> >>>>> Max open files 100000 100000 >>>>> files >>>>> >>>>> >>>>> On Sun, Feb 9, 2014 at 5:37 PM, Chamila Wijayarathna <[email protected] >>>>> > wrote: >>>>> >>>>>> It's about the same time as the previous occasion. >>>>>> >>>>>> limit file had the following content. >>>>>> >>>>>> Limit Soft Limit Hard Limit >>>>>> Units >>>>>> Max cpu time unlimited unlimited >>>>>> seconds >>>>>> Max file size unlimited unlimited >>>>>> bytes >>>>>> Max data size unlimited unlimited >>>>>> bytes >>>>>> Max stack size 8388608 unlimited >>>>>> bytes >>>>>> Max core file size 0 unlimited >>>>>> bytes >>>>>> Max resident set unlimited unlimited >>>>>> bytes >>>>>> Max processes unlimited unlimited >>>>>> processes >>>>>> Max open files 32768 32768 >>>>>> files >>>>>> Max locked memory 65536 65536 >>>>>> bytes >>>>>> Max address space unlimited unlimited >>>>>> bytes >>>>>> Max file locks unlimited unlimited >>>>>> locks >>>>>> Max pending signals 16382 16382 >>>>>> signals >>>>>> Max msgqueue size 819200 819200 >>>>>> bytes >>>>>> Max nice priority 0 0 >>>>>> Max realtime priority 0 0 >>>>>> Max realtime timeout unlimited unlimited us >>>>>> >>>>>> Thank You. >>>>>> >>>>>> >>>>>> On Sun, Feb 9, 2014 at 5:16 PM, Nirmal Fernando <[email protected]>wrote: >>>>>> >>>>>>> hmm.. is it at least after sometime than the previous occasion? >>>>>>> Also, did you verify that the limit has been set correctly (using the >>>>>>> limits file of process)? >>>>>>> >>>>>>> >>>>>>> On Sun, Feb 9, 2014 at 5:04 PM, Chamila Wijayarathna < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, I did edit the wso2server.sh files of both CEP and BAM. Now BAM >>>>>>>> is not giving the exception. But still CEP gives the same exception. >>>>>>>> >>>>>>>> Thank You. >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 9, 2014 at 1:04 PM, Chamila Wijayarathna < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Thank You for information. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Feb 9, 2014 at 12:54 PM, Nirmal Fernando >>>>>>>>> <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Feb 9, 2014 at 12:51 PM, Chamila Wijayarathna < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi , >>>>>>>>>>> Thank you for the information. >>>>>>>>>>> We are planning to run this for without stopping for a long >>>>>>>>>>> time. So will the 100000 limit exceed some day and give this error >>>>>>>>>>> again? >>>>>>>>>>> If so is there any way of avoiding that? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Which is highly unlikely! So, don't worry! :) >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thank You. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Feb 9, 2014 at 12:41 PM, Bhathiya Jayasekara < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> FYI: You can do the same by editing /etc/security/limits.conf file >>>>>>>>>>>> too. >>>>>>>>>>>> >>>>>>>>>>>> http://www.datastax.com/docs/1.0/troubleshooting/index#toomany >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Feb 9, 2014 at 12:18 PM, Nirmal Fernando < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> To avoid this please increase the open files limit of your OS. >>>>>>>>>>>>> 'ulimit -n' would give you the current number (by default it is >>>>>>>>>>>>> 1024 - most >>>>>>>>>>>>> of the cases). >>>>>>>>>>>>> >>>>>>>>>>>>> You can uncomment >>>>>>>>>>>>> >>>>>>>>>>>>> #ulimit -n 100000 (after uncommenting it would be : ulimit -n >>>>>>>>>>>>> 100000 ) >>>>>>>>>>>>> >>>>>>>>>>>>> in the wso2server.sh file of wso2 products in order to >>>>>>>>>>>>> increase the value. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Feb 9, 2014 at 12:02 PM, Chamila Wijayarathna < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hello all, >>>>>>>>>>>>>> I am working on "Infra Log Analyzer" intern project which >>>>>>>>>>>>>> uses BAM and CEP. Here we uses 2 instances and BAM and CEP >>>>>>>>>>>>>> communicating >>>>>>>>>>>>>> with each other. >>>>>>>>>>>>>> When I add these two into a remote server and ran it I am >>>>>>>>>>>>>> getting following warning and error. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [*2014-02-09 05:05:42,669] WARN >>>>>>>>>>>>>> {org.apache.cassandra.thrift.CustomTThreadPoolServer} - >>>>>>>>>>>>>> Transport error >>>>>>>>>>>>>> occurred during acceptance of message.* >>>>>>>>>>>>>> *org.apache.thrift.transport.TTransportException: >>>>>>>>>>>>>> java.net.SocketException: Too many open files* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.cassandra.thrift.TCustomServerSocket.acceptImpl(TCustomServerSocket.java:103)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.cassandra.thrift.TCustomServerSocket.acceptImpl(TCustomServerSocket.java:39)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:100)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)* >>>>>>>>>>>>>> *Caused by: java.net.SocketException: Too many open files* >>>>>>>>>>>>>> * at java.net.PlainSocketImpl.socketAccept(Native Method)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)* >>>>>>>>>>>>>> * at java.net.ServerSocket.implAccept(ServerSocket.java:462)* >>>>>>>>>>>>>> * at java.net.ServerSocket.accept(ServerSocket.java:430)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.cassandra.thrift.TCustomServerSocket.acceptImpl(TCustomServerSocket.java:97)* >>>>>>>>>>>>>> * ... 4 more* >>>>>>>>>>>>>> *[2014-02-09 05:05:42,666] ERROR >>>>>>>>>>>>>> {org.apache.thrift.transport.TSocket} - Could not configure >>>>>>>>>>>>>> socket.* >>>>>>>>>>>>>> *java.net.SocketException: Too many open files* >>>>>>>>>>>>>> * at java.net.Socket.createImpl(Socket.java:397)* >>>>>>>>>>>>>> * at java.net.Socket.getImpl(Socket.java:460)* >>>>>>>>>>>>>> * at java.net.Socket.setSoLinger(Socket.java:900)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.thrift.transport.TSocket.initSocket(TSocket.java:116)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.thrift.transport.TSocket.<init>(TSocket.java:107)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.client.HThriftClient.open(HThriftClient.java:123)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.client.HThriftClient.open(HThriftClient.java:26)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.ConcurrentHClientPool.createClient(ConcurrentHClientPool.java:147)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.ConcurrentHClientPool.<init>(ConcurrentHClientPool.java:53)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.RoundRobinBalancingPolicy.createConnection(RoundRobinBalancingPolicy.java:67)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.connection.HConnectionManager.<init>(HConnectionManager.java:67)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.service.AbstractCluster.<init>(AbstractCluster.java:67)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> me.prettyprint.cassandra.service.ThriftCluster.<init>(ThriftCluster.java:21)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.wso2.carbon.bam.cassandra.data.archive.mapred.CassandraMapReduceRowDeletion$RowKeyMapper.setup(CassandraMapReduceRowDeletion.java:119)* >>>>>>>>>>>>>> * at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)* >>>>>>>>>>>>>> * at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)* >>>>>>>>>>>>>> * at >>>>>>>>>>>>>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:211)* >>>>>>>>>>>>>> >>>>>>>>>>>>>> I get this when server runs for a while about 1.5 - 2 hours. >>>>>>>>>>>>>> I get the same error in both BAM and CEP. >>>>>>>>>>>>>> I didn't get this when I ran them on my local machine. >>>>>>>>>>>>>> What is the reason for this? How can I avoid this? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thank You! >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Chamila Wijayarathna* >>>>>>>>>>>>>> Engineering Intern, >>>>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>> Nirmal >>>>>>>>>>>>> >>>>>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Bhathiya Jayasekara* >>>>>>>>>>>> *Software Engineer,* >>>>>>>>>>>> *WSO2 inc., http://wso2.com <http://wso2.com>* >>>>>>>>>>>> >>>>>>>>>>>> *Phone: +94715478185 <%2B94715478185>* >>>>>>>>>>>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj >>>>>>>>>>>> <http://www.linkedin.com/in/bhathiyaj>* >>>>>>>>>>>> *Twitter: https://twitter.com/bhathiyax >>>>>>>>>>>> <https://twitter.com/bhathiyax>* >>>>>>>>>>>> *Blog: http://movingaheadblog.blogspot.com >>>>>>>>>>>> <http://movingaheadblog.blogspot.com/>* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Chamila Wijayarathna* >>>>>>>>>>> Engineering Intern, >>>>>>>>>>> WSO2 Inc. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & regards, >>>>>>>>>> Nirmal >>>>>>>>>> >>>>>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>>>>> Mobile: +94715779733 >>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Chamila Wijayarathna* >>>>>>>>> Engineering Intern, >>>>>>>>> WSO2 Inc. >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Chamila Wijayarathna* >>>>>>>> Engineering Intern, >>>>>>>> WSO2 Inc. >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & regards, >>>>>>> Nirmal >>>>>>> >>>>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>>>> Mobile: +94715779733 >>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Chamila Wijayarathna* >>>>>> Engineering Intern, >>>>>> WSO2 Inc. >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Senior Software Engineer- Platform Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Chamila Wijayarathna* >>>> Engineering Intern, >>>> WSO2 Inc. >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> *Lasantha Fernando* >>> Software Engineer - Data Technologies Team >>> WSO2 Inc. http://wso2.com >>> >>> email: [email protected] >>> mobile: (+94) 71 5247551 >>> >> >> >> >> -- >> *Chamila Wijayarathna* >> Engineering Intern, >> WSO2 Inc. >> >> > > > -- > > Thanks & regards, > Nirmal > > Senior Software Engineer- Platform Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: [email protected] mobile: (+94) 71 5247551
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
