Hi All,
The following changes I did in the MB product and achieve the $subject.
Copy the cassandra-env.sh with the changes I made to compatible with
org.wso2.carbon.core.bootup.validator.JVMValidator to the {MB_HOME}/bin
folder when pack is build
Load the cassandra-env.sh when MB start with cassandra profile
Set the -Xms -Xmx -XX:MaxPermSize parameters dynamically as calculated in
the cassandra-env.sh
This is working in the Linux. Now I have a problem with how to make this
changes which is compatible to the Windows environment. I have already make
the .bat file to start with the cassandra profile before. But now the
problem is I couldn't load the cassandra-env.sh in Windows. Is there
something which is relevant to cassandra-env.sh in Windows environment ?
Cheers!
On Wed, Apr 9, 2014 at 11:46 AM, Prabath Abeysekera <[email protected]>wrote:
>
> On Wed, Apr 9, 2014 at 11:16 AM, Dhanuka Ranasinghe <[email protected]>wrote:
>
>> Hi,
>>
>> What about other important JVM tuning parameters (GC threads and
>> strategy), Are you guys consider those as well?
>>
>
> If we are to make the Cassandra profile, fully Cassandra-like, yeah, it's
> a must we include all the required/recommended JVM parameters (including
> what's used for fine tuning the functionalities) in the start-up script as
> well.
>
> Cheers,
> Prabath
>
>
>>
>> Cheers,
>> Dhanuka
>>
>> *Dhanuka Ranasinghe*
>>
>> Senior Software Engineer
>> WSO2 Inc. ; http://wso2.com
>> lean . enterprise . middleware
>>
>> phone : +94 715381915
>>
>>
>> On Wed, Apr 9, 2014 at 11:08 AM, Shammi Jayasinghe <[email protected]>wrote:
>>
>>> Hi Deep,
>>>
>>> +1, that is what i wanted. It would be great if you can add this to the
>>> startup script of Cassandra profile.
>>>
>>> Hi Indika,
>>>
>>> Could you please talk to deep on this.
>>>
>>> thanks
>>> Shammi
>>>
>>>
>>> On Wed, Apr 9, 2014 at 10:51 AM, Deependra Ariyadewa <[email protected]>wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, Apr 9, 2014 at 8:51 AM, Shammi Jayasinghe <[email protected]>wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 8, 2014 at 7:29 PM, Kishanthan Thangarajah <
>>>>> [email protected]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 8, 2014 at 12:14 PM, Shammi Jayasinghe
>>>>>> <[email protected]>wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 8, 2014 at 11:55 AM, Deependra Ariyadewa
>>>>>>> <[email protected]>wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 8, 2014 at 11:32 AM, Shammi Jayasinghe <[email protected]
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi ,
>>>>>>>>>
>>>>>>>>> I think we should not ask the user to change the .sh file or .bat
>>>>>>>>> file when starting with the cassandra profile. What we can do is, as
>>>>>>>>> default cassandra server does, we need to fix MB in a way that , when
>>>>>>>>> we
>>>>>>>>> start the MB with Cassandra profile, it automatically allocates 1/3
>>>>>>>>> of the
>>>>>>>>> total memory for MB. WDYT ?
>>>>>>>>>
>>>>>>>>
>>>>>>>> We should be able to calculate required memory in the wso2server.sh.
>>>>>>>>
>>>>>>>
>>>>>>> Hi Sameera / Kishanthan,
>>>>>>>
>>>>>>> We have a requirement that the memory allocated when MB server
>>>>>>> starts with Cassandra profile should be a portion of the complete
>>>>>>> memory of
>>>>>>> the server. Is it possible to do it and if it is possible, how we should
>>>>>>> proceed ?
>>>>>>>
>>>>>>
>>>>>> We currently don't use profile specific configurations (such as
>>>>>> specific start-up scripts, etc) when starting up profiles. We start
>>>>>> profiles after the java process is started and all profiles use the same
>>>>>> configurations. So this will not be possible using carbon server startup
>>>>>> scripts.
>>>>>>
>>>>>> But can't you use MB/Cassandra specific startup script for this? This
>>>>>> script should set the relevant JVM parameters and call server start-up
>>>>>> with
>>>>>> cassandra profile.
>>>>>>
>>>>>
>>>>> Hi Kishanthan,
>>>>>
>>>>> Yes, we may able to use that scripts to fulfil this requirement.
>>>>> Actually what i wanted to know is whether we have inbuilt support from
>>>>> carbon for this requirement and thank you for clarifying.
>>>>>
>>>>> Hi Prabath/Deep,
>>>>>
>>>>> Do you have any idea on how cassandra handle the dynamic memory
>>>>> allocation for the product. I think , it will be a pain if we ask the user
>>>>> to change wso2server.sh or wso2server.bat if they want to start MB or SS
>>>>> with cassandra profile since with default memory allocation will not be
>>>>> enough for cassandra.
>>>>>
>>>>
>>>> We can add following logic use in cassandra-env.sh to Cassandra profile
>>>> startup section to calculate the heap sized.
>>>>
>>>> calculate_heap_sizes()
>>>> {
>>>> case "`uname`" in
>>>> Linux)
>>>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>>>> system_cpu_cores=`egrep -c 'processor([[:space:]]+):.*'
>>>> /proc/cpuinfo`
>>>> ;;
>>>> FreeBSD)
>>>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print
>>>> $2}'`
>>>> system_memory_in_mb=`expr $system_memory_in_bytes / 1024 /
>>>> 1024`
>>>> system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
>>>> ;;
>>>> SunOS)
>>>> system_memory_in_mb=`prtconf | awk '/Memory size:/ {print
>>>> $3}'`
>>>> system_cpu_cores=`psrinfo | wc -l`
>>>> ;;
>>>> Darwin)
>>>> system_memory_in_bytes=`sysctl hw.memsize | awk '{print
>>>> $2}'`
>>>> system_memory_in_mb=`expr $system_memory_in_bytes / 1024 /
>>>> 1024`
>>>> system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
>>>> ;;
>>>> *)
>>>> # assume reasonable defaults for e.g. a modern desktop or
>>>> # cheap server
>>>> system_memory_in_mb="2048"
>>>> system_cpu_cores="2"
>>>> ;;
>>>> esac
>>>>
>>>> # some systems like the raspberry pi don't report cores, use at
>>>> least 1
>>>> if [ "$system_cpu_cores" -lt "1" ]
>>>> then
>>>> system_cpu_cores="1"
>>>> fi
>>>>
>>>> # set max heap size based on the following
>>>> # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
>>>> # calculate 1/2 ram and cap to 1024MB
>>>> # calculate 1/4 ram and cap to 8192MB
>>>> # pick the max
>>>> half_system_memory_in_mb=`expr $system_memory_in_mb / 2`
>>>> quarter_system_memory_in_mb=`expr $half_system_memory_in_mb / 2`
>>>> if [ "$half_system_memory_in_mb" -gt "1024" ]
>>>> then
>>>> half_system_memory_in_mb="1024"
>>>> fi
>>>> if [ "$quarter_system_memory_in_mb" -gt "8192" ]
>>>> then
>>>> quarter_system_memory_in_mb="8192"
>>>> fi
>>>> if [ "$half_system_memory_in_mb" -gt "$quarter_system_memory_in_mb"
>>>> ]
>>>> then
>>>> max_heap_size_in_mb="$half_system_memory_in_mb"
>>>> else
>>>> max_heap_size_in_mb="$quarter_system_memory_in_mb"
>>>> fi
>>>> MAX_HEAP_SIZE="${max_heap_size_in_mb}M"
>>>>
>>>> # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4
>>>> * heap size)
>>>> max_sensible_yg_per_core_in_mb="100"
>>>> max_sensible_yg_in_mb=`expr $max_sensible_yg_per_core_in_mb "*"
>>>> $system_cpu_cores`
>>>>
>>>> desired_yg_in_mb=`expr $max_heap_size_in_mb / 4`
>>>>
>>>> if [ "$desired_yg_in_mb" -gt "$max_sensible_yg_in_mb" ]
>>>> then
>>>> HEAP_NEWSIZE="${max_sensible_yg_in_mb}M"
>>>> else
>>>> HEAP_NEWSIZE="${desired_yg_in_mb}M"
>>>> fi
>>>> }
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Deependra.
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> Thanks
>>>>> Shammi
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks
>>>>>>> Shammi
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Deependra.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Shammi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 8, 2014 at 11:03 AM, Nuwan Silva <[email protected]>wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> With WSO2 MB 2.2.0 we have introduced profiles to start an
>>>>>>>>>> internal Apache Cassandra and Zookeeper. While starting with
>>>>>>>>>> Cassandra
>>>>>>>>>> profile this starts a carbon server with Cassandra. According to the
>>>>>>>>>> MB
>>>>>>>>>> Tuning guide [1] we can set the MAX_HEAP_SIZE for Apache
>>>>>>>>>> Cassandra in cassandra-env.sh.
>>>>>>>>>>
>>>>>>>>>> AFAIK we do not pack this configuration file with WSO2 MB 2.2.0.
>>>>>>>>>> Instead cant we set the required memory parameters inside
>>>>>>>>>> wso2server.sh it
>>>>>>>>>> self? any idea where we can set the required memory?
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://docs.google.com/a/wso2.com/document/d/11Vpu2CQUyE-ZcEH-Y7uFI-MN6a9Dsx5_8m7s4NK0LMA/edit#
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Nuwan Silva*
>>>>>>>>>> *Senior Software Engineer - QA*
>>>>>>>>>> Mobile: +94779804543
>>>>>>>>>>
>>>>>>>>>> WSO2 Inc.
>>>>>>>>>> lean . enterprise . middlewear.
>>>>>>>>>> http://www.wso2.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>>
>>>>>>>>> * Shammi Jayasinghe*
>>>>>>>>> Associate Tech Lead
>>>>>>>>> WSO2, Inc.; http://wso2.com,
>>>>>>>>> mobile: +94 71 4493085
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Deependra Ariyadewa
>>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org
>>>>>>>>
>>>>>>>> email [email protected]; cell +94 71 403 5996 ;
>>>>>>>> Blog http://risenfall.wordpress.com/
>>>>>>>> PGP info: KeyID: 'DC627E6F'
>>>>>>>>
>>>>>>>> *WSO2 - Lean . Enterprise . Middleware*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>>
>>>>>>> * Shammi Jayasinghe*
>>>>>>> Associate Tech Lead
>>>>>>> WSO2, Inc.; http://wso2.com,
>>>>>>> mobile: +94 71 4493085
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Kishanthan Thangarajah*
>>>>>> Senior Software Engineer,
>>>>>> Platform Technologies Team,
>>>>>> WSO2, Inc.
>>>>>> lean.enterprise.middleware
>>>>>>
>>>>>> Mobile - +94773426635
>>>>>> Blog - *http://kishanthan.wordpress.com
>>>>>> <http://kishanthan.wordpress.com>*
>>>>>> Twitter - *http://twitter.com/kishanthan
>>>>>> <http://twitter.com/kishanthan>*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>>
>>>>> * Shammi Jayasinghe*
>>>>> Associate Tech Lead
>>>>> WSO2, Inc.; http://wso2.com,
>>>>> mobile: +94 71 4493085
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Deependra Ariyadewa
>>>> WSO2, Inc. http://wso2.com/ http://wso2.org
>>>>
>>>> email [email protected]; cell +94 71 403 5996 ;
>>>> Blog http://risenfall.wordpress.com/
>>>> PGP info: KeyID: 'DC627E6F'
>>>>
>>>> *WSO2 - Lean . Enterprise . Middleware*
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> * Shammi Jayasinghe*
>>> Associate Tech Lead
>>> WSO2, Inc.; http://wso2.com,
>>> mobile: +94 71 4493085
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list
>>> [email protected]
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>
>
> --
> Prabath Abeysekara
> Associate Technical Lead, Data TG.
> WSO2 Inc.
> Email: [email protected]
> Mobile: +94774171471
>
> _______________________________________________
> Dev mailing list
> [email protected]
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Indika Sampath
Software Engineer
WSO2 Inc.
http://wso2.com
Phone: +94 716 424 744
Blog: http://indikasampath.blogspot.com/
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
