Perhaps the "unlimited" keyword is what is not working? What should be
specified for Unlimited?


Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota

On Mon, Apr 25, 2016 at 9:48 PM, haosdent <[email protected]> wrote:

> According my test, it works in my side.
>
> * Before add it to /etc/init/mesos-slave.conf
>
> ```
> cat /proc/16550/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max cpu time              unlimited            unlimited            seconds
> Max file size             unlimited            unlimited            bytes
> Max data size             unlimited            unlimited            bytes
> Max stack size            10485760             unlimited            bytes
> Max core file size        0                    unlimited            bytes
> Max resident set          unlimited            unlimited            bytes
> Max processes             63655                63655
>  processes
> Max open files            8192                 8192                 files
> Max locked memory         65536                65536                bytes
> Max address space         unlimited            unlimited            bytes
> Max file locks            unlimited            unlimited            locks
> Max pending signals       63655                63655                signals
> Max msgqueue size         819200               819200               bytes
> Max nice priority         0                    0
> Max realtime priority     0                    0
> Max realtime timeout      unlimited            unlimited            us
> ```
>
> * After add it to /etc/init/mesos-slave.conf
>
> ```
> description "mesos slave"
>
> # I didn't use ulimit because I have already set ulimit globally.
> limit fsize 20001 20001
> limit nofile 20000 20000
> ```
>
> ```
> cat /proc/16602/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max cpu time              unlimited            unlimited            seconds
> Max file size             20001                20001                bytes
>   # <- Have changed
> Max data size             unlimited            unlimited            bytes
> Max stack size            10485760             unlimited            bytes
> Max core file size        0                    unlimited            bytes
> Max resident set          unlimited            unlimited            bytes
> Max processes             63655                63655
>  processes
> Max open files            8192                 8192                 files
> Max locked memory         65536                65536                bytes
> Max address space         unlimited            unlimited            bytes
> Max file locks            unlimited            unlimited            locks
> Max pending signals       63655                63655                signals
> Max msgqueue size         819200               819200               bytes
> Max nice priority         0                    0
> Max realtime priority     0                    0
> Max realtime timeout      unlimited            unlimited            us
> ```
>
> On Tue, Apr 26, 2016 at 3:51 AM, June Taylor <[email protected]> wrote:
>
>> Hello. We are running it as root, and it is able to specify the ulimit
>> for open files, as noted in our config file, but it is not setting the File
>> Size limit, which is remaining at 8MB.
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Mon, Apr 25, 2016 at 2:50 PM, Dick Davies <[email protected]>
>> wrote:
>>
>>> Hi June
>>>
>>> are you running Mesos as root, or a non-privileged user? Non-root
>>> won't be able to up their own ulimit too high
>>> (sorry, not an upstart expert as RHELs is laughably incomplete).
>>>
>>> On 25 April 2016 at 19:15, June Taylor <[email protected]> wrote:
>>> > What I'm saying is even putting them within the upstart script, per the
>>> > Mesos documentation, isn't working for the file block limit. We're
>>> still
>>> > getting 8MB useable, and as a result executors fail when attempting to
>>> write
>>> > larger files.
>>> >
>>> >
>>> > Thanks,
>>> > June Taylor
>>> > System Administrator, Minnesota Population Center
>>> > University of Minnesota
>>> >
>>> > On Mon, Apr 25, 2016 at 11:53 AM, haosdent <[email protected]> wrote:
>>> >>
>>> >> If you set in your upstart script, it isn't system wide and only
>>> effective
>>> >> in that session. I think need change /etc/security/limits.conf and
>>> >> /etc/sysctl.conf to make your ulimit work globally.
>>> >>
>>> >> On Tue, Apr 26, 2016 at 12:43 AM, June Taylor <[email protected]> wrote:
>>> >>>
>>> >>> Somewhere an 8MB maximum file size is being applied on just one of
>>> our
>>> >>> slaves, for example.
>>> >>>
>>> >>>
>>> >>> Thanks,
>>> >>> June Taylor
>>> >>> System Administrator, Minnesota Population Center
>>> >>> University of Minnesota
>>> >>>
>>> >>> On Mon, Apr 25, 2016 at 11:42 AM, June Taylor <[email protected]> wrote:
>>> >>>>
>>> >>>> We are operating a 6-node cluster running on Ubuntu, and have
>>> noticed
>>> >>>> that the ulimit settings within the slave context are difficult to
>>> set and
>>> >>>> predict.
>>> >>>>
>>> >>>> The documentation is a bit unclear on this point, as well.
>>> >>>>
>>> >>>> We have had some luck adding a configuration line to
>>> >>>> /etc/init/mesos-slave.conf as follows:
>>> >>>> limit nofile 20000 20000
>>> >>>> limit fsize unlimited unlimited
>>> >>>>
>>> >>>> The nofile limit seems to be respected, however the fsize limit does
>>> >>>> not.
>>> >>>>
>>> >>>> It is also mysterious that the system-wide limits are not inherited
>>> by
>>> >>>> the slave process. We would prefer to set all of these system-wide
>>> and have
>>> >>>> mesos-slave observe them.
>>> >>>>
>>> >>>> Can you please advise where you are setting your ulimits for the
>>> >>>> mesos-slave if it is working for you?
>>> >>>>
>>> >>>> Thanks,
>>> >>>> June Taylor
>>> >>>> System Administrator, Minnesota Population Center
>>> >>>> University of Minnesota
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Best Regards,
>>> >> Haosdent Huang
>>> >
>>> >
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Reply via email to