Re: [CentOS] Promo Store is now open

[Marian Marinov]

Hi guys,

Three things,
1st I would like to sponsor some stickers. Karan, you can reach me on Jabber if 
you want.

2nd I would also like to offer free hosting for the promo site.

3rd I can offer a storage area for delivery of TShirts, but it is in 
Bulgaria(Eastern Europe).


Regards,
Marian



signature.asc
Description: OpenPGP digital signature
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Are file system mounts costly?

[Marian Marinov]

On Wednesday 03 August 2011 23:29:48 Reynolds McClatchey wrote:
> I recall a kernel parameter on Unix System V of number
>  of mounted file systems. Max recommended was 8.
>  Larger numbers slowed down inode location and
>  impacted performance.
> 
> Has Linux solved that bottleneck? Are 20 or 30 mounted
>  efs and cifs file systems on one system OK?

I'm running servers with almost 1500 bind mounts... 
Before that I did check when the server performance is impacted.

So what I found was that when you reach around 2 mounts all commands start 
to take a long time to execute.

 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] lots of small files in a folder on Linux centos

[Marian Marinov]

On Sunday 24 July 2011 22:48:23 Always Learning wrote:
> > On Sun, Jul 24, 2011 at 5:13 PM, R P Herrold  wrote:
> > > then, we look to the leading letter of the hask, to design our
> > > egg carton bins.  We place pix1.jpg in directory: ./f/ and
> > > pix2.jpg in directory ./1/ and pix3.jpg in directory
> > > ./b/ and so forth -- if the directories get too full again,
> > > you might go to using the first two letters of the hash to
> > > perform the 'binning' process
> 
> If the pictures are named sequentially, why not store then at a 100 per
> directory structure something like this
> 
> /pix/0/00/pix1.jpg
> 
> /pix/0/26/pix02614.jpg
> 
> /pix/6/72/pix67255.jpg

As I have worked on projects where the 'coder' is not willing to do any 
changes, I offer you another temporary solution:

If the pictures are in /home/site/public_html/images, you simply need to  
create a tmpfs, copy the pictures there and then bind mount the tmpfs in that 
directory:

# mkdir /home/site/ram
# mount -t tmpfs -o size=200M none /home/site/ram
# cp -a /home/site/public_html/images/* /home/site/ram
# mount --bind /home/site/ram /home/site/public_html/images

Instant performance gain, while you wait for the coder to actually fix the 
problem. 

However you should make sure that you copy the new images from the ram to 
disk. Maybe with inotifywatch.

Keep in mind that this is only a temporary solution that should serve only as 
a proof that this is the problem and it needs to be fixed. Try to explain that 
this hack is not an actual solution.

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] managing a rack full of centos servers

[Marian Marinov]

On Thursday 21 July 2011 18:36:17 Devin Reade wrote:
> --On Wednesday, July 20, 2011 11:02:42 PM -0700 RC 
> 
> wrote:
> > On Wed, 20 Jul 2011 10:07:06 -0600 Devin Reade  wrote:
> >> It should be considered as complementing the automated config
> >> management tools like cfengine et al, not as a replacement for
> >> them (they're doing different jobs).
> > 
> > That's not entirely fair.  A little shell scripting and pdsh and pdcp
> > can certainly do everything cfengine/puppet can do
> 
> I wasn't referring to pdsh/pdcp; I was referring to pconsole.  The
> reason I said complementing is that sometimes it is good to have
> stuff under a configuration management system like cfengine/puppet,
> but sometimes you need to run ad-hoc commands, in an identical
> fashion, on lots of similar machines, which pconsole is good at
> (subject to the caveats I previously mentioned).
> 
> I made no comments on pdsh/pdcp at all, and make no claims on where
> it fits in the spectrum.
> 
> Devin
> 
You can actually achieve the same functionality of pdsh/pdcp and pconsole with 
a quite simple bash script :)

  http://multy-command.sourceforge.net/

I think it is a matter of what the admin will prefer to do. When you have a 
lot of identical machines, sometimes it is better to have cfengine/puppet, but 
sometimes it just an overkill to use them if you are the only one 
administrating those machines.

cfengine and puppet have a very good place on machines that are administered 
by a team of people. 

But solutions like pdsh/pconsole and multy-command, in my opinion are more 
suitable when there are only one or two guys administering those machines. 


Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] tripwire alternative

[Marian Marinov]

On Thursday 21 July 2011 09:27:28 Nguyen Vu Hung (VNC) wrote:
> Hello all,
> 
> Years ago, I used to work with tripwire for system monitoring.
> 
> Last time I checked with "yum search tripwire", there is no hit.
> IIRC, it used to be packed by default on older Redhat distros.
> 
> Any suggestion for an alternative of tripwire for my CentOS 5.6?
> 
> Cheers,

Previously I have used tripwire but for a few years now I have moved to 
Samhain - http://www.la-samhna.de/samhain/s_faq.html 

You can find an installation howto here:
  http://www.howtoforge.com/host-based-intrusion-detection-samhain

The tool is quite usefull but sometimes heavy on the central machine, once you 
go over 100 nodes.

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Mail Question

[Marian Marinov]

On Monday 27 June 2011 21:25:42 Bo Lynch wrote:
> Hello everyone,
> 
> Im having a issue that I just cant seem to figure out. We currently are
> running an email server Centos 5.6 Postfix/Dovecot with a squirrelmail
> frontend. I was setting up a testbed to do a migration to SoGo using
> LDAP/MYSQL/POSTFIX/Dovecot. I have everything up and functional but one
> thing is giving me an issue. The testbed box will not handle a users
> mailbox over 2gb. We are using mbox on the existing server with no issues
> at all. I have multiple users with inboxes over 2gb and and archive mbox
> file with over 50gb. I have check postfix and made sure that the
> mailbox_size_limit = 0
> Both are running i686 as returned by the arch command and both are Centos
> version 5.6.
> Any ideas would be greatly appreciated.

Did you checked the ulimits? Try adding:
  ulimit -f unlimited
to the postfix/dovecot init scripts.
Also if you are chrooting postfix or dovecot, check the limits in 
/etc/security/limits.conf.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] iptables port forwarding

[Marian Marinov]

On Monday 27 June 2011 07:15:33 muiz wrote:
> Marian,  I'm very happy you're online :)I think I have try the record you
> mention just now. And I would like to clear what I have done (the scripts
> I test):/sbin/iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080
> --to a.b.c.d:8181 /sbin/iptables -t nat -A POSTROUTING -j SNAT -s
> 192.168.0.0/255.255.255.0 --to 192.168.1.250 echo 1 >
> /proc/sys/net/ipv4/ip_fowardThen it's not to work!

You have to have some other iptables rules that block the traffic since this 
has 
to work.

Marian

> At 2011-06-27，"Marian Marinov"  wrote:
> >On Monday 27 June 2011 06:50:27 muiz wrote:
> >> Dear Marian and all,
> >> 
> >>   It seems don't works:
> >> /sbin/iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >> a.b.c.d:8181 /sbin/iptables -t nat -A POSTROUTING -j SNAT -s
> >> 192.168.0.0/255.255.255.0 --to a.b.c.d echo 1 >
> >> /proc/sys/net/ipv4/ip_foward
> >
> >Yup, its normal not to work... You got the SNAT rule wrong :)
> >
> >It should be to the IP of the server that is DOING the forwarding...
> >
> >so
> >
> >/sbin/iptables -t nat -A POSTROUTING -j SNAT -s 192.168.0.0/255.255.255.0
> >--to 192.168.1.250
> >
> >Marian
> >
> >> I check the Fedora iptables setting:  /etc/sysconfig/iptables files:
> >> ...
> >> 
> >> :POSTROUTING ACCEPT [0:0]
> >> 
> >> -A PREROUTING -i eth+ -p tcp --dport 8080 -j DNAT --to-destination
> >> a.b.c.d:8080 
> >> 
> >> :OUTPUT ACCEPT [0:0]
> >> 
> >> -A FORWARD -i eth+ -m state --state NEW -m tcp -p tcp -d a.b.c.d --dport
> >> 8080 -j ACCEPT
> >> 
> >> 
> >> And more rules I add is :
> >> /sbin/iptables -t nat -A POSTROUTING -d  a.b.c.d -p tcp --dport 8080 -j
> >> MASQUERADE
> >> 
> >> 
> >> Then it works!  But if I don't use system-config-firewall GUI tools,
> >> then how?
> >> 
> >> 
> >> 
> >> 
> >> Thanks very much !
> >> 
> >> At 2011-06-27，"Marian Marinov"  wrote:
> >> >On Monday 27 June 2011 00:08:08 muiz wrote:
> >> >> Thanks  Marian,
> >> >> The server only has one IP. I think I should add more iptables
> >> >> records, only one NAT record is not enough,isit correct?  If yes ,
> >> >> then how?
> >> >
> >> >Huh, I'm sorry yes you need a second rule. So the rules are:
> >> >iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >> >a.b.c.d:8181 iptables -t nat -A POSTROUTING -j SNAT -s
> >> >local_ip/local_net --to 192.168.1.250
> >> >echo 1 > /proc/sys/net/ipv4/ip_foward
> >> >
> >> >The Source NAT(SNAT) rule is needed, cause otherwise the packaets that
> >> >reach a.b.c.d will be comming from the ip of the local client not
> >> >192.168.1.250 and so 192.168.1.250 will never receive the replies from
> >> >a.b.c.d.
> >> >Since the packets reach the client directly from a.b.c.d, the client
> >> >will simply disregard them and will wait for packets comming from
> >> >.1.250.
> >> >
> >> >So the SNAT rule changes the SOURCE IP of the packets to 1.250 so
> >> >a.b.c.d will return the answares to the right source.
> >> >
> >> >Marian
> >> >
> >> >>  2011-06-26 23:38:58，"Marian Marinov"  wrote：
> >> >>  
> >> >> >On Sunday 26 June 2011 12:53:07 muiz wrote:
> >> >> >> Dear all,
> >> >> >> 
> >> >> >>   I would like to forward a port to an internet server, but
> >> >> >>   failed. can you
> >> >> >> 
> >> >> >> help me? Server:  eth0: 192.168.1.250, Port: 8080 TCP, CentOS 5.6
> >> >> >> Remote server:   IP: a.b.c.d  Port: 8181
> >> >> >> 
> >> >> >> 
> >> >> >> Forward path:  client1(192.168.1.10) -> 192.168.1.250:8080
> >> >> >> (forward) -> a.b.c.d  Port: 8181
> >> >> >> - In Fedora, I
> >> >> >> successfully to config the firewall using
> >> >> >> system-config-firewall and iptables command: 1. Run
> >> >> >> system-config-firewall
> >> >> >> 
> >> >> >>  1.1 open local port 8080
> >> >> >>  1.2 add a forward rule: local 8080 to remote a.b.c.d:8181, tcp
> >> >> >> 
> >> >> >> 2. echo 1 > /proc/sys/net/ipv4/ip_foward
> >> >> >> 3. add a iptables rule: /sbin/iptables -t nat -A POSTROUTING -d
> >> >> >> a.b.c.d -p tcp --dport 8181 -j MASQUERADE That's all.
> >> >> >> 
> >> >> >> 
> >> >> >> 
> >> >> >> 
> >> >> >> Thanks !
> >> >> >
> >> >> >You have to use Destination NAT for the job:
> >> >> >
> >> >> >iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >> >> >a.b.c.d:8181 echo 1 > /proc/sys/net/ipv4/ip_foward
> >> >> >
> >> >> >If you have more then one IPs on the local machine its a good idea
> >> >> >to specify the destination -d 192.168.1.250
> >> >> >
> >> >> >Marian
> >> >> 
> >> >> ___
> >> >> CentOS mailing list
> >> >> CentOS@centos.org
> >> >> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] iptables port forwarding

[Marian Marinov]

On Monday 27 June 2011 06:50:27 muiz wrote:
> Dear Marian and all,
>   It seems don't works:
> /sbin/iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> a.b.c.d:8181 /sbin/iptables -t nat -A POSTROUTING -j SNAT -s
> 192.168.0.0/255.255.255.0 --to a.b.c.d echo 1 >
> /proc/sys/net/ipv4/ip_foward

Yup, its normal not to work... You got the SNAT rule wrong :)

It should be to the IP of the server that is DOING the forwarding...

so 

/sbin/iptables -t nat -A POSTROUTING -j SNAT -s 192.168.0.0/255.255.255.0 --to 
192.168.1.250

Marian

> 
> 
> I check the Fedora iptables setting:  /etc/sysconfig/iptables files:
> ...
> 
> :POSTROUTING ACCEPT [0:0]
> 
> -A PREROUTING -i eth+ -p tcp --dport 8080 -j DNAT --to-destination
> a.b.c.d:8080 
> 
> :OUTPUT ACCEPT [0:0]
> 
> -A FORWARD -i eth+ -m state --state NEW -m tcp -p tcp -d a.b.c.d --dport
> 8080 -j ACCEPT
> 
> 
> And more rules I add is :
> /sbin/iptables -t nat -A POSTROUTING -d  a.b.c.d -p tcp --dport 8080 -j
> MASQUERADE
> 
> 
> Then it works!  But if I don't use system-config-firewall GUI tools, then
> how?
> 
> 
> 
> 
> Thanks very much !
> 
> At 2011-06-27，"Marian Marinov"  wrote:
> >On Monday 27 June 2011 00:08:08 muiz wrote:
> >> Thanks  Marian,
> >> The server only has one IP. I think I should add more iptables records,
> >> only one NAT record is not enough,isit correct?  If yes , then how?
> >
> >Huh, I'm sorry yes you need a second rule. So the rules are:
> >iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >a.b.c.d:8181 iptables -t nat -A POSTROUTING -j SNAT -s local_ip/local_net
> >--to 192.168.1.250
> >echo 1 > /proc/sys/net/ipv4/ip_foward
> >
> >The Source NAT(SNAT) rule is needed, cause otherwise the packaets that
> >reach a.b.c.d will be comming from the ip of the local client not
> >192.168.1.250 and so 192.168.1.250 will never receive the replies from
> >a.b.c.d.
> >Since the packets reach the client directly from a.b.c.d, the client will
> >simply disregard them and will wait for packets comming from .1.250.
> >
> >So the SNAT rule changes the SOURCE IP of the packets to 1.250 so a.b.c.d
> >will return the answares to the right source.
> >
> >Marian
> >
> >>  2011-06-26 23:38:58，"Marian Marinov"  wrote：
> >>  
> >> >On Sunday 26 June 2011 12:53:07 muiz wrote:
> >> >> Dear all,
> >> >> 
> >> >>   I would like to forward a port to an internet server, but failed.
> >> >>   can you
> >> >> 
> >> >> help me? Server:  eth0: 192.168.1.250, Port: 8080 TCP, CentOS 5.6
> >> >> Remote server:   IP: a.b.c.d  Port: 8181
> >> >> 
> >> >> 
> >> >> Forward path:  client1(192.168.1.10) -> 192.168.1.250:8080 (forward)
> >> >> -> a.b.c.d  Port: 8181 - In
> >> >> Fedora, I successfully to config the firewall using
> >> >> system-config-firewall and iptables command: 1. Run
> >> >> system-config-firewall
> >> >> 
> >> >>  1.1 open local port 8080
> >> >>  1.2 add a forward rule: local 8080 to remote a.b.c.d:8181, tcp
> >> >> 
> >> >> 2. echo 1 > /proc/sys/net/ipv4/ip_foward
> >> >> 3. add a iptables rule: /sbin/iptables -t nat -A POSTROUTING -d 
> >> >> a.b.c.d -p tcp --dport 8181 -j MASQUERADE That's all.
> >> >> 
> >> >> 
> >> >> 
> >> >> 
> >> >> Thanks !
> >> >
> >> >You have to use Destination NAT for the job:
> >> >
> >> >iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >> >a.b.c.d:8181 echo 1 > /proc/sys/net/ipv4/ip_foward
> >> >
> >> >If you have more then one IPs on the local machine its a good idea to
> >> >specify the destination -d 192.168.1.250
> >> >
> >> >Marian
> >> 
> >> ___
> >> CentOS mailing list
> >> CentOS@centos.org
> >> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] iptables port forwarding

[Marian Marinov]

On Monday 27 June 2011 00:08:08 muiz wrote:
> Thanks  Marian,
> The server only has one IP. I think I should add more iptables records,
> only one NAT record is not enough,isit correct?  If yes , then how?

Huh, I'm sorry yes you need a second rule. So the rules are:
iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to a.b.c.d:8181
iptables -t nat -A POSTROUTING -j SNAT -s local_ip/local_net --to 
192.168.1.250
echo 1 > /proc/sys/net/ipv4/ip_foward

The Source NAT(SNAT) rule is needed, cause otherwise the packaets that reach 
a.b.c.d will be comming from the ip of the local client not 192.168.1.250 and 
so 192.168.1.250 will never receive the replies from a.b.c.d.
Since the packets reach the client directly from a.b.c.d, the client will 
simply disregard them and will wait for packets comming from .1.250.

So the SNAT rule changes the SOURCE IP of the packets to 1.250 so a.b.c.d will 
return the answares to the right source.

Marian

> 
> 
>  2011-06-26 23:38:58，"Marian Marinov"  wrote：
> 
> >On Sunday 26 June 2011 12:53:07 muiz wrote:
> >> Dear all,
> >> 
> >>   I would like to forward a port to an internet server, but failed. can
> >>   you
> >> 
> >> help me? Server:  eth0: 192.168.1.250, Port: 8080 TCP, CentOS 5.6
> >> Remote server:   IP: a.b.c.d  Port: 8181
> >> 
> >> 
> >> Forward path:  client1(192.168.1.10) -> 192.168.1.250:8080 (forward) ->
> >> a.b.c.d  Port: 8181 -
> >> In Fedora, I successfully to config the firewall using
> >> system-config-firewall and iptables command: 1. Run
> >> system-config-firewall
> >> 
> >>  1.1 open local port 8080
> >>  1.2 add a forward rule: local 8080 to remote a.b.c.d:8181, tcp
> >> 
> >> 2. echo 1 > /proc/sys/net/ipv4/ip_foward
> >> 3. add a iptables rule: /sbin/iptables -t nat -A POSTROUTING -d  a.b.c.d
> >> -p tcp --dport 8181 -j MASQUERADE That's all.
> >> 
> >> 
> >> 
> >> 
> >> Thanks !
> >
> >You have to use Destination NAT for the job:
> >
> >iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to
> >a.b.c.d:8181 echo 1 > /proc/sys/net/ipv4/ip_foward
> >
> >If you have more then one IPs on the local machine its a good idea to
> >specify the destination -d 192.168.1.250
> >
> >Marian
> 
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] iptables port forwarding

[Marian Marinov]

On Sunday 26 June 2011 12:53:07 muiz wrote:
> Dear all,
>   I would like to forward a port to an internet server, but failed. can you
> help me? Server:  eth0: 192.168.1.250, Port: 8080 TCP, CentOS 5.6
> Remote server:   IP: a.b.c.d  Port: 8181
> 
> 
> Forward path:  client1(192.168.1.10) -> 192.168.1.250:8080 (forward) ->
> a.b.c.d  Port: 8181 -
> In Fedora, I successfully to config the firewall using
> system-config-firewall and iptables command: 1. Run system-config-firewall
>  1.1 open local port 8080
>  1.2 add a forward rule: local 8080 to remote a.b.c.d:8181, tcp
> 2. echo 1 > /proc/sys/net/ipv4/ip_foward
> 3. add a iptables rule: /sbin/iptables -t nat -A POSTROUTING -d  a.b.c.d -p
> tcp --dport 8181 -j MASQUERADE That's all.
> 
> 
> 
> 
> Thanks !
You have to use Destination NAT for the job:

iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 8080 --to a.b.c.d:8181
echo 1 > /proc/sys/net/ipv4/ip_foward

If you have more then one IPs on the local machine its a good idea to specify 
the destination -d 192.168.1.250 

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] ext4 in CentOS 5.6?

[Marian Marinov]

On Friday 24 June 2011 04:34:20 Smithies, Russell wrote:
> We have a single 27TB partition (35 x 1TB drives as RAID5+0 in an HP
> MDS600), just formatted it xfs and had no problems with it so far. It's
> used as scratch space so not too concerned about performance.
> 
> --Russell
> 

I have compared the performance of both XFS and Ext4. And since I use those 
big machines for backups, for me the write performance was very important. 
XFS was almost twice slower.  

But lets leave XFS alone :) Ext4 is the way to go :)

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] ext4 in CentOS 5.6?

[Marian Marinov]

On Thursday 23 June 2011 22:41:50 PJ wrote:
> On Thu, Jun 23, 2011 at 12:31 PM, PJ  wrote:
> > On Thu, Jun 23, 2011 at 1:07 PM, Marian Marinov  wrote:
> >> On Thursday 23 June 2011 19:16:37 PJ wrote:
> >>> I'm sure many are running ext4 FS's in production, but just want to be
> >>> re-assured that there are not currently any major issues before
> >>> starting a new project that looks like it will be using ext4.
> >>> 
> >>> I've previously been using xfs but the software for this project
> >>> requires ext3/ext4.
> >>> 
> >>> I'm always very cautious before jumping onto a new FS, (new in the
> >>> sense it is officially supported now)
> >>> 
> >>> Thanks in advance!
> >> 
> >> I'm running some 50 servers with ext4 each server has 2x15TB ext4
> >> partitions. I haven't had an issue with that setup. The first server
> >> was setup 3 years ago. It is quite faster then XFS in terms of write
> >> performance and thus far reliable without any major problem.
> >> 
> >> Keep in mind that user land tools are limited and the biggest partition
> >> you can create with them at the moment is 16TB. You can recompile the
> >> tools and remove this limitation if that is a problem for you.
> >> 
> >> Regards,
> >> Marian Marinov
> > 
> > Thanks for all the great replies everyone.
> > 
> > I've got an 18TB partition - the limit is 16TB even in x86_64?
> 
> Answering my own question yes, 16TB is the limit.
> Has anyone here successfully compiled their own version of e2fsprogs
> that works over 16TB?
> 
> Looking at https://ext4.wiki.kernel.org/index.php/Ext4_Howto it says:
> "The code to create file systems bigger than 16 TiB is, at the time of
> writing this article, not in any stable release of e2fsprogs. It will
> be in future releases."
> 
> Not sure if the wiki is out of date or not...

What I have seen is only a alpha/beta quality code that adds this 
functionality.

I would not suggest that you use those patches. At least not on a production 
machine. I only wanted to mention that there is such code... not that it is 
actually working :)

Marian
> 
> Thanks!
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] ext4 in CentOS 5.6?

[Marian Marinov]

On Thursday 23 June 2011 22:31:28 PJ wrote:
> On Thu, Jun 23, 2011 at 1:07 PM, Marian Marinov  wrote:
> > On Thursday 23 June 2011 19:16:37 PJ wrote:
> >> I'm sure many are running ext4 FS's in production, but just want to be
> >> re-assured that there are not currently any major issues before
> >> starting a new project that looks like it will be using ext4.
> >> 
> >> I've previously been using xfs but the software for this project
> >> requires ext3/ext4.
> >> 
> >> I'm always very cautious before jumping onto a new FS, (new in the
> >> sense it is officially supported now)
> >> 
> >> Thanks in advance!
> > 
> > I'm running some 50 servers with ext4 each server has 2x15TB ext4
> > partitions. I haven't had an issue with that setup. The first server was
> > setup 3 years ago. It is quite faster then XFS in terms of write
> > performance and thus far reliable without any major problem.
> > 
> > Keep in mind that user land tools are limited and the biggest partition
> > you can create with them at the moment is 16TB. You can recompile the
> > tools and remove this limitation if that is a problem for you.
> > 
> > Regards,
> > Marian Marinov
> 
> Thanks for all the great replies everyone.
> 
> I've got an 18TB partition - the limit is 16TB even in x86_64?

Yes. At least it was so, last year. I haven't checked recently. And I don't 
have a spare machine to repartition for the test. 
We have a 30TB RAID6 array and I was really annoyed that I had to make two 
partitions to utilze the whole space.

The wiki pages are still not updated: 
  http://en.wikipedia.org/wiki/Comparison_of_file_systems 
  https://ext4.wiki.kernel.org/index.php/Ext4_Howto

NOTE: Although very large fileystems are on ext4's feature list, current 
e2fsprogs currently still limits the filesystem size to 2^32 blocks (16TiB for 
a 4KiB block filesystem). Allowing filesystems larger than 16T is one of the 
very next high-priority features to complete for ext4. 



> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] ext4 in CentOS 5.6?

[Marian Marinov]

On Thursday 23 June 2011 19:16:37 PJ wrote:
> I'm sure many are running ext4 FS's in production, but just want to be
> re-assured that there are not currently any major issues before
> starting a new project that looks like it will be using ext4.
> 
> I've previously been using xfs but the software for this project
> requires ext3/ext4.
> 
> I'm always very cautious before jumping onto a new FS, (new in the
> sense it is officially supported now)
> 
> Thanks in advance!

I'm running some 50 servers with ext4 each server has 2x15TB ext4 partitions. 
I haven't had an issue with that setup. The first server was setup 3 years ago. 
It is quite faster then XFS in terms of write performance and thus far 
reliable without any major problem.

Keep in mind that user land tools are limited and the biggest partition you 
can create with them at the moment is 16TB. You can recompile the tools and 
remove this limitation if that is a problem for you.

Regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Using umask

[Marian Marinov]

On Tuesday 21 June 2011 18:27:11 John Hodrien wrote:
> On Tue, 21 Jun 2011, Todd Cary wrote:
> > Grasping a full understanding of setting default Users, Groups
> > and Masks has alluded me over the years, but now I find myself in
> > a situation where manually "setting" the file/directory
> > attributes is becoming a pain.
> > 
> > I understand the fundamentals of the file attributes, though from
> > time to time I have to review the "sticky bit"; what I do not
> > understand is where/how the attributes are set when a user
> > creates or modifies a file/directory.  Here is my situation:
> > 
> > My /var/www/html files have been manually set by me to
> > apache/apache 774.  This allows my PHP applications to access the
> > files, and I assume this is a "good" setting.
> > 
> > Now, my server is connected via Samba to my desktop.  If I create
> > a file, it is todd/todd 744, so Apache cannot access them.
> > 
> > If PHP (Apache) creates or modifies a file, it is apache/apache
> > 755, so I cannot access them (Write/Delete).
> > 
> > Is there a way to resolve this?  When I FTP to a friend's
> > rent-a-server, I can read/write/delete all of the files I have
> > placed there *and* the same for files touched by PHP (Apache).
> > 
> > My Linux Admin books as well as my Linux books do not appear to
> > cover this and/or my experience is lacking.
> 
> Either have a group that you're both a member of and have a SGID bit set on
> the relevent directories using that gruop, or look at ACLs.
> 
> jh

Or you can simply start using mod_suphp or suexec for running your php 
application. 

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] BIND9 - automatic zone definition replication to slave

[Marian Marinov]

On Tuesday 31 May 2011 11:41:47 Lars Hecking wrote:
> Martin Šťastný writes:
> > Hello,
> > 
> > I have simple question - is there a way to automatically replicate zone
> > definition (not zone itself - this is easy) to slave server using BIND9?
> > Is it BIND built-in or are there prebuilt scripts? Or I have to write
> > that script on my own (started by Cron, transfer file with zone names,
> > create conf file and finallly restart BIND?)?
> 
>  Check out incron from rpmforge. Together with rsync and possibly a bit of
>  custom scripting it should be perfect.
> 
>  I would not rsync config files directly into place, but use a staging
> area, watched by incron on the other end, so that a custom script could do
> some integrity checking before applying the update.
> 
> 

It can be done prety easy. If you move all zone definitions that has to be 
replicated into a single file and include it in the main configuration. Then 
you 
only need to rsync/scp that file and do 'rndc reconfig'  on the remote machine.

I'm doing this on our nameservers and it works like a charm. All zones that 
have no slaves are in different config. And I have one file per slave machine, 
which holds the zones that are replicated on it.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] local repo

[Marian Marinov]

On Saturday 28 May 2011 06:03:19 Steven Crothers wrote:
> You'll have to edit your repos in the %post section of your ks. The
> repos are provided by centos-release iirc.
> 
> On Fri, May 27, 2011 at 11:22 AM, Jerry Geis  wrote:
> > Hi all,
> > 
> > When I am installing I use kickstart and have a line like:
> >repo --name=Updates
> > --baseurl=http://192.168.1.14/centos/5.6/updates/x86_64/
> > and that works great for installing the OS.
> > 
> > After that the machine reboots and I have it automatically go into
> > additional installations running scripts.
> > These installations do "yum install XXX".
> > However, its no longer using my above repo its using the mirrorlist (as
> > expected).
> > 
> > My questions are :
> > 
> > 1) I dont see a way in yum to say "use this repo to install", is there a
> > way to point to my server in the office
> > and dont do the mirrorlist.
> > 
> > 2) Do I just drop a file called CentOS-office in the /etc/yum.repos.d
> > directory
> > that looks like this and it will be used first instead of the mirrorlist:
> > 
> > 
> > [base]
> > name=CentOS-$releasever - Base
> > mirrorlist=http://192.168.1.14/?release=$releasever&arch=$basearch&repo=o
> > s #baseurl=http://192.168.1.14/centos/$releasever/os/$basearch/
> > gpgcheck=1
> > gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
> > 
> > #released updates
> > [updates]
> > name=CentOS-$releasever - Updates
> > mirrorlist=http://192.168.1.14/?release=$releasever&arch=$basearch&repo=u
> > pdates #baseurl=http://192.168.1.14/centos/$releasever/updates/$basearch/
> > gpgcheck=1
> > gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
> > 
> > 
> > 
> > I dont want to mess anything up and I want additional package installs
> > in the office to go faster
> > in the office. I can remove the file when I am done installing.
> > 
> > Is there a better way or is this the way to do it?
> > 
> > jerry

Actually Steven is wrong. You can do

yum install --disablerepo http://192.168.1.14//package.rpm

But I agree with Steven, that the best way is to fix your repos in the post 
section.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] FTP Migration

[Marian Marinov]

On Tuesday 24 May 2011 05:24:07 listmail wrote:
> Hi All,
> 
> Please feel free to correct any misconceptions in my premises as I get to
> my question. I have about 6 ftp services running on a CentOS system that
> is going down for service, and I want to move the ftp services to a VM on
> another network. These are all running on Proftpd, with fairly complicated
> directory/permissions/rate control layouts, as proftpd nicely supports.
> 
> First, it appears that RH and CentOS have dropped proftpd since I last
> looked and are now only shipping vsftpd in the repositories.
> 
> Second, I looked at the vsftpd site, and noticed a complete absence of
> documentation (other than a basic bare-bones manpage), so I have no idea if
> vsftpd will support anything that I'm doing with proftpd, or any
> information about how to configure anything.
> 
> Obviously I could just install the latest version of proftpd from source on
> the new host and get on with my life, but is there any reason to bite the
> bullet and try to convert my ftp sites to a new, basically undocumented ftp
> server?
> 
> Any input appreciated, especially on conversions of complicated ftp sites
> from proftpd to vsftpd.
> 
> Thanks,
> --Bill

Bill, the proftpd is currently in the EPEL repository.

Information on additional CentOS repos is available at 
http://wiki.centos.org/AdditionalResources/Repositories Pay attention to the 
reference on yum-priorities.

And vsftpd doesn't have even half of the features that proftpd offers. So it 
will be a fairly complicated migration if you go that way.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Passing password to script for rpmsign of list of .rpm files

[Marian Marinov]

On Friday 20 May 2011 21:11:58 Ljubomir Ljubojevic wrote:
> John Hodrien wrote:
> > On Fri, 20 May 2011, Ljubomir Ljubojevic wrote:
> >> I am trying to automatize signing of unsigned .rpm files. My repo has at
> >> least 50 x 3 packages.
> >> 
> >> But I would have to type numerous passwords for each file. I can not see
> >> hot to pass pass phrase to script.
> >> 
> >> rpmsign --resign {--pass=??}  
> >> 
> >> Can someone advise me how to do that?
> > 
> > http://www.karan.org/blog/index.php/2011/05/06/sign-multiple-rpms-with-on
> > e-command
> 
> Thanks. I am bit behind visiting sites. I have found expect script for
> this but this is much more elegant.
> 
> Many thanks to KB also for solution.
> 

You should also check this:

http://blogs.23.nu/till/2008/12/rpm-addsign-with-gpg-agent/

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] allowing users to write to a web content area

[Marian Marinov]

On Monday 16 May 2011 06:19:49 David Mehler wrote:
> Hello,
> I've got apache running on a centos 5.6 machine. All of my users have
> a umask of 077 set in /etc/bashrc. I'm now wanting to give several of
> them permission to write to a web area so they can place content
> visible to the web server. I've got two groups webdev1 and webdev2
> which I want one to be able to write to site1 and the other to site2.
> I've got between 3 and 5 users in each group. I'd prefer not to mess
> with these users umask settings, but want the correct permissions and
> ownerships user:webdev1 or user:webdev2 where user is the username of
> the person who placed the file. Permissions I believe should be 664 so
> apache can read the files.
> 
> I'm wondering if I need to look in to ACLS which I've not used or if
> there's another solution?
> 
> Thanks.
> Dave.
It seams obvious... add the apache user to both webdev1 and webdev2 groups and 
you are done... no need to change umasks and perms :)

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Apache in chroot reporting every client is 16.0.0.0

[Marian Marinov]

On Saturday 14 May 2011 20:50:54 Jason Pyeron wrote:
> Not sure where to start on this. I went to examine a log file today and
> noticed a password protected internal file was being accessed from
> 16.0.0.0. Upon further review every log entry has the same IP. Accessing
> apache from localhost also reports 16.0.0.0.
> 
> Google is not being my friend right now, any advice?
> 
> Kernel: 2.6.9-89.0.29.Elsmp
> 
> In the chroot:
> 
> httpd-suexec-2.0.52-41.ent.7.centos4
> httpd-devel-2.0.52-41.ent.7.centos4
> httpd-2.0.52-41.ent.7.centos4
> 
> -jason

Check the resolv setup in the chroot. etc/resolv.conf, etc/hosts, 
etc/nsswitch.conf 

Marian

> 
> --
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> -   -
> - Jason Pyeron  PD Inc. http://www.pdinc.us -
> - Principal Consultant  10 West 24th Street #100-
> - +1 (443) 269-1555 x333Baltimore, Maryland 21218   -
> -   -
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> This message is copyright PD Inc, subject to license 20080407P00.
> 
> 
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Modify Parameters at system boot

[Marian Marinov]

On Friday 13 May 2011 07:04:33 Frederick Abrams wrote:
> Hi all.
> 
> i'm trying to modify some parameters but when system reboots it doesn't
> load. For the sysctl if I run sysctl -p then it changes
> 
> /etc/sysctl.conf
>  net.ipv4.netfilter.ip_conntrack_max = 1048576
> 
> /etc/modprobe.conf
>  options ip_conntrack hashsize=131072
> 
> after reboot results
> 
> cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
> 65536
> 
> cat /sys/module/nf_conntrack/parameters/hashsize
> 16384
> 
> expected results
> 
> cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
> 1048576
> 
> cat /sys/module/nf_conntrack/parameters/hashsize
> 131072
> 

It is possible that your iptables modules are loaded after the sysctl.conf is 
executed.
Keep in mind that sysctl.conf is loaded during network startup. 

About the options... at least with kernel  2.6.18.0194.el5 the right option 
is:

options ipt_hashsize 131072

Also keep in mind that you have to remove the '=' sign from modprobe.conf 

Marian
-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Finding wich files a writen to

[Marian Marinov]

On Thursday 05 May 2011 05:24:10 Marcelo Beckmann wrote:
> 2011/5/4 Nicolas Ross :
> > Hi !
> > 
> > I have a server (Centos 5) that is using a pair of SAS drives to store
> > the data. (Mail server) They are on an adaptec raid controler with a
> > battery backup and write back cache active.
> > 
> > >From time to time, I have sever peak io to those data disks (> 400 to
> > >500
> > 
> > iops, > 70 to 100 megs/sec).
> > 
> > With iostat, I find that it's almost a write i/o problem. How can I find
> > to which files the OS writes ? On OSX boxes, there is a utility called
> > fs_usage that can reports any disk activity for a particular process or
> > all processes. Is there any utility like this on Centos ?
> > 
> > iotop can points me to wich process, but that doesn't points me to what
> > files are the culprits...
> 
> I sugest a look for tools like this
> http://freshmeat.net/projects/fsniper
> 
> it helps to make a script to watch file activities, and it uses a kernel
> feature
> 
> I discovered inotify some months ago when I looked into every
> initscript in init.d
> 
> [23:13:35 root@gw init.d]# cat /etc/redhat-release
> CentOS release 5.3 (Final)
> [23:13:45 root@gw init.d]# head restorecond
> #!/bin/sh
> #
> # restorecond:  Daemon used to maintain path file context
> #
> # chkconfig:2345 12 87
> # description:  restorecond uses inotify to look for creation of new files
> \ # listed in the /etc/selinux/restorecond.conf file, and restores the \ #
> correct security context.
> 
> 
> more about inotify:
> http://linux.die.net/man/7/inotify
> 
> http://www.linuxjournal.com/article/8478
> What Is inotify?
> 
> inotify is a file change notification system—a kernel feature that
> allows applications to request the monitoring of a set of files
> against a list of events. When the event occurs, the application is
> notified. To be useful, such a feature must be simple to use,
> lightweight with little overhead and flexible. It should be easy to
> add new watches and painless to receive notification of events.

If you go the inotify route, do keep in mind that you need to monitor for 
modify events, otherwise you would not see the file changes before the 
applications finish with the files.

Regards,
Marian


-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Finding wich files a writen to

[Marian Marinov]

On Wednesday 04 May 2011 21:01:03 Jorge Fábregas wrote:
> On 05/04/2011 12:17 PM, Nicolas Ross wrote:
> > iotop can points me to wich process, but that doesn't points me to what
> > files are the culprits...
> 
> A rough way would be to change to the top-level directory where you
> suspect the files are being written and perform:
> 
> find . -type f -mmin -1 (that would search for all files modified
> within the last minute)
> 
> A more elegant way would be:
> 
> lsof -p PID  (where PID is the process ID...of the process iotop showed
> you)
> 

Just out of curiosity I decided to write a simple script which checks all the 
files from all pids on the system.

Here is what I got:
  http://hydra.azilian.net/scripts/read_fds.pl

The idea is to read all the /proc/PID/fdinfo/ files and check the difference in 
the pos lines (the position in the file descriptor). This is both write and 
read position depending on how the application has opened the file.
So in the end it lists all pids and the respective FDs which have changes:

hackman@gamelon:~$ sudo ./read_fds.pl 4
Pid: 14229 Position change: 22 blocks FD:   4(/home/hackman/f2.tst)
Pid: 14229 Position change: 12 blocks FD:   3(/home/hackman/f1.tst)

The argument to the script is the sleep between the two checks. 
I have tested the script on a few production servers... It works as a charm :)

Thank you for the good question... now I have one good tool in my arsenal :)

 --
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] /etc/bashrc help!

[Marian Marinov]

On Wednesday 20 April 2011 00:26:04 Marian Marinov wrote:
> On Tuesday 19 April 2011 20:18:38 Roland Roland wrote:
> >   Dear all,
> > 
> > i've appended the below to /etc/bashrc it works like a charm with ssh
> > connections though SFTP sessions fail since the below is being sent to
> > the intiator.
> > any way of limiting the below to none sftp sessions? or any other idea
> > for it to work?
> > 
> > 
> > # If id command returns zero, you’ve root access.
> > if [ $(id -u) -eq 0 ];
> > then # you are root, set red colour prompt
> > echo "###"
> > echo "### You are now working as ROOT. ###"
> > echo "### Pay attention to what you type. ###"
> > echo "###"
> > PS1="\\[$(tput setaf 1)\\]\\u@\\h:\\w #\\[$(tput sgr0)\\]"
> > else # normal
> > echo
> > echo " ###"
> > echo "Welcome $(whoami), here's something to start your day with:"
> > echo
> > echo `sh /etc/lines.sh /etc/quotes.txt`
> > echo " "
> > echo
> > PS1="[\\u@\\h:\\w] $"
> > fi
> 
> Rolan, you have two choices:
> 
> 1. Print the whole content on STDERR so you don't disturb the sftp
> 
> 2. if [ "$-" != 'hBc' ]; then echo 'your content here'; fi
> 
> If you go to the second option, the idea there is that $- is set to hBc
> every time you use the shell from SFTP (non-interactive mode). So you echo
> all the things you like only if it is an interactive shell.
> 
> Marian

Just to make things clear, this is from the bash manual:

   An  interactive  shell  is one started without non-option arguments and 
without the -c option whose standard input and error are both connected to 
terminals (as determined by isatty(3)), or one started with the -i option. PS1 
is set and $- includes i if bash is interactive, allowing a shell script or a 
startup file to test this state.

You can also test if you want with these two tests:

  if [[ "$-" =~ 'i' ]]; then echo interactive; fi

  if ( echo $- |grep i > /dev/null ); then echo interactive; fi

Marian


-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] /etc/bashrc help!

[Marian Marinov]

On Tuesday 19 April 2011 20:18:38 Roland Roland wrote:
>   Dear all,
> 
> i've appended the below to /etc/bashrc it works like a charm with ssh
> connections though SFTP sessions fail since the below is being sent to
> the intiator.
> any way of limiting the below to none sftp sessions? or any other idea
> for it to work?
> 
> 
> # If id command returns zero, you’ve root access.
> if [ $(id -u) -eq 0 ];
> then # you are root, set red colour prompt
> echo "###"
> echo "### You are now working as ROOT. ###"
> echo "### Pay attention to what you type. ###"
> echo "###"
> PS1="\\[$(tput setaf 1)\\]\\u@\\h:\\w #\\[$(tput sgr0)\\]"
> else # normal
> echo
> echo " ###"
> echo "Welcome $(whoami), here's something to start your day with:"
> echo
> echo `sh /etc/lines.sh /etc/quotes.txt`
> echo " "
> echo
> PS1="[\\u@\\h:\\w] $"
> fi
> 

Rolan, you have two choices:

1. Print the whole content on STDERR so you don't disturb the sftp

2. if [ "$-" != 'hBc' ]; then echo 'your content here'; fi

If you go to the second option, the idea there is that $- is set to hBc every 
time you use the shell from SFTP (non-interactive mode). So you echo all the 
things you like only if it is an interactive shell.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 17:36:39 John Jasen wrote:
> On 04/12/2011 10:21 AM, Boris Epstein wrote:
> > On Tue, Apr 12, 2011 at 3:36 AM, Alain Péan
> >  
> > > wrote:
> 
> 
> I would chime in with a dis-commendation for XFS. At my previous
> employer, two cases involving XFS resulted in irrecoverable data
> corruption. These were on RAID systems running from 4 to 20 TB.

Can someone(who actually knows) share with us, what is the state of xfs-utils, 
how stable and usable are they for recovery of broken XFS filesystems?

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 16:48:14 Markus Falb wrote:
> On 12.4.2011 15:02, Marian Marinov wrote:
> > On Tuesday 12 April 2011 15:56:54
> > rainer-rnrd0m5o0maboiyizis...@public.gmane.org wrote:
> > 
> > Yes... but with such RAID10 solution you get only half of the disk
> > space... so from 10 2TB drives you get only 10TB instead of 16TB with
> > RAID6.
> 
> From a somewhat theoretical view, this is true for standard raid10 but
> Linux md raid10 is much more flexible as I understood it. You could do 2
> copys over 2 disks, thats like standard 10. Or you could do 2 copys over
> 2 or 3 or ... x disks. Or you could do 3 copys over 3 or 4 or ... x
> disks. Do the math. See the manpage for md(4) and
> http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10
> 
> However, I have to admit that I have no experience with that but would
> like to hear about any disadvantages or if I am mislead. I am just
> interested.
Its like doing RAID50 or RAID60... Again the cheapest solution is RAID6. 
I really like the software raid in linux, it has good performance. But I have 
never tested it on such big volumes. And usually it is really hard to put 10 
or more drives on a machine without buying a sata controler. 

Marian



signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 16:20:22 m.r...@5-cent.us wrote:
> Rudi Ahlers wrote:
> > On Tue, Apr 12, 2011 at 2:47 PM, Marian Marinov  wrote:
> >> I'm managing machines with 30TB of storage for more then two years. And
> >> with good reporting and reaction we have never had to run fsck.
> >> 
> >> However I'm sure that if you have to run fsck on so big file systems, it
> >> will be fater to rebuild the array from other storage then waiting for
> 
> a few
> 
> >> weeks to finish.
> 
> 
> Here's a question: which would be faster on that huge a filesystem: fsck,
> or having a second 30TB filesystem, and rsyncing everything over?

For us, it was faster to transfer the information again. At least this was 
during the tests. We have never had to do it for real. 

I guess the time for the fsck depends on the amount of errors that you have. 
If it has to check only the jurnal the fsck will not take long. But i it has 
to do a full check of the FS... an rsync may be faster.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 15:56:54 rai...@ultra-secure.de wrote:
> > On Tuesday 12 April 2011 15:34:21 Torres, Giovanni (NIH/NINDS) [C] wrote:
> >> On Apr 12, 2011, at 3:23 AM, Matthew Feinberg wrote:
> >> 
> >> ext4 does not seem to be fully baked in 5.6 yet. parted 1.8 does not
> >> support creating ext4 (strange)
> >> 
> >> The CentOS homepage states that ext4 is now a fully supported filesystem
> >> in
> >> 5.6. ___
> >> CentOS mailing list
> >> CentOS@centos.org
> >> http://lists.centos.org/mailman/listinfo/centos
> > 
> > Steve,
> > I'm managing machines with 30TB of storage for more then two years. And
> > with
> > good reporting and reaction we have never had to run fsck.
> 
> That's not the issue.
> The issue is rebuild-time.
> The longer it takes, the more likely is another failure in the array.
> With RAID6, this does not instantly kill your RAID, as with RAID5 - but I
> assume it will further decrease overall-performance and the rebuild-time
> will go up significantly - adding the the risk.
> Thus, it's generally advisable to do just use RAID10 (in this case, a
> thin-striped array of RAID1-arrays).
> 

Yes... but with such RAID10 solution you get only half of the disk space... so 
from 10 2TB drives you get only 10TB instead of 16TB with RAID6.

Some of us really need the space. Rebuild time(while it is less then 4 days) 
is considered good enough. In my case I'm using these servers for backups and 
the raid rebuilds haven't made any changes to the performance of the backups.

I'm sure that if you use such storage with RAID6 for VMs it wont perform very 
well.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 15:34:21 Torres, Giovanni (NIH/NINDS) [C] wrote:
> On Apr 12, 2011, at 3:23 AM, Matthew Feinberg wrote:
> 
> ext4 does not seem to be fully baked in 5.6 yet. parted 1.8 does not
> support creating ext4 (strange)
> 
> The CentOS homepage states that ext4 is now a fully supported filesystem in
> 5.6. ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

Steve,
I'm managing machines with 30TB of storage for more then two years. And with 
good reporting and reaction we have never had to run fsck.

However I'm sure that if you have to run fsck on so big file systems, it will 
be fater to rebuild the array from other storage then waiting for a few weeks 
to finish.

On machines like that I use CentOS but I'm pratitioning them before the 
install with a rescue live cd that I have created for me.

Marian


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 40TB File System Recommendations

[Marian Marinov]

On Tuesday 12 April 2011 10:36:54 Alain Péan wrote:
> Le 12/04/2011 09:23, Matthew Feinberg a écrit :
> > Hello All
> > 
> > I have a brand spanking new 40TB Hardware Raid6 array to play around
> > with. I am looking for recommendations for which filesystem to use. I am
> > trying not to break this up into multiple file systems as we are going
> > to use it for backups. Other factors is performance and reliability.
> > 
> > CentOS 5.6
> > 
> > array is /dev/sdb
> > 
> > So here is what I have tried so far
> > reiserfs is limited to 16TB
> > ext4 does not seem to be fully baked in 5.6 yet. parted 1.8 does not
> > support creating ext4 (strange)
> > 
> > Anyone work with large filesystems like this that have any
> > suggestions/recommendations?
> 
> Hi Matthew,
> 
> I would go for xfs, which is now supported in CentOS. This is what I use
> for a 16 TB storage, with CentOS 5.3 (Rocks Cluster), and it woks fine.
> No problem with lengthy fsck, as with ext3 (which does not support such
> capacities). I did not try yet ext4...
> 
> Alain

I have Raid6 Arrays with 30TB. We have tested XFS and its write performance 
was really dissapointing. So we looked at Ext4. It is really good for our 
workloads, but it lacks the ability to grow over 16TB. So we crated two 
partitions on the raid with ext4. 

The RAID rebuild time is around 2 days, max 3 if the workload is higher. So I 
presume that for 40TB it will be around 4 days.

Marian
-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] sshd: Authentication Failures: 137 Time(s)

[Marian Marinov]

On Tuesday 05 April 2011 11:27:49 Rudi Ahlers wrote:
> On Tue, Apr 5, 2011 at 10:17 AM, John Hodrien  
wrote:
> > On Tue, 5 Apr 2011, rrich...@blythe.org wrote:
> >> 1) Move sshd to another
> >> port, one higher than 5000
> > 
> > I'd have mixed feelings about the Wisdom of running on a non-reserved
> > port.
> 
> Why,
> 
> We've been running SSH on hundreds of servers on a port higher than
> 5000 for year now and no problems at all.

I'm also running ssh on non standard port for more then 7 years and this is on 
a couple of thousend servers. Its not a problem if you simply add 'Port XXX' 
to your ~/.ssh/config . 

However, the traffic to ssh has reduced with only 40%. In the begining it was 
very good, we were surprised, how almost all failed attempts dissapeared. But 
in the following months that number increased and reached 60-65% of the 
original number. 

Introducing a Hawk helped us a lot. Tools like Hawk and fail2ban are quite 
useful, actually only thinks like that have good impact on the bruteforce 
attempts.


Regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] sshd: Authentication Failures: 137 Time(s)

[Marian Marinov]

On Monday 04 April 2011 21:08:45 David G.Miller wrote:
> Rainer Traut  writes:
> > Hi,
> > 
> > to prevent scripted dictionary attacks to sshd
> 
> > I applied those iptables rules:
> SNIP
> 
> 
> Lots of good advice from several people.  All of the suggested solutions
> mean you still have to wade through log entries from the unsuccessful
> attacks.
> 
> I've been quite happy with similar IP tables rules but I moved sshd to
> listen on something other than port 22 for external connections.  I
> haven't seen a single brute force attack since making the move and all
> unsuccessful attempts to login via ssh get logged so it's not like
> attackers can stay below my radar.

This does not help if you provide a public services like shared hosting. We 
have all of our ssh daemons listening on different port. It was ok for a month 
or two... and then it became almost the same.


> 
> It seems that the script kiddies who are responsible for most of these
> attacks don't bother scanning (nmap) before the attack.  If port 22 isn't
> open they move elsewhere.  If I ever see any failed login attempts I can
> assume that the perpetrator is at least a little more skilled than usual
> and possibly take additional action.
> 
> Cheers,
> Dave
> 
> 
> 
> 
> ___________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] sshd: Authentication Failures: 137 Time(s)

[Marian Marinov]

Guys, 
really... look at denyhosts and Hawk.

Both projects analyze the logs of the service and check for failed login 
attempts.

It is useless to battle the bruteforcers at the network level since they can 
adapt their behaviour to really easy surcomvent any firewalls.

In order to protect your applications you should build on them. Every daemon 
now has a decent log capabilities. And you can simply tail the log constantly 
and detect which IPs should be blocked. And then block them promptly.

It is hard to find someone that will enter the wrong password more then 10 
times :)

I don't know for denyhosts, but Hawk removes the blocks every day and you can 
configure how long you want to keep a single IP blocked. This way you have 
better control over the automated block/unblock procedure.

If you need more information about Hawk, contact me.

Marian

On Monday 04 April 2011 17:18:58 Jason Brown wrote:
> You could also try using tcpwrappers along with iptables.
> 
> On 04/04/2011 06:34 AM, Marian Marinov wrote:
> > On Monday 04 April 2011 12:18:43 Rainer Traut wrote:
> >> Hi,
> >> 
> >> to prevent scripted dictionary attacks to sshd
> >> I applied those iptables rules:
> >> 
> >> -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -m recent
> >> --update --seconds 60 --hitcount 4 --name SSH --rsource -j DROP
> >> -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -m recent --set
> >> --name SSH --rsource
> >> 
> >> And this is part of logwatch:
> >> 
> >> sshd:
> >>  Authentication Failures:
> >> unknown (www.telkom.co.ke): 137 Time(s)
> >> unknown (mkongwe.jambo.co.ke): 130 Time(s)
> >> unknown (212.49.70.24): 107 Time(s)
> >> root (195.191.250.101): 8 Time(s)
> >> 
> >> How is it possible for an attacker to try to logon more then 4 times?
> >> Can the attacker do this with only one TCP/IP connection without
> >> establishing a new one?
> >> Or have the scripts been adapted to this?
> > 
> > The attackers are not trying constantly.. Just a few bursts of trys.
> > 
> > Look at denyhosts ( http://denyhosts.sourceforge.net/ ).
> > I also have a tool for protecting from brute force attacks called Hawk (
> > https://github.com/hackman/Hawk-IDS-IPS ).
> > 
> > Marian
> > 
> >> Thx
> >> Rainer
> >> ___
> >> CentOS mailing list
> >> CentOS@centos.org
> >> http://lists.centos.org/mailman/listinfo/centos
> > 
> > ___
> > CentOS mailing list
> > CentOS@centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> 
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] sshd: Authentication Failures: 137 Time(s)

[Marian Marinov]

On Monday 04 April 2011 12:18:43 Rainer Traut wrote:
> Hi,
> 
> to prevent scripted dictionary attacks to sshd
> I applied those iptables rules:
> 
> -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -m recent
> --update --seconds 60 --hitcount 4 --name SSH --rsource -j DROP
> -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -m recent --set
> --name SSH --rsource
> 
> And this is part of logwatch:
> 
> sshd:
>  Authentication Failures:
> unknown (www.telkom.co.ke): 137 Time(s)
> unknown (mkongwe.jambo.co.ke): 130 Time(s)
> unknown (212.49.70.24): 107 Time(s)
> root (195.191.250.101): 8 Time(s)
> 
> How is it possible for an attacker to try to logon more then 4 times?
> Can the attacker do this with only one TCP/IP connection without
> establishing a new one?
> Or have the scripts been adapted to this?

The attackers are not trying constantly.. Just a few bursts of trys.

Look at denyhosts ( http://denyhosts.sourceforge.net/ ). 
I also have a tool for protecting from brute force attacks called Hawk ( 
https://github.com/hackman/Hawk-IDS-IPS ).

Marian
> 
> Thx
> Rainer
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

-- 
Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

[CentOS] ksplice within CentOS

[Marian Marinov]

Hello guys,
I saw that a few days back there was a talk about encorporating the ksplice 
toolchain into CentOS and creating rebootless upgrades to the CentOS kernel.

I'm really interested in helping for that.

Where/how we can start work ?

Best regards,
Marian Marinov


signature.asc
Description: This is a digitally signed message part.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos