Re: Millions of files in /var/www inode / out of space issue.
Hi, Or you could fix your application, to not do stupid things (like generating millions of files in a single directory) in the first place... ;-) On 2013-02-19 at 12:10 CET Paolo Aglialoro paol...@gmail.com wrote: Or you could just use ZFS, XFS, whateverFS in a separate unix/linux box and go NFS on it, simulating a true external storage appliance :) On Tue, Feb 19, 2013 at 11:47 AM, MJ m...@sci.fi wrote: Which app are you running that is generating millions of tiny files in a single directory? Regardless, in this case OpenBSD is not the right tool for the job. You need either FreeBSD or a Solaris variant to handle this problem because you need ZFS. What limits does ZFS have? --- The limitations of ZFS are designed to be so large that they will never be encountered in any practical operation. ZFS can store 16 Exabytes in each storage pool, file system, file, or file attribute. ZFS can store billions of names: files or directories in a directory, file systems in a file system, or snapshots of a file system. ZFS can store trillions of items: files in a file system, file systems, volumes, or snapshots in a pool. I'm not sure why ZFS hasn't yet been ported to OpenBSD, but if it were then that would pretty much eliminate the need for my one and only FreeBSD box ;-) On Feb 19, 2013, at 2:35 AM, Keith ke...@scott-land.net wrote: Q. How do I make the default web folder /var/www/ capable of holding millions of files (say 50GB worth of small 2kb-12kb files) so that I won't get inode issues ? The problem is that my server has the default disk layout as I didn't expect to have millions of files (I though they would be stored in the DB). When I started the app it generated all the files and I got out of space warnings. I tried moving the folder containing the files and making a symlink back but that didn't work because nginx is in a chroot. The two option I think I have are. 1. Reinstall the OS and make a dedicated /var/www partition but how I increase the inode limit I have no idea. 2. Make a new partition, format it, copy the files from the original partition and swap them around and restart nginx. ( Do i run newfs with some option to make more inodes ?) Thanks Keith. -- Greetings Rafal Bisingier
Re: Benchmark for nginx + php + mysql
Hi, On 2012-11-08 at 08:06 CET Raindy Long sop...@yeah.net wrote: Sorry , my php script just like ? phpinfo(); ? It's still nowhere near full configuration description... Show you nginx config. And , I think even the static html file test is unreasonable . That depends on many different factors. Like how many other daemons you have running on this system (and how much ram is left for cache) and what kind of disks you have there. From: Rafal Bisingier Date: 2012-11-08 00:42 To: sopato CC: misc Subject: Re: Benchmark for nginx + php + mysql Hi, On Wed, 07 nov 2012 at 23:43 CET Raindy Long sop...@yeah.net wrote: Hi @misc, Just create a webserver in openbsd5.2 by nginx+php+mysql , hardware is : 512M + 2.4G CPU + 40G disk . And I do some benchmark by ab/webbench tools, open 100 clients 10 process to do the test . the result is(close nginx access log in all tests) : (1)static html file498 requests/sec (2)php file 284 requests/sec , and five php-fpm process use 100% cpu !! :( I think the test result is so bad . next is my /etc/sysctl.conf context: --- kern.maxvnodes=131072 kern.maxproc=65536 kern.maxfiles=65536 kern.somaxconn=65536 kern.sominconn=256 kern.maxclusters=32768 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.udp.recvspace=65536 net.inet.udp.sendspace=65536 --- What can I do to improve the performance ? Thanks a lot . Try this: echo ?php exit 0; ? test.php PS. You didn't even show what are you testing (your configuration and the php script code) and want some improvement advices? You must be kidding... ;-) -- Greetings Rafal Bisingier
Re: Benchmark for nginx + php + mysql
Hi, On Wed, 07 nov 2012 at 23:43 CET Raindy Long sop...@yeah.net wrote: Hi @misc, Just create a webserver in openbsd5.2 by nginx+php+mysql , hardware is : 512M + 2.4G CPU + 40G disk . And I do some benchmark by ab/webbench tools, open 100 clients 10 process to do the test . the result is(close nginx access log in all tests) : (1)static html file498 requests/sec (2)php file 284 requests/sec , and five php-fpm process use 100% cpu !! :( I think the test result is so bad . next is my /etc/sysctl.conf context: --- kern.maxvnodes=131072 kern.maxproc=65536 kern.maxfiles=65536 kern.somaxconn=65536 kern.sominconn=256 kern.maxclusters=32768 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.udp.recvspace=65536 net.inet.udp.sendspace=65536 --- What can I do to improve the performance ? Thanks a lot . Try this: echo ?php exit 0; ? test.php PS. You didn't even show what are you testing (your configuration and the php script code) and want some improvement advices? You must be kidding... ;-) -- Greetings Rafal Bisingier
Re: relayd for lan servers with carp and pfsync
table servers:25 active (2 hosts) 3 host192.168.0.66100.00% up 4 host192.168.0.67100.00% up 3 redirectpop down 3 table servers:110 empty 5 host192.168.0.660.00% down 6 host192.168.0.670.00% down Seeking your ideas to solve this? where have I gone wrong? I referred to below 2 URLs http://www.openbsd.org/faq/pf/carp.html#failover http://meinit.nl/openbsd-loadbalancing-and-failover-relayd-pf-and-carp -- Thank you Indunil Jayasooriya -- Greetings Rafal Bisingier
Re: Virtualizing firewalling scenarios in one physical OpenBSD host
Hi On Wed, Jul 4, 2012 at 11:13 CEST C. L. Martinez carlopm...@gmail.com wrote: On Wed, Jul 4, 2012 at 10:49 AM, Jiri B ji...@devio.us wrote: On Wed, Jul 04, 2012 at 09:29:04AM +0200, C. L. Martinez wrote: I wonder if with OpenBSD is possible to create virtualized firewalled implementations of conventional physical topologies and designs such as central and remote DMZs (my question has nothing to do with virtualization platforms like ESXi/vSphere or Xen or KVM), like for example CheckPoint VSX does: http://www.checkpoint.com/products/vpn-1-power-vsx/index.html. So what is that doing? The link is full of marketing shit words :) The great catch here is what VSX does: you can deploy virtual firewalls within the same physical CheckPoint machine. And what does this mean? Anyway, read about rdomains in OpenBSD - that's how you'll get your virtual firewall, of course without the fancy (and mostly annoying) GUI like the CheckPoint's one. -- Greetings Rafal Bisingier
Re: carp mixed states
broadcast 192.168.5.255 inet6 fe80::204:23ff:fee3:c792%em0 prefixlen 64 scopeid 0x2 em1: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu 1500 lladdr 00:04:23:e3:c7:93 priority: 0 media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause) status: active inet 10.5.5.3 netmask 0xff00 broadcast 10.5.5.255 inet6 fe80::204:23ff:fee3:c793%em1 prefixlen 64 scopeid 0x3 enc0: flags=41UP,RUNNING priority: 0 groups: enc status: active pfsync0: flags=41UP,RUNNING mtu 1500 priority: 0 pfsync: syncdev: bge0 maxupd: 128 defer: off groups: carp pfsync pflog0: flags=141UP,RUNNING,PROMISC mtu 33196 priority: 0 groups: pflog carp1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:00:5e:00:01:01 priority: 0 carp: BACKUP carpdev em1 vhid 1 advbase 1 advskew 100 groups: carp status: backup inet6 fe80::200:5eff:fe00:101%carp1 prefixlen 64 scopeid 0x6 inet 10.5.5.1 netmask 0xff00 broadcast 10.5.5.255 carp2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:00:5e:00:01:02 priority: 0 carp: MASTER carpdev em0 vhid 2 advbase 1 advskew 100 groups: carp status: master inet6 fe80::200:5eff:fe00:102%carp2 prefixlen 64 scopeid 0x7 inet 192.168.5.1 netmask 0xff00 broadcast 192.168.5.255 It isn't normal. Check connectivity on em0 interface between both firewalls. When I hit something very similar, the reason turned out to be misconfigured vlans on switch ports. -- Greetings Rafal Bisingier
Re: disk management
Hi, On Friday, 13 Jan 2012 at 13:40 CET lilit-aibolit lilit-aibo...@mail.ru wrote: Hi misc. Here is newbee question. I have disk with unused space: # disklabel -p g wd0 16 partitions: # sizeoffset fstype [fsize bsize cpg] a: 1.0G63 4.2BSD 2048 163841 # / b: 1.2G 2097215swap c: 37.3G 0 unused d: 2.6G 4683375 4.2BSD 2048 163841 # /tmp e: 4.0G 10052439 4.2BSD 2048 163841 # /var f: 2.0G 18541648 4.2BSD 2048 163841 # /usr g: 1.0G 22735952 4.2BSD 2048 163841 # /usr/X11R6 h: 3.5G 24833104 4.2BSD 2048 163841 # /usr/local i: 1.9G 32229473 4.2BSD 2048 163841 # /usr/src j: 1.9G 36247864 4.2BSD 2048 163841 # /usr/obj k: 18.1G 40266255 4.2BSD 2048 163841 # /home So which one is unsed? Hint: partition c is always marked as unused, because it is the whole disk. You can NOT use it for anything. and I have /var with ending space: # df -h Filesystem SizeUsed Avail Capacity Mounted on /dev/wd0a 1005M206M749M22%/ /dev/wd0k 17.8G411M 16.5G 2%/home /dev/wd0d 2.5G6.0K2.4G 0%/tmp /dev/wd0f 2.0G927M985M48%/usr /dev/wd0g 1005M167M787M18%/usr/X11R6 /dev/wd0h 3.5G280M3.0G 8%/usr/local /dev/wd0j 1.9G993M841M54%/usr/obj /dev/wd0i 1.9G790M1.0G43%/usr/src /dev/wd0e 4.0G3.4G376M90%/var In /var I store some sites for apache and need more space for it. How can I use unused space for /var or it will be used automatically when /var reaches capacity 100%? No, it will not grow automagically. -- Greetings Rafal Bisingier
Re: relayd fails on POST 2GB
Hi, On friday, 06 Jan 2012 at 13:22 CET Gordon McAllister gordon.mcallis...@gmail.com wrote: Is there a knob to tweak to allow POSTs greater than 2GB or is this limit somehow hardcoded? A wild guess (since you didn't provide dmesg): do you use i386 arch? -- Greetings Rafal Bisingier
Re: PF rule match only packets for local machine
Hi,
On Thursday, 05 Jan 2012 at 09:00 CET
Robert Wolf r.wolf.c...@gmail.com wrote:
table OutNetworks const { }
pass quick proto tcp from OurNetworks to any port 22 no state
pass in quick proto tcp from any to any port rdr-to 127.0.0.1 port 22
block quick proto tcp from any to any port 22
But of course, the last rule blocks every SSH traffic going from unknown
networks to all hosts.
Could someone please help me to create PF rules to block only traffic going to
local machine from other networks as OutNetworks similary as the iptables rule
above?
Just replace to any to to self. Should do what you want.
I have read PF manual but not found any possibility to tell pf to
LOCAL-HOST. I have search with google but no relevant articles found, maybe I
have not asked correct.
Well, it's not very easy to find, but the self word is explained in
the manual.
--
Greetings
Rafal Bisingier
Re: Longsoon/Godson MIPS boxes, where to buy?
Hi, On, 03 Jan 2012 o 08:59 CET Anonymous cri...@ecn.org wrote: If you don't have signal to add to the thread at least don't add to the noise. Could you please follow your own advise and simply STFU? -- Rafal Bisingier
Re: Bug Tracking system does not work
Hi, I use Roundup Issue Tracker (www.roundup-tracker.org). It's not exactly a bug-tracker (well bugs.python.org use it), but it's highly and quite simply configurable (or more accurately modifiable by plugins). On Tue, Jul 19, 2011 at 15:16 CEST Johan Beisser j...@caustic.org wrote: To make it not suck: - easy to extend, modify, or add in plugins for new features (no patching, please) Not everything, but quite many things can be changed by plugins or modification of config files - simple database schema, no dumping required to upgrade I don't even look at database, and did an upgrade couple of times ;-) - functional search I wouldn't say it's flawless, but it works. - merging of tickets Didn't ever try that - automatically scheduled repeating tickets (heh) With external scripting it should be quite easy - ability to make API calls to the ticket software (i sometimes want to open/list/etc tickets remotely, without using the webt interface directly) There is CLI admin tool and XML-RPC access You get the idea. Try Roundup. For me the best thing is it's email gateway. I rarely look at the web frontend - most things are easily accessible through e-mails sent to Roundup. -- Greetings Rafal Bisingier
Re: Like OpenBSD? Like to see new stuff happening? You really need to order a CD today :)
Hi, On Thu, 21 Apr 2011 11:49:45 -0600 Theo de Raadt dera...@cvs.openbsd.org wrote: As long as it costs less than it brings in for funding what wrong could it do ? It costs time. Go do an install of OpenBSD 3.0 to understand the point. How about a new product: OpenBSD license for one machine, without media. This way it could be made tax-deductible even for europeans, and there won't be production and shipping cost. Yes, I know, this one also takes some time, so probably not really worth it... -- Greetings Rafal Bisingier
Re: Advice on pf no-sync
Hi, On Tue, 07 Dec 2010 21:15:13 -0700 Devin Reade g...@gno.org wrote: I understand (from pf.conf(5)) what no-sync is supposed to do, however the only example I've seen of it in use is on the pfsync and carp examples in pfsync(4). I was wondering if anyone had some advice on some specific examples of when the use of no-sync is appropriate, specifically in a two-node firewall cluster that uses pfsync. Assume that there are DMZ and internal network segments, some of which are routable and some of which are NAT'd private space. Further assume that some services are hosted from the firewall nodes themselves. I understand that most pf rules under these circumstances would *not* use no-sync, but it's not clear if there's anything other than pfsync/carp that should/might. In my understanding any connection made to the firewall own address or service (so not through the firewall, no nated or redirected one) should be no-sync'ed, because that connection would simply be invalid when carp-master will change. -- Greetings Rafal Bisingier
Re: Little update to authpf
Hi,
About a year ago I've sent a simple patch for authpf, which adds some
nice (as I think) feature to authpf. My patch was reviewed and extended
(and corrected) by couple of people, but since then did not get into
cvs. So now is my second try. I'd really like to get this kind of
functionality in authpf. And now the details:
- authpf can show a message to an user successfully logged in
- this message is read from /etc/authpf/authpf.message
- the message is the same for every user
- i'll want to change it ;-)
Patch (in the form proposed here on list last year) is below. This time
I've added adequate manpage changes.
--
Greetings
Rafal Bisingier
Index: authpf.8
===
RCS file: /cvs/src/usr.sbin/authpf/authpf.8,v
retrieving revision 1.47
diff -u -r1.47 authpf.8
--- authpf.86 Jan 2009 03:11:50 - 1.47
+++ authpf.86 Sep 2009 22:29:19 -
@@ -178,9 +178,13 @@
On successful invocation,
.Nm
displays a message telling the user he or she has been authenticated.
-It will additionally display the contents of the file
-.Pa /etc/authpf/authpf.message
-if the file exists and is readable.
+It will additionally display the contents of the file called
+.Pa authpf.message .
+This file will first be searched for in
+.Pa /etc/authpf/users/$USER/
+and then in
+.Pa /etc/authpf/ .
+Only first of these files will be used if both are present.
.Pp
There exist two methods for providing additional granularity to the control
offered by
Index: authpf.c
===
RCS file: /cvs/src/usr.sbin/authpf/authpf.c,v
retrieving revision 1.112
diff -u -r1.112 authpf.c
--- authpf.c10 Jan 2009 19:08:53 - 1.112
+++ authpf.c6 Sep 2009 22:29:19 -
@@ -320,10 +320,20 @@
}
while (1) {
+ struct stat sb;
+ char *path_message;
printf(\r\nHello %s. , luser);
printf(You are authenticated from host \%s\\r\n, ipsrc);
setproctitle(%...@%s, luser, ipsrc);
- print_message(PATH_MESSAGE);
+ if (asprintf(path_message, %s/%s/authpf.message,
+ PATH_USER_DIR, luser) == -1)
+ do_death(1);
+ if (stat(path_message, sb) == -1 || ! S_ISREG(sb.st_mode)) {
+ free(path_message);
+ if ((path_message = strdup(PATH_MESSAGE)) == NULL)
+ do_death(1);
+ }
+ print_message(path_message);
while (1) {
sleep(10);
if (want_death)
OpenBSD on IBM Power System?
Hi, Is there any chance to install OpenBSD on a logical hardware partition created on IBM p-Series machine? There is a mac-ppc port, but as I understand it's only for apple hardware? Is there any one having some experience on this field? -- Greetings Rafal Bisingier
Little update to authpf
Hi all,
I do not know if this is the correct list, or even method to send
patches, but did not found anything appropriate on the OpenBSD website.
I'd like to propose a little feature enhancement for the authpf. Here
are the details:
- authpf can show a message to an user successfully logged in
- this message is read from /etc/authpf/authpf.message
- the message is the same for every user
- i'll want to change it ;-)
Below is a patch which change current behavior, so that the message is
searched first in the /etc/authpf/USER dir, and if it's not found
there, then the old behavior is used (so fully backward compatible).
The patch looks very simple, but I did NOT tested it at all! Anyway it
would be nice, if something like this make it's way into the HEAD. ;-)
PS. Sorry for any language errors
--
Greetings
Rafal Bisingier
diff -u authpf.c.orig authpf.c
--- authpf.c.orig 2008-09-09 17:23:43.315714111 +0200
+++ authpf.c2008-09-10 21:07:06.258107858 +0200
@@ -314,10 +314,16 @@
signal(SIGQUIT, need_death);
signal(SIGTSTP, need_death);
while (1) {
+ char*fn = NULL;
printf(\r\nHello %s. , luser);
printf(You are authenticated from host \%s\\r\n,
ipsrc); setproctitle([EMAIL PROTECTED], luser, ipsrc);
- print_message(PATH_MESSAGE);
+ if (asprintf(fn, %s/%s/authpf.message,
+ PATH_USER_DIR, luser) == -1)
+ print_message(PATH_MESSAGE);
+ else
+ print_message(fn);
+ free(fn);
while (1) {
sleep(10);
if (want_death)
Re: Little update to authpf
Hi, On Thu, 11 Sep 2008 14:26:42 +0200 Ross Cameron [EMAIL PROTECTED] wrote: On Thu, Sep 11, 2008 at 2:09 PM, Rafal Bisingier [EMAIL PROTECTED]wrote: Below is a patch which change current behavior, so that the message is searched first in the /etc/authpf/USER dir, and if it's not found Would/etc/authpf/authpf.USER.messagenot be better? Sample change if (asprintf(fn, %s/authpf.%s.message, PATH_USER_DIR, luser) == -1) print_message(PATH_MESSAGE); else print_message(fn); Sample change Please bear in mind that I can at best read C so the above is probably wrong. Well, the /etc/authpf/user directory is already used for storing per user rules, and some other info, so adding there a message-file looks sensible for me... ;-) -- Greetings Rafal Bisingier
Re: Little update to authpf
Hi, On Thu, 11 Sep 2008 07:52:14 -0500 Todd T. Fries [EMAIL PROTECTED] wrote: I think you might want to check to see if the file exists not just if the asprintf succeeds.. Yes, that's a really good idea... ;-) But yes I do agree this is useful functionality that I've tested quite thoroughly... So, will it be available in OpenBSD 4.5? ;-) -- Greetings Rafal Bisingier
