Re: BSDstats Project v2.0 ...
Hello, On Sat, Aug 12, 2006 at 11:55:14PM -0300, Marc G. Fournier wrote: Since this is meant to provide stats for *BSD, not just FreeBSD, I've setup bsdstats.org as a more 'neutral' site ... Maybe you need to move data from bsdstats.hub.org to bsdstats.org? Because now there is bsdstats.hub.org with 1764 reported systems and there is a bsdstats.org with 22 reported systems. So question ... which one is correct? And will v2.x scripts report to correct one next month? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, 14 Aug 2006, Igor Robul wrote: Hello, On Sat, Aug 12, 2006 at 11:55:14PM -0300, Marc G. Fournier wrote: Since this is meant to provide stats for *BSD, not just FreeBSD, I've setup bsdstats.org as a more 'neutral' site ... Maybe you need to move data from bsdstats.hub.org to bsdstats.org? Because now there is bsdstats.hub.org with 1764 reported systems and there is a bsdstats.org with 22 reported systems. So question ... which one is correct? And will v2.x scripts report to correct one next month? As I just posted in another thread ... :) pre-v3.x scripts will not work with the new DB backend ... we've changed alot to a) eliminate storing any sensitive information and b) reduce the # of fetches that have to happen to do the reporting ... I've just put a redirect in from bsdstats.hub.org - bsdstats.org, so that ppl aren't confused from that perspective ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Friday 11 August 2006 22:29, Nikolas Britton wrote: On 8/11/06, Matthew Seaman [EMAIL PROTECTED] wrote: Marc G. Fournier wrote: On Fri, 11 Aug 2006, Nikolas Britton wrote: Ok... With my new script it took only 158 minutes to compute ALL TCP/IP address hashes. I'll repeat that... I have an md5 hash for every IP address in the world! All I need to do is grep your hash and it will tell me your IP address. yippee! :-) Can someone please explain to me what exactly you are trying to secure against in this case? He's trying to prevent any possibility of information disclosure about his servers. If I wanted to hack into his site, knowing what hosts he had running (ie. a bunch of live IP numbers) and what OS etc. each used would mean I'm already halfway to my goal. Now, while the design of bsdstats does not disclose that sort of stuff readily, any security conscious admin is going to worry about that data being collected and held outside of his administrative control. Having a completely anonymous and untraceable token to identify each of the hosts sending in information should make connecting the information back to the original sender practically impossible. YES! what he said... I don't want ANYTHING to trace back to me or my systems. Although, playing devil's advocate here, anyone that could steal the Apache log files from the bsdstats server would be able to work out that sort of data fairly readily. I guess the truly paranoid should only submit their data via some sort of anonymizing proxy. That's simple, don't keep the log files... * Can we trust Marc to delete them? * I thought this was going to be an official FreeBSD project hosted on freebsd.org? * Maybe we should get the OpenBSD people involved? Just thinking out loud :-/ honestly, should said security concious admins, really be participating 'using his bosses servers' in this project? probably not. even if all the security consious admins out there decline to have all their datacenters participate in bsdstats, im sure just the ones who decide that the risk of sending the same info your browser does (plus a bit more if you choose and deliberatly enable) is appropriate for them, is still going to give one hell of a great demographic report to bsdstats. 2 cents, jonathan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re[3]: BSDstats Project v2.0 ...
Hello Marc, Saturday, August 12, 2006, 12:55:13 AM, you wrote: On Sat, 12 Aug 2006, Daniel Gerzo wrote: It would be nice to see this in base system, that would help to raise this number enourmously. And surely it would be nice to see it somewhere under the freebsd.org domain. Actually, I've registered bsdstats.org for this ... I've been talking to various ppl from the other *BSDs about getting them involved as well, so went with the more 'neutral' domain instead of making this FreeBSD Only ... we share alot between us as it is, sharing marketing power is a good thing ... Ah, I see. I haven't been thinking about this this way :) -- Best regards, Danielmailto:[EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re[3]: BSDstats Project v2.0 ...
On Sat, 12 Aug 2006, Daniel Gerzo wrote: Hello Marc, Saturday, August 12, 2006, 12:55:13 AM, you wrote: On Sat, 12 Aug 2006, Daniel Gerzo wrote: It would be nice to see this in base system, that would help to raise this number enourmously. And surely it would be nice to see it somewhere under the freebsd.org domain. Actually, I've registered bsdstats.org for this ... I've been talking to various ppl from the other *BSDs about getting them involved as well, so went with the more 'neutral' domain instead of making this FreeBSD Only ... we share alot between us as it is, sharing marketing power is a good thing ... Ah, I see. I haven't been thinking about this this way :) I'm currently working on a v3.0 that will use bsdstats.org as the checkin server ... its being designed in such a way as to eliminate anything being left on the server, and, in fact, the hostname isn't even sent to the server ... Most of the code is ready, just working now on reducing the # of fetch's required down to 4 from about a dozen or more ... make it more efficient ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Sat, 12 Aug 2006, Jonathan Horne wrote: * Can we trust Marc to delete them? Won't be anything to delete ... except for any time I need to debug the server end, the logs will be set to /dev/null ... * I thought this was going to be an official FreeBSD project hosted on freebsd.org? Since this is meant to provide stats for *BSD, not just FreeBSD, I've setup bsdstats.org as a more 'neutral' site ... * Maybe we should get the OpenBSD people involved? I've sent Theo an email, and never heard back ... in fact, I've sent emails to the NetBSD, OpenBSD *and* DragonFlyBSD camps, and the only one that answered back with any sort of interest was the DF-BSD camp, and I have some mods to add to v3.0 to satisfy Matt's requirements to have it actually put into their base operating system ... and that one is simple, he just wants some sort of 'connectivity check' put in place Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Ok... With my new script it took only 158 minutes to compute ALL
TCP/IP address hashes. I'll repeat that... I have an md5 hash for
every IP address in the world! All I need to do is grep your hash and
it will tell me your IP address. yippee! :-)
Can we please find a new method to track hosts... perhaps my earlier
example: ifconfig |md5. If not please remove my entries in the
database.
I've attached the script used to make the hashes.
On 8/10/06, Nikolas Britton [EMAIL PROTECTED] wrote:
On 8/9/06, Nikolas Britton [EMAIL PROTECTED] wrote:
On 8/9/06, Marc G. Fournier [EMAIL PROTECTED] wrote:
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Marc G. Fournier wrote:
On Wed, 9 Aug 2006, Igor Robul wrote:
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote:
Could create problems long term .. one thing I will be using the
IPs to do is:
SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC;
to look for any 'abnormalities' like todays with Armenia ...
hashing it would make stuff like that fairly difficult ...
You can make _two_ hashes and then concatenate to form unique key.
Then you still be able to see a lot of single IPs. Personaly, I dont
care very much about IP/hostname disclosure :-)
Except that you are disclosing that each and every time you send out an
email, or hit a web site ... :)
The systems I'm concerned about are on private IP space, to not send email
and don't have X installed, much less a web browser and can only access
certain FreeBSD sites to update ports. In fact, they're not even
accessible
from *inside* our network except from certain hosts. In order to
successfully run the stats script on these hosts, I would have to open a
hole
in the firewall to bsdstats.hub.org on the correct port.
And yes, I *am* paranoid. But if you really want *all* statistics you can
get, then you'll have to deal with us paranoid types. My workstation,
which
is on a public IP, is already registered.
Done ... now I really hope that the US stats rise, maybe? I have a hard
time believing that Russia and the Ukraine have more deployments then the
'good ol'US of A' ... or do they? *raised eyebrow*
Here is what is now stored in the database (using my IP as a basis)
# select * from systems where ip = md5('24.224.179.167');
id |ip| hostname
| operating_system | release | architecture | country |report_date
--+--+--+--++--+-+---
1295 | 45c80b9266a5a6683eee9c9798bd6575 |
4a9110019f2ca076407ed838bf190017 | FreeBSD | 6.1-RC1| i386 | CA
| 2006-08-09 02:34:05.12579
1 | 45c80b9266a5a6683eee9c9798bd6575 |
9a45e58ab9535d89f0a7d2092b816364 | FreeBSD | 6.1-STABLE | i386 | CA
| 2006-08-09 16:01:03.34788
Why don't you just broadcast the ip address, it's what your doing now
anyways. 253^4 is a very small number.
infomatic# perl
my $num = 0;
system date;
while ($num = 409715208) {
$num++
}
system date;
Wed Aug 9 18:18:45 CDT 2006
Wed Aug 9 18:20:48 CDT 2006
2 minutes * 10 = 20 minutes to iterate though 4 billion IP addresses
on a very slow uni-proc system. I could even store every IP to md5
hash using less then 222GB of uncompressed space.
If you want... give me the md5 hash of a real ip address that is
unknown to me and I will hand you the ip address in two days... or
less. run the IP address though like this:
md5 -s xxx.xxx.xxx.xxx
I have other things to do with my time, so I don't really want to do
this, but if that's what it takes to stop this idea dead I'll do it.
Here's a better way to explain the problem:
Let's say we need to find Marc's IP address but we only have it's md5
hash value. Some of you may think this is hard to do but it's not. All
we need to do is compute every IP address into a hash and then match
Marc's hash to one in are list:
24.224.179.164 = e7e7a967c5f88d9fb10a1f22cd2133d2
24.224.179.165 = 3aa9b50aa7190f5aca1f78f075dc69c2
24.224.179.166 = c695175e48d649e3496ac715406a488d
24.224.179.167 = 45c80b9266a5a6683eee9c9798bd6575
So what is an IP address?... mathematically speaking it's 4 base 255
numbers grouped together:
{0, ..., 255}.{0, ..., 255}.{0, ..., 255}.{0, ..., 255}
To calculate how many combinations there could be you simply take the
base unit and raise it to the 4th power, since there are 4 of them.
This gives us 255^4 combinations or 4,228,250,625 TCP/IP addresses. We
also know that the first number can't be 0 or 255 and the others can't
be 255, we can also rule out all 127.x.y.z loopback and multicast
224.x.y.z - 239.x.y.z addresses:
(237^1) * (254^3)
This leaves us with 3,883,734,168 valid IP addresses. We can divide
this number by 5,000 and run it through a
Re: BSDstats Project v2.0 ...
Nikolas Britton wrote:
Ok... With my new script it took only 158 minutes to compute ALL
TCP/IP address hashes. I'll repeat that... I have an md5 hash for
every IP address in the world! All I need to do is grep your hash and
it will tell me your IP address. yippee! :-)
Can we please find a new method to track hosts... perhaps my earlier
example: ifconfig |md5. If not please remove my entries in the
database.
How about the attached diff. As discussed else-thread, this generates
a random ID 128bit token -- the chances of any two hosts generating the
same token are so minuscule as to be negligible. The token is cached in
a file /var/db/bsdstats for re-use in later months.
This also adds the capability for the paranoid to withhold the hostname
of the machine, and it removes what looks like a forgotten bit of debugging
code that would mean Marc would get quite a lot of e-mail each month...
I believe the default for CGI scripts is to ignore any extra parameters
that they weren't programmed to expect[1], so this should even be compatible
with the current bsdstats stuff.
Cheers,
Matthew
[1] No one would seriously contemplate running PHP with 'register_globals'
enabled in this day and age would they?
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW
--- /usr/ports/sysutils/bsdstats/files/300.statistics Thu Aug 10 10:58:00 2006
+++ 300.statistics Fri Aug 11 12:56:54 2006
@@ -5,7 +5,6 @@
# If there is a global system configuration file, suck it in.
#
-monthly_statistics_mailto=[EMAIL PROTECTED],root
if [ -r /etc/defaults/periodic.conf ]
then
. /etc/defaults/periodic.conf
@@ -37,22 +36,50 @@
/usr/bin/fetch -qo /dev/null http://$checkin_server/scripts/$1;
}
-checkin_server=bsdstats.hub.org;
+get_id_token () {
+if [ -f $id_token_file ] ;
+then
+ . $id_token_file
+else
+ IDTOKEN=$( openssl rand -base64 16 )
+ touch $id_token_file \
+ chown root:wheel $id_token_file \
+ chmod 600 $id_token_file \
+ echo IDTOKEN='$IDTOKEN' $id_token_file
+fi
+IDTOKEN=$( uri_escape $IDTOKEN )
+}
+
+checkin_server='bsdstats.hub.org'
+id_token_file='/var/db/bsdstats'
+
+# Send hostname to the stats server? Default yes -- set to NO
+# in periodic.conf if desired.
+
+monthly_statistics_reveal_hostname=${monthly_statisics_reveal_hostname-YES}
case $monthly_statistics_enable in
[Yy][Ee][Ss])
- HN=`/bin/hostname`
+ get_id_token
+ case $monthly_statistics_reveal_hostname in
+ [Yy][Ee][Ss])
+ HN=`/bin/hostname`
+ ;;
+ *)
+ HN='(no-hostname)'
+ ;;
+ esac
SYS=`/usr/bin/uname -r`
ARCH=`/usr/bin/uname -m`
OS=`/usr/bin/uname -s`
- do_fetch getid.php?hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
+ do_fetch getid.php?id=$IDTOKEN\hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
echo Posting monthly OS statistics to $checkin_server
case $monthly_statistics_report_devices in
[Yy][Ee][Ss])
IFS=
-do_fetch clear_devices.php?hn=$HN
+do_fetch clear_devices.php?id=$IDTOKEN\hn=$HN
for line in `/usr/sbin/pciconf -l | /usr/bin/grep -v none`
do
DRIVER=`echo $line | awk -F\@ '{print $1}'`
@@ -60,7 +87,7 @@
DEV=`echo $line | awk '{print $4}' | cut -c8-11`
CLASS=`echo $line | awk '{print $2}' | cut -c9-10`
SUBCLASS=`echo $line | awk '{print $2}' | cut -c11-14`
-do_fetch
report_device.php?driver=$DRIVER\vendor=$VEN\device=$DEV\class=$CLASS\subclass=$SUBCLASS\hn=$HN
+do_fetch
report_device.php?id=$IDTOKEN\driver=$DRIVER\vendor=$VEN\device=$DEV\class=$CLASS\subclass=$SUBCLASS\hn=$HN
done
echo Posting monthly device statistics to $checkin_server
@@ -69,10 +96,10 @@
DEV=$( uri_escape $( echo $line | cut -d ' ' -f 2- ) )
n=0
count=$( sysctl -n hw.ncpu )
-do_fetch clear_cpu.php?hn=$HN
+do_fetch clear_cpu.php?id=$IDTOKEN\hn=$HN
while [ $n -lt $count ]
do
-do_fetch
report_cpu.php?cpu_id=CPU$n\vendor=$VEN\cpu_type=$DEV\hn=$HN
+do_fetch
report_cpu.php?id=$IDTOKEN\cpu_id=CPU$n\vendor=$VEN\cpu_type=$DEV\hn=$HN
n=$(( $n + 1 ))
done
echo Posting monthly CPU statistics to $checkin_server
signature.asc
Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
On Fri, 11 Aug 2006, Nikolas Britton wrote: Ok... With my new script it took only 158 minutes to compute ALL TCP/IP address hashes. I'll repeat that... I have an md5 hash for every IP address in the world! All I need to do is grep your hash and it will tell me your IP address. yippee! :-) Can someone please explain to me what exactly you are trying to secure against in this case? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Fri, 11 Aug 2006, Nikolas Britton wrote: Ok... With my new script it took only 158 minutes to compute ALL TCP/IP address hashes. I'll repeat that... I have an md5 hash for every IP address in the world! All I need to do is grep your hash and it will tell me your IP address. yippee! :-) Can someone please explain to me what exactly you are trying to secure against in this case? He's trying to prevent any possibility of information disclosure about his servers. If I wanted to hack into his site, knowing what hosts he had running (ie. a bunch of live IP numbers) and what OS etc. each used would mean I'm already halfway to my goal. Now, while the design of bsdstats does not disclose that sort of stuff readily, any security conscious admin is going to worry about that data being collected and held outside of his administrative control. Having a completely anonymous and untraceable token to identify each of the hosts sending in information should make connecting the information back to the original sender practically impossible. Although, playing devil's advocate here, anyone that could steal the Apache log files from the bsdstats server would be able to work out that sort of data fairly readily. I guess the truly paranoid should only submit their data via some sort of anonymizing proxy. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
On Fri, Aug 11, 2006 at 02:38:48PM +0100, Matthew Seaman wrote: He's trying to prevent any possibility of information disclosure about his servers. If I wanted to hack into his site, knowing what hosts he had running (ie. a bunch of live IP numbers) and what OS etc. each used would mean I'm already halfway to my goal. Now, while the design of bsdstats does not disclose that sort of stuff readily, any security conscious admin is going to worry about that data being collected and held outside of his administrative control. Having a completely anonymous and untraceable token to identify each of the hosts sending in information should make connecting the information back to the original sender practically impossible. Yes, this kind of information leakage is particularly bad. Some script kiddie with a given hammer can go in search of just the right nails, and find them. If it's some work to extract info it's still worth it for a tidy list of hosts with a high probability of vulnerability. Although, playing devil's advocate here, anyone that could steal the Apache log files from the bsdstats server would be able to work out that sort of data fairly readily. I guess the truly paranoid should only submit their data via some sort of anonymizing proxy. It's easier than stealing log files. Anyone with access to traffic anywhere along the line can sniff this stuff without cracking into anyone's box. The suggestion to use a 128-bit random as an ID is a good one. Further, the stats server should have a public key and data sent to it should be encrypted. Or submissions could be over SSL. -- Darrin Chandler| Phoenix BSD Users Group [EMAIL PROTECTED] | http://bsd.phoenix.az.us/ http://www.stilyagin.com/ | ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Fri, 11 Aug 2006, Nikolas Britton wrote: Ok... With my new script it took only 158 minutes to compute ALL TCP/IP address hashes. I'll repeat that... I have an md5 hash for every IP address in the world! All I need to do is grep your hash and it will tell me your IP address. yippee! :-) Can someone please explain to me what exactly you are trying to secure against in this case? If you know my IP, my hostname, what OS I'm running and *every* driver I have enabled on my box, you're half way toward breaking in to my box. What he's saying is that you've chosen the IP address as the index key for the database. Even though you're hashing it with MD5, he has written a script that generates, in less than an hour, the MD5 hash for every single IP address in the world. *If* he can break in to your database and extract its information, he can simply match his hashes against yours and decode every IP address. Once he's done that, he has a big fat list of juicy targets to go after. This is the reason that the only hosts I've submitted on the two that are on public IP addresses. You can get the same info by probing them directly. You won't be getting my other boxes until this problem is solved. I think two suggestions have been made that are quite worthy of consideration. 1) encrypt the data being fed to your systems by the script - this should be relatively easy using keys and would ensure that a man in the middle attack would fail. You can connect using ssh and a unique key without having to reveal passwords to anyone. 2) use a unique hash, generated at the time of first conneciton, that identifies the box regardless of its IP, hostname, MAC address or any of the other myriad parameters that can all change over time. This would actually make your data more reliable, since parameters change (IPs, MACs, hostnames, peripherals, etc.), boxes do not. I realize everyone is very enthusiastic about this project, but, if you want a high adoption rate, you're going to have to consider the concerns of the more security conscious among us. -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
Paul Schmehl wrote: 1) encrypt the data being fed to your systems by the script - this should be relatively easy using keys and would ensure that a man in the middle attack would fail. You can connect using ssh and a unique key without having to reveal passwords to anyone. Uh... HTTPS surely? Because it's relatively simple to implement on both client and server, doesn't require extra software installed on every client beyond the monthly stats script itself and because of the way that HTTPS uses a one-sided Diffie Helmann exchange to create session keys which means that you don't have any trouble with key management on the many thousands of client boxes out there... In which case rewriting the monthly_stats script to send all the data to the server in one transaction would be a pretty good optimization. It's a pity that fetch(1) doesn't have the capability to do a HTTP POST rather than a GET though, given the amount of stuff to send. As a matter of interest, does the FreeBSD project or any of the other *BSDs have a CA anywhere that could sign the bsdstats web server cert? If not, then I guess some sort of appeal to raise the cash to get a cert signed by one of the Root CAs might well be in order. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
Matthew Seaman wrote: Paul Schmehl wrote: 1) encrypt the data being fed to your systems by the script - this should be relatively easy using keys and would ensure that a man in the middle attack would fail. You can connect using ssh and a unique key without having to reveal passwords to anyone. Uh... HTTPS surely? Because it's relatively simple to implement on both client and server, doesn't require extra software installed on every client beyond the monthly stats script itself and because of the way that HTTPS uses a one-sided Diffie Helmann exchange to create session keys which means that you don't have any trouble with key management on the many thousands of client boxes out there... I defer to your obviously greater experience and wisdom. :-) I would note that these issues appear to be impacting the project. As of right now, there are only 1612 systems reporting in, and I suspect there are a much greater number of systems distributed throughout the computing universe. Certainly some can be attributed to the newness of the project and the small amount of promotion done to date, but I can't help but think that at least some of it is due to hesitancy on the part of some to submit their data. For my part, I've submitted two public hosts. I have four others I will not submit until I'm certain the data are securely transmitted and stored. Surely I'm not alone? -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
At 11:49 AM -0500 8/11/06, Paul Schmehl wrote: I would note that these issues appear to be impacting the project. As of right now, there are only 1612 systems reporting in, ... For my part, I've submitted two public hosts. I have four others I will not submit until I'm certain the data are securely transmitted and stored. Surely I'm not alone? I know we are used to dealing in internet-time, where things happen instantly, but there could be many reasons that the host count is only 1612. Reasons that have nothing to do with the specific outcome of how these security issues are handled. I am certainly all for the improvements people have been talking about. I'm just saying that even if you make all those improvements, you're probably going to have to wait a few weeks before we see any significant number of hosts show up. That's just the way it is. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Fri, 11 Aug 2006, Garance A Drosihn wrote: At 11:49 AM -0500 8/11/06, Paul Schmehl wrote: I would note that these issues appear to be impacting the project. As of right now, there are only 1612 systems reporting in, ... For my part, I've submitted two public hosts. I have four others I will not submit until I'm certain the data are securely transmitted and stored. Surely I'm not alone? I know we are used to dealing in internet-time, where things happen instantly, but there could be many reasons that the host count is only 1612. Reasons that have nothing to do with the specific outcome of how these security issues are handled. I am certainly all for the improvements people have been talking about. I'm just saying that even if you make all those improvements, you're probably going to have to wait a few weeks before we see any significant number of hosts show up. That's just the way it is. Which was totally expected ... this wasn't meant to be a 'short term project', that's for sure :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re[2]: BSDstats Project v2.0 ...
Hello Garance, Friday, August 11, 2006, 9:59:41 PM, you wrote: At 11:49 AM -0500 8/11/06, Paul Schmehl wrote: I know we are used to dealing in internet-time, where things happen instantly, but there could be many reasons that the host count is only 1612. Reasons that have nothing to do with the specific outcome of how these security issues are handled. I am certainly all for the improvements people have been talking about. I'm just saying that even if you make all those improvements, you're probably going to have to wait a few weeks before we see any significant number of hosts show up. That's just the way it is. It would be nice to see this in base system, that would help to raise this number enourmously. And surely it would be nice to see it somewhere under the freebsd.org domain. -- Best regards, Danielmailto:[EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re[2]: BSDstats Project v2.0 ...
On Sat, 12 Aug 2006, Daniel Gerzo wrote: Hello Garance, Friday, August 11, 2006, 9:59:41 PM, you wrote: At 11:49 AM -0500 8/11/06, Paul Schmehl wrote: I know we are used to dealing in internet-time, where things happen instantly, but there could be many reasons that the host count is only 1612. Reasons that have nothing to do with the specific outcome of how these security issues are handled. I am certainly all for the improvements people have been talking about. I'm just saying that even if you make all those improvements, you're probably going to have to wait a few weeks before we see any significant number of hosts show up. That's just the way it is. It would be nice to see this in base system, that would help to raise this number enourmously. And surely it would be nice to see it somewhere under the freebsd.org domain. Actually, I've registered bsdstats.org for this ... I've been talking to various ppl from the other *BSDs about getting them involved as well, so went with the more 'neutral' domain instead of making this FreeBSD Only ... we share alot between us as it is, sharing marketing power is a good thing ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/11/06, Matthew Seaman [EMAIL PROTECTED] wrote: Marc G. Fournier wrote: On Fri, 11 Aug 2006, Nikolas Britton wrote: Ok... With my new script it took only 158 minutes to compute ALL TCP/IP address hashes. I'll repeat that... I have an md5 hash for every IP address in the world! All I need to do is grep your hash and it will tell me your IP address. yippee! :-) Can someone please explain to me what exactly you are trying to secure against in this case? He's trying to prevent any possibility of information disclosure about his servers. If I wanted to hack into his site, knowing what hosts he had running (ie. a bunch of live IP numbers) and what OS etc. each used would mean I'm already halfway to my goal. Now, while the design of bsdstats does not disclose that sort of stuff readily, any security conscious admin is going to worry about that data being collected and held outside of his administrative control. Having a completely anonymous and untraceable token to identify each of the hosts sending in information should make connecting the information back to the original sender practically impossible. YES! what he said... I don't want ANYTHING to trace back to me or my systems. Although, playing devil's advocate here, anyone that could steal the Apache log files from the bsdstats server would be able to work out that sort of data fairly readily. I guess the truly paranoid should only submit their data via some sort of anonymizing proxy. That's simple, don't keep the log files... * Can we trust Marc to delete them? * I thought this was going to be an official FreeBSD project hosted on freebsd.org? * Maybe we should get the OpenBSD people involved? Just thinking out loud :-/ -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/9/06, Nikolas Britton [EMAIL PROTECTED] wrote:
On 8/9/06, Marc G. Fournier [EMAIL PROTECTED] wrote:
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Marc G. Fournier wrote:
On Wed, 9 Aug 2006, Igor Robul wrote:
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote:
Could create problems long term .. one thing I will be using the
IPs to do is:
SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC;
to look for any 'abnormalities' like todays with Armenia ...
hashing it would make stuff like that fairly difficult ...
You can make _two_ hashes and then concatenate to form unique key.
Then you still be able to see a lot of single IPs. Personaly, I dont
care very much about IP/hostname disclosure :-)
Except that you are disclosing that each and every time you send out an
email, or hit a web site ... :)
The systems I'm concerned about are on private IP space, to not send email
and don't have X installed, much less a web browser and can only access
certain FreeBSD sites to update ports. In fact, they're not even accessible
from *inside* our network except from certain hosts. In order to
successfully run the stats script on these hosts, I would have to open a
hole
in the firewall to bsdstats.hub.org on the correct port.
And yes, I *am* paranoid. But if you really want *all* statistics you can
get, then you'll have to deal with us paranoid types. My workstation, which
is on a public IP, is already registered.
Done ... now I really hope that the US stats rise, maybe? I have a hard
time believing that Russia and the Ukraine have more deployments then the
'good ol'US of A' ... or do they? *raised eyebrow*
Here is what is now stored in the database (using my IP as a basis)
# select * from systems where ip = md5('24.224.179.167');
id |ip| hostname
| operating_system | release | architecture | country |report_date
--+--+--+--++--+-+---
1295 | 45c80b9266a5a6683eee9c9798bd6575 | 4a9110019f2ca076407ed838bf190017
| FreeBSD | 6.1-RC1| i386 | CA | 2006-08-09
02:34:05.12579
1 | 45c80b9266a5a6683eee9c9798bd6575 | 9a45e58ab9535d89f0a7d2092b816364
| FreeBSD | 6.1-STABLE | i386 | CA | 2006-08-09
16:01:03.34788
Why don't you just broadcast the ip address, it's what your doing now
anyways. 253^4 is a very small number.
infomatic# perl
my $num = 0;
system date;
while ($num = 409715208) {
$num++
}
system date;
Wed Aug 9 18:18:45 CDT 2006
Wed Aug 9 18:20:48 CDT 2006
2 minutes * 10 = 20 minutes to iterate though 4 billion IP addresses
on a very slow uni-proc system. I could even store every IP to md5
hash using less then 222GB of uncompressed space.
If you want... give me the md5 hash of a real ip address that is
unknown to me and I will hand you the ip address in two days... or
less. run the IP address though like this:
md5 -s xxx.xxx.xxx.xxx
I have other things to do with my time, so I don't really want to do
this, but if that's what it takes to stop this idea dead I'll do it.
Here's a better way to explain the problem:
Let's say we need to find Marc's IP address but we only have it's md5
hash value. Some of you may think this is hard to do but it's not. All
we need to do is compute every IP address into a hash and then match
Marc's hash to one in are list:
24.224.179.164 = e7e7a967c5f88d9fb10a1f22cd2133d2
24.224.179.165 = 3aa9b50aa7190f5aca1f78f075dc69c2
24.224.179.166 = c695175e48d649e3496ac715406a488d
24.224.179.167 = 45c80b9266a5a6683eee9c9798bd6575
So what is an IP address?... mathematically speaking it's 4 base 255
numbers grouped together:
{0, ..., 255}.{0, ..., 255}.{0, ..., 255}.{0, ..., 255}
To calculate how many combinations there could be you simply take the
base unit and raise it to the 4th power, since there are 4 of them.
This gives us 255^4 combinations or 4,228,250,625 TCP/IP addresses. We
also know that the first number can't be 0 or 255 and the others can't
be 255, we can also rule out all 127.x.y.z loopback and multicast
224.x.y.z - 239.x.y.z addresses:
(237^1) * (254^3)
This leaves us with 3,883,734,168 valid IP addresses. We can divide
this number by 5,000 and run it through a simple perl script to get a
time estimate on how long it will take to compute all these hashs. We
will split it into 4 parallel jobs:
my $number = 0;
while ($number = 194187) {
system md5 -s $number /usr/data/hashlist1;
$number++;
}
my $number = 194188;
while ($number = 388373) {
system md5 -s $number /usr/data/hashlist2;
$number++;
}
my $number = 388374;
while ($number = 582560) {
system md5 -s $number /usr/data/hashlist3;
$number++;
}
my $number = 582561;
while ($number = 776747) {
system md5 -s $number /usr/data/hashlist4;
$number++;
}
Ok, it took
Re: BSDstats Project v2.0 ...
On 8/10/06, Nikolas Britton [EMAIL PROTECTED] wrote:
On 8/9/06, Nikolas Britton [EMAIL PROTECTED] wrote:
On 8/9/06, Marc G. Fournier [EMAIL PROTECTED] wrote:
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Marc G. Fournier wrote:
On Wed, 9 Aug 2006, Igor Robul wrote:
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote:
Could create problems long term .. one thing I will be using the
IPs to do is:
SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC;
to look for any 'abnormalities' like todays with Armenia ...
hashing it would make stuff like that fairly difficult ...
You can make _two_ hashes and then concatenate to form unique key.
Then you still be able to see a lot of single IPs. Personaly, I dont
care very much about IP/hostname disclosure :-)
Except that you are disclosing that each and every time you send out an
email, or hit a web site ... :)
The systems I'm concerned about are on private IP space, to not send email
and don't have X installed, much less a web browser and can only access
certain FreeBSD sites to update ports. In fact, they're not even
accessible
from *inside* our network except from certain hosts. In order to
successfully run the stats script on these hosts, I would have to open a
hole
in the firewall to bsdstats.hub.org on the correct port.
And yes, I *am* paranoid. But if you really want *all* statistics you can
get, then you'll have to deal with us paranoid types. My workstation,
which
is on a public IP, is already registered.
Done ... now I really hope that the US stats rise, maybe? I have a hard
time believing that Russia and the Ukraine have more deployments then the
'good ol'US of A' ... or do they? *raised eyebrow*
Here is what is now stored in the database (using my IP as a basis)
# select * from systems where ip = md5('24.224.179.167');
id |ip| hostname
| operating_system | release | architecture | country |report_date
--+--+--+--++--+-+---
1295 | 45c80b9266a5a6683eee9c9798bd6575 |
4a9110019f2ca076407ed838bf190017 | FreeBSD | 6.1-RC1| i386 | CA
| 2006-08-09 02:34:05.12579
1 | 45c80b9266a5a6683eee9c9798bd6575 |
9a45e58ab9535d89f0a7d2092b816364 | FreeBSD | 6.1-STABLE | i386 | CA
| 2006-08-09 16:01:03.34788
Why don't you just broadcast the ip address, it's what your doing now
anyways. 253^4 is a very small number.
infomatic# perl
my $num = 0;
system date;
while ($num = 409715208) {
$num++
}
system date;
Wed Aug 9 18:18:45 CDT 2006
Wed Aug 9 18:20:48 CDT 2006
2 minutes * 10 = 20 minutes to iterate though 4 billion IP addresses
on a very slow uni-proc system. I could even store every IP to md5
hash using less then 222GB of uncompressed space.
If you want... give me the md5 hash of a real ip address that is
unknown to me and I will hand you the ip address in two days... or
less. run the IP address though like this:
md5 -s xxx.xxx.xxx.xxx
I have other things to do with my time, so I don't really want to do
this, but if that's what it takes to stop this idea dead I'll do it.
Here's a better way to explain the problem:
Let's say we need to find Marc's IP address but we only have it's md5
hash value. Some of you may think this is hard to do but it's not. All
we need to do is compute every IP address into a hash and then match
Marc's hash to one in are list:
24.224.179.164 = e7e7a967c5f88d9fb10a1f22cd2133d2
24.224.179.165 = 3aa9b50aa7190f5aca1f78f075dc69c2
24.224.179.166 = c695175e48d649e3496ac715406a488d
24.224.179.167 = 45c80b9266a5a6683eee9c9798bd6575
So what is an IP address?... mathematically speaking it's 4 base 255
numbers grouped together:
{0, ..., 255}.{0, ..., 255}.{0, ..., 255}.{0, ..., 255}
To calculate how many combinations there could be you simply take the
base unit and raise it to the 4th power, since there are 4 of them.
This gives us 255^4 combinations or 4,228,250,625 TCP/IP addresses. We
also know that the first number can't be 0 or 255 and the others can't
be 255, we can also rule out all 127.x.y.z loopback and multicast
224.x.y.z - 239.x.y.z addresses:
(237^1) * (254^3)
This leaves us with 3,883,734,168 valid IP addresses. We can divide
this number by 5,000 and run it through a simple perl script to get a
time estimate on how long it will take to compute all these hashs. We
will split it into 4 parallel jobs:
my $number = 0;
while ($number = 194187) {
system md5 -s $number /usr/data/hashlist1;
$number++;
}
my $number = 194188;
while ($number = 388373) {
system md5 -s $number /usr/data/hashlist2;
$number++;
}
my $number = 388374;
while ($number = 582560) {
system md5 -s $number /usr/data/hashlist3;
$number++;
}
Re: BSDstats Project v2.0 ...
On Aug 8, 2006, at 5:30 PM, Marc G. Fournier wrote: On Wed, 9 Aug 2006, Antony Mawer wrote: On 9/08/2006 9:16 AM, Marc G. Fournier wrote: Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... Marc G. Fournier Hub.Org Networking Services (http:// www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions- [EMAIL PROTECTED] Yes, that's true particularly if the server's were all the same hardware type and the software was compiled at the same time. Maybe my CPUID suggestion would come in handy? Also, maybe that person from Armenia installed the script in a distribution that's included in a virtual image (vmware comes to mind), and he's loading it on a bunch of different machines behind a (virtual) NAT or something... just a thought to consider. -Garrett ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote: Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... You can make _two_ hashes and then concatenate to form unique key. Then you still be able to see a lot of single IPs. Personaly, I dont care very much about IP/hostname disclosure :-) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/9/06, Chris [EMAIL PROTECTED] wrote: Nikolas Britton wrote: On 8/6/06, Marc G. Fournier [EMAIL PROTECTED] wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... This line is wrong: hptmv (1)Marvell Semiconductor (Was: Galileo Technology Ltd)MV88SX5081 8-port SATA PCI-X Controller1 Also why not track the ones with no driver attached... you should still be able to tell what the device is. How about some uptime stats as well? i don't see my tiny poor country philippines in the list? i already run /usr/local/etc/periodic/monthly/300.statistics btw is the syntax correct? monthly_statistics_enable=yes monthly_statistics_report_devices=yes or should the yes be YES ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Garrett Cooper wrote: Also, maybe that person from Armenia installed the script in a distribution that's included in a virtual image (vmware comes to mind), and he's loading it on a bunch of different machines behind a (virtual) NAT or something... just a thought to consider. If that's the case, those numbers should come back again in Sept ... but, the hostnames for the odd ones were all: http://www.domain.am; with the quotes included, which seemed a really odd value for 'hostname' to have produced :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Igor Robul wrote: On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote: Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... You can make _two_ hashes and then concatenate to form unique key. Then you still be able to see a lot of single IPs. Personaly, I dont care very much about IP/hostname disclosure :-) Except that you are disclosing that each and every time you send out an email, or hit a web site ... :) Regardless, though ... what do ppl suggest here? Simple 'md5' hash? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, jan gestre wrote: /usr/local/etc/periodic/monthly/300.statistics btw is the syntax correct? monthly_statistics_enable=yes monthly_statistics_report_devices=yes or should the yes be YES ? syntax is correct, and you are now on the countries list :) thx Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: If that's the case, those numbers should come back again in Sept ... but, the hostnames for the odd ones were all: http://www.domain.am; with the quotes included, which seemed a really odd value for 'hostname' to have produced :) Looks like a directadmin host. Moreover, resolves to an IP which is not in Armenia. Thought you were using some kind of IP to Country db like GeoIP to find geographic locations of the hosts. Otherwise, domains under f.e. .com gonna be shown as USA? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/9/06, Igor Robul [EMAIL PROTECTED] wrote: On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote: Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... You can make _two_ hashes and then concatenate to form unique key. Then you still be able to see a lot of single IPs. Personaly, I dont care very much about IP/hostname disclosure :-) I still like my idea the best for unique keys. It's a better way to detect hosts behind NATs, here it is again, four versions to pick from: # ifconfig | sha256 cbcc2f55a340c248af7e8a10871150d827af11d7051bbc782eefa04b0603248b # ifconfig | sha1 b607b9d45e6ad40c02ab20800e0d70245ab6db68 # ifconfig | md5 22a2a3eca61166fb113f1a688b3dd842 # ifconfig | cksum 3977021799 540 The only down side is it still can be faked, just like everything else. -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, Aug 09, 2006 at 05:54:26AM -0300, Marc G. Fournier wrote: Except that you are disclosing that each and every time you send out an email, or hit a web site ... :) Original poster concerned about this because he does not normaly use his servers for this kind of work, if I had understood him correctly these servers are for internal use only, and while they can connect to Internet, he is worried about secrets. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, Aug 09, 2006 at 05:41:55AM -0500, Nikolas Britton wrote: # ifconfig | sha256 cbcc2f55a340c248af7e8a10871150d827af11d7051bbc782eefa04b0603248b # ifconfig | sha1 b607b9d45e6ad40c02ab20800e0d70245ab6db68 # ifconfig | md5 22a2a3eca61166fb113f1a688b3dd842 # ifconfig | cksum 3977021799 540 The only down side is it still can be faked, just like everything else. IP from which connection is made cannot be faked, at least I dont know how to fake it. So there is at least one unfakable part of key. But there is no real need to keep real IP in database, for privacy reasons it is better to keep one-way hash in database. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/9/06, Nikolas Britton [EMAIL PROTECTED] wrote: On 8/9/06, Igor Robul [EMAIL PROTECTED] wrote: On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote: Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... You can make _two_ hashes and then concatenate to form unique key. Then you still be able to see a lot of single IPs. Personaly, I dont care very much about IP/hostname disclosure :-) I still like my idea the best for unique keys. It's a better way to detect hosts behind NATs, here it is again, four versions to pick from: # ifconfig | sha256 cbcc2f55a340c248af7e8a10871150d827af11d7051bbc782eefa04b0603248b # ifconfig | sha1 b607b9d45e6ad40c02ab20800e0d70245ab6db68 # ifconfig | md5 22a2a3eca61166fb113f1a688b3dd842 # ifconfig | cksum 3977021799 540 The only down side is it still can be faked, just like everything else. Based on the man pages: http://www.freebsd.org/cgi/man.cgi? md5 first appeared in 1.1.5.1-RELEASE sha1 first appeared in 4.10-RELEASE sha256 first appeared in 6.0-RELEASE, 5.5-RELEASE. That rules out sha256 and sha1, cksum was never a contender so this leaves md5. -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Someone mentioned having output from the script so you would know it was
running. This patch would do that, if you want to add that functionality.
--- 300.statistics.orig Wed Aug 9 09:49:35 2006
+++ 300.statistics Wed Aug 9 09:54:17 2006
@@ -44,6 +44,7 @@
SYS=`/usr/bin/uname -r`
ARCH=`/usr/bin/uname -m`
do_fetch getid.php?hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
+ echo Posting monthly OS statistics to bsdstats.hub.org\n
case $monthly_statistics_report_devices in
[Yy][Ee][Ss])
IFS=
@@ -57,6 +58,7 @@
DEV=`echo $line | awk '{print $4}' | cut -c8-11`
do_fetch
report_device.php?driver=$DRIVER\vendor=$VEN\device=$DEV\hn=$HN
done
+echo Posting monthly device statistics to bsdstats.hub.org\n
line=$( sysctl -n hw.model )
VEN=$( echo $line | cut -d ' ' -f 1 )
@@ -69,6 +71,7 @@
do_fetch
report_cpu.php?cpu_id=CPU$n\vendor=$VEN\cpu_type=$DEV\hn=$HN
n=$(( $n + 1 ))
done
+echo Posting monthly CPU statistics to bsdstats.hub.org\n
;;
esac
--
Paul Schmehl ([EMAIL PROTECTED])
Adjunct Information Security Officer
The University of Texas at Dallas
http://www.utdallas.edu/ir/security/
smime.p7s
Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Wed, 9 Aug 2006, Igor Robul wrote: On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote: Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... You can make _two_ hashes and then concatenate to form unique key. Then you still be able to see a lot of single IPs. Personaly, I dont care very much about IP/hostname disclosure :-) Except that you are disclosing that each and every time you send out an email, or hit a web site ... :) The systems I'm concerned about are on private IP space, to not send email and don't have X installed, much less a web browser and can only access certain FreeBSD sites to update ports. In fact, they're not even accessible from *inside* our network except from certain hosts. In order to successfully run the stats script on these hosts, I would have to open a hole in the firewall to bsdstats.hub.org on the correct port. And yes, I *am* paranoid. But if you really want *all* statistics you can get, then you'll have to deal with us paranoid types. My workstation, which is on a public IP, is already registered. Regardless, though ... what do ppl suggest here? Simple 'md5' hash? I think md5 is fine. SHA256 would probably be better. :-) -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
Igor Robul wrote: The only down side is it still can be faked, just like everything else. IP from which connection is made cannot be faked, at least I dont know how to fake it. So there is at least one unfakable part of key. But there is no real need to keep real IP in database, for privacy reasons it is better to keep one-way hash in database. We're using PAT. That means that, when I use a private host to access the internet, I could be on any one of a number of IP addresses. However, I was assuming that Marc is using the IP reported by ifconfig, which *should* be unique for each host, as opposed to the IP that connects to him, which could represent literally thousands of hosts in some cases. -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Marc G. Fournier wrote:
On Wed, 9 Aug 2006, Igor Robul wrote:
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote:
Could create problems long term .. one thing I will be using the
IPs to do is:
SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC;
to look for any 'abnormalities' like todays with Armenia ...
hashing it would make stuff like that fairly difficult ...
You can make _two_ hashes and then concatenate to form unique key.
Then you still be able to see a lot of single IPs. Personaly, I dont
care very much about IP/hostname disclosure :-)
Except that you are disclosing that each and every time you send out an
email, or hit a web site ... :)
The systems I'm concerned about are on private IP space, to not send email
and don't have X installed, much less a web browser and can only access
certain FreeBSD sites to update ports. In fact, they're not even accessible
from *inside* our network except from certain hosts. In order to
successfully run the stats script on these hosts, I would have to open a hole
in the firewall to bsdstats.hub.org on the correct port.
And yes, I *am* paranoid. But if you really want *all* statistics you can
get, then you'll have to deal with us paranoid types. My workstation, which
is on a public IP, is already registered.
Done ... now I really hope that the US stats rise, maybe? I have a hard
time believing that Russia and the Ukraine have more deployments then the
'good ol'US of A' ... or do they? *raised eyebrow*
Here is what is now stored in the database (using my IP as a basis)
# select * from systems where ip = md5('24.224.179.167');
id |ip| hostname | operating_system | release | architecture | country |report_date
--+--+--+--++--+-+---
1295 | 45c80b9266a5a6683eee9c9798bd6575 | 4a9110019f2ca076407ed838bf190017 |
FreeBSD | 6.1-RC1| i386 | CA | 2006-08-09
02:34:05.12579
1 | 45c80b9266a5a6683eee9c9798bd6575 | 9a45e58ab9535d89f0a7d2092b816364 |
FreeBSD | 6.1-STABLE | i386 | CA | 2006-08-09
16:01:03.34788
And yup, I have two hosts sitting behind a router ...
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED]
Yahoo . yscrappy Skype: hub.orgICQ . 7615664
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Paul Schmehl wrote: Igor Robul wrote: The only down side is it still can be faked, just like everything else. IP from which connection is made cannot be faked, at least I dont know how to fake it. So there is at least one unfakable part of key. But there is no real need to keep real IP in database, for privacy reasons it is better to keep one-way hash in database. We're using PAT. That means that, when I use a private host to access the internet, I could be on any one of a number of IP addresses. However, I was assuming that Marc is using the IP reported by ifconfig, which *should* be unique for each host, as opposed to the IP that connects to him, which could represent literally thousands of hosts in some cases. ifconfig most definitely wouldn't be unique for each host ... ifconfig on my machines here would show 192.168.1.2 and 192.168.1.99 ... I have no idea how many, but I imagine there are *alot* of hosts behind a NAT, or router, that would show those same numbers ... The uniqueness is a combination of IP+hostname ... again, as one pointed out with PCBSD, this isn't always necessarily the case, but, IMHO, that is a flaw of PCBSD having all hosts on the same network using the same hostname ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Vahan Yerkanian wrote: Marc G. Fournier wrote: If that's the case, those numbers should come back again in Sept ... but, the hostnames for the odd ones were all: http://www.domain.am; with the quotes included, which seemed a really odd value for 'hostname' to have produced :) Looks like a directadmin host. Moreover, resolves to an IP which is not in Armenia. Thought you were using some kind of IP to Country db like GeoIP to find geographic locations of the hosts. Otherwise, domains under f.e. .com gonna be shown as USA? I'm using GeoIP for this, based on the IP that is IP of the connection ... this is one flaw, IMHO, to using md5, its going to be a bit harder to spot stuff like this ... but, not impossible either ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
With minor mods, committed ... I moved bsdstats.hub.org to a variable, and
added an 'echo' for when the stats, or a part of them, is disabled, that
way if this ever does get into the base system, ppl reading monthly run
output will know that they exist, and how to turn it on ...
thx ...
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Someone mentioned having output from the script so you would know it was
running. This patch would do that, if you want to add that functionality.
--- 300.statistics.orig Wed Aug 9 09:49:35 2006
+++ 300.statistics Wed Aug 9 09:54:17 2006
@@ -44,6 +44,7 @@
SYS=`/usr/bin/uname -r`
ARCH=`/usr/bin/uname -m`
do_fetch getid.php?hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
+ echo Posting monthly OS statistics to bsdstats.hub.org\n
case $monthly_statistics_report_devices in
[Yy][Ee][Ss])
IFS=
@@ -57,6 +58,7 @@
DEV=`echo $line | awk '{print $4}' | cut -c8-11`
do_fetch
report_device.php?driver=$DRIVER\vendor=$VEN\device=$DEV\hn=$HN
done
+echo Posting monthly device statistics to bsdstats.hub.org\n
line=$( sysctl -n hw.model )
VEN=$( echo $line | cut -d ' ' -f 1 )
@@ -69,6 +71,7 @@
do_fetch
report_cpu.php?cpu_id=CPU$n\vendor=$VEN\cpu_type=$DEV\hn=$HN
n=$(( $n + 1 ))
done
+echo Posting monthly CPU statistics to bsdstats.hub.org\n
;;
esac
--
Paul Schmehl ([EMAIL PROTECTED])
Adjunct Information Security Officer
The University of Texas at Dallas
http://www.utdallas.edu/ir/security/
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED]
Yahoo . yscrappy Skype: hub.orgICQ . 7615664
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: The uniqueness is a combination of IP+hostname ... again, as one pointed out with PCBSD, this isn't always necessarily the case, but, IMHO, that is a flaw of PCBSD having all hosts on the same network using the same hostname ... That's the nice thing with the 'ifconfig|sha256' scheme. Because it would include the MAC address of the interfaces in the hash, the only 'identical' machines would be ones with no ethernet interfaces at all. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Howard Jones wrote: Marc G. Fournier wrote: The uniqueness is a combination of IP+hostname ... again, as one pointed out with PCBSD, this isn't always necessarily the case, but, IMHO, that is a flaw of PCBSD having all hosts on the same network using the same hostname ... That's the nice thing with the 'ifconfig|sha256' scheme. Because it would include the MAC address of the interfaces in the hash, the only 'identical' machines would be ones with no ethernet interfaces at all. Right, and the bad thing is if yu alias another IP on that device, the hash totally changes, so we see that one host now as being two different ones :) That's why we disqualified using ifconfig right at the beginning ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tuesday 08 August 2006 9:17 am, Marc G. Fournier wrote: But, there is no such ting as an 'index number' ... when everyone reports in next month, for instance, there is no 'number' that will be re-used for them that matches something used this month ... What about: indexnumber=$(md5 -q /etc/ssh/ssh_host_rsa_key.pub) That file gets generated the first time a host is booted with sshd_enable=YES and almost never changes afterward. Also, literally every BSD machine I've ever touched has sshd enabled (although usually severely locked down), and I suspect that's true for most people. -- Kirk Strauser ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Up to v2.2 ... ( Was: Re: BSDstats Project v2.0 ... )
On Tue, 8 Aug 2006, [EMAIL PROTECTED] wrote: --On August 9, 2006 9:32:18 AM +1000 Antony Mawer [EMAIL PROTECTED] wrote: On 9/08/2006 9:16 AM, Marc G. Fournier wrote: Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... I like that idea. I'm ready to submit my workstation, but I'm still a bit hesitant about some servers I adminA one way hash would alleviate my concerns. 'k, v2.2 brings us up to hashed unique keys, for more anonymity, and we've just added 'class' and 'subclass' to the devices report, so that we can improve the reporting, namely, so that we can group things better (ie. all RAID controllers or all ethernet controllers), that sort of thing ... the devices list is getting a bit big to load right now ... the 'all devices' list will still be available, but, for instance, ppl looking to see 'most popular ethernet controller', this should help speed things up a bit ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: Right, and the bad thing is if yu alias another IP on that device, the hash totally changes, so we see that one host now as being two different ones :) That's why we disqualified using ifconfig right at the beginning ... But didn't you say that you effectively wipe the database once a month, (or expire entries over that age)? I can't find the post that mentioned that now, naturally... :-) if you aren't using the 'key' as a database key, then what do you care that it changes as long as it uniquely identifies the system (which it definitely would)? I don't know how typical I am, but I don't really remember the last time I added an IP alias on a running server, for our few dozen production systems. I would imagine that those types of changes might well be lost of systems coming and going. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Howard Jones wrote: Marc G. Fournier wrote: Right, and the bad thing is if yu alias another IP on that device, the hash totally changes, so we see that one host now as being two different ones :) That's why we disqualified using ifconfig right at the beginning ... But didn't you say that you effectively wipe the database once a month, (or expire entries over that age)? I can't find the post that mentioned that now, naturally... :-) if you aren't using the 'key' as a database key, then what do you care that it changes as long as it uniquely identifies the system (which it definitely would)? I don't know how typical I am, but I don't really remember the last time I added an IP alias on a running server, for our few dozen production systems. I would imagine that those types of changes might well be lost of systems coming and going. I add/remove IPs from our servers several times each week, as we add VPS and remove them, or move then between boxes ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Wed, 9 Aug 2006, Howard Jones wrote: Marc G. Fournier wrote: Right, and the bad thing is if yu alias another IP on that device, the hash totally changes, so we see that one host now as being two different ones :) That's why we disqualified using ifconfig right at the beginning ... But didn't you say that you effectively wipe the database once a month, (or expire entries over that age)? I can't find the post that mentioned that now, naturally... :-) if you aren't using the 'key' as a database key, then what do you care that it changes as long as it uniquely identifies the system (which it definitely would)? I don't know how typical I am, but I don't really remember the last time I added an IP alias on a running server, for our few dozen production systems. I would imagine that those types of changes might well be lost of systems coming and going. I add/remove IPs from our servers several times each week, as we add VPS and remove them, or move then between boxes ... This problem is intractable: any scheme you can think of to generate a unique identifying number on a random host out there on the net will either fail to actually be unique, or suffer from mutating over time as machine configuration changes. How about the following. Use the bsdstats.hub.org to generate a random token and hand it to the client. 128 bits of randomness gives a sufficiently large domain (340,282,366,920,938,463,463,374,607,431,768,211,456 different possible combinations) that given a good RNG collisions are not a problem. You can generate that sort of token easily by, for example: % openssl rand -base64 16 KSOWkPuK03Od99S5vaPGdQ== Base64 encoded strings will have to be URL escaped if they are passed as parameters in a HTTP GET -- perhaps encoding as a string of hex digits might be a better idea: % openssl rand 16 | hexdump -e '16/1 %01x \n' 566fc9f2374a7e999d9587dc143373fc Anyhow, that's just implementation detail. So the transaction would go like this the first time a client machine tried to report its configuration: ClientServer - Check for cached ID token Not found Request new token from server -- Generate token Record it in DB Return token to client -- Cache token in file Generate OS version info Send to server with ID token --- If token is known, record data in DB Generate Driver info Send to server with ID token --- If token is known, record data in DB etc. etc. - Because the server generates the tokens, it knows which ones are valid, and can discard any data sent to it without a valid token. That doesn't prevent any vandal-minded person from requesting a metric butt-load of tokens to spam the database with, but that's no worse than the current situation. The neat thing is, the number of available tokens is so huge that it is infeasible to guess or accidentally collide with someone else's token. Eg. At 100Mb/s it would take about 10^33 seconds or 10^25 years to exhaustively search the whole token space. Thus spammed data will just time out at the end of the month without affecting anyone else's real data. Stealing an existing ID token by breaking into a machine or snooping on the net would be possible, but presumably sufficiently difficult to do in a large enough quantity that it wouldn't have a significant effect on the overall statistics. If snooping turns out to be a real problem, then using HTTPS is a possibility, but that will ramp up the load on the server quite a bit. For subsequent updates, the client machine just reuses the same token out of its cache file. If the cached token gets deleted, then the client machine will just have to request a new one and rely on the old data timing out at the end of the month. Saving away the token should be simple -- just make the server return the data to a 'get_token' query as MIME type text/plain and have fetch dump it in a cache file somewhere. /var/db/bsdstats for example. I can code up the client side of this in about 5 minutes, but the server end of things will take a little more work. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
Nikolas Britton wrote: I still like my idea the best for unique keys. It's a better way to detect hosts behind NATs, here it is again, four versions to pick from: # ifconfig | sha256 cbcc2f55a340c248af7e8a10871150d827af11d7051bbc782eefa04b0603248b # ifconfig | sha1 b607b9d45e6ad40c02ab20800e0d70245ab6db68 # ifconfig | md5 22a2a3eca61166fb113f1a688b3dd842 # ifconfig | cksum 3977021799 540 The only down side is it still can be faked, just like everything else. ifconfig output is by no means constant on a single host. Eg. Take my laptop; the media, status and ssid lines will change pretty often on my wireless nic. I mean several times during one session. Why not hash just the hostname? Or MAC-address? Of course these could also be fabricated, but you can't possibly avoid that as long as this is open source. (And the protocol would be pretty easy to reverse engineer anyway) How 'bout? $ ifconfig | grep ether | md5 This will change whenever one adds, removes or replaces a nic, though. Svein Halvor signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
Svein Halvor Halvorsen wrote: Why not hash just the hostname? Or MAC-address? Of course these could Disregard this. I see that the discussion has moved on. I'm with Matthew Seaman's suggested server generated id-string. Svein Halvor signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
In response to Matthew Seaman [EMAIL PROTECTED]: This problem is intractable: any scheme you can think of to generate a unique identifying number on a random host out there on the net will either fail to actually be unique, or suffer from mutating over time as machine configuration changes. Really? What if you just generate some sort of UID or GUID and store it in /var/db/bsdstats.guid (or similar)? If the file exists, use it, if it doesn't exist, generate a new ID. Not 100% error prone, but should be pretty damn reliable. -- Bill Moran Collaborative Fusion Inc. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
At 9:32 AM +1000 8/9/06, Antony Mawer wrote: What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... A scattered list of ideas: It might be useful to keep part of the domain-name in plain-text. Just a minimal part, such as '.edu' or '.co.uk'. So that would be one value sent/saved. Then have an MD5 hash of `hostname` (hashing the full hostname, including full domain), or maybe a hash of the output from: hostname ; ifconfig | grep ether Eg: hostname ; ifconfig | grep ether freefour.acs.rpi.edu ether 00:09:5b:01:02:03 ether 00:11:09:09:08:07 (this machine has two ethernet cards in it, and no, those are not the real MAC addresses of the cards... :-) == (hostname ; ifconfig | grep ether) | md5 0670be39b40dc52d996e1a6dcee6cca7 Maybe combine that with the partial-domain, to get 0670be39b40dc52d996e1a6dcee6cca7.edu Further, whatever value you decide to use to create a unique value, you could just save that value away in some file under /var/db . If the file does not exist, then create it and store the probably-unique value. That way you can pick some algorithm which should produce a unique result, and not worry if the value of that algorithm might change (on a single machine) over time. You'll only calculate it once, and then keep using that result. -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, Aug 09, 2006 at 03:16:29PM -0400, Bill Moran wrote: In response to Matthew Seaman [EMAIL PROTECTED]: This problem is intractable: any scheme you can think of to generate a unique identifying number on a random host out there on the net will either fail to actually be unique, or suffer from mutating over time as machine configuration changes. Really? What if you just generate some sort of UID or GUID and store it in /var/db/bsdstats.guid (or similar)? Well, exactly. What I neglected to say in the above was to generate a unique identifying number that encodes part of the machine configuration. However, you're right in that the client could just invent its own random ID number. Given the large number of possible ID numbers in the scheme I proposed, there shouldn't be any problem with collisions so long as all those machines are generating good random numbers[1]. On reflection, the advantages of having the server generate the ID numbers are not really all that compelling. Cheers, Matthew [1] In fact, it would be a pretty neat experiment to get a whole load of machines to generate a chunk'o'randomness and send it into a central machine and see just how evenly distributed the answers are. -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW pgpLb61vrzzw3.pgp Description: PGP signature
Re: BSDstats Project v2.0 ...
On 8/9/06, Marc G. Fournier [EMAIL PROTECTED] wrote:
On Wed, 9 Aug 2006, Paul Schmehl wrote:
Marc G. Fournier wrote:
On Wed, 9 Aug 2006, Igor Robul wrote:
On Tue, Aug 08, 2006 at 09:30:42PM -0300, Marc G. Fournier wrote:
Could create problems long term .. one thing I will be using the
IPs to do is:
SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC;
to look for any 'abnormalities' like todays with Armenia ...
hashing it would make stuff like that fairly difficult ...
You can make _two_ hashes and then concatenate to form unique key.
Then you still be able to see a lot of single IPs. Personaly, I dont
care very much about IP/hostname disclosure :-)
Except that you are disclosing that each and every time you send out an
email, or hit a web site ... :)
The systems I'm concerned about are on private IP space, to not send email
and don't have X installed, much less a web browser and can only access
certain FreeBSD sites to update ports. In fact, they're not even accessible
from *inside* our network except from certain hosts. In order to
successfully run the stats script on these hosts, I would have to open a hole
in the firewall to bsdstats.hub.org on the correct port.
And yes, I *am* paranoid. But if you really want *all* statistics you can
get, then you'll have to deal with us paranoid types. My workstation, which
is on a public IP, is already registered.
Done ... now I really hope that the US stats rise, maybe? I have a hard
time believing that Russia and the Ukraine have more deployments then the
'good ol'US of A' ... or do they? *raised eyebrow*
Here is what is now stored in the database (using my IP as a basis)
# select * from systems where ip = md5('24.224.179.167');
id |ip| hostname |
operating_system | release | architecture | country |report_date
--+--+--+--++--+-+---
1295 | 45c80b9266a5a6683eee9c9798bd6575 | 4a9110019f2ca076407ed838bf190017 |
FreeBSD | 6.1-RC1| i386 | CA | 2006-08-09
02:34:05.12579
1 | 45c80b9266a5a6683eee9c9798bd6575 | 9a45e58ab9535d89f0a7d2092b816364 |
FreeBSD | 6.1-STABLE | i386 | CA | 2006-08-09
16:01:03.34788
Why don't you just broadcast the ip address, it's what your doing now
anyways. 253^4 is a very small number.
infomatic# perl
my $num = 0;
system date;
while ($num = 409715208) {
$num++
}
system date;
Wed Aug 9 18:18:45 CDT 2006
Wed Aug 9 18:20:48 CDT 2006
2 minutes * 10 = 20 minutes to iterate though 4 billion IP addresses
on a very slow uni-proc system. I could even store every IP to md5
hash using less then 222GB of uncompressed space.
If you want... give me the md5 hash of a real ip address that is
unknown to me and I will hand you the ip address in two days... or
less. run the IP address though like this:
md5 -s xxx.xxx.xxx.xxx
I have other things to do with my time, so I don't really want to do
this, but if that's what it takes to stop this idea dead I'll do it.
--
BSD Podcasts @:
http://bsdtalk.blogspot.com/
http://freebsdforall.blogspot.com/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/08/2006 1:56 PM, David Schulz wrote: Ok i love the Idea of this, and will have all my machines running that in no time. Just make the Site look more sleek :) I will be hopefully the first one representing China on that list as well (brag :-) I'm working on it -- unfortunately have been very busy the past few days but hope to have something ready within the next couple of days. Patience... :-) Cheers Antony ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Mon, 7 Aug 2006, Matthew Seaman wrote: I think it would help uptake if when the bsdstats job is first run, it issues you with a 'registered system number' -- then all of the folks with low numbered systems get bragging rights... Actually, there is no registered system number, as there is no registration process ... that was the key requirement for doing this, was that it was as hands off as possible, and having to go to a web site to register each and every host was just too onerous of a task ... Yeah. You've leapt on the word 'register' there, and not addressed the salient point. 'Register' as in 'become cognisant of', and distinguished from 'signed up to.' Even though the process is simplified and automatic, you're still registering systems. Having some sort of return from the central stats machine when the periodic script runs, (rather than nothing at all, as at the moment) would be a good idea. Giving each system an index number shouldn't cost too much. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Mon, 7 Aug 2006, Chris wrote: Just my .02 worth - that Sparc64 listing Is mine Wheee! There are two Sparc64 listings ... both yours? The 8 in Panama are all mine :) Where's Chile? I just added 4 boxes and they're not listed. Excellent job Marc! Cheers, Mikhail. -- Mikhail Goriachev Webanoide Telephone: +61 (0)3 62252501 Mobile Phone: +61 (0)4 38255158 E-Mail: [EMAIL PROTECTED] Web: http://www.webanoide.org PGP Key ID: 0x4E148A3B PGP Key Fingerprint: D96B 7C14 79A5 8824 B99D 9562 F50E 2F5D 4E14 8A3B ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Hello, i have started to run this script , but for some reason i dont show up in the list. or maybe i do, but at least not the country from which i am submitting, china, has still zero entries. how can this be? my ip does resolve to a host in china when using some geoip lookup service... Thanks ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Mikhail Goriachev wrote: Marc G. Fournier wrote: On Mon, 7 Aug 2006, Chris wrote: Just my .02 worth - that Sparc64 listing Is mine Wheee! There are two Sparc64 listings ... both yours? The 8 in Panama are all mine :) Where's Chile? I just added 4 boxes and they're not listed. You are now :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, David Schulz wrote: Hello, i have started to run this script , but for some reason i dont show up in the list. or maybe i do, but at least not the country from which i am submitting, china, has still zero entries. how can this be? my ip does resolve to a host in china when using some geoip lookup service... I see China - 1 listed ... is that you? :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
cool yes, now i see it also, but it wasn't there before right after i executed my script. is there maybe some sort of delay before the data appears? On Aug 8, 2006, at 4:01 PM, Marc G. Fournier wrote: On Tue, 8 Aug 2006, David Schulz wrote: Hello, i have started to run this script , but for some reason i dont show up in the list. or maybe i do, but at least not the country from which i am submitting, china, has still zero entries. how can this be? my ip does resolve to a host in china when using some geoip lookup service... I see China - 1 listed ... is that you? :) Marc G. Fournier Hub.Org Networking Services (http:// www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions- [EMAIL PROTECTED] !DSPAM:1,44d8464a6298743259228! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote: On Tue, 8 Aug 2006, Mikhail Goriachev wrote: Marc G. Fournier wrote: On Mon, 7 Aug 2006, Chris wrote: Just my .02 worth - that Sparc64 listing Is mine Wheee! There are two Sparc64 listings ... both yours? The 8 in Panama are all mine :) Where's Chile? I just added 4 boxes and they're not listed. You are now :) Awesome! Thanks for that. Cheers, Mikhail. -- Mikhail Goriachev Webanoide Telephone: +61 (0)3 62252501 Mobile Phone: +61 (0)4 38255158 E-Mail: [EMAIL PROTECTED] Web: http://www.webanoide.org PGP Key ID: 0x4E148A3B PGP Key Fingerprint: D96B 7C14 79A5 8824 B99D 9562 F50E 2F5D 4E14 8A3B ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Hello, I think there is at least one error in country naming: should be Kazakhstan instead of Kazakstan. Our friends from Kazakhstan of course can correct me. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, Aug 07, 2006 at 12:42:27AM -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Only out of curiosity. What kind of webserver have you. If only a part of the FreeBSD Users install your script and it got executed at the same time, this will get an awfull lot of load for your server. Bye Estartu Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | EMail: [EMAIL PROTECTED] | on request Germany|| pgpGFZhFEDFkK.pgp Description: PGP signature
Re: BSDstats Project v2.0 ...
On Tuesday 08 August 2006 04:22, David Schulz wrote: cool yes, now i see it also, but it wasn't there before right after i executed my script. is there maybe some sort of delay before the data appears? On Aug 8, 2006, at 4:01 PM, Marc G. Fournier wrote: On Tue, 8 Aug 2006, David Schulz wrote: Hello, i have started to run this script , but for some reason i dont show up in the list. or maybe i do, but at least not the country from which i am submitting, china, has still zero entries. how can this be? my ip does resolve to a host in china when using some geoip lookup service... I see China - 1 listed ... is that you? :) Marc G. Fournier Undoubtedly there is going to be some time lag between the time the data is submitted and the time of its display. How log did you wait before checking to see if the site had been updated with your submission? By the way, please don't top post. It makes it hard to follow the thread. -- Gerard Seibert [EMAIL PROTECTED] Hofstadter's Law: It always takes longer than you expect, even when you take Hofstadter's Law into account. pgpXMM8SObJyT.pgp Description: PGP signature
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Matthew Seaman wrote: Marc G. Fournier wrote: On Mon, 7 Aug 2006, Matthew Seaman wrote: I think it would help uptake if when the bsdstats job is first run, it issues you with a 'registered system number' -- then all of the folks with low numbered systems get bragging rights... Actually, there is no registered system number, as there is no registration process ... that was the key requirement for doing this, was that it was as hands off as possible, and having to go to a web site to register each and every host was just too onerous of a task ... Yeah. You've leapt on the word 'register' there, and not addressed the salient point. 'Register' as in 'become cognisant of', and distinguished from 'signed up to.' Even though the process is simplified and automatic, you're still registering systems. Having some sort of return from the central stats machine when the periodic script runs, (rather than nothing at all, as at the moment) would be a good idea. Giving each system an index number shouldn't cost too much. But, there is no such ting as an 'index number' ... when everyone reports in next month, for instance, there is no 'number' that will be re-used for them that matches something used this month ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, David Schulz wrote: cool yes, now i see it also, but it wasn't there before right after i executed my script. is there maybe some sort of delay before the data appears? Yup, but only as the database grows ... I'm using the pear GeoIP module to determine country, of course, but am only storying the 2 char value in the main table ... that links up with a second 'full name' table that I have a script that I run periodically to populate ... On Aug 8, 2006, at 4:01 PM, Marc G. Fournier wrote: On Tue, 8 Aug 2006, David Schulz wrote: Hello, i have started to run this script , but for some reason i dont show up in the list. or maybe i do, but at least not the country from which i am submitting, china, has still zero entries. how can this be? my ip does resolve to a host in china when using some geoip lookup service... I see China - 1 listed ... is that you? :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] !DSPAM:1,44d8464a6298743259228! ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 07/08/2006 05:42, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... Hi Marc, thank you for your work! Is it considered 'stable' or still in development/testing? Should we go and tell others or is it too early yet? Best regards, Karol -- Karol Kwiatkowski freebsd at orchid dot homeunix dot org OpenPGP: http://www.orchid.homeunix.org/carlos/gpg/0x06E09309.asc signature.asc Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
Marc G. Fournier wrote:
On Tue, 8 Aug 2006, Matthew Seaman wrote:
Marc G. Fournier wrote:
On Mon, 7 Aug 2006, Matthew Seaman wrote:
I think it would help uptake if when the bsdstats job is first run, it
issues you with a 'registered system number' -- then all of the folks
with low numbered systems get bragging rights...
Actually, there is no registered system number, as there is no
registration process ... that was the key requirement for doing this,
was that it was as hands off as possible, and having to go to a web
site to register each and every host was just too onerous of a task ...
Yeah. You've leapt on the word 'register' there, and not addressed the
salient point. 'Register' as in 'become cognisant of', and distinguished
from 'signed up to.' Even though the process is simplified and
automatic,
you're still registering systems.
Having some sort of return from the central stats machine when the
periodic
script runs, (rather than nothing at all, as at the moment) would be a
good
idea. Giving each system an index number shouldn't cost too much.
But, there is no such ting as an 'index number' ... when everyone
reports in next month, for instance, there is no 'number' that will be
re-used for them that matches something used this month ...
You use the hostname as the identifying key for the data that's sent to you.
Presumably you've got to add that hostname to a table somewhere, so that
re-running the 300.statistics script doesn't spam the database with multiples
of the same set of data. i.e. it's the key to a unique index. Suggests that
row number in that table of host names is your unique index number...
Anyhow, how about the following little enhancement. This lists the CPUs on the
system pretending they are CPU0, CPU1, ... devices. The URI escape stuff should
be automatically decoded by PHP without any extra coding required.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW
--- /usr/ports/sysutils/bsdstats/files/300.statistics Tue Aug 8 09:21:34 2006
+++ 300.statistics Tue Aug 8 19:30:08 2006
@@ -14,25 +14,57 @@
oldmask=$(umask)
umask 066
+# RFC 2396
+uri_escape () {
+echo [EMAIL PROTECTED] | sed -e '
+s/%/%25/g
+s/;/%3b/g
+s,/,%2f,g
+s/?/%3f/g
+s/:/%3a/g
+s/@/%40/g
+s//%26/g
+s/=/%3d/g
+s/+/%2b/g
+s/\$/%24/g
+s/,/%2c/g
+s/ /%20/g
+'
+}
+
+do_fetch () {
+/usr/bin/fetch -qo /dev/null http://bsdstats.hub.org/scripts/$1;
+}
+
case $monthly_statistics_enable in
[Yy][Ee][Ss])
HN=`/bin/hostname`
SYS=`/usr/bin/uname -r`
ARCH=`/usr/bin/uname -m`
OS=`/usr/bin/uname -s`
- /usr/bin/fetch -qo /dev/null
http://bsdstats.hub.org/scripts/getid.php?hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
+ do_fetch getid.php?hn=$HN\sys=$SYS\arch=$ARCH\opsys=$OS
case $monthly_statistics_report_devices in
[Yy][Ee][Ss])
IFS=
-/usr/bin/fetch -qo /dev/null
http://bsdstats.hub.org/scripts/clear_devices.php?hn=$HN
+do_fetch clear_devices.php?hn=$HN
for line in `/usr/sbin/pciconf -l | /usr/bin/grep -v none`
do
DRIVER=`echo $line | awk -F\@ '{print $1}'`
VEN=`echo $line | awk '{print $4}' | cut -c12-15`
DEV=`echo $line | awk '{print $4}' | cut -c8-11`
-/usr/bin/fetch -qo /dev/null
http://bsdstats.hub.org/scripts/report_device.php?driver=$DRIVER\vendor=$VEN\device=$DEV\hn=$HN
+do_fetch
report_device.php?driver=$DRIVER\vendor=$VEN\device=$DEV\hn=$HN
+done
+line=$( sysctl -n hw.model )
+VEN=$( echo $line | cut -d ' ' -f 1 )
+DEV=$( uri_escape $( echo $line | cut -d ' ' -f 2- ) )
+n=0
+count=$( sysctl -n hw.ncpu )
+while [ $n -lt $count ]
+do
+do_fetch
report_device.php?driver=CPU$n\vendor=$VEN\device=$DEV\hn=$HN
+n=$(( $n + 1 ))
done
;;
esac
signature.asc
Description: OpenPGP digital signature
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Karol Kwiatkowski wrote: On 07/08/2006 05:42, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... Hi Marc, thank you for your work! Is it considered 'stable' or still in development/testing? Should we go and tell others or is it too early yet? Its was considered stable as soon as the uname output was sent out :) Any additions done are done in such a way that it shouldn't break the previous ones, and are add-ons, not core ... so even someone still running an *old* (now!) v1.0 script won't have any problems, it just means devices reports are sent out for them yet until they upgrade to v2.0 ... So, yes, defintely ... tell anyone and everyone that is running a *BSD system ... I'm having talks with Matt @ DragonflyBSD for one addition he'd like to see added to make sure that the script checks for network connectivity before trying to send its reports, after which he's planning on adding it to Dragonfly's base system install ... but, again, that won't affect older versions ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Gerhard Schmidt wrote: On Mon, Aug 07, 2006 at 12:42:27AM -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Only out of curiosity. What kind of webserver have you. If only a part of the FreeBSD Users install your script and it got executed at the same time, this will get an awfull lot of load for your server. I'm running Apache 2 + PHP 5.x ... with a PostgreSQL 8.1.4 database backedn to it ... I have a new Dual-CPU HP Proliant server on its way, upon which I'll put a second Apache 2 server and setup RR DNS so that both servers will answer and accept reports ... I've got 8 servers in place if things can't keep up, and more being added ... I'm not too worried about either server(s) or bandwidth ... and even less worried seeing that ppl have actually responded well to the whole thing ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Marc, I have a couple of questions. You use hostname and IP as a unique identifier for each host. For that reason, I have not submitted any of our systems. We use FreeBSD for sensitive security-related tasks, and we're loath to reveal that information. (When I submit or update ports, I always alter the uname information to hostname.utdallas.edu for that reason.) Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. Pending your (statisfactory) answer to the hostname-IP questions above, I'll submit our stuff. -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. sudden influx of hosts from Armenia - I installed BSDstats on my FreeBSD 6.1 servers :) Any problem with that? Just FYI, Vahan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. Ehm, actually something's really wrong :) Armenia 331 28.24% shouldn't be that much, I think I installed it on 4 servers :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Vahan Yerkanian wrote: Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. sudden influx of hosts from Armenia - I installed BSDstats on my FreeBSD 6.1 servers :) Any problem with that? Not from where I sit. :-) -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
Vahan Yerkanian wrote: Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. Ehm, actually something's really wrong :) Armenia 331 28.24% shouldn't be that much, I think I installed it on 4 servers :) Watch for the sequel of Servers Gone Wild. ;-) -- Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/ smime.p7s Description: S/MIME Cryptographic Signature
Re: BSDstats Project v2.0 ...
On 8/6/06, Marc G. Fournier [EMAIL PROTECTED] wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... This line is wrong: hptmv (1) Marvell Semiconductor (Was: Galileo Technology Ltd)MV88SX5081 8-port SATA PCI-X Controller 1 Also why not track the ones with no driver attached... you should still be able to tell what the device is. -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Nikolas Britton wrote: On 8/6/06, Marc G. Fournier [EMAIL PROTECTED] wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... This line is wrong: hptmv (1)Marvell Semiconductor (Was: Galileo Technology Ltd)MV88SX5081 8-port SATA PCI-X Controller1 Also why not track the ones with no driver attached... you should still be able to tell what the device is. How about some uptime stats as well? -- Best regards, Chris Anything good in life is either illegal, immoral or fattening. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/8/06, Chris [EMAIL PROTECTED] wrote: Nikolas Britton wrote: On 8/6/06, Marc G. Fournier [EMAIL PROTECTED] wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... This line is wrong: hptmv (1)Marvell Semiconductor (Was: Galileo Technology Ltd)MV88SX5081 8-port SATA PCI-X Controller1 Also why not track the ones with no driver attached... you should still be able to tell what the device is. How about some uptime stats as well? No. We agreed we would not track people. -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Paul Schmehl wrote: Marc, I have a couple of questions. You use hostname and IP as a unique identifier for each host. For that reason, I have not submitted any of our systems. We use FreeBSD for sensitive security-related tasks, and we're loath to reveal that information. (When I submit or update ports, I always alter the uname information to hostname.utdallas.edu for that reason.) Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. Have planned for it, and, in fact, am going to be making a couple of extra changes to the schema to allow for cleaning it up easier ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Vahan Yerkanian wrote: Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. sudden influx of hosts from Armenia - I installed BSDstats on my FreeBSD 6.1 servers :) Any problem with that? 317 servers, all with the exact same IP? I see 9 servers that look legit, and 317 that I would have classified as 'suspicious' ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Vahan Yerkanian wrote: Paul Schmehl wrote: Finally, it looks like your number one problem is going to be maintainence. Right now you're showing a .x and a F.x release. Not sure if that's tampering or what, but it's obviously not legit. You also have a sudden influx of hosts from Armenia. Again, don't know if they're legit or not, but keeping up with that stuff is going to require eyes-on type manual labor. I hope you've planned for that. Ehm, actually something's really wrong :) Armenia 331 28.24% shouldn't be that much, I think I installed it on 4 servers :) Yup, it was someone else from Armenia that it looks like modified the script and submitted all of their 'virtual hosts' as well as 'the server itself' ... although it does make the stats look good, please keep it to one entry per server (or even one per VPS, since that will then have a distinct IP) but not per virtual host :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Nikolas Britton wrote: How about some uptime stats as well? No. We agreed we would not track people. Again, if we add uptime states, it would be a *seperate* opt-in option ... the only quasi-not-opt-in (you still have to tell it to run the script) is the uname information ... even the pciconf information is purely opt-in ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Nikolas Britton wrote: Also why not track the ones with no driver attached... you should still be able to tell what the device is. I was looking at it from a 'what drivers / hardware is in use' not 'what hardware is available' ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 9/08/2006 9:16 AM, Marc G. Fournier wrote: Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... -Antony ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Antony Mawer wrote: On 9/08/2006 9:16 AM, Marc G. Fournier wrote: Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... Could create problems long term .. one thing I will be using the IPs to do is: SELECT ip, count(1) FROM systems GROUP BY ip ORDER BY count DESC; to look for any 'abnormalities' like todays with Armenia ... hashing it would make stuff like that fairly difficult ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Matthew Seaman wrote: Anyhow, how about the following little enhancement. This lists the CPUs on the system pretending they are CPU0, CPU1, ... devices. The URI escape stuff should be automatically decoded by PHP without any extra coding required. Perfect, added to script, as well as your clean ups ... thanks ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 8/8/06, Marc G. Fournier [EMAIL PROTECTED] wrote: On Tue, 8 Aug 2006, Matthew Seaman wrote: Anyhow, how about the following little enhancement. This lists the CPUs on the system pretending they are CPU0, CPU1, ... devices. The URI escape stuff should be automatically decoded by PHP without any extra coding required. Perfect, added to script, as well as your clean ups ... thanks ... What about PC-BSD? AFAIK they all have the same hostname. Some company could have 1000+ PC-BSD desktop systems hiding behind NAT. I just sent in one for you to look at... Here's it's uname -a: PCBSD# uname -a FreeBSD PCBSD.localhost 6.1-RELEASE-p2 FreeBSD 6.1-RELEASE-p2 #0: Fri Jun 16 09:21:34 PDT 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/PCBSDv1.11 i386 -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
--On August 9, 2006 9:32:18 AM +1000 Antony Mawer [EMAIL PROTECTED] wrote: On 9/08/2006 9:16 AM, Marc G. Fournier wrote: Can you tell me exactly what you do with those two pieces of data? Is there any way that information would be accessible from the internet? Absolutely nothing else we do with it ... it just gives us a unique key to work with ... in fact, assuming each of your servers use a different IP, there is no reason you couldn't do the uname trick above to hide the hostname ... Unless someone breaks into the server, or database, somehow, the data isn't accessible ... What if we improved upon this - if instead of storing the hostname and IP address, we stored a one-way hash of this information? OpenSSH in recent versions takes the same approach with its authorized_keys files... I like that idea. I'm ready to submit my workstation, but I'm still a bit hesitant about some servers I adminA one way hash would alleviate my concerns. Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/
Re: BSDstats Project v2.0 ...
On Tue, 8 Aug 2006, Nikolas Britton wrote: On 8/8/06, Marc G. Fournier [EMAIL PROTECTED] wrote: On Tue, 8 Aug 2006, Matthew Seaman wrote: Anyhow, how about the following little enhancement. This lists the CPUs on the system pretending they are CPU0, CPU1, ... devices. The URI escape stuff should be automatically decoded by PHP without any extra coding required. Perfect, added to script, as well as your clean ups ... thanks ... What about PC-BSD? AFAIK they all have the same hostname. Some company could have 1000+ PC-BSD desktop systems hiding behind NAT. I just sent in one for you to look at... Here's it's uname -a: PCBSD# uname -a FreeBSD PCBSD.localhost 6.1-RELEASE-p2 FreeBSD 6.1-RELEASE-p2 #0: Fri Jun 16 09:21:34 PDT 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/PCBSDv1.11 i386 Unfortunately, if they are *all* the same hostname, and behind NAT, they will just be seen as *one* host ... what is PCBSD? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On 9/08/2006 1:49 PM, Marc G. Fournier wrote: On Tue, 8 Aug 2006, Nikolas Britton wrote: PCBSD# uname -a FreeBSD PCBSD.localhost 6.1-RELEASE-p2 FreeBSD 6.1-RELEASE-p2 #0: Fri Jun 16 09:21:34 PDT 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/PCBSDv1.11 i386 Unfortunately, if they are *all* the same hostname, and behind NAT, they will just be seen as *one* host ... what is PCBSD? It's a user-friendly version of FreeBSD designed to provide a workstation environment for the more novice-style users.. see here for details: http://www.pcbsd.org/?p=learnhome It's not a different BSD OS per se (as opposed to Free/Dragonfly/Net/OpenBSD) but obviously the pre-defined hostname is a problem for determining uniqueness... I wonder if this is something better addressed by the PC-BSD developers as part of their setup process? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Wed, 9 Aug 2006, Antony Mawer wrote: On 9/08/2006 1:49 PM, Marc G. Fournier wrote: On Tue, 8 Aug 2006, Nikolas Britton wrote: PCBSD# uname -a FreeBSD PCBSD.localhost 6.1-RELEASE-p2 FreeBSD 6.1-RELEASE-p2 #0: Fri Jun 16 09:21:34 PDT 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/PCBSDv1.11 i386 Unfortunately, if they are *all* the same hostname, and behind NAT, they will just be seen as *one* host ... what is PCBSD? It's a user-friendly version of FreeBSD designed to provide a workstation environment for the more novice-style users.. see here for details: http://www.pcbsd.org/?p=learnhome It's not a different BSD OS per se (as opposed to Free/Dragonfly/Net/OpenBSD) but obviously the pre-defined hostname is a problem for determining uniqueness... I wonder if this is something better addressed by the PC-BSD developers as part of their setup process? In this case, I think it would have to be ... as much as I hate to refer to Windoze, but even they at least do something semi random when you setup the machine for a hostname, so that not everyone is 'the same' ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
While I'm definately interested in the statistical reporting, I do have one suggestion: Add a random sleep time to the update. Otherwise, the reporting server is gonna get HAMMERED once a month, assuming this project gains any kind of momentum. A random sleep timer at the beginning of the script over (for example) a 3 hour window could make the load bearable for the checkin server.Just something to consider. I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Hello Marc, Monday, August 7, 2006, 5:42:27 AM, you wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Maybe it would be better if you strip the ending number from the driver, so the page will list total number of driver usage not fxp0, fxp1 totals and so on. Let me know of any problems ... Seems like some people are already trying to be funny, because according to the page there are already people running (~40) FreeBSD 8.0-CURRENT :-) It's sad to see this to be happening. -- Best regards, Danielmailto:[EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, 7 Aug 2006, [EMAIL PROTECTED] wrote: While I'm definately interested in the statistical reporting, I do have one suggestion: Add a random sleep time to the update. Otherwise, the reporting server is gonna get HAMMERED once a month, assuming this project gains any kind of momentum. A random sleep timer at the beginning of the script over (for example) a 3 hour window could make the load bearable for the checkin server.Just something to consider. To be totally honest, 'load n the checkin server' isn't something I'm terribly worried about ... right now, we are at ~200 hosts checking in, from various time zones (see http://bsdstats.hub.org/bsd_statistics.php for countries that have checked in so far) ... so, even at month end, taking into consideration time zones, I don't expect a major hit ... now, if we get to 10's of thousands of hosts, which I'd *love*, then I may worry, but by then, I'll have a Dual-Core Dual CPU server in place that I can move the whole thing over onto ... IMHO, its going to be a slow ramp up over a long period of time, but, as I mentioned earlier ... it has to start somewhere, and we've accomplished that ... now to get everyone actually checking in, now that will be a feat to see :) I've also contacted folks @ NetBSD, OpenBSD and DragonflyBSD about participating ... again, our aim is to show that the *BSD camp is a market to look at, not just a group of hobbiests ... I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... This Phase of the script is optional, and not enabled by default ... I can't think of any reason why you wouldn't want to report it, but just in case someone feels it poses a problem, its an opt-in report ... pkg-message updated to reflect the extra line you need to add to /etc/periodic.conf: monthly_statistics_report_devices=yes I've written it to report driver + chip= information from pciconf -l, since even pciconf -lv doesn't seem to use card= ... the summary report will be extended next to show both vendor and chip statistics ... Let me know of any problems ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, Aug 07, 2006, at 00:42:27 -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... Great project idea, and glad to see it's growing :). Every time I refresh the page to check a host I add, I see several more since the last refresh just a few minutes before. The new percentage display added in the last few minutes is nice too. I haven't followed the whole thread so I'm not sure if this has been suggested yet or not, but my idea would be to make another table of just the major releases. Like an easy view on who's running 6.1-RELEASE regardless of patch numbers, or 5.5-RELEASE, or 4.11-RELEASE, just so people can get a quick idea on who's running which releases without having to look through every one including patch levels (kind of a quick summary). Or maybe even numbers and percentages for 4.x, vs 5.x, vs 6.x, etc, and another table for i386, amd64, sparc64, etc, so companies providing drivers or closed source binaries can quickly see the numbers for those. Just an idea :) -Mark -- Internet Radio: Party107 (Trance/Electronic) - http://www.party107.com Rock 101.9 The Edge (Rock) - http://www.rock1019.net IRC: MIXXnet IRC Network - irc.mixxnet.net (Nick: MIXX941) GnuPG Public Key: http://www.mkproductions.org/mk_pubkey.asc ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Mark Kane wrote: On Mon, Aug 07, 2006, at 00:42:27 -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... Great project idea, and glad to see it's growing :). Every time I refresh the page to check a host I add, I see several more since the last refresh just a few minutes before. The new percentage display added in the last few minutes is nice too. I haven't followed the whole thread so I'm not sure if this has been suggested yet or not, but my idea would be to make another table of just the major releases. Like an easy view on who's running 6.1-RELEASE regardless of patch numbers, or 5.5-RELEASE, or 4.11-RELEASE, just so people can get a quick idea on who's running which releases without having to look through every one including patch levels (kind of a quick summary). Or maybe even numbers and percentages for 4.x, vs 5.x, vs 6.x, etc, and another table for i386, amd64, sparc64, etc, so companies providing drivers or closed source binaries can quickly see the numbers for those. Just an idea :) -Mark Just my .02 worth - that Sparc64 listing Is mine Wheee! -- Best regards, Chris If builders built buildings the way programmers wrote programs, then the first woddpecker that came along would destroy civilization. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
Chris wrote: Mark Kane wrote: On Mon, Aug 07, 2006, at 00:42:27 -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... Great project idea, and glad to see it's growing :). Every time I refresh the page to check a host I add, I see several more since the last refresh just a few minutes before. The new percentage display added in the last few minutes is nice too. I haven't followed the whole thread so I'm not sure if this has been suggested yet or not, but my idea would be to make another table of just the major releases. Like an easy view on who's running 6.1-RELEASE regardless of patch numbers, or 5.5-RELEASE, or 4.11-RELEASE, just so people can get a quick idea on who's running which releases without having to look through every one including patch levels (kind of a quick summary). Or maybe even numbers and percentages for 4.x, vs 5.x, vs 6.x, etc, and another table for i386, amd64, sparc64, etc, so companies providing drivers or closed source binaries can quickly see the numbers for those. Just an idea :) -Mark Just my .02 worth - that Sparc64 listing Is mine Wheee! Let me clearify - 6.1-RELEASE sparc64 1 0.47% That one is mine. -- Best regards, Chris Ignorance should be painful. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, 7 Aug 2006, Mark Kane wrote: On Mon, Aug 07, 2006, at 00:42:27 -0300, Marc G. Fournier wrote: I've now committed v2.0 of the 300.statistics periodic script ... this one adds the device reporting that we'd talked about previously, and the summary reports now reflect the driver(s) in use for those deciding to report ... Great project idea, and glad to see it's growing :). Every time I refresh the page to check a host I add, I see several more since the last refresh just a few minutes before. The new percentage display added in the last few minutes is nice too. I haven't followed the whole thread so I'm not sure if this has been suggested yet or not, but my idea would be to make another table of just the major releases. Like an easy view on who's running 6.1-RELEASE regardless of patch numbers, or 5.5-RELEASE, or 4.11-RELEASE, just so people can get a quick idea on who's running which releases without having to look through every one including patch levels (kind of a quick summary). Or maybe even numbers and percentages for 4.x, vs 5.x, vs 6.x, etc, and another table for i386, amd64, sparc64, etc, so companies providing drivers or closed source binaries can quickly see the numbers for those. Just an idea :) Refresh the page, I've add a new table since, which I believe is what you are asking for :) Also, Antony, in Australia, is going to be doing some work on overall look of the pages, improve the presentation, give is a BSD feel to it ... I'm a grunt, hate making things look pretty :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, 7 Aug 2006, Chris wrote: Just my .02 worth - that Sparc64 listing Is mine Wheee! There are two Sparc64 listings ... both yours? The 8 in Panama are all mine :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, Aug 07, 2006, at 17:18:04 -0300, Marc G. Fournier wrote: Refresh the page, I've add a new table since, which I believe is what you are asking for :) Exactly. Just a couple minutes after I hit send, I saw that there and thought wow, that was fast! Then again I didn't see my post on the list yet so I thought maybe you already had that idea :) -Mark -- Internet Radio: Party107 (Trance/Electronic) - http://www.party107.com Rock 101.9 The Edge (Rock) - http://www.rock1019.net IRC: MIXXnet IRC Network - irc.mixxnet.net (Nick: MIXX941) GnuPG Public Key: http://www.mkproductions.org/mk_pubkey.asc ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: BSDstats Project v2.0 ...
On Mon, 7 Aug 2006, Mark Kane wrote: On Mon, Aug 07, 2006, at 17:18:04 -0300, Marc G. Fournier wrote: Refresh the page, I've add a new table since, which I believe is what you are asking for :) Exactly. Just a couple minutes after I hit send, I saw that there and thought wow, that was fast! Then again I didn't see my post on the list yet so I thought maybe you already had that idea :) I just added one in that is half way between the two ... so it shows major.minor releases, not just major, but stripping off everything after the - I'm going to start playing with the devices report next ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
