Re: Operation timed out with smtp.gmail.com - please help

[Jerry]

On Sat, 30 Mar 2013 10:17:55 -0700 (PDT)
Anton Shterenlikht articulated:

> Jerry, Matthew, thank you
> 
> I think I got it working.
> In addition to your advice, this guide was very helpful:
> 
> http://www.phinesolutions.com/sendmail-gmail-smtp-relay-howto.html
> 
> It seems these two options were required:
> 
> define(`RELAY_MAILER_ARGS', `TCP $h 587')
> define(`ESMTP_MAILER_ARGS', `TCP $h 587')

After reading that "How-to", I am so glad I use Postfix. Anyway, glad
you got it to work. You might find the idiot who wrote that first
"manual" you referenced and tell him/her they are a dumb-ass and post a
corrected manual.

-- 
Jerry ♔

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the Reply-To header.
__

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Operation timed out with smtp.gmail.com - please help

[Anton Shterenlikht]

Date: Sat, 30 Mar 2013 07:49:19 -0400
From: Jerry 
To: FreeBSD 
Subject: Re: Operation timed out with smtp.gmail.com - please help

On Sat, 30 Mar 2013 10:49:45 +
Matthew Seaman articulated:

> Given you're seeing that CONNECTED message there, it certainly does.
> The problem with that openssl command seems to be the 'unable to get
> local issuer certificate' part.  That's possibly openssl being pickier
> about verifying certs than sendmail would be, but that certificate
> verification step is probably where you're coming adrift.  You need to
> have the intermediate certs used by Google in your cacert.pem file, so
> sendmail will trust the smtp.gmail.com cert.  Check the 'confCACERT'
> setting in your sendmail.mc.  I have a block of code like this:
>=20
> define(`CERT_DIR', `MAIL_SETTINGS_DIR`'certs')dnl
> define(`confCACERT_PATH', `CERT_DIR')dnl
> define(`confCACERT', `CERT_DIR/cacert.pem')dnl
> define(`confSERVER_CERT', `CERT_DIR/cert.pem')dnl
> define(`confSERVER_KEY', `CERT_DIR/key.pem')dnl
> define(`confCLIENT_CERT', `CERT_DIR/cert.pem')dnl
> define(`confCLIENT_KEY', `CERT_DIR/key.pem')dnl
>=20
> which allows me to put all the keys and certs in /etc/mail/certs/

If you really need the Gmail certs, you can use this to get them:

openssl s_client -connect smtp.gmail.com:587 -starttls smtp -showcerts

If you feel you really need the "Equifax Secure Certificate Authority"
pem, go here <http://www.geotrust.com/resources/root-certificates/> and
download it.

Again, how to set up Sendmail is a task I leave for the student.

Jerry, Matthew, thank you

I think I got it working.
In addition to your advice, this guide was very helpful:

http://www.phinesolutions.com/sendmail-gmail-smtp-relay-howto.html

It seems these two options were required:

define(`RELAY_MAILER_ARGS', `TCP $h 587')
define(`ESMTP_MAILER_ARGS', `TCP $h 587')

Thanks again

Anton
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Operation timed out with smtp.gmail.com - please help

[Jerry]

On Sat, 30 Mar 2013 10:49:45 +
Matthew Seaman articulated:

> Given you're seeing that CONNECTED message there, it certainly does.
> The problem with that openssl command seems to be the 'unable to get
> local issuer certificate' part.  That's possibly openssl being pickier
> about verifying certs than sendmail would be, but that certificate
> verification step is probably where you're coming adrift.  You need to
> have the intermediate certs used by Google in your cacert.pem file, so
> sendmail will trust the smtp.gmail.com cert.  Check the 'confCACERT'
> setting in your sendmail.mc.  I have a block of code like this:
> 
> define(`CERT_DIR', `MAIL_SETTINGS_DIR`'certs')dnl
> define(`confCACERT_PATH', `CERT_DIR')dnl
> define(`confCACERT', `CERT_DIR/cacert.pem')dnl
> define(`confSERVER_CERT', `CERT_DIR/cert.pem')dnl
> define(`confSERVER_KEY', `CERT_DIR/key.pem')dnl
> define(`confCLIENT_CERT', `CERT_DIR/cert.pem')dnl
> define(`confCLIENT_KEY', `CERT_DIR/key.pem')dnl
> 
> which allows me to put all the keys and certs in /etc/mail/certs/

If you really need the Gmail certs, you can use this to get them:

openssl s_client -connect smtp.gmail.com:587 -starttls smtp -showcerts

If you feel you really need the "Equifax Secure Certificate Authority"
pem, go here  and
download it.

Again, how to set up Sendmail is a task I leave for the student.

-- 
Jerry ♔

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the Reply-To header.
__



signature.asc
Description: PGP signature

Re: Operation timed out with smtp.gmail.com - please help

[Jerry]

On Sat, 30 Mar 2013 10:14:44 GMT
Anton Shterenlikht articulated:

>   Date: Fri, 29 Mar 2013 15:36:19 -0400
>   From: Jerry 
>   To: FreeBSD 
>   Subject: Re: Operation timed out with smtp.gmail.com - please
> help
> 
>   On Fri, 29 Mar 2013 18:32:34 GMT
>   Anton Shterenlikht articulated:
> 
>   > Please help debug sendmail / smtp.gmail config.
>   > 
>   > My University just switched to gmail (dickheads)
>   > and I'm trying to figure out how to set it up.
>   > 
>   > It used to work ok with the University smtp auth
>   > server. Now I get in /var/log/maillog:
>   > 
>   >  sm-mta[72300]: r2TI0vQc072134: to=,
>   >  ctladdr= (1001/1001),
>   >  delay=00:20:01, xdelay=00:00:00, mailer=relay, pri=210424,
>   >  relay=smtp.gmail.com, dsn=4.0.0,
>   >  stat=Deferred: Operation timed out with smtp.gmail.com
>   > 
>   > I switched the firewall off completely.
>   > 
>   > I have:
>   > 
>   > # cat /etc/mail/auth/client-info
>   > AuthInfo:smtp.gmail.com "U:root" "I:me...@bristol.ac.uk"
>   > "P:x" # 
>   > 
>   > and this in /etc/mail/freebsd.mc:
>   > 
>   > define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5
>   > LOGIN')dnl define(`SMART_HOST', `smtp.gmail.com')dnl
>   > 
>   > I rebuilt (run make under /etc/mail. This just
>   > renames freebsd.mc to .mc, and freebsd.submit.mc
>   > to .submit.mc) and restarted sendmail.
>   > 
>   > I also use:
>   > 
>   > MASQUERADE_AS(`bristol.ac.uk')
>   > MASQUERADE_DOMAIN(`bristol.ac.uk')
>   > 
>   > to use the university domain instead of
>   > may .men.bris.ac.uk, which is not
>   > acceptable.
> 
>   Try this at the command line:
> 
>   openssl s_client -connect smtp.gmail.com:25 -starttls smtp
> 
>   If it times out, change the port number to 587 and try it
> again. If you cannot make a connect using either port number then you
> have a firewall problem.
> 
> Thank you, I get:
> 
> $ openssl s_client -connect smtp.gmail.com:25 -starttls smtp
> connect: Operation timed out
> connect:errno=60
> $ 
> 
> $ openssl s_client -connect smtp.gmail.com:587 -starttls smtp
> CONNECTED(0003)
> depth=1 C = US, O = Google Inc, CN = Google Internet Authority
> verify error:num=20:unable to get local issuer certificate
> verify return:0
> ---
> Certificate chain
>  0 s:/C=US/ST=California/L=Mountain View/O=Google
> Inc/CN=smtp.gmail.com i:/C=US/O=Google Inc/CN=Google Internet
> Authority 1 s:/C=US/O=Google Inc/CN=Google Internet Authority
>i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
> ---
> Server certificate
> -BEGIN CERTIFICATE-
> MIIDgDCCAumgAwIBAgIKO3T/ewBoqDANBgkqhkiG9w0BAQUFADBGMQswCQYD
> VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu
> dGVybmV0IEF1dGhvcml0eTAeFw0xMjA5MTIxMTU3NTBaFw0xMzA2MDcxOTQzMjda
> MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N
> b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw
> LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAv0UvQmjW1y96
> cOK6AdQVEYPRd3ZQ9UhxkKfuVaYS9riOESFkWxkz+b3Ts/EOA5SY8axkaJS7Qa/v
> N7laztYY8tTkx9Ml+eCY4xh0fFq9z4/WWADGqTY5I0wvqjZr+jBuYGulK1fU4ZUS
> QpuZMMO9x7Bmr5LVP9C5r2qnoqtMtJUCAwEAAaOCAVEwggFNMB0GA1UdJQQWMBQG
> CCsGAQUFBwMBBggrBgEFBQcDAjAdBgNVHQ4EFgQUaCtARMZ9urIDfdpR6v1AkQsr
> 44owHwYDVR0jBBgwFoAUv8Aw6/VDET5nup6R+/xq2uNrEiQwWwYDVR0fBFQwUjBQ
> oE6gTIZKaHR0cDovL3d3dy5nc3RhdGljLmNvbS9Hb29nbGVJbnRlcm5ldEF1dGhv
> cml0eS9Hb29nbGVJbnRlcm5ldEF1dGhvcml0eS5jcmwwZgYIKwYBBQUHAQEEWjBY
> MFYGCCsGAQUFBzAChkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dvb2dsZUludGVy
> bmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNydDAMBgNVHRMB
> Af8EAjAAMBkGA1UdEQQSMBCCDnNtdHAuZ21haWwuY29tMA0GCSqGSIb3DQEBBQUA
> A4GBADSkwmtEUhy/AhX2sIULT0Q5S9OlfKxbyE8hEc8nxls3jbk5yKZYd35Bzyy8
> raoUPFuD3IH+zP/FGj5LPQirjnJLUvuFDsiM4eowPUthQad9SGWWdz6hCx8HpEUZ
> 1ssGnwb3HX34e9RH57v9LdtVUPdFYQsBJ36miGPylWk6r0xx
> -END CERTIFICATE-
> subject=/C=US/ST=California/L=Mountain View/O=Google
> Inc/CN=smtp.gmail.com issuer=/C=US/O=Google Inc/CN=Google Internet
> Authority ---
> No client certificate CA names sent
> ---
> SSL handshake has read 2317 bytes and written 476 bytes
> ---
> New, TLSv1/SSLv3, Cipher is ECDHE-RSA-RC4-SHA
> Server public key is 1024 bit
> Secure Renegotiation IS supported
> Compression: NONE
> Expansion: NONE
> SSL-Session:
> Protocol  : TLSv1.2
>

Re: Operation timed out with smtp.gmail.com - please help

[Matthew Seaman]

On 30/03/2013 10:14, Anton Shterenlikht wrote:
> The university IT support page:
> http://www.bristol.ac.uk/it-services/applications/email/gmail/manual-config-gmail.html
> 
> actually says that port 465 SSL should be used,
> so I also tried:
> 
> $ openssl s_client -connect smtp.gmail.com:465 -starttls smtp
> CONNECTED(0003)
> ^C
> $ 
> 
> Not sure what to make of this.
> 
> Is the port set by sendmail config files?
> 
> Many thanks for your help
> 

Port 465 wouldn't use STARTTLS -- it requires SSL straight away.  Try:

% openssl s_client -connect  smtp.gmail.com:465

If it works you should see output to do with setting up session keys etc.

However, SMTP on port 465 seems to be mostly a windows thing, and
generally discouraged -- use of STARTTLS or equivalent to allow both SSL
and plaintext without having to allocate a separate port for SSL is
preferred.   I'm pretty sure that gmail does support STARTTLS...

> $ openssl s_client -connect smtp.gmail.com:587 -starttls smtp
> CONNECTED(0003)
> depth=1 C = US, O = Google Inc, CN = Google Internet Authority
> verify error:num=20:unable to get local issuer certificate
> verify return:0
> ---
> Certificate chain
>  0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com
>i:/C=US/O=Google Inc/CN=Google Internet Authority
>  1 s:/C=US/O=Google Inc/CN=Google Internet Authority
>i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
> ---

Given you're seeing that CONNECTED message there, it certainly does.
The problem with that openssl command seems to be the 'unable to get
local issuer certificate' part.  That's possibly openssl being pickier
about verifying certs than sendmail would be, but that certificate
verification step is probably where you're coming adrift.  You need to
have the intermediate certs used by Google in your cacert.pem file, so
sendmail will trust the smtp.gmail.com cert.  Check the 'confCACERT'
setting in your sendmail.mc.  I have a block of code like this:

define(`CERT_DIR', `MAIL_SETTINGS_DIR`'certs')dnl
define(`confCACERT_PATH', `CERT_DIR')dnl
define(`confCACERT', `CERT_DIR/cacert.pem')dnl
define(`confSERVER_CERT', `CERT_DIR/cert.pem')dnl
define(`confSERVER_KEY', `CERT_DIR/key.pem')dnl
define(`confCLIENT_CERT', `CERT_DIR/cert.pem')dnl
define(`confCLIENT_KEY', `CERT_DIR/key.pem')dnl

which allows me to put all the keys and certs in /etc/mail/certs/

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Operation timed out with smtp.gmail.com - please help

[Anton Shterenlikht]

Date: Fri, 29 Mar 2013 15:36:19 -0400
From: Jerry 
To: FreeBSD 
Subject: Re: Operation timed out with smtp.gmail.com - please help

On Fri, 29 Mar 2013 18:32:34 GMT
Anton Shterenlikht articulated:

> Please help debug sendmail / smtp.gmail config.
> 
> My University just switched to gmail (dickheads)
> and I'm trying to figure out how to set it up.
> 
> It used to work ok with the University smtp auth
> server. Now I get in /var/log/maillog:
> 
>  sm-mta[72300]: r2TI0vQc072134: to=,
>  ctladdr= (1001/1001),
>  delay=00:20:01, xdelay=00:00:00, mailer=relay, pri=210424,
>  relay=smtp.gmail.com, dsn=4.0.0,
>  stat=Deferred: Operation timed out with smtp.gmail.com
> 
> I switched the firewall off completely.
> 
> I have:
> 
> # cat /etc/mail/auth/client-info
> AuthInfo:smtp.gmail.com "U:root" "I:me...@bristol.ac.uk" "P:x"
> # 
> 
> and this in /etc/mail/freebsd.mc:
> 
> define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
> define(`SMART_HOST', `smtp.gmail.com')dnl
> 
> I rebuilt (run make under /etc/mail. This just
> renames freebsd.mc to .mc, and freebsd.submit.mc
> to .submit.mc) and restarted sendmail.
> 
> I also use:
> 
> MASQUERADE_AS(`bristol.ac.uk')
> MASQUERADE_DOMAIN(`bristol.ac.uk')
> 
> to use the university domain instead of
> may .men.bris.ac.uk, which is not
> acceptable.

Try this at the command line:

openssl s_client -connect smtp.gmail.com:25 -starttls smtp

If it times out, change the port number to 587 and try it again. If you
cannot make a connect using either port number then you have a firewall
problem.

Thank you, I get:

$ openssl s_client -connect smtp.gmail.com:25 -starttls smtp
connect: Operation timed out
connect:errno=60
$ 

$ openssl s_client -connect smtp.gmail.com:587 -starttls smtp
CONNECTED(0003)
depth=1 C = US, O = Google Inc, CN = Google Internet Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-BEGIN CERTIFICATE-
MIIDgDCCAumgAwIBAgIKO3T/ewBoqDANBgkqhkiG9w0BAQUFADBGMQswCQYD
VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu
dGVybmV0IEF1dGhvcml0eTAeFw0xMjA5MTIxMTU3NTBaFw0xMzA2MDcxOTQzMjda
MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N
b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw
LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAv0UvQmjW1y96
cOK6AdQVEYPRd3ZQ9UhxkKfuVaYS9riOESFkWxkz+b3Ts/EOA5SY8axkaJS7Qa/v
N7laztYY8tTkx9Ml+eCY4xh0fFq9z4/WWADGqTY5I0wvqjZr+jBuYGulK1fU4ZUS
QpuZMMO9x7Bmr5LVP9C5r2qnoqtMtJUCAwEAAaOCAVEwggFNMB0GA1UdJQQWMBQG
CCsGAQUFBwMBBggrBgEFBQcDAjAdBgNVHQ4EFgQUaCtARMZ9urIDfdpR6v1AkQsr
44owHwYDVR0jBBgwFoAUv8Aw6/VDET5nup6R+/xq2uNrEiQwWwYDVR0fBFQwUjBQ
oE6gTIZKaHR0cDovL3d3dy5nc3RhdGljLmNvbS9Hb29nbGVJbnRlcm5ldEF1dGhv
cml0eS9Hb29nbGVJbnRlcm5ldEF1dGhvcml0eS5jcmwwZgYIKwYBBQUHAQEEWjBY
MFYGCCsGAQUFBzAChkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dvb2dsZUludGVy
bmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNydDAMBgNVHRMB
Af8EAjAAMBkGA1UdEQQSMBCCDnNtdHAuZ21haWwuY29tMA0GCSqGSIb3DQEBBQUA
A4GBADSkwmtEUhy/AhX2sIULT0Q5S9OlfKxbyE8hEc8nxls3jbk5yKZYd35Bzyy8
raoUPFuD3IH+zP/FGj5LPQirjnJLUvuFDsiM4eowPUthQad9SGWWdz6hCx8HpEUZ
1ssGnwb3HX34e9RH57v9LdtVUPdFYQsBJ36miGPylWk6r0xx
-END CERTIFICATE-
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com
issuer=/C=US/O=Google Inc/CN=Google Internet Authority
---
No client certificate CA names sent
---
SSL handshake has read 2317 bytes and written 476 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-RC4-SHA
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol  : TLSv1.2
Cipher: ECDHE-RSA-RC4-SHA
Session-ID: 8CAF4204FADB72F58FA6334A62F65B7182EF06F3C9AD8042FD44B9F726E8C9D5
Session-ID-ctx: 
Master-Key: 
45312AE23341AAFA1414BDDD30740E4FB40655986FD410A606CD351206BBAC5E5496F77DDF4DBE32B0E9B7E7FFA1057
Key-Arg   : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 100800 (seconds)
TLS session ticket:
 - 63 53 11 b3 92 0d 59 63-15 90 58 10 84 f2 f7 6a   cSYc..Xj
0010 - 7c 7c 

Re: Operation timed out with smtp.gmail.com - please help

[Jerry]

On Fri, 29 Mar 2013 18:32:34 GMT
Anton Shterenlikht articulated:

> Please help debug sendmail / smtp.gmail config.
> 
> My University just switched to gmail (dickheads)
> and I'm trying to figure out how to set it up.
> 
> It used to work ok with the University smtp auth
> server. Now I get in /var/log/maillog:
> 
>  sm-mta[72300]: r2TI0vQc072134: to=,
>  ctladdr= (1001/1001),
>  delay=00:20:01, xdelay=00:00:00, mailer=relay, pri=210424,
>  relay=smtp.gmail.com, dsn=4.0.0,
>  stat=Deferred: Operation timed out with smtp.gmail.com
> 
> I switched the firewall off completely.
> 
> I have:
> 
> # cat /etc/mail/auth/client-info
> AuthInfo:smtp.gmail.com "U:root" "I:me...@bristol.ac.uk" "P:x"
> # 
> 
> and this in /etc/mail/freebsd.mc:
> 
> define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
> define(`SMART_HOST', `smtp.gmail.com')dnl
> 
> I rebuilt (run make under /etc/mail. This just
> renames freebsd.mc to .mc, and freebsd.submit.mc
> to .submit.mc) and restarted sendmail.
> 
> I also use:
> 
> MASQUERADE_AS(`bristol.ac.uk')
> MASQUERADE_DOMAIN(`bristol.ac.uk')
> 
> to use the university domain instead of
> may .men.bris.ac.uk, which is not
> acceptable.

Try this at the command line:

openssl s_client -connect smtp.gmail.com:25 -starttls smtp

If it times out, change the port number to 587 and try it again. If you
cannot make a connect using either port number then you have a firewall
problem.

-- 
Jerry ♔

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the Reply-To header.
__

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Operation timed out with smtp.gmail.com - please help

[CeDeROM]

gmail has blocking mechanism when you use it from different devices,
try this maybe it will help:

https://accounts.google.com/DisplayUnlockCaptcha

--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Operation timed out with smtp.gmail.com - please help

[Anton Shterenlikht]

Please help debug sendmail / smtp.gmail config.

My University just switched to gmail (dickheads)
and I'm trying to figure out how to set it up.

It used to work ok with the University smtp auth
server. Now I get in /var/log/maillog:

 sm-mta[72300]: r2TI0vQc072134: to=,
 ctladdr= (1001/1001),
 delay=00:20:01, xdelay=00:00:00, mailer=relay, pri=210424,
 relay=smtp.gmail.com, dsn=4.0.0,
 stat=Deferred: Operation timed out with smtp.gmail.com

I switched the firewall off completely.

I have:

# cat /etc/mail/auth/client-info
AuthInfo:smtp.gmail.com "U:root" "I:me...@bristol.ac.uk" "P:x"
# 

and this in /etc/mail/freebsd.mc:

define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
define(`SMART_HOST', `smtp.gmail.com')dnl

I rebuilt (run make under /etc/mail. This just
renames freebsd.mc to .mc, and freebsd.submit.mc
to .submit.mc) and restarted sendmail.

I also use:

MASQUERADE_AS(`bristol.ac.uk')
MASQUERADE_DOMAIN(`bristol.ac.uk')

to use the university domain instead of
may .men.bris.ac.uk, which is not
acceptable.

What else am I missing?

Thanks

Anton
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

FreeBSD server for sons tv shows. please help

[Brent Clark]

Good day

Im not it its because of a power failure or what, but for some reason my 
'download server', has lost its raid (0/ stripe).


Im trying to fix this, for the raid contains quite a few shows for my son.

If I go

[root@torry /usr/home/bclark]# gstripe list
Geom name: st0
State: UP
Status: Total=3, Online=3
Type: AUTOMATIC
Stripesize: 65536
ID: 1006591079
Providers:
1. Name: stripe/st0
   Mediasize: 360102297600 (335G)
   Sectorsize: 512
   Stripesize: 65536
   Stripeoffset: 0
   Mode: r0w0e0
Consumers:
1. Name: ada0
   Mediasize: 120034123776 (111G)
   Sectorsize: 512
   Mode: r0w0e0
   Number: 0
2. Name: ada1
   Mediasize: 120034123776 (111G)
   Sectorsize: 512
   Mode: r0w0e0
   Number: 2
3. Name: ada4
   Mediasize: 120034123776 (111G)
   Sectorsize: 512
   Mode: r0w0e0
   Number: 1

I see 'State: UP'

if i:
[root@torry /usr/home/bclark]# mount -t ufs /dev/stripe/st0a /mnt/
mount: /dev/stripe/st0a: Invalid argument

[root@torry /usr/home/bclark]# fsck /dev/stripe/st0a
fsck: Could not determine filesystem type

[root@torry /usr/home/bclark]# fsck_ufs /dev/stripe/st0a
** /dev/stripe/st0a
Cannot find file system superblock
ioctl (GCINFO): Inappropriate ioctl for device
fsck_ufs: /dev/stripe/st0a: can't read disk label

If someone could help, it would be appreciated, of what the next step 
is, it would be appreciated.





___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Mon, 01 Oct 2012 15:00:40 -0500,  wrote:



Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 5  
ee 60 16 0 1 0 0
Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI  
Status Error

Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3  
ef 42 51 0 1 0 0
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI  
Status Error

Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3  
ef 64 51 0 1 0 0
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI  
Status Error

Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3  
ef 66 51 0 1 0 0
Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI  
Status Error

Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
...
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0  
41 f3 94 99 0 1 0 0
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI  
Status Error

Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): Retrying command



Sometimes you'll see this before a crash, but not every time.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[guy . helmer]

On Wednesday, June 6, 2012 8:36:04 PM UTC-5, Mark Felder wrote:
> Hi guys I'm excitedly posting this from my phone. Good news for you guys, bad 
> news for us -- we were building HA storage on vmware for a client and can now 
> replicate the crash on demand. I'll be posting details when I get home to my 
> PC tonight, but this hopefully is enough to replicate the crash for any 
> curious followers:
> 
> 
> 
> ESXi 5
> 
> 9 or 9-STABLE
> 
> HAST 
> 
> 1 cpu is fine
> 
> 1GB of ram
> 
> UFS SUJ on HAST device
> 
> No special loader.conf, sysctl, etc
> 
> No need for VMWare tools
> 
> Run Bonnie++ on the HAST device
> 
> 
> 
> We can get the crash to happen on the first run of bonnie++ right now. I'll 
> post the exact specs and precise command run in the PR. We found an old post 
> from 2004 when we looked up the process state obtained from CTRL+T -- flswai 
> -- which describes the symptoms nearly perfectly.
> 
> 
> 
>  http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2004-02/0250.html 
> 
> 
> 
> Hopefully this gets us closer to a fix...

Is this a crash or a hang? Over the past couple of weeks, I've been working 
with a FreeBSD 9.1RC1 system under VMware ESXi 5.0 with a 64GB UFS root FS and 
2TB ZFS filesystem mounted via a virtual LSI SAS interface. Sometimes during 
heavy I/O load (rsync from other servers) on the ZFS FS, this shows up in 
/var/log/messages:

Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 5 ee 60 
16 0 1 0 0 
Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status Error
Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef 42 
51 0 1 0 0 
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status Error
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef 64 
51 0 1 0 0 
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status Error
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): Retrying command
Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef 66 
51 0 1 0 0 
Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status Error
Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
...
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 41 f3 94 
99 0 1 0 0 
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status Error
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy
Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): Retrying command

These have been happening roughly every other day.

mpt0 and em0 were sharing int 18, so today I put 
hint.mpt.0.msi_enable="1"
into /boot/devices.hints and rebooted; now mpt0 is using int 256. I'll see if 
it helps.

Guy
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Hi guys I'm excitedly posting this from my phone. Good news for you guys, bad 
news for us -- we were building HA storage on vmware for a client and can now 
replicate the crash on demand. I'll be posting details when I get home to my PC 
tonight, but this hopefully is enough to replicate the crash for any curious 
followers:

ESXi 5
9 or 9-STABLE
HAST 
1 cpu is fine
1GB of ram
UFS SUJ on HAST device
No special loader.conf, sysctl, etc
No need for VMWare tools
Run Bonnie++ on the HAST device

We can get the crash to happen on the first run of bonnie++ right now. I'll 
post the exact specs and precise command run in the PR. We found an old post 
from 2004 when we looked up the process state obtained from CTRL+T -- flswai -- 
which describes the symptoms nearly perfectly.

 http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2004-02/0250.html 

Hopefully this gets us closer to a fix...


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[John Baldwin]

On Thursday, May 31, 2012 11:11:11 am Mark Felder wrote:
> So when this hang happens, there never is a real panic. It just sits in a  
> state which I describe as like being in a deadlock. How would I go about  
> getting a crashdump if it never panics? Is it possible to do the dump over  
> a network or something because I don't believe it can write through the  
> controller at all.

You can break into ddb and run 'call doadump'.  It should use polled IO, so 
there is a slight chance of it working.

> Also, thank you for the KTR_SCHED tip. This is the type of info I was  
> looking for. Unfortunately I've only ever seen this crash once on a kernel  
> with debugging enabled. The machine which is currently prepared to do this  
> work used to crash a few times a week and now it has 70 days uptime...  
> however, it is an example of a machine with mpt0 and em0 sharing an IRQ so  
> I might be able to trigger it using Dane's method.
> 
> $ vmstat -i
> interrupt  total   rate
> irq1: atkbd0 392  0
> irq6: fdc0 9  0
> irq14: ata0   34  0
> irq18: em0 mpt0   1189748491218
> cpu0: timer   2174263198400
> Total 3364012124619
> 
> 
> I'm doing my best to get you guys the info you need, but this is one heck  
> of a Heisenbug...

Thanks.

-- 
John Baldwin
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

So when this hang happens, there never is a real panic. It just sits in a  
state which I describe as like being in a deadlock. How would I go about  
getting a crashdump if it never panics? Is it possible to do the dump over  
a network or something because I don't believe it can write through the  
controller at all.


Also, thank you for the KTR_SCHED tip. This is the type of info I was  
looking for. Unfortunately I've only ever seen this crash once on a kernel  
with debugging enabled. The machine which is currently prepared to do this  
work used to crash a few times a week and now it has 70 days uptime...  
however, it is an example of a machine with mpt0 and em0 sharing an IRQ so  
I might be able to trigger it using Dane's method.


$ vmstat -i
interrupt  total   rate
irq1: atkbd0 392  0
irq6: fdc0 9  0
irq14: ata0   34  0
irq18: em0 mpt0   1189748491218
cpu0: timer   2174263198400
Total 3364012124619


I'm doing my best to get you guys the info you need, but this is one heck  
of a Heisenbug...

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[John Baldwin]

On Wednesday, May 30, 2012 3:56:02 pm Mark Felder wrote:
> On Wed, 30 May 2012 12:17:07 -0500, John Baldwin  wrote:
> 
> >
> > Humm, can you test it with 2 CPUs?
> >
> 
> We primarily only run with 1 CPU. We have seen it crash on multiple CPU  
> VMs. Also, Dane Foster appeared to have been using multiple CPUs in his  
> video transcoding VMs.
> 
> Unfortunately I can't give you more information at the moment. I'm working  
> with Dane to compile easy to follow steps that recreate this failure. I  
> have not been successful in getting this to crash on demand in my  
> environment, but Dane has so we're trying to recreate his.

Ok.  It would be really helpful if we could get a crashdump, though I realize 
that may not be doable.  Otherwise, full DDB ps output from a hang would be a 
good start.  Primarily I would want to see what the system is doing and why it 
isn't running the threads on the run queue.  It might also be useful to add 
KTR_SCHED tracing so we can get the output of that via 'show ktr' from DDB 
when it hangs.

-- 
John Baldwin
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Wed, 30 May 2012 12:17:07 -0500, John Baldwin  wrote:



Humm, can you test it with 2 CPUs?



We primarily only run with 1 CPU. We have seen it crash on multiple CPU  
VMs. Also, Dane Foster appeared to have been using multiple CPUs in his  
video transcoding VMs.


Unfortunately I can't give you more information at the moment. I'm working  
with Dane to compile easy to follow steps that recreate this failure. I  
have not been successful in getting this to crash on demand in my  
environment, but Dane has so we're trying to recreate his.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Wed, 30 May 2012 10:06:13 -0500, John Baldwin  wrote:



Do you only have one CPU in this VM?  If not, do you know which threads
the other CPUs were running (e.g. do you have ps7.png, etc.)?


correct, only one CPU in the VM
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[John Baldwin]

On Thursday, May 24, 2012 9:47:46 am Mark Felder wrote:
> On Wed, 23 May 2012 17:30:40 -0500, Adrian Chadd   
> wrote:
> 
> > Hi,
> >
> > can you please, -please- file a PR? And place all of the above
> > information in it so we don't lose it?
> >
> 
> I'd be glad to post a PR and assist in helping to get it permanently  
> fixed. I certainly don't want this data to get lost and honestly our  
> business uses FreeBSD on VMWare so much that we really need a permanent  
> fix as much as anyone else :-)
> 
> The reason I've hesitated to post a PR so far is that I didn't have any  
> truly useful or concrete evidence of where the problem lies. After Dane  
> Foster contacted me and told me he could recreate the crash on demand with  
> his workload it was easier to narrow things down. The suggestion that it  
> was an interrupts issue (by possibly Bjoern Zeeb?) and Dane's discovery  
> that his crashes ceased when em0 and mpt0 share an IRQ, but em0 is  
> completely unused was starting to prove there is some strong evidence here  
> in favor of the interrupts issue.
> 
> Dane, what's the status on your end? Has your fix still been successful?  
> Is it also stable if you simply set hint.mpt.0.msi_enable="1" ?

Hmm, so the set of ps output you have from DDB shows a lot of runnable 
processes and swi6 (Giant taskq) as the only running thread (all consistent
with your hang).  (And that is from your Ctrl-Alt-Esc)

Do you only have one CPU in this VM?  If not, do you know which threads
the other CPUs were running (e.g. do you have ps7.png, etc.)?

-- 
John Baldwin
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Adrian Chadd]

Hi,

You guys now absolutely, positively have enough information for a PR.

It's still not clear whether it's a device/interrupt layer issue in
FreeBSD, or whether vmware is doing something wrong with how it
implements shared interrupts, or a bit of both..

Adrian

On 24 May 2012 13:54, dane foster  wrote:
> Hey all,
>
> On 25/05/2012, at 1:47 AM, Mark Felder wrote:
>
>> On Wed, 23 May 2012 17:30:40 -0500, Adrian Chadd  wrote:
>>
>>> Hi,
>>>
>>> can you please, -please- file a PR? And place all of the above
>>> information in it so we don't lose it?
>>>
>>
>> I'd be glad to post a PR and assist in helping to get it permanently fixed. 
>> I certainly don't want this data to get lost and honestly our business uses 
>> FreeBSD on VMWare so much that we really need a permanent fix as much as 
>> anyone else :-)
>>
>> The reason I've hesitated to post a PR so far is that I didn't have any 
>> truly useful or concrete evidence of where the problem lies. After Dane 
>> Foster contacted me and told me he could recreate the crash on demand with 
>> his workload it was easier to narrow things down. The suggestion that it was 
>> an interrupts issue (by possibly Bjoern Zeeb?) and Dane's discovery that his 
>> crashes ceased when em0 and mpt0 share an IRQ, but em0 is completely unused 
>> was starting to prove there is some strong evidence here in favor of the 
>> interrupts issue.
>>
>> Dane, what's the status on your end? Has your fix still been successful? Is 
>> it also stable if you simply set hint.mpt.0.msi_enable="1" ?
>>
>
> The situation I've got that's stable now is:
>
> hw.pci.enable_msi="0"
> hw.pci.enable_msix="0"
>
> in /boot/loader.conf
>
> and:
>
> samael:~:% vmstat -i                                                  [ 
> 6:31PM]
> interrupt                          total       rate
> irq1: atkbd0                           6          0
> irq18: em0 mpt0                  3061100         15
> irq19: em1                       6891706         35
> cpu0: timer                    166383735        868
> cpu1: timer                    166382123        868
> cpu3: timer                    166382123        868
> cpu2: timer                    166382121        868
> Total                          675482914       3525
>
> Not using em0. This works for 8 (FreeBSD samael.slush.ca 8.3-STABLE FreeBSD 
> 8.3-STABLE #1: Mon May  7 11:51:03 NZST 2012     
> r...@samael.slush.ca:/usr/obj/usr/src/sys/DENE  amd64).
>
> Neither of those settings on their own seem to stop it from happening.
>
> The 9 box I've tried this on still hangs almost every time i run handbrake, 
> no matter whether MSI/MSIX is enabled, or I have separate IRQs for mpt0 and 
> em0/1
>
> I can cause the hang mostly on demand, but not quite sure what information to 
> provide from the hung system. If somebody can let me know what they need, 
> including root access, I can make that happen.
>
> Cheers,
>
> Dane
>
>
>
>>
>> Thanks!
>
>
>
>
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Bjoern A. Zeeb]

On 24. May 2012, at 13:47 , Mark Felder wrote:

> On Wed, 23 May 2012 17:30:40 -0500, Adrian Chadd  wrote:
> 
>> Hi,
>> 
>> can you please, -please- file a PR? And place all of the above
>> information in it so we don't lose it?
>> 
> 
> I'd be glad to post a PR and assist in helping to get it permanently fixed. I 
> certainly don't want this data to get lost and honestly our business uses 
> FreeBSD on VMWare so much that we really need a permanent fix as much as 
> anyone else :-)
> 
> The reason I've hesitated to post a PR so far is that I didn't have any truly 
> useful or concrete evidence of where the problem lies. After Dane Foster 
> contacted me and told me he could recreate the crash on demand with his 
> workload it was easier to narrow things down. The suggestion that it was an 
> interrupts issue (by possibly Bjoern Zeeb?) 

Just for the public archives.  Interrupts wasn't me.   I might have mentioned 
disabling cdrom and fdc as good as possible but everything else I cannot 
remember...


> and Dane's discovery that his crashes ceased when em0 and mpt0 share an IRQ, 
> but em0 is completely unused was starting to prove there is some strong 
> evidence here in favor of the interrupts issue.
> 
> Dane, what's the status on your end? Has your fix still been successful? Is 
> it also stable if you simply set hint.mpt.0.msi_enable="1" ?

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[dane foster]

Hey all,

On 25/05/2012, at 1:47 AM, Mark Felder wrote:

> On Wed, 23 May 2012 17:30:40 -0500, Adrian Chadd  wrote:
> 
>> Hi,
>> 
>> can you please, -please- file a PR? And place all of the above
>> information in it so we don't lose it?
>> 
> 
> I'd be glad to post a PR and assist in helping to get it permanently fixed. I 
> certainly don't want this data to get lost and honestly our business uses 
> FreeBSD on VMWare so much that we really need a permanent fix as much as 
> anyone else :-)
> 
> The reason I've hesitated to post a PR so far is that I didn't have any truly 
> useful or concrete evidence of where the problem lies. After Dane Foster 
> contacted me and told me he could recreate the crash on demand with his 
> workload it was easier to narrow things down. The suggestion that it was an 
> interrupts issue (by possibly Bjoern Zeeb?) and Dane's discovery that his 
> crashes ceased when em0 and mpt0 share an IRQ, but em0 is completely unused 
> was starting to prove there is some strong evidence here in favor of the 
> interrupts issue.
> 
> Dane, what's the status on your end? Has your fix still been successful? Is 
> it also stable if you simply set hint.mpt.0.msi_enable="1" ?
> 

The situation I've got that's stable now is:

hw.pci.enable_msi="0"
hw.pci.enable_msix="0"

in /boot/loader.conf

and:

samael:~:% vmstat -i  [ 6:31PM]
interrupt  total   rate
irq1: atkbd0   6  0
irq18: em0 mpt0  3061100 15
irq19: em1   6891706 35
cpu0: timer166383735868
cpu1: timer166382123868
cpu3: timer166382123868
cpu2: timer166382121868
Total  675482914   3525

Not using em0. This works for 8 (FreeBSD samael.slush.ca 8.3-STABLE FreeBSD 
8.3-STABLE #1: Mon May  7 11:51:03 NZST 2012 
r...@samael.slush.ca:/usr/obj/usr/src/sys/DENE  amd64).

Neither of those settings on their own seem to stop it from happening.

The 9 box I've tried this on still hangs almost every time i run handbrake, no 
matter whether MSI/MSIX is enabled, or I have separate IRQs for mpt0 and em0/1

I can cause the hang mostly on demand, but not quite sure what information to 
provide from the hung system. If somebody can let me know what they need, 
including root access, I can make that happen.

Cheers,

Dane



> 
> Thanks!




___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Wed, 23 May 2012 17:30:40 -0500, Adrian Chadd   
wrote:



Hi,

can you please, -please- file a PR? And place all of the above
information in it so we don't lose it?



I'd be glad to post a PR and assist in helping to get it permanently  
fixed. I certainly don't want this data to get lost and honestly our  
business uses FreeBSD on VMWare so much that we really need a permanent  
fix as much as anyone else :-)


The reason I've hesitated to post a PR so far is that I didn't have any  
truly useful or concrete evidence of where the problem lies. After Dane  
Foster contacted me and told me he could recreate the crash on demand with  
his workload it was easier to narrow things down. The suggestion that it  
was an interrupts issue (by possibly Bjoern Zeeb?) and Dane's discovery  
that his crashes ceased when em0 and mpt0 share an IRQ, but em0 is  
completely unused was starting to prove there is some strong evidence here  
in favor of the interrupts issue.


Dane, what's the status on your end? Has your fix still been successful?  
Is it also stable if you simply set hint.mpt.0.msi_enable="1" ?



Thanks!
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Adrian Chadd]

Hi,

can you please, -please- file a PR? And place all of the above
information in it so we don't lose it?

If this is indeed the problem then I really think we should root cause
why the driver and/or interrupt handling code is getting angry with
the shared interrupt.

I'd also appreciate it if you and the other people who can reproduce
this could work with the em/mpt driver people and root cause why this
is going. I think having FreeBSD on vmware work stable out of the box
without these kinds of tweaks is the way to go - who knows what else
is lurking here..

I'm very very glad you've persisted with this and if I had them, I'd
send you a "FreeBSD persistent bug reporter!" t-shirt.

Thanks,


Adrian
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Mon, 21 May 2012 13:47:45 -0500, Michael Powell  
 wrote:


Very curious how 'irq 22 at device 22.0' and 'dev.mpt.0.%location:  
slot=22'

all match with a '22'.


Strangely here in ESXi that doesn't work the same. Emulated BIOS must be  
considerably different... :/


$ vmstat -i
interrupt  total   rate
irq1: atkbd0   6  0
irq6: fdc0 9  0
irq15: ata1   34  0
irq16: em162  0
irq18: em0178079 17
cpu0: timer  4136470400
irq256: mpt0  112544 10
Total4427204428
$ sysctl -a | grep mpt
kern.sched.preemption: 1
kern.sched.preempt_thresh: 64
dev.mpt.0.%desc: LSILogic SAS/SATA Adapter
dev.mpt.0.%driver: mpt
dev.mpt.0.%location: slot=0 function=0 handle=\_SB_.PCI0.PE40.S1F0
dev.mpt.0.%pnpinfo: vendor=0x1000 device=0x0054 subvendor=0x15ad  
subdevice=0x1976 class=0x010700

dev.mpt.0.%parent: pci3
dev.mpt.0.debug: 3
dev.mpt.0.role: 1
dev.mpt.0.wake: 0

irq256 and slot ... 0. Interesting.


The obvious thing here is we are comparing a userland Vbox guest to a  
VMWare
hypervisor. From what little I know concerning any of this, to me it  
sounds

vaguely like an APIC, LAPIC, and IO/APIC bug. There are known bugs wrt to
BIOS setting up IRQ routing incorrectly, and/or providing incorrect ACPI
and/or IMS tables to operating systems.


FWIW, VirtualBox and ESXi are nearly the same except ESXi just has as  
minimal an OS as possible for performance reasons. And what you're  
describing is exactly what I've been thinking for a long time but I just  
haven't had the proof.


The parallel in this case would be the logical or synthetic so-called  
"BIOS"
that the VMWare hypervisor presents to the FreeBSD guest at guest boot  
time.
In this case the truest fix for the problem would fall to VMWare, e.g.  
if the

hypervisor is setting up tables in such a way as to create the shared IRQ
problem in the first place.
If my idea/theory/potential hypothesis has any merit. I do not understand
why any of this would be different depending upon which guest is  
installed,

but I also know absolutely nothing about VMWare hypervisor internals.


I don't know enough about how it's supposed to work but hopefully we're  
getting close to nailing down the real VMWare bug and we can finally tell  
their engineering to fix it.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Michael Powell]

Mark Felder wrote:

> OK guys I've been talking with another user who can recreate this crash
> and the last bit of information we've learned seems to be leaning towards
> interrupts/IRQ issues like someone (bz@ perhaps?) suggested.
> 
> I'm still trying to test this myself, but the other user was able to
> recreate my crash pretty much on demand. The fix was to not use the first
> NIC in the VM because it will always share an IRQ with mpt0. Once mpt0 is
> on its own the crash does not seem to be reproducible anymore.
> 
[snip]

I am not anywhere near your level in this subject area. My understanding is 
limited and do not have the in-depth experience. However, please allow me to 
possibly add an idea or two.

I am shakedown testing FreeBSD 9 in a VirtualBox VM - so there is definitely 
a degree of 'apples vs oranges' present. VirtualBox (as I am using it) is a 
userland app and not a bare-metal hypervisor. When I set up the VM I chose 
to use the synthetic SAS controller as that would best represent actual 
server hardware in my workplace, along with the corresponding mpt driver in 
the FreeBSD 9 guest.

Please note some of the following for comparative purposes only:

[...]
Event timer "LAPIC" quality 400
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0  irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0:  on motherboard
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
Timecounter "HPET" frequency 14318180 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
[...]
em0:  port 0xd000-0xd007 
mem 0xf000-0xf001 irq 19 at device 3.0 on pci0
[...]
mpt0:  port 0xd100-0xd1ff mem 
0xf082-0xf083,0xf084-0xf085 irq 22 at device 22.0 on pci0
mpt0: MPI Version=1.5.0.0
[...]

The em0 is the first Intel NIC in Vbox and notice how it and mpt0 come up 
with distinctly different IRQs.

A sysctl -a |grep mpt returns this:

device  mpt
kern.sched.preemption: 1
kern.sched.preempt_thresh: 80
dev.mpt.0.%desc: LSILogic SAS/SATA Adapter
dev.mpt.0.%driver: mpt
dev.mpt.0.%location: slot=22 function=0
dev.mpt.0.%pnpinfo: vendor=0x1000 device=0x0054 subvendor=0x1000 
subdevice=0x8000 class=0x01
dev.mpt.0.%parent: pci0
dev.mpt.0.debug: 3
dev.mpt.0.role: 1

Very curious how 'irq 22 at device 22.0' and 'dev.mpt.0.%location: slot=22' 
all match with a '22'.

The obvious thing here is we are comparing a userland Vbox guest to a VMWare 
hypervisor. From what little I know concerning any of this, to me it sounds 
vaguely like an APIC, LAPIC, and IO/APIC bug. There are known bugs wrt to 
BIOS setting up IRQ routing incorrectly, and/or providing incorrect ACPI 
and/or IMS tables to operating systems.

The parallel in this case would be the logical or synthetic so-called "BIOS" 
that the VMWare hypervisor presents to the FreeBSD guest at guest boot time. 
In this case the truest fix for the problem would fall to VMWare, e.g. if the 
hypervisor is setting up tables in such a way as to create the shared IRQ 
problem in the first place.

If my idea/theory/potential hypothesis has any merit. I do not understand 
why any of this would be different depending upon which guest is installed, 
but I also know absolutely nothing about VMWare hypervisor internals.

> 
> Is there any other way we can make mpt0 get its own dedicated IRQ without
> having to do this? The problem is that it causes us to have to make
> rc.conf changes, pf.conf changes, and who knows what other software could
> be on these machines that is trying to bind to a specific NIC...
> 

Very possibly Andrew's device.hints is probably your best shot at a 
workaround. 

Wish you the best of luck in any case. You have done quite a job in 
researching this problem even to arrive at this point. Thank-you for that, 
and for sharing it with the community. Even though I can't really offer the 
kind of assistance you require, I have followed along with interest for self 
edification.

-Mike

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Mon, 21 May 2012 12:01:19 -0500, Andrew Boyer   
wrote:


You could try switching mpt to MSI.  MSI interrupts are never shared.   
Add this to /boot/device.hints:



hint.mpt.0.msi_enable="1"



Currently implementing this on the known crashy servers. I've been looking  
around and all of our VM's that do NOT crash also do not share interrupts  
between em0/mpt0.


Thank you very much if this is the fix we will be SO grateful.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Andrew Boyer]

On May 21, 2012, at 12:41 PM, Mark Felder wrote:

> OK guys I've been talking with another user who can recreate this crash and 
> the last bit of information we've learned seems to be leaning towards 
> interrupts/IRQ issues like someone (bz@ perhaps?) suggested.
> 
> I'm still trying to test this myself, but the other user was able to recreate 
> my crash pretty much on demand. The fix was to not use the first NIC in the 
> VM because it will always share an IRQ with mpt0. Once mpt0 is on its own the 
> crash does not seem to be reproducible anymore.
> 
> Before:
> 
> $ vmstat -i
> interrupt  total   rate
> irq1: atkbd0 378  0
> irq6: fdc0 9  0
> irq15: ata1   34  0
> irq16: em1687237  1
> irq18: em0 mpt0319094024539
> cpu0: timer236770821400
> Total  556552503940
> 
> After:
> 
> $ vmstat -i
> interrupt  total   rate
> irq1: atkbd0  38  0
> irq6: fdc0 9  0
> irq15: ata1   34  0
> irq16: em1  2811 15
> irq17: em2 5  0
> cpu0: timer71013398
> irq256: mpt0   12163 68
> Total  86073483
> 
> 
> Is there any other way we can make mpt0 get its own dedicated IRQ without 
> having to do this? The problem is that it causes us to have to make rc.conf 
> changes, pf.conf changes, and who knows what other software could be on these 
> machines that is trying to bind to a specific NIC...
> 
> 
> Thanks!
> 

You could try switching mpt to MSI.  MSI interrupts are never shared.  Add this 
to /boot/device.hints:

> hint.mpt.0.msi_enable="1"


-Andrew

--
Andrew Boyerabo...@averesystems.com




___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

OK guys I've been talking with another user who can recreate this crash  
and the last bit of information we've learned seems to be leaning towards  
interrupts/IRQ issues like someone (bz@ perhaps?) suggested.


I'm still trying to test this myself, but the other user was able to  
recreate my crash pretty much on demand. The fix was to not use the first  
NIC in the VM because it will always share an IRQ with mpt0. Once mpt0 is  
on its own the crash does not seem to be reproducible anymore.


Before:

$ vmstat -i
interrupt  total   rate
irq1: atkbd0 378  0
irq6: fdc0 9  0
irq15: ata1   34  0
irq16: em1687237  1
irq18: em0 mpt0319094024539
cpu0: timer236770821400
Total  556552503940

After:

$ vmstat -i
interrupt  total   rate
irq1: atkbd0  38  0
irq6: fdc0 9  0
irq15: ata1   34  0
irq16: em1  2811 15
irq17: em2 5  0
cpu0: timer71013398
irq256: mpt0   12163 68
Total  86073483


Is there any other way we can make mpt0 get its own dedicated IRQ without  
having to do this? The problem is that it causes us to have to make  
rc.conf changes, pf.conf changes, and who knows what other software could  
be on these machines that is trying to bind to a specific NIC...



Thanks!
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Quick update:

I have received word last night that this crash has been consistently  
happening to someone on FreeBSD 9 and they're looking for more ideas. I  
changed the following 41 days ago:


- Video memory to "auto" if it wasn't already
- SCSI controller changed from LSI Logic Parallel to LSI Logic SAS

It uses the same driver (da) but so far has been holding steady for us. As  
far as the video memory -- many of our servers somehow had video memory  
set to 1MB which seemed strange; newer builds of FreeBSD on ESXi do not  
show this option. Perhaps there was a build of ESXi in the past that had a  
different setting for video memory when you selected FreeBSD?


Another change people might want to do as suggested to us by VMWare  
Support:


- Change CPU/MMU Virtualization to the bottom option -- "Use Intel  
VTx/AMD-V for instruction set virtualization and Intel EPT / AMD RVI for  
MMU virtualization"


Supposedly there are autodetection issues here with some OSes -- they  
named some BSDs and Netware.



I'll provide further updates if anything changes, but this seems to be  
working well so far. We won't begin to trust it until we can hit at least  
100 days of uptime, though. Unfortunately I was hoping to upgrade these  
servers to 8.3 before then...


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Doug Barton]

On 4/2/2012 3:59 PM, Joe Greco wrote:
>> On 4/2/2012 11:43 AM, Joe Greco wrote:
>>> As a user, you can't win.  If you don't report
>>> a problem, you get criticized.  If you report a problem but can't figure
>>> out how to reproduce it, you get criticized.  If you can reproduce it
>>> but you don't submit a workaround, you get criticized.  If you submit a
>>> workaround but you don't submit a patch, you get criticized.  If you
>>> submit a patch but it's not in the preferred format, you get criticized.
>>
>> I'm still not sure what you're taking as criticism. Nothing I've said
>> was intended that way, nor should it be read that way. If you feel that
>> you've been criticized by others in the manner you describe, you should
>> probably take it up with them on an individual basis.
> 
> It certainly seemed to me that
> 
>> As much as I'm sensitive to your production requirements, realistically
>> it's not likely that you'll get a helpful result without testing a newer
>> version. 8.2 came out over a year ago, many many things have changed
>> since then.
> 
> was an unwarranted criticism for reasons that I've already explained.

Everything in that paragraph is a fact. If you feel criticized when
people state facts, I'm not sure how much I can help you.

Please note, I didn't say, "You're an idiot for running old stuff." I
even explicitly stated that I understood *why* the OP is running an old
version. Nevertheless, the facts are what they are. The only way we can
deal rationally with the world is to acknowledge reality and deal with
it. Wishing it were otherwise isn't really useful. :)

> Or perhaps this:
> 
>> And since you can't reliably reproduce the problem, how do you expect us
>> to? I understand that these sorts of bugs are difficult/annoying, etc.
>> Been there, done that.
> 
> Which would appear to be suggesting that either (or possibly both):
> 
> 1) The reporter has a duty to be able to "reliably reproduce the problem"
>prior to reporting, and/or
> 
> 2) That there was some unreasonable expectation on the reporter's part
>that you were expected to reproduce it.

Quite the contrary, I was responding to your implication that there is
some other answer that we should be able to give the OP, other than "Try
a newer version."

Various people have chimed in on the thread, all have offered
suggestions, none of which (AFAICS) have helped. I continue to maintain
that the best course of action for the OP would be to try the latest
8-stable.

And BTW, there are (at least) 2 reasons for that. First, the bug may
actually be fixed. But second, we're in the middle of a release cycle
for 8.3 right now. If the bug persists in the latest code it will be
easier to get the right eyes onto the problem. That benefits both the OP
and the community at large.

Doug
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Guys,

The crash on my machine with debugging has evaded me for a few days. I'm  
still looking for further suggestions of things I should grab from the DDB  
when it happens again.


Thanks for the help everyone!
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On 4/2/2012 11:43 AM, Joe Greco wrote:
> > As a user, you can't win.  If you don't report
> > a problem, you get criticized.  If you report a problem but can't figure
> > out how to reproduce it, you get criticized.  If you can reproduce it
> > but you don't submit a workaround, you get criticized.  If you submit a
> > workaround but you don't submit a patch, you get criticized.  If you
> > submit a patch but it's not in the preferred format, you get criticized.
> 
> I'm still not sure what you're taking as criticism. Nothing I've said
> was intended that way, nor should it be read that way. If you feel that
> you've been criticized by others in the manner you describe, you should
> probably take it up with them on an individual basis.

It certainly seemed to me that

> As much as I'm sensitive to your production requirements, realistically
> it's not likely that you'll get a helpful result without testing a newer
> version. 8.2 came out over a year ago, many many things have changed
> since then.

was an unwarranted criticism for reasons that I've already explained.

Or perhaps this:

> And since you can't reliably reproduce the problem, how do you expect us
> to? I understand that these sorts of bugs are difficult/annoying, etc.
> Been there, done that.

Which would appear to be suggesting that either (or possibly both):

1) The reporter has a duty to be able to "reliably reproduce the problem"
   prior to reporting, and/or

2) That there was some unreasonable expectation on the reporter's part
   that you were expected to reproduce it.

I consider 1) to be ridiculous, as long as the reporter is reasonably
willing to work to resolve the issue, that should certainly be good
enough, and he's certainly been interactive enough to _my_ comments,
and 2) seems to be nowhere in sight in the reporter's comments, but
is nonetheless present in your response.

Please respect Reply-to.  Thanks.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Doug Barton]

On 4/2/2012 11:43 AM, Joe Greco wrote:
> As a user, you can't win.  If you don't report
> a problem, you get criticized.  If you report a problem but can't figure
> out how to reproduce it, you get criticized.  If you can reproduce it
> but you don't submit a workaround, you get criticized.  If you submit a
> workaround but you don't submit a patch, you get criticized.  If you
> submit a patch but it's not in the preferred format, you get criticized.

I'm still not sure what you're taking as criticism. Nothing I've said
was intended that way, nor should it be read that way. If you feel that
you've been criticized by others in the manner you describe, you should
probably take it up with them on an individual basis.

My experience of FreeBSD as a community is that we tend to be both less
critical of users, and less tolerant of it. Especially when compared to
other communities that I've interacted with.

Doug

-- 

This .signature sanitized for your protection
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On 03/30/2012 07:41, Joe Greco wrote:
> >> On 3/29/2012 7:01 AM, Joe Greco wrote:
>  On 3/28/2012 1:59 PM, Mark Felder wrote:
> > FreeBSD 8-STABLE, 8.3, and 9.0 are untested
> 
>  As much as I'm sensitive to your production requirements, realistically
>  it's not likely that you'll get a helpful result without testing a newer
>  version. 8.2 came out over a year ago, many many things have changed
>  since then.
> 
>  Doug
> >>>
> >>> So you're saying that he should have been using 8.3-RELEASE, then.
> >>
> >> That isn't what I said at all, sorry if I wasn't clear. The OP mentioned
> >> 9.0-RELEASE, and in the context of his message (which I snipped) he
> >> mentioned 8-stable. That's what I was referring to.
> > 
> > And since both the poster and I made it clear that this doesn't seem
> > to be a case of "it fails reliably on a machine of your choosing",
> > just installing random other versions and hoping that it's going to
> > cause a fail ... well, let's just say that doesn't make a whole lot
> > of sense.  Or at least it's a recipe for a hell of a lot of busywork,
> > busywork not guaranteed to return any sort of useful result.
> 
> And since you can't reliably reproduce the problem, how do you expect us
> to? I understand that these sorts of bugs are difficult/annoying, etc.
> Been there, done that.

Nobody expected you to.  We're trying to figure out any commonalities
that might exist; these may serve to help shed light on where the
problem lies.

The interesting thing is that I took it and looked at it and came to a
conclusion that might have been wrong, though I think the trail of
reasoning I used was itself reasonable, given my exceedingly small (one
example of problem) sample size.

Mark's able to actually *reproduce* the problem on separate installs
and with circumstances that are at least somewhat different than what
my theory involved, though it is not quite possible to rule out some
sort of corruption.

Since I have to *assume* that many sites run some sort of FreeBSD on
their VMware gear, given that VMware actually lists it as a supported
version and VMware generally does things "for profit", I am still kind
of of the opinion that this is some sort of corruption bug, one that I
triggered inadvertently, but one that Mark's environment reproduces
rather more frequently.  That just seems so unlikely, but more unlikely
things have come to pass, so I'm holding onto it as my working theory ;-)

I still plan to try to recover my broken VM from backups at some point
if time permits.

But in short, to answer your question:  I don't *care* if you can
reproduce the problem.  As a user, you can't win.  If you don't report
a problem, you get criticized.  If you report a problem but can't figure
out how to reproduce it, you get criticized.  If you can reproduce it
but you don't submit a workaround, you get criticized.  If you submit a
workaround but you don't submit a patch, you get criticized.  If you
submit a patch but it's not in the preferred format, you get criticized.

Hm.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Doug Barton]

On 03/30/2012 07:41, Joe Greco wrote:
>> On 3/29/2012 7:01 AM, Joe Greco wrote:
 On 3/28/2012 1:59 PM, Mark Felder wrote:
> FreeBSD 8-STABLE, 8.3, and 9.0 are untested

 As much as I'm sensitive to your production requirements, realistically
 it's not likely that you'll get a helpful result without testing a newer
 version. 8.2 came out over a year ago, many many things have changed
 since then.

 Doug
>>>
>>> So you're saying that he should have been using 8.3-RELEASE, then.
>>
>> That isn't what I said at all, sorry if I wasn't clear. The OP mentioned
>> 9.0-RELEASE, and in the context of his message (which I snipped) he
>> mentioned 8-stable. That's what I was referring to.
> 
> And since both the poster and I made it clear that this doesn't seem
> to be a case of "it fails reliably on a machine of your choosing",
> just installing random other versions and hoping that it's going to
> cause a fail ... well, let's just say that doesn't make a whole lot
> of sense.  Or at least it's a recipe for a hell of a lot of busywork,
> busywork not guaranteed to return any sort of useful result.

And since you can't reliably reproduce the problem, how do you expect us
to? I understand that these sorts of bugs are difficult/annoying, etc.
Been there, done that.

> In the meantime, it's unrealistic to tell people to use supported
> releases, to wait fifteen months between releases, and then to criticize
> people complaining about problems with a supported release for "using
> old code".

Just to be clear, I didn't criticize anyone. And I share your
frustration with the length of the 8.3 release cycle. I really wish I
had a better answer, but as much as you and I may wish that things were
different, "Try a newer version" is the best answer we have atm.

Doug

-- 

This .signature sanitized for your protection
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Jim Bryant]

Mark Felder wrote:

On Thu, 29 Mar 2012 12:24:30 -0500,  wrote:



I just started reading this tread, but I am wondering if I missed
something here. What does this have to do with "Windows 7"?


I emailed him off-list but I'm guessing he thought this was on VMWare 
Workstation or another product that would virtualize FreeBSD on top of 
Windows as the host OS.

___


Correct...My bad.

jim

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Fri, 30 Mar 2012 11:53:10 -0500, Joe Greco  wrote:


On the same vmdk files?  "Deleting the VM" makes it sound like not.


Fresh new VMDK files every time, and always thick provisioned.


None of the other VM's, even the VM's that had been abused in this
horribly insensitive manner of being placed on intolerably slow iSCSI,
developed this condition.


We've seen similar results. Baffling how VMs you know are worse off never  
develop this condition.



There are dozens of other VM's running on these hosts, alongside the
one that was exhibiting this behaviour.
The VM continued to exhibit this behaviour even after having been moved
onto a different ESXi platform and architecture (Opteron->Xeon).
For the problem to "follow" the VM in this manner, and afflict *only*
the one VM, strongly suggests that it is something that is contained
within the VM files that constitute this VM.  That is consistent with
the observation that the problem arose at a point where the VM is
known to have had all those files moved from one location to a dodgy
location.


We were hoping that was the explanation as well, but rebuilding the VM  
entirely from scratch on a new host and seeing the crash come back was a  
big blow to morale. :(



That's why I believe the evidence points to corruption of some sort.
Of course, your case makes this all interesting.


For the last year I've been convinced it's something hidden inside ESXi's  
I/O virtualization layer that happens to trigger on only certain VMs.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On Fri, 30 Mar 2012 09:44:47 -0500, Joe Greco  wrote:
> > Have you migrated these hosts, or were they installed in-place and
> > never moved?
> > fwiw the apparent integrity of things on the VM is consistent with
> > our experience too.
> 
> VMMotion and StorageVMMotion does not seem to affect the stability. Even  
> deleting the VM, rebuilding from scratch, re-installing all packages from  
> scratch, copying over a few configs and then copying in any other data  
> (perhaps website data) does not solve the problem.

On the same vmdk files?  "Deleting the VM" makes it sound like not.

> However, our two most notorious for crashing happen to be webservers. We  
> moved one to hardware. We simply rsync'd the exact data (entire OS and  
> files) off the VM onto hardware, made a few config changes (fstab, network  
> interface) and it's been running for 4+ months now with zero crashes.

That part doesn't shock me at all.

> I don't think it's corruption :/

Then it is hard to see what it is.

>From my perspective:

We had a perfectly functional, nearly zero-traffic VM, since Jabber
traffic averages no more than a few messages per hour.  It was working
for quite some time.

We moved it from a local datastore to an iSCSI datastore that ended up
getting periodically crushed by the load (in particular during the
periodic daily load imposed by a bunch of VM's all running at once).
At this point, this one VM started hanging on I/O.  We expected that
this would clear up upon return to a host with a local datastore.  It
did not.

This ended up as a broken VM, one that would hang up overnite, maybe
not every night, but several times a week at least.

But wait:

None of the other VM's, even the VM's that had been abused in this
horribly insensitive manner of being placed on intolerably slow iSCSI,
developed this condition.

There are dozens of other VM's running on these hosts, alongside the
one that was exhibiting this behaviour.

The VM continued to exhibit this behaviour even after having been moved
onto a different ESXi platform and architecture (Opteron->Xeon).

For the problem to "follow" the VM in this manner, and afflict *only*
the one VM, strongly suggests that it is something that is contained
within the VM files that constitute this VM.  That is consistent with
the observation that the problem arose at a point where the VM is
known to have had all those files moved from one location to a dodgy
location.

That's why I believe the evidence points to corruption of some sort.

Of course, your case makes this all interesting.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

We don't have any indications that before the crash processes will take  
unusual amounts of CPU. The only time there is high CPU usage is at the  
point where it does enter the crashed state and no longer seems to be able  
to communicate with the disk.


I'm not sure this is the same bug but we'll keep an eye out for it. Thanks  
for reporting your experience!

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Fri, 30 Mar 2012 09:44:47 -0500, Joe Greco  wrote:


Have you migrated these hosts, or were they installed in-place and
never moved?
fwiw the apparent integrity of things on the VM is consistent with
our experience too.


VMMotion and StorageVMMotion does not seem to affect the stability. Even  
deleting the VM, rebuilding from scratch, re-installing all packages from  
scratch, copying over a few configs and then copying in any other data  
(perhaps website data) does not solve the problem.


However, our two most notorious for crashing happen to be webservers. We  
moved one to hardware. We simply rsync'd the exact data (entire OS and  
files) off the VM onto hardware, made a few config changes (fstab, network  
interface) and it's been running for 4+ months now with zero crashes.


I don't think it's corruption :/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On Thu, 29 Mar 2012 19:27:31 -0500, Joe Greco  wrote:
> 
> > It also doesn't explain the experience here, where one VM basically
> > crapped out but only after a migration - and then stayed crapped out.
> > It would be interesting to hear about your datastore, how busy it is,
> > what technology, whether you're using thin, etc.  I just have this real
> > strong feeling that it's some sort of corruption with the vmfs3 and thin
> > provisioned disk format, but it'd be interesting to know if that's
> > totally off-track.
> 
> We've ruled out SAN, but we haven't ruled out VMFS. Even FreeBSD Guests on  
> standalone ESXi servers with no SAN exhibit this crash.
>
> For the record, we only use thick provisioning and if it was corruption  
> I'm not sure what layer the corruption could be at. The crashy servers  
> show no abnormalities when I run either `freebsd-update IPS` or  
> `pkg_libchk` to confirm checksums of all installed programs. Now the other  
> data on there... it's not exactly verified, but our backups via rsnapshot  
> seem to prove there is no issue there or we'd have lots of new files each  
> run.

Crud, there goes part of my theory :-)

Have you migrated these hosts, or were they installed in-place and
never moved?

fwiw the apparent integrity of things on the VM is consistent with
our experience too.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On 3/29/2012 7:01 AM, Joe Greco wrote:
> >> On 3/28/2012 1:59 PM, Mark Felder wrote:
> >>> FreeBSD 8-STABLE, 8.3, and 9.0 are untested
> >>
> >> As much as I'm sensitive to your production requirements, realistically
> >> it's not likely that you'll get a helpful result without testing a newer
> >> version. 8.2 came out over a year ago, many many things have changed
> >> since then.
> >>
> >> Doug
> > 
> > So you're saying that he should have been using 8.3-RELEASE, then.
> 
> That isn't what I said at all, sorry if I wasn't clear. The OP mentioned
> 9.0-RELEASE, and in the context of his message (which I snipped) he
> mentioned 8-stable. That's what I was referring to.

And since both the poster and I made it clear that this doesn't seem
to be a case of "it fails reliably on a machine of your choosing",
just installing random other versions and hoping that it's going to
cause a fail ... well, let's just say that doesn't make a whole lot
of sense.  Or at least it's a recipe for a hell of a lot of busywork,
busywork not guaranteed to return any sort of useful result.

What you suggest is a fine solution for "My ASUS Sempron box fails 
when I do X!" -- in such a case, "Try a different version of FreeBSD" 
makes lots of sense.  The problem is, in a virtualization environment,
theoretically the virtual hosts are all the same sort of hardware 
(modulo any specific configuration changes of course), so when someone
presents a problem that afflicts only a percentage of their VM's, it
is important to keep in mind that you are not interacting with physical
hardware, and that reinstalling an OS on a "problem" VM...?  

Well, let's just say I like real hardware better for many reasons.

In the meantime, it's unrealistic to tell people to use supported
releases, to wait fifteen months between releases, and then to criticize
people complaining about problems with a supported release for "using
old code".

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Adrian Chadd]

Again, it's starting to sound like an interrupt handling issue which
may or may not be limited to the storage device.

You'll have to engage someone who knows those device drivers and
likely have them add some debugging to the driver which can be easily
flipped on (via binaries in a ramdisk - very important if you can't
run sysctl because your disk IO has locked up!) to see what the
current state of things.

It's likely that the BSD mpt(4) and other storage drivers, and/or our
interrupt handling code, is just slightly different enough to confuse
the snot out of VMWare. I'd first look at the obvious - (eg, if you've
just stopped receiving interrupts, even if new IO is scheduled). I'd
also ask VMware if they have any tools that they can run on a VM to
get the state of the internal emulated driver. For example, register
dumps of the device to see if it's in a hung state, register dumps of
the PIC/APIC to see what state they're in, etc.

Maybe pull in someone like ixsystems and see if they can help debug
this kind of stuff? If you're paying vmware for support, you could
pull them into things with ixsystems and see if the two of them can
help you sort this out?

Thanks,



Adrian
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Jerome Herman]

On 28/03/2012 22:59, Mark Felder wrote:
Alright guys, I'm at the end of my rope here. For those that haven't 
seen my previous emails here's the (not so) quick breakdown:


Overview:

FreeBSD ?? - 7.4 never crash
FreeBSD 8.0 - 8.2 crashes
FreeBSD 8-STABLE, 8.3, and 9.0 are untested (Sorry, not possible in 
our production at this time, and we were hoping we could base some 
stuff on 8.3 for long term stability...)
ESXi: Confirmed ESXi 4.0 - 5.0 has this problem. Haven't tested on 
others.



History:

Over the course of the last 2 years we've been banging our heads on 
the wall. VMWare is done debugging this. They claim it's not a VMWare 
issue. They can't identify what the heck happens. We had a glimmer of 
hope with ESXi 5.0 fixing it because we never saw any crashes in the 
handful of deployments, but our dreams were crushed today -- two days 
before an outage to begin migration to ESXi 5.0 -- when a customer's 
ESXi 5.0 server and FreeBSD 8.2 guest crashed.



Crash Details:

The keyboard/mouse usually stops responding for input on the console; 
normally we can't type in a username or password. However, we can 
switch VTs.


If there's a shell on the console and we can type, we can only run 
things in memory. Any time we try to access the disk it will hang 
indefinitely.


The server still has network access. We can ping it without issue. SSH 
of course kicks you out because it can't do any I/O.


If we were to serve a lightweight http server off a memory backed 
filesystem I'm confident it would run just fine as long as it wasn't 
logging or anything.


On ESXi you see that there is a CPU spike of 100% that goes on 
indefinitely. No idea what the FreeBSD OS itself thinks it is doing 
because we can't run top during the crash.


This crash can affect a server and happen multiple times a week. It 
can also not show up for 180 days or more. But it does happen. The 
server can be 100% idle and crash. We have servers that do more I/O 
than the ones that crash could ever attempt to do and these don't 
crash at all. Completely inexplicable.



Things we've looked into:

Nothing about the installed software matters. We've tried cross 
referencing the crashed servers by the programs they run but the base 
OS is the only common denominator due to the wide variety of servers 
it has affected.


Storage doesn't matter. We've tried different iSCSI SANs, we've tried 
different switches, we've tried local datastores on the ESXi servers 
themselves.


HP servers, Dell servers -- doesn't seem to matter either. (All with 
latest firmwares, BIOSes, etc)


VMWare gave us a ton of debugging tasks, and we've given them 
gigabytes of debugging info and data; they can't find anything.


VMWare tools -- with, without, using open-vm-tools makes no 
difference. I think we've done a fair job ruling out VMWare.



I think we've finally found enough data that this is definitely 
something in the FreeBSD world. I'm going to begin prepping some of 
the known crashy servers with more debugging. Any suggestions on what 
I should build the kernel with? They never do a proper panic, but I 
definitely want to at least *try* to get into the debugger the next 
time it crashes. And when it crashes, what the heck should I be 
running? I've never played with the KDB before...



Thank you for any suggestions and help you can give me
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to 
"freebsd-questions-unsubscr...@freebsd.org"



Sorry, coming a bit late to the party,

I have seen similar behavior on a few vm. All of them either Debian and 
FreeBSD. Even though CPU indication are not necessarily relevant in a 
VM, vi launched through crontab -e would take insane amount of CPU (up 
to 84%) and Apache was hanging around 350% 400% (quad CPU VM).
Now the thing is that making a VM snapshot and deploying the snapshot a 
while later, or on a different (way less loaded) VMWare platform would 
basically make it perfectly usable again.
Shutting down the VM and starting it again with only one CPU would also 
basically solve the problem. In a way Debian seemed to be able to 
survive the crisis but Disk I/O have latencies of many seconds, 
sometimes minutes. This would happen only on heavily loaded VMWare. In a 
quite similar way older version of Debian never shown the problem.


Can you test whether you have similar behavior on your platform ?
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 19:27:31 -0500, Joe Greco  wrote:


It also doesn't explain the experience here, where one VM basically
crapped out but only after a migration - and then stayed crapped out.
It would be interesting to hear about your datastore, how busy it is,
what technology, whether you're using thin, etc.  I just have this real
strong feeling that it's some sort of corruption with the vmfs3 and thin
provisioned disk format, but it'd be interesting to know if that's
totally off-track.


We've ruled out SAN, but we haven't ruled out VMFS. Even FreeBSD Guests on  
standalone ESXi servers with no SAN exhibit this crash.


For the record, we only use thick provisioning and if it was corruption  
I'm not sure what layer the corruption could be at. The crashy servers  
show no abnormalities when I run either `freebsd-update IPS` or  
`pkg_libchk` to confirm checksums of all installed programs. Now the other  
data on there... it's not exactly verified, but our backups via rsnapshot  
seem to prove there is no issue there or we'd have lots of new files each  
run.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> > And then there is this one with similar symptoms and a workaround:
> >
> > http://forums.freebsd.org/showthread.php?t=3D27899
> 
> I'm now investigating those loader.conf options. I have my crashy machine
> set to use them on next boot so we'll see if it crashes now that I'm using
> LSI SAS emulated controller. If it still crashes, we'll see what happens 
> after that with those loader.conf options enabled.

Um, if I may, that's something completely different.

VMDirectPath, or PCIe passthru, is making a hardware device on a VMware
host available directly to a guest.  It'll take your LSI controller, in
the example cited, and make it unavailable to VMware ESXi, and present
it instead inside the guest environment.  You do this when you have an
app whose performance would suffer greatly when made to operate through
the indirection that a VM naturally lives in; for example, it is quite
common for FreeNAS users to pass a disk controller through to a VM guest
in order to allow a virtualized FreeNAS instance to directly manage the
physical disks.

In that case, there are some issues with ESXi and interrupt delivery to
the guest VM; virtualization doesn't actually get rid of the possibility
of ESXi problems, since the hypervisor is still ultimately involved.  It
is certainly possible that there's some common issue involving interrupt
delivery somehow, but I wouldn't get my hopes up.

It also doesn't explain the experience here, where one VM basically
crapped out but only after a migration - and then stayed crapped out.
It would be interesting to hear about your datastore, how busy it is,
what technology, whether you're using thin, etc.  I just have this real
strong feeling that it's some sort of corruption with the vmfs3 and thin
provisioned disk format, but it'd be interesting to know if that's 
totally off-track.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Doug Barton]

On 3/29/2012 7:01 AM, Joe Greco wrote:
>> On 3/28/2012 1:59 PM, Mark Felder wrote:
>>> FreeBSD 8-STABLE, 8.3, and 9.0 are untested
>>
>> As much as I'm sensitive to your production requirements, realistically
>> it's not likely that you'll get a helpful result without testing a newer
>> version. 8.2 came out over a year ago, many many things have changed
>> since then.
>>
>> Doug
> 
> So you're saying that he should have been using 8.3-RELEASE, then.

That isn't what I said at all, sorry if I wasn't clear. The OP mentioned
9.0-RELEASE, and in the context of his message (which I snipped) he
mentioned 8-stable. That's what I was referring to.

> If you'll kindly go over to http://www.freebsd.org and look under
> "Latest Releases", please note that "8.2" is a production release.
> If you don't want it to be a production release, then find a way
> to make it so, but please don't snipe at people who are using the
> code that the FreeBSD project has indicated is a current production
> offering.
> 
> There are many good reasons not to run arbitrary snapshots on your
> production gear.  It's unrealistic to expect people to run non-
> RELEASE non-production code on their production gear.  We can have
> that discussion if you don't understand that, drop me a note off-
> list and I'll be happy to explain it.

I can see that you're upset about something, sorry if my message caused
you additional stress. I actually understand the realities of production
environments quite well, and believe it or not I agree with some of your
frustration about how we handle support for our "supported" releases.
We've had various public threads about these issues, which have sparked
some quite-lively private discussions amongst our committers, and I'm
hoping that once the long-overdue 8.3-RELEASE is out we'll be able to
buckle down and start putting some of those ideas into action.

Meanwhile, this is still a volunteer project, and as a result sometimes
the best way to get attention to a problem is to verify that it hasn't
already been fixed. You've been around more than long enough to
understand this Joe. We can spend time arguing about what *should* be
(actually we can't ...) but my point was in trying to help the OP get
the most/best help the fastest way possible.

Doug
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 15:53:52 -0500, Adam Vande More  
 wrote:




Doesn't VMWare offer different types of emulated disk controllers?  If  
so,
that might be the easiest way to narrow the field.  Another thing maybe  
to

try would be to backport the mpt


Yes, they offer Paravirtual (not applicable for FreeBSD), LSI Parallel  
(default option), LSI SAS, and Buslogic (not available for 64bit).


Both LSI SAS and LSI Parallel use the mpt driver.



Also, it's not VMWare's place to claim "not our problem" when you are
paying for support.  If this doesn't happen on bare metal, it's a VMWare
issue, or they need to demonstrate it's not their issue.  At least that
would be the expectation I have.


You're right, but we've thrown a ton of money at their support and had  
direct phone access to their engineers. The best we can get out of them is  
"no indication this is a VMWare problem". It's easy for them to blow  
people off when they're as big as they've grown to be.


There is also a comment on this post indicating someone else with the  
issue

and who has received unofficial vmware feedback.

http://www.hailang.me/tech/virtual/freebsd-vmware-esx-a-weird-error-with-san-storage/


I found that post ages ago and that's me, "mf", as the only person to  
comment on it. Unfortunately our problem does not align with what he's  
describing.




And then there is this one with similar symptoms and a workaround:

http://forums.freebsd.org/showthread.php?t=27899



I'm now investigating those loader.conf options. I have my crashy machine  
set to use them on next boot so we'll see if it crashes now that I'm using  
LSI SAS emulated controller. If it still crashes, we'll see what happens  
after that with those loader.conf options enabled.



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Adam Vande More]

On Thu, Mar 29, 2012 at 1:22 PM, Mark Felder  wrote:

>
> If we assume mpt is the culprit
>

Doesn't VMWare offer different types of emulated disk controllers?  If so,
that might be the easiest way to narrow the field.  Another thing maybe to
try would be to backport the mpt

Also, it's not VMWare's place to claim "not our problem" when you are
paying for support.  If this doesn't happen on bare metal, it's a VMWare
issue, or they need to demonstrate it's not their issue.  At least that
would be the expectation I have.

There is also a comment on this post indicating someone else with the issue
and who has received unofficial vmware feedback.

http://www.hailang.me/tech/virtual/freebsd-vmware-esx-a-weird-error-with-san-storage/

And then there is this one with similar symptoms and a workaround:

http://forums.freebsd.org/showthread.php?t=27899

-- 
Adam Vande More
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On Thursday 29 March 2012 17:49:30 Joe Greco wrote:
> > > On Thursday 29 March 2012 15:42:42 Joe Greco wrote:
> > > > > Hi,
> > > 
> > > Do both 32- and 64-bit versions of FreeBSD crash?
> > 
> > We've only seen it happen on one virtual machine.  That was a 32-bit
> > version.  And it's not so much a crash as it is a "disk I/O hang".
> 
> It almost sounds like the lost interrupt issue I've seen with USB EHCI 
> devices, though disk I/O should have a retry timeout?

That doesn't seem to fit.  Why would a perfectly functional VM suddenly
develop this problem when given a slow underlying datastore (fits so far)
but then the problem *remains* when returned to a fast local datastore,
even on a different host and architecture?  And why wouldn't the other
VM's running alongside develop the same problem?

> What does "wmstat -i" output?

No idea, we reloaded the VM months ago.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 11:53:02 -0500, Alan Cox  wrote:



Not so long ago, VMware implemented a clever scheme for reducing the
overhead of virtualized interrupts that must be delivered by at least  
some

(if not all) of their emulated storage controllers:

http://static.usenix.org/events/atc11/tech/techAbstracts.html#Ahmad

Perhaps, there is a bad interaction between this scheme and FreeBSD's mpt
driver.

Alan


If we assume mpt is the culprit how can I go about diagnosing this more  
accurately? Is there something I should be looking for in vmstat -i? Too  
many interrupts? Not enough? Rate too high or too low? Or is this  
something that is much harder to track down because we're dealing with  
emulated hardware?


If any BSD devs are interested in access to our environment I think we  
could comply. I might even be able to get authorization to give you an  
account on the most crash-prone server which doesn't have any sensitive  
customer data on it. I think at this point we'd even be willing to pay  
someone to look at a server in this state just so we (and hopefully  
others) can benefit and hopefully we end up with a more reliable  
FreeBSD-on-VMWare for everyone.


I know Doug mentioned running newer OS versions and that is definitely  
tempting but because it's not 100% reproducible on demand it's hard to  
prove it fixes it without waiting 6 months. We're fighting internally here  
with "trust 9.0 fixes it" vs "jump back to 7.4 because we KNOW it doesn't  
happen there". Having someone look at this and say "oh, yes, that's a  
deficiency in mpt that appears to be fixed in the newer driver that was  
MFC'd to 8-STABLE and you'll find in 8.3-RELEASE and 9.0-RELEASE" would be  
more comforting.


Thanks to everyone for their time on this!
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 12:24:30 -0500,  wrote:



I just started reading this tread, but I am wondering if I missed
something here. What does this have to do with "Windows 7"?


I emailed him off-list but I'm guessing he thought this was on VMWare  
Workstation or another product that would virtualize FreeBSD on top of  
Windows as the host OS.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 12:05:30 -0500, Mark Atkinson   
wrote:




If this is an interrupt problem with disk i/o, then you might want to
look into (DDB(4))
show intr
show intrcount
maybe
show allrman



Thank you! I really don't know what things we should be running in DDB to  
diagnose this and we will try this upon the next crash.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Jerry]

On Thu, 29 Mar 2012 11:43:45 -0500
Jim Bryant articulated:

> Mark Felder wrote:
> > Alright guys, I'm at the end of my rope here. For those that
> > haven't seen my previous emails here's the (not so) quick breakdown:
> >
> > Overview:
> >
> > FreeBSD ?? - 7.4 never crash
> > FreeBSD 8.0 - 8.2 crashes
> > FreeBSD 8-STABLE, 8.3, and 9.0 are untested (Sorry, not possible in 
> > our production at this time, and we were hoping we could base some 
> > stuff on 8.3 for long term stability...)
> > ESXi: Confirmed ESXi 4.0 - 5.0 has this problem. Haven't tested on 
> > others.
> >
> >
> > History:
> >
> > Over the course of the last 2 years we've been banging our heads on 
> > the wall. VMWare is done debugging this. They claim it's not a
> > VMWare issue. They can't identify what the heck happens. We had a
> > glimmer of hope with ESXi 5.0 fixing it because we never saw any
> > crashes in the handful of deployments, but our dreams were crushed
> > today -- two days before an outage to begin migration to ESXi 5.0
> > -- when a customer's ESXi 5.0 server and FreeBSD 8.2 guest crashed.
> >
> >
> > Crash Details:
> >
> > The keyboard/mouse usually stops responding for input on the
> > console; normally we can't type in a username or password. However,
> > we can switch VTs.
> >
> > If there's a shell on the console and we can type, we can only run 
> > things in memory. Any time we try to access the disk it will hang 
> > indefinitely.
> >
> > The server still has network access. We can ping it without issue.
> > SSH of course kicks you out because it can't do any I/O.
> >
> > If we were to serve a lightweight http server off a memory backed 
> > filesystem I'm confident it would run just fine as long as it
> > wasn't logging or anything.
> >
> > On ESXi you see that there is a CPU spike of 100% that goes on 
> > indefinitely. No idea what the FreeBSD OS itself thinks it is doing 
> > because we can't run top during the crash.
> >
> > This crash can affect a server and happen multiple times a week. It 
> > can also not show up for 180 days or more. But it does happen. The 
> > server can be 100% idle and crash. We have servers that do more I/O 
> > than the ones that crash could ever attempt to do and these don't 
> > crash at all. Completely inexplicable.
> >
> >
> > Things we've looked into:
> >
> > Nothing about the installed software matters. We've tried cross 
> > referencing the crashed servers by the programs they run but the
> > base OS is the only common denominator due to the wide variety of
> > servers it has affected.
> >
> > Storage doesn't matter. We've tried different iSCSI SANs, we've
> > tried different switches, we've tried local datastores on the ESXi
> > servers themselves.
> >
> > HP servers, Dell servers -- doesn't seem to matter either. (All
> > with latest firmwares, BIOSes, etc)
> >
> > VMWare gave us a ton of debugging tasks, and we've given them 
> > gigabytes of debugging info and data; they can't find anything.
> >
> > VMWare tools -- with, without, using open-vm-tools makes no 
> > difference. I think we've done a fair job ruling out VMWare.
> >
> >
> > I think we've finally found enough data that this is definitely 
> > something in the FreeBSD world. I'm going to begin prepping some of 
> > the known crashy servers with more debugging. Any suggestions on
> > what I should build the kernel with? They never do a proper panic,
> > but I definitely want to at least *try* to get into the debugger
> > the next time it crashes. And when it crashes, what the heck should
> > I be running? I've never played with the KDB before...
> >
> >
> > Thank you for any suggestions and help you can give me
> 
> This sounds just like a race condition that happens under Windows 7
> on this laptop.  The race condition, as far as I can tell involves
> heavy disk access and heavy network access, and usually leaves the
> drive light on, while all activity monitors (alldisk, allcpu,
> allnetwork) are still active, although on this laptop disk takes
> priority, and network slows to a crawl.  occasionally, the mouse will
> stop working, along with everything else, but usually not.  keyboard
> is lower priority, and doesn't do anything.
> 
> You might want to check with mickeysoft, this might just be their 
> problem.  This sounds so freaking similar to the issue I get, and I 
> think it's a race condition (shared interrupts??).
> 
> This laptop is a Compaq Presario C300 series, with the 945GM chipset
> and a T7600 Core2 Duo CPU, with 3G of RAM.

{TOP POSTING CORRECTED}

I just started reading this tread, but I am wondering if I missed
something here. What does this have to do with "Windows 7"?

-- 
Jerry ♔

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the Reply-To header.
__

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listin

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Atkinson]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 03/29/2012 07:03, Mark Felder wrote:
> Alright, new data. It happened to crash about 10 minutes after I
> came in this morning and I ran some stuff in the DDB. I have no
> idea what information is useful, but perhaps someone will see
> something out of the ordinary?
> 
> 
> http://feld.me/freebsd/esx_crash/

If this is an interrupt problem with disk i/o, then you might want to
look into (DDB(4))

show intr
show intrcount

maybe

show allrman
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk90lloACgkQrDN5kXnx8yaCZACbBamQksNyWC26PUsOn5N9LJLV
ql0AoJwYCFDfXhCpZIN735V9qg0VepFf
=fCLN
-END PGP SIGNATURE-

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Alan Cox]

On Thu, Mar 29, 2012 at 11:27 AM, Mark Felder  wrote:

> On Thu, 29 Mar 2012 10:55:36 -0500, Hans Petter Selasky 
> wrote:
>
>>
>> It almost sounds like the lost interrupt issue I've seen with USB EHCI
>> devices, though disk I/O should have a retry timeout?
>>
>> What does "wmstat -i" output?
>>
>> --HPS
>>
>
>
> Here's a server that has a week uptime and is due for a crash any hour now:
>
> root@server:/# vmstat -i
> interrupt  total   rate
> irq1: atkbd0  34  0
> irq6: fdc0 9  0
> irq15: ata1   34  0
> irq16: em1778061  1
> irq17: mpt0 19217711 31
> irq18: em0 283674769460
> cpu0: timer246571507400
> Total  550242125892
>
>

Not so long ago, VMware implemented a clever scheme for reducing the
overhead of virtualized interrupts that must be delivered by at least some
(if not all) of their emulated storage controllers:

http://static.usenix.org/events/atc11/tech/techAbstracts.html#Ahmad

Perhaps, there is a bad interaction between this scheme and FreeBSD's mpt
driver.

Alan
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Jim Bryant]

This sounds just like a race condition that happens under Windows 7 on 
this laptop.  The race condition, as far as I can tell involves heavy 
disk access and heavy network access, and usually leaves the drive light 
on, while all activity monitors (alldisk, allcpu, allnetwork) are still 
active, although on this laptop disk takes priority, and network slows 
to a crawl.  occasionally, the mouse will stop working, along with 
everything else, but usually not.  keyboard is lower priority, and 
doesn't do anything.


You might want to check with mickeysoft, this might just be their 
problem.  This sounds so freaking similar to the issue I get, and I 
think it's a race condition (shared interrupts??).


This laptop is a Compaq Presario C300 series, with the 945GM chipset and 
a T7600 Core2 Duo CPU, with 3G of RAM.


Mark Felder wrote:
Alright guys, I'm at the end of my rope here. For those that haven't 
seen my previous emails here's the (not so) quick breakdown:


Overview:

FreeBSD ?? - 7.4 never crash
FreeBSD 8.0 - 8.2 crashes
FreeBSD 8-STABLE, 8.3, and 9.0 are untested (Sorry, not possible in 
our production at this time, and we were hoping we could base some 
stuff on 8.3 for long term stability...)
ESXi: Confirmed ESXi 4.0 - 5.0 has this problem. Haven't tested on 
others.



History:

Over the course of the last 2 years we've been banging our heads on 
the wall. VMWare is done debugging this. They claim it's not a VMWare 
issue. They can't identify what the heck happens. We had a glimmer of 
hope with ESXi 5.0 fixing it because we never saw any crashes in the 
handful of deployments, but our dreams were crushed today -- two days 
before an outage to begin migration to ESXi 5.0 -- when a customer's 
ESXi 5.0 server and FreeBSD 8.2 guest crashed.



Crash Details:

The keyboard/mouse usually stops responding for input on the console; 
normally we can't type in a username or password. However, we can 
switch VTs.


If there's a shell on the console and we can type, we can only run 
things in memory. Any time we try to access the disk it will hang 
indefinitely.


The server still has network access. We can ping it without issue. SSH 
of course kicks you out because it can't do any I/O.


If we were to serve a lightweight http server off a memory backed 
filesystem I'm confident it would run just fine as long as it wasn't 
logging or anything.


On ESXi you see that there is a CPU spike of 100% that goes on 
indefinitely. No idea what the FreeBSD OS itself thinks it is doing 
because we can't run top during the crash.


This crash can affect a server and happen multiple times a week. It 
can also not show up for 180 days or more. But it does happen. The 
server can be 100% idle and crash. We have servers that do more I/O 
than the ones that crash could ever attempt to do and these don't 
crash at all. Completely inexplicable.



Things we've looked into:

Nothing about the installed software matters. We've tried cross 
referencing the crashed servers by the programs they run but the base 
OS is the only common denominator due to the wide variety of servers 
it has affected.


Storage doesn't matter. We've tried different iSCSI SANs, we've tried 
different switches, we've tried local datastores on the ESXi servers 
themselves.


HP servers, Dell servers -- doesn't seem to matter either. (All with 
latest firmwares, BIOSes, etc)


VMWare gave us a ton of debugging tasks, and we've given them 
gigabytes of debugging info and data; they can't find anything.


VMWare tools -- with, without, using open-vm-tools makes no 
difference. I think we've done a fair job ruling out VMWare.



I think we've finally found enough data that this is definitely 
something in the FreeBSD world. I'm going to begin prepping some of 
the known crashy servers with more debugging. Any suggestions on what 
I should build the kernel with? They never do a proper panic, but I 
definitely want to at least *try* to get into the debugger the next 
time it crashes. And when it crashes, what the heck should I be 
running? I've never played with the KDB before...



Thank you for any suggestions and help you can give me
___
freebsd-hack...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to 
"freebsd-hackers-unsubscr...@freebsd.org"



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 10:49:30 -0500, Joe Greco  wrote:


I explained it at the time to one of my VMware friends:



This is 100% identical to what we see, Joe! And we're so unlucky that we  
have this happen on probably a dozen servers, but a handful are the really  
bad ones. We've rebuilt them from scratch many times with no improvement.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 10:55:36 -0500, Hans Petter Selasky   
wrote:


It almost sounds like the lost interrupt issue I've seen with USB EHCI
devices, though disk I/O should have a retry timeout?

What does "wmstat -i" output?

--HPS



Here's a server that has a week uptime and is due for a crash any hour now:

root@server:/# vmstat -i
interrupt  total   rate
irq1: atkbd0  34  0
irq6: fdc0 9  0
irq15: ata1   34  0
irq16: em1778061  1
irq17: mpt0 19217711 31
irq18: em0 283674769460
cpu0: timer246571507400
Total  550242125892
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 10:31:24 -0500, Eduardo Morras   
wrote:




Don't know about ESXi but on others VM Managers i can change the chipset  
emulation from ICH10 to ICH4. Can you change it to an older chipset too?


Unfortunately there's no setting in the GUI for that but I'll keep looking  
to see if there's a hidden option -- perhaps in the VM's config file.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Hans Petter Selasky]

On Thursday 29 March 2012 17:49:30 Joe Greco wrote:
> > On Thursday 29 March 2012 15:42:42 Joe Greco wrote:
> > > > Hi,
> > 
> > Do both 32- and 64-bit versions of FreeBSD crash?
> 
> We've only seen it happen on one virtual machine.  That was a 32-bit
> version.  And it's not so much a crash as it is a "disk I/O hang".

It almost sounds like the lost interrupt issue I've seen with USB EHCI 
devices, though disk I/O should have a retry timeout?

What does "wmstat -i" output?

--HPS
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On Thursday 29 March 2012 15:42:42 Joe Greco wrote:
> > > Hi,
> 
> Do both 32- and 64-bit versions of FreeBSD crash?

We've only seen it happen on one virtual machine.  That was a 32-bit
version.  And it's not so much a crash as it is a "disk I/O hang".

The fact that it was happening regularly to that one VM, while a
bunch of other similar VM's were running alongside it without any
incident, along with the problem moving with the VM as it is moved
from host to host and from Opteron to Xeon, strongly points at 
something being wrong with the VM itself.  Our systems are built
mostly by script; I rebuilt the VM a few months ago and the
problem vanished.  The rebuilt system "should" have been virtually
identical to the original.  I never actually compared them though.

My working theory was that something bad had happened to the VM
during a migration from one datastore to another.  We have a really
slow-writing iSCSI server that it had been migrated onto for a little
bit, which was where the problem first appeared, I believe.  At
first I thought it was the nightly cron jobs just exceeding the iSCSI
server's capacity to cope, so we migrated the VM onto a host with
local datastores, and it remained broken thereafter.

So my conclusion was that it seemed likely that somehow VMware's 
thin provisioned disk image had gotten fouled up, and under some
unknown use case, it could be teased into locking up further I/O
on the VM.  I wasn't able to prove it.  I tried a read-dd of the
entire disk - passed, flying.  I tried several things to duplicate
the nightly periodic tasks where it seemed so prone to locking up.
They all ran fine.  But if I left the machine run, it'd do it
again eventually.

I explained it at the time to one of my VMware friends:

> But here's where it gets weird.  Three times, now, one VM - our Jabber
> server - has gone wonky in the wee early AM hours.  Disk I/O on the VM
> just locks up.  You can type at the console until it does I/O, so you
> can put in "root" at the login: prompt but never get a pw prompt.  My
> systems all run "top" from /etc/ttys and I can see that a whole bunch
> of processes are stopped in "getblk".  It's like the iSCSI disk has gone
> away, except it hasn't, since the other VM's are all happily churning
> away, on the same datastore, on the same VMware host.

http://www.sol.net/tmp/freebsd/freebsd-esxi-lockup.gif

> Now it's *possible* that the problem actually happens after the 3AM cron
> run (note slight CPU/memory drop) but the Jabber implosion actually
> happens around 0530, see drop in memory%.  But the root problem at the
> VM level seems to be that disk I/O has frozen.  I can't tell for sure when
> that happens.  All three instances are similar to this.
> 
> I can't explain this or figure out how to debug it.  Since it's locked up
> right now, thought I'd ping you for ideas before resetting it.

Now that was actually before we migrated it back to local datastore,
but when we did, the problem remained, suggesting that whatever has
happened to the VM, it is contained within the VM's vmdk or other
files.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Eduardo Morras]

At 16:03 29/03/2012, you wrote:

Alright, new data. It happened to crash about 10 minutes after I came in
this morning and I ran some stuff in the DDB. I have no idea what
information is useful, but perhaps someone will see something out of the
ordinary?


http://feld.me/freebsd/esx_crash/


Don't know about ESXi but on others VM Managers i can change the 
chipset emulation from ICH10 to ICH4. Can you change it to an older 
chipset too?




Thanks...



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 09:58:16 -0500, Hans Petter Selasky   
wrote:



Do both 32- and 64-bit versions of FreeBSD crash?


Correct, we see both i386 and amd64 flavors crash in the same way.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> On 3/28/2012 1:59 PM, Mark Felder wrote:
> > FreeBSD 8-STABLE, 8.3, and 9.0 are untested
> 
> As much as I'm sensitive to your production requirements, realistically
> it's not likely that you'll get a helpful result without testing a newer
> version. 8.2 came out over a year ago, many many things have changed
> since then.
> 
> Doug

So you're saying that he should have been using 8.3-RELEASE, then.

If you'll kindly go over to http://www.freebsd.org and look under
"Latest Releases", please note that "8.2" is a production release.
If you don't want it to be a production release, then find a way
to make it so, but please don't snipe at people who are using the
code that the FreeBSD project has indicated is a current production
offering.

There are many good reasons not to run arbitrary snapshots on your
production gear.  It's unrealistic to expect people to run non-
RELEASE non-production code on their production gear.  We can have
that discussion if you don't understand that, drop me a note off-
list and I'll be happy to explain it.

Otherwise, you've told him to run a "newer version," of which NONE
IS AVAILABLE, unless you're thinking 9.0, but FreeBSD has a rather
catastrophic history of "point zero" releases, and most clueful
admins won't run those in production without carefully measuring
the risks and benefits.  So you've basically told him to run a
newer version without any such version being realistically 
available.

WTF? 

You want people not to use releases that "came out over a year 
ago"?  The generally sensible solution to that is to release 
RELEASEs more than once every fourteen or fifteen months.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Hans Petter Selasky]

On Thursday 29 March 2012 15:42:42 Joe Greco wrote:
> > Hi,

Do both 32- and 64-bit versions of FreeBSD crash?

--HPS
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Joe Greco]

> Hi,
> 
> * have you filed a PR?
> * is the crash easily reproducable?
> * are you able to boot some ramdisk-only FreeBSD-8.2 images (eg create
> a ramdisk image using nanobsd?) and do some stress testing inside
> that?
> 
> It sounds like you've established it's a storage issue, or at least
> interrupt handling for storage issue. So I'd definitely try the
> ramdisk-only boot and thrash it using lighttpd/httperf or something.
> If that survives fine, I'd look at trying to establish whether there's
> something wrong in the disk driver(s) freebsd is using. I'm not that
> cluey on ESXi, but there may be some PIC/APIC/ACPI change between 7.x
> and 8.0 which has caused this to surface.

We've seen this.  Or something that seems really like it.

We run dozens of FreeBSD VM's, many of which are 8.mumble.  We have a
scripted build environment dating back many years, so generally servers
come out in a fairly reproducible form.

After several months of smooth running, we had need to shuffle some
things around, and migrated some servers to a different datastore.
Suddenly, one particular VM, our corp Jabber server, started randomly 
disconnecting people every morning.  Some inspection showed that the
machine was running, but disk I/O in the VM was freezing up.  
Subsequent inspection suggested that it was happening during the 
periodic daily, though we never managed to get it to happen by manually 
forcing periodic daily, so that's only a theory.  Given that several 
times it appeared that one of the find commands was running, I was 
guessing that something in the thin provisioned disk image for the 
system had gone bad, but reading the entire disk with dd didn't cause 
a hang, running the periodic daily by hand didn't cause a hang, etc.

Migrating the VM to a different host and datastore did not fix the
issue.  Migrating the VM from an Opteron to a Xeon host with all the
latest ESXi 4 patches also didn't make any difference.  Migrating the
disk image from thin to full seemed to fix it, but I only gave it a
day or two, then decided there were other good reasons to reload the
VM, so I nuked the VM, which, of course, fixed it.

In the meantime, a dozen other similar VM's alongside it run just
fine.  My conclusion was that it was something specific that had gone
awry in the virtual machine, probably in the disk image, but I could
not identify it without significant digging that I had no particular
reason or inclination to do; since it appeared to be a VMware problem,
the "reload it and be done with it" seemed the quickest path to 
resolution.

That having been said, if anyone has any brilliant ideas about what 
would constitute useful further steps to isolate this, I can look at
recovering the faulty VM from backup and seeing if it still exhibits
the problem.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Alright, new data. It happened to crash about 10 minutes after I came in  
this morning and I ran some stuff in the DDB. I have no idea what  
information is useful, but perhaps someone will see something out of the  
ordinary?



http://feld.me/freebsd/esx_crash/


Thanks...
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Thank you for the suggestion. We'll put it in our toolbox and see if it  
helps!

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 02:36:49 -0500, Doug Barton  wrote:


As much as I'm sensitive to your production requirements, realistically
it's not likely that you'll get a helpful result without testing a newer
version. 8.2 came out over a year ago, many many things have changed
since then.


The sad part is that VMWare's "supported FreeBSD versions" are a joke, and  
we've been trying to keep VMWare happy by only running "supported  
versions". I honestly don't think they even test. It's so stupid.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Wed, 28 Mar 2012 18:31:38 -0500, Adrian Chadd   
wrote:



* have you filed a PR?


No


* is the crash easily reproducable?


Unfortunately not. It's totally random. Some servers will "get the bug"  
and crash daily, some will crash weekly, some might seem to be fine but 3  
months later hit this crash.



* are you able to boot some ramdisk-only FreeBSD-8.2 images (eg create
a ramdisk image using nanobsd?) and do some stress testing inside
that?


That's a plan I'd like to execute but my free time for building that  
environment is rather short at the moment :(



I'm not that
cluey on ESXi, but there may be some PIC/APIC/ACPI change between 7.x
and 8.0 which has caused this to surface.


Was there a setting to revert ACPI behavior from 8.x to 7.x? I thought I  
read about that at one point or perhaps this was something available  
back in the dev cycle when 8 was -CURRENT. *shrug* I know 9.0 and onward  
has even more ACPI changes so assuming it truly is an ACPI bug I guess we  
could cross our fingers and hope that the bug has mysteriously vanished?

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Michael Powell]

Mark Felder wrote:

> Alright guys, I'm at the end of my rope here. For those that haven't seen
> my previous emails here's the (not so) quick breakdown:
> 
> Overview:
> 
> FreeBSD ?? - 7.4 never crash
> FreeBSD 8.0 - 8.2 crashes
> FreeBSD 8-STABLE, 8.3, and 9.0 are untested (Sorry, not possible in our
> production at this time, and we were hoping we could base some stuff on
> 8.3 for long term stability...)
> ESXi: Confirmed ESXi 4.0 - 5.0 has this problem. Haven't tested on others.
> 
[snip]
> 
> 
> I think we've finally found enough data that this is definitely something
> in the FreeBSD world. I'm going to begin prepping some of the known crashy
> servers with more debugging. Any suggestions on what I should build the
> kernel with? They never do a proper panic, but I definitely want to at
> least *try* to get into the debugger the next time it crashes. And when it
> crashes, what the heck should I be running? I've never played with the KDB
> before...
> 
> 
> Thank you for any suggestions and help you can give me

I am definitely out of my league here and this is way over my head, to be 
sure. Just a couple of shots in the dark for possibly covering a couple more 
data points for your research. And I am a tad fuzzy on both as I have never 
needed to dig into either because I've not had any trouble with either.

IIRC there are three different timer subsystems one may choose from. You may 
want to look into expirementation with each of the three, just to see if 
this changes any observed behaviors. Or to possibly rule it out. 

Your situation sounds like a candidate for reverse logic - if I can't get 
any handle on what's wrong I start at the opposite end and try to make a 
list of what is "right" in an attempt to leave a smaller subset to probe.

I also think this most likely has nothing to do with what's happening, but 
for some reason it just pops into my head. Try disabling msi in 
/boot/loader.conf like this:

hw.pci.enable_msi="0"
hw.pci.enable_msix="0"

At least if it makes no difference maybe this will exclude it from being a 
'possible'. Developers who are more in-depth aware of what the differences 
are between 7.x and 8.x/9.x in the development timeline can probably provide 
a better picture so as to narrow the field of what to look at. This is way 
over my head, just wish I could help - I know and have experienced the kind 
of quandary you have here (I feel for you).   :-)

-Mike
 

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Doug Barton]

On 3/28/2012 1:59 PM, Mark Felder wrote:
> FreeBSD 8-STABLE, 8.3, and 9.0 are untested

As much as I'm sensitive to your production requirements, realistically
it's not likely that you'll get a helpful result without testing a newer
version. 8.2 came out over a year ago, many many things have changed
since then.

Doug
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Adrian Chadd]

Hi,

* have you filed a PR?
* is the crash easily reproducable?
* are you able to boot some ramdisk-only FreeBSD-8.2 images (eg create
a ramdisk image using nanobsd?) and do some stress testing inside
that?

It sounds like you've established it's a storage issue, or at least
interrupt handling for storage issue. So I'd definitely try the
ramdisk-only boot and thrash it using lighttpd/httperf or something.
If that survives fine, I'd look at trying to establish whether there's
something wrong in the disk driver(s) freebsd is using. I'm not that
cluey on ESXi, but there may be some PIC/APIC/ACPI change between 7.x
and 8.0 which has caused this to surface.

2c,


Adrian
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

Alright guys, I'm at the end of my rope here. For those that haven't seen  
my previous emails here's the (not so) quick breakdown:


Overview:

FreeBSD ?? - 7.4 never crash
FreeBSD 8.0 - 8.2 crashes
FreeBSD 8-STABLE, 8.3, and 9.0 are untested (Sorry, not possible in our  
production at this time, and we were hoping we could base some stuff on  
8.3 for long term stability...)

ESXi: Confirmed ESXi 4.0 - 5.0 has this problem. Haven't tested on others.


History:

Over the course of the last 2 years we've been banging our heads on the  
wall. VMWare is done debugging this. They claim it's not a VMWare issue.  
They can't identify what the heck happens. We had a glimmer of hope with  
ESXi 5.0 fixing it because we never saw any crashes in the handful of  
deployments, but our dreams were crushed today -- two days before an  
outage to begin migration to ESXi 5.0 -- when a customer's ESXi 5.0 server  
and FreeBSD 8.2 guest crashed.



Crash Details:

The keyboard/mouse usually stops responding for input on the console;  
normally we can't type in a username or password. However, we can switch  
VTs.


If there's a shell on the console and we can type, we can only run things  
in memory. Any time we try to access the disk it will hang indefinitely.


The server still has network access. We can ping it without issue. SSH of  
course kicks you out because it can't do any I/O.


If we were to serve a lightweight http server off a memory backed  
filesystem I'm confident it would run just fine as long as it wasn't  
logging or anything.


On ESXi you see that there is a CPU spike of 100% that goes on  
indefinitely. No idea what the FreeBSD OS itself thinks it is doing  
because we can't run top during the crash.


This crash can affect a server and happen multiple times a week. It can  
also not show up for 180 days or more. But it does happen. The server can  
be 100% idle and crash. We have servers that do more I/O than the ones  
that crash could ever attempt to do and these don't crash at all.  
Completely inexplicable.



Things we've looked into:

Nothing about the installed software matters. We've tried cross  
referencing the crashed servers by the programs they run but the base OS  
is the only common denominator due to the wide variety of servers it has  
affected.


Storage doesn't matter. We've tried different iSCSI SANs, we've tried  
different switches, we've tried local datastores on the ESXi servers  
themselves.


HP servers, Dell servers -- doesn't seem to matter either. (All with  
latest firmwares, BIOSes, etc)


VMWare gave us a ton of debugging tasks, and we've given them gigabytes of  
debugging info and data; they can't find anything.


VMWare tools -- with, without, using open-vm-tools makes no difference. I  
think we've done a fair job ruling out VMWare.



I think we've finally found enough data that this is definitely something  
in the FreeBSD world. I'm going to begin prepping some of the known crashy  
servers with more debugging. Any suggestions on what I should build the  
kernel with? They never do a proper panic, but I definitely want to at  
least *try* to get into the debugger the next time it crashes. And when it  
crashes, what the heck should I be running? I've never played with the KDB  
before...



Thank you for any suggestions and help you can give me
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Can someone please help me set up my tata-photon (huawei) modem ?

[Manish Jain]

   Hello Matthias,
   I got it to work !  I symlinked /usr/local/bin/tclsh8.2 as tclsh, but
   that was probably not the only factor involved. In a streak of
   unintended brilliance, I had set the device in ppp.conf as /dev/cuaU0,
   whereas there is no such device - as /var/log/ppp.log complained.
   (Thanks for your suggestion for looking up the log). I changed
   /dev/cuaU0 to /dev/cuaU0.0, as a matter of courtesy. Now my FreeBSD
   box is connected to the internet and I don't have to boot into XP
   again (as long as I can hope).
   Thanks again
   Manish Jain
   On 29-Jun-11 00:55, Manish Jain wrote:

   Hello Matthias,
   Yes, I do have /dev/cuaU0.0 along with corresponding lock and init
   files, although I cannot give you the output you want right now since
   I am booted into XP with the FreeBSD 8.2 dvd image being downloaded.
   (This will hopefully finish in about 15 hours, as cygwin::ncftpget
   suggests).
   I also tried a few other things, most notably installing
   usb_modeswitch, and putting this into /etc/devd.conf :
   attach 100 {
   device-name "ugen[0-9]+";
   match "vendor" "0x12d1";
   match "product" "0x1446";
   action "/usr/local/bin/usb_modeswitch";
   };
   Running kldload for u3g and ucom indicates they are both already
   loaded. Running usb_modeswitch seems to have also generated some
   /dev/ttyU. files, along with corresponding lock and init files -
   just like the cua entries.
   The only thing that seems out of place with the current setup is that
   usb_modeswitch's wrapper (usb_modeswitch_dispatcher) needs
   /usr/local/bin/tclsh, and for some reason the FreeBSD installer
   installed it as /usr/local/bin/tclsh8.2
   I'll try fixing the above tomorrow and will try out anything
   you/someone else might have to suggest. BTW, I made a slight error in
   my original message. My system's release version is 8.1, not 8.0.
   Thank you &
   Regards
   Manish Jain
   [1]invalid.poin...@gmail.com
   On 29-Jun-11 00:16, Matthias Apitz wrote:

El día Tuesday, June 28, 2011 a las 01:47:31PM +0530, Manish Jain escribió:

   Hi all,
   My tata-photon (huawei) modem works painlessly on Win XP. Every
   setting is auto (including getting the nameserver). The auth protocol
   is CHAP. But when I boot into FreeBSD 8.0 (stable), and run 'ppp -auto
   pmdemand' followed by 'ping [1]www.google.com' (or yahoo or any other
   site, for that matter), I always get "Hostname lookup failure". BTW, I
   don't have an /etc/resolv.conf. I only configured ppp.conf (please see
   below for details). Can someone please help me set up the modem so
   that I can dial out and connect to the internet from my FreeBSD box ?
   Thanks in advance.
   Manish Jain
   This is my ppp.conf :

...

Hello Manish,

Before digging into the ppp details (there is a log about in
/var/log/ppp.log), do you have after attaching the Huawei modem a
device /dev/cuaU0 at all? What does the following command gives:

ls -l /dev/cua*
chat -vs < /dev/cuaU0.0 > /dev/cuaU0.0 '' AT OK 'ATI2' O
printf "\n"

HIH

matthias

References

   1. mailto:invalid.poin...@gmail.com
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Can someone please help me set up my tata-photon (huawei) modem ?

[Manish Jain]

   Hello Matthias,
   Yes, I do have /dev/cuaU0.0 along with corresponding lock and init
   files, although I cannot give you the output you want right now since
   I am booted into XP with the FreeBSD 8.2 dvd image being downloaded.
   (This will hopefully finish in about 15 hours, as cygwin::ncftpget
   suggests).
   I also tried a few other things, most notably installing
   usb_modeswitch, and putting this into /etc/devd.conf :
   attach 100 {
   device-name "ugen[0-9]+";
   match "vendor" "0x12d1";
   match "product" "0x1446";
   action "/usr/local/bin/usb_modeswitch";
   };
   Running kldload for u3g and ucom indicates they are both already
   loaded. Running usb_modeswitch seems to have also generated some
   /dev/ttyU. files, along with corresponding lock and init files -
   just like the cua entries.
   The only thing that seems out of place with the current setup is that
   usb_modeswitch's wrapper (usb_modeswitch_dispatcher) needs
   /usr/local/bin/tclsh, and for some reason the FreeBSD installer
   installed it as /usr/local/bin/tclsh8.2
   I'll try fixing the above tomorrow and will try out anything
   you/someone else might have to suggest. BTW, I made a slight error in
   my original message. My system's release version is 8.1, not 8.0.
   Thank you &
   Regards
   Manish Jain
   [1]invalid.poin...@gmail.com
   On 29-Jun-11 00:16, Matthias Apitz wrote:

El día Tuesday, June 28, 2011 a las 01:47:31PM +0530, Manish Jain escribió:

   Hi all,
   My tata-photon (huawei) modem works painlessly on Win XP. Every
   setting is auto (including getting the nameserver). The auth protocol
   is CHAP. But when I boot into FreeBSD 8.0 (stable), and run 'ppp -auto
   pmdemand' followed by 'ping [1]www.google.com' (or yahoo or any other
   site, for that matter), I always get "Hostname lookup failure". BTW, I
   don't have an /etc/resolv.conf. I only configured ppp.conf (please see
   below for details). Can someone please help me set up the modem so
   that I can dial out and connect to the internet from my FreeBSD box ?
   Thanks in advance.
   Manish Jain
   This is my ppp.conf :

...

Hello Manish,

Before digging into the ppp details (there is a log about in
/var/log/ppp.log), do you have after attaching the Huawei modem a
device /dev/cuaU0 at all? What does the following command gives:

ls -l /dev/cua*
chat -vs < /dev/cuaU0.0 > /dev/cuaU0.0 '' AT OK 'ATI2' O
printf "\n"

HIH

matthias

References

   1. mailto:invalid.poin...@gmail.com
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Can someone please help me set up my tata-photon (huawei) modem ?

[Matthias Apitz]

El día Tuesday, June 28, 2011 a las 01:47:31PM +0530, Manish Jain escribió:

> 
>Hi all,
>My tata-photon (huawei) modem works painlessly on Win XP. Every
>setting is auto (including getting the nameserver). The auth protocol
>is CHAP. But when I boot into FreeBSD 8.0 (stable), and run 'ppp -auto
>pmdemand' followed by 'ping [1]www.google.com' (or yahoo or any other
>site, for that matter), I always get "Hostname lookup failure". BTW, I
>don't have an /etc/resolv.conf. I only configured ppp.conf (please see
>below for details). Can someone please help me set up the modem so
>that I can dial out and connect to the internet from my FreeBSD box ?
>Thanks in advance.
>Manish Jain
>This is my ppp.conf :
...

Hello Manish,

Before digging into the ppp details (there is a log about in
/var/log/ppp.log), do you have after attaching the Huawei modem a
device /dev/cuaU0 at all? What does the following command gives:

ls -l /dev/cua*
chat -vs < /dev/cuaU0.0 > /dev/cuaU0.0 '' AT OK 'ATI2' O
printf "\n"

HIH

matthias
-- 
Matthias Apitz
t +49-89-61308 351 - f +49-89-61308 399 - m +49-170-4527211
e  - w http://www.unixarea.de/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Can someone please help me set up my tata-photon (huawei) modem ?

[Manish Jain]

   Hi all,
   My tata-photon (huawei) modem works painlessly on Win XP. Every
   setting is auto (including getting the nameserver). The auth protocol
   is CHAP. But when I boot into FreeBSD 8.0 (stable), and run 'ppp -auto
   pmdemand' followed by 'ping [1]www.google.com' (or yahoo or any other
   site, for that matter), I always get "Hostname lookup failure". BTW, I
   don't have an /etc/resolv.conf. I only configured ppp.conf (please see
   below for details). Can someone please help me set up the modem so
   that I can dial out and connect to the internet from my FreeBSD box ?
   Thanks in advance.
   Manish Jain
   This is my ppp.conf :
   default:
set log Phase Chat LCP IPCP CCP tun command
allow users guru bourne
   # ident user-ppp VERSION (built COMPILATIONDATE)
set device /dev/cuaU0
set speed 115200
set dial "ABORT BUSY ABORT NO\\sCARRIER TIMEOUT 5 \
\"\" AT OK-AT-OK AT&FE0V1X1&D2&C1S0=0 OK \\dATDT\\T TIMEOUT 40
   CONNECT"
set timeout 180 # 3 minute idle timer (the default)
enable dns # request DNS info (for resolv.conf)
   pmdemand:
set phone "#777"
set login
set authname "internet"
set authkey ""
set timeout 0
disable ipv6cp
set ifaddr 10.0.0.1/0 10.0.0.2/0 255.255.255.0 0.0.0.0
add default HISADDR
accept CHAP

References

   1. http://www.google.com/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: NDIS failed under 8.0, please help

[Xihong Yin]

ub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
GEOM: ad0s4: geometry does not match label (255h,63s != 16h,63s).
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/ad0s4a
drm0:  on vgapci0
vgapci0: child drm0 requested pci_enable_busmaster
info: [drm] AGP at 0xc000 256MB
info: [drm] Initialized i915 1.6.0 20080730
drm0: [ITHREAD]
drm0: [ITHREAD]

Please help!

Thanks,
Xihong
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: NDIS failed under 8.0, please help

[Paul B Mahol]

Make use of google.

You need to create wlanX first.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

NDIS failed under 8.0, please help

[Xihong Yin]

: geometry does not match label (255h,63s != 16h,63s).
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/ad0s4a
drm0:  on vgapci0
vgapci0: child drm0 requested pci_enable_busmaster
info: [drm] AGP at 0xc000 256MB
info: [drm] Initialized i915 1.6.0 20080730
drm0: [ITHREAD]
drm0: [ITHREAD]

Please help!

Thanks,
Xihong
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Please help me

[Jonathan Chen]

On Wed, Jan 20, 2010 at 01:10:48PM +0630, komyo gyi wrote:
> hi,
> 
> >  yesterday i have modify squid.conf file.i have use vi editior.but i
> > cannot delete in text message. following error ^? appear.How to do it?.
> > please help me.

Looks like you're hitting the Delete key. That's not a valid vi
command. You need to be in command mode and hit the 'x' key.
-- 
Jonathan Chen 
--
The human mind ordinarily operates at only ten percent of its capacity
 -- the rest is overhead for the operating system.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Please help me

[komyo gyi]

hi,

>  yesterday i have modify squid.conf file.i have use vi editior.but i
> cannot delete in text message. following error ^? appear.How to do it?.
> please help me.
>
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Can't get wifi working in 8.0, please help.

[alex_p]

Hello!
Here's my working rc.conf:

wlans_ral0="wlan0"
create_args_wlan0="wlanmode hostap mode 11g"
ifconfig_wlan0="inet 192.168.0.1 netmask 0xff00 ssid btest channel
11"

or you can do it by hand:
# ifconfig wlan0 create wlandev ral0 wlanmode hostap
# ifconfig wlan0 inet 192.168.0.1 netmask 0xff00 ssid btest
channel 11



On 5 янв, 05:31, Eric Webster  wrote:
> Hello, I recently upgraded from bsd 6.4 to bsd8.0release ( new install ) and 
> I am having issues getting mywifitowork. Before the upgrade it worked 
> perfectly in 6.4.
> I am a bit confused as I have read different things about this. The 
> handbookhttp://www.freebsd.org/doc/en/books/handbook/network-wireless.html#NE...
> under basic settings says check to see if your device supports host based ap 
> mode by doing
>
> ifconfig wlan0 list caps
>
> # ifconfig wlan0 list caps
> drivercaps=2985cd01
>
> Then it says the wireless device can now be put into AP mode by doing:
>
> ifconfig wlan0 ssid freebsdap mode 11g mediaopt hostap inet 10.0.0.1 netmask 
> 255.255.255.0
>
> Which returns:
> ifconfig: inet: bad value
>
> I read some place that you need to put inet before ssid so I tried it with 
> the following:
>
> # ifconfig wlan0 inet 10.0.0.1/24 ssid gangsta wepmode on weptxkey 1 wepkey 
> apasswordhere mode 11g mediaopt hostap
>
> Which returns:
> ifconfig: SIOCSIFMEDIA (media): Device not configured
>
> If I try to configure it like it was in 6.4 I get another error:
>
> ifconfig wlan0 ssid gangsta channel 8 wepmode on weptxkey 1 wepkey 
> apasswordhere mode 11g mediaopt hostap 10.0.0.1 netmask 255.255.255.0
>
> Which returns:
> ifconfig: SIOCSIFMEDIA (media): Device not configured
>
> I know wep is not secure, I am just trying to get it working.
>
> At start of the guide it tells you to configure the following in 
> /boot/loader.conf which I did.
>
> /boot/loader.conf
>
> if_ral_load="YES"
> wlan_load="YES"
> wlan_scan_ap_load="YES"
> wlan_scan_sta_load="YES"
> wlan_wep_load="YES"
> wlan_ccmp_load="YES"
> wlan_Tkip_load="YES"
>
> When I run kldstat I see the if_ral is loaded. I don't know if its supposed 
> to show the other modules.
>
> kldstat
>
> Id Refs Address    Size Name
>  1    9 0xc040 b22548   kernel
>  2    1 0xc0f23000 13e4c    if_ral.ko
>  3    1 0xc357b000 35000    ipl.ko
>
> Here is rc.conf
>
> check_quotas="NO"
> gateway_enable="YES"
> hostname="router.foo.bar"
> ibcs2_enable="NO"
> ifconfig_sk0="DHCP"
> ifconfig_xl0="inet 192.168.0.1  netmask 255.255.255.0"
> wlans_ral0="wlan0"
> ifconfig_wlan0="inet 10.0.0.1/24 ssid gangsta wepmode on weptxkey 1 wepkey 
> apasswordhere mode 11g mediaopt hostap"
> ipfilter_enable="YES"
> ipfilter_rules="/etc/ipf.rules"
> ipmon_enable="YES"
> ipmon_flags="-Ds"
> ipnat_enable="YES"
> ipnat_rules="/etc/ipnat.rules"
>
> Here is ifconfig -a
>
> xl0: flags=8843 metric 0 mtu 1500
>     options=8
>     ether 00:60:97:7f:3e:6c
>     inet 192.168.0.1 netmask 0xff00 broadcast 192.168.0.255
>     media: Ethernet autoselect (100baseTX )
>     status: active
> sk0: flags=8843 metric 0 mtu 1500
>     options=b
>     ether 00:0c:41:e4:7e:83
>     inet x.x.x.x netmask 0xf800 broadcast 255.255.255.255
>     media: Ethernet autoselect (100baseTX )
>     status: active
> ral0: flags=8843 metric 0 mtu 2290
>     ether 00:14:bf:78:a2:a7
>     media: IEEE 802.11 Wireless Ethernet autoselect mode 11g
>     status: associated
> plip0: flags=8810 metric 0 mtu 1500
> lo0: flags=8049 metric 0 mtu 16384
>     options=3
>     inet 127.0.0.1 netmask 0xff00
> wlan0: flags=8843 metric 0 mtu 1500
>     ether 00:14:bf:78:a2:a7
>     media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
>     status: no carrier
>     ssid gangsta channel 11 (2462 Mhz 11g)
>     country US authmode OPEN privacy ON deftxkey 1 wepkey 1:104-bit
>     txpower 0 bmiss 7 scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250
>     roam:rssi 7 roam:rate 5 protmode CTS bintval 0
>
> netstat -rn
>
> Routing tables
>
> Internet:
> Destination    Gateway    Flags    Refs  Use  Netif Expire
> default    x.x.x.x    UGS 0   35    sk0
> x.x.x.x/21 link#2 U   0    0    sk0
> x.x.x.x    link#2 UHS 0    0    lo0
> 127.0.0.1  link#5 UH  0   32    lo0
> 192.168.0.0/24 link#1 U   1  568    xl0
> 192.168.0.1    link#1 UHS 0    0    lo0
>
> Many thanks in advance!!
>
> Eric
>
> ___freebsd-questi...@freebsd.org 
> mailing listhttp://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Can't get wifi working in 8.0, please help.

[Eric Webster]

Hello, I recently upgraded from bsd 6.4 to bsd 8.0 release ( new install ) and 
I am having issues getting my wifi to work. Before the upgrade it worked 
perfectly in 6.4. 
I am a bit confused as I have read different things about this. The handbook 
http://www.freebsd.org/doc/en/books/handbook/network-wireless.html#NETWORK-WIRELESS-AP-BASIC
under basic settings says check to see if your device supports host based ap 
mode by doing 

ifconfig wlan0 list caps

# ifconfig wlan0 list caps
drivercaps=2985cd01

Then it says the wireless device can now be put into AP mode by doing:

ifconfig wlan0 ssid freebsdap mode 11g mediaopt hostap inet 10.0.0.1 netmask 
255.255.255.0

Which returns:
ifconfig: inet: bad value

I read some place that you need to put inet before ssid so I tried it with the 
following:

# ifconfig wlan0 inet 10.0.0.1/24 ssid gangsta wepmode on weptxkey 1 wepkey 
apasswordhere mode 11g mediaopt hostap

Which returns:
ifconfig: SIOCSIFMEDIA (media): Device not configured

If I try to configure it like it was in 6.4 I get another error:

ifconfig wlan0 ssid gangsta channel 8 wepmode on weptxkey 1 wepkey 
apasswordhere mode 11g mediaopt hostap 10.0.0.1 netmask 255.255.255.0

Which returns:
ifconfig: SIOCSIFMEDIA (media): Device not configured

I know wep is not secure, I am just trying to get it working. 

At start of the guide it tells you to configure the following in 
/boot/loader.conf which I did.

/boot/loader.conf

if_ral_load="YES"
wlan_load="YES"
wlan_scan_ap_load="YES"
wlan_scan_sta_load="YES"
wlan_wep_load="YES"
wlan_ccmp_load="YES"
wlan_Tkip_load="YES"

When I run kldstat I see the if_ral is loaded. I don't know if its supposed to 
show the other modules.

kldstat

Id Refs Address    Size Name
 1    9 0xc040 b22548   kernel
 2    1 0xc0f23000 13e4c    if_ral.ko
 3    1 0xc357b000 35000    ipl.ko


Here is rc.conf


check_quotas="NO"
gateway_enable="YES"
hostname="router.foo.bar"
ibcs2_enable="NO"
ifconfig_sk0="DHCP"
ifconfig_xl0="inet 192.168.0.1  netmask 255.255.255.0"
wlans_ral0="wlan0"
ifconfig_wlan0="inet 10.0.0.1/24 ssid gangsta wepmode on weptxkey 1 wepkey 
apasswordhere mode 11g mediaopt hostap"
ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"
ipnat_enable="YES"
ipnat_rules="/etc/ipnat.rules"

Here is ifconfig -a

xl0: flags=8843 metric 0 mtu 1500
    options=8
    ether 00:60:97:7f:3e:6c
    inet 192.168.0.1 netmask 0xff00 broadcast 192.168.0.255
    media: Ethernet autoselect (100baseTX )
    status: active
sk0: flags=8843 metric 0 mtu 1500
    options=b
    ether 00:0c:41:e4:7e:83
    inet x.x.x.x netmask 0xf800 broadcast 255.255.255.255
    media: Ethernet autoselect (100baseTX )
    status: active
ral0: flags=8843 metric 0 mtu 2290
    ether 00:14:bf:78:a2:a7
    media: IEEE 802.11 Wireless Ethernet autoselect mode 11g
    status: associated
plip0: flags=8810 metric 0 mtu 1500
lo0: flags=8049 metric 0 mtu 16384
    options=3
    inet 127.0.0.1 netmask 0xff00 
wlan0: flags=8843 metric 0 mtu 1500
    ether 00:14:bf:78:a2:a7
    media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
    status: no carrier
    ssid gangsta channel 11 (2462 Mhz 11g)
    country US authmode OPEN privacy ON deftxkey 1 wepkey 1:104-bit
    txpower 0 bmiss 7 scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250
    roam:rssi 7 roam:rate 5 protmode CTS bintval 0

netstat -rn

Routing tables

Internet:
Destination    Gateway    Flags    Refs  Use  Netif Expire
default    x.x.x.x    UGS 0   35    sk0
x.x.x.x/21 link#2 U   0    0    sk0
x.x.x.x    link#2 UHS 0    0    lo0
127.0.0.1  link#5 UH  0   32    lo0
192.168.0.0/24 link#1 U   1  568    xl0
192.168.0.1    link#1 UHS 0    0    lo0


Many thanks in advance!! 

Eric






___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help me make sense of top's CPU output

[Chris Rees]

2009/11/3 Chris Stankevitz :
> Dan Nelson wrote:
>>
>> Junior Hacker Project: add an instantaneous-CPU value (calculated by
>> subtracting successive ki_runtime values) to the list of things top
>> calculates and toggle it and weighted-CPU when pressing C.  The toggling
>> code is already there; it just toggles between two different weighted-cpu
>> values at the moment.
>>
>
> Makes sense, thank you.  If I want to hack a port program, I go to the
> "work" directory, edit the source, and rebuild.  How do I hack a non-port
> program like top?
>
> Chris

Look in the Makefile for /usr/src/usr.bin/top, and you'll see the
source is in /usr/src/contrib/top

Hack away!

Chris
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in a mailing list?
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help me make sense of top's CPU output

[Chris Stankevitz]

Dan Nelson wrote:

Junior Hacker Project: add an instantaneous-CPU value (calculated by
subtracting successive ki_runtime values) to the list of things top
calculates and toggle it and weighted-CPU when pressing C.  The toggling
code is already there; it just toggles between two different weighted-cpu
values at the moment.



Makes sense, thank you.  If I want to hack a port program, I go to the 
"work" directory, edit the source, and rebuild.  How do I hack a 
non-port program like top?


Chris
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help me make sense of top's CPU output

[Dan Nelson]

In the last episode (Nov 02), Chris Stankevitz said:
> I recently performed a CPU intensive task with Xorg.  When I completed the
> task and Xorg no longer was using the CPU, I got this result from top:
> 
> ===
> 
> last pid:  1201;  load averages:  0.24,  0.10,  0.09up 0+00:29:42
> 63 processes:  1 running, 62 sleeping
> CPU:  1.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.0% idle
> Mem: 161M Active, 67M Inact, 68M Wired, 1240K Cache, 41M Buf, 1676M Free
> Swap: 4060M Total, 4060M Free
> 
>   PID USERNAME THR PRI NICE   SIZERES STATE  C   TIMECPU
>  1017 cstankevitz1 1040   366M   331M select 0   3:25 35.89% Xorg

The CPU column in the process list is a decaying average (more useful to the
kernel scheduler than an instantaneous value).  You'll see it slowly drop to
0 over 10-15 seconds.

Junior Hacker Project: add an instantaneous-CPU value (calculated by
subtracting successive ki_runtime values) to the list of things top
calculates and toggle it and weighted-CPU when pressing C.  The toggling
code is already there; it just toggles between two different weighted-cpu
values at the moment.

-- 
Dan Nelson
dnel...@allantgroup.com
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

please help me make sense of top's CPU output

[Chris Stankevitz]

Hello,

I recently performed a CPU intensive task with Xorg.  When I completed the task 
and Xorg no longer was using the CPU, I got this result from top:

===

last pid:  1201;  load averages:  0.24,  0.10,  0.09up 0+00:29:42
63 processes:  1 running, 62 sleeping
CPU:  1.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.0% idle
Mem: 161M Active, 67M Inact, 68M Wired, 1240K Cache, 41M Buf, 1676M Free
Swap: 4060M Total, 4060M Free

  PID USERNAME THR PRI NICE   SIZERES STATE  C   TIMECPU
 1017 cstankevitz1 1040   366M   331M select 0   3:25 35.89% Xorg

===

Note that the "CPU" row reports 99% idle.
Note that the "CPU" column reports 36% Xorg

I have two questions:

1. Why do these two numbers seem to not agree?  One reports the CPU is not 
being used, the other reports Xorg is using the CPU.

2. How can I change my system so that these two numbers seem to agree?

Thank you,

Chris

PS: conky does the same thing -- I assume this means the seemingly disagreeing 
numbers are coming from the FreeBSD kernel.





  
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Please help me win Ipod nano :)

[gosha-necr]

Hello friends! I want to ask you for help in winning ipod nano, if you register 
on biggest social network here: http://vkontakte.ru/reg632660 i'm can win :) 
Thank you! :)
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Some Questions About Freebsd Please Help Me !!

[xn.bravo]

Hi , 

i want to ask some questions about freebsd , one of my friend have freebsd 
in his server , he is using it , he have ips issu on his server and he is 
converting ips in proxies ( Socks 4/5 ) , i want to know how i can do that , 
how i can set firewall that or what i need to do? , like let me show you what 
he have did , he have give me SSH access , in that when i go i need to put 
commands , i will give you some commands , to make ip as proxy i need to give 
this command in ( Putty ) socks -d -p14344 -i204.18.245.9 -e204.18.245.9 , it 
will convert ip in proxy , but i dun know how to do that in freebsd , i will 
show you 1 screenshot as well ,  here is screenshot  ( 
http://i36.tinypic.com/wuoro6.png ) , you can have a look on that as well , 
please help me if you can , like this i am going to buy may be 20 freebsd for 
that , i want to know how can i set all this in that , if you can help me in 
that , please send me steps how i can make ip in proxy with the help of freebsd.

Here is Some More Commands.

To Stop Socks Here is Command : killall -9 socks

To Start Socks Here is Command : socks -d -p14344 -i204.18.245.9 -e204.18.245.9

Regrads

Bravo
italy
00393888992300
   

Alice Messenger ;-) chatti anche con gli amici di Windows Live Messenger e 
tutti i telefonini TIM!
Vai su http://maileservizi.alice.it/alice_messenger/index.html?pmk=footer
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Ian Smith]

On Mon, 17 Aug 2009 17:23:29 -0700 Walt Pawley 
 > At 4:44 PM +0200 8/17/09, Heiner Strauß wrote:
[..]
 > >Putting the symbol names in one word helped the linker / loader a lot.
 > >Live was so easy.
 > >
 > >Heiner
 > >
 > >C(one word = 32 bit) .NOT. (some word processor software)
 > 
 > As something of an ancient curmudgeon these days, I've enjoyed
 > this discussion. As speculation on my part, perhaps the six
 > character limitation is less a software issue than an early
 > architecture issue - DEC's PDP-6/10 design used 36-bit words
 > and packed six characters (clearly from a limited subset of the
 > then current ASCII) per word, making simple searches very
 > effective through symbol tables with a simple word level
 > compare loop.

Can I play in the ancient curmudgeonly nostalgia reunion too?

 > While likely not all that closely related to the issue, I
 > recall a technique I was introduced to on Control Data systems
 > called COSY, in which one punched binary coded Hollerith cards
 > with two characters per column encoded (six bits per
 > character). Of course, such cards required excellent handling
 > equipment (which Control Data had) because a stack of cards
 > punched with 960 holes in each one had lots of opportunity for
 > hanging chads.

First real systems programming job was converting $multinat's data files 
from NCR 315 format (12-bit 'slabs' holding 2 6-bit alphanum upper-case 
characters or 3 4-bit BCD numbers, on 7 track tape and some paper tape) 
to IBM 360 format (8 bit EBCDIC chars or BCD numerics, on 9 track tape), 
which only took about 4 months, replacing a whole floor & tons of gear.

The NCR was also clearly designed around 80-column punch cards; 2 alphas 
or 3 digits or one 12-bit instruction code per column.  The programmer's 
art was judged (by peers, not management :) on what your best single 
card 80-slab program could do once booted .. test runs of which involved 
turning up at the end of The Operator's shift and likely offering some 
$inducement, after conning one of the punch girls into typing 160 chars 
of utter gibberish for no apparent reason ..



cheers, Ian___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Polytropon]

On Wed, 19 Aug 2009 12:22:11 +0200, Heiner Strauß  wrote:
> Didn't need lower case at this time. REAL PROGRAMMERS USED FORTRAN
> 
> http://www.pbm.com/~lindahl/real.programmers.html

When you're there, don't miss "The story about Mel". By the
way... we have a Mel on our mailing list... :-)

http://www.pbm.com/~lindahl/mel.html

Little history lesson from far away:

In approx. 1950, the IBM N.O.R.C. processed numerical values
only. There were no plans to make the printer print characters
because "no need for this" was seen. Even the need for a
programming language like FORTRAN wasn't seen. :-)





-- 
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Heiner Strauß]

Am Mittwoch, den 19.08.2009, 07:59 + schrieb
freebsd-questions-requ...@freebsd.org:
> On Tue, Aug 18, 2009 at 01:45:27PM -0400, Karl Vogel wrote:
> 
> > >> On Mon, 17 Aug 2009 17:23:29 -0700, 
> > >> Walt Pawley  said:
> > 
> > W> As speculation on my part, perhaps the six character limitation
> is less
> > W> a software issue than an early architecture issue - DEC's
> PDP-6/10
> > W> design used 36-bit words and packed six characters (clearly from
> a
> > W> limited subset of the then current ASCII) per word, making simple
> > W> searches very effective through symbol tables with a simple word
> level
> > W> compare loop.
> > 
> >I'll second that.  My first job for Uncle Sugar was on a DEC
> 10/55
> >for the Air Force, and 36-bit words were a fact of life.  There
> were
> >lots of programs around for conversion to/from 32-bit words, just
> so
> >we could talk to everybody else on Earth.
> 
> CDC (Control Data) mainframe machines used 6 bit characters.
> I believe the 3600 series had 36 bit words.
> The 6000 series (6400, 6500, etc, plus 170/750) used 60 bit words
> but still used 6 bit characters.  So, everything was all upper case.
> It had 12 bit 'peripheral processors' which tended the 60 bit main 
> processor[s] so later started to use 12 bit characters or sometimes 8 
> in 12 to allow for upper/lower case.   That was a Seymour Cray thing.
> He designed their early mainframes before he bolted to make his
> own companies (so he wouldn't have to conform to corporate control).

And I always thought it was 14 bit with 7 bit characters, perhaps this
is why my outputs looked strange :) This was the last model I've used:

http://www.cray-cyber.org/systems/cy960.php

> Later CDC came out with their 180 series that used 64 bit words
> and 8 bit bytes. It was kind of a nice system but it was too late for 
> them.  The world was turning to clusters of cheap CPU chips running
> UNIX
> instead of massive mainframes running proprietary OSen and CDC didn't 
> jump on that bandwagon soon or strongly or cheaply enough.
> 
> Anyway, in those earliest of days, 6 bits was the economical character
> set.  But it was an obstacle to upper/lower case characters without
> using some shift code.   IBM and DEC started doing 8 bit bytes - I
> don't 
> know just when - and that allowed eash use of upper/lower characters
> and
> so quickly determined the standard character size for a long time.

Didn't need lower case at this time. REAL PROGRAMMERS USED FORTRAN

http://www.pbm.com/~lindahl/real.programmers.html

The problem was, the programmers packed the string into integer arrays.
2 characters in 1 integer saved a lot of space, but the VAX didn't like
this style.

>   Now 
> that 8 bit byte is a thorn in the side of those who want to create
> and 
> universalize a character set that is international.  
> 
> jerry
> 

Wasn't it just 3 or 4 releases ago FreeBSD went 8 bit clean ?



___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Jerry McAllister]

On Tue, Aug 18, 2009 at 01:45:27PM -0400, Karl Vogel wrote:

> >> On Mon, 17 Aug 2009 17:23:29 -0700, 
> >> Walt Pawley  said:
> 
> W> As speculation on my part, perhaps the six character limitation is less
> W> a software issue than an early architecture issue - DEC's PDP-6/10
> W> design used 36-bit words and packed six characters (clearly from a
> W> limited subset of the then current ASCII) per word, making simple
> W> searches very effective through symbol tables with a simple word level
> W> compare loop.
> 
>I'll second that.  My first job for Uncle Sugar was on a DEC 10/55
>for the Air Force, and 36-bit words were a fact of life.  There were
>lots of programs around for conversion to/from 32-bit words, just so
>we could talk to everybody else on Earth.

CDC (Control Data) mainframe machines used 6 bit characters.
I believe the 3600 series had 36 bit words.
The 6000 series (6400, 6500, etc, plus 170/750) used 60 bit words
but still used 6 bit characters.  So, everything was all upper case.
It had 12 bit 'peripheral processors' which tended the 60 bit main 
processor[s] so later started to use 12 bit characters or sometimes 8 
in 12 to allow for upper/lower case.   That was a Seymour Cray thing.
He designed their early mainframes before he bolted to make his
own companies (so he wouldn't have to conform to corporate control).
Later CDC came out with their 180 series that used 64 bit words
and 8 bit bytes. It was kind of a nice system but it was too late for 
them.  The world was turning to clusters of cheap CPU chips running UNIX
instead of massive mainframes running proprietary OSen and CDC didn't 
jump on that bandwagon soon or strongly or cheaply enough.

Anyway, in those earliest of days, 6 bits was the economical character
set.  But it was an obstacle to upper/lower case characters without
using some shift code.   IBM and DEC started doing 8 bit bytes - I don't 
know just when - and that allowed eash use of upper/lower characters and
so quickly determined the standard character size for a long time.  Now 
that 8 bit byte is a thorn in the side of those who want to create and 
universalize a character set that is international.  

jerry

> 
> -- 
> Karl Vogel  I don't speak for the USAF or my company
> 
> Men are liars.  We'll lie about lying if we have to.  I'm an
> algebra liar.  I figure two good lies make a positive.  --Tim Allen
> ___
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Karl Vogel]

>> On Mon, 17 Aug 2009 17:23:29 -0700, 
>> Walt Pawley  said:

W> As speculation on my part, perhaps the six character limitation is less
W> a software issue than an early architecture issue - DEC's PDP-6/10
W> design used 36-bit words and packed six characters (clearly from a
W> limited subset of the then current ASCII) per word, making simple
W> searches very effective through symbol tables with a simple word level
W> compare loop.

   I'll second that.  My first job for Uncle Sugar was on a DEC 10/55
   for the Air Force, and 36-bit words were a fact of life.  There were
   lots of programs around for conversion to/from 32-bit words, just so
   we could talk to everybody else on Earth.

-- 
Karl Vogel  I don't speak for the USAF or my company

Men are liars.  We'll lie about lying if we have to.  I'm an
algebra liar.  I figure two good lies make a positive.  --Tim Allen
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Walt Pawley]

At 4:44 PM +0200 8/17/09, Heiner Strauß wrote:
>> On Mon, Aug 17, 2009 at 06:18:45PM +0800, Erich Dollansky wrote:
>> > On 17 August 2009 pm 18:09:06 cpghost wrote:
>> > > On Mon, Aug 17, 2009 at 10:25:29AM +0200, Polytropon wrote:
>> > > > By the way, where did I read that #define macro names have to
>> > > > be unique within the first 6 (six) letters? :-)
>> > >
>> > > The 6 letters limit was actually a restriction of earlier
>> > > linkers and it affected all identifiers of linkable objects
>> >
>> > I did not know that linkers resolved macros those days.
>>
>> Of course they didn't. But knowing that linkers restricted the
>> identifiers' length to 6 chars, it made sense for preprocessors
>> to restrict them as well before passing them to the compiler
>> and linker.
>>
>> Actually, it's a bit more complicated than that, but the basic
>> restriction came from the linkers, the preprocessors only inherited
>> it.
>>
>> > Interesting.
>> >
>> > Erich
>>
>> Regards,
>> -cpghost.
>
>Putting the symbol names in one word helped the linker / loader a lot.
>Live was so easy.
>
>Heiner
>
>C(one word = 32 bit) .NOT. (some word processor software)

As something of an ancient curmudgeon these days, I've enjoyed
this discussion. As speculation on my part, perhaps the six
character limitation is less a software issue than an early
architecture issue - DEC's PDP-6/10 design used 36-bit words
and packed six characters (clearly from a limited subset of the
then current ASCII) per word, making simple searches very
effective through symbol tables with a simple word level
compare loop.

While likely not all that closely related to the issue, I
recall a technique I was introduced to on Control Data systems
called COSY, in which one punched binary coded Hollerith cards
with two characters per column encoded (six bits per
character). Of course, such cards required excellent handling
equipment (which Control Data had) because a stack of cards
punched with 960 holes in each one had lots of opportunity for
hanging chads.
--

Walter M. Pawley 
Wump Research & Company
676 River Bend Road, Roseburg, OR 97471
 541-672-8975
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: please help to uninstall FreeBSD!!!

[Heiner Strauß]

> On Mon, Aug 17, 2009 at 06:18:45PM +0800, Erich Dollansky wrote:
> > On 17 August 2009 pm 18:09:06 cpghost wrote:
> > > On Mon, Aug 17, 2009 at 10:25:29AM +0200, Polytropon wrote:
> > > > By the way, where did I read that #define macro names have to
> > > > be unique within the first 6 (six) letters? :-)
> > >
> > > The 6 letters limit was actually a restriction of earlier
> > > linkers and it affected all identifiers of linkable objects
> > 
> > I did not know that linkers resolved macros those days.
> 
> Of course they didn't. But knowing that linkers restricted the
> identifiers' length to 6 chars, it made sense for preprocessors
> to restrict them as well before passing them to the compiler
> and linker.
> 
> Actually, it's a bit more complicated than that, but the basic
> restriction came from the linkers, the preprocessors only inherited
> it.
> 
> > Interesting.
> > 
> > Erich
> 
> Regards,
> -cpghost.

Putting the symbol names in one word helped the linker / loader a lot.
Live was so easy.

Heiner

C(one word = 32 bit) .NOT. (some word processor software)


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"