[Savannah-hackers-public] Re: [EMAIL PROTECTED] senddigests cron job

[Bob Proulx]

Karl Berry wrote:
> (Bob, if you have any quick thoughts, please pass along ...)

Hmm...  This seems to have been happening for some time.  Google found
me this message in the archive from last September referencing gnu.org
lists with this problem then.

  http://mail.python.org/pipermail/mailman-users/2005-September/046460.html

A patch is suggested in the thread.  But also an additional problem
possibly being a bad date string in a message that is being digested.

Bob

[Savannah-hackers-public] Re: reliable, incremental git->cvs ?

[Bob Proulx]

Sylvain Beucler wrote:
> I haven't setup an auto-packing commit hook so may need to have me do
> that. I'm planning to add something like in post-update:
> 
>   export GIT_DIR=coreutils.git
>   git-count-objects
>   # If > 5120k
>   git repack
>   git prune

A useful reference for update scripts is the update-hook-example.txt
example shipped with git.  Mostly it covers access control.  In
Debian's git-doc package this is normally installed in
/usr/share/doc/git-doc/howto/update-hook-example.txt.  There is a lot
of room for policy differences but that implements a nice access
policy.

Something that Carl set up for us is a cron task that does a full
repack (e.g. git-repack -q -a -d) periodically.  Also along with that
the cron task also touches up permissions to add extra protection to
the tags directory.  Here are some ideas.

  find $repodir/refs/tags -type d ! -perm -=t -print0 | xargs -r0 chmod +t
  find $repodir/refs/tags -type f ! -perm +a=w -print0 | xargs -r0 chmod a-w
  find $repodir/objects -type d ! -perm -=t -print0 | xargs -r0 chmod +t
  find $repodir/objects -type f ! -perm +a=w -print0 | xargs -r0 chmod a-w

Bob

[Savannah-hackers-public] Re: [Savannah-cvs] [ListHelperAntiSpam] (edit) wiki markup cleanup

[Bob Proulx]

Sylvain Beucler wrote:
> http://savannah.gnu.org/maintenance/ListHelperAntiSpam
> Thanks for the mark-up clean-up Bob :)

I just learned of that wiki yesterday and noticed that page on
listhelper there.  Seems like a good place for more information.  I
should add some more information in that section.  But doing a little
format cleanup there was all I had time for at that moment.

Bob

Re: [Savannah-hackers-public] Re: savannah shell access?

[Bob Proulx]

Jim Meyering wrote:
> I'll be happy to move things to
>   /var/lib/git/git-to-cvs
> once things are settled -- or wherever else seems appropriate.

I am thinking about the problem now and trying to decide whether
/var/cache/git-to-cvs is more appropirate than /var/lib/git-to-cvs.
Using /var/cache just "feels" like the better location to me.

  http://www.pathname.com/fhs/pub/fhs-2.3.html#VARCACHEAPPLICATIONCACHEDATA
  http://www.pathname.com/fhs/pub/fhs-2.3.html#VARLIBVARIABLESTATEINFORMATION

I would be cautious about using small, short, generic names like "git"
because of possible name collisions as the name here in case the git
project itself eventually starts to use a /var/ location and creates a
collision.  A top level project name unique to this task should
probably be used instead.

And what if this were expanded to be offerred to other version control
systems (without starting discussion -- such as 'arch' or something)
then would this cache be useful there too?  In which case I am
thinking that perhaps /var/cache/git-srv or something slightly longer
and more reliably avoiding name collisions might be good.

But since things can be modified in the event of actual found problems
I don't feel strongly about this.

> One small measure that may help avoid casual or inadvertent
> interference: leave each working directory "chmod 0", and have
> the git update hook do "chmod 755 ...", do its job, then "chmod 0"
> to re-protect it.

I think this is not so good.  I understand how it would make it more
difficult to have accidental crosstalk between projects happen by
making the windows of vulnerability smaller, like a time-lock safe.
But I think that if there is a concern about interference that the
process either should stand on its own or not.  Withing being able to
completely verbalize why, I think that doing this type of door opening
and closing just "feels" like less than a good thing to do.

Bob

Re: [Savannah-hackers-public] Re: locate on sv.gnu.org?

[Bob Proulx]

Jim Meyering wrote:
> [EMAIL PROTECTED] (Karl Berry) wrote:
> > [using locate] doesn't happen very often.  I've maybe done it half
> > a dozen times
> > since I got access.  I'll miss it on the occasions when it's needed, but
> > if no one else cares, never mind.
> 
> I too find locate very useful, and used it just a day or two
> ago when looking for git-cvsexportcommit.

I think this is already close to resolution but I thought I would pipe
in that I often use 'locate' on systems and if I were working on a
system without it I would often find that I missed it.

Bob

Re: [Savannah-hackers-public] Savannah backup exclusion

[Bob Proulx]

Sylvain Beucler wrote:
> apt-proxy is pretty buggy

Agreed.  I wanted to mention that it was replaced by 'apt-cacher',
which worked great in Sarge but in Etch seems to be suffering from
featuritis, but is still better than apt-proxy.  I am told that many
people are having good luck using 'approx' but I have yet to try it.

> so I'm trying to use a Debian Etch binary-only mirror instead, which
> takes around 15GB. It is in /vservers/internal/var/www/ , so you can
> exclude it from the backup.

Of course using a mirror gives the best performance and I always set
up a mirror when desiring better reliability.

Bob

[Savannah-hackers-public] Setting default_member_moderation to yes for all lists?

[Bob Proulx]

Recently an address at yahoo.com subscribed to a large number of
mailing lists and then sent spam messages to them.  Because the
address was subscribed to the mailing list Mailman was configured to
send the messages along without delay and without filtering.

I would like to set default_member_moderation to yes on all of the
mailing lists.  That setting sets the hold for moderation bit for new
subscribers until it is cleared manually.  The effect is that someone
can subscribe but their first posting will be held for human review
the same as a non-subscriber.  Once they have posted a message and a
human listhelper has reviewed it they would clear the moderate bit for
that address.  Subsequent mail messages from them would be passed
through without delay and without filtering the same as it is done
now.

On many of the mailing lists the default_member_moderation is already
set to yes to protect against this problem.  I always set it on the
mailing lists that I watch to closely.  But so far it is far from
globally implemented.

By doing this initial hold it would avoid the types of spam attacks
that we are vulnerable to at this moment.  I fear that because of the
popularity of Mailman more attackers might automate this process and
slip more spam through the system.

I think we should ensure that default_member_moderation is set on
every list.  What do you think?

Bob

Re: [Savannah-hackers-public] new mailing list blurb

[Bob Proulx]

Karl Berry wrote:
> Finally, here is my proposed text to install in about_list_creation.txt
> about changes to the default mailing list configuration.  Final
> suggestions, comments on either the text or the changes?

Looks good to me.

> No footer is added to messages by default.

Apologies for not keeping up with the past discussion and I hate to
revisit an already discussed topic but the footer is going away by
default?  That surprised me since it seems to be the only defense
against the endless people sending unsubscribe messages to the mailing
lists.  Of course they seem to do that anyway and so it probably
doesn't matter.  And it does tidy up the look of the messages.

Bob

Re: [Savannah-hackers-public] new mailing list blurb

[Bob Proulx]

Sylvain Beucler wrote:
> - We at Savannah represent a strong stand against services that keep
>   their source code private (such as SourceForge, of course). Hence,
>   we need a permanent location where the listhelper source code can be
>   downloaded, if possible with instructions on how to set it up at
>   other places. If the license is AGPL, all the better :)

The custom source is licensed under the GPL and is free software.  The
main part of listhelper is a shell script and a procmail file plus a
few other associated files and crontabs.  The message catagorization
is done by SpamAssassin.  There are some custom spam filter rules
based upon 'grep' and 'md5sum'.  Some Ruby is used for mime component
handling.  Listhelper is an aggregation of available technologies.
All of the projects used are available under a free software license.

Most of the contribution is really the server cpu time it takes to
process messages plus the babysitting needed to manage it because the
nature of spam is that the environment is always in motion around it.
I frequently write custom spam filter rules that are good for about
two weeks before they become stale.  A snapshot of the custom part of
the spam rules this winter would be quite stale this next spring.

The babysitting of the hold queues by the listhelper volunteers is a
critical element.  Without the human review the system would not work.
The listhelper scripts enable human review of the mailing list hold
queues more efficiently than was previously possible.

A problem with distributing listhelper code is one of preparing it for
generic distribution such that it would work in a generic environment.
I wrote it in situ specifically for the purpose for which it is
working now and it would take some effort to make it transportable.
Also the GNU Mail system was announced which if completed would make
listhelper obsolete.  Listhelper was written to bridge the gap until
then.  Therefore there wasn't a lot of reason to do this in
anticipation of it being replaced.

Bob

Re: [Savannah-hackers-public] new mailing list blurb

[Bob Proulx]

Sylvain Beucler wrote:
> Another thing we may want to mention is that list maintainers
> (including [EMAIL PROTECTED]) get one Mailman "request for approval"
> per incoming mail - something that users would usually rather get in
> daily batches.

That statement confuses me.  To me it implies that an owner, because
only the mailing list owners and moderators get the moderator
messages, would be *reading* the mailing list traffic through the
moderator messages as opposed to *reviewing* the mailing list traffic
through the moderator messages.  Is that what you meant?

I review the mailing list traffic with the moderator messages (using
the listhelper scripts) but I _read_ the lists by being subscribed
normally.  Otherwise replies would get strange and things like that.

If someone is inclined to read them in daily batches (meaning to me
digest format) then they probably would not want to be seeing the
moderator messages at all.  They should probably be encouraged more
strongly to filter them out.  It would be nicer of Mailman if it did
not send moderator messages to the mailing list owner.

Bob

Re: [Savannah-hackers-public] new mailing list blurb

[Bob Proulx]

Karl Berry wrote:
> http://tug.org/~karl/alc.html is my expanded text to be shown in the
> list creation page.  More comments?

>> By default, Mailman sends request for  notification on every message,
>> and this is critical for listhelper's operation. You will probably
>> want to filter these from your inbox, with a pattern such as this:
>> ^Subject: confirm [a-f0-9]{40}

That pattern occurs in the body of the message.  Specifically in a
mime attachment in the body of the message.  But I would guess that
upon reading it most people would assume that was the subject of the
message.  The Subject: is a red herring.  It probably should have a
note saying that it is a body rule and not a header rule.

But many message handling programs are more efficient at scanning just
the message headers.  The following may be a better subject match
because it scans the header.  I use:

  ^Subject: [-a-zA-Z0-9]+ post from

But this next is perhaps a little tighter and less prone to false hits.

  ^Subject: [^[:space:]]* post from [^[:space:]]* requires approval

Unfortunately I think that message is one that gets localized
depending upon language settings.  I also have this in one of my
rules.

  ^Subject: El envio a [-a-zA-Z0-9]+ de.*precisa.*de.*aprobacion

Bob

Re: [Savannah-hackers-public] new mailing list blurb

[Bob Proulx]

Sylvain Beucler wrote:
> Since listhelper rely on this 'admin_immed_notify' parameter being
> turn on, list admins need to be careful not to turn it off. We can
> suggest that list admins, if they don't want the "requests for
> approval" mails, use a filter rule in their MUA instead.

Thanks.  That clears things up for me.

Bob

Re: [Savannah-hackers-public] help-grub and bug-grub

[Bob Proulx]

Yoshinori K. Okuji wrote:
> Robert Millan wrote:
> > With [EMAIL PROTECTED] we didn't have any problem with spam, since it's
> > subscriber-only.  If help-grub is made subscriber-only as well, we can
> > avoid the spam too.

It would certainly be inconvenient to users if they needed to
subscribe in order to post bug reports.

> I think, bug-* and help-* have been traditionally open to
> non-subscribers.

You are correct.  Traditionally those have been open to
non-subscribers.  This makes it easy for users to submit bug reports.
Submitting bug reports has always been encouraged.  The boilerplate
documentation asks users to submit bug reports to the bug-*
addresses.

I think if a user needed to subscribe the number of users that would
actually subscribe to send in bug reports would be dramatically
reduced.  I doubt if *I* would subscribe to post a bug report.

> As far as I see, Information For Maintainers of GNU Software does
> not mention any rule about how open contacts should be. So I guess
> it is acceptable to make the list writable only for subscribers,
> although it would be surely inconvenient for the user who just wants
> to ask one question.

I think it would be unfortunate if bug-grub were different from the
other bug-* mailing lists and required subscription to post.  I
strongly recommend going with the flow and doing the same thing that
the other bug-* mailing lists are doing.

Bob

Re: [Savannah-hackers-public] help-grub and bug-grub

[Bob Proulx]

Robert Millan wrote:
> Which is that?  Either someone actively moderates the list, or we accept spam
> as a part of it.

We moderate non-subscriber email.  Messages from subscribed users are
passed through normally without delay.  Here is a quick overview:

  https://savannah.gnu.org/maintenance/ListHelperAntiSpam

Bob

Re: [Savannah-hackers-public] help-grub and bug-grub

[Bob Proulx]

Robert Millan wrote:
> Karl Berry wrote:
> > We moderate non-subscriber email.
> > 
> > That is, "we", meaning a small group of volunteers of whom Bob and I are
> > two, are willing to moderate non-subscriber email to any GNU lists for
> > which there is no one else willing.  (We already handle dozens.)  If you
> > want us to help with help-grub, just say the word and I will add
> > [EMAIL PROTECTED] as a moderator.
> 
> Hi Karl,
> 
> I do really appreciate your efforts, although I still think it's a bad
> approach at dealing with spam.  May I suggest you accept mail from non
> subscribers, but only when it comes in an envelope with traceable domain
> name (and that domain isn't RHBLed)?

That would simply be great!  Actually a number of people have
suggested some really good things that would help reduce spam.  I have
been doing something similar to what you describe on my site and I
agree it can really help.

Unfortunately we are constrained by the current abilities of the
Mailman implementation as it exists on gnu.org.  To the best of my
knowledge Mailman on gnu.org does not have any of this capability.  I
would be extremely happy if you were to educate me otherwise.  We have
been trying to make the best of things.  It is a compromise at best.

> I've been doing that for years, and it's very reliable.  Of course, there's
> yet the people who insist on their envelopes being untraceable, but they
> aren't many and it's up to you how to handle them.

I don't think anyone is in disagreement with you.  That is not the
issue here.  The only problem is actually doing it since we are
constrained by the current mailing list manager implementation of
Mailman on gnu.org.

Bob

Re: [Savannah-hackers-public] Re: GNU mailing lists prefixes

[Bob Proulx]

Karl Berry wrote:
> I remember seeing a document that recommended using help- info- and
> [EMAIL PROTECTED] for GNU packages mailing lists. I can't find it
> anymore (neither 'maintain' nor 'standards'), did that changed?
> 
> No, it didn't change.  The bug- thing in the Mail node of maintain.texi.
> We should add help- there as well.
> 
> The other place you've seen it discussed is rms' dubbing message.  I had
> him change that to recommend the circumlotion of creating PKG-bugs in
> savannah and making bug-PKG an alias, purely because they can make that
> list on their own, but they can't make bug-PKG.  It would be great if
> people could make bug-PKG in savannah.

Let me also vote to make it possible for users to create their own
bug-PKG automatically.  I think the critical mass of convention is
behind bug-PKG and help-PKG and so it would be great if that simply
happened by default.  If there had been a consistent form from the
start I don't think it would matter which way that it was but now that
it has been bug-PKG for so long that is the form I would promote.

Bob

> File: maintain.info,  Node: Mail,  Next: Old Versions,  Prev: Platforms,  Up: 
> Top
> 
> 7 Dealing With Mail
> ***
> 
> Once a program is in use, you will get bug reports for it.  Most GNU
> programs have their own special lists for sending bug reports.  The
> advertised bug-reporting email address should always be
> [EMAIL PROTECTED]', to help show users that the program is a GNU
> package, but it is ok to set up that list to forward to another site
> for further forwarding.  The package distribution should state the name
> of the bug-reporting list in a prominent place, and ask users to help
> us by reporting bugs there.

[Savannah-hackers-public] Savannah Bug Spam

[Bob Proulx]

I saw these but there must be many others.  It is online pharmacy spam
with links in the signature.

  http://savannah.gnu.org/bugs/?15280
  http://savannah.gnu.org/bugs/?16179
  http://savannah.gnu.org/bugs/?1212<-- already marked as spam

Can someone find all submissions from user 66134 and clean out the spam?

Thanks
Bob

Re: [Savannah-hackers-public] Re: CVS server clogged up

[Bob Proulx]

Steve White wrote:
> Karl Berry wrote:
> > FWIW, I have had similar problems with CVS servers (including but not
> >  limited to savannah) in the past, even though the server is running
> >  fine.  It was due to network hobgoblins between me and the server.
>
> I don't deny that explanation, but it needs confirmation.  Otherwise,
> it would sound "convenient" for the sysadmin.

I have also experienced asymmetrical connection issues with systems on
the network in general and not necessarily the gnu.org machines in
particular.  Network issues can be very frustrating.  In my case I
have logins on a few other machines and can test connectivity from
different backbone locations to help isolate the problem.  It always
"feels" strange to be able to ping a machine from one location but not
another but it does make sense under some failure conditions.

I suggest that you try 'ping' when it is working and get a feel for
the normal latency between you and Savannah.  Then when you are
experiencing the problem run 'ping' again.  One potential problem is
that it is connected but the network latency between will be too high
to be practically functional for use with cvs over ssh.  This could be
confirmed by the data from ping.

Secondly I would try a snapshot of the public anonymous read-only cvs
repository.  You can prepare it at any time and leave it around ready
for testing when you need it.  When you are experiencing this cvs
problem jump over to the read-only pserver cvs repository and test
whether it is functional.  Since it uses a different service protocol
it might provide a different data point.

> A day down is very inconvenient to a busy developer.

This is a blatant plug for the distributed version control systems
such as Git.  The Internet is a wonderful magical machine that
connects the world together[*].  But it is susceptible to the
practical problems of WANs across geographically separated sites.  By
using a distributed VCS most of the problems can be ameliorated.  The
number of projects using Git on Savannah has been growing at a rapid
rate and can be viewed here:

  http://git.sv.gnu.org/gitweb/

Bob

-- 
[*] ... One Ring to bring them all and in the darkness bind them.

Re: [Savannah-hackers-public] rules for email

[Bob Proulx]

nick alvaro wrote:
> I got a bounce for a e-mail that I sent to this list.
> Are there any rules for e-mailing to savannah-hackers?

Show us the bounce?  Posting it to a pastebin and posting the URL to
it would be best.

> The title of the bounce was "The results of your email commands."

That sounds like your To: address went to a robot -request address
instead of to the mailing list itself.

Bob

Re: [Savannah-hackers-public] Mailing list posting size limits

[Bob Proulx]

Joshua 'jag' Ginsberg wrote:
> We've had another incident in the wee-hours of the night (eastern US)
> where a mailing list administrator disabled maximum message size limits
> on their list and a user posted a 4MB email to their list, generating
> about 1.2GB of traffic. The GNU mail system does not have infinite
> bandwidth, and posting such a message degrades quality-of-service to
> unacceptably low levels and is an unreasonable use of limited resources.

I have been routinely rejecting large messages with an explanation
message for any that I have noticed are large.  In the reject message
I ask them to either compress the attachment or to post it to one of
the many free pastebins that are available and then post a URL to it.
But it has been quite easy to miss large messages since the Mailman
web interface doesn't display the message size on the same page as the
message details.

> We have set on the MTA a hard limit on postings to lists.gnu.org.
> Messages with attachments greater than 200KB (before encoding to a 7bit
> representation) will be rejected.

Just to clarify the original poster will get an MTA rejection as
feedback?  I am imagining that it simply says something simply such as
"message too big" or some such.  Was there or is there a URL that we
could point them to with hints and instructions on how to make a large
data file available to a mailing list?

> We feel this strikes an appropriate balance between the utility of
> sending large attachments and the need to provide a reasonable
> quality of service for all GNU mail traffic.

I am certainly in agreement.

Bob

Re: [Savannah-hackers-public] spam posting replies

[Bob Proulx]

Karl Berry wrote:
> Presumably user "gronholm" is a fake.  When this happens, is there some
> way to disable the user from posting again?  Or is that already done?
> Or is it useless?  (The posting is already marked as spam.)

Apparently not.  Here is some previous discussion.

  https://savannah.gnu.org/support/?106304

> I can't say I've noticed the same user posting spam multiple times, but
> I haven't exactly paid strict attention either.

There definitely has been problems with repeat offenses from the same
user account.

> Is there some way we can include a CAPTCHA or something as part of
> registration?  The continued posting of spam replies is a drag.  (But
> requiring people to be group members to post would be even worse, so ...)

There is already a weak captcha.  But I don't think that even with a
strong captcha that it would really be improved.  Other sites with
very strong captchas are finding it insufficient.

I think the best that we can do is to create a way to block accounts
that are found to be offending and to react with removal of the spam
when spam is posted.

Bob

Re: [Savannah-hackers-public] [gnu.org #365733] Helping Improve Savannah Volunteer

[Bob Proulx]

Karl Berry wrote:
> http://savannah.gnu.org/maintenance/SavaneTasks
> ...
> Working on spam would be great.
> 
> Yes
> 
> The after-the-fact approach we have now is a big drag, because spam
> often gets sent to the project mailing lists as soon as it is posted.
> 
> With spam, it seems to me that making harder captchas is one very useful
> thing to do.  That cuts off the spammers from being able to post at all.

I also think it would be useful to be able to clean up the bugs after
spam has arrived.  Right now once spam is posted it can never be
removed.  It can only be collapsed into a small font tag if five
people vote that it is spam.  But it never goes away.  I would like to
see it possible to remove it forever from the database.

Bob

Re: [Savannah-hackers-public] [gnu.org #365733] Helping Improve Savannah Volunteer

[Bob Proulx]

Karl Berry wrote:
> Right now once spam is posted it can never be removed.  It can only
> be collapsed into a small font tag if five people vote that it is
> spam.
> 
> BTW, isn't that an awfully high barrier?  I don't know of any other
> system that requires more than one admin to look at it and say it's
> junk.  I can understand how one random savannah user shouldn't be able
> to declare any random post is spam, but if the user is a project or
> savannah admin, how about taking their word as gospel?

I think that if you are voting on a spam in a project for which you
are listed as admin then you get five votes.  So for your own projects
you can vote it to be spam yourself.  But for other projects where you
are not listed as an admin then you only get one vote.  But even for
your own projects the spam isn't really removed.

What I would like to see is something like what the Debian folks have
which is to flag a bug as having spam in it and have a moderator team
work through those and do the cleanup.  That seems to work fairly
well.

> Anyway, Andrew, as you can see, lots for you to contemplate :).

Yes.  Definitely a lot to think about! :-)

Bob

Re: [Savannah-hackers-public] Two question which I have to pass on

[Bob Proulx]

Sylvain Beucler wrote:
> Sebastian Gerhardt wrote:
> > a maintainer asked me about the reason we give the adive to write
> > explicit copyright dates--not ranges. Why is it better to do this?
> 
> That's what the FSF lawyers recommend
> http://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html
> 
> "Do not abbreviate the year list using a range; for instance, do not
>  write '1996--1998'; instead, write '1996, 1997, 1998'."
> 
> The document doesn't justify this recommendation.  Maybe Karl knows,
> otherwise we can ask licens...@gnu.org, otherwise we can assume that,

As I recall from previous discussion this is so that searching for a
specific year can locate the document more easily.  For instance in
the range 2005-2009 searching for "2007" won't locate that document.
Specifying the list of dates facilitates this part from the
Copyright-Notices document:

  Don't delete old year numbers, though; they are significant since they
  indicate when older versions might theoretically go into the public
  domain, if the movie companies don't continue buying laws to further
  extend copyright.  If you copy a file into the package from some other
  program, keep the copyright years that come with the file.

The Copyright-Notices document also says:

  Sometimes a program has an overall copyright notice that refers to the
  whole program.  It might be in the README file, or it might be
  displayed when the program starts up.  This copyright notice should
  mention the year of completion of the most recent major version; it
  can mention years of completion of previous major versions, but that
  is optional.

Therefore an overall date such as in --version output may use just the
latest year and does not need to display the long list of dates that
are needed in a file's copyright notice.

Of course many of these rules and guidelines are very general best
practices and date from before projects *always* had version control
meta-information available for them.  (All projects today use a
version control system, right? :-) Some of these I am certain are
there to benefit a project that exists as distribution only without
any way to look at the version control history.  And having had to
deal with lawyers on these matters I can say from first hand
experience that most of them do not understand the concept of version
control systems and so it is much better to have a clean paper trail
plainly visible in the files and not need to fall back to using the
meta-information of a version control system.  So I think these are
still good ideas today too.

Bob

Re: [Savannah-hackers-public] Re: ssh logins to lists.gnu.org

[Bob Proulx]

Ward Vandewege wrote:
> If the inconvenience is simply having to jump through a machine to get to
> lists, you could use a .ssh/config stanza like this to automate it:
> 
>  Host lists
>ProxyCommand ssh acco...@bastion.host -C $SSH_PROXY_FLAGS nc -w60 
> lists.gnu.org 22

I have often had trouble with 'nc' not detecting connections closures
(apparently by design) and remaining around indefinitely and needing
to have the processes cleaned up by other means.  Because of this I
found nc to be less than great for this purpose.  Which was unfortunate.

I recommend using the 'connect' command instead.  It was designed
specificially for the purpose of use with ssh's ProxyCommand but is
also a useful general purpose tool.  Using connect the line would be:

  ProxyCommand ssh -qq acco...@bastion.host connect %h %p

For me -qq worked best because the outermost ssh would report the
error appropriately.  Having the subprocess ssh report errors too
created less helpful output.  YMMV.

The connect program is in the "connect-proxy" package.

However jumping through a secondary machine requires either forwarding
your ssh agent through the intermediate machine or typing in your
password for each connection.  Many people dislike forwarding their
agent onto secondary machines.  Requiring that the password be typed
in again and again isn't good for security either.

On the original topic, I also think that it isn't a problem to have
ssh access to lists from anywhere.  If the noise in the logfiles annoy
you then using 'fail2ban' works well to curb it.  But I am confident
that with reasonable passwords that brute force attacks cannot
succeed.  Restricting login to ssh rsa keys only would prevent any
worry at all that a brute force attacks and would be better than IP
restrictions.

Bob

Re: [Savannah-hackers-public] Two question which I have to pass on

[Bob Proulx]

Noah Slater wrote:
> On Thu, Jan 22, 2009 at 02:23:27PM -0700, Bob Proulx wrote:
> >   Don't delete old year numbers, though; they are significant since they

Just to be pedantic, I didn't write that but was only quoting it from
the Copyright-Notices file that we were discussing.
  http://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html

> >   indicate when older versions might theoretically go into the public
> >   domain, if the movie companies don't continue buying laws to further
> >   extend copyright.  If you copy a file into the package from some other
> >   program, keep the copyright years that come with the file.
> 
> I don't buy this argument.
> 
> If a software package has a range of years that apply to all files, then from 
> a
> legal standpoint, the current year is the only one you can take into account.
> You could not take one file from the current release GNU Emacs and claim that 
> it
> had gone into the public domain because the first copyright year is listed as
> being over X years ago.
> 
> The older years are of no use because they apply to older versions of the same
> software. Older versions that are not included in the current release, by
> definition. There is no part of that software package that could be used with
> one of the older copyright years.
> ...

I think the clue here is this part "indicate when older versions might
theoretically go into the public domain".  It doesn't indicate that
the current version of the file goes into the public domain.  It only
gives a clue that one could look into the history of the file and find
an older version that might possibly go into the public domain.

> At some point in the future, I have downloaded the most recent release of GNU
> Emacs and I notice that the first year in the 17 line list of years (this is a
> serious issue!) is actually outside of the current copyright limit. So what 
> do I
> do? I certainly can't use any of the software in the current release as if it
> was public domain. Nope, I have to go through the release archives, guessing
> which one might be from that year.  Eventually after some trial an error I 
> will
> find a package which lists that year as the most recent year and I can safely
> assume that this package is now in the public domain.

As I understand it yes it would go something like that but perhaps
without all of the guessing about which year is which year.  Aren't
the old distributions of software projects usually organized a little
better than that?  But in any case as I read it these are only a hint
providing a clear paper trail for legal purposes.

> If you only ever used the most recent year, what difference would that make? I
> would have to do the very same thing! I might wonder to my self if any GNU 
> Emacs
> package is in the public domain, and then I have to perform a manual search 
> as I
> find the actual release that I can use. This is exactly the same process!
> 
> So, not only do I see the current recommendation as buying us nothing, it has
> the very strange side-affect of producing copyright statements that could 
> easily
> take up an entire terminal screen-full. A huge 17 line list of years!

The full license is already quite long.  Incrementally increasing it
in the areas that we see the full list of dates doesn't seem too much
of a hardship.  The long list of dates isn't required to be seen by
the user (e.g. in such locations such as --version output).  So this
shouldn't be a problem of consuming too much screen real estate.  Sure
when looking at a source file we need to page past the file's license
statement.  But we already have to page through the file's license
statement.

Perhaps you could provide an example where the full list
of dates creates a problem?  I am having a hard time thinking of one
and could benefit from the example.  Hoping for a URL into a version
control repository.  Something like: 
http://git.savannah.gnu.org/gitweb/?p=emacs.git;a=blob;f=src/alloc.c;h=1be5e2b8f1d789c0c94f035aa41a679302be5f2d;hb=HEAD

I am not emotionally attached to this issue but curiousity has me
discussing it.  For a change in the wording of the lawyer's guidelines
I am certainly not the person to convince.

Bob

Re: [Savannah-hackers-public] Side comments

[Bob Proulx]

Karl Berry wrote:
> However, I just now tried to open https://savannah.gnu.org in the
> firefox3 from CentOS, and, it seems that the cacert root is in ff3 after
> all!  I was not aware of that.  So -- yay!  Everything seems good.

I just tried this in a freshly installed Firefox 3 from mozilla.org
upstream and it didn't include the cacert root certificate.  (The
cacert root may have been installed by your distro or you may have
taken it yourself like I have done.)  Since it isn't in the upstream
and a lot of people install from the upstream I think that a lot of
users will still be seeing the untrusted cert dialog in FF3.  Darn.  I
was excited that it was going to be in there.

Bob

[Savannah-hackers-public] Recent spam to mailing list.

[Bob Proulx]

The recent spam to the mailing list was due to a subscribed address.
It appears that at some time previously subscribed members were
automatically whitelisted and unmoderated.  An old address came back
from the past to haunt us.

To clear this problem and also avoid more future problems of the same
type I have reset everyone's moderate bit for the mailing list.  This
will result in a small increase in delays as we revalidate postings.
But everything should return to normal very quickly.

Bob

Re: [Savannah-hackers-public] Savannah bzr server errors out

[Bob Proulx]

Bake Timmons wrote:
> Thanks to all--I've learned from you all.  I will try again next month
> when I get more dialup hours.  In the long run, I would hope for either
> being able to resume the download of a bazaar pack or for smaller
> maximum pack sizes, say 30MB.

Here is an idea: Perhaps you could do a bzr pull onto a different
internet connected host with a faster network connection such as
fencepost or some other host.  Then use 'rsync --partial' to transfer
that copy to your system.

This would seem to have the advantage of being able to stop and
restart the rsync along the way as needed without losing the data that
has already been transferred.  You can restart it as needed and it
should eventually be able to transfer the files.  After finishing the
rsync then you should have a result that is the same as a bzr pull.
However I am not an expert on bzr and use it only sporadically.

Bob

Re: [Savannah-hackers-public] Getting an error when trying to make changes on a private branch

[Bob Proulx]

Nikolas Kallis wrote:
> I am performing the Coreutils Contribution Guidelines:
> http://git.savannah.gnu.org/cgit/coreutils.git/tree/HACKING

I will go ahead and try to answer your question but let's take any
future discussion over to the coreutils list please.

> I am up to 'Make your changes on a private "topic" branch - Now,
> modify some file and commit it:' (line 129).
> 
> I modified the file 'DIFF' and executed the command 'git commit
> DIFF', and got the message "error: pathspec 'DIFF' did not match any
> file(s) known to git.".
> 
> What's up with that?

You didn't modify an existing file.  You created a new file.  I am
sure the file you created was from line 77:

git format-patch --stdout -1 > DIFF

That is for creating a patch for submission by email.  You would not
want to commit that file to version control.  The contents are already
in version control.

Let's say you were going to modify the sort.c file.  You would make
your modifications to that file.  You would then commit that file.
That file is already in version control.  Therefore to commit the
changes to that file by the instructions on line 129 you would say:

  git commit sort.c

Since it is already in version control this action will succeed.  If
you are creating a new file then you would need to 'git add' the file
first.

You may want to start with one of the many git tutorials on the web.
There are several available.  By looking at other tutorials you may
find that one covers things better for you than another.

I would start here:

  http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html

I also found this reference of references a useful list of references.

  http://sixrevisions.com/resources/git-tutorials-beginners/

Bob

Re: [Savannah-hackers-public] Problem with Coreutils Contribution Guidelines

[Bob Proulx]

Nikolas Kallis wrote:
> There is a problem with the Coreutils Contribution Guidelines:
> 
> http://git.savannah.gnu.org/cgit/coreutils.git/tree/HACKING

This would be a great discussion to have on the coreutils project
discussion list over at coreut...@gnu.org instead of here.  This is
the savannah hackers mailing list for people who hack on Savannah.
Savannah is the software forge site for free software projects.

I will try to give you an answer here anyway...  But let's take any
future discussion over to the coreutils list.

> The instructions should specify the command needs to be executed
> from the coreutils/ directory, or perhaps its a better idea to make
> the document specify to enter the coreutils/ directory first, as I
> suspect most - if not all commands in the instructions should
> generally be executed inside the coreutils/ directory.
> This will help put the instructions in line with implementation.

If you send this feedback to the coreutils project I am sure they will
greatfully appreciate it.

Bob

Re: [Savannah-hackers-public] how to build libntlm

[Bob Proulx]

Bill Liu wrote:
> To whom it may be concerned:

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

> Hello. I am newbie to libntlm. I recently download the libntlm by:
> git clone git://git.savannah.nongnu.org/libntlm.git

But you are asking about the libntlm project.  We don't know too much
about specific projects here.  It would be better to look at the
project README file and see what it says about where to get help.

For libntlm it says:

  If you need help to use Libntlm, or wish to help others, you are
  invited to join our mailing list libntlm at nongnu.org, see
  .

> I'm using Ubuntu 12.04. After "cd libntlm", I could NOT build it
> because there is no "configure" file to run. If I run "make" directly,
> it would fail.
> Where is the "configure" file?
> 
> Please help!

If you are starting from source code then you probably need to create
it.  Make sure you have all of the proper autotools installed and then
run:

  autoreconf --install

That will run the various autotools and create the configure file.
For any more assistance please contact the project team because
different projects will have variations on the theme.

Good luck!
Bob

Re: [Savannah-hackers-public] making sure I don't damage a git repo

[Bob Proulx]

Aharon Robbins wrote:
> I have copies of those branches on my personal system and want to push
> them back up.
> ...
> I did some experimenting on some test repos, and I believe that all I
> need to do is
> 
>   git push origin num-handler # and likewise for long-double
> 
> and then things will be back in sync.

I think so too.  Seems correct to me.  Go for it!

[Sometimes I am slightly confused by the various syntax due to the
available git syntax shortcuts and like you have done I like to test
it out with a local scratch repository to verify it before doing it to
a public remote.  Definitely a good idea.]

> My question is - what mechanisms are in place to recover the main repo
> as it is now, in case I screw something up?

AFAIK there are two.  1. Things are backed up.  If we need to recover
we can request a backup restore from sysadmin.  This is the blanket
site protection against catastrophe.  However also AFAIK that is only
available as a sysadmin function.  We would put in a sysadmin request.

2. More nicely to us because we have direct access is that if you have
your own backup then the Savannah hackers may restore directly from
your backup copy.  This is what we would normally do.

Since every full git repository is a full backup the operation for you
as the maintainer is safe as long as you have your own backup of the
repository with all of the branches that need to be backed up.  Since
as you see you can always push the copy back up to the repository
using git itself for the operation.  In this way it is operated as a
distributed backup system.

Additionally a Savannah hacker could make an individual backup as a
special request if you felt that was necessary.  In this case though
as long as you yourself have a good full git repository then we would
more easily be able to use it to restore through the git push
interface.  If for whatever reason there was something preventing it
(hooks preventing the push?) then we would deal with it directly.

So I say, "Go for it!"

Bob

Re: [Savannah-hackers-public] memory leak or bad uasge ?

[Bob Proulx]

Noam weissman wrote:
> I am running a simple TCP terminal server that works like a telnet
> server but with no Protocol. Actually a s simple TCP server. I use
> raw TCP terminal.

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

> It works ok but as far as I was able to spot I have memory leaks inside
> the LWIP stack.

But you are asking about the LWIP project.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

Looking at the Savannah web page here I see pointers to both the
lwip-users and lwip-devel mailing lists.

  https://savannah.nongnu.org/mail/?group=lwip

I suggest starting off by sending your request to the lwip-users
mailing list as a general good place to start.

Good luck!
Bob

> My server accepts a pBuf and process it.
> 
> The transferred data in that pBuf is simple text lines. My server reads
> char by char and when it finds CR it 
> 
> copies this string to a command processor (text parser). This text
> parser reads the text and if necessary 
> sends some data back to the client terminal.
> 
> If I pass a large file larger than the command processor queue I need to
> stall the pBuf processing until some space
> 
> Is freed in the command processor. If all is OK I do not free the
> accepted pBuf until It is processed.
> 
> That means the original pBuf is not immediately freed and at the same
> time the command processor sends  data to
> 
> The client TCP terminal. The pBuf is eventually freed after all the data
> is processed or if there is an error etc...
> 
> All is well so far.
> 
> If I disconnect the terminal (connected to my server) either in the
> middle of the file transfer or even if I wait to the file
> 
> End I can do that about more than 15-20 times. I have added error
> handling, polling, abort code etc... 
> 
> Nothing helped and after 20 or something iteration the TCP stoped
> working.
> 
> Although the Stack should free all pBuf's I added code to free the
> received pBuf in case of abort. I also added
> 
> Freeing the recived pBuf in all the cases (abort, close or error).
> 
> Adding that extra freeing stopped the server from being completely un
> responsive but it is not able to send large amount 
> Of data.
> 
> All the above suggests that memory handling is not working properly
> inside LwIP and therefore it is leaking.
> 
> Normally we work with an older stable version of LWIP (1.32 + FreeRTOS) 
> 
> I have tried using the latest version 1.41, I even added a few patches
> and installed the latest FreeRTOS but it behaves the same.
> 
> I feel that something basic is not working properly inside LwIP.
> 
> The code is running on STR91 with 96K RAM 
> 
> I have attached my TCP server source code + lwipopts.h
> 
> Any input will help.
> 
> Thanks,
> Noam.

Re: [Savannah-hackers-public] Answer to a task in order to join Savannah Administration

[Bob Proulx]

Hi Adrián,

Adrián wrote:
> following the indications published here[1], I chose the following task:

Thank you for picking up the task.

> task #12149: please delete master branch for FreeON project

  https://savannah.gnu.org/task/?12149
  I ran 'git filter-branch' because I grafted that branch onto previous
  history I converted from CVS. This changed all SHA commit tags, and
  makes it impossible for me to push my master to savannah. Please
  delete master so that I can push it.

I cannot tell from what is said here but I think that maybe more
history was added to the version history.  Usually when filter-branch
is used it is used to delete history.  But it is powerful and can do
many things.

Please ask for more details.  Was more history added?  Or was history
deleted?  Basically really try to understand what happened that is
causing a non-fast-forward.

I would ask them to upload the proposed master to a proposed-master
branch and then compare the history of the two branches.  Then you can
really know what they are wanting to do.  If they are fixing,
repairing, improving, the project history then I would help them do
it.  If they are deleting history then I would get a second opinion.

[Early in project's conversion there are often some missteps in
getting the git repository converted.  It sometimes happens that it
takes a few attempts before the right recipe is found.  If it is an
improvement to the conversion then I would definitely go ahead and
upload an improved master history.  I did this with another project
not too long ago.  If however history is being deleted then that would
need more thinking.  Because we don't condone deleting history as a
general statement.]

> (Note I chose this old task mainly because the others are quite
> similar).
>
> My answer will be:
> 
> ***
> 
> Hello Nico,
> 
> I am writing you concerning the Savannah administration task#12149 (I am
> helping with the tasks there).

Very nice.

> You have the required git instructions available here:
> 
> http://savannah.gnu.org/maintenance/UsingGit
> 
> -
> *Removing a branch*
> 
> We don't really like this, but there's a way to remove remote
> branches. We don't like it because this allow project members to
> potentially remove free software from Savannah, willingly or by
> mistake. This is mitigated by the fact it is very easy to push back
> changes.
> 
> When removing a branch, only its reference is removed, and the commit
> are still reachable by their identifiers. However, Savannah may prune
> unreachable commits (git gc), so don't count on this.
> 
> To remove a branch:
> 
> git push origin :mybranch

As you respond, we don't really like that.  See the recent note about
an accidental deletion in another project.  I imagine that if we had
the technology written already that we would disallow this activity
for all branches.  It just happens to be allowed by default.

In any case this happens to work for any branch that is not "master".
But the master branch is special in that it is the default current
branch and it isn't possible to remove the master branch.  Therefore
the above instructions won't actually work when we are talking about
the master branch.  The master branch cannot be deleted.

If this action is to be done (dependent upon learning more about the
actual happening of the version history) then have them upload a
proposed-master branch.  Then as a savannah hacker rename the
branches.  I would move master to master-$DATE and move
proposed-master to master.  (If master-$DATE is really just trash then
I think it is okay to delete it.  But of course once it is no longer
the master branch the it can be deleted with the above.)

> git push origin :master (this will delete the master branch remotely)

That will fail because master is the current branch.  It will produce
a message like this one.  (I created a test repo in order to capture
the error.)

  remote: error: refusing to update checked out branch: refs/heads/master
  remote: error: By default, updating the current branch in a non-bare 
repository
  remote: error: is denied, because it will make the index and work tree 
inconsistent
  remote: error: with what you pushed, and will require 'git reset --hard' to 
match
  remote: error: the work tree to HEAD.
  remote: error: 
  remote: error: You can set 'receive.denyCurrentBranch' configuration variable 
to
  remote: error: 'ignore' or 'warn' in the remote repository to allow pushing 
into
  remote: error: its current branch; however, this is not recommended unless you
  remote: error: arranged to update its work tree to match what you pushed in 
some
  remote: error: other way.
  remote: error: 
  remote: error: To squelch this message and still keep the default behaviour, 
set
  remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
  To ...
   ! [remote rejected] master (branch is currently checked out)
  error: failed to push some re

Re: [Savannah-hackers-public] git accumulations on vcs

[Bob Proulx]

James Cloos wrote:
> KB> exec timeout 120m /usr/lib/git-core/git-daemon "$@"
> ...
> Anyway, from git/daemon.c:run_service():
> 
> /*
>  * We'll ignore SIGTERM from now on, we have a
>  * good client.
>  */
> signal(SIGTERM, SIG_IGN);
> 
> so that is why timeout(1) fails.

Good debugging!

> exec timeout --signal=SIGKILL 120m /usr/lib/git-core/git-daemon "$@"
> 
> or:
> 
> exec timeout --signal=SIGRTMIN --kill-after=240m 120m 
> /usr/lib/git-core/git-daemon "$@"

For what it is worth I would vote for the longer timeout.  It is still
within the current day.  The problem isn't processes stuck hanging
around from breakfast or lunch but processes stuck hanging around from
the last New Year's party.  Either will do that.  And if someone is
running a slow download which is still active then it would be a
tragedy to kill them too soon.

Bob

Re: [Savannah-hackers-public] git accumulations on vcs

[Bob Proulx]

Karl Berry wrote:
> I forgot to mention it, but I increased the timeout to eight hours
> while I was there.

Good.

> Maybe it should be even more, but I'm doubtful that anything that hasn't
> completed in that long will ever finish.  Our repositories aren't that
> big.

Unrelated but related anecdote.  I did a bzr checkout of emacs source
some time ago and it timed out on me before the download completed.  I
had to take some special actions to get it to complete for me.  Some
projects are rather large.

In a perfect world it would be great to have two timers.  One looking
at how long it was I/O idle without making any data movement progress.
That is the stuck case that we would like to kill off.  It is holding
a process slot and is dead.  Detecting idle I/O timer would be an
interesting tool.  Then another timer looking at the overall total
runtime.  That would catch an infinite loop dumping data that will
never finish.  But the world isn't perfect. :-)

Bob

[Savannah-hackers-public] frontend /etc/init.d/sql_injection_monitor errors

[Bob Proulx]

While cleaning up munin I noticed that there are errors from insserv.

  insserv: Script sql_injection_monitor is broken: incomplete LSB comment.
  insserv: missing `Required-Start:' entry: please add even if empty.
  insserv: missing `Required-Stop:'  entry: please add even if empty.
  insserv: missing `Default-Stop:'   entry: please add even if empty.

Looking I see that the file is very sparse.  Too sparse.

  #!/bin/bash
  ### BEGIN INIT INFO
  # Provides:  sql_injection_monitor
  # Default-Start: 2 3 4 5
  #1;2403;0c Default-Stop:  0 1 6
  ### END INIT INFO
  /root/bin/sql_injection_monitor &

That really isn't a proper init.d script.  And it will start a new
copy of it during shutdown too.

Initially to stop the errors I am going to change that to this
following partial skeleton-copy rewrite.  I have inspected it closely
but I don't have an easy way to test this.

Bob

#!/bin/sh
### BEGIN INIT INFO
# Provides:  sql_injection_monitor
# Required-Start:$network $remote_fs $syslog
# Required-Stop: $network $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: SQL Injection Monitor
# Description:   SQL Injection Monitor
### END INIT INFO

# Author: Unknown but likely Michael J. Flickinger or Sylvain Beucler.
# Updated: Bob Proulx.

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="SQL Injection Monitor"
NAME=sql_injection_monitor
SCRIPTNAME=/etc/init.d/$NAME

# Read configuration variable file if it is present
test -r /etc/default/$NAME && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

case $1 in
  start)
log_daemon_msg "Starting $DESC" "$NAME"
/root/bin/sql_injection_monitor &
log_end_msg 0
;;
  stop)
log_daemon_msg "Stopping $DESC" "$NAME"
log_end_msg 0
;;
  restart|force-reload)
log_daemon_msg "Restarting $DESC" "$NAME"
do_start
log_end_msg 0
;;
  *)
echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" 1>&2
exit 3
;;
esac

exit 0

[Savannah-hackers-public] internal.savannah.gnu.org web server access

[Bob Proulx]

Is internal.savannah.gnu.org's web server available from the outside
world?  It appears that was where munin had run previously.  I would
restore it there if it were accessible.  I am thinking that access is
blocked at the firewall these days however.  But thought I might as
well ask about it.

Bob

Re: [Savannah-hackers-public] internal.savannah.gnu.org web server access

[Bob Proulx]

Hi Ryan,

Ryan Doyle wrote:
> I'm thinking it would make sense to have this on the mgt
> server. This is externally accessible and has other monitoring-type
> applications running on it (Nagios).
> Thoughts?

Ah!  mgt's web server is externally accessible.  That would work.

My thoughts are that monitoring such as this is useful.  Generally I
would like to get it working again.  If mgt is the place for it then I
would happily set it up there.

Bob

[Savannah-hackers-public] mgt insserv sysync loop issues

[Bob Proulx]

I started installing (re-installing) munin on mgt.  I didn't expect
any problems.  But this displayed various init.d script errors
relating to sysync.  (Somewhat abreviated.  I removed some of the
repetition.)

  insserv: warning: script 'K20sysync' missing LSB tags and overrides
  insserv: warning: script 'sysync' missing LSB tags and overrides

The above is the root of the problem that cascades down through these
others below.

  insserv: There is a loop at service sysync if started
  insserv: There is a loop between service munin-node and checkroot if started
  insserv:  loop involving service checkroot at depth 3
  insserv:  loop involving service mountdevsubfs at depth 2
  insserv: There is a loop at service munin-node if started
  insserv: There is a loop between service munin-node and mountnfs if started
  insserv:  loop involving service mountnfs at depth 2
  insserv:  loop involving service networking at depth 1
  insserv:  loop involving service mountnfs-bootclean at depth 4
  insserv:  loop involving service screen-cleanup at depth 6
  insserv: There is a loop between service sysync and mountall-bootclean if 
started
  insserv:  loop involving service mountall-bootclean at depth 2
  insserv:  loop involving service mountall at depth 1
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  ...
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv:  loop involving service mountoverflowtmp at depth 8
  insserv:  loop involving service checkfs at depth 5
  insserv: There is a loop between service sysync and ifupdown-clean if started
  insserv:  loop involving service ifupdown-clean at depth 1
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv: Starting sysync depends on munin-node and therefore on system 
facility `$all' which can not be true!
  insserv:  loop involving service hwclockfirst at depth 4
  insserv:  loop involving service hostname at depth 5
  insserv: There is a loop between service munin-node and mountall if started
  insserv:  loop involving service mtab at depth 7
  insserv: There is a loop between service munin-node and mountoverflowtmp if 
started
  insserv: exiting now without changing boot order!
  update-rc.d: error: insserv rejected the script header
  dpkg: error processing munin-node (--configure):
   subprocess installed post-installation script returned error exit status 1
  configured to not write apport reports
Errors were encountered while 
processing:
   munin-node
  E: Sub-process /usr/bin/dpkg returned an error code (1)

I would like to learn more about sysync so that I could fill in the
Description and Short-Description appropriately but in the meantime I
am going to add the following LSB header to address this problem.

  ### BEGIN INIT INFO
  # Provides:  sysync
  # Required-Start:$remote_fs $syslog
  # Required-Stop: $remote_fs $syslog
  # Default-Start: 2 3 4 5
  # Default-Stop:
  # Short-Description: Sysync user/group sync tool
  # Description:   Sysync is a tool to manage users/groups and
  #configuration files across multiple hosts.
  ### END INIT INFO

That makes insserv happy when ordering the boot time rc symlinks.

I rather silently did not install any stop symlinks because I don't
think this service needs to be stopped at reboot.  There has been a
general move to avoid spending cpu cycles to stop processes that don't
need an explicit shutdown sequence to be happy and thinking this is in
that category I didn't stop it.

And with the above package installation is happy again.  Yay!

Bob

Re: [Savannah-hackers-public] internal.savannah.gnu.org web server access

[Bob Proulx]

Ryan Doyle wrote:
> Yup, sure is. You could setup munin.savannah.gnu.org or similar and
> used name-based virtual hosting on Apache.

I don't think we need a separate virtual server for it.  At least not
yet.  I set it up here.

  http://mgt.savannah.gnu.org/munin/

The advantage to a separate URL though is that the service could be
moved from node to node without external disruption.

Except that there appears to be firewall blocking between mgt and all
of vcs, frontend, download.  It can only connect to internal.

> Also, if you are just interested in monitoring and performance data,
> I installed Performance Co-Pilot (pcp) a while ago on the savannah
> servers. It's got a bit of a learning curve but I find it extremely
> useful.

I didn't start out with the idea that we needed to monitor performance
data.  I started out thinking that dpkg and apt should not produce
errors when installing additional tools on the machine.  So I wanted
to fix that problem throughout.

But munin itself appears to have been broken since 2009.  So I wanted
to either have it work or to uninstall it entirely.  I don't feel
strongly either way.

But I do like munin and use it myself on my machines.  Therefore I am
inclined to make it work.  Monitoring and historical data is one of
those things that becomes useful later when you unexpectedly have a
reason to go look at it.

> Part of the pcp toolset is pmchart, this will display performance
> metrics in real time, charted in a time series plot. You'd run
> pmchart on your local machine and point it to the box you wish to
> monitor.
> 
> Come to think of it, I should add this info to the wiki!

Yep!

Bob

[Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

I would like to walk through the Savannah VMs and get all of them
their security upgrades.  I would like to discuss the procedure for
doing this.  The VMs are at:

  download: 6.0.2
  frontend: 6.0
  internal: 6.0.2
  mgt: 6.0.2
  vcs: 6.0.1

I think mgt might be the closest to savannah-hackers and the furthest
from the public and therefore I propose that we upgrade it first.
Have to start somewhere.

Does anyone have a preference for doing one thing or another?

When would be a good time to perform this upgrade?

How long of a waiting time for major events such as this should we
have between posting a proposal for action and then performing the
action?

Other comments?

Bob

Re: [Savannah-hackers-public] internal.savannah.gnu.org web server access

[Bob Proulx]

Bob Proulx wrote:
> Except that there appears to be firewall blocking between mgt and all
> of vcs, frontend, download.  It can only connect to internal.

I plan on adding this rule to the /etc/default/iptables-rules files on
the various VMs.

  # Allow munin from 140.186.70.74 mgt.savannah.gnu.org
  -A INPUT -m tcp -p tcp --dport 4949 --src 140.186.70.74 -j ACCEPT

Bob

Re: [Savannah-hackers-public] internal.savannah.gnu.org web server access

[Bob Proulx]

Karl Berry wrote:
>   # Allow munin from 140.186.70.74 mgt.savannah.gnu.org
>   -A INPUT -m tcp -p tcp --dport 4949 --src 140.186.70.74 -j ACCEPT
> 
> Sounds good to me, FWIW.

Okay.  Will do during daylight sysadmin hours.  Just in case.

> I think mgt might be the closest to savannah-hackers and the furthest
> from the public and therefore I propose that we upgrade it first.
> 
> Also sounds good to me.

This was asked in the other thread.  I will divert this answer back to
it so that we can keep everything straight.

Bob

Re: [Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

Karl Berry wrote in the other message:
> I think mgt might be the closest to savannah-hackers and the furthest
> from the public and therefore I propose that we upgrade it first.
> 
> Also sounds good to me.

Sounds good.  Will upgrade mgt first and then decide what to do next.

> When would be a good time to perform this upgrade?
> 
> I don't think it really matters, assuming downtime is basically a matter
> of a reboot.  (Especially for mgt.)

A VM should boot very quickly.  As a risk management I will coordinate
with sysadmin just in case something goes really bad and it needs a
rescue.  I don't expect that to be needed.  But just in case.

> I would suggest posting a news item so users have a chance of knowing
> what is going on.  https://savannah.gnu.org/news/?group=administration

Okay.  I see that there hasn't been a news posting there since May 2012.

> How long of a waiting time for major events such as this should we
> have between posting a proposal for action and then performing the
> action?
> 
> Once Michael confirms, you're good :).  Otherwise ... a few days
> at least?

I would want to do this during daylight sysadmin hours too.  Same
thing with the firewall rule changes too.  Just in case.

> Other comments?
> 
> Do you have any sense of what breakage might ensue from the upgrade?

I don't expect any breakage.  But examples of past breakage would be
those /etc/init.d/* scripts that I just posted about fixing because
they were locally added but didn't have LSB headers.  Without those
the system tool 'insserv' could not determine how to make the symlinks
and order the boot.  Since we hit those a couple of times already
there might be another VM or two with a problem like that lurking.  If
the machine hasn't been rebooted since those were installed then the
reboot path for them won't have been tested.  And we all know the
truism of untested code.  It is why I like to reboot first before
making major changes.

I already fixed mgt for these.  But I can't predict any other problems
beyond those similar to the past ones.  I can only do my best looking
at things going in and react to them as they occur.

I will also look for packages that contain obsolete conffiles
especially any leaving files behind in /etc/init.d since those can be
troublesome due to missing LSB headers.  And will look for held
packages and try to understand why and to clean them up if I see them
too.  Plus I will look for old/new conffiles '*.dpkg-*' and '*.ucf-*'
in /etc and clean those up both before to start clean so that any left
are from this upgrade.  And then look afterward to leave it clean for
next time.

But since I expect that the Savannah Hackers all do reasonable things
then I don't expect any breakage.  :-) Missing LSB headers in init
scripts is kind'a a common gotcha that I have seen everywhere since
those have newly appeared on the scene as required.  Minor in the
grand scheme of things.

But not expecting any breakage doesn't mean there won't be any.  Must
plan for problems regardless.

I always worry about the possibility that someone has compiled in
custom Apache modules that might be disconnected by the Apache upgrade
and need to be rebuilt.  But so far haven't hit any.  (No Rails
Phusion installed that I can see for example.)

> What will the new version be?

The current Debian Stable "Squeeze" point release version is 6.0.7.

> Is there a super-high-top-level NEWS-ish list for each Debian
> release?

Since the latest on the VMs is 6.0.2 it means we have missed the last
five point releases and if you are interested in the changes then all
of those would need to be examined.  And for frontend the first two
point releases as well.

Remember that there isn't any change in functionality.  These are all
considered Debian 6.0.  These are security patches for the most part.
Along with other things that are considered safe but important enough
to backport into the release branch.  We will be picking up the last
two year's of security upgrades all at once.  Here are the individual
release notes for the ones we will be getting.

  http://www.debian.org/News/2013/20130223 6.0.7 released
  http://www.debian.org/News/2012/20120929 6.0.6 released
  http://www.debian.org/News/2012/20120512 6.0.5 released
  http://www.debian.org/News/2012/20120128 6.0.4 released
  http://www.debian.org/News/2011/20111008 6.0.3 released
  http://www.debian.org/News/2011/20110625 6.0.2 released
  http://www.debian.org/News/2011/20110319 6.0.1 released

Bob

Re: [Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

A few days turned into a few more because I wanted to wait for weekday
daylight hours when everyone would be around.

I am going to check with sysadmin then will reboot mgt today.  After
the reboot we can check that everything got restarted correctly.  It
has been 336 days since the init.d scripts were tested. :-) After that
it is more cleaning before getting the upgrade under way.

Bob

Re: [Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

Bob Proulx wrote:
> I am going to check with sysadmin then will reboot mgt today.  After
> the reboot we can check that everything got restarted correctly.  It
> has been 336 days since the init.d scripts were tested. :-) After that
> it is more cleaning before getting the upgrade under way.

The mgt VM has been freshly rebooted.  It cycled down and up very
quickly in less than a minute.  Next steps are cleaning old config
files (*.dpkg-* and *.ucf-*) and "obsolete conffiles from init.d" if
any so as to have things clean.  Then the upgrade.  Probably ready to
reboot tomorrow to the latest kernel that will get installed.

Bob

Re: [Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

Bob Proulx wrote:
> Next steps are cleaning old config files (*.dpkg-* and *.ucf-*) and
> "obsolete conffiles from init.d" if any so as to have things clean.
> Then the upgrade.  Probably ready to reboot tomorrow to the latest
> kernel that will get installed.

All done.  The mgt VM has been upgraded.  Please let me know if you
find any problems.

There is a problem left unresolved yet.  Michael and I were chatting
on IRC about the problem.  Here is a summary of the issue.

The user accounts in /etc/passwd, /etc/group, and /etc/shadow are
managed through a sysync process.  The intent is to manage added
accounts across the collection of VMs.

This comes into conflict with users created by packages when they are
installed.  A package is installed and creates a new local account for
a pseudo user.  Then the sysync management process removes this
unmanaged account.  Then another package is installed and creates
another local pseudo account and reuses the same uids.  This causes
several packages to have the same pseudo users with duplicated uids.
Some duplicated uids are problematic and need to be fixed on mgt.

At the moment we don't have a solution.  Yet.  It is the next thing to
figure out.  I am thinking that sysync should be modified to ignore
accounts less than a specified min configured value.  But ideas are
still flowing.  More on this later.  It needs to be solved to finish
the cleanup and before we work through the other VMs.

Bob

Re: [Savannah-hackers-public] Unable to run GUI application using cross-compiled jamvm for arm

[Bob Proulx]

Hello Aditya,

aditya sen wrote:
> Hello everyone,
> 
> First of all I apologize for this direct message. I went through the
> website https://savannah.gnu.org/ and only found this address where I can
> post my query. I am sorru if I posted at the wrong place. Below is the
> description of the problem we are facing.

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

But you are asking about GNU Classpath.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

Looking at the Savannah web page here I see this list of mailing lists
for the Classpath project.

  https://savannah.gnu.org/mail/?group=classpath

I suggest starting off by sending your request to the
classp...@gnu.org mailing list as a general good place to start asking
questions about your problem.

Good luck!
Bob


> We have cross-compiled the gnu classpath 0.98 and jamvm 1.5_4 for our arm
> xscale device. When we run helloworld program, it works perfectly fine. But
> when we run gui applications, there is no response. Below are our commands
> which we gave for cross-compiling:
> 
> GNU CLASSPATH:
> ./configure --host=arm-xscale-linux-gnueabi  --prefix=/usr/local
> --disable-examples --with-x --disable-qt-peer --disable-gtk-peer
> --disable-gconf-peer --disable-plugin --enable-alsa --disable-dssi
> --with-escher=/home/warrior/java-project/escher-0.3/src
> --enable-local-sockets --enable-collections --enable-xmlj --disable-Werror
>  --enable-tools  --enable-default-toolkit=gnu.java.awt.peer.x.XToolkit
> 
> make
> 
> make install
> 
> JAMVM:
> 
> ./configure --host arm-xscale-linux-gnueabi --prefix=/usr/local
> --with-classpath-install-dir=/usr/local –enable-ffi
> 
> make
> 
> make install
> 
> We then transfer the required files to our device.
> 
> 
> we run our programs as
> 
> jamvm helloworld
> 
> jamvm -verbose Button
> 
> helloworld runs perfectly fine and gives the desired output in proper time.
> But when we run any gui sample we receive following error:
> 
> Exception in thread "main" java.lang.ExceptionInInitializerError
>at gnu.java.awt.peer.x.XToolkit.getClasspathFontPeer(XToolkit.java:208)
>at java.awt.Font.getPeerFromToolkit(Font.java:329)
> ...
> Caused by: java.lang.NullPointerException
>at java.io.InputStreamReader.(InputStreamReader.java:208)
>at java.util.Properties.load(Properties.java:380)
>at gnu.java.awt.font.OpenTypeFontPeer.(OpenTypeFontPeer.java:93)
>at gnu.java.awt.peer.x.XToolkit.getClasspathFontPeer(XToolkit.java:208)
> 
> Can someone please guide us on this. This is becoming very critical.
> 
> Earlier we thought this to be jamvm issue, but on jamvm forums , it can out
> that we are missing something while cross-compiling gnu classpath.
> 
> It will be a great help if someone can direct us towards fixing this to see
> what we are missing to get the gui applications running with jamvm. We have
> made a small sample using java swing which creates a frame and 2 buttons
> over it.
> 
> Thanks in advance.
> Aditya

[Savannah-hackers-public] Next Savannah VM system upgrade - internal

[Bob Proulx]

Continuing in the upgrades I plan on doing the same recipe on
"internal" next.  I think it is completely normal and so good for the
next pick of low hanging fruit.  It runs MySQL and I will make sure we
have a good db backup prior to the upgrade.  But there is the
preparatory cleaning to do on it before then.

Bob

Re: [Savannah-hackers-public] Savannah VM system upgrades

[Bob Proulx]

Bob Proulx wrote:
> There is a problem left unresolved yet.  Michael and I were chatting
> on IRC about the problem.  Here is a summary of the issue.
>
> The user accounts in /etc/passwd, /etc/group, and /etc/shadow are
> managed through a sysync process.  The intent is to manage added
> accounts across the collection of VMs.
> 
> This comes into conflict with users created by packages when they are
> installed.  A package is installed and creates a new local account for
> a pseudo user.  Then the sysync management process removes this
> unmanaged account.  Then another package is installed and creates
> another local pseudo account and reuses the same uids.  This causes
> several packages to have the same pseudo users with duplicated uids.
> Some duplicated uids are problematic and need to be fixed on mgt.
> 
> At the moment we don't have a solution.  Yet.  It is the next thing to
> figure out.  I am thinking that sysync should be modified to ignore
> accounts less than a specified min configured value.  But ideas are
> still flowing.  More on this later.  It needs to be solved to finish
> the cleanup and before we work through the other VMs.

There was some more discussion concerning this and we decided that for
the moment we would suspend the operation of the sysync process until
we figure out the best way to deal with the above problem.  So for the
moment the system sync is suspended.  This enables us to proceed
onward through the rest of the VMs.

As a final bit of stuff on mgt I am going to check the current system
user accounts to make sure those are okay and fix any that are not.
Then give the VM a reboot to the kernel that was installed.  The
previous reboot was very quick and I expect this one to be too.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - internal

[Bob Proulx]

Bob Proulx wrote:
> Continuing in the upgrades I plan on doing the same recipe on
> "internal" next.  I think it is completely normal and so good for the
> next pick of low hanging fruit.  It runs MySQL and I will make sure we
> have a good db backup prior to the upgrade.  But there is the
> preparatory cleaning to do on it before then.

In preparation I rebooted "internal" today.  It had been up 346 days.
All good.

Bob

Re: [Savannah-hackers-public] (no subject)

[Bob Proulx]

Ankith S wrote:
>I have found some c source code of UNIX commands.page is
> coreutils.git - GNU coreutils.htm.   And is it possible to compile
> those codes using gcc compiler??if possible please guide me how to
> compile??

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

But you are asking about GNU coreutils.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

Looking at the Savannah web page for the project you are asking about
I see this list of mailing lists.

  https://savannah.gnu.org/mail/?group=coreutils

Please send your question to the coreut...@gnu.org mailing list for
discussion about that project.

For compiling the coreutils project please read these docs:

  
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=README-hacking;hb=HEAD

Bob

Re: [Savannah-hackers-public] A wasteland of a wiki

[Bob Proulx]

Karl Berry wrote:
> We can't copy from the old wiki.  That's the whole problem (well, the
> biggest part of it, anyway).  At least I don't know how.

As I read that suggestion I think perhaps the suggestion was to bring
up the current web page and then cut-n-paste from it into a new
location.  All history would be lost but we would then have the
current snapshot to import.  It would be very labor intensive.  It
would require reformatting.  Simple, brute force, but not without
problems.

> To me, neither losing everything that has been written nor updating
> starting from the garbage that is in the wiki now is a viable way to
> proceed.  OTOH, I am not the boss.  Whatever the consensus is.

And I think that is a valid point.  It would be shame to lose
everything.

> Everyone completely agrees and has ever since the wiki was cracked, but
> no one has made progress on actually recovering it.
> 
> If someone (you? Ryan? anyone?) wants to set up ikiwiki (or some other
> software -- we could discuss) on savannah, that would IMHO be another
> useful thing to do, independent of (and prerequisite to) actually adding
> content.  As long as we have been dead in the water for X months, I
> suggest it would be a good time to take this opportunity to move to a
> different wiki software.

After I finish the current round of VM upgrades I would be willing to
look at the wiki problem.  But I want to finish the one task before
starting another.

I am completely unfamiliar with administrating the current Zwiki or
with the proposed Ikiwiki.  I used to maintain a MoinMoin wiki.  But I
found that quite unpleasant to work through all of the upgrades for
MoinMoin.  The Ikiwiki suggestion seems most attractive to me.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - internal

[Bob Proulx]

The VM internal has been upgraded.  For the most part everything is
happy and working fine.  Please let me know if there are any problems.

Here are the issues I know about and am continuing to work through.

Problem 1:

  internal:~# debsums -c
  /lib/modules/2.6.26-2-xen-686/modules.symbols
  /lib/modules/2.6.26-2-xen-686/modules.pcimap
  /lib/modules/2.6.26-2-xen-686/modules.dep
  /lib/modules/2.6.26-2-xen-686/modules.usbmap
  /lib/modules/2.6.26-2-xen-686/modules.isapnpmap
  /lib/modules/2.6.26-2-xen-686/modules.inputmap
  /lib/modules/2.6.26-2-xen-686/modules.alias
  /lib/modules/2.6.26-2-xen-686/modules.seriomap
  /lib/modules/2.6.26-2-xen-686/modules.ieee1394map
  debsums: missing file /var/lib/pcp/pmns/.NeedRebuild (from pcp package)

There are two separate complains.  I have no idea why the above is
complaining for the kernel package.  The pcp complaint just looks like
a simple package bug.

Problem 2:

  internal:~# dpkg-reconfigure grub-pc
  /usr/sbin/grub-probe: warn: disk does not exist, so falling back to partition 
device /dev/xvda2.
  /usr/sbin/grub-probe: warn: disk does not exist, so falling back to partition 
device /dev/xvda2.
  /usr/sbin/grub-probe: warn: disk does not exist, so falling back to partition 
device /dev/xvda2.
  /usr/sbin/grub-probe: warn: disk does not exist, so falling back to partition 
device /dev/xvda2.
  /usr/sbin/grub-setup: warn: disk does not exist, so falling back to partition 
device /dev/xvda2.
  /usr/sbin/grub-setup: error: cannot guess the root device. Specify the option 
`--root-device'.

I imagine this problem is due to it being a Xen domU.  The previous VM
mgt doesn't have grub installed.  I chatted with Ward and he says that
the dom0 will bootstrap the xen domU based upon the presence of the
menu.lst file.  Therefore I think internal needs the same treatment as
mgt.  Meaning that I think grub needs to be purged off leaving only
the menu.lst file for the dom0 to boot.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - internal

[Bob Proulx]

> Problem 1:
> 
>   internal:~# debsums -c
>   /lib/modules/2.6.26-2-xen-686/modules.symbols
>   /lib/modules/2.6.26-2-xen-686/modules.pcimap
>   /lib/modules/2.6.26-2-xen-686/modules.dep
>   /lib/modules/2.6.26-2-xen-686/modules.usbmap
>   /lib/modules/2.6.26-2-xen-686/modules.isapnpmap
>   /lib/modules/2.6.26-2-xen-686/modules.inputmap
>   /lib/modules/2.6.26-2-xen-686/modules.alias
>   /lib/modules/2.6.26-2-xen-686/modules.seriomap
>   /lib/modules/2.6.26-2-xen-686/modules.ieee1394map
>   debsums: missing file /var/lib/pcp/pmns/.NeedRebuild (from pcp package)
> 
> There are two separate complains.  I have no idea why the above is
> complaining for the kernel package.  The pcp complaint just looks like
> a simple package bug.

Both of those problems have been fixed.  The 2.6.26-2-xen-686 package
wasn't used.  It was replaced previously with 2.6.26-5-xen-686 which
is what the system is running but the older -2 was still hanging
around.  I purged it.

The pcp package includes the NeedRebuild file but it had been
removed.  I restored it to silence the noise.

Bob

Re: [Savannah-hackers-public] vc load

[Bob Proulx]

Karl Berry wrote:
> Apparently loggerhead is started once by rc and lives until the system
> reboots (251 days and counting).  I suppose some kind of balance finally
> got tipped.  Sure looking forward to Bob's reboot program completing.

Each of the VMs has its own set of quirks that I am learning about
them as I go along.  I am just finishing up VM "internal" today.  I am
chatting on IRC with the FSF admins about the Xen bootstrap process
and applying the final cleanup on it now.  (A little scary to be
purging grub and then rebooting but that is the plan today for that
xen domU system.)  That being behind us I will reboot "frontend" and
hit it next.  It is the furthest behind.  But getting into the rhythm
of things now.

But for the initial reboot we might as well reboot "vcs" and
"download" today along with the others.  That will give themn the
initial reboot-before-making-changes since I like being able to
separate old problems that may have been there for a while from new
problems that I may have just introduced.

> There may have been contention with git, since the next three processes
> ...
> Don't know what to make of that.  The git commands all went back to
> inetd but it wasn't obvious which project they were associated with, if
> any.  They completed reasonably quickly (a few minutes) after loggerhead
> was killed, but still.

With a load of 38 they were probably just so slow in running that they
were just still running.  I don't think there would have been any real
contention other than general machine resources.  In particular I have
found that VM I/O is never as fast as native I/O and therefore
anything that really hits the disk hard has a disproportionate effect
of disruption bogging down the machine.

I just installed 'iotop' on all of the machines.  Just run it 'iotop'
and perhaps it will be useful helping to diagnose issues in the
future.  It is like top but shows processes by I/O.  The default
column to sort upon is all IO.  Look for the '>' character to indicate
the sort field.  Use keyboard-left or keyboard-right to change sort
fields.  A useful tool.

> Finally, in /etc/xinetd.d/*, I noticed that all the rlimit_cpu values
> were commented out, as in:
> service git  # or bzr or cvs or whatever
> {
>   disable = no
>   #rlimit_cpu  = 20
> 
> I've never used rlimit_cpu, but it sure sounds like a good idea.  Did it
> prove problematic in practice?

I have no information here.

Bob

Re: [Savannah-hackers-public] savannah git seems to have just died

[Bob Proulx]

Aharon Robbins wrote:
> Hi. I did a push into the gawk repo a few minutes ago (it's now 14:21
> EDT) and it worked, but now trying to do a new push, or a pull, I'm
> getting:
> 
> $ git pull
> Permission denied (publickey).
> fatal: The remote end hung up unexpectedly
> 
> Can someone look into this please?

Please try it again now.

I think the problem is that the database is on VM "internal".  I
rebooted it around :24 and then it failed to come back up afterward.
Ward was our guardian angel and rescued the VM.  It is back online.
I didn't realize there was a connection between "internal" and "vcs"
but I know the sshd on vcs is modified for authorized keys.  If it
pulls them from the database then that would be a dependency.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - internal

[Bob Proulx]

Bob Proulx wrote:
> Problem 2:
>   internal:~# dpkg-reconfigure grub-pc
>   /usr/sbin/grub-probe: warn: disk does not exist, so falling back to 
> partition device /dev/xvda2.
> ...
> I imagine this problem is due to it being a Xen domU.  The previous VM
> mgt doesn't have grub installed.  I chatted with Ward and he says that
> the dom0 will bootstrap the xen domU based upon the presence of the
> menu.lst file.  Therefore I think internal needs the same treatment as
> mgt.  Meaning that I think grub needs to be purged off leaving only
> the menu.lst file for the dom0 to boot.

This has been done.  The grub* packages were purged.  Because grub-pc
(aka grub 2) wasn't functional at all.  I saved a copy of the menu.lst
file off first just in case.  But I wasn't paranoid enough and had a
problem.

I didn't like the idea of having menu.lst be only updated manually.
The problem is with grub2.  I therefore tried grub-legacy (aka grub 1).
Which seems like it will work and keep menu.lst updated.  I think it
will be just fine.  Eventually.

But... and there is always a "but" the device.map file isn't generated
correctly by grub-mkdevicemap.  And that filtered through to the new
menu.lst file that I created with the grub-legacy version of
update-grub.  Among the other differences I didn't catch the
difference before rebooting.  The root was /dev/xvda2 instead of hd0.
The machine went down but did not come back up due to the error in
menu.lst.

Internal was down for from about 12:25 -0600 until about twenty
minutes later.  Ward was our guardian angel and rescued the system.
Thanks Ward!

I still think menu.lst needs to be updated automatically as upgrades
are applied.  I am going to continue to work that problem.  But for
the moment I am going to let internal sit and call it good for now.
Since internal seems to be critical for vcs access.  I will focus on
this problem on a different VM and then transport the eventual
solution back to internal later.

Bob

[Savannah-hackers-public] Next Savannah VM system upgrade - frontend

[Bob Proulx]

I am going to start the upgrade flow for "frontend" and reboot it in a
few minutes.  Nothing has been done to it yet.  It has been up for a
year.  Just getting it ready for the upgrade.  It should reboot quickly.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - frontend

[Bob Proulx]

Bob Proulx wrote:
> I am going to start the upgrade flow for "frontend" and reboot it in a
> few minutes.  Nothing has been done to it yet.  It has been up for a
> year.  Just getting it ready for the upgrade.  It should reboot quickly.

Due to other urgent events unrelated to frontend I missed the window
of opportunity today.  I am going to spring it at the next earliest
opportunity that presents itself.  I want to make sure it is daylight
hours for the admins who have access to the dom0 as a risk mitigation.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - frontend

[Bob Proulx]

Topic: Zope

I have mostly upgraded "frontend".  Most importantly this includes
Apache2 and in general all of the subsystem components.

  frontend:~# cat /etc/debian_version
  6.0.7

But I ran across something that raised questions.  The machine has
Zope installed from when it was Lenny 5.0.  Zope was removed from
Squeeze 6.0.  I didn't look for why but probably the typical reason is
due to unfixed release critical bugs.  It has reappeared in Wheezy 7
and a backport of it exists.

I imagine that the use of Zope is for the Zwiki.  I imagine that I can
install the 2.12 version from Backports without problem.  But that
would be a major version upgrade as opposed to the minor security
upgrades of everything else.  And of course Zwiki is already broken so
this might make things worse.

Therefore I held off upgrading Zope at this moment.  I marked it as
"held" in the package manager.

I also held off upgrading ssh too.  I know that some of the VMs have a
customized ssh and need special handling.  I think that is only vcs
and download which I had put off until last due to that reason.  But I
want to verify this before I upgrade it on frontend.

TODO Summary: Double check ssh and upgrade it.  Install grub-legacy
and verify menu.lst setup.  Understand Zope and upgrade it to
backports.  Cross fingers and reboot.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - frontend

[Bob Proulx]

> I have mostly upgraded "frontend".  Most importantly this includes
> Apache2 and in general all of the subsystem components.

A quick status summary:

  frontend:~# apt-show-versions |grep -v -e uptodate
  libdb4.5 4.5.20-13 installed: No available version in archive
  openssh-client/squeeze upgradeable from 1:5.5p1-6 to 1:5.5p1-6+squeeze3
  openssh-server/squeeze upgradeable from 1:5.5p1-6 to 1:5.5p1-6+squeeze3
  python2.4 2.4.6-1+lenny1 installed: No available version in archive
  python2.4-minimal 2.4.6-1+lenny1 installed: No available version in archive
  zope-common/squeeze upgradeable from 0.5.45 to 0.5.50
  zope2.10 2.10.6-1+lenny1 installed: No available version in archive

Zope uses libdb4.5.  Fixing up Zope will fix up libdb4.5.

/usr/bin/python is already python2.6.  I think python2.4 is an orphan.
Unless there is a reason to keep it I plan on removing it and leaving
the default as python2.6.

The openssh files are as already explained.  Just waiting temporarily
while I double check them because I don't want to accidentally lock us
out.

Bob

Re: [Savannah-hackers-public] Next Savannah VM system upgrade - frontend

[Bob Proulx]

Karl Berry wrote:
> 1) Can you figure out enough Zope that you can extract the content?
> That is what is needed, so that it can be a starting place for a sane
> wiki.

Sorry that I haven't even started looking at it.  Soon.  And then
there is Aljosha Papsch's work at grabbing the current state of the
pages.  But I haven't started tracing down through the Zwiki paths
yet.

> > Customized ssh and need special handling
> 
> Looking at frontend:/etc/ssh/sshd_config, I concur that frontend does
> not have the customized ssh login stuff.  But you should look too.

Looks normal.  I upgraded it.  Done.

> On another front: I noticed that on mgt and frontend (and probably
> internal, but didn't check), "locate" no longer works.  I really really
> miss it.  Can you enable whatever needs to be enabled, please?

I will be happy to look into it.

Bob

Re: [Savannah-hackers-public] Keeping web pages in something other than CVS

[Bob Proulx]

Karl Berry wrote:
> Glenn Morris wrote:
> Maybe they'd just want
> to hide the CVS web pages repo on their Savannah project page then.
> 
> I don't think there's a way to do that (without disabling web pages
> completely, by deselecting "homepage" in "select features").
> 
> Anyway, we'll see if any of the CVS-haters want to do the necessary
> work.  I'm not holding my breath.

I find CVS to be more than sufficient for the web pages.  The web
pages don't usually change at the rapid pace that a project might be
developed.  CVS seems fine.

For those that are really chafing at CVS there are the interfaces.  I
don't know about bzr but in git there is git-cvs (and git-svn and
git-bzr) that will allow you to use git locally but other repositories
remotely.  I haven't found the need to use git-cvs, the web pages are
not that big of a deal, but git-svn for svn hosted projects is
excellent and git-bzr a sanity saver for me.

Bob

[Savannah-hackers-public] MTA problems on internal

[Bob Proulx]

The VM internal is having problems with the MTA.  Running mailq there
shows many messages that are "frozen".  Doing some testing it looks
like Exim is still delivering messages to at least some addresses.
But others, such as root email, is not being delivered.

Does anyone know if there is anything special about the MTA
configuration on the VMs?  Internal seems misconfigured.

  $ connect localhost 25
  220 colonialone.fsf.org ESMTP Exim 4.72 Thu, 25 Apr 2013 23:17:59 +
  QUIT
  221 colonialone.fsf.org closing connection

As to the problems the logs are indicating:

  2013-04-25 23:06:55 1UVVFf-00012Z-As == r...@internal.savannah.gnu.org 
R=dnslookup T=remote_smtp defer (111): Connection refused

Connection refused sounds like a configuration problem.  But if I test
mail delivery manually I can do it.  Mail is sent through okay.  So
the problem seems to be local to frontend and not a firewall or
misconfiguration on the other end.

Thanks,
Bob

Re: [Savannah-hackers-public] MTA problems on internal

[Bob Proulx]

Hi Ward,

Ward Vandewege wrote:
> It's only mail to r...@internal.savannah.gnu.org that was stuck. That mail is
> configured to be forwarded to savannah-reports-priv...@gnu.org via
> /etc/aliases.
> 
> But exim on internal didn't know that internal.savannah.gnu.org was its own
> domain/hostname. 

I could tell that it didn't have the right hostname configuration.
But not being familiar with exim meant I didn't know where to debug it.

> In fact someone (in 2011) had hardcoded the old hostname of that machine in
> /etc/exim4/exim4.conf.template... The proper way for a simple exim setup like
> this would have been to use dpkg-reconfigure exim4-config and tell Exim its
> local domain. 
> 
> I removed the hardcoded hostname, updated /etc/exim4/update-exim4.conf.conf
> (which is what dpkg-reconfigure exim4-config does), and redelivered the
> frozen messages.

Yay!  Thanks Ward!

Bob

[Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

Summary:

Tonight I upgraded VM vcs.  Mostly.  I will keep chipping away at
things until they are completely through.

I spot checked cvs, hg, git, and mercurial using their various web
interfaces.  All appeared to be working.  Please report any problems
if something is not working.

Does anyone know anything about how python-bzrlib is used?

This was a little bit of a rough upgrade.  Things did not go
smoothly.  Hopefully there won't be any resulting problems.

Details:

The python packages were in a very sad state.  They did not upgrade
cleanly.  When the python packages were upgraded the result was many
errors being unable to byte compile the python scripts.  I am not sure
how they got into their present bad state but python2.4 was still
hanging around and the presences of it was causing python package
upgrades to fail due to errors from python2.4.  However python2.4
wasn't listed as being installed.  The filesystem state was out of
sync with the package manager state.

It looks like old files were left hanging around on the system but the
package manager thought they were removed.  Also there were various
postinst script files from 2.4 still hanging around.  Also there were
pkg diversions still hanging around.  It was a mess!  I left a
"typescript" file in /root with details of the problem if anyone
wants to look.

I needed to take some manual cleaning steps to get things back into an
upgradable shape.  I ended up needing to manually remove many
python2.4 remnants and manually patch up the dpkg database.  Obviously
that isn't good because mistakes can be made and it can get out of
sync but it was in a bad way and needed help.

Additionally python-bzrlib appears to have been installed by simply
copying files into place in /usr/lib/python2.6/dist-packages/bzrlib.
Does anyone know anything about how those files arrived there?  I
think as a local addition they should be installed in
/usr/local/lib/python2.6/dist-packages/bzrlib instead.  I left them in
place but think that they should go into /usr/local instead.  Or we
can create a backports package for it.  The python-bzrlib package does
not exist in Squeeze 6 but appears in Wheezy 7.  If we wait a month
then we can install the next version when it releases.

Packages that depend upon python working correctly include mercurial.
I have no idea about the mercurial repositories on vcs.  If anyone who
has hg repositories could test then and report if they are still happy
or not that would be great.

Since the ssh there is custom I "held" those packages for the moment
and did not upgrade those.  Ssh is the only package that has not been
upgraded.  Also the VM has not been rebooted.  It needs to have the
grub configuration double reviewed before rebooting.  I didn't touch
the grub config yet and it probably needs it.  I still need to look
through all of the old/new config files in /etc and clean and/or merge
the 17 or so of them.

Bob

Re: [Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

Ineiev wrote:
> I can't update anonymous checkout of web pages for
> `www' and `gnun'; cvs update says:
> 
> cvs [update aborted]: unrecognized auth response from cvs.sv.gnu.org: cvs: 
> unrecognized option '--allow-root-regexp=^/srv/cvs/sources/.*$'

I see reports from two others that bzr+ssh isn't isn't working
either.  I am looking at it now.

Bob

Re: [Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

Eli Zaretskii wrote:
> Indeed, bzr+ssh doesn't work.  Trying to update from the Emacs
> repository, I get this:

Michael debugged it and fixed it.  Thanks Michael!  Please try it
again now.  I don't have a way to test it.

Bob

Re: [Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

Ineiev wrote:
> Another issue: in `www' web pages,  cvs commit says:
> 
> Failed to exec cvs: No such file or directory at /usr/local/bin/sv_membersh 
> line 110.

Thanks for the report.  That is actually just another face of the same
issue.  The cvs that had been installed was locally patched.  The
"packaged version" was simply a dummy equivs package to hold the name
in the package manager.  The upgrade removed it and overwrote the
locally patched files with the new packaged files.  So now we need to
recover the previously patched version.  Tracing through the steps I
installed the previously installed package before I realized it was
simply a dummy.  Argh.  I reinstalled the packaged version.  I have
put things back to the inbetween state where auth cvs works but anon
cvs through the pserver is still unhappy.  Michael put me on the path
of the notes from Derek for the patching of the cvs server.

Sorry for all of the trouble!

Bob

Re: [Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

Anon cvs through the pserver should be working again.  Please try it
and let us know!

In order to solve the problem as quickly as possible I asked Nico to
restore /usr/bin/cvs from backup from last week.  That restored the
patched version.  Now we can more calmly deal with the issue.

Bob

Re: [Savannah-hackers-public] VM vcs upgraded (mostly)

[Bob Proulx]

I am working through the various notifications.  I see that the vcs
/etc/cron.hourly/bzr_commit_mail_notification script is producing
errors.

/etc/cron.hourly/bzr_commit_mail_notification:
Traceback (most recent call last):
  File "/usr/src/bzr-hookless-email/bzr_hookless_email.py", line 328, in 

main()
  File "/usr/src/bzr-hookless-email/bzr_hookless_email.py", line 79, in main
branch.update()
  File "/usr/src/bzr-hookless-email/bzr_hookless_email.py", line 132, in update
for revision in self._revisions_to_send():
  File "/usr/src/bzr-hookless-email/bzr_hookless_email.py", line 149, in 
_revisions_to_send
revision_history = self._branch.revision_history()
AttributeError: 'BzrBranch6' object has no attribute 'revision_history'

Does a bzr person want to take a shot at looking at this problem?

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Glenn Morris wrote:
> I'm pining for commit notifications...

Sorry!

> Maybe try asking baz...@lists.canonical.com for help?

Karl suggested asking on gnu-prog-discuss.

> They'll probably want to know what the versions of bzr etc were before
> and after the upgrade.
> Did you upgrade to bzr 2.6?

In theory bzr wasn't upgraded.  The package manager thinks it has
2.1.2-1 installed and there isn't an upgrade for it in Stable so there
shouldn't have been any changes.  The log doesn't show any changes to
it.  However bzr itself reports 2.6b2 so I know that it is out of sync
with the package manager.  I don't know when that was installed.

Python went from 2.6.6-3+squeeze6 to 2.6.6-3+squeeze7 at the upgrade
and is reporting itself as Python 2.6.6.  Due to the special nature of
the hand installation on that machine it did cause problems since
things weren't really under package management.  The bzrlib addon did
probably change too.  I don't know what version it is installed now.

> I think that may have removed the revision_history attribute that
> bzr-hookless uses. It was deprecated in 2.5:
> 
> https://bugs.launchpad.net/bzr-hookless-email/+bug/988195
> 
> has a possible patch.

Thank you for doing that research!  That is very likely the problem.
I will try the patch and see if that fixes things.

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Glenn Morris wrote:
> Bob Proulx wrote:
> > In theory bzr wasn't upgraded.  The package manager thinks it has
> > 2.1.2-1 installed and there isn't an upgrade for it in Stable so there
> > shouldn't have been any changes.  The log doesn't show any changes to
> > it.  However bzr itself reports 2.6b2 so I know that it is out of sync
> > with the package manager.  I don't know when that was installed.
> 
> It used to be 2.5, AFAIK.
> http://savannah.gnu.org/forum/forum.php?forum_id=7240
> 
> That was fairly important, since older bzr versions were slower.
> The last release is 2.5.1, a minor bug fix.
> 2.6 is still in beta (and seems likely to stay there).
> I don't know where 2.6b2 would have come from.
> Maybe staying with 2.5 would have been better.

Probably.  When there was the initial problem I put out the call for
help and Michael responded to the call and jumped in and fixed that
part of the system.  He should know the details best.  He may have
upgraded it to 2.6 at that time.  I don't know.

Debian Wheezy 7 is scheduled for release next weekend May 4th.  After
we sort out the issues we learned from this Squeeze "point release
security upgrade" I will want to upgrade the machine to Wheezy 7.
Wheezy 7 has bzr 2.6.0~bzr6526-1 in it at this time.  Being the new
Stable release it will definitely have a lot of users of it.  I am
sure that if problems were found and fixed before that they would
after that point.  Upgrading to it will be good because it will wash
out some of the customizations and it will then be on the security
team supported track which would be good.  It would simplify things
somewhat and simple is always good.

Monday morning there was a small bit of discussion on IRC about simply
going directly to Wheezy 7 in order to get the newer bits.  But it was
only brief discussion because almost certainly that would break other
stuff since with that many things on the system will move to new API
versions.  It has just been mentioned in passing and we let it pass. :-) 
But pretty soon that will be the thing to do.

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Bob Proulx wrote:
> Glenn Morris wrote:
> > I think that may have removed the revision_history attribute that
> > bzr-hookless uses. It was deprecated in 2.5:
> > 
> > https://bugs.launchpad.net/bzr-hookless-email/+bug/988195
> > 
> > has a possible patch.
> 
> Thank you for doing that research!  That is very likely the problem.
> I will try the patch and see if that fixes things.

I applied that patch.  It didn't apply cleanly due to file
differences.  I manually and rather mechanically patched the
rejections into the vcs.savannah copy of the file.  I ran the hourly
cron script just now and if no output is good output then it was good
because there was no output.  At least the errors that were emitted
before are now silent.  I have my fingers crossed.

Did you get any notifications from it?

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Eli Zaretskii wrote:
> > Bob Proulx wrote:
> > Did you get any notifications from it?
> 
> Yes, 3 minutes before you sent this message.  Does that sound right?

Yes.  I ran it manually so I could see any output without waiting for
the cron mail to deliver it.  Then sent that message.

I assume that means that all of the new commits that hadn't been
notified in the last couple of days all went out at that time?  Good
deal.  Thanks for the confirmation.

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Glenn Morris wrote:
> If you have time, would you be willing to try the patch from
> https://bugs.launchpad.net/bzr-hookless-email/+bug/533009/comments/3
> 
> - msg.add_inline_attachment(diff_val, diff_name)
> + msg.add_inline_attachment(diff_val, diff_name, 'x-diff')
> 
> It is just a piece of trivia that should improve the MIME type of
> commit messages. It's not at all urgent, so if it doesn't seem to
> work, forget about it.

Seems reasonable.  I added that change to the script.  I imagine we
will need to wait for the next commit before there is a notification.
If you would keep an eye open and report on the result that would be
great.

Bob

Re: [Savannah-hackers-public] bzr commit notifications [was Re: VM vcs upgraded (mostly)]

[Bob Proulx]

Glenn Morris wrote:
> Bob Proulx wrote:
> > I added that change to the script. I imagine we will need to wait
> > for the next commit before there is a notification. If you would
> > keep an eye open and report on the result that would be great.
> 
> Seems ok, thanks.
> 
> http://lists.gnu.org/archive/html/emacs-diffs/2013-05/msg00017.html
> 
> is still readable, and I get pretty colours when I view it in Gnus now.

Yay!

I think that was the last of the urgent issues for vcs.  Still more to
do but all of the services seem to be functioning correctly.  I will
move on to the download system before returning to it.

Bob

Re: [Savannah-hackers-public] Dragora

[Bob Proulx]

anthony wrote:
>   I am a LMDE user since the beginning. I have always wanted to run just
> straight Debian and yet the inability to change some things have kept me
> from it
>
> Have been doing all I can to install Dragora and it is not simple as I
> am doing it from a usb stick and cannot seem to create a path when it
> ask for mounted directory. Also I I find on the net is trouble with
> getting it in and no internet connection once it is loaded.

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

> So is there anywhere that i can get his information on mounting from a
> usb and how to get it to connect once installed.

But you are asking about Dragora GNU/Linux.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

Looking at http://www.dragora.org/ they suggest this:

  How To Get Help
  Communication channels:
  Mailing List
  To discuss or inform about bugs in Dragora, the mailing list
  dragora-...@nongnu.org is the best place. Please, send your messages
  in plain text, including a descriptive line in the subject of the
  email. If all the subjects are "error in Dragora" it's impossible to
  tell them apart.

  The mailing list archive is available in
  http://lists.nongnu.org/mailman/listinfo/dragora-bug

  IRC
  For a real time chat, you can join #dragora with an IRC client on the
  server irc.freenode.net. Also via web by visiting
  http://webchat.freenode.net/?channels=dragora

  Social Network
  The group, in the free social network Identica. 

> I like the Gnewsense as well which is going to Debian as its base
> (Excellent) with 3.0 and once again I have put it on a stick and cannot
> get it to even show a wireless connection to get on the web.
> Is there a workaround anywhere? 

You would have to ask the http://www.gnewsense.org/ folks.

Bob

Re: [Savannah-hackers-public] [gnu.org #826036] zope restore on frontend.savannah

[Bob Proulx]

Nicolás César via RT wrote:
> Give us a day and I'll restore the tapes labeled 2012-07-19
> 
> Sorry for the inconveniences! 

Thanks for working the problem.  Sorry for the hassle!

Bob

Re: [Savannah-hackers-public] bzr commit notifications

[Bob Proulx]

Glenn Morris wrote:
> Recently Savannah bzr was updated to bzr 2.6.
> This breaks the bzr-hookless-email script that was being used for
> commit notifications. [1]
> 
> To try and fix this, I suggest installing the bzr-email plugin.
> `apt-get install bzr-email' (the version from Debian testing is the
> latest version).

Thank you for researching this problem and proposing a solution.
That's awesome!

> IIUC, historically Savannah had some theoretical security concerns
> about bzr-email. I've looked at it and IMO the one issue that there
> might be can be trivially patched away. [2]
> But I don't really know why you went with bzr-hookless-email.

This was before my time and I have no information on it.

> If you are willing to install it, I can test it.

I am willing to give it a try.  Here are my comments.  There will be
enough delay that if there are other comments I will hear them before
I get this done.  But otherwise I will implement your plan as filed.

> [1] If you want the details,
> https://lists.ubuntu.com/archives/bazaar/2013q2/075520.html

With the exception of your comment there that the 2.6 wasn't in the
then stable Squeeze 6.  It is in the now stable Wheezy 7.  But vcs.sv
is still at this instant running Squeeze 6 and would normally have had
bzr 2.1.2.  The bzr that was installed on vcs.sv was not from the
Debian packages but from local compilation.

> [2] The only issue I can see is this:
> Anyone with write access to a bzr branch can set plugin options in
> .bzr/branch/branch.conf (this is actually good, because it means we
> will be able to control our own commit notifications without needing
> to bug Savannah admins).

If those are simply options I don't see the issue.  But with security
concerns it was probably possible to set configurations that executed
commands.  I assume.  I haven't looked.

> One option is "post_commit_mailer".
> This is either 'smtplib' (an internal Python library), or an external
> command like "/bin/mail".
> (I am assuming smtplib will be the correct option for Savannah.)

I don't think it matters.  My personal preference is to use the local
command.  I would use /usr/sbin/sendmail as a general local mailer
because that takes no configuration.  But /bin/mail or /usr/bin/mailx
would be okay too.  In that role they are effectively simply a wrapper
around /usr/sbin/sendmail.

Editorial comment: I think the general preference that we often see in
documentation for smtp is that people developing on an MS or Apple
client don't usually have an mta running.  Using smtp they can point
off to their mail relay and it works for them.  Using
/usr/sbin/sendmail only works on GNU/BSD/Unix machines.  But we have
one of those.  :-)

> Someone could try and set this to something nasty like "rm -rf /".
> So all we need to do is hard-code that option to "smtplib".
> Patching emailer.py is one trivial way to do that (see end).

That would be unpleasant.  Talking out the problem  It would run
on the server with the user:group of the person who is running the
commit, who is probably not the person who set up the attack.  The
permissions would be similar though.  It would take quite a long time
to traverse the entire filesystem slowing things down for a long time.
It would eventually remove the project files that are accessible by
that user:group.  It wouldn't have permission to remove other projects
files or anything else on the server.  Worse would be if a single user
were in several groups because then it would have permission to remove
all of the files in all of the groups of that user.

> Alternatively, I am told that options set in ~/.bzr/locations.conf
> will take precedence over branch options. If bzr on Savannah runs
> under a single user, that could be a better way to do it.

AFAIK write access is through ssh+bzr invoking sv_membersh which is a
restricted shell which screens incoming actions.  Each process is run
as a specific uid:gid and access is allowed or denied by group control.
There isn't a single process owner.  It is compartmentalized into
individually owned processes.

> If you want to review it, the code is at
> https://launchpad.net/bzr-email

If a python person would look that over it would be great.  I am not a
python person so must rely upon it being "obvious".

> I suggest a patch something like:
> 
> ***
> *** 206,212 
>   if mailer == 'smtplib':
>   self._send_using_smtplib()
>   else:
> ! self._send_using_process()
>   finally:
>   self.repository.unlock()
>   self.branch.unlock()
> --- 206,213 
>   if mailer == 'smtplib':
>   self._send_using_smtplib()
>   else:
> ! raise errors.BzrError("Bad value for post_commit_mailer")
> ! # self._send_using_process()
>   finally:
>   self.repository.unlock()
>   self.branch.unlock()

Seems okay.  Throw an 

Re: [Savannah-hackers-public] [Savannah-hackers-private] [gnu.org #830937] Correction Request gnu.org (savannah)

[Bob Proulx]

Ward Vandewege via RT wrote:
> Could you purge the spam from the bug linked below?
> http://savannah.gnu.org/bugs/?28247

Karl organized the deletion of it.  It's done now.

Bob

Re: [Savannah-hackers-public] plesase, help me

[Bob Proulx]

van quang wrote:
> I am unable  reading SMS message from Nokia 3110c cell phone
> I can send SMS success, but getsms failure.

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

> My file configure: /etc/gnokii:

But you are asking about the gnokii project.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

The project web site:

  https://savannah.nongnu.org/mail/?group=gnokii

Contains information about mailing lists.

Bob

> [glocal]
> port  = /dev/ttyACMO
> model = AT
> initlength = default
> connection = serial
> use_locking = yes
> serial_baudrate = 10
> [gnokiid]
> bindr = /usr/local/bin/
> [logging]
> debug = on
> rlpdebug = off
> xdebug = off
> 
>  # when I use model = 3110|3110c, it will got result with the following
> error: Device is locked and cannot be unlocked. (although, I restart OS)
> # if I use connection = dku2libusb , result following error: unsupported
> [global] connection value "dku2libusb"
> My OS is centos 6.3, gnokii version 0.6.31

[Savannah-hackers-public] web.vcs.savannah timeouts

[Bob Proulx]

Dora has noticed and reported that the viewcvs interface has been
producing timeouts at various times.

  http://web.cvs.savannah.gnu.org/viewcvs/

I put together a very brute force check by running wget against one of
the pages every five minutes and I have seen this so far.  During the
times below I have been seeing failures.

  Jul 12 00:30 - 02:10 -0600 Jul 13 00:35 - 02:05 -0600
  Jul 12 06:10 - 07:00 -0600 Jul 13 06:10 - 07:00 -0600
  Jul 12 10:05 - 10:10 -0600 Jul 13 10:05 - 10:30 -0600

And similary for today the 14th again too but I haven't been rigorous
about looking at the data yet.  But it certainly seems unhappy.  Too
similar from day to day to be coincidence so I think something is
going on there.

Just now I added a quick check to run a 'ps -efH' every five minutes
too.  Perhaps that will show something interesting in tonight's
overnight run.

Bob

Re: [Savannah-hackers-public] web.vcs.savannah timeouts

[Bob Proulx]

Bob Proulx wrote:
> Just now I added a quick check to run a 'ps -efH' every five minutes
> too.  Perhaps that will show something interesting in tonight's
> overnight run.

And in the last night's run I am seeing very large loads being
recorded on vcs.  I can see load averages up in the 20+ range for
extended periods of time.  I haven't had time to look at the ps dumps
yet to see which processes are driving the load there.

Bob

Re: [Savannah-hackers-public] lwip

[Bob Proulx]

K P KANZIYA wrote:
> i  am experienced  in 8051  and   PIC  families of microcontrollers .
> i have recently  stared  stm32f107. i   could  do  GPIO  ,LCD etc in  it .
> now  trying  to begin for ethernet  application.
> 
> i am just a beginner in ethernet . i have downloaded lwip stack. but
> could not understand how to use this materials ..please help..

You have reached the mailing list for people who hack on Savannah.
Savannah is the software forge for people committed to free software.
The web site, the version control hosting, that type of thing.

But you are asking about the lwip project.  We don't know too much
about specific projects here.  It would be better to look at the
project and see what it says about where to get help.

The project web site:

  https://savannah.nongnu.org/mail/?group=lwip

Contains information about the project mailing lists.

Bob

Re: [Savannah-hackers-public] ikiwiki on savannah

[Bob Proulx]

Karl Berry wrote:
> Bob, Michael, John, Ineiev, anyone -

Hi Karl!

> So, about getting ikiwiki installed, so the savannah documentation wiki
> can be updated again and we can move forward.

Yes.  This task has been hanging out for a while now.  I was out of
commission for a while.  But then afterward hadn't gotten back into
working through the todo list.  Sorry about that.  Things are looking
pretty good again for me though so I am ready to be involved again.

> Presumably it has to be installed on frontend, just as zope is now.

Yes.  It would install on frontend.

> Is there a Debian package for ikiwiki under Debian 6[.0]?  (Which is
> what frontend is running.)  Or is installing it from the original
> source the way to go?

Yes.  There is a Debian package for ikiwiki.  I haven't yet gone
through a complete installation and use of ikiwiki anywhere yet
though.  Therefore I am still missing some knowledge myself about how
ikiwiki works.  Since ikiwiki is a wiki compiler it isn't quite as
simple as doing an apt-get install ikiwki and having that be enough.
I think if I had installed it elsewhere already then I would know the
details well enough to proceed.  I think that for someone
knowledgeable in it that it really is as simple as installing the
package and then pointing Apache to it. 

> I imagine we will need subversion installed on frontend too -- at least
> I can't imagine any viable way to use the svn on the download vhost, nor
> does it even seem particularly desirable to me.  Having installed svn
> from original source for many years, I'm pretty sure that using the
> Debian svn package will be the path of least resistance there :).

I know you are not a fan of git but the path of least resistance would
be to use git for the ikiwiki backend.  And probably in either case it
would be to vcs.savannah for the source repository.

This seems to me to be one of the best features of ikiwiki.  It uses a
vcs as a backend.  Therefore in addition to making changes using the
web interface it is also possible for people with commit access to
make changes using the vcs backend directly.  That is you can check
out the source files of the wiki and make changes and commit them and
the changes appears on the wiki.  Plus it means that it avoids the
black box backend of most of the other wikis.  Backup and restore is
the same as backup and restore of any other source repository.  Which
would save us from the problems Savannah had with the previous wiki.

> I don't have much Debian experience.  Help?

I don't see ikiwiki setup being Debian specific.  I just haven't
installed and configured ikiwiki myself.  For me I need to install it
someplace in a scratch area and try it before doing it on the
production Savannah.

Has anyone on the list set up ikiwiki before?  Can I get a tutorial?
I have test servers available to play with so that we can work through
the issues.  Otherwise I will plan on setting up a victim test
installation where folks here can play with it and then we will all
"learn through it" together.

Bob

[Savannah-hackers-public] frontend VM rebooted

[Bob Proulx]

After 491 days of uptime I rebooted frontend.savannah just now.  Nico
stood by as a safety net.  No trouble.  All good.

Bob

Re: [Savannah-hackers-public] ikiwiki on savannah

[Bob Proulx]

Aljosha Papsch wrote:
> After the little discussion a few months back, I experimented a little
> with ikiwiki

Cool!

> The basic setup (which involves installing the package and running
> ikiwiki --setup) makes four directories / files (where  is the
> wiki name):

Thanks for the info.

> http://ikiwiki.info/setup/#index6h2 explains changing the locations
> pretty well. For our case, where the "master" git is somewhere remote,
> this explains it well which config files one needs to edit:
> http://ikiwiki.info/tips/Hosting_Ikiwiki_and_master_git_repository_on_different_machines/
>  

Perfect!  Just what is needed.

> Apart from moving directories around and editing the config files, we
> would also have to set up CGI in Apache (if it's not already done).

Apache will need some configuration for it.  But this part I think I
understand fairly well.  So I am not concerned about that part.  And
that part is also very easy to modify as we go.

> If you don't want to move directories around, ikiwiki also has some
> command line options, with which you can specify the directories.

I will look at those.

> What I forgot to mention: ikiwiki has really good themeability! When
> experimenting, I cooked up a little stylesheet which resembled the style
> used in Savannah's tracker messages. Unfortunately, the CSS file is long
> gone, along with the experiments I did. But it shouldn't be too hard to
> write it once more.

Excellent!

When you say resemble Savannah's tracker do you mean the gnats bug
tracker web pages?  Or something different?

I don't think it needs to match a particular part of Savane.  But it
would be good if it had a nice theme that made it fit into the overall
Savannah site.  A person can get whiplash if the colors and layout are
radically different between pages.

Bob

[Savannah-hackers-public] php xcache errors on mgt

[Bob Proulx]

Today the mgt.savannah node started producing a new error message.

  Failed loading /usr/lib/php5/20090626+lfs/xcache.so: 
/usr/lib/php5/20090626+lfs /xcache.so: cannot open shared object file: No such 
file or directory

That was due to this state:

  mgt:~# dpkg -l php5-xcache
  Desired=Unknown/Install/Remove/Purge/Hold
  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name   VersionDescription
  +++-==-==-
  rc  php5-xcache1.3.0-7Fast, stable PHP opcode cacher

It was installed today around that time and then removed.

  2013-08-19 19:21:20 install php5-xcache  1.3.0-7
  ...
  2013-08-19 19:34:53 remove php5-xcache 1.3.0-7 1.3.0-7

It waqs installed and then removed.  But with the configuration files
left behind the php session cleanup cron scripts complained.  This has
been a long standing issue with php.  No one has figured out how to
make the scripts optionally load components that aren't there anymore.

  mgt:~# php5 -r ''
  Failed loading /usr/lib/php5/20090626+lfs/xcache.so: 
/usr/lib/php5/20090626+lfs /xcache.so: cannot open shared object file: No such 
file or directory

Since the package was removed I simply purged it.

  mgt:~# dpkg --purge php5-xcache

That silenced the new cron messages.

Bob

[Savannah-hackers-public] Old Wiki Is Offline

[Bob Proulx]

Probably something did not come back up after the reboot today.  The
files for the old wiki are offline.  I am getting a 503 error Service
Temporarily Unavailable error.  I am trying to figure it out now.  But
if another Savannah Hacker wants to jump in and help that would be
great.  It is after midnight for me and I have an early morning alarm
tomorrow.

  https://savannah.gnu.org/maintenance/HowToBecomeASavannahHacker
  503 Service Temporarily Unavailable

Bob

Re: [Savannah-hackers-public] Old Wiki Is Offline

[Bob Proulx]

The problem is the same one Karl reported back in May:

http://lists.gnu.org/archive/html/savannah-hackers-public/2013-05/msg00028.html
> Then I ran /etc/init.d/zope2.10 restart.  (Several times, killing old
> processes, etc.)
> 
> The results were quite mysterious, with the wiki failing to come up at
> random times, but anyway, as things stand right now,
> http://savannah.gnu.org/maintenance/FrontPage looks like the old (good)
> version.  Whether it will persist through a reboot, I can't say.

And that comment was correct.  It did not survive through a reboot.

I have tried restarting zope several times.  But the daemon does not
start.  Nothing ever listens on port 9673.

  service zope2.10 restart

I traced through /etc/init.d/zope2.10 and see that it is executing this:

  cd /var/lib/zope2.10/instance
  dzhandle -z 2.10 zopectl default start

It then emits:

  . daemon process started, pid=19317

But that pid is never alive afterward.  I don't see anything being
logged to the /var/log/zope2.10/default/* files.

Bob

Re: [Savannah-hackers-public] our wiki status

[Bob Proulx]

Karl Berry wrote:
> - the ikiwiki package is installed and basically configured on frontend.

Thank you Karl for doing this work!  It's awesome!

Bob

Re: [Savannah-hackers-public] ok to install "locales" package on vcs.sv?

[Bob Proulx]

> Anyone know of a reason why I should not `apt-get install locales' on
> vcs.sv? I'd leave the default locale as "None", as it is now.

I think every system should have the locales package installed.  So
definitely installing it is the right thing to do.

(BTW: PAM uses the default locale setting for things that "log in" to
the system.  So in theory as I understand it the default locale should
only affect people who log in by some method through PAM.  This area
has been changing in recent years though and so I might be wrong now.
The desire would be to have the default as None which maps to C as you
have noted.)

> It actually looks like the Savannah gecos data (real names etc) is in
> latin-1 rather than utf-8; does anyone know if that is indeed the case?
>
> I do not know, but it wouldn't surprise me.

Probably an artifact of history.  As far as I know there is a
goal/vision in the Debian system that everything will be uniformly
UTF-8.  Therefore I assume that the gecos names would also be assumed
to be in UTF-8.  But it would have been easy for a cut-n-paste error
from history to have left some latin-1 encodings behind.

I think it would be reasonable to convert the encoding from latin-1 to
utf-8 if you had the time and resources to do it.  That would be great
and it would clean that part up.

Bob

Re: [Savannah-hackers-public] rsync process load on vcs

[Bob Proulx]

Karl Berry wrote:
> I know Bob et al. were gathering stats on frontend, but I didn't see
> anything on vcs.

The firewalls on the machines were blocking me from setting it up on
vcs.  I am going to open the firewalls to allow it.  Originally I
didn't understand how they were set up.  But now I understand how they
were implemented on savannah and am comfortable making changes to them
now.  This should get us stats on vcs.

Bob

Re: [Savannah-hackers-public] rsync process load on vcs

[Bob Proulx]

Bob Proulx wrote:
> Karl Berry wrote:
> > I know Bob et al. were gathering stats on frontend, but I didn't see
> > anything on vcs.
> 
> The firewalls on the machines were blocking me from setting it up on
> vcs.  I am going to open the firewalls to allow it.  Originally I
> didn't understand how they were set up.  But now I understand how they
> were implemented on savannah and am comfortable making changes to them
> now.  This should get us stats on vcs.

Done.  Munin stats are now available for vcs.  And since last night
they do show some very high load spikes.  It will be useful to see a
day or so of data collected to see the trend.

Bob

Re: [Savannah-hackers-public] savane git cannot be committed

[Bob Proulx]

Tomasz Konojacki wrote:
> I was trying to login to vcs.sv.gnu.org as a root, but I still
> get "access denied". I think that something with the key
> must be messed up.

To log into vcs, first log into mgt.  Then from mgt log into vcs.  To
log into mgt, first log into fencepost.  There is a firewall on all of
the VMs that prevents direct access from the Internet.

Bob

Re: [Savannah-hackers-public] savane git cannot be committed

[Bob Proulx]

Karl Berry wrote:
> To log into vcs, first log into mgt.  
> 
> He can't.  He doesn't have a static IP address (as far as I know --
> Tomasz, if I'm wrong, let me know).  I asked him to ask sysadmin for an
> account on fencepost but I don't think that has happened yet.

But should be able to get to fencepost as I described.

> There is a firewall on all of the VMs that prevents direct access
> from the Internet.
> 
> At least at the iptables level, that is not true for vcs, which is
> precisely why I (tried to) set Tomasz up the way I did.  It allows ssh
> from anywhere since that is necessary for some VC access.

Ah!  Yes you are right.  I just tried it now and of course it worked.
Otherwise version control can't work.  (My hand smacks my forehead.)
A silly mistake on my part.  Sorry for the noise.  I was confused
because each VM has its own iptables rules that are unique.

> However, do you know if there is some other "firewall"-type thing
> (whatever that means) that would stop "ssh r...@vcs.sv.gnu.org" at the
> user level (not the ssh level), resulting in the usual "Permission
> denied (publickey)."?  Tomasz hasn't been able to log in yet.  All very
> frustrating.

Nope.  And in fact I tested this and it worked okay for me just now.
Which would lead back to a problem on the Tomasz's client side again.

Bob