Re: cvs2gitdump dumps core when trying process src

[Dinesh Thirumurthy]

Dear Yasuoka San, and Stuart Henderson,

> Last argument for cvs2gitdump should be a CVS repository, not source
> code.

Thanks very much.

It worked well. It took about 2.5 hours and generated 27GB of "git dump"
data.

box$ pwd
/home/user/rsync
box$ cvs2gitdump -k OpenBSD -e openbsd.org cvs0 > ob.dump
** walk cvs tree
Ignore cvs0/ports/audio/opencore-amr/.git: cannot handle the path named
'.git'
Ignore cvs0/ports/devel/log4cplus/.git: cannot handle the path named
'.git'
** cvs has 346530 changeset
** dumped
box$ ls -l ob.dump
-rw-r--r--  1 user  user  28981973872 Dec 25 01:19 ob.dump
box$ du -h *.dump
27.0G   ob.dump
box$ ^D

The git import also worked very well. Thanks.

Script started on Mon Dec 25 01:29:43 2017
box$ pwd
/home/user/rsync
box$ ls -l
total 56612448
drwxr-xr-x  7 user  user  512 Dec 24 18:30 cvs0
-rw-r--r--  1 user  user  28981973872 Dec 25 01:19 ob.dump
box$ file ob.dump
ob.dump: ASCII English text
box$ wc -l ob.dump
 881911531 ob.dump
box$ git init --bare git0.git
Initialized empty Git repository in /home/user/rsync/git0.git/
box$ git --git-dir git0.git fast-import < ob.dump
/usr/local/libexec/git/git-fast-import statistics:
-
Alloc'd objects:3685000
Total objects:  3681554 (240725 duplicates  )
  blobs  :  1405136 (231986 duplicates1212707 deltas of
1391574 attempts)
  trees  :  1929889 (  8739 duplicates1791569 deltas of
1901228 attempts)
  commits:   346529 ( 0 duplicates  0 deltas of
0 attempts)
  tags   :0 ( 0 duplicates  0 deltas of
0 attempts)
Total branches:  65 ( 1 loads )
  marks: 1073741824 (   1983651 unique)
  atoms: 163649
Memory total:211233 KiB
   pools: 38499 KiB
 objects:172734 KiB
-
pack_report: getpagesize()=   4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit  = 35184372088832
pack_report: pack_used_ctr=192
pack_report: pack_mmap_calls  = 64
pack_report: pack_open_windows=  1 /  1
pack_report: pack_mapped  =  588175141 /  588175141
-

box$ ^D

Script done on Mon Dec 25 02:41:14 2017

This is my/user/operator error. 

I thought it would be good for cvs2gitdump
to complain if user specified cvsrepo argument
a. does not exist
b. is not a directory
c. does not have a CVSROOT subdirectory (I am assuming every CVS
repository has one. I could be wrong).

Then there is good chance the user is not making an error. 
So, I thought it might be a good idea to add that:

https://github.com/yasuoka/cvs2gitdump/compare/master...hakrtech:master

Kindly consider and let me know what you think.

Thank you and Stuart Henderson for helping me get past this problem.

Osoreirimasu. Thank you very much.

Regards,
Dinesh

Re: rdomain/rtable

[Scott Nicholas]

Hello

You may need a direct route to the gateway as well

Happy holidays,
Scott


On Dec 24, 2017 4:08 PM, "Paul B. Henson"  wrote:

Thanks for the info. I don't want to move any interfaces to a
non-default routing domain, I just want to be able to run a process with
a different default route. I can make that work, via the route -T 10
exec you mention after setting a default route in that domain.

But I can't seem to get traffic for my local subnet sent out my
internal interface, even after I add a route to it in the non-default
routing domain. Dunno, maybe I'm missing something.

I set it up like:

Internet:
DestinationGatewayFlags   Refs  Use   Mtu  Prio
Iface
default24.x.x.x  UGS02 - 8 umb0
10.0/1610.128.0.20UGS00 - 8 em0

But 'ping 10.128.0.20' shows the packets going out umb0, not em0?

Thanks again.

On Sat, Dec 23, 2017 at 05:07:37PM +0100, Sebastian Benoit wrote:
>
> When you create a new routing domain, for example by adding an interface
to
> a routing domain (e.g. ifconfig umb0 rdomain 10), you create a new routing
> table 10. It will be empty until you add an address on umb0 or, for
example
> add your default route.
>
> This routing table will be used to forward packets that are "in that
routing
> domain" (the packet is marked with the rdomain or rather the rtable it
will
> use). How does the packet get marked?
>
> Three ways:
>
> * with pf, as you have discovered. As the manpage documents, the
> mark needs to be set before route lookup is done.
>
> * when a paket comes in on an interface in rdomain 10, it will stay in
> rdomain 10 (unless pf changes it).
>
> * a packet is generated on the local machine by a process that "is in that
> routing domain". I.e. processes are also marked with a rdomain.
>
> To start a process in a specific rdomain (10), use "route -T 10 exec
> command", for example
>
>   route -T 10 exec ping -n ip
>
> or even
>
>   route -T 10 exec ksh
>
> Processes spawned by that shell will inherit the rdomain.
>
> Note that i used -n in the ping example. DNS resolving using the resolvers
> in resolv.conf might not work, as long as those resolvers are not
reachable
> in rdomain 10.
>
> Hope this helps ...

Re: rdomain/rtable

[Paul B. Henson]

Thanks for the info. I don't want to move any interfaces to a
non-default routing domain, I just want to be able to run a process with
a different default route. I can make that work, via the route -T 10
exec you mention after setting a default route in that domain.

But I can't seem to get traffic for my local subnet sent out my
internal interface, even after I add a route to it in the non-default
routing domain. Dunno, maybe I'm missing something.

I set it up like:

Internet:
DestinationGatewayFlags   Refs  Use   Mtu  Prio Iface
default24.x.x.x  UGS02 - 8 umb0
10.0/1610.128.0.20UGS00 - 8 em0

But 'ping 10.128.0.20' shows the packets going out umb0, not em0?

Thanks again.

On Sat, Dec 23, 2017 at 05:07:37PM +0100, Sebastian Benoit wrote:
> 
> When you create a new routing domain, for example by adding an interface to
> a routing domain (e.g. ifconfig umb0 rdomain 10), you create a new routing
> table 10. It will be empty until you add an address on umb0 or, for example
> add your default route.
> 
> This routing table will be used to forward packets that are "in that routing
> domain" (the packet is marked with the rdomain or rather the rtable it will
> use). How does the packet get marked?
> 
> Three ways:
> 
> * with pf, as you have discovered. As the manpage documents, the
> mark needs to be set before route lookup is done.
> 
> * when a paket comes in on an interface in rdomain 10, it will stay in
> rdomain 10 (unless pf changes it).
> 
> * a packet is generated on the local machine by a process that "is in that
> routing domain". I.e. processes are also marked with a rdomain.
> 
> To start a process in a specific rdomain (10), use "route -T 10 exec
> command", for example
> 
>   route -T 10 exec ping -n ip
> 
> or even
> 
>   route -T 10 exec ksh
> 
> Processes spawned by that shell will inherit the rdomain.
> 
> Note that i used -n in the ping example. DNS resolving using the resolvers
> in resolv.conf might not work, as long as those resolvers are not reachable
> in rdomain 10.
> 
> Hope this helps ...

Re: relayd stops processing traffic intermittently

[Claudio Jeker]

On Sat, Dec 23, 2017 at 02:04:19PM +0100, Mischa Peters wrote:
> 
> > On 23 Dec 2017, at 13:08, Claudio Jeker  wrote:
> > 
> >> On Sat, Dec 23, 2017 at 11:40:57AM +0100, Mischa wrote:
> >> Hi All,
> >> 
> >> Since OpenBSD 6.2, just confirmed this in the latest snapshot 
> >> (GENERIC.MP#305) as well, for some reason relayd stops processing traffic 
> >> and starts flooding the log file with the following message:
> >> 
> >> Dec 23 11:19:11 lb2 relayd[22515]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:12 lb2 relayd[52110]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:12 lb2 relayd[69641]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:12 lb2 relayd[22515]: rsae_send_imsg: poll timeout
> >> [snip]
> >> Dec 23 11:19:17 lb2 relayd[69641]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:18 lb2 relayd[22515]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:18 lb2 relayd[52110]: rsae_send_imsg: poll timeout
> >> Dec 23 11:19:18 lb2 relayd[69641]: rsae_send_imsg: poll timeout
> >> ...etc...
> >> 
> >> Restarting the daemon "fixes" the problem.
> >> Not sure how to trouble shoot this but I am able to reproduce this 
> >> consistently by pointing SSLLabs towards relayd.
> >> Would be great to get some pointers.
> >> 
> > 
> > I have seen this as well on our production systems. This is a problem in
> > the privsep part of the TLS code. I could not do more testing yet but my
> > assumption is that a new option / feature is freaking this code out.
> 
> Anything I can do or collect to give you more information? 

So, I think I found the problem. The ca process did not handle errors from
RSA_private_encrypt correctly. So once you got a bad signature in the
system chocked and stopped. This diff seems to work for me (against
SSLlabs).

Cheers
-- 
:wq Claudio

Index: ca.c
===
RCS file: /cvs/src/usr.sbin/relayd/ca.c,v
retrieving revision 1.31
diff -u -p -r1.31 ca.c
--- ca.c28 Nov 2017 00:20:23 -  1.31
+++ ca.c24 Dec 2017 18:01:20 -
@@ -266,9 +266,15 @@ ca_dispatch_relay(int fd, struct privsep
break;
}
 
+   if (cko.cko_tlen == -1) {
+   char buf[256];
+   log_warnx("%s: %s", __func__,
+   ERR_error_string(ERR_get_error(), buf));
+   }
+
iov[c].iov_base = 
iov[c++].iov_len = sizeof(cko);
-   if (cko.cko_tlen) {
+   if (cko.cko_tlen > 0) {
iov[c].iov_base = to;
iov[c++].iov_len = cko.cko_tlen;
}
@@ -381,12 +387,12 @@ rsae_send_imsg(int flen, const u_char *f
 
IMSG_SIZE_CHECK(, ());
memcpy(, imsg.data, sizeof(cko));
-   if (IMSG_DATA_SIZE() !=
-   (sizeof(cko) + cko.cko_tlen))
-   fatalx("data size");
 
ret = cko.cko_tlen;
-   if (ret) {
+   if (ret > 0) {
+   if (IMSG_DATA_SIZE() !=
+   (sizeof(cko) + ret))
+   fatalx("data size");
toptr = (u_char *)imsg.data + sizeof(cko);
memcpy(to, toptr, ret);
}

Re: Picking the nearest (not necessarily fastest) anoncvs server

[Dinesh Thirumurthy]

Peter,
 
 Thanks very much! 

> When I download from Canada and the transit goes through the USA it is
> downloading (and thus exporting) from Canada and not the USA.

Thanks for the clarification! I always viewed it as taking a CD from
Canada to USA to France to India. Too bad USA has Hotel California
policy on OpenBSD. 

"We are programmed to receive 
 You can check out any time you like 
 But OpenBSD can never leave!"

Never thought of links+routers (flights+airports) being pass through
staging points.

This whole thing reminds of the "medicine for goat" scene in the movie
Terminal.

A Bulgarian is taking some medicine from Canada, to his hometown in
Bulgaria. That medicine is impounded by customs is New York Airport.
He is transiting through New York because of some law.

https://www.youtube.com/watch?v=xrTUmYxnNlo

Just kind of reminds of this OpenBSD/Canada/USA combination.
 
> That said, have a merry christmas!

Wish you and all on this list a merry christmas!

Thanks again.

Regards,
Dinesh

Re: cvs2gitdump dumps core when trying process src

[Dinesh Thirumurthy]

Yasuoka San,

  Doumo arigatou gozaimasu.

> src1 seems to be a check outted source code.
> Last argument for cvs2gitdump should be a CVS repository, not source
> code.
> To get a copy of OpenBSD CVS repository, you can see
> http://www.openbsd.org/cvsync.html

 You made my day and also made me realise I need to know cvs.
 No getting around it. 

 Osoreirimasu.

Regards,
Dinesh

https://www.livinglanguage.com/blog/2013/06/20/how-to-say-thank-you-in-japanese-the-9-expressions-you-need/

Re: cvs2gitdump dumps core when trying process src

[YASUOKA Masahiko]

Hi,

On Sun, 24 Dec 2017 03:32:05 +0530
Dinesh Thirumurthy  wrote:
>> The conversion on github is done with cvs2gitdump. 
> 
> git2cvsdump dumps core on latest current.
> I am stumped after some basic investigation.
> 
> /usr/local/bin/cvs2gitdump dumps core.
(snip)
> What I did:
> 
> mkdir x
> cd x
> cvs -qd anon...@anoncvs.jp.openbsd.org:/cvs checkout -P src
> mv src src0 # save a copy for later use
> cp -r src0 src1 # use a copy of the repo

src1 seems to be a check outted source code.

> pkg_add -vvv cvs2gitdump 
> # follow instructions given in source also at
> https://github.com/yasuoka/cvs2gitdump/blob/master/cvs2gitdump.py
> git init --bare git1.git
> cvs2gitdump -k OpenBSD -e openbsd.org /home/user/x/src1 > openbsd.dump

Last argument for cvs2gitdump should be a CVS repository, not source
code.

To get a copy of OpenBSD CVS repository, you can see
http://www.openbsd.org/cvsync.html

--yasuoka