Re: [fossil-users] ForceCommand SSH key with fossil?

[Andy Bradford]

Thus said Andy Bradford on 29 Jun 2013 20:34:07 -0600:

 So it  seems to think  that the server  didn't respond, but  it surely
 looks like it did. Will this approach even work?

Ok,  the problem  was that  the connection  was closed  (due to  missing
keep-alive, so SSH closed connection)  in xfer.c:client_sync. I was able
to workround it by moving  the call to transport_global_startup() at the
top of the  while ( go ) loop, and  adding a transport_global_shutdown()
at the  bottom. This now seems  to work, but  it seems like it's  no the
cleanest solution:

$ fossil clone --sshserver ssh://amb@localhost//tmp/temp.fossil test.fossil
ssh -e none -T amb@localhost
Round-trips: 1   Artifacts sent: 0  received: 1
ssh -e none -T amb@localhost
Round-trips: 2   Artifacts sent: 0  received: 2
Clone finished with 487 bytes sent, 1119 bytes received
Rebuilding repository meta-data...
  100.0% complete...
project-id: b3fa0981e117c6a57a0850875f58973eec3a6453
admin-user: amb (password is 3c1aaf)

But it fails at this point because it doesn't know about the --sshserver
option:

$ fossil open test.fossil 
$ touch file
$ fossil add file
ADDED  file
$ fossil ci -m test
Autosync:  ssh://amb@localhost//tmp/temp.fossil
ssh -e none -T amb@localhost
$ echo $?
141

So somehow it  needs to be taught that when  autosync'ing, it should use
--sshserver.

Guess I'll have to  leave that for another day... Does  this seem like a
worthwhile addition? It seems that  it would make handling SSH tunneling
configurations much easier---no need  to worry about parsing interactive
shell responses.

Thoughts?

Thanks,

Andy
--
TAI64 timestamp: 400051cfda10
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] ForceCommand SSH key with fossil?

[Rene]

On 2013-06-30 02:19, Andy Bradford wrote:

Hello,

Next hurdle.  SSL or SSH on Windows.

I just  discovered that fossil.exe  does not  have SSL support,  so 
that
kind of presents a hurdle. By the  way, it works nicely in a chroot 
with

stunnel and SSL client certificates on an OpenBSD server.

So, now on to SSH...

I'm trying to setup a force  command with ssh keys that restricts 
access

with that key to simply ``fossil http REPOSITORY''.

Is there  some trick I need  to tell the  client that it already  has 
an
open fossil http  server waiting to be used on  stdin/stdout and to 
just
start talking  HTTP? SSH will have  already taken care of  wiring up 
the
stdin/stdout  on the  server side,  so this  really just  seems to  be 
a

client side problem.

I setup a ForceCommand SSH key but  the client doesn't seem to know 
what

to do with it and crashes:

$ fossil ver
This is fossil version 1.26 [c9cb6e7293] 2013-06-18 21:09:23 UTC
$ fossil clone ssh://amb@localhost//tmp/test.fossil test.fossil
ssh -e none -T amb@localhost
...
debug1: Offering RSA public key: /home/amb/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: Authentication succeeded (publickey).
...
debug1: channel 1: new [client-session]
debug1: Entering interactive session.
debug1: Remote: Forced command.
debug1: Remote: Forced command.
...
debug1: Exit status 0
$ echo $?
141

As can be seen,  when my SSH key is used, it will  be forced into 
fossil
http  mode,  but  the  client  crashes.  Here  is  what  I  have  in  
my

authorized_keys:

$ grep fossil /home/amb/.ssh/authorized_keys
command=/home/amb/bin/fossil http /tmp/test.fossil ssh-rsa
B3NzaC1yc2EDAQABAAABAQDqKI393xubO69Rs+Y6fbnxDpAgX5kTe31qWepFyfu08wxNj5iX57vNIYFIcC7bwKW4EHMakuEIYr2eDzXl3e1pXLzteFESISaZkXrmspNIMRh4oW/3LqV+pGXfimA//YlmbJOMzEHerSCTi+QG0O6LNyvjlZgJmP8dJgc0ktzw6nAVcpdFxwoNa+tQJb+g7wLHGRCsl9uvf6rfdzXVUm/tAtD/TyPITU7Ni2q7aTm/m8YKsXDUif91UP9XUH8phwwEucQa3MagtIcmUKJzrkuwHT+rr2K/0W8vpjO3iq3g7ejONqaTfqEW2Rc5uydYsc1B5IjsmPm0bVkbB3B6ZBxF
amb

Is there any  way to tell the  local fossil client command  that it 
just
needs to start talking HTTP to  whatever file descriptor it has open 
for
SSH? Or,  is there  another mode that  fossil has that  will do  this? 
I

couldn't find it in the documentation.

By the way, SSH keys, and  just SSH with password authentication do 
work
for cloning, but I want to restrict the access that users make to 
simply
the fossil  command (no  shell access), but  if I just  use the  SSH 
key

without restricting the command, they gain full shell access.

Thanks,

Andy


One possibility seems to be to use the urlShell :
Wed, Feb 6, 2013 at 5:31 PM, Richard Hipp drh at sqlite.org wrote:


I want to understand the problem before I put in the fix.

To try to help better understand what is happening, I have added a new 
query parameter to the ssh: url scheme.  You can now say:


fossil clone ssh://user@host/path/to/repo?shell=/bin/bash 
new.fossil


and that will cause Fossil to add the /bin/bash argument to the end 
of the ssh command.  Please note that you can also do
--sshtrace to get some interactive information on what the ssh command 
is doing.


Tell you users that they can only acces fossil with:
//user@host/path/to/repo?shell=/andy/sh

make the line in authorized hosts

command=/home/amb/bin/forcedcommand.sh

check in the script if the command is /andy/sh in SSH_ORIGINAL_COMMAND.
 If so exec /bin/sh, otherwise close the connection.

S
Look in SSH_ORIGINAL_COMMAND. probably this is always /bin/sh or 
something like that.

Maybe you can check on the ip number if its trusted and then you know.
If not echo back something like you this is me and I want to know who 
you are

If it's fossil it wil send you


then make the dance with the fossil client.

--
Rene
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] ForceCommand SSH key with fossil?

[Rene]

On 2013-06-30 15:24, Rene wrote:
[snip]

Sorry this should have not been part of the message.
It could be an alternative way. But the urlShell seems to be much 
easier!




S
Look in SSH_ORIGINAL_COMMAND. probably this is always /bin/sh or
something like that.
Maybe you can check on the ip number if its trusted and then you know.
If not echo back something like you this is me and I want to know who 
you are

If it's fossil it wil send you


then make the dance with the fossil client.


--
Rene
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Importing moderately sized repositories

[Isaac Jurado]

I'm playing with ways to easily reduce the tame it takes to import a
moderately sized repository (around 50MB in git).

My first attempt looks like this:

Index: src/import.c
==
--- src/import.c
+++ src/import.c
@@ -731,10 +731,11 @@
   char *zPassword;
   FILE *pIn;
   Stmt q;
   int forceFlag = find_option(force, f, 0)!=0;
   int incrFlag = find_option(incremental, i, 0)!=0;
+  int fastFlag = find_option(fast, 0, 0)!=0;
 
   find_option(git,0,0);  /* Skip the --git option for now */
   verify_all_options();
   if( g.argc!=3   g.argc!=4 ){
 usage(REPOSITORY-NAME);
@@ -749,10 +750,18 @@
 if( forceFlag ) file_delete(g.argv[2]);
 db_create_repository(g.argv[2]);
   }
   db_open_repository(g.argv[2]);
   db_open_config(0);
+
+  if( fastFlag ){
+db_multi_exec(
+PRAGMA journal_mode=OFF;
+PRAGMA synchronous=OFF;
+PRAGMA temp_store=MEMORY;
+);
+  }
 
   /* The following temp-tables are used to hold information needed for
   ** the import.
   **
   ** The XMARK table provides a mapping from fast-import marks and symbols

However, it doesn't seem to make much difference.  Is there any other
SQLite tunable that can improve performance even at the cost of
resistance to hardware failure?

Thanks in advance.

-- 
Isaac Jurado

The noblest pleasure is the joy of understanding.
  Leonardo da Vinci
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Why delta manifests?

[Isaac Jurado]

Hello all,

I've been working further on improving the import command.  This time I
played with the generation of delta manifests with the same heuristic
used by the commit command.

But, after comparing with and without delta manifests, I noticed that
even though the total raw size of the artifacts was indeed reduced, the
real disk size was not.

After the results, the conclusion is obvious: the generic artifact delta
compression is outperforming the delta manifest convention.

So the question is, what is the rationale behind delta manifests?  After
checking Fossil's history, it looks like they were added after artifact
delta compression.  Which actually seems even more confusing.

Best regards.

-- 
Isaac Jurado

The noblest pleasure is the joy of understanding.
  Leonardo da Vinci
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Importing moderately sized repositories

[Eduardo]

On Sun, 30 Jun 2013 23:37:03 +0200
Isaac Jurado dipto...@gmail.com wrote:

 I'm playing with ways to easily reduce the tame it takes to import a
 moderately sized repository (around 50MB in git).
 
 My first attempt looks like this:
 
 Index: src/import.c
 ==
 --- src/import.c
 +++ src/import.c
 @@ -731,10 +731,11 @@
char *zPassword;
FILE *pIn;
Stmt q;
int forceFlag = find_option(force, f, 0)!=0;
int incrFlag = find_option(incremental, i, 0)!=0;
 +  int fastFlag = find_option(fast, 0, 0)!=0;
  
find_option(git,0,0);  /* Skip the --git option for now */
verify_all_options();
if( g.argc!=3   g.argc!=4 ){
  usage(REPOSITORY-NAME);
 @@ -749,10 +750,18 @@
  if( forceFlag ) file_delete(g.argv[2]);
  db_create_repository(g.argv[2]);
}
db_open_repository(g.argv[2]);
db_open_config(0);
 +
 +  if( fastFlag ){
 +db_multi_exec(
 +PRAGMA journal_mode=OFF;
 +PRAGMA synchronous=OFF;
 +PRAGMA temp_store=MEMORY;
 +);
 +  }
  
/* The following temp-tables are used to hold information needed for
** the import.
**
** The XMARK table provides a mapping from fast-import marks and symbols
 
 However, it doesn't seem to make much difference.  Is there any other
 SQLite tunable that can improve performance even at the cost of
 resistance to hardware failure?

a) Increase cache_size to 500-1GB RAM (negative values in the pragma)
b) I use 2 steps: 

b1) git fast-export  xz -vv --lzma2=preset3,dict=256MiB git.xz
b2) xzcat -vv git.xz  fossil import git.fossil

The -vv on b2 step shows you the %done, speed of import in KB/sec, compression 
rate and estimated time to finish. git.xz and git.fossil archives can be in 
different disks.

c) In manifest.c you can set #define MX_MANIFEST_CACHE 6 to bigger value, like 
64. Bigger values may be a waste because it may take less recalculate than 
linear search the manifest cache.

In my tests, importing freebsd git repository using b) method, setting 
cache_size to 2GB was the best. 

If left as default (cache size=2000 pages), fossil drops to 17-20% of cpu 
waiting for IO. The db file size was 230 GB before rebuilding. In speed (with 
default cache), it began with 3MB/sec and ends with 240KB/sec (of uncompressed 
data).

Note that I didn't used git fast-export --all.

 Thanks in advance.
 
 -- 
 Isaac Jurado

-- 
Eduardo emorr...@yahoo.es
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users