Fwd: git-remote-fd problem

2014-12-29 Thread Jiri Sevcik
 The remote-fd expects the transport to pass half-closes. So you can't
 close all at once.

 Let there be pipes W and R and transport connection C.

 - W-read should be closed after being passed to remote-fd.
 - R-write should be closed after being passed to remote-fd.
 - Upon receiving no more data from C, close W-write.
 - Upon receiving EOF from R-read, close it and signal no more data
   to C.

Hi, I followed your advices, correctly close pipes but git clone still
doesnt finish and hanging on.
Code is in an attachement (its part of big system).

#create pipes
w_pipe = os.pipe()
r_pipe = os.pipe()

client_process = subprocess.Popen(/usr/bin/git clone fd::{0},{1} /tmp/gittest.format(r_pipe[0], w_pipe[1]), shell=True)
#closing pipes
os.close(r_pipe[0]) 
os.close(w_pipe[1])

epoll = select.epoll()
epoll.register(w_pipe[0], select.EPOLLIN)
epoll.register(proc.fd, select.EPOLLIN)

remoteGit = proc.runDaemon(git-upload-pack /tmp/testgit)

while True:
events = epoll.poll(1)

for fd, event in events:
if fd == w_pipe[0]:
if event  select.EPOLLIN:
rd = os.read(w_pipe[0], 1)
if rd:
#write data to remove git server
remoteGit.writeToChannel(rd)
else:
proc.writeError(Local socket write error)
return 1
else:
proc.writeError(Local socket error)
return 1

elif fd == proc.fd:
if event  select.EPOLLIN:
#read data from remote git server
data = remoteGit.getAll()
remoteGit.stderrWrite()

if not data:
#remote server send EOF, close local pipe
#but git clone is still running
os.close(r_pipe[1])
return 0

want = len(data)

writed = 0
offset = 0

while(writed != want):
#write data from remote git server to local pipe
wr = os.write(r_pipe[1], data[offset:])

if(wr  0):
return 1

writed += wr
offset += wr

else:
return -1  


Re: Fwd: git-remote-fd problem

2014-12-29 Thread Ilari Liusvaara
On Mon, Dec 29, 2014 at 10:47:58AM +0100, Jiri Sevcik wrote:
  The remote-fd expects the transport to pass half-closes. So you can't
  close all at once.
 
  Let there be pipes W and R and transport connection C.
 
  - W-read should be closed after being passed to remote-fd.
  - R-write should be closed after being passed to remote-fd.
  - Upon receiving no more data from C, close W-write.
  - Upon receiving EOF from R-read, close it and signal no more data
to C.
 
 Hi, I followed your advices, correctly close pipes but git clone still
 doesnt finish and hanging on.
 Code is in an attachement (its part of big system).

Few ideas:
- Check that git clone (and its subprocesses) don't inherit
w_pipe[0] (/proc/pid/fd on Linux might be handy). If they do, that
prevents this program from closing the pipe.

- Setting environment variable GIT_TRANSLOOP_DEBUG to 1 might make
  git spew lots of messages to stderr about reads, writes and closes.

 
 #create pipes
 w_pipe = os.pipe()
 r_pipe = os.pipe()
 
 client_process = subprocess.Popen(/usr/bin/git clone fd::{0},{1} 
 /tmp/gittest.format(r_pipe[0], w_pipe[1]), shell=True)
 #closing pipes
 os.close(r_pipe[0]) 
 os.close(w_pipe[1])
 
 epoll = select.epoll()
 epoll.register(w_pipe[0], select.EPOLLIN)
 epoll.register(proc.fd, select.EPOLLIN)
 
 remoteGit = proc.runDaemon(git-upload-pack /tmp/testgit)
 
 while True:
 events = epoll.poll(1)
 
 for fd, event in events:
 if fd == w_pipe[0]:
 if event  select.EPOLLIN:
 rd = os.read(w_pipe[0], 1)
 if rd:
 #write data to remove git server
 remoteGit.writeToChannel(rd)
 else:
 proc.writeError(Local socket write error)
 return 1
 else:
 proc.writeError(Local socket error)
 return 1
 
 elif fd == proc.fd:
 if event  select.EPOLLIN:
 #read data from remote git server
 data = remoteGit.getAll()
 remoteGit.stderrWrite()
 
 if not data:
 #remote server send EOF, close local pipe
 #but git clone is still running
 os.close(r_pipe[1])
 return 0
 
 want = len(data)
 
 writed = 0
 offset = 0
 
 while(writed != want):
 #write data from remote git server to local pipe
 wr = os.write(r_pipe[1], data[offset:])
 
 if(wr  0):
 return 1
 
 writed += wr
 offset += wr
 
 else:
 return -1  

-Ilari
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html