Re: [sqlite] (dot) output call locks up when used on a named pipe

[Rowan Worth]

On 1 January 2017 at 08:55, James K. Lowden 
wrote:

> On Sat, 31 Dec 2016 15:16:19 -0500
> Paul Lambert  wrote:
>
> > I have used the (dot) .output call in conjunction with a both a file
> > and name pipe on Linux with Sqlite 3.13 installed.  With a file this
> > functions successfully  and completes.  When using a named pipe this
> > functions locks up sqlite and it must be aborted for termination.
>
> Is something draining the pipe?  If not, SQLite will naturally block
> when the pipe fills.
>

Yeah, this. Even *opening* a named pipe for writing blocks until a reader
shows up (in linux at least). You can easily observe this behaviour:

$ mkfifo /tmp/fifo
$ echo hello > /tmp/fifo

Notice that you don't get the prompt back immediately like you normally
would when running echo. But if you open another terminal and do:

$ cat /tmp/fifo

You'll see "hello" in the new terminal and get the prompt back in your
first terminal.

In my testing this is consistent with the behaviour of sqlite3's .output
command.
-Rowan
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (dot) output call locks up when used on a named pipe

[James K. Lowden]

On Sat, 31 Dec 2016 15:16:19 -0500
Paul Lambert  wrote:

> I have used the (dot) .output call in conjunction with a both a file
> and name pipe on Linux with Sqlite 3.13 installed.  With a file this
> functions successfully  and completes.  When using a named pipe this
> functions locks up sqlite and it must be aborted for termination.

Is something draining the pipe?  If not, SQLite will naturally block
when the pipe fills.  

> I have observed that the file write mode of this function when using
> the same file name a second time will delete the previous data in the
> file. This is most likely accomplished by deleting the file and
> recreating a new one.  

Better than guessing is strace(1).  That would have shown you the exact
syscall and parameters.  

> I believe this command needs to be modified so that it performs a
> "stat" 

In general that's usually not the best approach.  Any file operation
inappropriate for a directory elicits EISDIR.  Since there are many
potential error conditions, it's better to simply operate on the name
and handle any error, than to try to anticipate each one.  

--jkl
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (dot) output call locks up when used on a named pipe

[Richard Hipp]

On 12/31/16, Paul Lambert  wrote:
> The "wb" your listing has add the "b" but based on what I have found
> the "b"  adds nothing and is the same as just "w."

True for Linux.  But SQLite is x-platform.  And the "w" is significant
on other systems.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (dot) output call locks up when used on a named pipe

[Paul Lambert]

I will need to do some testing.  Here is an example of fopen using a named
pipe

   mkfifo fifoname

 /* issue fopen for write end of the fifo */
wr_stream = fopen(fifoname,"w");

from:  
http://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.cbcpx01/fifopip.htm

The "wb" your listing has add the "b" but based on what I have found
the "b"  adds nothing and is the same as just "w."  The documentation
I found states it is only added for ISO C standard conformance.

I will need to do some more testing and report back.


On Sat, Dec 31, 2016 at 3:32 PM, Richard Hipp  wrote:

> On 12/31/16, Paul Lambert  wrote:
> > I have observed that the file write mode of this function when using the
> > same file name a second time will delete the previous data in the file.
> > This is most likely accomplished by deleting the file and recreating a
> new
> > one.  if so then this explains why the .output function locks up when
> using
> > a named pipe as the pipe is not responding to a delete command.
>
> It just calls fopen().  http://www.sqlite.org/src/
> artifact/6095531aa9?ln=2609
>
> I don't know what fopen() is doing behind the scenes.  Probably it
> works differently on different systems, but I would have guessed that
> it just invoked open() with O_TRUNC.
>
>
> >
> > Pipes are files and the ability to launch an application and place a
> > pending read on the pipe is such that the system will block the read and
> > the application will hang on the read statement until something shows up.
> > In the case of a file this is not true.
> >
> > I believe this command needs to be modified so that it performs a "stat"
> to
> > determine if the target is a file or named pipe.  If a named pipe then
> omit
> > the call to delete it.  This function should additionally check to make
> > sure the target is a file and not a directory too as user typos will most
> > certainly lead to unknown results.
> > ___
> > sqlite-users mailing list
> > sqlite-users@mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (dot) output call locks up when used on a named pipe

[Richard Hipp]

On 12/31/16, Paul Lambert  wrote:
> I have observed that the file write mode of this function when using the
> same file name a second time will delete the previous data in the file.
> This is most likely accomplished by deleting the file and recreating a new
> one.  if so then this explains why the .output function locks up when using
> a named pipe as the pipe is not responding to a delete command.

It just calls fopen().  http://www.sqlite.org/src/artifact/6095531aa9?ln=2609

I don't know what fopen() is doing behind the scenes.  Probably it
works differently on different systems, but I would have guessed that
it just invoked open() with O_TRUNC.


>
> Pipes are files and the ability to launch an application and place a
> pending read on the pipe is such that the system will block the read and
> the application will hang on the read statement until something shows up.
> In the case of a file this is not true.
>
> I believe this command needs to be modified so that it performs a "stat" to
> determine if the target is a file or named pipe.  If a named pipe then omit
> the call to delete it.  This function should additionally check to make
> sure the target is a file and not a directory too as user typos will most
> certainly lead to unknown results.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users