Re: [sqlite] sqlite3 mailing list broken

[Cal Leeming]

Follow up, eventually I got it working by using the following;
http://fabianpeter.de/cloud/owncloud-migrating-from-sqlite-to-mysql/

Pretty dirty, but it got the job done... took me 6 *hours* to discover
this fix, it's untested/alpha as hell and I didn't even write the
code.

Hopefully this helps others that might come across similar issues in future.

I can only argue that it really shouldn't have been this complicated :/

Cal

On Thu, Feb 12, 2015 at 1:40 PM, Richard Hipp  wrote:
> Tnx for email.  I'm in a meeting.  Reply later.
>
> On 2/12/15, Cal Leeming  wrote:
>> Thanks for the quick reply, appreciated.
>>
>> I've been having a problem where iterdump() is exporting in a format
>> which is unsuitable for MySQL, although it previously appeared to be
>> corruption (hence the previous comment of seriousness), it does
>> actually appear to be documented incompatibility. Tools such as [2]
>> don't work with popular libraries such as Django, due to its use of
>> pysqlite2 and patching scripts such as [3] also don't seem to work.
>>
>> It seems the backup features of sqlite3 are not exposed in higher
>> level APIs such as Python, and getting data out of a sqlite3 in-memory
>> DB in python into a file is proving to be a difficult task, something
>> which is arguable a common requirement. (something which I'm still
>> trying to do as we speak, and will report back with my eventual fix)
>>
>> I'm not sure what my current proposal would be, but my first
>> suggestion would be either exposing those backup features to the
>> higher level APIs.
>>
>> Also, I'd like to +1 having this project on some sort of social collab
>> platform, be it github, bitbucket etc. It would make external
>> contributions much easier, as I nearly gave up trying to report this
>> issue out of frustration.
>>
>> Thanks in advance
>>
>> Cal
>>
>> [2]: https://github.com/husio/python-sqlite3-backup
>> [3]: http://www.redmine.org/attachments/download/6239/sqlite3-to-mysql.py
>>
>> On Thu, Feb 12, 2015 at 12:53 PM, Richard Hipp  wrote:
>>> Sorry you had trouble.  You can send email directly to me if you want.
>>> Or
>>> to sqlite-users@sqlite.org wherr it will be held for moderation.
>>>
>>> --
>>> D. Richard Hipp
>>> Sent from phone - Excuse brevity
>>>
>>> On Feb 12, 2015 7:31 AM, "Cal Leeming"  wrote:

 Hi,

 Tried to report an issue to sqlite3 mailing list, I got my
 confirmation email but it keeps saying password incorrect.

 Not sure if you still have any say over the project, but would +1 ask
 for this to be on Github, I've been trying for 30 minutes to report a
 rather serious problem with sqlite3 and now giving up because it's too
 hard to report.

 Cal
>>
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/12/2015 05:15 AM, Cal Leeming wrote:
> I've been having a problem where iterdump() is exporting in a
> format which is unsuitable for MySQL,

iterdump is part of pysqlite, and has no code from the official SQLite
project.  iterdump itself is just an inadequate 50 lines of python code:

 https://github.com/ghaering/pysqlite/blob/master/lib/dump.py

BTW there is a python sqlite group which will suit your python <->
sqlite needs better:

  https://groups.google.com/forum/#!forum/python-sqlite

> It seems the backup features of sqlite3 are not exposed in higher 
> level APIs such as Python,

SQLite exposes a C API.  It is then up to the wrappers to expose that
in their wrapped form.  pysqlite does expose it but seems to leave it
out of the documentation.

  https://github.com/ghaering/pysqlite/blob/master/src/backup.c

> and getting data out of a sqlite3 in-memory DB in python into a
> file is proving to be a difficult task, something which is arguable
> a common requirement.

It is pretty simple.  You can use the backup API, you can use textual
dumps, and you attach databases and copy that way.  I suggest posting
to the python sqlite list with what it is you are trying to do.

Disclosure:  I'm the author of an alternate Python SQLite wrapper:

  http://rogerbinns.github.io/apsw/

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAlTc3AQACgkQmOOfHg372QSejACg5etNEUcBnNq0rKQV2cABkPo6
bjwAn0FagmG7B8jUxTCSEWDzLbMhOXwo
=4h4h
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[justin]

On 2015-02-12 14:00, Stephan Beal wrote:

On Thu, Feb 12, 2015 at 2:45 PM,  wrote:
And yeah, I'm aware of fossil, but (to my thinking ;>) that 
shouldn't hold

back _this_ bit of software. ;)


FWIW, fossil was/is designed _specifically_ for sqlite's hosting 
(that's
neither a joke nor an exaggeration), so it's _exceedingly_ unlikely 
to go

anywhere, regardless of how many +1s people collect to the contrary.


Yeah, I know. ;)

+ Justin
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[Stephan Beal]

On Thu, Feb 12, 2015 at 2:45 PM,  wrote:

> And yeah, I'm aware of fossil, but (to my thinking ;>) that shouldn't hold
> back _this_ bit of software. ;)
>

FWIW, fossil was/is designed _specifically_ for sqlite's hosting (that's
neither a joke nor an exaggeration), so it's _exceedingly_ unlikely to go
anywhere, regardless of how many +1s people collect to the contrary.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[Cal Leeming]

Thanks for the quick reply, appreciated.

I've been having a problem where iterdump() is exporting in a format
which is unsuitable for MySQL, although it previously appeared to be
corruption (hence the previous comment of seriousness), it does
actually appear to be documented incompatibility. Tools such as [2]
don't work with popular libraries such as Django, due to its use of
pysqlite2 and patching scripts such as [3] also don't seem to work.

It seems the backup features of sqlite3 are not exposed in higher
level APIs such as Python, and getting data out of a sqlite3 in-memory
DB in python into a file is proving to be a difficult task, something
which is arguable a common requirement. (something which I'm still
trying to do as we speak, and will report back with my eventual fix)

I'm not sure what my current proposal would be, but my first
suggestion would be either exposing those backup features to the
higher level APIs.

Also, I'd like to +1 having this project on some sort of social collab
platform, be it github, bitbucket etc. It would make external
contributions much easier, as I nearly gave up trying to report this
issue out of frustration.

Thanks in advance

Cal

[2]: https://github.com/husio/python-sqlite3-backup
[3]: http://www.redmine.org/attachments/download/6239/sqlite3-to-mysql.py

On Thu, Feb 12, 2015 at 12:53 PM, Richard Hipp  wrote:
> Sorry you had trouble.  You can send email directly to me if you want.  Or
> to sqlite-users@sqlite.org wherr it will be held for moderation.
>
> --
> D. Richard Hipp
> Sent from phone - Excuse brevity
>
> On Feb 12, 2015 7:31 AM, "Cal Leeming"  wrote:
>>
>> Hi,
>>
>> Tried to report an issue to sqlite3 mailing list, I got my
>> confirmation email but it keeps saying password incorrect.
>>
>> Not sure if you still have any say over the project, but would +1 ask
>> for this to be on Github, I've been trying for 30 minutes to report a
>> rather serious problem with sqlite3 and now giving up because it's too
>> hard to report.
>>
>> Cal
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[justin]

On 2015-02-12 13:40, Richard Hipp wrote:

On 2/12/15, Cal Leeming  wrote:
Also, I'd like to +1 having this project on some sort of social 
collab

platform, be it github, bitbucket etc. It would make external
contributions much easier, as I nearly gave up trying to report this
issue out of frustration.


+1 for GitHub.  Purely because of the large qty of potential 
contributors

there, many of whom actively get involved.

And yeah, I'm aware of fossil, but (to my thinking ;>) that shouldn't 
hold

back _this_ bit of software. ;)

Regards and best wishes,

Justin Clift
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 mailing list broken

[Richard Hipp]

Tnx for email.  I'm in a meeting.  Reply later.

On 2/12/15, Cal Leeming  wrote:
> Thanks for the quick reply, appreciated.
>
> I've been having a problem where iterdump() is exporting in a format
> which is unsuitable for MySQL, although it previously appeared to be
> corruption (hence the previous comment of seriousness), it does
> actually appear to be documented incompatibility. Tools such as [2]
> don't work with popular libraries such as Django, due to its use of
> pysqlite2 and patching scripts such as [3] also don't seem to work.
>
> It seems the backup features of sqlite3 are not exposed in higher
> level APIs such as Python, and getting data out of a sqlite3 in-memory
> DB in python into a file is proving to be a difficult task, something
> which is arguable a common requirement. (something which I'm still
> trying to do as we speak, and will report back with my eventual fix)
>
> I'm not sure what my current proposal would be, but my first
> suggestion would be either exposing those backup features to the
> higher level APIs.
>
> Also, I'd like to +1 having this project on some sort of social collab
> platform, be it github, bitbucket etc. It would make external
> contributions much easier, as I nearly gave up trying to report this
> issue out of frustration.
>
> Thanks in advance
>
> Cal
>
> [2]: https://github.com/husio/python-sqlite3-backup
> [3]: http://www.redmine.org/attachments/download/6239/sqlite3-to-mysql.py
>
> On Thu, Feb 12, 2015 at 12:53 PM, Richard Hipp  wrote:
>> Sorry you had trouble.  You can send email directly to me if you want.
>> Or
>> to sqlite-users@sqlite.org wherr it will be held for moderation.
>>
>> --
>> D. Richard Hipp
>> Sent from phone - Excuse brevity
>>
>> On Feb 12, 2015 7:31 AM, "Cal Leeming"  wrote:
>>>
>>> Hi,
>>>
>>> Tried to report an issue to sqlite3 mailing list, I got my
>>> confirmation email but it keeps saying password incorrect.
>>>
>>> Not sure if you still have any say over the project, but would +1 ask
>>> for this to be on Github, I've been trying for 30 minutes to report a
>>> rather serious problem with sqlite3 and now giving up because it's too
>>> hard to report.
>>>
>>> Cal
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users