Re: [Maya-Python] How reliable is SQLite in production?

Tue, 17 Sep 2013 00:19:05 -0700

Hmm. I actually like the shelve option, now that I've seen it. Like PyQt, the Python library is large enough that I definitely haven't scratched the surface on all that it has available, so I certainly appreciate the tip. That actually solves some of my other non-interface issues too. Thanks! 

On 9/17/2013 12:07 AM, Justin Israel wrote:
For the purely single-user type tool, I was wondering about the data requirement to see if suggesting the python Shelve <http://docs.python.org/2/library/shelve.html> module would be an appropriate replacement for Sqlite. That would be a suitable replacement if you access your data in a key-value approach, where the value can be any serialized data structure. 


But if you need the query support, then I suppose sqlite would need to 
be the solution. And for studio-wide settings, you could either use 
MySQL, with your client API hitting it directly (2 tier solution), or 
an sqlite solution, made available over a REST api that serializes all 
access to the database (3 tier solution). That is, clients would hit 
the REST api, and a single worker is performing all writes (and I 
assume its fine that multiple workers could serve read requests).
The same thing could be done with MySQL, in a 3 tier fashion, if you 
want more speed and reliability over the sqlite backend.







On Tue, Sep 17, 2013 at 6:19 PM, Joe Weidenbach <[email protected] 
<mailto:[email protected]>> wrote:


    As they say, when you have a hammer, all you see is nails.  In my
    previous life I was a database programmer, hence my proclivity
    towards using them.  Mostly, I use this tool for finding rig paths
    in an easy fashion.  The tool loads a list of characters that it
    knows about (one query), and then loads the known control and
    envelope rigs for those characters (another query), finally using
    that info for another query to actually feed the file paths into a
    referencer to make the appropriate constraints.  It would be
    simple enough to write a text parser for a flat file, but I like
    SQL a lot better than string parsing.

    That's most of it :)  However, on a much larger tool I'm working
    on, it's going to be managing studio-wide settings and
    preferences.  I was planning on using sqlite for the single-user
    version, and MySQL for the enterprise.  Is that the wrong approach?


    On 9/16/2013 9:35 PM, Justin Israel wrote:

    Is your application running queries against the data, or simply
    retrieving a payload of rig data for a given key?


    On Tue, Sep 17, 2013 at 4:09 PM, Joe Weidenbach
    <[email protected] <mailto:[email protected]>> wrote:

        John,

        Are there any other decent cross-platform embedded
        solutions?  In my case, this is a distributed project, so
        it's all single user, and they're only reading the
        database--the TD's (me mostly) load it up with rig data for
        loading the characters on the animation end, and the
        animators just install the tool, load the project, and use
        the tool to load the characters. In a production situation,
        I'd go MySQL in a heartbeat, but I don't know of any other
        reliable options for this scenario.

        What other options would you suggest?

        Thanks,

        Joe


        On 9/16/2013 9:24 AM, John Patrick wrote:

        I'd advise strongly against this.  There is a worse problem
        than performance that you'll encounter eventually -
        concurrent writes can corrupt the database.

        Like Justin mentioned, sqlite uses a file-locking mechanism
        to control concurrent writes, which seems to work 99.9% of

        the time. The time it doesn't will corrupt your database. 
        I've seen it happen.


        IMO, sqlite is great for storing single-user application
        data and as a test database, but if you're intending to
        store critical data that will be written to by multiple
        machines over a network, I would bite the bullet with
        PostgreSQL, MySQL, etc.



        On Sun, Sep 15, 2013 at 8:55 PM, Joe Weidenbach
        <[email protected] <mailto:[email protected]>> wrote:

            That's for 2012+ by the way.


            On 9/15/2013 8:49 PM, Jay Goodman wrote:

                Are you planning to connect to the DB from within
                maya?  There are (or were) issues with maya and
                sqlite: from the dll not being included with the
                maya install to memory access issues.  I thought I
                read this was fixed as of 2012, but I'm stuck on
                2009, and haven't verified.

                If anyone has resolved the memory access issues
                (which are very rare and random)--I would be
                interested in that solution.
                thanks



            -- 
            You received this message because you are subscribed to

            the Google Groups "Python Programming for Autodesk Maya"
            group.
            To unsubscribe from this group and stop receiving emails
            from it, send an email to
            [email protected]
            <mailto:python_inside_maya%[email protected]>.
            To post to this group, send email to
            [email protected]
            <mailto:[email protected]>.
            For more options, visit
            https://groups.google.com/groups/opt_out.





        -- 
        John Patrick

        404-242-2675 <tel:404-242-2675>
        [email protected] <mailto:[email protected]>
        http://www.canyourigit.com

        -- 
        You received this message because you are subscribed to the

        Google Groups "Python Programming for Autodesk Maya" group.
        To unsubscribe from this group and stop receiving emails
        from it, send an email to
        [email protected]
        <mailto:[email protected]>.
        To post to this group, send email to
        [email protected]
        <mailto:[email protected]>.
        For more options, visit
        https://groups.google.com/groups/opt_out.



        -- 
        You received this message because you are subscribed to the

        Google Groups "Python Programming for Autodesk Maya" group.
        To unsubscribe from this group and stop receiving emails from
        it, send an email to
        [email protected]
        <mailto:python_inside_maya%[email protected]>.
        To post to this group, send email to
        [email protected]
        <mailto:[email protected]>.
        For more options, visit https://groups.google.com/groups/opt_out.



    -- 
    You received this message because you are subscribed to the

    Google Groups "Python Programming for Autodesk Maya" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to [email protected]
    <mailto:[email protected]>.
    To post to this group, send email to
    [email protected]
    <mailto:[email protected]>.
    For more options, visit https://groups.google.com/groups/opt_out.



    -- 
    You received this message because you are subscribed to the Google

    Groups "Python Programming for Autodesk Maya" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to [email protected]
    <mailto:python_inside_maya%[email protected]>.
    To post to this group, send email to
    [email protected]
    <mailto:[email protected]>.
    For more options, visit https://groups.google.com/groups/opt_out.


--

You received this message because you are subscribed to the Google 
Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to [email protected].

To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.


--
You received this message because you are subscribed to the Google Groups "Python 
Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.























					Reply via email to