Oh yeah...you need to upgrade your sqlite. You are hitting the SD card which is dog slow.
Keeping your DB in memory may solve all of your database speed concerns. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems ________________________________________ From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of kritesh tripathi [tripathi.krit...@gmail.com] Sent: Tuesday, August 28, 2012 11:34 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Pragma Synchronous=OFF is not working Hi, Sorry but i am not using memory database since sqlite version -3.6.4 does not support i guess ..I am not indexing any data and i am sure that all insertion , i am doing under one transaction . Do you think creating index or i n memory database or binding the values using sqlite_binding will effect the performance .. Is performance due to hardware like SD card speed or Filesystem ? Cheers kritesh On Tue, Aug 28, 2012 at 9:55 PM, Black, Michael (IS) <michael.bla...@ngc.com > wrote: > So you're already doing the smart thing....good....and I believe you did > say you're using a memory database, right? Not storing it on the SD card? > > Are you able to run your timing test on a standard PC? > > 50 inserts/sec is definitely NOT fast (that's your .02 number). > Thousands/sec is more like it on a standard PC. > > Did you create any indexes on your data? > > Are you sure you have the entire load process in one transaction? > > Care to show us your code? > > Another thing you can do is output SQL statements to stdout so you can run > those inserts inside an sqlite3 shell and test different configurations > that way. > > > > Michael D. Black > Senior Scientist > Advanced Analytics Directorate > Advanced GEOINT Solutions Operating Unit > Northrop Grumman Information Systems > > ________________________________________ > From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] > on behalf of kritesh tripathi [tripathi.krit...@gmail.com] > Sent: Tuesday, August 28, 2012 11:09 AM > To: General Discussion of SQLite Database > Subject: EXT :Re: [sqlite] Pragma Synchronous=OFF is not working > > HI Michael, > > I am not stroing the video data in the table .For example -Suppose i have > one image - ABC.jpg in the folder contains in SD Card > 1- First i am parsing the image > 2- Second Get the values like - Fullpath (Wht is the exact path of image in > SD Card),file Size and create unique id . > 3-Third , I am inserting these values in Image table . > > > Executing the same steps for all images (folder where all images stored ) > in SD card . > > Total time to parsing and insert the 500 image =42 sec > Time to insert one image in table after parsing = .02 Sec > Time to parsing one images = .04 sec > > Cheers > kritesh > > On Tue, Aug 28, 2012 at 9:22 PM, Black, Michael (IS) < > michael.bla...@ngc.com > > wrote: > > > I think the first thing you'll hear is to NOT store the video data in the > > database. > > Just store a file path. That is much faster and should complete a lot > > faster than your expectations. > > > > Is there some specific reason why you want the blob data in your > database? > > > > How long does it take you just to read and parse the images and skip the > > database insert? > > > > What exactly are you inserting in those 3 tables? Sounds like some big > > data. > > > > > > Michael D. Black > > Senior Scientist > > Advanced Analytics Directorate > > Advanced GEOINT Solutions Operating Unit > > Northrop Grumman Information Systems > > > > ________________________________________ > > From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] > > on behalf of kritesh tripathi [tripathi.krit...@gmail.com] > > Sent: Tuesday, August 28, 2012 10:31 AM > > To: General Discussion of SQLite Database > > Subject: EXT :Re: [sqlite] Pragma Synchronous=OFF is not working > > > > Hi Michael, > > > > Righ Now i am inserting only 500 records in three different tables in 42 > > sec . I have 500 imagesor video (.jpg or MP4) in the SD card which i am > > parsing and then inserting one by one in the video table or image table > > . I am expecting this in between 10-20 sec . Presently i am using the > > Micro itron embedded RTOS in my device . > > > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > > > > -- > Regards > kritesh tripathi > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- Regards kritesh tripathi _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
