On Sun, May 05, 2013 at 01:50:41AM -0700, Newbie89 scratched on the wall: > let say is .txt file
99+% of log files are text files. This tells us nothing useful. Is it an Apache log file? Or some other well-known format that someone might have already written a parser for? Does it have a fixed format? Does it have a fixed number of columns? How many columns does it have? How are the columns differentiated? Commas? White-space? Tabs? What are the types of each column? Integers? Text? Dates? How do you want to map columns to database tables? What is the format of the final table? You need to figure this out, not some magic one function library. All of these questions need to be asked before we even get to the import process, yet you provided none of this information. That makes it very difficult/impossible to help, except to help you ask better questions. This is long, but good: http://www.catb.org/esr/faqs/smart-questions.html In my own dealings with log files and SQLite, I usually avoid importing them all together. Importing large files is slow. Most log file searches require scans, due to the type of ad-hoc or summary queries that are typically run against logs. If the file has a well-defined format, I usually just write a virtual table module to scan the log file directly. This is MUCH faster than importing the file, and can be used against "live" log files. Chapter 10 of "Using SQLite" is all about virtual tables: http://shop.oreilly.com/product/9780596521196.do The big example in that chapter is about exposing Apache/httpd format log files the database through a virtual table. Example code can be found here: http://examples.oreilly.com/9780596521196/ -j -- Jay A. Kreibich < J A Y @ K R E I B I.C H > "Intelligence is like underwear: it is important that you have it, but showing it to the wrong people has the tendency to make them feel uncomfortable." -- Angela Johnson _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
