Cecil,, Linux reporting 8 cores is due to hyper threading on the four cores. Thats normal.
One of the cores reporting a high usage is normal if you have single threaded app that simply cannot be moved, e.g. many perl programs exhibit this behaviour. Thats fine and to be expected. I do not have the whole thread for what you reported but I did read somebody ask if you have put your inserts between a BEGIN/END transaction. That will make a massive difference to your speed. Also I?m unclear as to how Java fits in all of this. Perhaps you gave a better indication further up the thread I do not have. There are a lot of very, very talented people on the list, I am an not one of them :), I would strongly urge you to provide more information rather than less, e.g. you provide summary information for Java by RSS. It would be a lot more useful to have the full raw data so people can look for quirks and anomalies rather than simply you interpreting it for us. I am NOT the best person to talk about SQLite performance, however I am more familiar with Unix/Linux performance and administration. Since most people here do not have the same issues as you and I have no doubt they are hitting Sqlite far harder than I or you can, I would look at what you are doing that is different and Java keeps jumping out at me. I have run (though not programmed as I?m not a Java developer) a number of programs running Java as the backend onto large DB2 databases and the first thing the developers would do would be to increase the amount of memory that Java is allowed to allocate and use. They would tend to throw 4GB at each JVM or more if the sysadmins would let them. I still don?t get a feeling for where the problem is though as the information is fragmentary. Rob On 17 Apr 2016, at 8:40, Cecil Westerhof wrote: > 2016-04-17 1:03 GMT+02:00 Keith Medcalf <kmedcalf at dessus.com>: > >>> Have another problem also. My CPU is about 15%, but the load average >>> is >>> also about 15. (This is on a Linux system.) This results (sometimes) >>> in a >>> very sluggish system. Can the load be a SQLite problem, or is it a >>> Java >>> problem? (When the program is not running, the load average is a lot >>> lower.) >> >> You have slow CPU with multiple cores (15% means you probably have >> quad >> core -- probably with Intel Hyper-Slowness as well (as in not SMT), >> or you >> have dual-core SMT), and the dispatcher is brain-dead and dispatching >> the >> single threaded application on the same CPU as more critical OS >> workers >> resulting in swamping that core with compute use and preventing >> competing >> threads (such as the OS or the gooey) from being other than -- well >> -- >> gooey. >> >> You claim this is Linux. There are many brands and flavours (and >> versions) or Linux, all tuned to behave differently. You do not >> specify >> the particular version and brand it is. >> > > ?It never hurts to give some extra information. I only expected > SQLite > help. But I do not mind to get other help. :-D > > The CPU is: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz. > It is a little strange. It has four cores but Linux thinks it has > eight. > None of the eight has a high load: sometimes one 60-70%, but most of > the > time a lot lower. > The operating system is openSUSE 13.2. > > free -m gives: > total used free shared buffers > cached > Mem: 15923 15737 185 1300 105 > 8131 > -/+ buffers/cache: 7500 8422 > Swap: 2053 1659 394? > > > ?RSSMemory usage for java > ======================================== > RSSMemory 6 MB by PID=9705 > RSSMemory 54 MB by PID=9731 > RSSMemory 75 MB by PID=28844 > RSSMemory 112 MB by PID=18743 > RSSMemory 121 MB by PID=28880 > RSSMemory 345 MB by PID=1036 > ======================================== > Total used RSSMemory: 713 MB > > swap usage for java > ======================================== > swap 2 MB by PID=28880 > swap 7 MB by PID=28844 > swap 128 MB by PID=9731 > swap 132 MB by PID=9705 > swap 442 MB by PID=1036 > ======================================== > Total used swap: 711 MB > > The program has PID 18743 and does not use swap. > > ps -l 18743 gives: > F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD > 0 S 1000 18743 28670 4 80 0 - 1718302 futex_ pts/12 45:29 java > RandomUUID CheckUUID.properties > ? > My own psPid 18743 gives: > USER PID PPID TT START ELAPSED TIME STAT COMMAND > cecil 18743 28670 pts/12 Apr16 16:29:42 00:45:27 Sl+ java > RandomUUID CheckUUID.properties > > So it does not have a high CPU usage about 5%. > > I think I have to cancel the program, because I have: > 23:03:12: Inserted 6.00e+07 UUID's > 00:50:33: Inserted 6.10e+07 UUID's > 02:56:31: Inserted 6.20e+07 UUID's > 04:56:06: Inserted 6.30e+07 UUID's > 06:53:03: Inserted 6.40e+07 UUID's > 09:14:08: Inserted 6.50e+07 UUID's > > That will take at least another 3 days. > > This is the text based UUID. I expected it to take longer, but the > blob > based took 4? hours. So that is a very big difference. But that is > something for another post. > > It took almost half an hour to write, because of the sluggish > behaviour. > :'-( > > -- > Cecil Westerhof > _______________________________________________ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
