Re: [PERFORM] Certain query eating up all free memory (out of memory error)
On Mon, May 24, 2010 at 12:50 PM, Łukasz Dejneka l.dejn...@gmail.com wrote: Hi group, I could really use your help with this one. I don't have all the details right now (I can provide more descriptions tomorrow and logs if needed), but maybe this will be enough: I have written a PG (8.3.8) module, which uses Flex Lexical Analyser. It takes text from database field and finds matches for defined rules. It returns a set of two text fields (value found and value type). When I run query like this: SELECT * FROM flex_me(SELECT some_text FROM some_table WHERE id = 1); It works perfectly fine. Memory never reaches more than 1% (usually its below 0.5% of system mem). But when I run query like this: SELECT flex_me(some_text_field) FROM some_table WHERE id = 1; Memory usage goes through the roof, and if the result is over about 10k matches (rows) it eats up all memory and I get out of memory error. I'm not sure exactly what's happening in your particular case, but there is some known suckage in this area. http://archives.postgresql.org/pgsql-hackers/2010-05/msg00230.php http://archives.postgresql.org/pgsql-hackers/2010-05/msg00395.php -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] Certain query eating up all free memory (out of memory error)
EXPLAIN ANALYSE on smaller query: Seq Scan on teksty (cost=0.00..1353.50 rows=1 width=695) (actual time=0.220..12.354 rows=368 loops=1) Filter: (id = 1) Total runtime: 12.488 ms Memory config: # - Memory - shared_buffers = 24MB temp_buffers = 8MB max_prepared_transactions = 5 work_mem = 16MB# min 64kB maintenance_work_mem = 16MB# min 1MB max_stack_depth = 2MB# min 100kB # - Free Space Map - max_fsm_pages = 153600 #max_fsm_relations = 1000 Memory info from logs: TopMemoryContext: 49416 total in 6 blocks; 7680 free (8 chunks); 41736 used TopTransactionContext: 8192 total in 1 blocks; 7856 free (1 chunks); 336 used Type information cache: 8192 total in 1 blocks; 1800 free (0 chunks); 6392 used CFuncHash: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used MbProcContext: 1024 total in 1 blocks; 928 free (6 chunks); 96 used Operator class cache: 8192 total in 1 blocks; 3848 free (0 chunks); 4344 used Operator lookup cache: 24576 total in 2 blocks; 14072 free (6 chunks); 10504 used MessageContext: 8192 total in 1 blocks; 752 free (0 chunks); 7440 used smgr relation table: 8192 total in 1 blocks; 2808 free (0 chunks); 5384 used TransactionAbortContext: 32768 total in 1 blocks; 32752 free (0 chunks); 16 used Portal hash: 8192 total in 1 blocks; 3912 free (0 chunks); 4280 used PortalMemory: 8192 total in 1 blocks; 8040 free (0 chunks); 152 used PortalHeapMemory: 1024 total in 1 blocks; 880 free (0 chunks); 144 used ExecutorState: 516096 total in 6 blocks; 15368 free (7 chunks); 500728 used SRF multi-call context: 2499608 total in 276 blocks; 714136 free (38704 chunks); 1785472 used ExprContext: 3157941940 total in 12908 blocks; 505592 free (11 chunks); 3157436348 used Relcache by OID: 8192 total in 1 blocks; 3376 free (0 chunks); 4816 used CacheMemoryContext: 667472 total in 20 blocks; 239368 free (1 chunks); 428104 used pg_toast_150116_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_database_datname_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_index_indrelid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_ts_dict_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_aggregate_fnoid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_language_name_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_statistic_relid_att_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_ts_dict_dictname_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_namespace_nspname_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_opfamily_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_opclass_oid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_ts_parser_prsname_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_amop_fam_strat_index: 1024 total in 1 blocks; 88 free (0 chunks); 936 used pg_opclass_am_name_nsp_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used pg_trigger_tgrelid_tgname_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_cast_source_target_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_auth_members_role_member_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_attribute_relid_attnum_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_ts_config_cfgname_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_authid_oid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_ts_config_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_conversion_default_index: 1024 total in 1 blocks; 88 free (0 chunks); 936 used pg_language_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_enum_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_proc_proname_args_nsp_index: 1024 total in 1 blocks; 152 free (0 chunks); 872 used pg_ts_parser_oid_index: 1024 total in 1 blocks; 344 free (0 chunks); 680 used pg_database_oid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_conversion_name_nsp_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_class_relname_nsp_index: 1024 total in 1 blocks; 240 free (0 chunks); 784 used pg_attribute_relid_attnam_index: 1024 total in 1 blocks; 280 free (0 chunks); 744 used pg_class_oid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_amproc_fam_proc_index: 1024 total in 1 blocks; 88 free (0 chunks); 936 used pg_operator_oprname_l_r_n_index: 1024 total in 1 blocks; 88 free (0 chunks); 936 used pg_index_indexrelid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_type_oid_index: 1024 total in 1 blocks; 304 free (0 chunks); 720 used pg_rewrite_rel_rulename_index: 1024 total in 1 blocks;
