[ 
https://issues.apache.org/jira/browse/HAWQ-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Diachenko updated HAWQ-1084:
--------------------------------------
    Fix Version/s:     (was: backlog)
                   2.0.1.0-incubating

> Sometimes psql crashes with "out of memory" on \d hcatalog.* command
> --------------------------------------------------------------------
>
>                 Key: HAWQ-1084
>                 URL: https://issues.apache.org/jira/browse/HAWQ-1084
>             Project: Apache HAWQ
>          Issue Type: Bug
>          Components: PXF
>            Reporter: Oleksandr Diachenko
>            Assignee: Oleksandr Diachenko
>             Fix For: 2.0.1.0-incubating
>
>
> Normally it returns definition of all Hive tables:
> {code}
> PXF Hive Table "default.hive_abc"
>  Column |  Type  
> --------+--------
>  t0     | text
>  num1   | int4
>  d1     | float8
>  t1     | text
> PXF Hive Table "default.hive_abc222"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  t0     | text
>  num1   | int4
>  t1     | text
> PXF Hive Table "default.hive_avro_table"
>  Column |  Type  
> --------+--------
>  t0     | text
>  d1     | float8
>  num1   | int4
>  t1     | text
> PXF Hive Table "default.hive_binary"
>  Column | Type  
> --------+-------
>  b1     | bytea
> PXF Hive Table "default.hive_collections_table"
>  Column |  Type  
> --------+--------
>  ut1    | text
>  sr1    | text
>  m1     | text
>  a1     | text
>  s1     | text
>  f1     | float4
> PXF Hive Table "default.hive_many_partitioned_table"
>  Column |   Type    
> --------+-----------
>  s1     | text
>  s2     | text
>  n1     | int4
>  d1     | float8
>  dc1    | numeric
>  tm     | timestamp
>  f      | float4
>  bg     | int8
>  b      | bool
>  tn     | int2
>  sml    | int2
>  dt     | date
>  vc1    | varchar
>  c1     | bpchar
>  bin    | bytea
> PXF Hive Table "default.hive_orc_all_types"
>  Column |   Type    
> --------+-----------
>  bin    | bytea
>  c1     | bpchar
>  vc1    | varchar
>  dt     | date
>  sml    | int2
>  tn     | int2
>  b      | bool
>  bg     | int8
>  f      | float4
>  tm     | timestamp
>  dc1    | numeric
>  d1     | float8
>  n1     | int4
>  s2     | text
>  s1     | text
> PXF Hive Table "default.hive_orc_snappy"
>  Column |  Type  
> --------+--------
>  t0     | text
>  t1     | text
>  num1   | int4
>  d1     | float8
> PXF Hive Table "default.hive_orc_table"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_orc_zlib"
>  Column |  Type  
> --------+--------
>  t0     | text
>  t1     | text
>  num1   | int4
>  d1     | float8
> PXF Hive Table "default.hive_parquet_table"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_partitioned_clustered_sorted_table"
>  Column |  Type  
> --------+--------
>  fmt    | text
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_partitioned_clustered_table"
>  Column |  Type  
> --------+--------
>  fmt    | text
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_partitioned_skewed_stored_table"
>  Column |  Type  
> --------+--------
>  fmt    | text
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_partitioned_skewed_table"
>  Column |  Type  
> --------+--------
>  fmt    | text
>  t0     | text
>  d1     | float8
>  num1   | int4
>  t1     | text
> PXF Hive Table "default.hive_partitioned_table"
>  Column |  Type  
> --------+--------
>  t0     | text
>  t1     | text
>  num1   | int4
>  d1     | float8
>  fmt    | text
> PXF Hive Table "default.hive_rc_table"
>  Column |  Type  
> --------+--------
>  t0     | text
>  d1     | float8
>  num1   | int4
>  t1     | text
> PXF Hive Table "default.hive_rc_table_no_serde"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> PXF Hive Table "default.hive_sequence_table"
>  Column |  Type  
> --------+--------
>  t0     | text
>  t1     | text
>  num1   | int4
>  d1     | float8
> PXF Hive Table "default.hive_small_data"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  n1     | int4
>  s2     | text
>  s1     | text
> PXF Hive Table "default.hive_small_data_no_data_file"
>  Column |  Type  
> --------+--------
>  d1     | float8
>  n1     | int4
>  s2     | text
>  s1     | text
> PXF Hive Table "default.hive_table"
>  Column |  Type  
> --------+--------
>  s1     | text
>  n1     | int4
>  d1     | float8
>  bg     | int8
>  b      | bool
> PXF Hive Table "default.hive_types"
>  Column |   Type    
> --------+-----------
>  bin    | bytea
>  c1     | bpchar
>  vc1    | varchar
>  dt     | date
>  sml    | int2
>  tn     | int2
>  b      | bool
>  bg     | int8
>  f      | float4
>  tm     | timestamp
>  dc1    | numeric
>  d1     | float8
>  n1     | int4
>  s2     | text
>  s1     | text
> PXF Hive Table "default.hivebinaryrc"
>  Column | Type  
> --------+-------
>  data2  | bytea
>  data1  | bytea
>  n1     | int4
>  s1     | text
> PXF Hive Table "default.many_partitions"
>  Column | Type 
> --------+------
>  date_i | text
>  i      | int4
> PXF Hive Table "default.rc_hive_types"
>  Column |   Type    
> --------+-----------
>  bg     | int8
>  b      | bool
>  si     | int2
>  bin    | bytea
>  f      | float4
>  tm     | timestamp
>  d1     | float8
>  n1     | int4
>  s1     | text
>  key    | text
> PXF Hive Table "default.reg_collections"
>  Column |  Type  
> --------+--------
>  sr1    | text
>  m1     | text
>  a1     | text
>  f1     | float4
>  s1     | text
> PXF Hive Table "default.reg_heterogen"
>  Column |  Type  
> --------+--------
>  fmt    | text
>  d1     | float8
>  num1   | int4
>  t1     | text
>  t0     | text
> {code}
> Sometimes, not consistently it fails with OOM:
> {code}
> pxfautomation=# set client_min_messages=DEBUG1;
> DEBUG1:  Dropping in memory mapping OidInMemHeapMapping
> DEBUG1:  Dropping in memory mapping OidInMemOnlyMapping
> SET
> pxfautomation=# \d hcatalog.*
> DEBUG1:  Message type Q received by from libpq, len = 151
> DEBUG1:  Inserted entry for query (sessionid=17, commandcnt=3)
> DEBUG1:  SISC READER (shareid=0, slice=-1): exec dependency on slice -1, 
> driver_slice is -1
> DEBUG1:  SISC READER (shareid=0, slice=-1): exec dependency on slice -1, 
> driver_slice is -1
> DEBUG1:  SISC (shareid=0, slice=-1): No tuplestore yet, initializing 
> tuplestore
> DEBUG1:  Loopback interface IP address: [::1]
> DEBUG1:  Replaced url: 
> http://[::1]:51200/pxf/v14/Metadata/getMetadata?profile=Hive&pattern=*
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field b1, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field f1, type float4
> DEBUG1:  Parsing field a1, type text
> DEBUG1:  Parsing field m1, type text
> DEBUG1:  Parsing field sr1, type text
> DEBUG1:  Parsing field ut1, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field s2, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field dc1, type numeric
> DEBUG1:  modifier[0]: 38
> DEBUG1:  modifier[1]: 18
> DEBUG1:  Parsing field tm, type timestamp
> DEBUG1:  Parsing field f, type float4
> DEBUG1:  Parsing field bg, type int8
> DEBUG1:  Parsing field b, type bool
> DEBUG1:  Parsing field tn, type int2
> DEBUG1:  Parsing field sml, type int2
> DEBUG1:  Parsing field dt, type date
> DEBUG1:  Parsing field vc1, type varchar
> DEBUG1:  modifier[0]: 5
> DEBUG1:  Parsing field c1, type bpchar
> DEBUG1:  modifier[0]: 3
> DEBUG1:  Parsing field bin, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field s2, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field dc1, type numeric
> DEBUG1:  modifier[0]: 38
> DEBUG1:  modifier[1]: 18
> DEBUG1:  Parsing field tm, type timestamp
> DEBUG1:  Parsing field f, type float4
> DEBUG1:  Parsing field bg, type int8
> DEBUG1:  Parsing field b, type bool
> DEBUG1:  Parsing field tn, type int2
> DEBUG1:  Parsing field sml, type int2
> DEBUG1:  Parsing field dt, type date
> DEBUG1:  Parsing field vc1, type varchar
> DEBUG1:  modifier[0]: 5
> DEBUG1:  Parsing field c1, type bpchar
> DEBUG1:  modifier[0]: 3
> DEBUG1:  Parsing field bin, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field s2, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field s2, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field bg, type int8
> DEBUG1:  Parsing field b, type bool
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field s2, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field dc1, type numeric
> DEBUG1:  modifier[0]: 38
> DEBUG1:  modifier[1]: 18
> DEBUG1:  Parsing field tm, type timestamp
> DEBUG1:  Parsing field f, type float4
> DEBUG1:  Parsing field bg, type int8
> DEBUG1:  Parsing field b, type bool
> DEBUG1:  Parsing field tn, type int2
> DEBUG1:  Parsing field sml, type int2
> DEBUG1:  Parsing field dt, type date
> DEBUG1:  Parsing field vc1, type varchar
> DEBUG1:  modifier[0]: 5
> DEBUG1:  Parsing field c1, type bpchar
> DEBUG1:  modifier[0]: 3
> DEBUG1:  Parsing field bin, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field data1, type bytea
> DEBUG1:  Parsing field data2, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field i, type int4
> DEBUG1:  Parsing field date_i, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field key, type text
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field n1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field tm, type timestamp
> DEBUG1:  Parsing field f, type float4
> DEBUG1:  Parsing field bg, type int8
> DEBUG1:  Parsing field b, type bool
> DEBUG1:  Parsing field si, type int2
> DEBUG1:  Parsing field bin, type bytea
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field s1, type text
> DEBUG1:  Parsing field f1, type float4
> DEBUG1:  Parsing field a1, type text
> DEBUG1:  Parsing field m1, type text
> DEBUG1:  Parsing field sr1, type text
> DEBUG1:  Parsed item , namespace
> DEBUG1:  Parsing field t0, type text
> DEBUG1:  Parsing field t1, type text
> DEBUG1:  Parsing field num1, type int4
> DEBUG1:  Parsing field d1, type float8
> DEBUG1:  Parsing field fmt, type text
> DEBUG1:  SISC (shareid=0, slice=-1): No tuplestore yet, initializing 
> tuplestore
> DEBUG1:  Dropping in memory mapping OidInMemHeapMapping
> DEBUG1:  Deleted entry for query (sessionid=17, commandcnt=3)
> DEBUG1:  Dropping in memory mapping OidInMemOnlyMapping
> out of memory
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to