Maybe I just paste some main thing on mail , and congratulation on IPO
thing.
Unregister query: 17m42s (17m42s)
Fetched 317246 row(s) in 1062.84s
Query Runtime Profile:
Query (id=8149e2439f43b15a:f08e570d7fbf1085):
Summary:
Session ID: 35436d1112b79287:9045c79c795858a5
Session Type: BEESWAX
Start Time: 2017-04-28 11:50:00.292615000
End Time: 2017-04-28 12:07:43.133484000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.5.0-cdh5-INTERNAL RELEASE (build
43880282edc04c03c162bbea6fc85b5388e7fdde)
User: impala
Connected User: impala
Delegated User:
Network Address: ::ffff:10.44.10.186:36325
Default Db: sjzy
Sql Statement: select
MRECID,UNITID,PCQDM,PCQMC,PCXQDM,PCXQMC,DM,H001,H002,H003,H021,H022,H023,H024,H025,H026A,H026B,H026C,H026D,H026E,H026F,H026G,H027,H028,H029,H030,H031,H032,H033,H034,H035,H036,H037A,H037B,H037C,H038,H039,H040,H041,H042,H043A,H043B,H043C,H043D,H043E,H043F,H043G,H043H,H043I,H043J,H043K,H043L,H044A,H044B,H044C,H044D,H044E,H044F,H044G,H044H,H044I,H050,H051,H052,H053,H054,H055,H056,H061,H062,H063,H064,H065,H066,H070,H071,H072,H073,H074,H075,H080,H100,H111,H112,H113,H120,H200,H201,H202,H203,H204,H205,H206,H207,H208,H209,H210,H211,H300,H320,H321,H322,H323,H324,H400,H401,H402,H403,H404,H405,H406,H500,H600,H601,H602,H603,H604,H605,H606,H607,H608,H609,H610,H611,H612,H613,H614,H615,H616,H621A,H621B,H621C,H621D,H621E,H621F,H622A,H622B,H622C,H801,H802,H803,H804,H901,H902,H903
FROM NP_2017_NP601 WHERE DS_AREACODE LIKE '445281%'
Coordinator: node1.sky.org:22000
Query Options (non default):
Plan:
----------------
Estimated Per-Host Requirements: Memory=4.50GB VCores=1
01:EXCHANGE [UNPARTITIONED]
| hosts=4 per-host-mem=unavailable
| tuple-ids=0 row-size=1.67KB cardinality=1155911
|
00:SCAN HDFS [sjzy.np_2017_np601, RANDOM]
partitions=1/1 files=20 size=1.06GB
predicates: DS_AREACODE LIKE '445281%'
table stats: 11559109 rows total
column stats: all
hosts=4 per-host-mem=4.50GB
tuple-ids=0 row-size=1.67KB cardinality=1155911
----------------
Estimated Per-Host Mem: 4831838208
Estimated Per-Host VCores: 1
Request Pool: default-pool
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem
Est. Peak Mem Detail
-------------------------------------------------------------------------------------------------------------
01:EXCHANGE 1 32.314ms 32.314ms 317.25K 1.16M 0
-1.00 B UNPARTITIONED
00:SCAN HDFS 20 1s137ms 1s348ms 317.25K 1.16M 163.85 MB
4.50 GB sjzy.np_2017_np601
Planner Timeline: 53.683ms
- Analysis finished: 24.565ms (24.565ms)
- Equivalence classes computed: 26.389ms (1.823ms)
- Single node plan created: 33.607ms (7.218ms)
- Runtime filters computed: 33.684ms (76.568us)
- Distributed plan created: 39.125ms (5.441ms)
- Planning finished: 53.683ms (14.558ms)
Query Timeline: 17m42s
- Start execution: 43.792us (43.792us)
- Planning finished: 60.640ms (60.596ms)
- Ready to start 20 remote fragments: 65.111ms (4.471ms)
- All 20 remote fragments started: 74.572ms (9.461ms)
- Rows available: 744.300ms (669.728ms)
- First row fetched: 790.128ms (45.828ms)
- Unregister query: 17m42s (17m42s)
ImpalaServer:
- ClientFetchWaitTimer: 17m31s
- RowMaterializationTimer: 10s024ms
2017-04-28 19:44 GMT+08:00 Jim Apple <[email protected]>:
> dev@ does not appear to accept attachments. You can upload it somewhere
> and
> post a link, though.
>
> On Thu, Apr 27, 2017 at 11:35 PM, 吴朱华 <[email protected]> wrote:
>
> > Oops, I just resend it, you know the chinese network^_^
> >
> > 2017-04-28 14:20 GMT+08:00 Mostafa Mokhtar <[email protected]>:
> >
> >> Btw the profile wasn't attached.
> >> Please resend.
> >>
> >> On Thu, Apr 27, 2017 at 11:11 PM, 吴朱华 <[email protected]> wrote:
> >>
> >>> Profile is in the attachment, thanks
> >>>
> >>>
> >>> 2017-04-28 13:10 GMT+08:00 Dimitris Tsirogiannis <
> >>> [email protected]>:
> >>>
> >>>> Maybe you also want to post some information about the schema (how
> wide
> >>>> your table is, does it use nested types, etc) as well as the profile
> of
> >>>> the
> >>>> slow query.
> >>>>
> >>>> Dimitris
> >>>>
> >>>> On Thu, Apr 27, 2017 at 9:30 PM, 吴朱华 <[email protected]> wrote:
> >>>>
> >>>> > Hi guys:
> >>>> > we can facing a big issue when select * from a big table.
> >>>> > The performance is 17minutes for retrieving 400MB data. Even slow
> >>>> under
> >>>> > JDBC situation.
> >>>> > Is there anyway to improve it?^_^
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
