[ 
https://issues.apache.org/jira/browse/IMPALA-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Tauber-Marshall resolved IMPALA-6713.
--------------------------------------------
    Resolution: Fixed

commit ffb74e7c0fb75e3062ad08451e474be59264f6a4 (asf-gerrit/master)
Author: Thomas Tauber-Marshall <tmarsh...@cloudera.com>
Date:   Thu Apr 12 03:36:06 2018 +0000

    IMPALA-6713: Fix request for unneeded memory in partial sort
    
    When a Sorter::Run is initialized, if it is an initial run and has
    varlen data, it requests an extra buffer to have space to sort the
    varlen data should it need to spill to disk.
    
    This extra buffer is not needed in the case of partial sorts, which
    do not spill, and because this extra buffer was not included in the
    calculation of the minimum required reservation, requesting it caused
    the partial sort to fail in cases where the partial sort only had its
    minimum reservation available to use.
    
    The solution is to not request the extra memory for partial sorts.
    
    Testing:
    - Added a test to test_sort.py that ensures the partial sort can
      complete successfully even if additional memory requests beyond its
      minimum reservation are denied.
    
    Change-Id: I2d9c0863009021340d8b684669b371a2cfb1ecad
    Reviewed-on: http://gerrit.cloudera.org:8080/10031
    Reviewed-by: Thomas Tauber-Marshall <tmarsh...@cloudera.com>
    Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>

> Failed to increase reservation at Sorter
> ----------------------------------------
>
>                 Key: IMPALA-6713
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6713
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 2.12.0
>            Reporter: yyzzjj
>            Assignee: Thomas Tauber-Marshall
>            Priority: Critical
>              Labels: crash, newbie
>         Attachments: IMPALA-6713.zip, query-impala-26340.csv, table schema.txt
>
>
> {color:#d04437}E0321 17:12:01.525718 17305 substitute.cc:41] 
> strings::Substitute format string invalid: asked for "$2", but only 2 args 
> were given. Full format string was: "Unexpected error trying to reserve $0 
> bytes for a sorted run: $2".{color}
> I0321 17:12:01.538334 17305 status.cc:128]
>  @ 0x7ff90612e98a impala::Status::Status()
>  @ 0x7ff908acd9e4 impala::Sorter::Run::Init()
>  @ 0x7ff908ace9e9 impala::Sorter::Open()
>  @ 0x7ff9086c5a1f impala::PartialSortNode::GetNext()
>  @ 0x7ff908a8fda2 impala::FragmentInstanceState::ExecInternal()
>  @ 0x7ff908a928ff impala::FragmentInstanceState::Exec()
>  @ 0x7ff908aabb8a impala::QueryState::ExecFInstance()
>  @ 0x7ff909775a2f impala::Thread::SuperviseThread()
>  @ 0x7ff9097762ba boost::detail::thread_data<>::run()
>  @ 0x7ff905cf20f5 thread_proxy
>  
> https://github.com/apache/impala/blob/1d38c584aef2d0f852af8ee659157d00eb3b061a/be/src/runtime/sorter.cc#L637



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to