date:20161026

[Impala-ASF-CR] IMPALA-4383: Ensure plan fragment report thread is always started

2016-10-26 Thread Henry Robinson (Code Review)

Henry Robinson has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4864

Change subject: IMPALA-4383: Ensure plan fragment report thread is always 
started
..

IMPALA-4383: Ensure plan fragment report thread is always started

PlanFragmentExecutor::report_thread_active_ was set during
OpenInternal() after ReportProfile() - run in a different thread -
signalled that it was running.

Howeer, ReportProfile() reads report_thread_active_ to determine if it
should exit its loop. If it runs fast enough, ReportProfile() will never
see report_thread_active_ == true.

This patch moves setting report_thread_active_ to ReportProfile() -
slightly earlier, but visible at almost the same time because it is now
correctly protected by report_thread_lock_.

Testing:
* TestRPCTimeout would hit this in certain configurations. Ran
  test_execplanfragment_timeout() for 90 minutes with no failures;
  previously it would fail within the first five attempts repeatedly.

Change-Id: I5d3cab4cc5245014758e6b70dec7a706a7fa929b
---
M be/src/runtime/plan-fragment-executor.cc
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/4864/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4864
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d3cab4cc5245014758e6b70dec7a706a7fa929b
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson

[Impala-ASF-CR] IMPALA 2523: Make HdfsTableSink aware of clustered input

2016-10-26 Thread Lars Volker (Code Review)

Lars Volker has posted comments on this change.

Change subject: IMPALA_2523: Make HdfsTableSink aware of clustered input
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4863/1/be/src/exec/hdfs-table-sink.cc
File be/src/exec/hdfs-table-sink.cc:

Line 594: WriteClusteredRowBatch(state, batch);
this case is exercised by the tests for IMPALA-2521, however we don't have 
tests (yet) that make sure we actually ever have one partition output file 
open. Any ideas how I could test this easily with our current infrastructure?


-- 
To view, visit http://gerrit.cloudera.org:8080/4863
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibeda0bdabbfe44c8ac95bf7c982a75649e1b82d0
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Lars Volker 
Gerrit-Reviewer: Lars Volker 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA 2523: Make HdfsTableSink aware of clustered input

2016-10-26 Thread Lars Volker (Code Review)

Lars Volker has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4863

Change subject: IMPALA_2523: Make HdfsTableSink aware of clustered input
..

IMPALA_2523: Make HdfsTableSink aware of clustered input

IMPALA-2521 introduced clustering for insert statements. This change
makes the HdfsTableSink aware of clustered inputs, so that partitions
are opened, written, and closed one by one.

Change-Id: Ibeda0bdabbfe44c8ac95bf7c982a75649e1b82d0
---
M be/src/exec/hdfs-table-sink.cc
M be/src/exec/hdfs-table-sink.h
M be/src/exec/hdfs-table-writer.cc
M common/thrift/DataSinks.thrift
M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/UpdateStmt.java
M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java
M fe/src/main/java/org/apache/impala/planner/TableSink.java
9 files changed, 105 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/4863/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4863
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibeda0bdabbfe44c8ac95bf7c982a75649e1b82d0
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Lars Volker

[Impala-ASF-CR] IMPALA-3346: DeepCopy() Kudu rows into Impala tuples.

2016-10-26 Thread Alex Behm (Code Review)

Alex Behm has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4862

Change subject: IMPALA-3346: DeepCopy() Kudu rows into Impala tuples.
..

IMPALA-3346: DeepCopy() Kudu rows into Impala tuples.

Implements additional changes to make the memory layout
of Kudu rows identical to Impala tuples.
In particular, Kudu rows allocate a null bit even for
non-nullable columns, and Impala now does the same
for Kudu scan tuples.

This change exploits the now-identical Kudu and Impala
tuple layouts to avoid the expensive translation.

Perf: Mostafa is still confirming. Will update.

Testing: A private core/hdfs run passed.

Change-Id: Ic911e4eff9fe98bf28d8a1bab5c9d7e9ab66d9cb
---
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-scanner.h
M be/src/runtime/tuple.cc
M be/src/runtime/tuple.h
M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java
M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
7 files changed, 122 insertions(+), 205 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/4862/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4862
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic911e4eff9fe98bf28d8a1bab5c9d7e9ab66d9cb
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm

[Impala-ASF-CR] IMPALA-4309: Introduce Expr rewrite phase and supporting classes.

2016-10-26 Thread Alex Behm (Code Review)

Alex Behm has posted comments on this change.

Change subject: IMPALA-4309: Introduce Expr rewrite phase and supporting
classes.
..

Patch Set 7:

(2 comments)

Please take another look at the latest patch set.

- Did some final polish
- Resurrected the ExprRewriterTest

http://gerrit.cloudera.org:8080/#/c/4746/7/fe/src/main/java/org/apache/impala/planner/HdfsPartitionPruner.java
File fe/src/main/java/org/apache/impala/planner/HdfsPartitionPruner.java:

Line 102: public List prunePartitions(Analyzer analyzer,
List conjuncts)
> make this an ImmutableList to express that it doesn't change conjuncts? alt
The conjuncts list is actually modified. See existing comment.

I don't know of a simple way to guard against changing the Exprs in place.

Line 122: Expr clonedConjunct = exprRewriter_.rewrite(conjunct.clone(),
analyzer);
> why not simply directly apply the rule like before?
Because the conjunct could have nested BetweenPredicates and we now need an
ExprRewriter to handle that.

--
To view, visit http://gerrit.cloudera.org:8080/4746
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2279dc984bcf7742db4fa3b1aa67283ecbb05e6e
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Dimitris Tsirogiannis
Gerrit-Reviewer: Marcel Kornacker
Gerrit-Reviewer: Tim Armstrong
Gerrit-HasComments: Yes

57 matches

Mail list logo