[jira] [Resolved] (IMPALA-8785) DEBUG build images should be tagged differently from release build

2019-07-30 Thread Tim Armstrong (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong resolved IMPALA-8785.
---
   Resolution: Fixed
Fix Version/s: Impala 3.3.0

> DEBUG build images should be tagged differently from release build
> --
>
> Key: IMPALA-8785
> URL: https://issues.apache.org/jira/browse/IMPALA-8785
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: docker
> Fix For: Impala 3.3.0
>
>
> Currently the container build gives debug and release images the same tag 
> names. This is fine for development, since you'll generally be working on one 
> build at a time, but is confusing if you need to integrate this into a more 
> general build process, since debug and release images are really separate 
> artifacts.
> One example I ran into is a case when I ran ./buildall.sh ... 
> -release_and_debug, then wanted to build docker images for the release 
> artifacts. This is not currently possible in a sane way.
> I think we should tag images with a -debug suffix if they were generated from 
> debug artifacts. start-impala-cluster would need to be smart about picking 
> the image to use. We may want to have different build targets to build the 
> different flavours of image, too.
> Note that there are many flavours of debug images, e.g. ASAN, but I think 
> simply separating release and debug would avoid a lot of confusion.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8816) custom cluster tests in precommit are taking close to 2 hours

2019-07-30 Thread Tim Armstrong (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896672#comment-16896672
 ] 

Tim Armstrong commented on IMPALA-8816:
---

Actually, even without a special mode for start-impala-cluster, we can have 
detect the failure by polling that the processes still exist. I prototyped it 
like this:

{noformat}
tarmstrong@tarmstrong-box:~/Impala/incubator-impala$ git show
commit ff30f358118e392bf0912565b31cf4137f71a656
Author: Tim Armstrong 
Date:   Tue Jul 30 18:37:41 2019 -0700

Prototype fast fail

diff --git a/be/src/service/impala-server.cc b/be/src/service/impala-server.cc
index d2f44d4..2a983e7 100644
--- a/be/src/service/impala-server.cc
+++ b/be/src/service/impala-server.cc
@@ -2495,6 +2495,9 @@ Status ImpalaServer::Start(int32_t thrift_be_port, 
int32_t beeswax_port, int32_t
   // then wait for the initial catalog update.
   RETURN_IF_ERROR(exec_env_->StartStatestoreSubscriberService());

+  SleepForMs(1);
+  exit(1);
+
   if (FLAGS_is_coordinator) exec_env_->frontend()->WaitForCatalog();

   SSLProtocol ssl_version = SSLProtocol::TLSv1_0;
diff --git a/tests/common/impala_cluster.py b/tests/common/impala_cluster.py
index 6136091..fb900ea 100644
--- a/tests/common/impala_cluster.py
+++ b/tests/common/impala_cluster.py
@@ -169,9 +169,18 @@ class ImpalaCluster(object):
 if expected_num_ready_impalads is None:
   expected_num_ready_impalads = len(self.impalads)

+def impalads_went_away():
+  print "Check"
+  self.refresh()
+  # Number of impalads should not change.
+  assert expected_num_impalads == len(self.impalads)
+  assert self.statestored is not None
+  assert self.catalogd is not None
+
+
 for impalad in self.impalads:
   
impalad.service.wait_for_num_known_live_backends(expected_num_ready_impalads,
-  timeout=CLUSTER_WAIT_TIMEOUT_IN_SECONDS, interval=2)
+  timeout=CLUSTER_WAIT_TIMEOUT_IN_SECONDS, interval=2, 
early_term_fn=impalads_went_away)
   if (impalad._get_arg_value("is_coordinator", default="true") == "true" 
and
  impalad._get_arg_value("stress_catalog_init_delay_ms", default=0) == 
0):
 impalad.wait_for_catalog()
diff --git a/tests/common/impala_service.py b/tests/common/impala_service.py
index cba0a0f..7b06f3d 100644
--- a/tests/common/impala_service.py
+++ b/tests/common/impala_service.py
@@ -251,9 +251,10 @@ class ImpaladService(BaseImpalaService):
 return False

   def wait_for_num_known_live_backends(self, expected_value, timeout=30, 
interval=1,
-  include_shutting_down=True):
+  include_shutting_down=True, early_term_fn=lambda: False):
 start_time = time()
 while (time() - start_time < timeout):
+  early_term_fn()
   value = None
   try:
 value = self.get_num_known_live_backends(timeout=timeout, 
interval=interval,
{noformat}

> custom cluster tests in precommit are taking close to 2 hours
> -
>
> Key: IMPALA-8816
> URL: https://issues.apache.org/jira/browse/IMPALA-8816
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
>
> This is affecting precommit times substantially. We should either speed up 
> the tests or, more likely, move some to exhaustive.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8816) custom cluster tests in precommit are taking close to 2 hours

2019-07-30 Thread Tim Armstrong (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896665#comment-16896665
 ] 

Tim Armstrong commented on IMPALA-8816:
---

A large part of the problem is that start-impala-cluster.py is not designed to 
support these kind of startup failure tests. Some startup failure tests end up 
waiting for CLUSTER_WAIT_TIMEOUT_IN_SECONDS for a metric to increment. This is 
non-deterministic. It seems like much of the time the tests fail with a shorter 
timeout when processes fail to appear here - 
https://github.com/apache/impala/blob/72c9370856d7436885adbee3e8da7e7d9336df15/tests/common/impala_cluster.py#L166.
 But unlucky ones get stuck waiting for the longer metric timeout.
{noformat}
custom_cluster.test_query_event_hooks.TestHooksStartupFail.test_hook_startup_fail
 (from pytest)
Took 4 min 14 sec.
add description
Standard Error

-- 2019-07-30 22:53:23,755 INFO MainThread: Starting cluster with command: 
/home/ubuntu/Impala/bin/start-impala-cluster.py --cluster_size=1 
--num_coordinators=1 
--log_dir=/home/ubuntu/Impala/logs/custom_cluster_tests/test_hooks_startup_fail_uLTriL
 --log_level=1 
'--impalad_args=--query_event_hook_classes=org.apache.impala.testutil.AlwaysErrorQueryEventHook
 --minidump_path=/tmp/tmpBDmylN ' 
'--state_store_args=--statestore_update_frequency_ms=50 
--statestore_priority_update_frequency_ms=50 
...[truncated 14624 chars]...
ome/ubuntu/Impala/tests/common/impala_cluster.py", line 174, in wait_until_ready
timeout=CLUSTER_WAIT_TIMEOUT_IN_SECONDS, interval=2)
  File "/home/ubuntu/Impala/tests/common/impala_service.py", line 270, in 
wait_for_num_known_live_backends
assert 0, 'num_known_live_backends did not reach expected value in time'
AssertionError: num_known_live_backends did not reach expected value in time
-- 2019-07-30 22:57:29,083 DEBUGMainThread: Found 0 impalad/1 statestored/1 
catalogd process(es)
{noformat}

I think we could fix this entirely by specialising start-impala-cluster to have 
an "expected startup failure mode" that waits for the number of impalads to hit 
0, which should be quick, then throws an exception.

> custom cluster tests in precommit are taking close to 2 hours
> -
>
> Key: IMPALA-8816
> URL: https://issues.apache.org/jira/browse/IMPALA-8816
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
>
> This is affecting precommit times substantially. We should either speed up 
> the tests or, more likely, move some to exhaustive.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work started] (IMPALA-8668) Document impala-shell HS2 support

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-8668 started by Alex Rodoni.
---
> Document impala-shell HS2 support
> -
>
> Key: IMPALA-8668
> URL: https://issues.apache.org/jira/browse/IMPALA-8668
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Docs
>Reporter: Tim Armstrong
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: docs, future_release_doc, in_33
>
> https://gerrit.cloudera.org/#/c/13961/



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-8668) Document impala-shell HS2 support

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni updated IMPALA-8668:

Description: https://gerrit.cloudera.org/#/c/13961/

> Document impala-shell HS2 support
> -
>
> Key: IMPALA-8668
> URL: https://issues.apache.org/jira/browse/IMPALA-8668
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Docs
>Reporter: Tim Armstrong
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: docs, future_release_doc, in_33
>
> https://gerrit.cloudera.org/#/c/13961/



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-8791) Handle the case where there is no fragment scheduled on the coordinator for a query

2019-07-30 Thread Bikramjeet Vig (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bikramjeet Vig updated IMPALA-8791:
---
Description: 
For insert statements executed on a dedicated coord, the fragments get schduled 
only on executors but a query state object still gets started up on the coord 
host with the coord_mem_limit. we end up with a situation where the mem 
admitted is zero for the coord but the mem_reserved is non-zero which would 
affect other admission decisions.

There is also a case where there is no coordinator fragment but the execution 
fragment gets scheduled on the coord (eg. insert into  values) for 
this case, the mem admitted is per_backend_mem_limit_ but the mem limit applied 
to the coord query state is coord_backend_mem_limit_ which again causes a 
inconsistency.

  was:For insert statements executed on a dedicated coord, the fragments get 
schduled only on executors but a query state object still gets started up on 
the coord host with the coord_mem_limit. we end up with a situation where the 
mem admitted is zero for the coord but the mem_reserved is non-zero which would 
affect other admission decisions.


> Handle the case where there is no fragment scheduled on the coordinator for a 
> query
> ---
>
> Key: IMPALA-8791
> URL: https://issues.apache.org/jira/browse/IMPALA-8791
> Project: IMPALA
>  Issue Type: Bug
>Affects Versions: Product Backlog
>Reporter: Bikramjeet Vig
>Assignee: Bikramjeet Vig
>Priority: Major
>
> For insert statements executed on a dedicated coord, the fragments get 
> schduled only on executors but a query state object still gets started up on 
> the coord host with the coord_mem_limit. we end up with a situation where the 
> mem admitted is zero for the coord but the mem_reserved is non-zero which 
> would affect other admission decisions.
> There is also a case where there is no coordinator fragment but the execution 
> fragment gets scheduled on the coord (eg. insert into  values) for 
> this case, the mem admitted is per_backend_mem_limit_ but the mem limit 
> applied to the coord query state is coord_backend_mem_limit_ which again 
> causes a inconsistency.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8456) Impala Doc: Document HTTP based HS2/beeswax endpoints on coordinators

2019-07-30 Thread Alex Rodoni (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896639#comment-16896639
 ] 

Alex Rodoni commented on IMPALA-8456:
-

https://gerrit.cloudera.org/#/c/13960/

> Impala Doc: Document HTTP based HS2/beeswax endpoints on coordinators
> -
>
> Key: IMPALA-8456
> URL: https://issues.apache.org/jira/browse/IMPALA-8456
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Docs
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: future_release_doc, in_33
>
> Also document IMPALA-8717
> https://issues.apache.org/jira/browse/IMPALA-8783



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work started] (IMPALA-8456) Impala Doc: Document HTTP based HS2/beeswax endpoints on coordinators

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-8456 started by Alex Rodoni.
---
> Impala Doc: Document HTTP based HS2/beeswax endpoints on coordinators
> -
>
> Key: IMPALA-8456
> URL: https://issues.apache.org/jira/browse/IMPALA-8456
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Docs
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: future_release_doc, in_33
>
> Also document IMPALA-8717
> https://issues.apache.org/jira/browse/IMPALA-8783



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8816) custom cluster tests in precommit are taking close to 2 hours

2019-07-30 Thread Tim Armstrong (JIRA)
Tim Armstrong created IMPALA-8816:
-

 Summary: custom cluster tests in precommit are taking close to 2 
hours
 Key: IMPALA-8816
 URL: https://issues.apache.org/jira/browse/IMPALA-8816
 Project: IMPALA
  Issue Type: Bug
  Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong


This is affecting precommit times substantially. We should either speed up the 
tests or, more likely, move some to exhaustive.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8815) Ranger fail to start in minicluster if you source set-classpath.sh

2019-07-30 Thread Tim Armstrong (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896625#comment-16896625
 ] 

Tim Armstrong commented on IMPALA-8815:
---

After I fixed the immediate failure, there were some more failures that I think 
were because of incompatible versions of dependencies - it was unable to find 
some methods in some jackson class.

> Ranger fail to start in minicluster if you source set-classpath.sh
> --
>
> Key: IMPALA-8815
> URL: https://issues.apache.org/jira/browse/IMPALA-8815
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>
> The below appear in logs/cluster/ranger/catalina.out
> {noformat}
> java.io.FileNotFoundException: 
> /home/tarmstrong/Impala/incubator-impala/toolchain/cdh_components-1173663/hive-2.1.1-cdh6.x-SNAPSHOT/lib/datanucleus-api-jdo-3.2.1.jar
>  (No such file or directory)
> at java.util.zip.ZipFile.open(Native Method)
> at java.util.zip.ZipFile.(ZipFile.java:225)
> at java.util.zip.ZipFile.(ZipFile.java:155)
> at java.util.jar.JarFile.(JarFile.java:166)
> at java.util.jar.JarFile.(JarFile.java:103)
> at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93)
> at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)
> at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99)
> at 
> sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
> at 
> sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)
> at org.apache.tomcat.util.scan.FileUrlJar.(FileUrlJar.java:48)
> at 
> org.apache.tomcat.util.scan.JarFactory.newInstance(JarFactory.java:34)
> at 
> org.apache.catalina.startup.ContextConfig$FragmentJarScannerCallback.scan(ContextConfig.java:2682)
> at 
> org.apache.tomcat.util.scan.StandardJarScanner.process(StandardJarScanner.java:325)
> at 
> org.apache.tomcat.util.scan.StandardJarScanner.doScanClassPath(StandardJarScanner.java:235)
> at 
> org.apache.tomcat.util.scan.StandardJarScanner.scan(StandardJarScanner.java:196)
> at 
> org.apache.catalina.startup.ContextConfig.processJarsForWebFragments(ContextConfig.java:1913)
> at 
> org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1264)
> at 
> org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:881)
> at 
> org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:388)
> at 
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
> at 
> org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
> at 
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5606)
> at 
> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> at 
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
> at 
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8785) DEBUG build images should be tagged differently from release build

2019-07-30 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896626#comment-16896626
 ] 

ASF subversion and git services commented on IMPALA-8785:
-

Commit def70c241dffcdab8301cbc7456ea6680ca01692 in impala's branch 
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=def70c2 ]

IMPALA-8785: give debug docker images a different name

* Build scripts are generalised to have different targets for release
  and debug images.
* Added new targets for the debug images: docker_debug_images,
  statestored_debug images. The release images still have the
  same names.
* Separate build contexts are set up for the different base
  images.
* The debug or release base image can be specified as the FROM
  for the daemon images.
* start-impala-cluster.py picks the correct images for the build type

Future work:
We would like to generalise this to allow building from
non-ubuntu-16.04 base images. This probably requires another
layer of dockerfiles to specify a base image for impala_base
with the required packages installed.

Change-Id: I32d2e19cb671beacceebb2642aba01191bd7a244
Reviewed-on: http://gerrit.cloudera.org:8080/13905
Reviewed-by: Joe McDonnell 
Tested-by: Impala Public Jenkins 


> DEBUG build images should be tagged differently from release build
> --
>
> Key: IMPALA-8785
> URL: https://issues.apache.org/jira/browse/IMPALA-8785
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: docker
>
> Currently the container build gives debug and release images the same tag 
> names. This is fine for development, since you'll generally be working on one 
> build at a time, but is confusing if you need to integrate this into a more 
> general build process, since debug and release images are really separate 
> artifacts.
> One example I ran into is a case when I ran ./buildall.sh ... 
> -release_and_debug, then wanted to build docker images for the release 
> artifacts. This is not currently possible in a sane way.
> I think we should tag images with a -debug suffix if they were generated from 
> debug artifacts. start-impala-cluster would need to be smart about picking 
> the image to use. We may want to have different build targets to build the 
> different flavours of image, too.
> Note that there are many flavours of debug images, e.g. ASAN, but I think 
> simply separating release and debug would avoid a lot of confusion.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8815) Ranger fail to start in minicluster if you source set-classpath.sh

2019-07-30 Thread Tim Armstrong (JIRA)
Tim Armstrong created IMPALA-8815:
-

 Summary: Ranger fail to start in minicluster if you source 
set-classpath.sh
 Key: IMPALA-8815
 URL: https://issues.apache.org/jira/browse/IMPALA-8815
 Project: IMPALA
  Issue Type: Bug
  Components: Infrastructure
Reporter: Tim Armstrong
Assignee: Tim Armstrong


The below appear in logs/cluster/ranger/catalina.out
{noformat}
java.io.FileNotFoundException: 
/home/tarmstrong/Impala/incubator-impala/toolchain/cdh_components-1173663/hive-2.1.1-cdh6.x-SNAPSHOT/lib/datanucleus-api-jdo-3.2.1.jar
 (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:225)
at java.util.zip.ZipFile.(ZipFile.java:155)
at java.util.jar.JarFile.(JarFile.java:166)
at java.util.jar.JarFile.(JarFile.java:103)
at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93)
at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)
at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99)
at 
sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
at 
sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)
at org.apache.tomcat.util.scan.FileUrlJar.(FileUrlJar.java:48)
at 
org.apache.tomcat.util.scan.JarFactory.newInstance(JarFactory.java:34)
at 
org.apache.catalina.startup.ContextConfig$FragmentJarScannerCallback.scan(ContextConfig.java:2682)
at 
org.apache.tomcat.util.scan.StandardJarScanner.process(StandardJarScanner.java:325)
at 
org.apache.tomcat.util.scan.StandardJarScanner.doScanClassPath(StandardJarScanner.java:235)
at 
org.apache.tomcat.util.scan.StandardJarScanner.scan(StandardJarScanner.java:196)
at 
org.apache.catalina.startup.ContextConfig.processJarsForWebFragments(ContextConfig.java:1913)
at 
org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1264)
at 
org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:881)
at 
org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:388)
at 
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at 
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at 
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5606)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at 
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
at 
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work started] (IMPALA-8814) Impala Doc: Webui and SPNEGO

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-8814 started by Alex Rodoni.
---
> Impala Doc: Webui and SPNEGO
> 
>
> Key: IMPALA-8814
> URL: https://issues.apache.org/jira/browse/IMPALA-8814
> Project: IMPALA
>  Issue Type: Task
>  Components: Docs
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: future_release_doc, in_33
>
> https://gerrit.cloudera.org/#/c/13774/
> Doc review at: https://gerrit.cloudera.org/#/c/13958/



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-8814) Impala Doc: Webui and SPNEGO

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni updated IMPALA-8814:

Description: 
https://gerrit.cloudera.org/#/c/13774/

Doc review at: https://gerrit.cloudera.org/#/c/13958/

  was:https://gerrit.cloudera.org/#/c/13774/


> Impala Doc: Webui and SPNEGO
> 
>
> Key: IMPALA-8814
> URL: https://issues.apache.org/jira/browse/IMPALA-8814
> Project: IMPALA
>  Issue Type: Task
>  Components: Docs
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: future_release_doc, in_33
>
> https://gerrit.cloudera.org/#/c/13774/
> Doc review at: https://gerrit.cloudera.org/#/c/13958/



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8738) Add a column representing the type(table or view) in the show tables output

2019-07-30 Thread Thomas Tauber-Marshall (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896561#comment-16896561
 ] 

Thomas Tauber-Marshall commented on IMPALA-8738:


Seems like a reasonable suggestion. Do you know if other DBs, eg. Hive, do 
something similar? We tend to try to stick to general conventions with this 
sort of stuff

> Add a column representing the type(table or view) in the show tables output
> ---
>
> Key: IMPALA-8738
> URL: https://issues.apache.org/jira/browse/IMPALA-8738
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Catalog, Frontend
>Reporter: baotuquan
>Assignee: baotuquan
>Priority: Minor
>  Labels: features
>
> Now the output of the +*show tables*+ command in the system is as follows:
> {code:java}
> default> show tables;
> Query: show tables
> +--+
> | name |
> +--+
> | table1 |
> | view1 |
> +--+
> {code}
> I think we should add a column for the representation type,The output should 
> be like this
> {code:java}
> default> show tables;
> Query: show tables
> +---+
> | name | type |
> ++
> | table1 |  table   |
> | view1 |   view   |
> ++
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8814) Impala Doc: Webui and SPNEGO

2019-07-30 Thread Alex Rodoni (JIRA)
Alex Rodoni created IMPALA-8814:
---

 Summary: Impala Doc: Webui and SPNEGO
 Key: IMPALA-8814
 URL: https://issues.apache.org/jira/browse/IMPALA-8814
 Project: IMPALA
  Issue Type: Task
  Components: Docs
Reporter: Alex Rodoni
Assignee: Alex Rodoni


https://gerrit.cloudera.org/#/c/13774/



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8813) Impala Doc: Read/Write Hive ACID tables

2019-07-30 Thread Alex Rodoni (JIRA)
Alex Rodoni created IMPALA-8813:
---

 Summary: Impala Doc: Read/Write Hive ACID tables
 Key: IMPALA-8813
 URL: https://issues.apache.org/jira/browse/IMPALA-8813
 Project: IMPALA
  Issue Type: Sub-task
  Components: Docs
Reporter: Alex Rodoni
Assignee: Alex Rodoni






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Assigned] (IMPALA-8347) Support read/write Hive ACID tables

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni reassigned IMPALA-8347:
---

Assignee: (was: Alex Rodoni)

> Support read/write Hive ACID tables
> ---
>
> Key: IMPALA-8347
> URL: https://issues.apache.org/jira/browse/IMPALA-8347
> Project: IMPALA
>  Issue Type: Epic
>  Components: Backend, Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Lars Volker
>Priority: Critical
>  Labels: impala-acid
>
> Hive added ACID transactional support in HIVE-5317. Impala should be able to 
> read Hive ACID tables.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work started] (IMPALA-8347) Support read/write Hive ACID tables

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-8347 started by Alex Rodoni.
---
> Support read/write Hive ACID tables
> ---
>
> Key: IMPALA-8347
> URL: https://issues.apache.org/jira/browse/IMPALA-8347
> Project: IMPALA
>  Issue Type: Epic
>  Components: Backend, Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Lars Volker
>Assignee: Alex Rodoni
>Priority: Critical
>  Labels: impala-acid
>
> Hive added ACID transactional support in HIVE-5317. Impala should be able to 
> read Hive ACID tables.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work stopped] (IMPALA-8347) Support read/write Hive ACID tables

2019-07-30 Thread Alex Rodoni (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-8347 stopped by Alex Rodoni.
---
> Support read/write Hive ACID tables
> ---
>
> Key: IMPALA-8347
> URL: https://issues.apache.org/jira/browse/IMPALA-8347
> Project: IMPALA
>  Issue Type: Epic
>  Components: Backend, Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Lars Volker
>Assignee: Alex Rodoni
>Priority: Critical
>  Labels: impala-acid
>
> Hive added ACID transactional support in HIVE-5317. Impala should be able to 
> read Hive ACID tables.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8636) Implement INSERT for insert-only ACID tables

2019-07-30 Thread Alex Rodoni (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896484#comment-16896484
 ] 

Alex Rodoni commented on IMPALA-8636:
-

[~boroknagyz] Any user-facing doc impact?

> Implement INSERT for insert-only ACID tables
> 
>
> Key: IMPALA-8636
> URL: https://issues.apache.org/jira/browse/IMPALA-8636
> Project: IMPALA
>  Issue Type: New Feature
>Reporter: Zoltán Borók-Nagy
>Assignee: Zoltán Borók-Nagy
>Priority: Critical
>  Labels: impala-acid
> Fix For: Impala 4.0, Impala 3.3.0
>
>
> Impala should support insertion for insert-only ACID tables.
> For this we need to allocate a write ID for the target table, and write the 
> data into the base/delta directories.
> INSERT operation should create a new delta directory with the allocated write 
> ID.
> INSERT OVERWRITE should create a new base directory with the allocated write 
> ID. This new base directory will only contain the data coming from this 
> operation.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8812) Impala Doc: SPLIT_PART to support negative indexes

2019-07-30 Thread Alex Rodoni (JIRA)
Alex Rodoni created IMPALA-8812:
---

 Summary: Impala Doc: SPLIT_PART to support negative indexes
 Key: IMPALA-8812
 URL: https://issues.apache.org/jira/browse/IMPALA-8812
 Project: IMPALA
  Issue Type: Sub-task
  Components: Docs
Reporter: Alex Rodoni
Assignee: Alex Rodoni






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-7770) SPLIT_PART to support negative indexes

2019-07-30 Thread Alex Rodoni (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896456#comment-16896456
 ] 

Alex Rodoni commented on IMPALA-7770:
-

[~norbertluksa] Are you targeting the 3.3 release for this feature?

> SPLIT_PART to support negative indexes
> --
>
> Key: IMPALA-7770
> URL: https://issues.apache.org/jira/browse/IMPALA-7770
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Reporter: Tristan Stevens
>Assignee: Norbert Luksa
>Priority: Major
>  Labels: built-in-function
>
> Request is for SPLIT_PART to support negative indexes i.e. support right to 
> left searching.
> See Snowflake documentation for details: 
> [https://docs.snowflake.net/manuals/sql-reference/functions/split_part.html|https://docs.snowflake.net/manuals/sql-reference/functions/split_part.html]
> partNr: Requested part of the split (1-based). 0 is treated as 1. If the 
> value is negative, the parts are counted from the right side of the string.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8811) Impala Doc: query option to change default ACID type of new tables

2019-07-30 Thread Alex Rodoni (JIRA)
Alex Rodoni created IMPALA-8811:
---

 Summary: Impala Doc: query option to change default ACID type of 
new tables
 Key: IMPALA-8811
 URL: https://issues.apache.org/jira/browse/IMPALA-8811
 Project: IMPALA
  Issue Type: Sub-task
  Components: Docs
Reporter: Alex Rodoni
Assignee: Alex Rodoni






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8808) Add query option to change default ACID type of new tables

2019-07-30 Thread Alex Rodoni (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896438#comment-16896438
 ] 

Alex Rodoni commented on IMPALA-8808:
-

[~csringhofer] Are you targeting this feature for 3.3?

> Add query option to change default ACID type of new tables
> --
>
> Key: IMPALA-8808
> URL: https://issues.apache.org/jira/browse/IMPALA-8808
> Project: IMPALA
>  Issue Type: New Feature
>  Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8810) ExplainTest.testScanNodeFsScheme fails if Orc support is disabled

2019-07-30 Thread Csaba Ringhofer (JIRA)
Csaba Ringhofer created IMPALA-8810:
---

 Summary: ExplainTest.testScanNodeFsScheme fails if Orc support is 
disabled
 Key: IMPALA-8810
 URL: https://issues.apache.org/jira/browse/IMPALA-8810
 Project: IMPALA
  Issue Type: Bug
  Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Csaba Ringhofer


The problem is caused by going into this block when Orc support is disabled: 
https://github.com/apache/impala/blob/51f30a6b9088cf3a0217a49a5792f1b41a8ad51b/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java#L416
+ that ExplainTest only mocks HdfsPartition.getFileFormat() but not 
HdfsPartition.part.getInputFormatDescriptor().





--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8809) Refresh a subset of partitions for ACID tables

2019-07-30 Thread Gabor Kaszab (JIRA)
Gabor Kaszab created IMPALA-8809:


 Summary: Refresh a subset of partitions for ACID tables
 Key: IMPALA-8809
 URL: https://issues.apache.org/jira/browse/IMPALA-8809
 Project: IMPALA
  Issue Type: Improvement
Affects Versions: Impala 3.3.0
Reporter: Gabor Kaszab


Enhancing REFRESH logic to handle ACID tables was covered by this change: 
https://issues.apache.org/jira/browse/IMPALA-8600
Basically each user initiated REFRESH PARTITION is rejected meanwhile the 
REFRESH_PARTITION event in event processor are actually doing a full table load 
for ACID tables.

There is room for improvement: When a full table refresh is being executed on 
an ACID table we can have 2 scenarios:
- If there was some schema changes then reload the full table. Identify such a 
scenario should be possible by checking the table-level writeId. However, there 
is a bug in Hive that it doesn't update that field for partitioned tables 
(https://issues.apache.org/jira/browse/HIVE-22062). This would be the desired 
way but could also be workarounded by checking other fields lik lastDdlChanged 
or such.
- If a full table refresh is not needed then we should fetch the 
partition-level writeIds and reload only the ones that are out-of-date locally.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8339) Coordinator should be more resilient to fragment instances startup failure

2019-07-30 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896223#comment-16896223
 ] 

ASF subversion and git services commented on IMPALA-8339:
-

Commit dfc968dff1eabc71871e2d941fbe9539944c3e88 in impala's branch 
refs/heads/master from Thomas Tauber-Marshall
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=dfc968d ]

IMPALA-8339: Add local executor blacklist to coordinators

This patch adds the concept of a blacklist of executors to the
coordinator, which removes executors from consideration for query
scheduling. Blacklisting decisions are local to a given coordinator
and are not included in statestore updates.

The intention is to allow coordinators to be more aggressive about
deciding that an exeutor is unhealthy or unavailable, to minimize
failed queries in environments where cluster membership may be more
variable, rather than having to wait on the statestore heartbeat
mechanism to decide that the executor is down.

For the first patch, executors will only be blacklisted if the KRPC
status for Exec() is an error. Followup work will add blacklisting of
executors in more complex scenarios, eg. if an executor appears to be
a straggler.

When a query is scheduled and there is currently some blacklisted
executors, a new line 'Blacklisted Executors:' is added to the profile
listing the hostnames of all such executors.

Testing:
- Added a case to the cluster mgr BE unit test that uses blacklisting.
- Added e2e test cases for killing and restarting an impalad.
- Manual randomized testing locally with iptables.
TODO
- Add an e2e test case where an impalad becomes briefly unreachable.
- Manual/stress tests on a real cluster.

Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74
Reviewed-on: http://gerrit.cloudera.org:8080/13868
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 


> Coordinator should be more resilient to fragment instances startup failure
> --
>
> Key: IMPALA-8339
> URL: https://issues.apache.org/jira/browse/IMPALA-8339
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Distributed Exec
>Reporter: Michael Ho
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
>  Labels: Availability, resilience
>
> Impala currently relies on statestore for cluster membership. When an Impala 
> executor goes offline, it may take a while for statestore to declare that 
> node as unavailable and for that information to be propagated to all 
> coordinator nodes. Within this window, some coordinator nodes may still 
> attempt to issue RPCs to the faulty node, resulting in RPC failures which 
> resulted in query failures. In other words, many queries may fail to start 
> within this window until all coordinator nodes get the latest information on 
> cluster membership.
> Going forward, coordinator may need to fall back to using backup executors 
> for each fragments in case some of the executors are not available. Moreover, 
> *coordinator should treat the cluster membership information from statestore 
> (or any external source of truth e.g. etcd) as hints instead of ground truth* 
> and adjust the scheduling of fragment instances based on the availability of 
> the executors from the coordinator's perspective.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Assigned] (IMPALA-8808) Add query option to change default ACID type of new tables

2019-07-30 Thread Csaba Ringhofer (JIRA)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csaba Ringhofer reassigned IMPALA-8808:
---

Assignee: Csaba Ringhofer

> Add query option to change default ACID type of new tables
> --
>
> Key: IMPALA-8808
> URL: https://issues.apache.org/jira/browse/IMPALA-8808
> Project: IMPALA
>  Issue Type: New Feature
>  Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-8808) Add query option to change default ACID type of new tables

2019-07-30 Thread Csaba Ringhofer (JIRA)
Csaba Ringhofer created IMPALA-8808:
---

 Summary: Add query option to change default ACID type of new tables
 Key: IMPALA-8808
 URL: https://issues.apache.org/jira/browse/IMPALA-8808
 Project: IMPALA
  Issue Type: New Feature
  Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Csaba Ringhofer






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8600) Reload partition does not work for transactional tables

2019-07-30 Thread Gabor Kaszab (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896178#comment-16896178
 ] 

Gabor Kaszab commented on IMPALA-8600:
--

I ran into a Hive issue during the implementation: Hive doesn't increment any 
writeIds if I change the schema of a partitioned ACID table. It increments the 
high watermark, however. I opened a Jira: 
https://issues.apache.org/jira/browse/HIVE-22062


> Reload partition does not work for transactional tables
> ---
>
> Key: IMPALA-8600
> URL: https://issues.apache.org/jira/browse/IMPALA-8600
> Project: IMPALA
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Gabor Kaszab
>Priority: Major
>  Labels: impala-acid
>
> If a table is transactional, a reload partition call should fetch the valid 
> writeIds. Without doing this, the reload will skip adding all the newly 
> created delta files of the transactional table pertaining to the new writeIds.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-7770) SPLIT_PART to support negative indexes

2019-07-30 Thread Norbert Luksa (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896104#comment-16896104
 ] 

Norbert Luksa commented on IMPALA-7770:
---

Feel free to review:

https://gerrit.cloudera.org/#/c/13880/

> SPLIT_PART to support negative indexes
> --
>
> Key: IMPALA-7770
> URL: https://issues.apache.org/jira/browse/IMPALA-7770
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Reporter: Tristan Stevens
>Assignee: Norbert Luksa
>Priority: Major
>  Labels: built-in-function
>
> Request is for SPLIT_PART to support negative indexes i.e. support right to 
> left searching.
> See Snowflake documentation for details: 
> [https://docs.snowflake.net/manuals/sql-reference/functions/split_part.html|https://docs.snowflake.net/manuals/sql-reference/functions/split_part.html]
> partNr: Requested part of the split (1-based). 0 is treated as 1. If the 
> value is negative, the parts are counted from the right side of the string.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8752) Add Jaro-winkler edit distance and similarity built-in function

2019-07-30 Thread Norbert Luksa (JIRA)


[ 
https://issues.apache.org/jira/browse/IMPALA-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896106#comment-16896106
 ] 

Norbert Luksa commented on IMPALA-8752:
---

https://gerrit.cloudera.org/#/c/13870/

> Add Jaro-winkler edit distance and similarity built-in function
> ---
>
> Key: IMPALA-8752
> URL: https://issues.apache.org/jira/browse/IMPALA-8752
> Project: IMPALA
>  Issue Type: New Feature
>Reporter: Norbert Luksa
>Assignee: Norbert Luksa
>Priority: Major
>  Labels: built-in-function
>
> References:
>  * [Apache commons - JaroWinklerDistance 
> |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerDistance.html]]
>  * [Apache commons - JaroWinklerSimilarity 
> |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerSimilarity.html]]
>  * [Oracle - 
> JARO_WINKLER[_SIMILARITY]|[https://oracle-base.com/articles/11g/utl_match-string-matching-in-oracle]]
> Notable difference:
>  * With similarity, the Oracle version returns a normalized result ranging 
> from 0 to 100.
>  * In the Appache version, null values result in exceptions.
>  * Apache rounds the values to two digits 
> The scaling factor of the algorithm can be added as an extra/default argument.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org