[jira] [Created] (IMPALA-13099) ImpalaOperator getAllowedSignatures needs to be implemented
Steve Carlin created IMPALA-13099: - Summary: ImpalaOperator getAllowedSignatures needs to be implemented Key: IMPALA-13099 URL: https://issues.apache.org/jira/browse/IMPALA-13099 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin This method is used to show what syntax is allowed for a given function. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13098) computeEvalCost needs better implementation in Calcite planner
Steve Carlin created IMPALA-13098: - Summary: computeEvalCost needs better implementation in Calcite planner Key: IMPALA-13098 URL: https://issues.apache.org/jira/browse/IMPALA-13098 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin Right now, computeEvalCost is always returning UNKNOWN_COST. The costing needs to be calculated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13097) Better exception for Analyze*Expr class in Calcite Planner
Steve Carlin created IMPALA-13097: - Summary: Better exception for Analyze*Expr class in Calcite Planner Key: IMPALA-13097 URL: https://issues.apache.org/jira/browse/IMPALA-13097 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin Some of the Analyzed*Expr classes throw a RuntimeException. There should be a cleaner exception thrown in these cases. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13095) Handle UDFs in Calcite planner
Steve Carlin created IMPALA-13095: - Summary: Handle UDFs in Calcite planner Key: IMPALA-13095 URL: https://issues.apache.org/jira/browse/IMPALA-13095 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13042) Need to add partition pruning
Steve Carlin created IMPALA-13042: - Summary: Need to add partition pruning Key: IMPALA-13042 URL: https://issues.apache.org/jira/browse/IMPALA-13042 Project: IMPALA Issue Type: Bug Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13011) Need to remove awkward Authorization instantiation.
Steve Carlin created IMPALA-13011: - Summary: Need to remove awkward Authorization instantiation. Key: IMPALA-13011 URL: https://issues.apache.org/jira/browse/IMPALA-13011 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin There is a reference to the Authorization instance in CalcitePhysPlanCreator in order to instantiate the Analyzer object. Authorization needs to happen earlier. This should be refactored so it is not referenced in this part of the code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12964) Implement aggregation capability
Steve Carlin created IMPALA-12964: - Summary: Implement aggregation capability Key: IMPALA-12964 URL: https://issues.apache.org/jira/browse/IMPALA-12964 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12961) Use a Map instead of an ArrayList for Expr in HDFS RelNode
Steve Carlin created IMPALA-12961: - Summary: Use a Map instead of an ArrayList for Expr in HDFS RelNode Key: IMPALA-12961 URL: https://issues.apache.org/jira/browse/IMPALA-12961 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin This came up in code review in ImpalaHdfsScanRel: "For wide tables where we are only needing a few columns projected, we will end up with a long list with mostly Nulls. A LinkedHashMap (preserves Insertion order) where the key is position and value is the SlotRef would be better suited despite the cpu cost of hashing. In general, in a query planner, memory is the most precious commodity since the plan search space can be large, so anything we can do to reduce memory footprint would be preferred." One counter argument: The list is used in other Rel Nodes, and it seems more natural. For instance, the Project RelNode will have a RexInputRef RexNode which is "$2". It seems more natural to have an array in this case. Every other RelNode works this way except for the ScanNode. To add to the counter argument: Let's take a worst case scenario of a query that has 10 tables with 500 columns apiece. If we are allocating 8 byte pointers, we would need 10*500*8 to hold this information, which is 40,000 bytes. While reducing the memory footprint is more important, reducing it by 40,000 bytes really isn't going to make an impact. Even if we take into account that multiple queries would be running simultaneously, this is a very shortlived code path. So should we go with the more natural approach versus the less memory intensive approach? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12959) Calcite planner: Need to code Parquet Count Star Optimization
Steve Carlin created IMPALA-12959: - Summary: Calcite planner: Need to code Parquet Count Star Optimization Key: IMPALA-12959 URL: https://issues.apache.org/jira/browse/IMPALA-12959 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12958) Calcite planner: Invert Joins should be implemented as a rule
Steve Carlin created IMPALA-12958: - Summary: Calcite planner: Invert Joins should be implemented as a rule Key: IMPALA-12958 URL: https://issues.apache.org/jira/browse/IMPALA-12958 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12954) Implement Sort capability
Steve Carlin created IMPALA-12954: - Summary: Implement Sort capability Key: IMPALA-12954 URL: https://issues.apache.org/jira/browse/IMPALA-12954 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12940) Implement filtering conditions
Steve Carlin created IMPALA-12940: - Summary: Implement filtering conditions Key: IMPALA-12940 URL: https://issues.apache.org/jira/browse/IMPALA-12940 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12935) Allow function parsing for Impala Calcite planner
Steve Carlin created IMPALA-12935: - Summary: Allow function parsing for Impala Calcite planner Key: IMPALA-12935 URL: https://issues.apache.org/jira/browse/IMPALA-12935 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin We need the ability to parse and validate Impala functions using the Calcite planner This commit is not attended to work for all functions, or even most functions. It will work as a base to be reviewed, and at least some functions will work. More complicated functions will be added in a later commit. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12934) Import parser files from Calcite into Impala
Steve Carlin created IMPALA-12934: - Summary: Import parser files from Calcite into Impala Key: IMPALA-12934 URL: https://issues.apache.org/jira/browse/IMPALA-12934 Project: IMPALA Issue Type: Sub-task Reporter: Steve Carlin Since the Impala sql syntax is different from the Calcite sql syntax, Impala needs it's own parsing files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12872) Support basic queries using Calcite on the frontend
Steve Carlin created IMPALA-12872: - Summary: Support basic queries using Calcite on the frontend Key: IMPALA-12872 URL: https://issues.apache.org/jira/browse/IMPALA-12872 Project: IMPALA Issue Type: Sub-task Components: fe Reporter: Steve Carlin First commit for the Calcite planner. The idea is to make minimal changes to the current frontend code and keep this code in a separate jar. The first commit will support a basic select statement without any filters or functions (e.g. select * from tbl and select c1 from tbl) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12871) Implement front end using Calcite
Steve Carlin created IMPALA-12871: - Summary: Implement front end using Calcite Key: IMPALA-12871 URL: https://issues.apache.org/jira/browse/IMPALA-12871 Project: IMPALA Issue Type: Epic Components: fe Reporter: Steve Carlin Calcite provides a framework for compilation that will allow us to provide optimizations more easily and also allow more support for SQL standard queries. This Epic will track the multiple commits needed to support this -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12871) Implement front end using Calcite
[ https://issues.apache.org/jira/browse/IMPALA-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-12871: - Assignee: Steve Carlin > Implement front end using Calcite > - > > Key: IMPALA-12871 > URL: https://issues.apache.org/jira/browse/IMPALA-12871 > Project: IMPALA > Issue Type: Epic > Components: fe >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > Calcite provides a framework for compilation that will allow us to provide > optimizations more easily and also allow more support for SQL standard > queries. This Epic will track the multiple commits needed to support this -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12424) Allow third party extensibility for JniFrontend
[ https://issues.apache.org/jira/browse/IMPALA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-12424: - Assignee: Steve Carlin > Allow third party extensibility for JniFrontend > --- > > Key: IMPALA-12424 > URL: https://issues.apache.org/jira/browse/IMPALA-12424 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > The JniFrontend java class is called through Jni on the backend. We should > allow a developer to create their own JniFrontend class if they want to use > their own planner to create an Impala request -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12424) Allow third party extensibility for JniFrontend
Steve Carlin created IMPALA-12424: - Summary: Allow third party extensibility for JniFrontend Key: IMPALA-12424 URL: https://issues.apache.org/jira/browse/IMPALA-12424 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Steve Carlin The JniFrontend java class is called through Jni on the backend. We should allow a developer to create their own JniFrontend class if they want to use their own planner to create an Impala request -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12078) The ".test" file shows NULL as a string
Steve Carlin created IMPALA-12078:
-
Summary: The ".test" file shows NULL as a string
Key: IMPALA-12078
URL: https://issues.apache.org/jira/browse/IMPALA-12078
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Steve Carlin
This seems minor.
In the alter-table.sql file, after the
{code:java}
insert overwrite table t_part partition(j=NULL, s=NULL) select 4 from
functional.alltypes limit 5{code}
there is a select that should show s as NULL. It is showing s as 'NULL'
(within a string)
Running it in impala-shell shows that it is actually NULL, so the framework is
calculating equivalency correctly, it is just not displaying the value
correctly in the text file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12012) Java UDFs currently do not support the TIMESTAMP datatype
Steve Carlin created IMPALA-12012: - Summary: Java UDFs currently do not support the TIMESTAMP datatype Key: IMPALA-12012 URL: https://issues.apache.org/jira/browse/IMPALA-12012 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11895) Need accessor methods for third party extension
Steve Carlin created IMPALA-11895: - Summary: Need accessor methods for third party extension Key: IMPALA-11895 URL: https://issues.apache.org/jira/browse/IMPALA-11895 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin Some accessor methods are needed for a third party extension on some of the StatementBase classes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11894) Need accessor methods for third party extension
Steve Carlin created IMPALA-11894: - Summary: Need accessor methods for third party extension Key: IMPALA-11894 URL: https://issues.apache.org/jira/browse/IMPALA-11894 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin Some accessor methods are needed for a third party extension on some of the StatementBase classes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11766) In HdfsScanNode.buildBinaryStatsPredicate, an Expr needs to be cloned
Steve Carlin created IMPALA-11766:
-
Summary: In HdfsScanNode.buildBinaryStatsPredicate, an Expr needs
to be cloned
Key: IMPALA-11766
URL: https://issues.apache.org/jira/browse/IMPALA-11766
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Steve Carlin
In HdfsScanNode.buildBinaryStatsPredicate exists the following code:
{code:java}
private void buildBinaryStatsPredicate(Analyzer analyzer, SlotRef inputSlot,
BinaryPredicate inputPred, BinaryPredicate.Operator op) {
// Obtain the rhs expr of the input predicate
Expr constExpr = inputPred.getChild(1);
...
BinaryPredicate statsPred = new BinaryPredicate(op, slot, constExpr);{code}
This contains a potential bug. The constExpr is mutable and the creation of
the new BinaryPredicate can change the innards of the inputPred Expr.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10997) Refactor code for UDF implementation
[ https://issues.apache.org/jira/browse/IMPALA-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-10997. --- Resolution: Fixed > Refactor code for UDF implementation > > > Key: IMPALA-10997 > URL: https://issues.apache.org/jira/browse/IMPALA-10997 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > In its current form, Impala supports Java UDFs that are derived from the > UDF.class. > The UDF.class is legacy code and Hive now supports implementation off of the > GenericUDF.class. > This Jira is to track the refactoring the current UDF code to allow easier > implementation when we eventually support the newer class. Common code will > be put in a base HiveExecutor class and UDF.class specific code will be in a > derived class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11528) hive-exec.pom doesn't include UDAF class
[
https://issues.apache.org/jira/browse/IMPALA-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585600#comment-17585600
]
Steve Carlin commented on IMPALA-11528:
---
I should also note that I don't think the issue is that we want to include the
UDAF class in the pom.xml. At this point, we don't support java user defined
aggregate functions. It just so happens that this UDF was derived from UDAF
which wasn't in the class path. But as shown in IMPALA-11109, it could have
been any class that was missing and the same error would have happened.
> hive-exec.pom doesn't include UDAF class
>
>
> Key: IMPALA-11528
> URL: https://issues.apache.org/jira/browse/IMPALA-11528
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.1.0
>Reporter: Gabor Kaszab
>Priority: Major
>
> For hive-exec we load "*UDF*" that excludes the UDAF class:
> https://github.com/apache/impala/blob/master/java/shaded-deps/hive-exec/pom.xml#L102
> As a result when a UDAF is being loaded to catalog we get a
> NoClassDefFoundException.
> {code:java}
> I0819 09:20:07.777845 1 HiveUdfLoader.java:63] Loading UDF
> 'eu.radoop.datahandler.hive.udf.GenericUDAFCorrelationMatrix' from
> file:/tmp/e5a348f5-753a-485a-b37d-2a1420b09df7.jar
> I0819 09:20:07.780457 1 MetastoreEventsProcessor.java:700] Metastore
> event processing restarted. Last synced event id was updated from 902310 to
> 902310
> I0819 09:20:07.780704 1 jni-util.cc:286] java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> at
> org.apache.hadoop.hive.ql.exec.FunctionUtils.getUDFClassType(FunctionUtils.java:157)
> at
> org.apache.impala.hive.executor.HiveUdfLoader.(HiveUdfLoader.java:68)
> at
> org.apache.impala.hive.executor.HiveUdfLoader.createWithLocalPath(HiveUdfLoader.java:155)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:47)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.loadJavaFunctions(CatalogServiceCatalog.java:1756)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.invalidateDb(CatalogServiceCatalog.java:1862)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:1994)
> at org.apache.impala.service.JniCatalog.(JniCatalog.java:166)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hive.ql.exec.UDAF
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 9 more
> I0819 09:20:07.780738 1 status.cc:129] NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> CAUSED BY: ClassNotFoundException: org.apache.hadoop.hive.ql.exec.UDAF
> @ 0xd99193
> @ 0x162f6d9
> @ 0xd5f447
> @ 0xd2ba46
> @ 0xc85b28
> @ 0xbd6fd0
> @ 0x7ff4f9bf7554
> @ 0xc7ba86
> E0819 09:20:07.781023 1 catalog.cc:87] NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> CAUSED BY: ClassNotFoundException: org.apache.hadoop.hive.ql.exec.UDAF
> . Impalad exiting.
> {code}
> In Impla 3.4 we handled this exception gracefully but apaprently in 4.1
> Catalog gets terminated by this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11528) hive-exec.pom doesn't include UDAF class
[
https://issues.apache.org/jira/browse/IMPALA-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585592#comment-17585592
]
Steve Carlin commented on IMPALA-11528:
---
I think this is a duplicate of
https://issues.apache.org/jira/browse/IMPALA-11109
If we try this in the most current version of Impala, I think the issue is
fixed.
> hive-exec.pom doesn't include UDAF class
>
>
> Key: IMPALA-11528
> URL: https://issues.apache.org/jira/browse/IMPALA-11528
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.1.0
>Reporter: Gabor Kaszab
>Priority: Major
>
> For hive-exec we load "*UDF*" that excludes the UDAF class:
> https://github.com/apache/impala/blob/master/java/shaded-deps/hive-exec/pom.xml#L102
> As a result when a UDAF is being loaded to catalog we get a
> NoClassDefFoundException.
> {code:java}
> I0819 09:20:07.777845 1 HiveUdfLoader.java:63] Loading UDF
> 'eu.radoop.datahandler.hive.udf.GenericUDAFCorrelationMatrix' from
> file:/tmp/e5a348f5-753a-485a-b37d-2a1420b09df7.jar
> I0819 09:20:07.780457 1 MetastoreEventsProcessor.java:700] Metastore
> event processing restarted. Last synced event id was updated from 902310 to
> 902310
> I0819 09:20:07.780704 1 jni-util.cc:286] java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> at
> org.apache.hadoop.hive.ql.exec.FunctionUtils.getUDFClassType(FunctionUtils.java:157)
> at
> org.apache.impala.hive.executor.HiveUdfLoader.(HiveUdfLoader.java:68)
> at
> org.apache.impala.hive.executor.HiveUdfLoader.createWithLocalPath(HiveUdfLoader.java:155)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:47)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.loadJavaFunctions(CatalogServiceCatalog.java:1756)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.invalidateDb(CatalogServiceCatalog.java:1862)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:1994)
> at org.apache.impala.service.JniCatalog.(JniCatalog.java:166)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hive.ql.exec.UDAF
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 9 more
> I0819 09:20:07.780738 1 status.cc:129] NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> CAUSED BY: ClassNotFoundException: org.apache.hadoop.hive.ql.exec.UDAF
> @ 0xd99193
> @ 0x162f6d9
> @ 0xd5f447
> @ 0xd2ba46
> @ 0xc85b28
> @ 0xbd6fd0
> @ 0x7ff4f9bf7554
> @ 0xc7ba86
> E0819 09:20:07.781023 1 catalog.cc:87] NoClassDefFoundError:
> org/apache/hadoop/hive/ql/exec/UDAF
> CAUSED BY: ClassNotFoundException: org.apache.hadoop.hive.ql.exec.UDAF
> . Impalad exiting.
> {code}
> In Impla 3.4 we handled this exception gracefully but apaprently in 4.1
> Catalog gets terminated by this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11323) Invalid inferred predicates based on casted null values being equivalent
[
https://issues.apache.org/jira/browse/IMPALA-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Carlin resolved IMPALA-11323.
---
Resolution: Fixed
> Invalid inferred predicates based on casted null values being equivalent
>
>
> Key: IMPALA-11323
> URL: https://issues.apache.org/jira/browse/IMPALA-11323
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Joe McDonnell
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: correctness
>
> This query should return two rows, but instead it returns nothing:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v union all select distinct a,b,c from v;
> Fetched 0 row(s) in 0.14s{noformat}
> Without the union all, each side is producing a single row, so it stands to
> reason that the union all would produce two rows:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v;
> +---+--+--+
> | a | b | c |
> +---+--+--+
> | 1 | NULL | NULL |
> +---+--+--+
> Fetched 1 row(s) in 0.14s
> {noformat}
> Both b and c are hard coded to null in the definition of v. The plan shows
> that it is producing a b = c predicate:
> {noformat}
> +---+
> | Explain String |
> +---+
> | Max Per-Host Resource Reservation: Memory=7.88MB Threads=1 |
> | Per-Host Resource Estimates: Memory=24MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | | pass-through-operands: all |
> | | row-size=5B cardinality=2 |
> | | |
> | |--08:SELECT |
> | | | predicates: b = c |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 07:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 06:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT) |
> | | | row-size=3B cardinality=1 |
> | | | |
> | | 05:UNION |
> | | constant-operands=1 |
> | | row-size=1B cardinality=1 |
> | | |
> | 04:SELECT |
> | | predicates: b = c |
> | | row-size=5B cardinality=1 |
> | | |
> | 03:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | row-size=5B cardinality=1 |
> | | |
> | 02:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT) |
> | | row-size=3B cardinality=1 |
> | | |
> | 01:UNION |
> | constant-operands=1 |
> | row-size=1B cardinality=1 |
> +---+
> {noformat}
> I added tracing to PlanNode's addConjunctsToNode() and the
[jira] [Resolved] (IMPALA-11162) Provide support for Hive Generic UDFs
[ https://issues.apache.org/jira/browse/IMPALA-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-11162. --- Resolution: Fixed > Provide support for Hive Generic UDFs > - > > Key: IMPALA-11162 > URL: https://issues.apache.org/jira/browse/IMPALA-11162 > Project: IMPALA > Issue Type: New Feature > Components: Frontend >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > Impala supports the first generation of Hive UDFs. > Impala should also support the second generaton of Hive UDFs. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] (IMPALA-11323) Invalid inferred predicates based on casted null values being equivalent
[ https://issues.apache.org/jira/browse/IMPALA-11323 ]
Steve Carlin deleted comment on IMPALA-11323:
---
was (Author: scarlin):
Not sure if this is the issue. Trying to understand the code...
Two things I noticed on my first try on debugging:
1) The code above got moved to PlanNode.addConjunctsToNode()
2) This part of the code doesn't get called on the above query.
> Invalid inferred predicates based on casted null values being equivalent
>
>
> Key: IMPALA-11323
> URL: https://issues.apache.org/jira/browse/IMPALA-11323
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Joe McDonnell
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: correctness
>
> This query should return two rows, but instead it returns nothing:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v union all select distinct a,b,c from v;
> Fetched 0 row(s) in 0.14s{noformat}
> Without the union all, each side is producing a single row, so it stands to
> reason that the union all would produce two rows:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v;
> +---+--+--+
> | a | b | c |
> +---+--+--+
> | 1 | NULL | NULL |
> +---+--+--+
> Fetched 1 row(s) in 0.14s
> {noformat}
> Both b and c are hard coded to null in the definition of v. The plan shows
> that it is producing a b = c predicate:
> {noformat}
> +---+
> | Explain String |
> +---+
> | Max Per-Host Resource Reservation: Memory=7.88MB Threads=1 |
> | Per-Host Resource Estimates: Memory=24MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | | pass-through-operands: all |
> | | row-size=5B cardinality=2 |
> | | |
> | |--08:SELECT |
> | | | predicates: b = c |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 07:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 06:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT) |
> | | | row-size=3B cardinality=1 |
> | | | |
> | | 05:UNION |
> | | constant-operands=1 |
> | | row-size=1B cardinality=1 |
> | | |
> | 04:SELECT |
> | | predicates: b = c |
> | | row-size=5B cardinality=1 |
> | | |
> | 03:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | row-size=5B cardinality=1 |
> | | |
> | 02:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT) |
> | | row-size=3B cardinality=1 |
> | | |
> | 01:UNION |
> | constant-operands=1 |
> | row-size=1B
[jira] [Comment Edited] (IMPALA-11323) Invalid inferred predicates based on casted null values being equivalent
[
https://issues.apache.org/jira/browse/IMPALA-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544993#comment-17544993
]
Steve Carlin edited comment on IMPALA-11323 at 6/1/22 4:03 PM:
---
Not sure if this is the issue. Trying to understand the code...
Two things I noticed on my first try on debugging:
1) The code above got moved to PlanNode.addConjunctsToNode()
2) This part of the code doesn't get called on the above query.
was (Author: scarlin):
Not sure if this is the issue. Trying to understand the code...
Two things I noticed on my first try on debugging:
1) The code above got moved to PlanNode.addConjunctsToNode()
2) This method doesn't get called on the above query.
> Invalid inferred predicates based on casted null values being equivalent
>
>
> Key: IMPALA-11323
> URL: https://issues.apache.org/jira/browse/IMPALA-11323
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Joe McDonnell
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: correctness
>
> This query should return two rows, but instead it returns nothing:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v union all select distinct a,b,c from v;
> Fetched 0 row(s) in 0.14s{noformat}
> Without the union all, each side is producing a single row, so it stands to
> reason that the union all would produce two rows:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v;
> +---+--+--+
> | a | b | c |
> +---+--+--+
> | 1 | NULL | NULL |
> +---+--+--+
> Fetched 1 row(s) in 0.14s
> {noformat}
> Both b and c are hard coded to null in the definition of v. The plan shows
> that it is producing a b = c predicate:
> {noformat}
> +---+
> | Explain String |
> +---+
> | Max Per-Host Resource Reservation: Memory=7.88MB Threads=1 |
> | Per-Host Resource Estimates: Memory=24MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | | pass-through-operands: all |
> | | row-size=5B cardinality=2 |
> | | |
> | |--08:SELECT |
> | | | predicates: b = c |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 07:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 06:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT) |
> | | | row-size=3B cardinality=1 |
> | | | |
> | | 05:UNION |
> | | constant-operands=1 |
> | | row-size=1B cardinality=1 |
> | | |
> | 04:SELECT |
> | | predicates: b = c |
> | | row-size=5B cardinality=1 |
> | | |
> | 03:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | row-size=5B cardinality=1 |
> | | |
> | 02:AGGREGATE [FINALIZE] |
> | | group by:
[jira] [Commented] (IMPALA-11323) Invalid inferred predicates based on casted null values being equivalent
[
https://issues.apache.org/jira/browse/IMPALA-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544993#comment-17544993
]
Steve Carlin commented on IMPALA-11323:
---
Not sure if this is the issue. Trying to understand the code...
Two things I noticed on my first try on debugging:
1) The code above got moved to PlanNode.addConjunctsToNode()
2) This method doesn't get called on the above query.
> Invalid inferred predicates based on casted null values being equivalent
>
>
> Key: IMPALA-11323
> URL: https://issues.apache.org/jira/browse/IMPALA-11323
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Joe McDonnell
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: correctness
>
> This query should return two rows, but instead it returns nothing:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v union all select distinct a,b,c from v;
> Fetched 0 row(s) in 0.14s{noformat}
> Without the union all, each side is producing a single row, so it stands to
> reason that the union all would produce two rows:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v;
> +---+--+--+
> | a | b | c |
> +---+--+--+
> | 1 | NULL | NULL |
> +---+--+--+
> Fetched 1 row(s) in 0.14s
> {noformat}
> Both b and c are hard coded to null in the definition of v. The plan shows
> that it is producing a b = c predicate:
> {noformat}
> +---+
> | Explain String |
> +---+
> | Max Per-Host Resource Reservation: Memory=7.88MB Threads=1 |
> | Per-Host Resource Estimates: Memory=24MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | | pass-through-operands: all |
> | | row-size=5B cardinality=2 |
> | | |
> | |--08:SELECT |
> | | | predicates: b = c |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 07:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 06:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT) |
> | | | row-size=3B cardinality=1 |
> | | | |
> | | 05:UNION |
> | | constant-operands=1 |
> | | row-size=1B cardinality=1 |
> | | |
> | 04:SELECT |
> | | predicates: b = c |
> | | row-size=5B cardinality=1 |
> | | |
> | 03:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | row-size=5B cardinality=1 |
> | | |
> | 02:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT) |
> | | row-size=3B cardinality=1 |
> | | |
> | 01:UNION |
> | constant-operands=1
[jira] [Assigned] (IMPALA-11323) Invalid inferred predicates based on casted null values being equivalent
[
https://issues.apache.org/jira/browse/IMPALA-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Carlin reassigned IMPALA-11323:
-
Assignee: Steve Carlin
> Invalid inferred predicates based on casted null values being equivalent
>
>
> Key: IMPALA-11323
> URL: https://issues.apache.org/jira/browse/IMPALA-11323
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.2.0
>Reporter: Joe McDonnell
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: correctness
>
> This query should return two rows, but instead it returns nothing:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v union all select distinct a,b,c from v;
> Fetched 0 row(s) in 0.14s{noformat}
> Without the union all, each side is producing a single row, so it stands to
> reason that the union all would produce two rows:
> {noformat}
> [localhost:21050] default> with t as (select 1 a), v as (select distinct a,
> cast(null as smallint)b, cast(null as smallint)c from t) select distinct
> a,b,c from v;
> +---+--+--+
> | a | b | c |
> +---+--+--+
> | 1 | NULL | NULL |
> +---+--+--+
> Fetched 1 row(s) in 0.14s
> {noformat}
> Both b and c are hard coded to null in the definition of v. The plan shows
> that it is producing a b = c predicate:
> {noformat}
> +---+
> | Explain String |
> +---+
> | Max Per-Host Resource Reservation: Memory=7.88MB Threads=1 |
> | Per-Host Resource Estimates: Memory=24MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | | pass-through-operands: all |
> | | row-size=5B cardinality=2 |
> | | |
> | |--08:SELECT |
> | | | predicates: b = c |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 07:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | | row-size=5B cardinality=1 |
> | | | |
> | | 06:AGGREGATE [FINALIZE] |
> | | | group by: a, CAST(NULL AS SMALLINT) |
> | | | row-size=3B cardinality=1 |
> | | | |
> | | 05:UNION |
> | | constant-operands=1 |
> | | row-size=1B cardinality=1 |
> | | |
> | 04:SELECT |
> | | predicates: b = c |
> | | row-size=5B cardinality=1 |
> | | |
> | 03:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
> | | row-size=5B cardinality=1 |
> | | |
> | 02:AGGREGATE [FINALIZE] |
> | | group by: a, CAST(NULL AS SMALLINT) |
> | | row-size=3B cardinality=1 |
> | | |
> | 01:UNION |
> | constant-operands=1 |
> | row-size=1B cardinality=1 |
> +---+
> {noformat}
> I added tracing to PlanNode's addConjunctsToNode()
[jira] [Created] (IMPALA-11213) impala-shell strict hs2 mode large fetches fail
Steve Carlin created IMPALA-11213: - Summary: impala-shell strict hs2 mode large fetches fail Key: IMPALA-11213 URL: https://issues.apache.org/jira/browse/IMPALA-11213 Project: IMPALA Issue Type: Bug Components: Clients Reporter: Steve Carlin The strict hs2 protocol mode is broken when fetching large results. The FetchResults.hasMoreRows field is always returned as false. When there are on more results, Hive returns an empty batch with no rows. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11162) Provide support for Hive Generic UDFs
Steve Carlin created IMPALA-11162: - Summary: Provide support for Hive Generic UDFs Key: IMPALA-11162 URL: https://issues.apache.org/jira/browse/IMPALA-11162 Project: IMPALA Issue Type: New Feature Components: Frontend Reporter: Steve Carlin Impala supports the first generation of Hive UDFs. Impala should also support the second generaton of Hive UDFs. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11101) Change code to help with third party extensions
Steve Carlin created IMPALA-11101: - Summary: Change code to help with third party extensions Key: IMPALA-11101 URL: https://issues.apache.org/jira/browse/IMPALA-11101 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Steve Carlin I need a minor change to help with a third party extension. Specifically, I need to make HdfsTable.setAvroSchema() a protected method. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11097) Execute sometimes fails in call to Hive in test framework
[ https://issues.apache.org/jira/browse/IMPALA-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin updated IMPALA-11097: -- Description: Hive can fail if you call fetch before the execute succeeds. We should call wait_to_finish before doing any fetch results. (was: Hive can fail if you call fetch before the execute succeeds. We can get around this by calling Hive synchronously in the test framework.) > Execute sometimes fails in call to Hive in test framework > - > > Key: IMPALA-11097 > URL: https://issues.apache.org/jira/browse/IMPALA-11097 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Steve Carlin >Priority: Major > > Hive can fail if you call fetch before the execute succeeds. We should call > wait_to_finish before doing any fetch results. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11097) Execute sometimes fails in call to Hive in test framework
Steve Carlin created IMPALA-11097: - Summary: Execute sometimes fails in call to Hive in test framework Key: IMPALA-11097 URL: https://issues.apache.org/jira/browse/IMPALA-11097 Project: IMPALA Issue Type: Bug Components: Infrastructure Reporter: Steve Carlin Hive can fail if you call fetch before the execute succeeds. We can get around this by calling Hive synchronously in the test framework. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11096) impala-shell strict hs2 mode does not support get_summary call
[ https://issues.apache.org/jira/browse/IMPALA-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-11096: - Assignee: Steve Carlin > impala-shell strict hs2 mode does not support get_summary call > -- > > Key: IMPALA-11096 > URL: https://issues.apache.org/jira/browse/IMPALA-11096 > Project: IMPALA > Issue Type: Bug > Components: Clients >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > The impala-shell strict hs2 mode does not support the get_summary call. This > is called when a query is taking a long time in the RUNNING or QUEUED state. > We need to avoid this call in strict hs2 mode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11096) impala-shell strict hs2 mode does not support get_summary call
Steve Carlin created IMPALA-11096: - Summary: impala-shell strict hs2 mode does not support get_summary call Key: IMPALA-11096 URL: https://issues.apache.org/jira/browse/IMPALA-11096 Project: IMPALA Issue Type: Bug Components: Clients Reporter: Steve Carlin The impala-shell strict hs2 mode does not support the get_summary call. This is called when a query is taking a long time in the RUNNING or QUEUED state. We need to avoid this call in strict hs2 mode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11095) impala-shell strict hs2 mode insert is broken
[ https://issues.apache.org/jira/browse/IMPALA-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-11095: - Assignee: Steve Carlin > impala-shell strict hs2 mode insert is broken > - > > Key: IMPALA-11095 > URL: https://issues.apache.org/jira/browse/IMPALA-11095 > Project: IMPALA > Issue Type: Bug > Components: Clients >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > The insert command using impala-shell in strict_hs2_mode is not working > The close_dml statement should return two values and it is only returning one. > However, the rows returned is not supported by the strict hs2 protocol so > some infrastructure has to be changed as well. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11095) impala-shell strict hs2 mode insert is broken
Steve Carlin created IMPALA-11095: - Summary: impala-shell strict hs2 mode insert is broken Key: IMPALA-11095 URL: https://issues.apache.org/jira/browse/IMPALA-11095 Project: IMPALA Issue Type: Bug Components: Clients Reporter: Steve Carlin The insert command using impala-shell in strict_hs2_mode is not working The close_dml statement should return two values and it is only returning one. However, the rows returned is not supported by the strict hs2 protocol so some infrastructure has to be changed as well. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11062) The now_string option doesn't work in plannedstatement mode
[ https://issues.apache.org/jira/browse/IMPALA-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-11062: - Assignee: Steve Carlin > The now_string option doesn't work in plannedstatement mode > --- > > Key: IMPALA-11062 > URL: https://issues.apache.org/jira/browse/IMPALA-11062 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > The "set now_string=" command isn't working properly when the > query is being run through the ExecutePlannedStatement thrift API -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11056) Allow Java UDFs to fail query when exception is thrown
[ https://issues.apache.org/jira/browse/IMPALA-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-11056: - Assignee: Steve Carlin > Allow Java UDFs to fail query when exception is thrown > -- > > Key: IMPALA-11056 > URL: https://issues.apache.org/jira/browse/IMPALA-11056 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > Currently, the functionality for Hive UDFs is such that if the java code > throws an exception, the backend logs it as a warning and treats the function > as if it returned NULL. > Sometimes it is desirable to allow the Java function to diagnose an error as > fatal to the query. > Since this feature would break backward compatibility, the default behavior > should be the current behavior. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11075) Use newer API for get_partitions_ps_with_auth() call
[ https://issues.apache.org/jira/browse/IMPALA-11075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned IMPALA-11075: - Assignee: Steve Carlin > Use newer API for get_partitions_ps_with_auth() call > > > Key: IMPALA-11075 > URL: https://issues.apache.org/jira/browse/IMPALA-11075 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > > Change the code to keep the API in sync with the newer API for > get_partitions_ps_with_auth() -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11075) Use newer API for get_partitions_ps_with_auth() call
Steve Carlin created IMPALA-11075: - Summary: Use newer API for get_partitions_ps_with_auth() call Key: IMPALA-11075 URL: https://issues.apache.org/jira/browse/IMPALA-11075 Project: IMPALA Issue Type: Improvement Components: Catalog Reporter: Steve Carlin Change the code to keep the API in sync with the newer API for get_partitions_ps_with_auth() -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-11070) Add test_section.py to framework
[ https://issues.apache.org/jira/browse/IMPALA-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin closed IMPALA-11070. - Resolution: Won't Fix Determined it was not needed. > Add test_section.py to framework > > > Key: IMPALA-11070 > URL: https://issues.apache.org/jira/browse/IMPALA-11070 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Steve Carlin >Priority: Minor > > In the test framework, the parsed sections from the ".test" file could be > made more object oriented in case there are extensions. > This Jira serves as a first step to restructure the code with minimal logic > change. After this Jira, another Jira will be filed to move more logic into > the class. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11070) Add test_section.py to framework
Steve Carlin created IMPALA-11070: - Summary: Add test_section.py to framework Key: IMPALA-11070 URL: https://issues.apache.org/jira/browse/IMPALA-11070 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Steve Carlin In the test framework, the parsed sections from the ".test" file could be made more object oriented in case there are extensions. This Jira serves as a first step to restructure the code with minimal logic change. After this Jira, another Jira will be filed to move more logic into the class. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11062) The now_string option doesn't work in plannedstatement mode
Steve Carlin created IMPALA-11062: - Summary: The now_string option doesn't work in plannedstatement mode Key: IMPALA-11062 URL: https://issues.apache.org/jira/browse/IMPALA-11062 Project: IMPALA Issue Type: Bug Components: Backend Reporter: Steve Carlin The "set now_string=" command isn't working properly when the query is being run through the ExecutePlannedStatement thrift API -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11056) Allow Java UDFs to fail query when exception is thrown
Steve Carlin created IMPALA-11056: - Summary: Allow Java UDFs to fail query when exception is thrown Key: IMPALA-11056 URL: https://issues.apache.org/jira/browse/IMPALA-11056 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Steve Carlin Currently, the functionality for Hive UDFs is such that if the java code throws an exception, the backend logs it as a warning and treats the function as if it returned NULL. Sometimes it is desirable to allow the Java function to diagnose an error as fatal to the query. Since this feature would break backward compatibility, the default behavior should be the current behavior. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-11046) When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
[
https://issues.apache.org/jira/browse/IMPALA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455276#comment-17455276
]
Steve Carlin edited comment on IMPALA-11046 at 12/8/21, 4:31 PM:
-
Ah, I see.
I won't push hard on this, but one general comment as to why I'd still like to
make the change:
When we run it on jenkins, A DCHECK will bring down the server. Once the server
is brought down, it will fail many other tests behind it, thus potentially
masking some errors which would only be exposed on the next Jenkins run.
It's also a slight pain on the debugging side for me on my local machine since
I'd have to restart the server (but yeah, this is minor as well).
Also, this is more of a consistency issue, since the same error happens in two
places, but one is a DCHECK and the other prints out an error message. In this
one case, it allows the caller to add more logging information, which I think
is better than just throwing the DCHECK error.
But I suppose these are not serious issues.
was (Author: scarlin):
Ah, I see.
I won't push hard on this, but one general comment as to why I'd still like to
make the change:
When we run it on jenkins, A DCHECK will bring down the server. Once the server
is brought down, it will fail many other tests behind it, thus potentially
masking some errors which would only be exposed on the next Jenkins run.
It's also a slight pain on the debugging side for me on my local machine since
I'd have to restart the server (but yeah, this is minor as well).
Also, this is more of a consistency issue, since the same error happens in two
places, but one is a DCHECK and the other prints out an error message.
But I suppose these are not serious issues.
> When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
>
>
> Key: IMPALA-11046
> URL: https://issues.apache.org/jira/browse/IMPALA-11046
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Steve Carlin
>Assignee: Steve Carlin
>Priority: Major
>
> The following code exists in runtime/descriptors.cc:
> int RowDescriptor::GetTupleIdx(TupleId id) const {
> DCHECK_LT(id, tuple_idx_map_.size()) << "RowDescriptor: " << DebugString();
> return tuple_idx_map_[id];
> }
>
> If the id doesn't exist in the map, it returns INVALID_IDX. However, if the
> id >= tuple_idx_size, it crashes the server.
> I was working on an issue on the frontend where I passed an incorrect index
> and it failed the query when I passed a bad index in both instances, but it
> was much preferable not to crash the server and only fail the query. So the
> proposal here is to get rid of the DCHECK_LT and replace it with a "return
> INVALID_IDX" when it fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-11046) When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
[
https://issues.apache.org/jira/browse/IMPALA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455276#comment-17455276
]
Steve Carlin edited comment on IMPALA-11046 at 12/8/21, 2:34 PM:
-
Ah, I see.
I won't push hard on this, but one general comment as to why I'd still like to
make the change:
When we run it on jenkins, A DCHECK will bring down the server. Once the server
is brought down, it will fail many other tests behind it, thus potentially
masking some errors which would only be exposed on the next Jenkins run.
It's also a slight pain on the debugging side for me on my local machine since
I'd have to restart the server (but yeah, this is minor as well).
Also, this is more of a consistency issue, since the same error happens in two
places, but one is a DCHECK and the other prints out an error message.
But I suppose these are not serious issues.
was (Author: scarlin):
Ah, I see.
I won't push hard on this, but one general comment as to why I'd still like to
make the change:
When we run it on jenkins, A DCHECK will bring down the server. Once the server
is brought down, it will fail many other tests behind it, thus potentially
masking some errors which would only be exposed on the next Jenkins run.
It's also a slight pain on the debugging side for me on my local machine since
I'd have to restart the server (but yeah, this is minor as well).
But I suppose these are not serious issues.
> When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
>
>
> Key: IMPALA-11046
> URL: https://issues.apache.org/jira/browse/IMPALA-11046
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Steve Carlin
>Assignee: Steve Carlin
>Priority: Major
>
> The following code exists in runtime/descriptors.cc:
> int RowDescriptor::GetTupleIdx(TupleId id) const {
> DCHECK_LT(id, tuple_idx_map_.size()) << "RowDescriptor: " << DebugString();
> return tuple_idx_map_[id];
> }
>
> If the id doesn't exist in the map, it returns INVALID_IDX. However, if the
> id >= tuple_idx_size, it crashes the server.
> I was working on an issue on the frontend where I passed an incorrect index
> and it failed the query when I passed a bad index in both instances, but it
> was much preferable not to crash the server and only fail the query. So the
> proposal here is to get rid of the DCHECK_LT and replace it with a "return
> INVALID_IDX" when it fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11046) When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
[
https://issues.apache.org/jira/browse/IMPALA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455276#comment-17455276
]
Steve Carlin commented on IMPALA-11046:
---
Ah, I see.
I won't push hard on this, but one general comment as to why I'd still like to
make the change:
When we run it on jenkins, A DCHECK will bring down the server. Once the server
is brought down, it will fail many other tests behind it, thus potentially
masking some errors which would only be exposed on the next Jenkins run.
It's also a slight pain on the debugging side for me on my local machine since
I'd have to restart the server (but yeah, this is minor as well).
But I suppose these are not serious issues.
> When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
>
>
> Key: IMPALA-11046
> URL: https://issues.apache.org/jira/browse/IMPALA-11046
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Steve Carlin
>Assignee: Steve Carlin
>Priority: Major
>
> The following code exists in runtime/descriptors.cc:
> int RowDescriptor::GetTupleIdx(TupleId id) const {
> DCHECK_LT(id, tuple_idx_map_.size()) << "RowDescriptor: " << DebugString();
> return tuple_idx_map_[id];
> }
>
> If the id doesn't exist in the map, it returns INVALID_IDX. However, if the
> id >= tuple_idx_size, it crashes the server.
> I was working on an issue on the frontend where I passed an incorrect index
> and it failed the query when I passed a bad index in both instances, but it
> was much preferable not to crash the server and only fail the query. So the
> proposal here is to get rid of the DCHECK_LT and replace it with a "return
> INVALID_IDX" when it fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11046) When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
[
https://issues.apache.org/jira/browse/IMPALA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Carlin reassigned IMPALA-11046:
-
Assignee: Steve Carlin
> When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
>
>
> Key: IMPALA-11046
> URL: https://issues.apache.org/jira/browse/IMPALA-11046
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Steve Carlin
>Assignee: Steve Carlin
>Priority: Major
>
> The following code exists in runtime/descriptors.cc:
> int RowDescriptor::GetTupleIdx(TupleId id) const {
> DCHECK_LT(id, tuple_idx_map_.size()) << "RowDescriptor: " << DebugString();
> return tuple_idx_map_[id];
> }
>
> If the id doesn't exist in the map, it returns INVALID_IDX. However, if the
> id >= tuple_idx_size, it crashes the server.
> I was working on an issue on the frontend where I passed an incorrect index
> and it failed the query when I passed a bad index in both instances, but it
> was much preferable not to crash the server and only fail the query. So the
> proposal here is to get rid of the DCHECK_LT and replace it with a "return
> INVALID_IDX" when it fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11046) When GetTupleIdx fails, it should return INVALID_IDX, not bring down impalad
Steve Carlin created IMPALA-11046:
-
Summary: When GetTupleIdx fails, it should return INVALID_IDX, not
bring down impalad
Key: IMPALA-11046
URL: https://issues.apache.org/jira/browse/IMPALA-11046
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Steve Carlin
The following code exists in runtime/descriptors.cc:
int RowDescriptor::GetTupleIdx(TupleId id) const {
DCHECK_LT(id, tuple_idx_map_.size()) << "RowDescriptor: " << DebugString();
return tuple_idx_map_[id];
}
If the id doesn't exist in the map, it returns INVALID_IDX. However, if the id
>= tuple_idx_size, it crashes the server.
I was working on an issue on the frontend where I passed an incorrect index and
it failed the query when I passed a bad index in both instances, but it was
much preferable not to crash the server and only fail the query. So the
proposal here is to get rid of the DCHECK_LT and replace it with a "return
INVALID_IDX" when it fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10997) Refactor code for UDF implementation
Steve Carlin created IMPALA-10997: - Summary: Refactor code for UDF implementation Key: IMPALA-10997 URL: https://issues.apache.org/jira/browse/IMPALA-10997 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin In its current form, Impala supports Java UDFs that are derived from the UDF.class. The UDF.class is legacy code and Hive now supports implementation off of the GenericUDF.class. This Jira is to track the refactoring the current UDF code to allow easier implementation when we eventually support the newer class. Common code will be put in a base HiveExecutor class and UDF.class specific code will be in a derived class. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10896) Tests in TestImpalaShellInteractive failed in S3 build when strict_hs2_protocol=True
[
https://issues.apache.org/jira/browse/IMPALA-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411387#comment-17411387
]
Steve Carlin commented on IMPALA-10896:
---
I have something that might fix this?
I do have something posted on gerrit: [https://gerrit.cloudera.org/#/c/17820/]
But I have yet to test this (need to figure out how to test this on the S3 run).
> Tests in TestImpalaShellInteractive failed in S3 build when
> strict_hs2_protocol=True
>
>
> Key: IMPALA-10896
> URL: https://issues.apache.org/jira/browse/IMPALA-10896
> Project: IMPALA
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Steve Carlin
>Priority: Blocker
> Labels: broken-build
>
> See test failures and timeouts in a S3 build:
> {code:java}
> 23:38:38
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_write_delimited[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: False | protocol: beeswax] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> PASSED
> 23:38:38
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_write_delimited[table_format_and_file_extension:
> ('textfile', '.txt') | strict_hs2_protocol: True | protocol: hs2] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 23:38:43
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_write_delimited[table_format_and_file_extension:
> ('textfile', '.txt') | strict_hs2_protocol: False | protocol: hs2-http] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> PASSED
> 01:38:43
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_write_delimited[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: True | protocol: hs2-http] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 01:38:43
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_write_delimited[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: True | protocol: hs2] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 03:38:43
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_change_delimiter[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: False | protocol: beeswax] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> PASSED
> 05:38:43
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_change_delimiter[table_format_and_file_extension:
> ('textfile', '.txt') | strict_hs2_protocol: True | protocol: hs2] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 05:38:44
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_change_delimiter[table_format_and_file_extension:
> ('textfile', '.txt') | strict_hs2_protocol: False | protocol: hs2-http] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> PASSED
> 07:38:44
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_change_delimiter[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: True | protocol: hs2-http] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 07:38:44
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_change_delimiter[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: True | protocol: hs2] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> FAILED
> 09:38:44
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_print_to_file[table_format_and_file_extension:
> ('parquet', '.parq') | strict_hs2_protocol: False | protocol: beeswax] <-
> ../../../../../../../data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/tests/shell/test_shell_interactive.py
> PASSED
> 11:38:44
> shell/test_shell_interactive.py::TestImpalaShellInteractive::test_print_to_file[table_format_and_file_extension:
> ('textfile', '.txt') | strict_hs2_protocol: True | protocol: hs2] <-
>
[jira] [Created] (IMPALA-10858) On full acid table, HMS call not returning file descriptors
Steve Carlin created IMPALA-10858: - Summary: On full acid table, HMS call not returning file descriptors Key: IMPALA-10858 URL: https://issues.apache.org/jira/browse/IMPALA-10858 Project: IMPALA Issue Type: Bug Components: Catalog Reporter: Steve Carlin Bug happens on a full acid table which has both insert and delete file descriptors. When a metadata request hits catalogd, the file descriptors are not being sent back. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10845) Increase visibility on methods and variables for extensibility
Steve Carlin created IMPALA-10845: - Summary: Increase visibility on methods and variables for extensibility Key: IMPALA-10845 URL: https://issues.apache.org/jira/browse/IMPALA-10845 Project: IMPALA Issue Type: Task Components: Frontend Reporter: Steve Carlin Increased the visibility on a couple methods for extensibility. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10832) Analyzing UDFs should not require the Profile thread to exist
Steve Carlin created IMPALA-10832: - Summary: Analyzing UDFs should not require the Profile thread to exist Key: IMPALA-10832 URL: https://issues.apache.org/jira/browse/IMPALA-10832 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Steve Carlin There is code in FunctionCallExpr which requires the Profile thread to exist if a UDF is being analyzed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10827) Impala-shell fixes for strict hs2 mode
Steve Carlin created IMPALA-10827: - Summary: Impala-shell fixes for strict hs2 mode Key: IMPALA-10827 URL: https://issues.apache.org/jira/browse/IMPALA-10827 Project: IMPALA Issue Type: Bug Components: Clients Reporter: Steve Carlin The strict hs2 protocol was added with IMPALA-10778. Some tests did not work as expected. These issues are most likely due to some kind of incompatibility when connecting with HiveServer2 directly, as not all features in Impala work in Hive. The following tests need to be investigated: In test_shell_interactive test_cancellation_mid_command test_unicode_input test_reconnect test_line_ends_with_comment In test_shell_commandline test_completed_query_errors_1 test_completed_query_errors_2 test_cancellation test_query_time_and_link_message test_get_log_once test_large_sql test_type_formatting test_null_values test_fetch_size -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10778) Allow impala-shell to connect directly to HS2
Steve Carlin created IMPALA-10778: - Summary: Allow impala-shell to connect directly to HS2 Key: IMPALA-10778 URL: https://issues.apache.org/jira/browse/IMPALA-10778 Project: IMPALA Issue Type: Improvement Components: Clients Reporter: Steve Carlin Impala-shell already uses HS2 protocol to connect to Impalad. We can also allow impala-shell to connect to any server using the hs2 protocol. We will need to disable features and tests specifically related to Impala -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10760) Add accessor methods to Function classes
[ https://issues.apache.org/jira/browse/IMPALA-10760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin updated IMPALA-10760: -- Description: A few of the functions in AggregateFunction and ScalarFunction don't have accessor methods, namely: In AggregateFunction removeFnSymbol getValueFnSymbol In ScalarFunction prepareFnSymbol closeFnSymbol These are necessary for an external frontend to access these symbols. was: A few of the functions in AggregateFunction and ScalarFunction don't have accessor methods, namely: In AggregateFunction removeFnSymbol getValueFnSymbol In ScalarFunction prepareFnSymbol closeFnSymbol > Add accessor methods to Function classes > > > Key: IMPALA-10760 > URL: https://issues.apache.org/jira/browse/IMPALA-10760 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Steve Carlin >Priority: Major > > A few of the functions in AggregateFunction and ScalarFunction don't have > accessor methods, namely: > In AggregateFunction > removeFnSymbol > getValueFnSymbol > In ScalarFunction > prepareFnSymbol > closeFnSymbol > > These are necessary for an external frontend to access these symbols. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10760) Add accessor methods to Function classes
Steve Carlin created IMPALA-10760: - Summary: Add accessor methods to Function classes Key: IMPALA-10760 URL: https://issues.apache.org/jira/browse/IMPALA-10760 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin A few of the functions in AggregateFunction and ScalarFunction don't have accessor methods, namely: In AggregateFunction removeFnSymbol getValueFnSymbol In ScalarFunction prepareFnSymbol closeFnSymbol -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10750) Changes needed for impala-shell to make it more HS2 compatible
Steve Carlin created IMPALA-10750: - Summary: Changes needed for impala-shell to make it more HS2 compatible Key: IMPALA-10750 URL: https://issues.apache.org/jira/browse/IMPALA-10750 Project: IMPALA Issue Type: Improvement Components: Clients Reporter: Steve Carlin Need some changes to impala-shell to make the client more HS2 compatible, including: * when the fetch returns the bitset containing nulls, the lack of presence of bits means it is not null. Currently it will fail the query. * adding fetchType to TCLIServiceThrift structure (though unused currently in Impala) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10524) Change HdfsPartition to allow third party extensions
[ https://issues.apache.org/jira/browse/IMPALA-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-10524. --- Resolution: Fixed > Change HdfsPartition to allow third party extensions > > > Key: IMPALA-10524 > URL: https://issues.apache.org/jira/browse/IMPALA-10524 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Steve Carlin >Priority: Minor > > Changes needed to HdfsPartition and other related classes to allow third > party extensions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10525) Add param to BuiltinsDb to defer initialization
[ https://issues.apache.org/jira/browse/IMPALA-10525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-10525. --- Resolution: Fixed > Add param to BuiltinsDb to defer initialization > --- > > Key: IMPALA-10525 > URL: https://issues.apache.org/jira/browse/IMPALA-10525 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Steve Carlin >Priority: Major > > BuiltinsDb.java currently initializes all the builtin functions on > initialization. Part of the initialization task is to interact with the C++ > code to fetch the signatures of the functions. This doesn't work if a third > party wants to use the BuiltinDb but does not have access to the C++ library > at runtime. > The solution is to add a parameter to the initialization which will allow the > initialization to be deferred. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10510) Change code to help with third party extensions
[ https://issues.apache.org/jira/browse/IMPALA-10510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-10510. --- Resolution: Fixed > Change code to help with third party extensions > --- > > Key: IMPALA-10510 > URL: https://issues.apache.org/jira/browse/IMPALA-10510 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Steve Carlin >Priority: Minor > > I'm adding some third party extensions and I need some of the code to be more > visible. > Specifically, I need the ArithmeticExpr.Operator to be public and the > HdfsTable.loadConstraintsInfo method to be protected (to be called from a > derived class) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10593) Skip runtime filter for outer joins when Expr not constant after null substitution
[ https://issues.apache.org/jira/browse/IMPALA-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin resolved IMPALA-10593. --- Resolution: Fixed > Skip runtime filter for outer joins when Expr not constant after null > substitution > -- > > Key: IMPALA-10593 > URL: https://issues.apache.org/jira/browse/IMPALA-10593 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Reporter: Steve Carlin >Priority: Minor > > Currently there is code that asserts that an Expr is not constant after > substituting SlotRefs with constant nulls. > A third party tool needs this restriction to be weakened. In a case where an > Expr is checked and the Expr is not constant even after substituting nulls, > the result will be to not generate a runtime filter for that Expr. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10593) Skip runtime filter for outer joins when Expr not constant after null substitution
Steve Carlin created IMPALA-10593: - Summary: Skip runtime filter for outer joins when Expr not constant after null substitution Key: IMPALA-10593 URL: https://issues.apache.org/jira/browse/IMPALA-10593 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Steve Carlin Currently there is code that asserts that an Expr is not constant after substituting SlotRefs with constant nulls. A third party tool needs this restriction to be weakened. In a case where an Expr is checked and the Expr is not constant even after substituting nulls, the result will be to not generate a runtime filter for that Expr. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10556) Casting to a date without date portion should return error?
Steve Carlin created IMPALA-10556:
-
Summary: Casting to a date without date portion should return
error?
Key: IMPALA-10556
URL: https://issues.apache.org/jira/browse/IMPALA-10556
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Steve Carlin
The query select cast(cast('23:59:59' as timestamp) as date) returns NULL.
Is this the desired behavior or should we return an error? Postgres returns an
error in this case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10525) Add param to BuiltinsDb to defer initialization
Steve Carlin created IMPALA-10525: - Summary: Add param to BuiltinsDb to defer initialization Key: IMPALA-10525 URL: https://issues.apache.org/jira/browse/IMPALA-10525 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Steve Carlin BuiltinsDb.java currently initializes all the builtin functions on initialization. Part of the initialization task is to interact with the C++ code to fetch the signatures of the functions. This doesn't work if a third party wants to use the BuiltinDb but does not have access to the C++ library at runtime. The solution is to add a parameter to the initialization which will allow the initialization to be deferred. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10524) Change HdfsPartition to allow third party extensions
Steve Carlin created IMPALA-10524: - Summary: Change HdfsPartition to allow third party extensions Key: IMPALA-10524 URL: https://issues.apache.org/jira/browse/IMPALA-10524 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Steve Carlin Changes needed to HdfsPartition and other related classes to allow third party extensions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10510) Change code to help with third party extensions
[ https://issues.apache.org/jira/browse/IMPALA-10510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin updated IMPALA-10510: -- Parent: IMPALA-10514 Issue Type: Sub-task (was: Improvement) > Change code to help with third party extensions > --- > > Key: IMPALA-10510 > URL: https://issues.apache.org/jira/browse/IMPALA-10510 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Steve Carlin >Priority: Minor > > I'm adding some third party extensions and I need some of the code to be more > visible. > Specifically, I need the ArithmeticExpr.Operator to be public and the > HdfsTable.loadConstraintsInfo method to be protected (to be called from a > derived class) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10510) Change code to help with third party extensions
Steve Carlin created IMPALA-10510: - Summary: Change code to help with third party extensions Key: IMPALA-10510 URL: https://issues.apache.org/jira/browse/IMPALA-10510 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Steve Carlin I'm adding some third party extensions and I need some of the code to be more visible. Specifically, I need the ArithmeticExpr.Operator to be public and the HdfsTable.loadConstraintsInfo method to be protected (to be called from a derived class) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
