[jira] [Commented] (DRILL-6253) HashAgg Unit Testing And Refactoring
[ https://issues.apache.org/jira/browse/DRILL-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423429#comment-16423429 ] Boaz Ben-Zvi commented on DRILL-6253: - Comments added to the design doc ... > HashAgg Unit Testing And Refactoring > > > Key: DRILL-6253 > URL: https://issues.apache.org/jira/browse/DRILL-6253 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Fix For: 1.14.0 > > > This is a parent issue to hold all the subtasks required to refactor HashAgg > to make it unit testable. Design doc > https://docs.google.com/document/d/110BAWg3QXMfdmuqB0p3HuaoKpPGY-lqCRtHFxdh53Ds/edit?usp=sharing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-5977) predicate pushdown support kafkaMsgOffset
[
https://issues.apache.org/jira/browse/DRILL-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker reassigned DRILL-5977:
Assignee: Abhishek Ravi
> predicate pushdown support kafkaMsgOffset
> -
>
> Key: DRILL-5977
> URL: https://issues.apache.org/jira/browse/DRILL-5977
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: B Anil Kumar
>Assignee: Abhishek Ravi
>Priority: Major
> Fix For: 1.14.0
>
>
> As part of Kafka storage plugin review, below is the suggestion from Paul.
> {noformat}
> Does it make sense to provide a way to select a range of messages: a starting
> point or a count? Perhaps I want to run my query every five minutes, scanning
> only those messages since the previous scan. Or, I want to limit my take to,
> say, the next 1000 messages. Could we use a pseudo-column such as
> "kafkaMsgOffset" for that purpose? Maybe
> SELECT * FROM WHERE kafkaMsgOffset > 12345
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6202) Deprecate usage of IndexOutOfBoundsException to re-alloc vectors
[ https://issues.apache.org/jira/browse/DRILL-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423360#comment-16423360 ] ASF GitHub Bot commented on DRILL-6202: --- Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1144 It is not clear why get/set Byte/Char/Short/Int/Long/Float/Double do not delegate to UDLE, while get/set Bytes delegates to UDLE and relies on netty 'AbstractByteBuf` for bounds checking. IMO, it will be good to have the behavior consistent for all methods. In many cases including `VariableLengthVectors`, there is no need to rely on UDLE boundary checking as a caller already provides or can provide a guarantee that an index is within a buffer boundaries. In those cases, boundary check becomes an extra cost. IMO, it will be good to have a consistent behavior with ability to enable bounds checking for debugging. > Deprecate usage of IndexOutOfBoundsException to re-alloc vectors > > > Key: DRILL-6202 > URL: https://issues.apache.org/jira/browse/DRILL-6202 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Major > Fix For: 1.14.0 > > > As bounds checking may be enabled or disabled, using > IndexOutOfBoundsException to resize vectors is unreliable. It works only when > bounds checking is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated DRILL-6287: -- Labels: ready-to-commit (was: ) > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > Labels: ready-to-commit > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423315#comment-16423315 ] Vlad Rozov commented on DRILL-6272: --- [~volodymyr.tkach] There is a difference between a json and a jar file. A json file represents original source submitted to ASF, while a jar file is a compilation from a source code. > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4091) Support more functions in gis contrib module
[ https://issues.apache.org/jira/browse/DRILL-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423150#comment-16423150 ] ASF GitHub Bot commented on DRILL-4091: --- Github user ChrisSandison commented on a diff in the pull request: https://github.com/apache/drill/pull/258#discussion_r178652351 --- Diff: contrib/gis/src/main/java/org/apache/drill/exec/expr/fn/impl/gis/STUnionAggregate.java --- @@ -0,0 +1,114 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.drill.exec.expr.fn.impl.gis; + +import javax.inject.Inject; + +import org.apache.drill.exec.expr.DrillAggFunc; +import org.apache.drill.exec.expr.annotations.FunctionTemplate; +import org.apache.drill.exec.expr.annotations.Output; +import org.apache.drill.exec.expr.annotations.Param; +import org.apache.drill.exec.expr.annotations.Workspace; +import org.apache.drill.exec.expr.holders.BigIntHolder; +import org.apache.drill.exec.expr.holders.IntHolder; +import org.apache.drill.exec.expr.holders.NullableVarBinaryHolder; +import org.apache.drill.exec.expr.holders.ObjectHolder; +import org.apache.drill.exec.expr.holders.UInt1Holder; + +import com.esri.core.geometry.SpatialReference; + +import io.netty.buffer.DrillBuf; + +@FunctionTemplate(name = "st_unionaggregate", scope = FunctionTemplate.FunctionScope.POINT_AGGREGATE) --- End diff -- Is there documentation for that for aggregate functions? > Support more functions in gis contrib module > > > Key: DRILL-4091 > URL: https://issues.apache.org/jira/browse/DRILL-4091 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Reporter: Karol Potocki >Assignee: Karol Potocki >Priority: Major > > Support for commonly used gis functions in gis contrib module: relate, > contains, crosses, intersects, touches, difference, disjoint, buffer, union > etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DRILL-6009) No drillbits on index page
[ https://issues.apache.org/jira/browse/DRILL-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati closed DRILL-6009. Resolution: Cannot Reproduce > No drillbits on index page > -- > > Key: DRILL-6009 > URL: https://issues.apache.org/jira/browse/DRILL-6009 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > Attachments: empty_drillbits.JPG > > > After DRILL-4286 once I saw that index page showed no drillbits at all but it > was working, so at least one drillbit was online (empty_drillbits.JPG). After > refresh everything was fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4091) Support more functions in gis contrib module
[
https://issues.apache.org/jira/browse/DRILL-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422981#comment-16422981
]
ASF GitHub Bot commented on DRILL-4091:
---
Github user ChrisSandison commented on a diff in the pull request:
https://github.com/apache/drill/pull/258#discussion_r178619522
--- Diff:
contrib/gis/src/main/java/org/apache/drill/exec/expr/fn/impl/gis/STXFunc.java
---
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.expr.fn.impl.gis;
+
+import java.sql.Types;
+
+import javax.inject.Inject;
+
+import org.apache.drill.exec.expr.DrillSimpleFunc;
+import org.apache.drill.exec.expr.annotations.FunctionTemplate;
+import org.apache.drill.exec.expr.annotations.Output;
+import org.apache.drill.exec.expr.annotations.Param;
+import org.apache.drill.exec.expr.holders.Float8Holder;
+import org.apache.drill.exec.expr.holders.VarBinaryHolder;
+
+import com.esri.core.geometry.Geometry.Type;
+import com.esri.core.geometry.ogc.OGCPoint;
+
+import io.netty.buffer.DrillBuf;
+
+@FunctionTemplate(name = "st_x", scope =
FunctionTemplate.FunctionScope.SIMPLE,
+ nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
+public class STXFunc implements DrillSimpleFunc {
+ @Param
+ VarBinaryHolder geomParam;
+
+ @Output
+ Float8Holder out;
+
+ @Inject
+ DrillBuf buffer;
+
+ public void setup() {
+ }
+
+ public void eval() {
+
+com.esri.core.geometry.ogc.OGCGeometry geom;
+
+geom = com.esri.core.geometry.ogc.OGCGeometry
+.fromBinary(geomParam.buffer.nioBuffer(geomParam.start,
geomParam.end - geomParam.start));
+
+if(geom != null && geom.geometryType().equals("Point")){
+ out.value = ((com.esri.core.geometry.ogc.OGCPoint) geom).X();
+} else {
+ out.value = Double.NaN;
--- End diff --
@cgivre it looks like the assigning of `NaN` is breaking the test suite. Is
this the behaviour, or could this be the default null handling that is
provided? Ditto for other times this is assigned
> Support more functions in gis contrib module
>
>
> Key: DRILL-4091
> URL: https://issues.apache.org/jira/browse/DRILL-4091
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Reporter: Karol Potocki
>Assignee: Karol Potocki
>Priority: Major
>
> Support for commonly used gis functions in gis contrib module: relate,
> contains, crosses, intersects, touches, difference, disjoint, buffer, union
> etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422931#comment-16422931 ] Volodymyr Tkach edited comment on DRILL-6272 at 4/2/18 6:30 PM: [~vrozov] those binaries serve us as data input for testing purposes, they are not intended to be executed/compiled by users. I think that this paragraph [http://www.apache.org/legal/release-policy.html#compiled-packages] is saying about binaries that users will execute manually or programmatically, but that is not our case, for us those files are raw data, like json files for testing purposes. was (Author: volodymyr.tkach): [~vrozov] those binaries serve us as data input for testing purposes, they are not intended to be executed/compiled by users. I think that this paragraph [http://www.apache.org/legal/release-policy.html#compiled-packages] is saying about binaries that users will execute manually or programmatically, but that is not our case, for us those files just raw data, like json files for testing purposes. > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422931#comment-16422931 ] Volodymyr Tkach commented on DRILL-6272: [~vrozov] those binaries serve us as data input for testing purposes, they are not intended to be executed/compiled by users. I think that this paragraph [http://www.apache.org/legal/release-policy.html#compiled-packages] is saying about binaries that users will execute manually or programmatically, but that is not our case, for us those files just raw data, like json files for testing purposes. > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6306) Should not be able to run queries against disabled storage plugins
Krystal created DRILL-6306: -- Summary: Should not be able to run queries against disabled storage plugins Key: DRILL-6306 URL: https://issues.apache.org/jira/browse/DRILL-6306 Project: Apache Drill Issue Type: Bug Components: Storage - Other Affects Versions: 1.13.0 Reporter: Krystal Currently, queries against disabled storage plugins are returning data. This should not be the case. Queries against disabled storage plugins should fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-5977) predicate pushdown support kafkaMsgOffset
[
https://issues.apache.org/jira/browse/DRILL-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua reassigned DRILL-5977:
---
Assignee: (was: Bhallamudi Venkata Siva Kamesh)
> predicate pushdown support kafkaMsgOffset
> -
>
> Key: DRILL-5977
> URL: https://issues.apache.org/jira/browse/DRILL-5977
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: B Anil Kumar
>Priority: Major
> Fix For: 1.14.0
>
>
> As part of Kafka storage plugin review, below is the suggestion from Paul.
> {noformat}
> Does it make sense to provide a way to select a range of messages: a starting
> point or a count? Perhaps I want to run my query every five minutes, scanning
> only those messages since the previous scan. Or, I want to limit my take to,
> say, the next 1000 messages. Could we use a pseudo-column such as
> "kafkaMsgOffset" for that purpose? Maybe
> SELECT * FROM WHERE kafkaMsgOffset > 12345
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-4547) Javadoc fails with Java8
[
https://issues.apache.org/jira/browse/DRILL-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422648#comment-16422648
]
Vitalii Diravka commented on DRILL-4547:
So what is the next step for this Jira?
I suppose it should be the refactoring of Drill java docs and moving onto Java8
doclint. But the task is really minor.
> Javadoc fails with Java8
>
>
> Key: DRILL-4547
> URL: https://issues.apache.org/jira/browse/DRILL-4547
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build Test
>Affects Versions: 1.6.0
>Reporter: Laurent Goujon
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> Javadoc cannot be generated when using Java8 (likely because the parser is
> now more strict).
> Here's an example of issues when trying to generate javadocs in module
> {{drill-fmpp-maven-plugin}}
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:jar (attach-javadocs) on
> project drill-fmpp-maven-plugin: MavenReportException: Error while creating
> archive:
> [ERROR] Exit code: 1 -
> /Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:44:
> error: unknown tag: goal
> [ERROR] * @goal generate
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:45:
> error: unknown tag: phase
> [ERROR] * @phase generate-sources
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:25:
> error: unknown tag: goal
> [ERROR] * @goal help
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:26:
> error: unknown tag: requiresProject
> [ERROR] * @requiresProject false
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:27:
> error: unknown tag: threadSafe
> [ERROR] * @threadSafe
> [ERROR] ^
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/bin/javadoc
> @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in
> '/Users/laurent/devel/drill/tools/fmpp/target/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :drill-fmpp-maven-plugin
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422641#comment-16422641 ] ASF GitHub Bot commented on DRILL-6287: --- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/1182#discussion_r178572285 --- Diff: pom.xml --- @@ -66,6 +66,7 @@ 4096 4096 +-Xdoclint:none --- End diff -- Thanks > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422629#comment-16422629 ] Vlad Rozov edited comment on DRILL-6272 at 4/2/18 3:40 PM: --- [~arina] Please see http://www.apache.org/legal/release-policy.html#compiled-packages and http://www.apache.org/dev/release-publishing.html#valid: ??The Apache Software Foundation exists to create open source software. Thus, the fundamental requirement for a release is that it consist of the necessary *source code* to build the project. Optionally, a release may also be accompanied by *compiled binaries* for the convenience of users.?? was (Author: vrozov): [~arina] Please see http://www.apache.org/dev/release-publishing.html#valid: ??The Apache Software Foundation exists to create open source software. Thus, the fundamental requirement for a release is that it consist of the necessary *source code* to build the project. Optionally, a release may also be accompanied by *compiled binaries* for the convenience of users.?? > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422629#comment-16422629 ] Vlad Rozov commented on DRILL-6272: --- [~arina] Please see http://www.apache.org/dev/release-publishing.html#valid: ??The Apache Software Foundation exists to create open source software. Thus, the fundamental requirement for a release is that it consist of the necessary *source code* to build the project. Optionally, a release may also be accompanied by *compiled binaries* for the convenience of users.?? > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422616#comment-16422616 ] ASF GitHub Bot commented on DRILL-6287: --- Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1182#discussion_r178567909 --- Diff: pom.xml --- @@ -66,6 +66,7 @@ 4096 4096 +-Xdoclint:none --- End diff -- @vdiravka Please see DRILL-4547. > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422590#comment-16422590 ] ASF GitHub Bot commented on DRILL-6287: --- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/1182#discussion_r178561972 --- Diff: pom.xml --- @@ -66,6 +66,7 @@ 4096 4096 +-Xdoclint:none --- End diff -- Do we need a task (new Jira) for refactoring the Drill java docs and moving onto Java 8 doclint? > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422560#comment-16422560 ] ASF GitHub Bot commented on DRILL-6287: --- Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1182 Sorry, Maven not being a strong point, I didn't understand initially what I was looking at. +1 > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422255#comment-16422255 ] ASF GitHub Bot commented on DRILL-6287: --- Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1182 There are two issues with enabling `apache-release` by default: - it triggers creating source `apache-drill-...-src.tar.gz` and `apache-drill-...-src.zip` archives. - maven build for any sub-module fails. The change disables activation of the `apache-release` profile based on JDK version and requires explicit activation during the Apache release process. JDK 1.7 is not supported. See DRILL-1491 and #1143. > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6016) Error reading INT96 created by Apache Spark
[ https://issues.apache.org/jira/browse/DRILL-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422075#comment-16422075 ] ASF GitHub Bot commented on DRILL-6016: --- Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1166 @rajrahul thanks for making all the changes (and of course for the fix)! > Error reading INT96 created by Apache Spark > --- > > Key: DRILL-6016 > URL: https://issues.apache.org/jira/browse/DRILL-6016 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.13.0 >Reporter: Rahul Raj >Assignee: Rahul Raj >Priority: Major > Labels: ready-to-commit > Fix For: 1.14.0 > > > Hi, > I am getting the error - SYSTEM ERROR : ClassCastException: > org.apache.drill.exec.vector.TimeStampVector cannot be cast to > org.apache.drill.exec.vector.VariableWidthVector while trying to read a spark > INT96 datetime field on Drill 1.11 in spite of setting the property > store.parquet.reader.int96_as_timestamp to true. > I believe this was fixed in drill > 1.10(https://issues.apache.org/jira/browse/DRILL-4373). What could be wrong. > I have attached the dataset at > https://github.com/rajrahul/files/blob/master/result.tar.gz -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6223) Drill fails on Schema changes
[
https://issues.apache.org/jira/browse/DRILL-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422069#comment-16422069
]
Parth Chandra commented on DRILL-6223:
--
{quote}Sorry to say, I still disagree with this statement: "This pull request
adds logic to detect and eliminate dangling columns".
{quote}
Adding some more background -
[~paul-rogers] has this exactly right. At the high level, dropping any columns
for star queries is not the solution as a star queries explicitly asks for all
columns.
More importantly, as Paul pointed out, the solution (in some cases) is to
provide a schema.
For Parquet (or any other source that has schema information), we might solve
the problem by creating a union of the columns (and also take care of type
changes by promoting columns to a common type). Parquet currently does not do
that. Worse, it overrides the schema seen so far with the new schema when a
change is encountered (I would suggest, once again, a perusal of the Parquet
metadata cache code). This means in the presence of schema change with Parquet
files, even the planning gets off on the wrong foot.
The only solution, again, as Paul mentioned, is to provide a composite schema
by inferring it or asking the user to provide one. The latter is hard because
sometimes the user does not have the schema or because it is mutating all the
time (it happens when people move fast and break things). This is exactly when
they want to use Drill, and, in fact, this was a primary use case for the early
design of Drill. As a design constraint, we had to assume that we could not
know the schema until runtime (we may have taken it too far :( ).
We could try to infer the schema by doing a complete scan but that has the same
issues as asking the user to specify as schema and, in addition, can take
really really long.
> Drill fails on Schema changes
> --
>
> Key: DRILL-6223
> URL: https://issues.apache.org/jira/browse/DRILL-6223
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.10.0, 1.12.0
>Reporter: salim achouche
>Assignee: salim achouche
>Priority: Major
> Fix For: 1.14.0
>
>
> Drill Query Failing when selecting all columns from a Complex Nested Data
> File (Parquet) Set). There are differences in Schema among the files:
> * The Parquet files exhibit differences both at the first level and within
> nested data types
> * A select * will not cause an exception but using a limit clause will
> * Note also this issue seems to happen only when multiple Drillbit minor
> fragments are involved (concurrency higher than one)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6009) No drillbits on index page
[ https://issues.apache.org/jira/browse/DRILL-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422046#comment-16422046 ] Arina Ielchiieva commented on DRILL-6009: - [~vdonapati] as I have mentioned I saw this scenario only once, so I guess if you cannot reproduce it, just close Jira with won't reproduce. If issue pops up again, we'll re-open it. > No drillbits on index page > -- > > Key: DRILL-6009 > URL: https://issues.apache.org/jira/browse/DRILL-6009 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > Attachments: empty_drillbits.JPG > > > After DRILL-4286 once I saw that index page showed no drillbits at all but it > was working, so at least one drillbit was online (empty_drillbits.JPG). After > refresh everything was fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6223) Drill fails on Schema changes
[
https://issues.apache.org/jira/browse/DRILL-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422042#comment-16422042
]
Parth Chandra commented on DRILL-6223:
--
{quote}To your point about compensation logic in the context of Schema Changes
{quote} *
{quote}Why do you think it is ok to dynamically include new columns?{quote}
*
{quote}Yet it is not ok to exclude them?{quote}
Usually, in real world data with dynamically changing schema's, new columns are
added and not removed.
*
{quote}Consider a batch of 32k rows{quote}
*
{quote}A VV with null integer values will require 32kb (bits) + 32kb * 4 =
160kb{quote}
*
{quote}Each missing column will require that much memory per
mini-fragment{quote}
One of the guarantees provided by value vectors is that elements can be
accessed by index in constant time (or, in the case of nested elements in O(m)
where m is the level of nesting) . The representation is based on providing
this guarantee. It comes at the cost of additional memory usage, which is a
deliberate tradeoff.
{quote}This is unless (similarly to the implicit columns) we optimize the VV
storage representation or / and push the column preservation to higher layers
such as the client or foreman
{quote}
It would be wonderful to improve vectors to use much less memory while
providing the same guarantees. A proposal would be welcome.
> Drill fails on Schema changes
> --
>
> Key: DRILL-6223
> URL: https://issues.apache.org/jira/browse/DRILL-6223
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.10.0, 1.12.0
>Reporter: salim achouche
>Assignee: salim achouche
>Priority: Major
> Fix For: 1.14.0
>
>
> Drill Query Failing when selecting all columns from a Complex Nested Data
> File (Parquet) Set). There are differences in Schema among the files:
> * The Parquet files exhibit differences both at the first level and within
> nested data types
> * A select * will not cause an exception but using a limit clause will
> * Note also this issue seems to happen only when multiple Drillbit minor
> fragments are involved (concurrency higher than one)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6287) apache-release profile should be disabled by default
[ https://issues.apache.org/jira/browse/DRILL-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421992#comment-16421992 ] ASF GitHub Bot commented on DRILL-6287: --- Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1182 I don't understand why the apache-release be disabled by default. And I don't see how this change achieves that anyway. Also, moving -Xdoclint:none to all profiles implies we are no longer supporting development using JDK7 ? I'm OK with that, but not sure if we concluded that at the time of the 1.13 release. If that's what we want to do, I'm fine with this change. > apache-release profile should be disabled by default > > > Key: DRILL-6287 > URL: https://issues.apache.org/jira/browse/DRILL-6287 > Project: Apache Drill > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6272: --- Assignee: Arina Ielchiieva (was: Volodymyr Tkach) > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Arina Ielchiieva >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6272) Remove binary jars files from source distribution
[ https://issues.apache.org/jira/browse/DRILL-6272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421975#comment-16421975 ] Arina Ielchiieva commented on DRILL-6272: - [~vrozov] these files are needed for Dynamic UDF support tests (when we dynamically load functions into Drill). For that purpose we need both source and binary files present. We cannot compile them during Drill build since they are created from separate project, neither they can be pulled from maven repo, since this project is not published. Example of the project: https://github.com/arina-ielchiieva/drillUDF. Also can you please point to Apache documentation where it is indicated that we cannot have such files in release. > Remove binary jars files from source distribution > - > > Key: DRILL-6272 > URL: https://issues.apache.org/jira/browse/DRILL-6272 > Project: Apache Drill > Issue Type: Task >Reporter: Vlad Rozov >Assignee: Volodymyr Tkach >Priority: Critical > Fix For: 1.14.0 > > > Per [~vrozov] the source distribution contains binary jar files under > exec/java-exec/src/test/resources/jars -- This message was sent by Atlassian JIRA (v7.6.3#76005)
