[jira] [Created] (DRILL-6310) limit batch size for hash aggregate

Padma Penumarthy created DRILL-6310: --- Summary: limit batch size for hash aggregate Key: DRILL-6310 URL: https://issues.apache.org/jira/browse/DRILL-6310 Project: Apache Drill Issue Type:

Re: "Death of Schema-on-Read"

On Thu, Apr 5, 2018 at 9:02 AM, Aman Sinha wrote: > All good discussions in this thread. It clearly shows that Drill's > schema-on-read is not only a nice-to-have but for applications like IOT, it > is a must-have. > Absolutely. It must be possible to have schema on read

Re: "Death of Schema-on-Read"

On Thu, Apr 5, 2018 at 12:12 PM, Charles Givre wrote: > I’ll weigh in here. IMHO, Drill’s schema discovery features are > excellent, but what would be nice is if Drill could: > 1. Accept hints or directives for the schema > 2. Remember these hints if the file doesn’t change.

[GitHub] drill pull request #1202: DRILL-6311: No logging information in drillbit.log...

Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1202#discussion_r179626383 --- Diff: exec/vector/pom.xml --- @@ -69,6 +69,7 @@ org.apache.drill drill-common ${project.version} + test ---

[GitHub] drill issue #1199: DRILL-6303: Provide a button to copy the Drillbit's JStac...

Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1199 @arina-ielchiieva I've made updates based on the review.  I've also skipped

[GitHub] drill issue #1195: DRILL-6273: Removed dependency licensed under Category X

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1195 LGTM ---

No log prints in drillbit.log on latest master

Hi All, On latest Apache master, commit for DRILL-6234 added a dependency inside /exec/vector/pom.xml file for drill-common artifact. But the change missed to specify the scope as test, due to this a separate jar was created inside

New Bee Questions

Hi Team, I recently started working on Drill and got a challenging work and I have implemented the query in SQL Server.I have to use the same logic in Drill. First - I was wondering if we have unpivot option in Drill in order to convert columns to rows which is needed for my requirement.

[GitHub] drill pull request #1202: DRILL-6311: No logging information in drillbit.log...

GitHub user sohami opened a pull request: https://github.com/apache/drill/pull/1202 DRILL-6311: No logging information in drillbit.log / drillbit.out You can merge this pull request into a Git repository by running: $ git pull https://github.com/sohami/drill DRILL-6311

[GitHub] drill issue #1202: DRILL-6311: No logging information in drillbit.log / dril...

Github user sohami commented on the issue: https://github.com/apache/drill/pull/1202 @vrozov - Please help to review. ---

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

Github user priteshm commented on the issue: https://github.com/apache/drill/pull/1184 @parthchandra can you please review it. @jiang-wu Parth is traveling and he would be able to review it next week. ---

Re: "Death of Schema-on-Read"

On Thu, Apr 5, 2018 at 7:24 AM, Joel Pfaff wrote: > Hello, > > A lot of versioning problems arise when trying to share data through kafka > between multiple applications with different lifecycles and maintainers, > since by default, a single message in Kafka is just a blob.

[jira] [Created] (DRILL-6311) No logging information in drillbit.log / drillbit.out

Sorabh Hamirwasia created DRILL-6311: Summary: No logging information in drillbit.log / drillbit.out Key: DRILL-6311 URL: https://issues.apache.org/jira/browse/DRILL-6311 Project: Apache Drill

Re: "Death of Schema-on-Read"

Great discussion. Really appreciate the insight from the Drill users! To Ted's points: the simplest possible solution is to allow a table function to express types. Just making stuff up: SELECT a FROM schema(myTable, (a: INT)) Or, a SQL extension: SELECT a FROM myTable(a: INT) Or, really

Re: "Death of Schema-on-Read"

Hi Aman, To use an analogy, Drill is the first self-driving query engine (with respect to schema.) Like all self-driving or auto-pilot mechanisms, it needs a human to take control in dicey situations. Suppose we automatically do sampling, or we remember prior queries. We need a way to pass

Re: [DISCUSS] DrillBuf

Hi Vlad, > I'd suggest to keep focus on DrillBuf design and implementation as the only > gate for accessing raw (direct) memory. I was doing that. By explaining where DrillBuf fits in the overall design, we see that DrillBuf should be the only access point for direct memory. The context

Re: "Death of Schema-on-Read"

Hello, Thank you Paul for starting this discussion. However, I was not clear on the latest point as to how providing hints and creating a view(mechanism which already exists in DRILL) is different. I do think that creating a view can be cumbersome (in terms of syntax). Providing hints are

[GitHub] drill issue #1199: DRILL-6303: Provide a button to copy the Drillbit's JStac...

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1199 @kkhatua thanks for making the changes, it believe it's very useful but I suggest we do consider some design changes: 1. Information and auto-refresh display as badege and do not

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179405848 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179406094 --- Diff: exec/java-exec/src/main/resources/rest/profile/profile.ftl --- @@ -372,6 +372,16 @@ table.sortable thead .sorting_desc { background-image:

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179403811 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179402877 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -38,7 +39,16 @@ * Wrapper class

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179403081 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -38,7 +39,16 @@ * Wrapper class

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179405507 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill issue #1126: DRILL-6179: Added pcapng-format support

Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1126 @kkhatua better ping @tdunning, ---

[jira] [Created] (DRILL-6309) Show currently logged in user inside SQL session eg. SELECT CURRENT_USER();

Hari Sekhon created DRILL-6309: -- Summary: Show currently logged in user inside SQL session eg. SELECT CURRENT_USER(); Key: DRILL-6309 URL: https://issues.apache.org/jira/browse/DRILL-6309 Project:

Re: "Death of Schema-on-Read"

Hello, A lot of versioning problems arise when trying to share data through kafka between multiple applications with different lifecycles and maintainers, since by default, a single message in Kafka is just a blob. One way to solve that is to agree on a single serialization format, friendly with

Re: "Death of Schema-on-Read"

All good discussions in this thread. It clearly shows that Drill's schema-on-read is not only a nice-to-have but for applications like IOT, it is a must-have. For other types of data that is slowly changing, in order to improve overall user experience where the user is willing to run offline

Re: [DISCUSS] DrillBuf

Hi Paul, Thank you, it is good to have a different angle view on Vectors, Operators and Batches. For this thread, I'd suggest to keep focus on DrillBuf design and implementation as the only gate for accessing raw (direct) memory. It will be good for the community to agree on 1. Boundary

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179551936 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179552510 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -38,7 +39,16 @@ * Wrapper class for

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179552379 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -38,7 +39,16 @@ * Wrapper class for

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179551687 --- Diff: exec/java-exec/src/main/resources/rest/profile/profile.ftl --- @@ -372,6 +372,16 @@ table.sortable thead .sorting_desc { background-image:

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179557189 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill pull request #1197: DRILL-6279: UI indicates operators that spilled in...

Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/1197#discussion_r179552086 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/OperatorWrapper.java --- @@ -190,8 +223,59 @@ public void

[GitHub] drill issue #1197: DRILL-6279: UI indicates operators that spilled in-memory...

Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1197 @arina-ielchiieva done the requested changes. ---

Re: "Death of Schema-on-Read"

I’ll weigh in here. IMHO, Drill’s schema discovery features are excellent, but what would be nice is if Drill could: 1. Accept hints or directives for the schema 2. Remember these hints if the file doesn’t change. 3. Allow these hints to be applied to a directory of files. Therefore, I do