[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179026313 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027582 --- Diff: distribution/src/resources/drillbit.sh --- @@ -154,6 +192,7 @@ start_bit ( ) nohup nice -n $DRILL_NICENESS "$DRILL_HOME/bin/runbit" exec

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025617 --- Diff: distribution/src/resources/drill-env.sh --- @@ -86,6 +86,12 @@ #export DRILL_PID_DIR=${DRILL_PID_DIR:-$DRILL_HOME} +#

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028127 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028625 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027761 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028078 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179026462 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025777 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179028377 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179025400 --- Diff: distribution/src/resources/drill-env.sh --- @@ -86,6 +86,12 @@ #export DRILL_PID_DIR=${DRILL_PID_DIR:-$DRILL_HOME} +#

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179027910 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

Re: "Death of Schema-on-Read"

Well, the restart strategy still works for your examples. And you only pay once. From them you look at the cached type information and used an upper bound data type as you read the data. Since it works to read the values in the right order, it is obviously possible to push down typing information

Re: "Death of Schema-on-Read"

Subtle point. I can provide schema with Parquet, as you note. (Actually, for Parquet, Drill is schema-required: I can't not provide a schema due to the nature of Parquet...) But, I can't provide a schema for JSON, CSV, etc. The point is, Drill forbids the user from providing a schema; only the

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023466 --- Diff: src/main/resources/checkstyle-config.xml --- @@ -30,10 +30,15 @@ + --- End diff -- Do we want to do

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022090 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -777,23 +778,20 @@ public int getActualMemoryConsumed() { * @return

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022533 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -534,15 +534,11 @@ public void setSafe(int index, byte[] bytes) {

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023079 --- Diff: exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/writer/BaseScalarWriter.java --- @@ -211,7 +211,7 @@ protected boolean

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179023675 --- Diff: src/main/resources/checkstyle-suppressions.xml --- @@ -16,4 +16,13 @@ +

[GitHub] drill pull request #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsExc...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1144#discussion_r179022338 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -534,15 +534,11 @@ public void setSafe(int index, byte[] bytes) {

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1144 My two cents... DrillBuf is the only memory-level abstraction that (low level) Drill code should reference. The UDLE and other bits should be fully encapsulated. This guideline lets us evolve

[GitHub] drill issue #1200: DRILL-143: Support CGROUPs resource management

Github user Ben-Zvi commented on the issue: https://github.com/apache/drill/pull/1200 Could there be users with an older Linux (pre 4.5, circa March 2016) which does not support cgroups V2 ? ---

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

Github user Ben-Zvi commented on a diff in the pull request: https://github.com/apache/drill/pull/1200#discussion_r179005732 --- Diff: distribution/src/resources/drillbit.sh --- @@ -127,6 +127,44 @@ check_before_start() fi } +check_after_start(){ +

[jira] [Created] (DRILL-6307) Handle empty batches in record batch sizer correctly

Padma Penumarthy created DRILL-6307: --- Summary: Handle empty batches in record batch sizer correctly Key: DRILL-6307 URL: https://issues.apache.org/jira/browse/DRILL-6307 Project: Apache Drill

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1144 IMO, this PR or DRILL-6202 is not the best place to discuss boundary checking as the PR and JIRA deals with `IndexOutOfBoundsException` but does not change how DrillBuf, Vector or Operators ensure

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

Github user sachouche commented on the issue: https://github.com/apache/drill/pull/1144 I have added a comment within this PR associated JIRA: [DRILL-6202](https://issues.apache.org/jira/browse/DRILL-6202) ---

[GitHub] drill pull request #1195: DRILL-6273: Removed dependency licensed under Cate...

Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1195#discussion_r178953868 --- Diff: tools/fmpp/src/main/java/bsh/package-info.java --- @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: "Death of Schema-on-Read"

I don't see why you say that Drill is schema-forbidden. The Parquet reader, for instance, makes strong use of the implied schema to facilitate reading of typed data. Likewise, the vectorized internal format is strongly typed and, as such, uses schema information. Views are another way to

Re: "Death of Schema-on-Read"

Here is another way to think about it. Today, Drill is "schema-forbidden": even if I know the schema, I can't communicate that to Drill; Drill must figure it out on its own, making the same mistakes every time on ambiguous schemas. Contrast this with Hive, which is "schema-required": I must

Re: Drill Hangout today at 10 am PST

Hi All, I'll be giving a presentation about unit testing in Drill. Thanks, Tim From: Vitalii Diravka Sent: Tuesday, April 3, 2018 7:53:49 AM To: dev@drill.apache.org Subject: Re: Drill Hangout today at 10 am PST I have a small

[GitHub] drill issue #1201: DRILL-4091: Support for additional gis operations in gis ...

Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1201 @cgivre could you review this? ---

[GitHub] drill issue #1198: DRILL-6294: Changes to support Calcite 1.16.0

Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/1198 @chunhui-shi, I have addressed all CR comments, could you please take a look again? ---

Re: Drill Hangout today at 10 am PST

I have a small topic/question regarding "Jenkins builds" for drill-scm project [1] It started to fail due to the old Java version. I'm interested in the possibility of updating the JDK version there. 1. https://builds.apache.org/blue/organizations/jenkins/drill-scm/activity Kind regards Vitalii

[GitHub] drill pull request #1201: DRILL-4091: Support for additional gis operations ...

GitHub user ChrisSandison opened a pull request: https://github.com/apache/drill/pull/1201 DRILL-4091: Support for additional gis operations in gis contrib module This based off of PR https://github.com/apache/drill/pull/258. I have attempted to address the comments from the PR

Drill Hangout today at 10 am PST

We will have our bi-weekly hangout today Apr 3rd at 10 am PST. Please reply to this post with proposed topics to discuss. Hangout link:

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1144 I think we need to include a few other folks into this. @paul-rogers, @sachouche, have also looked into the issue of excessive bounds checking and ways to write to direct memory with minimum

Re: "Death of Schema-on-Read"

This, of course, begs the question [1], doesn't it? If you have the schema, then you have either a) spent time designing and documenting your data (both the schema and dictionary containing the semantics) or b) spent time "finding, interpreting, and cleaning data" to discover the data schema and

[GitHub] drill pull request #814: Add clickable localhost URL

Github user ebuildy closed the pull request at: https://github.com/apache/drill/pull/814 ---

[GitHub] drill issue #1200: DRILL-143: Support CGROUPs resource management

Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1200 @paul-rogers / @Ben-Zvi , could you review this? ---

[GitHub] drill pull request #1200: DRILL-143: Support CGROUPs resource management

GitHub user kkhatua opened a pull request: https://github.com/apache/drill/pull/1200 DRILL-143: Support CGROUPs resource management Introduces the `DRILLBIT_CGROUP` option in defined in `drill-env.sh` The startup script checks if the specified CGroup (ver 2) is available and