[jira] [Created] (DRILL-5096) Query return null rows .., parquet file creation failed

[Ravan (JIRA)]

Ravan created DRILL-5096:


 Summary: Query return null rows .., parquet file creation failed
 Key: DRILL-5096
 URL: https://issues.apache.org/jira/browse/DRILL-5096
 Project: Apache Drill
  Issue Type: Bug
Reporter: Ravan
 Fix For: 1.8.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-5095) Selecting Join Columns returning null values in the second column

[Ravan (JIRA)]

Ravan created DRILL-5095:


 Summary: Selecting Join Columns returning null values in the 
second column
 Key: DRILL-5095
 URL: https://issues.apache.org/jira/browse/DRILL-5095
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.8.0
Reporter: Ravan


select a.empno,a.ename,a.deptno,b.deptno deptno deptno_1,b.loc from emp a FULL 
JOIN dept b ON a.deptno = b.deptno

Returning O/P:- 

empno ename   deptno  deptno_1  loc

7369.0  SMITH   20.0nullDALLAS
7499.0  ALLEN   30.0nullCHICAGO

Estimated O/P:-

empno ename   deptno  deptno_1  loc

7369.0  SMITH   20.020.0DALLAS
7499.0  ALLEN   30.030.0CHICAGO




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Window function

[Nitin Pawar]

any help on this ?

from  class NoFrameSupportTemplate, I see that

inIndex is hard coded to point  to previous row in case of lag and
next row in case of lead.

Is there a way I can modify this and pass it as parameter to pic
appropriate row?


On Fri, Nov 25, 2016 at 2:57 PM, Nitin Pawar 
wrote:

> adding dev list for comments
>
> On Wed, Nov 23, 2016 at 7:04 PM, Nitin Pawar 
> wrote:
>
>> Hi,
>>
>> according to DRILL-3596
>> , lead or lag function
>> are limited to use offset as 1.
>>
>> according to documentation on postgres
>> lag(value any [, offset integer [, default any ]]) same type as value
>> returns value evaluated at the row that is offset rows before the
>> current row within the partition; if there is no such row, instead return
>> default. Both offset and default are evaluated with respect to the
>> current row. If omitted, offset defaults to 1 and default to null
>>
>>
>> is there any plan to allow offset according to needs but not restrict
>> equal to 1
>>
>> usecase :
>>
>> I have daily data for a month.
>> every day I want to do a delta with last week same day like compare
>> monday with monday and tuesday with tuesday so basically do a lag(col, 7)
>>
>> --
>> Nitin Pawar
>>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Re: [ANNOUNCE] - New Apache Drill Committer - Chris Westin

[yuliya Feldman]

Congratulations Chris!!!

  From: Jacques Nadeau 
 To: dev  
 Sent: Thursday, December 1, 2016 8:54 AM
 Subject: [ANNOUNCE] - New Apache Drill Committer - Chris Westin
   
On behalf of the Apache Drill PMC, I am very pleased to announce that Chris
Westin has accepted the invitation to become a committer in the project.

Welcome Chris and thanks for your great contributions!


--
Jacques Nadeau
CTO and Co-Founder, Dremio


   

[GitHub] drill pull request #675: DRILL-5094: Comparator should guarantee transitive ...

[chunhui-shi]

GitHub user chunhui-shi opened a pull request:

https://github.com/apache/drill/pull/675

DRILL-5094: Comparator should guarantee transitive attribute.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chunhui-shi/drill DRILL-5094

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/675.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #675


commit 90e874a3e38cdfdac9067d55e6829c048bcb459b
Author: chunhui-shi 
Date:   2016-12-02T01:55:12Z

DRILL-5094: Comparator should guarantee transitive attribute.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #638: DRILL-4982: Separate Hive reader classes for differ...

[bitblender]

Github user bitblender commented on a diff in the pull request:

https://github.com/apache/drill/pull/638#discussion_r90574793
  
--- Diff: 
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveAbstractReader.java
 ---
@@ -218,17 +229,18 @@ private void init() throws ExecutionSetupException {
 }
   }
 } catch (Exception e) {
-  throw new ExecutionSetupException("Failure while initializing 
HiveRecordReader: " + e.getMessage(), e);
+  throw new ExecutionSetupException("Failure while initializing 
HiveOrcReader: " + e.getMessage(), e);
--- End diff --

"HiveOrcReader:" in the exception-string should be changed so that it 
prints the actual reader type. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5094) Assure Comparator to be transitive

[Chunhui Shi (JIRA)]

Chunhui Shi created DRILL-5094:
--

 Summary: Assure Comparator to be transitive
 Key: DRILL-5094
 URL: https://issues.apache.org/jira/browse/DRILL-5094
 Project: Apache Drill
  Issue Type: Bug
Reporter: Chunhui Shi
Assignee: Chunhui Shi
Priority: Critical


In AssignmentCreator.java, one Comparator could break transitive attribute 
required for a Comparator implementation and the result is not correct.

E.g. for:
long IntPlusOne = 0x8000L;
[0]=2 * IntPlusOne + 5, [1] = 2* IntPlusOne + 8, [2] = 4 * IntPlusOne + 4,

the compare results will be like:
compare([0],[1]) = -3,
compare([1],[2]) = 4,
compare([0],[2]) = 1 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request #673: DRILL-4764: Parquet file with INT_16, etc. logical ...

[parthchandra]

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/673#discussion_r90570343
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet2/TestDrillParquetReader.java
 ---
@@ -84,4 +94,43 @@ public void test4349() throws Exception {
   .sqlBaselineQuery("SELECT columns[0] id, CAST(NULLIF(columns[1], '') 
AS DOUBLE) val FROM cp.`parquet2/4349.csv.gz` WHERE columns[0] = 'b'")
   .go();
   }
+
+  @Test
+  public void testUnsignedByteShortIntLong() throws Exception {
+Path file = new Path(getDfsTestTmpSchemaLocation(), 
"uint_types.parquet");
+Configuration conf = new Configuration();
+MessageType schema = parseMessageType(
+  "message test { "
++ "required int32 int8_field (INT_8); "
++ "required int32 uint8_field (UINT_8); "
++ "required int32 int16_field (INT_16); "
++ "required int32 uint16_field (UINT_16); "
++ "required int32 uint32_field (UINT_32); "
++ "required int64 uint64_field (UINT_64); "
++"} ");
+GroupWriteSupport.setSchema(schema, conf);
+
+ParquetWriter writer = ExampleParquetWriter.builder(file)
+  .withType(schema)
+  .withConf(conf)
+  .build();
+
+SimpleGroupFactory groupFactory = new SimpleGroupFactory(schema);
+writer.write(
+  groupFactory.newGroup()
+.append("int8_field", -128)
+.append("uint8_field", 127)
+.append("int16_field", -32768)
--- End diff --

Can you add test values for unsigned types as well? 0, -1, 256, 65536, etc


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #648: DRILL-5015: Randomly select the drillbit from the list pro...

[sudheeshkatkam]

Github user sudheeshkatkam commented on the issue:

https://github.com/apache/drill/pull/648
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5093) Explain plan shows all partitions when query scans all partitions, and filter pushdown is used with metadata caching.

[Robert Hou (JIRA)]

Robert Hou created DRILL-5093:
-

 Summary: Explain plan shows all partitions when query scans all 
partitions, and filter pushdown is used with metadata caching.
 Key: DRILL-5093
 URL: https://issues.apache.org/jira/browse/DRILL-5093
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 1.9.0
Reporter: Robert Hou
Assignee: Jinfeng Ni


This query scans all the partitions because the partitions cannot be pruned.  
When metadata caching is used, the explain plan shows all the partitions, when 
it should only show the parent.

0: jdbc:drill:zk=10.10.100.186:5181/drill/rho> explain plan for select \* from 
orders_parts_metadata;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(*=[$0])
00-02Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath 
[path=/drill/testdata/filter/orders_parts_metadata/0_0_1.parquet], 
ReadEntryWithPath 
[path=/drill/testdata/filter/orders_parts_metadata/0_0_3.parquet], 
ReadEntryWithPath 
[path=/drill/testdata/filter/orders_parts_metadata/0_0_4.parquet], 
ReadEntryWithPath 
[path=/drill/testdata/filter/orders_parts_metadata/0_0_5.parquet], 
ReadEntryWithPath 
[path=/drill/testdata/filter/orders_parts_metadata/0_0_2.parquet]], 
selectionRoot=/drill/testdata/filter/orders_parts_metadata, numFiles=5, 
usedMetadataFile=true, 
cacheFileRoot=/drill/testdata/filter/orders_parts_metadata, columns=[`*`]]])


Here is the same query with a table that does not have metadata caching.

0: jdbc:drill:zk=10.10.100.186:5181/drill/rho> explain plan for select \* from 
orders_parts;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(*=[$0])
00-02Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath 
[path=maprfs:///drill/testdata/filter/orders_parts]], 
selectionRoot=maprfs:/drill/testdata/filter/orders_parts, numFiles=1, 
usedMetadataFile=false, columns=[`*`]]])




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] - New Apache Drill Committer - Chris Westin

[Bridget Bevens]

Congratulations, Chris!
http://drill.apache.org/team/

On Thu, Dec 1, 2016 at 3:07 PM, Jason Altekruse  wrote:

> Congrats Chris!
>
> Jason Altekruse
> Software Engineer at Dremio
> Apache Drill Committer
>
> On Thu, Dec 1, 2016 at 2:05 PM, Sudheesh Katkam 
> wrote:
>
> > Congratulations, Chris!
> >
> > > On Dec 1, 2016, at 10:30 AM, Aditya  wrote:
> > >
> > > Congratulations Chris!
> > >
> > > On Thu, Dec 1, 2016 at 9:56 AM, Parth Chandra 
> wrote:
> > >
> > >> Congrats Chris. And thank you for all your cool contributions!
> > >>
> > >>
> > >>
> > >> On Thu, Dec 1, 2016 at 8:54 AM, Jacques Nadeau 
> > wrote:
> > >>
> > >>> On behalf of the Apache Drill PMC, I am very pleased to announce that
> > >> Chris
> > >>> Westin has accepted the invitation to become a committer in the
> > project.
> > >>>
> > >>> Welcome Chris and thanks for your great contributions!
> > >>>
> > >>>
> > >>> --
> > >>> Jacques Nadeau
> > >>> CTO and Co-Founder, Dremio
> > >>>
> > >>
> >
> >
>

[jira] [Created] (DRILL-5092) JDBC driver includes Drill test code

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5092:
--

 Summary: JDBC driver includes Drill test code
 Key: DRILL-5092
 URL: https://issues.apache.org/jira/browse/DRILL-5092
 Project: Apache Drill
  Issue Type: Improvement
Affects Versions: 1.8.0
Reporter: Paul Rogers
Priority: Minor


The Drill JDBC driver ({{DrillConnectionImpl}} contains test code and links to 
test code. This should be factored out separately so that production code does 
not contain test classes or options to enable test features.

In particular:

{code}
import org.apache.drill.exec.util.TestUtilities;
...
  private static void makeTmpSchemaLocationsUnique(StoragePluginRegistry 
pluginRegistry, Properties props) {
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] - New Apache Drill Committer - Chris Westin

[Jason Altekruse]

Congrats Chris!

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Thu, Dec 1, 2016 at 2:05 PM, Sudheesh Katkam 
wrote:

> Congratulations, Chris!
>
> > On Dec 1, 2016, at 10:30 AM, Aditya  wrote:
> >
> > Congratulations Chris!
> >
> > On Thu, Dec 1, 2016 at 9:56 AM, Parth Chandra  wrote:
> >
> >> Congrats Chris. And thank you for all your cool contributions!
> >>
> >>
> >>
> >> On Thu, Dec 1, 2016 at 8:54 AM, Jacques Nadeau 
> wrote:
> >>
> >>> On behalf of the Apache Drill PMC, I am very pleased to announce that
> >> Chris
> >>> Westin has accepted the invitation to become a committer in the
> project.
> >>>
> >>> Welcome Chris and thanks for your great contributions!
> >>>
> >>>
> >>> --
> >>> Jacques Nadeau
> >>> CTO and Co-Founder, Dremio
> >>>
> >>
>
>

[jira] [Created] (DRILL-5091) JDBC unit test fails to set up test storage plugin config on Java 8

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5091:
--

 Summary: JDBC unit test fails to set up test storage plugin config 
on Java 8
 Key: DRILL-5091
 URL: https://issues.apache.org/jira/browse/DRILL-5091
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.8.0
 Environment: Java 8
Reporter: Paul Rogers


Run the {{TestJDBCQuery}} unit tests. They will fail with errors relating to 
the default name space.

The problem is due to a failure (that is ignored, DRILL-4510) to set up the 
test DFS name space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-5090) JDBC tests silently ignore failure to set up test storage plugin

[Paul Rogers (JIRA)]

Paul Rogers created DRILL-5090:
--

 Summary: JDBC tests silently ignore failure to set up test storage 
plugin
 Key: DRILL-5090
 URL: https://issues.apache.org/jira/browse/DRILL-5090
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.8.0
Reporter: Paul Rogers


Run the {{TestJDBCQuery}} test cases using Java 8. The tests will fail with 
error such as:

{code}
VALIDATION ERROR ...
Current default schema: No default schema selected
{code}

The problem is that the tests try to set up a test schema, but fail. The code 
that does this setup just silently ignores the error:

{code}
try {
TestUtilities.updateDfsTestTmpSchemaLocation(pluginRegistry, 
tmpDirPath);
TestUtilities.makeDfsTmpSchemaImmutable(pluginRegistry);
  }
} catch(Throwable e) {
  // ... This is unlikely to
  // happen, but just a safeguard to avoid failing user applications.
  logger.warn("Failed to update tmp schema locations. This step is purely 
for testing purpose. " +
  "Shouldn't be seen in production code.");
  // Ignore the error and go with defaults
}
{code}

The reason that the error is ignored is that the JDBC driver _itself_ contains 
test code in the form of the following check:

{code}
  if (props != null && 
"true".equalsIgnoreCase(props.getProperty("drillJDBCUnitTests"))) {
{code}

That is, the JDBC driver itself contains the code needed to set up the test 
schemas. This means that test code is shipped in production. And, we must 
handle the failure gracefully in case the user set the property in production.

Requested changes:

* Find a way to do test setup without including test code in JDBC.
* If test setup fails, issue a fatal error to direct the developer to the 
actual problem.

And, of course, fix the Java 8 error so that the tests pass. Fixing that error 
is a separate issue. If things were to fail again, we need the above fixes to 
avoid wasting developer's time finding the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] - New Apache Drill Committer - Chris Westin

[Sudheesh Katkam]

Congratulations, Chris!

> On Dec 1, 2016, at 10:30 AM, Aditya  wrote:
> 
> Congratulations Chris!
> 
> On Thu, Dec 1, 2016 at 9:56 AM, Parth Chandra  wrote:
> 
>> Congrats Chris. And thank you for all your cool contributions!
>> 
>> 
>> 
>> On Thu, Dec 1, 2016 at 8:54 AM, Jacques Nadeau  wrote:
>> 
>>> On behalf of the Apache Drill PMC, I am very pleased to announce that
>> Chris
>>> Westin has accepted the invitation to become a committer in the project.
>>> 
>>> Welcome Chris and thanks for your great contributions!
>>> 
>>> 
>>> --
>>> Jacques Nadeau
>>> CTO and Co-Founder, Dremio
>>> 
>> 

Re: [ANNOUNCE] Apache Drill 1.9.0 Released

[Sudheesh Katkam]

Thank you for the information, Sebastian.

This maybe our first time posting to the announce list. I posted on the
list to provide more visibility about the release, but I used the previous
announcement template.

I will add a note about the brief details to our release process
instructions.

- Sudheesh

On Wed, Nov 30, 2016 at 10:35 AM, sebb  wrote:

> What is the project about? Why should I be interested in it?
> [rhetorical questions]
>
> The Announce emails are sent to people not on the developer or user lists.
> Most will have no idea what the project is about.
>
> So the e-mails should contain at least brief details of what the
> product does, and some info on why the new release might be of
> interest to them.
>
> Readers should not have to click the link to find out the basic information
> (although of course it is useful to have such links for further detail).
>
> Please can you add that information to future announce mails?
>
> Thanks.
>
>
> On 29 November 2016 at 23:52, Sudheesh Katkam  wrote:
> > On behalf of the Apache Drill community, I am happy to announce the
> release
> > of Apache Drill 1.9.0.
> >
> > For information about Apache Drill, and to get involved, visit the
> project
> > website:
> >
> > https://drill.apache.org/
> >
> > This release introduces new features and enhancements, including
> > asynchronous Parquet reader, Parquet filter pushdown, dynamic UDF support
> > and HTTPD format plugin. In all, 70 issues have been resolved.
> >
> > The binary and source artifacts are available here:
> >
> > https://drill.apache.org/download/
> >
> > Review the release notes for a complete list of fixes and enhancements:
> >
> > https://drill.apache.org/docs/apache-drill-1-9-0-release-notes/
> >
> > Thanks to everyone in the community who contributed to this release!
> >
> > - Sudheesh
>

[jira] [Created] (DRILL-5089) Skip initializing all enabled storage plugins for every query

[Abhishek Girish (JIRA)]

Abhishek Girish created DRILL-5089:
--

 Summary: Skip initializing all enabled storage plugins for every 
query
 Key: DRILL-5089
 URL: https://issues.apache.org/jira/browse/DRILL-5089
 Project: Apache Drill
  Issue Type: Improvement
  Components: Query Planning & Optimization
Affects Versions: 1.9.0
Reporter: Abhishek Girish


In a query's lifecycle, at attempt is made to initialize each enabled storage 
plugin, while building the schema tree. This is done regardless of the actual 
plugins involved within a query. 

Sometimes, when one or more of the enabled storage plugins have issues - either 
due to misconfiguration or the underlying datasource being slow or being down, 
the overall query time taken increases drastically. Most likely due the attempt 
being made to register schemas from a faulty plugin.

For example, when a jdbc plugin is configured with SQL Server, and at one point 
the underlying SQL Server db goes down, any Drill query starting to execute at 
that point and beyond begin to slow down drastically. 

We must skip registering unrelated schemas (& workspaces) for a query. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] - New Apache Drill Committer - Chris Westin

[Parth Chandra]

Congrats Chris. And thank you for all your cool contributions!



On Thu, Dec 1, 2016 at 8:54 AM, Jacques Nadeau  wrote:

> On behalf of the Apache Drill PMC, I am very pleased to announce that Chris
> Westin has accepted the invitation to become a committer in the project.
>
> Welcome Chris and thanks for your great contributions!
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>

[GitHub] drill pull request #594: DRILL-4842: SELECT * on JSON data results in Number...

[chunhui-shi]

Github user chunhui-shi commented on a diff in the pull request:

https://github.com/apache/drill/pull/594#discussion_r90494185
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
 ---
@@ -59,12 +59,13 @@
   private final ListVectorOutput listOutput;
   private final boolean extended = true;
   private final boolean readNumbersAsDouble;
+  private List path = Lists.newArrayList();
 
   /**
* Collection for tracking nullable fields during reading
* and storing them for creating default typed vectors
*/
-  private final Set nullableFields  = Sets.newLinkedHashSet();
+  private final Set nullableFields  = 
Sets.newLinkedHashSet();
--- End diff --

I did not see update on this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[ANNOUNCE] - New Apache Drill Committer - Chris Westin

[Jacques Nadeau]

On behalf of the Apache Drill PMC, I am very pleased to announce that Chris
Westin has accepted the invitation to become a committer in the project.

Welcome Chris and thanks for your great contributions!


--
Jacques Nadeau
CTO and Co-Founder, Dremio

[GitHub] drill pull request #672: DRILL-5085: Add / update description for dynamic UD...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/672#discussion_r90484226
  
--- Diff: exec/java-exec/src/main/resources/drill-module.conf ---
@@ -207,11 +207,12 @@ drill.exec: {
   // Set this property if custom absolute root should be used for 
remote directories, ex: root: "/app/drill".
   // root: "",
 
-  // Relative base directory for all udf directories (local and 
remote).
+  // Base directory for all udf directories (local and remote).
   base: ${drill.exec.zk.root}"/udf",
 
-  // Relative path to local udf directory.
-  // Generated temporary directory is used as root unless 
${DRILL_TMP_DIR} or ${drill.tmp-dir} is set.
+  // Relative local udf directory. Drill temporary directory is used 
as root for this directory.
+  // Drill temporary directory is set to generated temporary directory
+  // unless ${DRILL_TMP_DIR} or ${drill.tmp-dir} is set.
--- End diff --

Updated to local: "${drill.cluster-tmp-dir}/udf/local".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #672: DRILL-5085: Add / update description for dynamic UD...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/672#discussion_r90483802
  
--- Diff: distribution/src/resources/drill-env.sh ---
@@ -141,4 +141,9 @@
 # Arguments passed to sqlline (the Drill shell) at all times: whether
 # Drill is embedded in Sqlline or not.
 
-#export DRILL_SHELL_JAVA_OPTS="..."
\ No newline at end of file
+#export DRILL_SHELL_JAVA_OPTS="..."
+
+# Location Drill should use for temporary files, such as downloaded 
dynamic UDFs jars.
+# Set to "/tmp" by default.
+#
+# export DRILL_TMP_DIR="..."
--- End diff --

Done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #669: DRILL-5044: After the dynamic registration of multiple jar...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/669
  
@paul-rogers, I have made changes after CR, please review when possible.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5088) Error when reading DBRef column

[Guillaume Champion (JIRA)]

Guillaume Champion created DRILL-5088:
-

 Summary: Error when reading DBRef column
 Key: DRILL-5088
 URL: https://issues.apache.org/jira/browse/DRILL-5088
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
 Environment: drill 1.9.0
mongo 3.2
Reporter: Guillaume Champion


In a mongo database with DBRef, when a DBRef is inserted in the first line of 
mongo's collection drill query failed :
{code}
0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
Error: SYSTEM ERROR: CodecConfigurationException: Can't find a codec for class 
com.mongodb.DBRef.
{code}

Simple example to reproduce:
In mongo instance
{code}
db.contact2.drop();
db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8938"), "account" : 
DBRef("contact", ObjectId("999cbf116b69060001fd8611")) });
{code}

In drill :
{code}
0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
Error: SYSTEM ERROR: CodecConfigurationException: Can't find a codec for class 
com.mongodb.DBRef.


[Error Id: 2944d766-e483-4453-a706-3d481397b186 on Analytics-Biznet:31010] 
(state=,code=0)
{code}

If the first line doesn't contain de DBRef, drill will querying correctly :
In a mongo instance :
{code}
db.contact2.drop();
db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8939") });
db.contact2.insert({ "_id" : ObjectId("582081d96b69060001fd8938"), "account" : 
DBRef("contact", ObjectId("999cbf116b69060001fd8611")) });
{code}

In drill :
{code}
0: jdbc:drill:zk=local> select * from mongo.mydb.contact2;
+--+---+
| _id  |account 
   |
+--+---+
| {"$oid":"582081d96b69060001fd8939"}  | {"$id":{}} 
   |
| {"$oid":"582081d96b69060001fd8938"}  | 
{"$ref":"contact","$id":{"$oid":"999cbf116b69060001fd8611"}}  |
+--+---+
2 rows selected (0,563 seconds)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] Apache Drill 1.9.0 Released

[Arina Yelchiyeva]

Sudheesh,

I guess the following Jiras has been included in Improvement list by
mistake, they are not in 1.9 scope:
[DRILL-4309] - Make this option
store.hive.optimize_scan_with_native_readers=true default
[DRILL-4864] - Add ANSI format for date/time functions
[DRILL-4865] - Add ANSI format for date/time functions

I guess the confusion was that they had Fix Version indicated as 1.9, it
was intended version, in reality they are still in progress / under review.
I have updated their Fix version to Future to avoid the confusion. Could
you please exclude these three Jiras from Improvement list?

Kind regards
Arina


On Wed, Nov 30, 2016 at 8:35 PM, sebb  wrote:

> What is the project about? Why should I be interested in it?
> [rhetorical questions]
>
> The Announce emails are sent to people not on the developer or user lists.
> Most will have no idea what the project is about.
>
> So the e-mails should contain at least brief details of what the
> product does, and some info on why the new release might be of
> interest to them.
>
> Readers should not have to click the link to find out the basic information
> (although of course it is useful to have such links for further detail).
>
> Please can you add that information to future announce mails?
>
> Thanks.
>
>
> On 29 November 2016 at 23:52, Sudheesh Katkam  wrote:
> > On behalf of the Apache Drill community, I am happy to announce the
> release
> > of Apache Drill 1.9.0.
> >
> > For information about Apache Drill, and to get involved, visit the
> project
> > website:
> >
> > https://drill.apache.org/
> >
> > This release introduces new features and enhancements, including
> > asynchronous Parquet reader, Parquet filter pushdown, dynamic UDF support
> > and HTTPD format plugin. In all, 70 issues have been resolved.
> >
> > The binary and source artifacts are available here:
> >
> > https://drill.apache.org/download/
> >
> > Review the release notes for a complete list of fixes and enhancements:
> >
> > https://drill.apache.org/docs/apache-drill-1-9-0-release-notes/
> >
> > Thanks to everyone in the community who contributed to this release!
> >
> > - Sudheesh
>

[jira] [Resolved] (DRILL-4888) putIfAbsent for ZK stores is not atomic

[Arina Ielchiieva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arina Ielchiieva resolved DRILL-4888.
-
Resolution: Fixed

Fixed in scope of DRILL-4726.
Merged in Apache master with commit 89f2633f612a645666de8f51dcb19c6f8044a95e

> putIfAbsent for ZK stores is not atomic
> ---
>
> Key: DRILL-4888
> URL: https://issues.apache.org/jira/browse/DRILL-4888
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.9.0
>
>
> putIfAbsent method in ZookeeperPersistentStore [1] and ZkEphemeralStore [2] 
> is not atomic. First is checks if znode is absent and then puts data into it, 
> but by that time someone might have created znode.
> [1] 
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/store/ZookeeperPersistentStore.java#L89
> [2] 
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/coord/zk/ZkEphemeralStore.java#L90



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)