[jira] [Resolved] (HIVE-24420) Druid test failures
[ https://issues.apache.org/jira/browse/HIVE-24420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa resolved HIVE-24420. - Resolution: Fixed not an issue with latest apache, not seeing it there. was on a different branch > Druid test failures > > > Key: HIVE-24420 > URL: https://issues.apache.org/jira/browse/HIVE-24420 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Test Result (11 failures / ±0) > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_semijoin_reduction_all_types] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24421) DruidOutputFormat and DruidStorageHandler use different filesystem causing issues in data loading
[ https://issues.apache.org/jira/browse/HIVE-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-24421: Description: Seeing below message in the logs - Directory /tmp/workingDir/.staging/segmentDir does not exist, ignore this if it is create statement or inserts of 0 rows, no Druid segments to move, cleaning working directory > DruidOutputFormat and DruidStorageHandler use different filesystem causing > issues in data loading > - > > Key: HIVE-24421 > URL: https://issues.apache.org/jira/browse/HIVE-24421 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Seeing below message in the logs - > Directory /tmp/workingDir/.staging/segmentDir does not exist, ignore this if > it is create statement or inserts of 0 rows, no Druid segments to move, > cleaning working directory -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24421) DruidOutputFormat and DruidStorageHandler use different filesystem causing issues in data loading
[ https://issues.apache.org/jira/browse/HIVE-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-24421: --- > DruidOutputFormat and DruidStorageHandler use different filesystem causing > issues in data loading > - > > Key: HIVE-24421 > URL: https://issues.apache.org/jira/browse/HIVE-24421 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24420) Druid test failures
[ https://issues.apache.org/jira/browse/HIVE-24420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-24420: --- > Druid test failures > > > Key: HIVE-24420 > URL: https://issues.apache.org/jira/browse/HIVE-24420 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Test Result (11 failures / ±0) > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_semijoin_reduction_all_types] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert] > org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23770) Druid filter translation unable to handle inverted between
[ https://issues.apache.org/jira/browse/HIVE-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23770: Attachment: HIVE-23770.1.patch > Druid filter translation unable to handle inverted between > -- > > Key: HIVE-23770 > URL: https://issues.apache.org/jira/browse/HIVE-23770 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23770.1.patch, HIVE-23770.patch > > > Druid filter translation happens in Calcite and does not uses HiveBetween > inverted flag for translation this misses a negation in the planned query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23770) Druid filter translation unable to handle inverted between
[ https://issues.apache.org/jira/browse/HIVE-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147468#comment-17147468 ] Nishant Bangarwa commented on HIVE-23770: - [~jcamachorodriguez] Can you please help review this one. > Druid filter translation unable to handle inverted between > -- > > Key: HIVE-23770 > URL: https://issues.apache.org/jira/browse/HIVE-23770 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23770.patch > > > Druid filter translation happens in Calcite and does not uses HiveBetween > inverted flag for translation this misses a negation in the planned query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23770) Druid filter translation unable to handle inverted between
[ https://issues.apache.org/jira/browse/HIVE-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23770: Status: Patch Available (was: Open) > Druid filter translation unable to handle inverted between > -- > > Key: HIVE-23770 > URL: https://issues.apache.org/jira/browse/HIVE-23770 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23770.patch > > > Druid filter translation happens in Calcite and does not uses HiveBetween > inverted flag for translation this misses a negation in the planned query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23770) Druid filter translation unable to handle inverted between
[ https://issues.apache.org/jira/browse/HIVE-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23770: Attachment: HIVE-23770.patch > Druid filter translation unable to handle inverted between > -- > > Key: HIVE-23770 > URL: https://issues.apache.org/jira/browse/HIVE-23770 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23770.patch > > > Druid filter translation happens in Calcite and does not uses HiveBetween > inverted flag for translation this misses a negation in the planned query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23770) Druid filter translation unable to handle inverted between
[ https://issues.apache.org/jira/browse/HIVE-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-23770: --- > Druid filter translation unable to handle inverted between > -- > > Key: HIVE-23770 > URL: https://issues.apache.org/jira/browse/HIVE-23770 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Druid filter translation happens in Calcite and does not uses HiveBetween > inverted flag for translation this misses a negation in the planned query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.4.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.2.patch, > HIVE-23184.3.patch, HIVE-23184.4.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082800#comment-17082800 ] Nishant Bangarwa commented on HIVE-23184: - test failures are unrelated, reattaching same patch. > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.2.patch, > HIVE-23184.3.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.3.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.2.patch, > HIVE-23184.3.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.2.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.2.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082572#comment-17082572 ] Nishant Bangarwa commented on HIVE-23184: - +cc [~ashutoshc] Please review. > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: (was: HIVE-23184.1.patch) > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.1.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.1.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23184.1.patch, HIVE-23184.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Status: Patch Available (was: Open) > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23184.patch > > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-23184: Attachment: HIVE-23184.patch > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-23184.patch > > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23184) Upgrade druid to 0.17.1
[ https://issues.apache.org/jira/browse/HIVE-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-23184: --- > Upgrade druid to 0.17.1 > --- > > Key: HIVE-23184 > URL: https://issues.apache.org/jira/browse/HIVE-23184 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Upgrade to druid latest release 0.17.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22933) Allow connecting kerberos-enabled Hive to connect to a non-kerberos druid cluster
[ https://issues.apache.org/jira/browse/HIVE-22933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22933: Attachment: HIVE-22933.patch > Allow connecting kerberos-enabled Hive to connect to a non-kerberos druid > cluster > - > > Key: HIVE-22933 > URL: https://issues.apache.org/jira/browse/HIVE-22933 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-22933.patch > > > Currently, If kerberos is enabled for hive, it can only connect to external > druid clusters which are kerberos enabled, Since the Druid client used to > connect to druid is always KerberosHTTPClient, This task is to allow a > kerberos enabled hiverserver2 to connect to non-kerberized druid cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22933) Allow connecting kerberos-enabled Hive to connect to a non-kerberos druid cluster
[ https://issues.apache.org/jira/browse/HIVE-22933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-22933: --- > Allow connecting kerberos-enabled Hive to connect to a non-kerberos druid > cluster > - > > Key: HIVE-22933 > URL: https://issues.apache.org/jira/browse/HIVE-22933 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Currently, If kerberos is enabled for hive, it can only connect to external > druid clusters which are kerberos enabled, Since the Druid client used to > connect to druid is always KerberosHTTPClient, This task is to allow a > kerberos enabled hiverserver2 to connect to non-kerberized druid cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010682#comment-17010682 ] Nishant Bangarwa commented on HIVE-22395: - committed via https://github.com/apache/hive/commit/948144a49753d3955505f428d427fb7b2fb9642a > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, > HIVE-22395.2.patch, HIVE-22395.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Resolution: Fixed Status: Resolved (was: Patch Available) > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, > HIVE-22395.2.patch, HIVE-22395.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Attachment: HIVE-22395.2.patch > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, > HIVE-22395.2.patch, HIVE-22395.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Attachment: HIVE-22395.2.patch > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, HIVE-22395.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Attachment: HIVE-22395.1.patch > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22395.1.patch, HIVE-22395.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22394) Duplicate Jars in druid classpath causing issues
[
https://issues.apache.org/jira/browse/HIVE-22394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962490#comment-16962490
]
Nishant Bangarwa commented on HIVE-22394:
-
merged to master via 1f5a2064aa1e2466901548991ab8720c0f8c6ddb
https://github.com/apache/hive/commit/1f5a2064aa1e2466901548991ab8720c0f8c6ddb
> Duplicate Jars in druid classpath causing issues
>
>
> Key: HIVE-22394
> URL: https://issues.apache.org/jira/browse/HIVE-22394
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22394.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> hive-druid-handler jar has shaded version of druid classes,
> druid-hdfs-storage also has non-shaded classes.
> {code}
> [hive@hiveserver2-1 lib]$ ls |grep druid
> calcite-druid-1.19.0.7.0.2.0-163.jar
> druid-bloom-filter-0.15.1.7.0.2.0-163.jar
> druid-hdfs-storage-0.15.1.7.0.2.0-163.jar
> hive-druid-handler-3.1.2000.7.0.2.0-163.jar
> hive-druid-handler.jar
> {code}
> Exception below -
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:177)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:765)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:630)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
> ... 3 more
> Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:96)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:104)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:743)
> ... 6 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.copyFilesWithChecks(HdfsDataSegmentPusher.java:163)
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.push(HdfsDataSegmentPusher.java:145)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:747)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22394) Duplicate Jars in druid classpath causing issues
[
https://issues.apache.org/jira/browse/HIVE-22394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-22394:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Duplicate Jars in druid classpath causing issues
>
>
> Key: HIVE-22394
> URL: https://issues.apache.org/jira/browse/HIVE-22394
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22394.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> hive-druid-handler jar has shaded version of druid classes,
> druid-hdfs-storage also has non-shaded classes.
> {code}
> [hive@hiveserver2-1 lib]$ ls |grep druid
> calcite-druid-1.19.0.7.0.2.0-163.jar
> druid-bloom-filter-0.15.1.7.0.2.0-163.jar
> druid-hdfs-storage-0.15.1.7.0.2.0-163.jar
> hive-druid-handler-3.1.2000.7.0.2.0-163.jar
> hive-druid-handler.jar
> {code}
> Exception below -
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:177)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:765)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:630)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
> ... 3 more
> Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:96)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:104)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:743)
> ... 6 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.copyFilesWithChecks(HdfsDataSegmentPusher.java:163)
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.push(HdfsDataSegmentPusher.java:145)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:747)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Status: Patch Available (was: Open) > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-22395.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-22395: Attachment: HIVE-22395.patch > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-22395.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22395) Add ability to read Druid metastore password from jceks
[ https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-22395: --- > Add ability to read Druid metastore password from jceks > --- > > Key: HIVE-22395 > URL: https://issues.apache.org/jira/browse/HIVE-22395 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22394) Duplicate Jars in druid classpath causing issues
[
https://issues.apache.org/jira/browse/HIVE-22394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-22394:
Status: Patch Available (was: Open)
> Duplicate Jars in druid classpath causing issues
>
>
> Key: HIVE-22394
> URL: https://issues.apache.org/jira/browse/HIVE-22394
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-22394.patch
>
>
> hive-druid-handler jar has shaded version of druid classes,
> druid-hdfs-storage also has non-shaded classes.
> {code}
> [hive@hiveserver2-1 lib]$ ls |grep druid
> calcite-druid-1.19.0.7.0.2.0-163.jar
> druid-bloom-filter-0.15.1.7.0.2.0-163.jar
> druid-hdfs-storage-0.15.1.7.0.2.0-163.jar
> hive-druid-handler-3.1.2000.7.0.2.0-163.jar
> hive-druid-handler.jar
> {code}
> Exception below -
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:177)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:765)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:630)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
> ... 3 more
> Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:96)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:104)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:743)
> ... 6 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.copyFilesWithChecks(HdfsDataSegmentPusher.java:163)
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.push(HdfsDataSegmentPusher.java:145)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:747)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22394) Duplicate Jars in druid classpath causing issues
[
https://issues.apache.org/jira/browse/HIVE-22394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-22394:
Attachment: HIVE-22394.patch
> Duplicate Jars in druid classpath causing issues
>
>
> Key: HIVE-22394
> URL: https://issues.apache.org/jira/browse/HIVE-22394
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-22394.patch
>
>
> hive-druid-handler jar has shaded version of druid classes,
> druid-hdfs-storage also has non-shaded classes.
> {code}
> [hive@hiveserver2-1 lib]$ ls |grep druid
> calcite-druid-1.19.0.7.0.2.0-163.jar
> druid-bloom-filter-0.15.1.7.0.2.0-163.jar
> druid-hdfs-storage-0.15.1.7.0.2.0-163.jar
> hive-druid-handler-3.1.2000.7.0.2.0-163.jar
> hive-druid-handler.jar
> {code}
> Exception below -
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:177)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:765)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:630)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
> ... 3 more
> Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:96)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:104)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:743)
> ... 6 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.copyFilesWithChecks(HdfsDataSegmentPusher.java:163)
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.push(HdfsDataSegmentPusher.java:145)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:747)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22394) Duplicate Jars in druid classpath causing issues
[
https://issues.apache.org/jira/browse/HIVE-22394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa reassigned HIVE-22394:
---
> Duplicate Jars in druid classpath causing issues
>
>
> Key: HIVE-22394
> URL: https://issues.apache.org/jira/browse/HIVE-22394
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
>
> hive-druid-handler jar has shaded version of druid classes,
> druid-hdfs-storage also has non-shaded classes.
> {code}
> [hive@hiveserver2-1 lib]$ ls |grep druid
> calcite-druid-1.19.0.7.0.2.0-163.jar
> druid-bloom-filter-0.15.1.7.0.2.0-163.jar
> druid-hdfs-storage-0.15.1.7.0.2.0-163.jar
> hive-druid-handler-3.1.2000.7.0.2.0-163.jar
> hive-druid-handler.jar
> {code}
> Exception below -
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:177)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:765)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:630)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$1.apply(Futures.java:713)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861)
> ... 3 more
> Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:96)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:104)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:743)
> ... 6 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.fs.HadoopFsWrapper
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.copyFilesWithChecks(HdfsDataSegmentPusher.java:163)
> at
> org.apache.hive.druid.org.apache.druid.storage.hdfs.HdfsDataSegmentPusher.push(HdfsDataSegmentPusher.java:145)
> at
> org.apache.hive.druid.org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:747)
> at
> org.apache.hive.druid.org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.10.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.10.patch,
> HIVE-20683.2.patch, HIVE-20683.3.patch, HIVE-20683.4.patch,
> HIVE-20683.5.patch, HIVE-20683.6.patch, HIVE-20683.8.patch, HIVE-20683.patch
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.8.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch,
> HIVE-20683.6.patch, HIVE-20683.8.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.6.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch,
> HIVE-20683.6.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.5.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: (was: HIVE-20683.5.patch)
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901406#comment-16901406
]
Nishant Bangarwa commented on HIVE-20683:
-
[~jcamachorodriguez] this seems ready to be pushed, unless you have any
comments ?
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.5.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.5.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.4.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.4.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896539#comment-16896539
]
Nishant Bangarwa commented on HIVE-20683:
-
fixed checkstyle and updated qfile for dynamic_semijoin_reduction_3 and
semijoin_hint
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.3.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch,
> HIVE-20683.3.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: (was: HIVE-20683.2.patch)
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.2.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.2.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.2.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Status: Patch Available (was: Open)
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20683:
Attachment: HIVE-20683.1.patch
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885407#comment-16885407
]
Nishant Bangarwa commented on HIVE-20683:
-
[~jcamachorodriguez] rebased and created
https://github.com/apache/hive/pull/723 . please review.
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885224#comment-16885224
]
Nishant Bangarwa commented on HIVE-20683:
-
[~jcamachorodriguez] yeah it can be checked in now, working on rebase.
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-20683.patch
>
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Attachment: HIVE-21612.2.patch > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.1.patch, HIVE-21612.1.patch, > HIVE-21612.1.patch, HIVE-21612.2.patch, HIVE-21612.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Attachment: HIVE-21612.1.patch > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.1.patch, HIVE-21612.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Status: Patch Available (was: Open) > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Status: Open (was: Patch Available) > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821018#comment-16821018 ] Nishant Bangarwa commented on HIVE-21612: - [~bslim] [~ashutoshc] Any comments here ? > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816629#comment-16816629 ] Nishant Bangarwa commented on HIVE-21612: - +cc [~ashutoshc] [~bslim] Please review. > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21612.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Status: Patch Available (was: Open) > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-21612.patch > > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-21612: Attachment: HIVE-21612.patch > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-21612.patch > > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21612) Upgrade druid to 0.14.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa reassigned HIVE-21612: --- > Upgrade druid to 0.14.0-incubating > -- > > Key: HIVE-21612 > URL: https://issues.apache.org/jira/browse/HIVE-21612 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Druid 0.14.0-incubating is released. > This task is to upgrade hive to use 0.14.0-incubating version of druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.7.patch > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.5.patch, > HIVE-20546.6.patch, HIVE-20546.7.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.6.patch > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.5.patch, > HIVE-20546.6.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.5.patch > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.5.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778008#comment-16778008 ] Nishant Bangarwa edited comment on HIVE-20546 at 2/26/19 2:51 PM: -- rebased and updated patch, [~bslim] any more comments here ? the checkstyle voilations are fixed now. was (Author: nishantbangarwa): rebased and updated patch, [~bslim] any comments here ? > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.5.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778008#comment-16778008 ] Nishant Bangarwa commented on HIVE-20546: - rebased and updated patch, [~bslim] any comments here ? > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.5.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746682#comment-16746682 ] Nishant Bangarwa edited comment on HIVE-20546 at 1/18/19 9:33 PM: -- Fixed Findbugs error and updated the patch. was (Author: nishantbangarwa): Fixed Findbugs error and fixed the patch. > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746682#comment-16746682 ] Nishant Bangarwa commented on HIVE-20546: - Fixed Findbugs error and fixed the patch. > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.4.patch > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.4.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.3.patch > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, > HIVE-20546.3.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Druid 0.13.0
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.2.patch > Upgrade to Druid 0.13.0 > --- > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0-incubating
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Summary: Upgrade to Apache Druid 0.13.0-incubating (was: Upgrade to Apache Druid 0.13.0) > Upgrade to Apache Druid 0.13.0-incubating > - > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Apache Druid 0.13.0
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Summary: Upgrade to Apache Druid 0.13.0 (was: Upgrade to Druid 0.13.0) > Upgrade to Apache Druid 0.13.0 > -- > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.2.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Druid 0.13.0
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Status: Patch Available (was: Open) > Upgrade to Druid 0.13.0 > --- > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20546) Upgrade to Druid 0.13.0
[ https://issues.apache.org/jira/browse/HIVE-20546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20546: Attachment: HIVE-20546.1.patch > Upgrade to Druid 0.13.0 > --- > > Key: HIVE-20546 > URL: https://issues.apache.org/jira/browse/HIVE-20546 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20546.1.patch, HIVE-20546.patch > > > This task is to upgrade to druid 0.13.0 when it is released. Note that it > will hopefully be first apache release for Druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692797#comment-16692797 ] Nishant Bangarwa commented on HIVE-20932: - [~bslim] Can you also share some benchmark results regarding the performance gains ? > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.3.patch, HIVE-20932.4.patch, HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686468#comment-16686468 ] Nishant Bangarwa edited comment on HIVE-19026 at 11/14/18 12:50 PM: pushed to master. Thanks [~ashutoshc] and [~bslim] for review. was (Author: nishantbangarwa): pushed to master > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.7.patch, HIVE-19026.8.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.7.patch, HIVE-19026.8.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686468#comment-16686468 ] Nishant Bangarwa commented on HIVE-19026: - pushed to master > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.7.patch, HIVE-19026.8.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.8.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.7.patch, HIVE-19026.8.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.7.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.7.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684838#comment-16684838 ] Nishant Bangarwa commented on HIVE-19026: - resolved merge conflicts and updates patch. > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.6.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, > HIVE-19026.6.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.5.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.5.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683537#comment-16683537 ] Nishant Bangarwa commented on HIVE-19026: - [~bslim] [~vgarg] updated patch based on latest review comments. > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.4.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.4.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Description: https://issues.apache.org/jira/browse/HIVE-18976 introduces support for setting up druid kafka-indexing service. Input serialization should be configurable. for now we can say we only support json, but there should be a mechanism to support other formats. Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. Also add support to ingest timestamp column when the input timestamp column name in input is not `__time`. e.g. CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other columns...) STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' TBLPROPERTIES ( "druid.timestamp.column" = "myinputColumnTimestamp" other ppts ) was: https://issues.apache.org/jira/browse/HIVE-18976 introduces support for setting up druid kafka-indexing service. Input serialization should be configurable. for now we can say we only support json, but there should be a mechanism to support other formats. Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. > Also add support to ingest timestamp column when the input timestamp column > name in input is not `__time`. > e.g. > CREATE EXTERNAL TABLE druid_kafka_test_avro(__time timestamp , other > columns...) > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.timestamp.column" = "myinputColumnTimestamp" > other ppts > ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671852#comment-16671852 ] Nishant Bangarwa commented on HIVE-19026: - +cc [~vgarg] Please help with the review. > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.3.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, > HIVE-19026.3.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.2.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.2.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19026: Attachment: HIVE-19026.1.patch > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.1.patch, HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20812) Update jetty dependency to 9.3.25.v20180904
[ https://issues.apache.org/jira/browse/HIVE-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664361#comment-16664361 ] Nishant Bangarwa commented on HIVE-20812: - :+1 > Update jetty dependency to 9.3.25.v20180904 > --- > > Key: HIVE-20812 > URL: https://issues.apache.org/jira/browse/HIVE-20812 > Project: Hive > Issue Type: Task >Reporter: Thejas M Nair >Assignee: Thejas M Nair >Priority: Critical > Attachments: HIVE-20812.1.patch > > > The jetty version 9.3.20.v20170531 being used currently in master has several > CVE associated with it. > Version 9.3.25.v20180904 has those issues resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20490) UDAF: Add an `approx_distinct` to Hive
[ https://issues.apache.org/jira/browse/HIVE-20490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654765#comment-16654765 ] Nishant Bangarwa commented on HIVE-20490: - +1 > UDAF: Add an `approx_distinct` to Hive > -- > > Key: HIVE-20490 > URL: https://issues.apache.org/jira/browse/HIVE-20490 > Project: Hive > Issue Type: Improvement > Components: UDF >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-20490.1.patch, HIVE-20490.2.patch > > > This gives a direct pathway to call the equivalent of Hive's default > compute_stats() UDAF which computes a HyperLogLog bitVector + approx distinct > on a column. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20700) Add config to disable rollup for druid
[ https://issues.apache.org/jira/browse/HIVE-20700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643836#comment-16643836 ] Nishant Bangarwa commented on HIVE-20700: - unrelated failures, reattaching patch > Add config to disable rollup for druid > -- > > Key: HIVE-20700 > URL: https://issues.apache.org/jira/browse/HIVE-20700 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20700.1.patch, HIVE-20700.2.patch, > HIVE-20700.3.patch, HIVE-20700.patch > > > Add a table property - 'druid.rollup' to allow disabling rollup for druid > tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20700) Add config to disable rollup for druid
[ https://issues.apache.org/jira/browse/HIVE-20700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20700: Attachment: HIVE-20700.3.patch > Add config to disable rollup for druid > -- > > Key: HIVE-20700 > URL: https://issues.apache.org/jira/browse/HIVE-20700 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20700.1.patch, HIVE-20700.2.patch, > HIVE-20700.3.patch, HIVE-20700.patch > > > Add a table property - 'druid.rollup' to allow disabling rollup for druid > tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20686) Sync QueryIDs across hive and druid
[ https://issues.apache.org/jira/browse/HIVE-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643755#comment-16643755 ] Nishant Bangarwa commented on HIVE-20686: - reuploaded patch. > Sync QueryIDs across hive and druid > --- > > Key: HIVE-20686 > URL: https://issues.apache.org/jira/browse/HIVE-20686 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20686.1.patch, HIVE-20686.2.patch, > HIVE-20686.3.patch, HIVE-20686.4.patch, HIVE-20686.5.patch, HIVE-20686.patch > > > For the queries that hive passes to druid, pass on additional queryID as > query context. > It will be useful in tracing query level metrics across druid and hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20698) Better error instead of NPE when timestamp is null for any row when ingesting to druid
[
https://issues.apache.org/jira/browse/HIVE-20698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20698:
Attachment: HIVE-20698.2.patch
> Better error instead of NPE when timestamp is null for any row when ingesting
> to druid
> --
>
> Key: HIVE-20698
> URL: https://issues.apache.org/jira/browse/HIVE-20698
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-20698.1.patch, HIVE-20698.2.patch, HIVE-20698.patch
>
>
> Currently when ingesting data to druid we get a wierd NPE when timestamp is
> null for any row.
> We should provide an error with a better message which helps user to know
> what is actually wrong.
> {code}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.druid.serde.DruidSerDe.serialize(DruidSerDe.java:364)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:957)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:111)
> at org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:965)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:480)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20686) Sync QueryIDs across hive and druid
[ https://issues.apache.org/jira/browse/HIVE-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20686: Attachment: HIVE-20686.5.patch > Sync QueryIDs across hive and druid > --- > > Key: HIVE-20686 > URL: https://issues.apache.org/jira/browse/HIVE-20686 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20686.1.patch, HIVE-20686.2.patch, > HIVE-20686.3.patch, HIVE-20686.4.patch, HIVE-20686.5.patch, HIVE-20686.patch > > > For the queries that hive passes to druid, pass on additional queryID as > query context. > It will be useful in tracing query level metrics across druid and hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19452) Avoid Deserializing and Serializing Druid query in DruidRecordReaders
[ https://issues.apache.org/jira/browse/HIVE-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa resolved HIVE-19452. - Resolution: Won't Fix With the ability to push dynamic filters, we need to do serde of query on record readers. > Avoid Deserializing and Serializing Druid query in DruidRecordReaders > - > > Key: HIVE-19452 > URL: https://issues.apache.org/jira/browse/HIVE-19452 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > > Druid record reader deserializes and serializes the Druid query before > sending it to druid. > This can be avoided and we can stop packaging some of druid dependencies e.g. > org.antlr from druid-handler selfcontained jar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19026) Configurable serde for druid kafka indexing
[ https://issues.apache.org/jira/browse/HIVE-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643569#comment-16643569 ] Nishant Bangarwa commented on HIVE-19026: - +cc [~ashutoshc] > Configurable serde for druid kafka indexing > > > Key: HIVE-19026 > URL: https://issues.apache.org/jira/browse/HIVE-19026 > Project: Hive > Issue Type: Task >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19026.patch > > > https://issues.apache.org/jira/browse/HIVE-18976 introduces support for > setting up druid kafka-indexing service. > Input serialization should be configurable. for now we can say we only > support json, but there should be a mechanism to support other formats. > Perhaps, we can make use of Hive's serde library like LazySimpleSerde etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
