[jira] [Commented] (KNOX-1362) Add documentation for the interaction with SHS

2018-06-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519036#comment-16519036 ] Marco Gaido commented on KNOX-1362: --- [~lmccay] I created the issue as you suggested in K

[jira] [Created] (KNOX-1362) Add documentation for the interaction with SHS

2018-06-21 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1362: - Summary: Add documentation for the interaction with SHS Key: KNOX-1362 URL: https://issues.apache.org/jira/browse/KNOX-1362 Project: Apache Knox Issue Type: Improv

[jira] [Commented] (KNOX-1315) Spark UI urls issue: Jobs, stdout/stderr and threadDump links

2018-06-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519032#comment-16519032 ] Marco Gaido commented on KNOX-1315: --- [~lmccay] this is actually a patch on YARN UI. As I

[jira] [Commented] (SPARK-24607) Distribute by rand() can lead to data inconsistency

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518196#comment-16518196 ] Marco Gaido commented on SPARK-24607: - [~viirya] please check the description in the

[jira] [Updated] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24606: Priority: Major (was: Blocker) > Decimals multiplication and division may be null due to the resu

[jira] [Commented] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518130#comment-16518130 ] Marco Gaido commented on SPARK-24606: - Critical and Blocker are reserved for committ

[jira] [Resolved] (SPARK-24606) Decimals multiplication and division may be null due to the result precision overflow

2018-06-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-24606. - Resolution: Duplicate > Decimals multiplication and division may be null due to the result preci

[jira] [Commented] (SPARK-23901) Data Masking Functions

2018-06-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16514687#comment-16514687 ] Marco Gaido commented on SPARK-23901: - These functions can be used as any other func

[jira] [Updated] (KNOX-1358) Create new version definition for SHS

2018-06-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated KNOX-1358: -- Attachment: KNOX-1358.patch > Create new version definition for SHS > --

[jira] [Created] (KNOX-1358) Create new version definition for SHS

2018-06-15 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1358: - Summary: Create new version definition for SHS Key: KNOX-1358 URL: https://issues.apache.org/jira/browse/KNOX-1358 Project: Apache Knox Issue Type: New Feature

[jira] [Commented] (KNOX-1353) SHS always showing link to incomplete applications

2018-06-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513503#comment-16513503 ] Marco Gaido commented on KNOX-1353: --- Sorry [~lmccay], I'll be more careful next time. Th

[jira] [Created] (SPARK-24562) Allow running same tests with multiple configs in SQLQueryTestSuite

2018-06-14 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24562: --- Summary: Allow running same tests with multiple configs in SQLQueryTestSuite Key: SPARK-24562 URL: https://issues.apache.org/jira/browse/SPARK-24562 Project: Spark

[jira] [Updated] (KNOX-1356) X-Forwarded-Context header is not rewritten everywhere for SHS

2018-06-13 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated KNOX-1356: -- Attachment: KNOX-1356.patch > X-Forwarded-Context header is not rewritten everywhere for SHS > -

[jira] [Created] (KNOX-1356) X-Forwarded-Context header is not rewritten everywhere for SHS

2018-06-13 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1356: - Summary: X-Forwarded-Context header is not rewritten everywhere for SHS Key: KNOX-1356 URL: https://issues.apache.org/jira/browse/KNOX-1356 Project: Apache Knox I

[jira] [Commented] (KNOX-1354) Revert KNOX-1160 in favor of using spark.ui.proxyBase to solve spark UI issues

2018-06-13 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510906#comment-16510906 ] Marco Gaido commented on KNOX-1354: --- this LGTM. What do you think [~smore] [~lmccay]? >

[jira] [Updated] (SPARK-23931) High-order function: array_zip(array1, array2[, ...]) → array

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-23931: Description: Ref: https://prestodb.io/docs/current/functions/array.html Merges the given arrays,

[jira] [Commented] (SPARK-23931) High-order function: array_zip(array1, array2[, ...]) → array

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510079#comment-16510079 ] Marco Gaido commented on SPARK-23931: - I just edited the title/description in order

[jira] [Updated] (SPARK-23931) High-order function: array_zip(array1, array2[, ...]) → array

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-23931: Summary: High-order function: array_zip(array1, array2[, ...]) → array (was: High-order function:

[jira] [Created] (SPARK-24531) HiveExternalCatalogVersionsSuite failing due to missing 2.2.0 version

2018-06-12 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24531: --- Summary: HiveExternalCatalogVersionsSuite failing due to missing 2.2.0 version Key: SPARK-24531 URL: https://issues.apache.org/jira/browse/SPARK-24531 Project: Spark

[jira] [Resolved] (SPARK-24481) GeneratedIteratorForCodegenStage1 grows beyond 64 KB

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-24481. - Resolution: Not A Problem I am resolving this as no further action can be taken IMHO (other than

[jira] [Comment Edited] (SPARK-24481) GeneratedIteratorForCodegenStage1 grows beyond 64 KB

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509662#comment-16509662 ] Marco Gaido edited comment on SPARK-24481 at 6/12/18 2:03 PM:

[jira] [Commented] (KNOX-1160) Bug fixes to make spark history UI work

2018-06-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509391#comment-16509391 ] Marco Gaido commented on KNOX-1160: --- Thanks for your answer [~weihan]. Yes, sure. Let's

[jira] [Created] (KNOX-1353) SHS always showing link to incomplete applications

2018-06-11 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1353: - Summary: SHS always showing link to incomplete applications Key: KNOX-1353 URL: https://issues.apache.org/jira/browse/KNOX-1353 Project: Apache Knox Issue Type: Bu

[jira] [Updated] (KNOX-1353) SHS always showing link to incomplete applications

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated KNOX-1353: -- Attachment: KNOX-1353.patch > SHS always showing link to incomplete applications > -

[jira] [Commented] (KNOX-1160) Bug fixes to make spark history UI work

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508225#comment-16508225 ] Marco Gaido commented on KNOX-1160: --- [~lmccay] This patch breaks versions less than 2.4.

[jira] [Commented] (SPARK-24509) Spark WebUI [security] - Web Server Version Disclosure

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508032#comment-16508032 ] Marco Gaido commented on SPARK-24509: - I see the point, but Spark is open source, so

[jira] [Commented] (KNOX-1160) Bug fixes to make spark history UI work

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508020#comment-16508020 ] Marco Gaido commented on KNOX-1160: --- [~smore] this patch breaks the SHS UI (old or new d

[jira] [Commented] (SPARK-24510) Spark WebUI filters use Basic Authentication [security]

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507976#comment-16507976 ] Marco Gaido commented on SPARK-24510: - I am not sure this is a real issue. You can c

[jira] [Created] (KNOX-1351) Regression on SHS UI caused by KNOX-1160

2018-06-11 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1351: - Summary: Regression on SHS UI caused by KNOX-1160 Key: KNOX-1351 URL: https://issues.apache.org/jira/browse/KNOX-1351 Project: Apache Knox Issue Type: Bug Affe

[jira] [Commented] (KNOX-1160) Bug fixes to make spark history UI work

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507819#comment-16507819 ] Marco Gaido commented on KNOX-1160: --- Looks like I cannot reopen. Than I'll create a new

[jira] [Commented] (KNOX-1160) Bug fixes to make spark history UI work

2018-06-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507817#comment-16507817 ] Marco Gaido commented on KNOX-1160: --- I do think this is a bad patch. This patch assumes

[jira] [Commented] (SPARK-24481) GeneratedIteratorForCodegenStage1 grows beyond 64 KB

2018-06-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506183#comment-16506183 ] Marco Gaido commented on SPARK-24481: - Yes, because before SPARK-22520 code generati

[jira] [Updated] (KNOX-1347) X-Forwarded-Context header is not rewritten correctly

2018-06-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated KNOX-1347: -- Attachment: KNOX-1347.patch > X-Forwarded-Context header is not rewritten correctly > --

[jira] [Created] (KNOX-1347) X-Forwarded-Context header is not rewritten correctly

2018-06-08 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1347: - Summary: X-Forwarded-Context header is not rewritten correctly Key: KNOX-1347 URL: https://issues.apache.org/jira/browse/KNOX-1347 Project: Apache Knox Issue Type:

[jira] [Commented] (SPARK-24481) GeneratedIteratorForCodegenStage1 grows beyond 64 KB

2018-06-07 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504828#comment-16504828 ] Marco Gaido commented on SPARK-24481: - Thanks for reporting this. I am investigating

[jira] [Commented] (SPARK-24468) DecimalType `adjustPrecisionScale` might fail when scale is negative

2018-06-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16501655#comment-16501655 ] Marco Gaido commented on SPARK-24468: - Thanks for reporting this. I will submit soon

[jira] [Created] (NIFI-5262) ListFile should retrieve file attributes only once

2018-06-04 Thread Marco Gaido (JIRA)
Marco Gaido created NIFI-5262: - Summary: ListFile should retrieve file attributes only once Key: NIFI-5262 URL: https://issues.apache.org/jira/browse/NIFI-5262 Project: Apache NiFi Issue Type: Im

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-06-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498201#comment-16498201 ] Marco Gaido commented on SPARK-24437: - I checked and it seems that the leakage you a

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-06-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498072#comment-16498072 ] Marco Gaido commented on SPARK-24437: - [~tanejagagan] the map you are referring seem

[jira] [Created] (YARN-8385) Clean local directories when a container is killed

2018-05-31 Thread Marco Gaido (JIRA)
Marco Gaido created YARN-8385: - Summary: Clean local directories when a container is killed Key: YARN-8385 URL: https://issues.apache.org/jira/browse/YARN-8385 Project: Hadoop YARN Issue Type: Bu

[jira] [Created] (YARN-8385) Clean local directories when a container is killed

2018-05-31 Thread Marco Gaido (JIRA)
Marco Gaido created YARN-8385: - Summary: Clean local directories when a container is killed Key: YARN-8385 URL: https://issues.apache.org/jira/browse/YARN-8385 Project: Hadoop YARN Issue Type: Bu

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-05-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496910#comment-16496910 ] Marco Gaido commented on SPARK-24437: - Reproducing the issue is quite easy: you just

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-05-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496598#comment-16496598 ] Marco Gaido commented on SPARK-24437: - Do you have dynamic allocation enabled? > Me

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-05-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496585#comment-16496585 ] Marco Gaido commented on SPARK-24437: - I just remembered that I started working on t

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-05-31 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496331#comment-16496331 ] Marco Gaido commented on SPARK-24437: - I remember another JIRA about this. Anyway, t

[jira] [Commented] (SPARK-24395) Fix Behavior of NOT IN with Literals Containing NULL

2018-05-30 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495068#comment-16495068 ] Marco Gaido commented on SPARK-24395: - The main issue here is that {{(null, null) =

[jira] [Commented] (SPARK-24373) "df.cache() df.count()" no longer eagerly caches data when the analyzed plans are different after re-analyzing the plans

2018-05-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493548#comment-16493548 ] Marco Gaido commented on SPARK-24373: - [~wbzhao] as I answered on the PR, the fix is

[jira] [Commented] (SPARK-24401) Aggreate on Decimal Types does not work

2018-05-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16492781#comment-16492781 ] Marco Gaido commented on SPARK-24401: - I followed the repro steps using the file you

[jira] [Resolved] (SPARK-24389) describe() can't work on column that name contain dots

2018-05-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-24389. - Resolution: Cannot Reproduce > describe() can't work on column that name contain dots > -

[jira] [Commented] (SPARK-24373) "df.cache() df.count()" no longer eagerly caches data when the analyzed plans are different after re-analyzing the plans

2018-05-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491271#comment-16491271 ] Marco Gaido commented on SPARK-24373: - [~smilegator] yes, you're right, the impact wo

[jira] [Commented] (SPARK-24373) "df.cache() df.count()" no longer eagerly caches data

2018-05-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491124#comment-16491124 ] Marco Gaido commented on SPARK-24373: - [~smilegator] I think an eager API is not rela

[jira] [Commented] (SPARK-24389) describe() can't work on column that name contain dots

2018-05-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490797#comment-16490797 ] Marco Gaido commented on SPARK-24389: - I cannot reproduce on current master. Probably

[jira] [Commented] (SPARK-24373) "df.cache() df.count()" no longer eagerly caches data

2018-05-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490551#comment-16490551 ] Marco Gaido commented on SPARK-24373: - [~wbzhao] yes, I do agree with you. That is th

[jira] [Commented] (SPARK-24341) Codegen compile error from predicate subquery

2018-05-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483846#comment-16483846 ] Marco Gaido commented on SPARK-24341: - This is an issue in the Optimizer, rather than

[jira] [Commented] (SPARK-24344) Spark SQL Thrift Server issue

2018-05-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483829#comment-16483829 ] Marco Gaido commented on SPARK-24344: - I moved to Major as Critical and Blocker are r

[jira] [Updated] (SPARK-24344) Spark SQL Thrift Server issue

2018-05-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24344: Priority: Major (was: Blocker) > Spark SQL Thrift Server issue > - > >

[jira] [Resolved] (SPARK-24315) Multiple streaming jobs detected error causing job failure

2018-05-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-24315. - Resolution: Not A Bug > Multiple streaming jobs detected error causing job failure >

[jira] [Created] (NIFI-5218) TestProcessorLifecycle.validateProcessorCanBeStoppedWhenOnTriggerThrowsException is flaky

2018-05-20 Thread Marco Gaido (JIRA)
Marco Gaido created NIFI-5218: - Summary: TestProcessorLifecycle.validateProcessorCanBeStoppedWhenOnTriggerThrowsException is flaky Key: NIFI-5218 URL: https://issues.apache.org/jira/browse/NIFI-5218 Proj

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Description: Several functions working on collection return incorrect result for complex data types

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Summary: Collection functions interpreted execution doesn't work with complex types (was: array_co

[jira] [Created] (SPARK-24315) Multiple streaming jobs detected error causing job failure

2018-05-18 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24315: --- Summary: Multiple streaming jobs detected error causing job failure Key: SPARK-24315 URL: https://issues.apache.org/jira/browse/SPARK-24315 Project: Spark Issu

[jira] [Updated] (SPARK-24313) array_contains/array_position interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Description: The functions {{array_contains}} and {{array_position}} return incorrect result for co

[jira] [Updated] (SPARK-24313) array_contains/array_position interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Summary: array_contains/array_position interpreted execution doesn't work with complex types (was:

[jira] [Created] (SPARK-24313) array_contains interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24313: --- Summary: array_contains interpreted execution doesn't work with complex types Key: SPARK-24313 URL: https://issues.apache.org/jira/browse/SPARK-24313 Project: Spark

[jira] [Commented] (SPARK-24298) PCAModel Memory in Pipeline

2018-05-17 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478802#comment-16478802 ] Marco Gaido commented on SPARK-24298: - May you please provide a small program/simple

[jira] [Updated] (SPARK-23778) SparkContext.emptyRDD confuses SparkContext.union

2018-05-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-23778: Priority: Trivial (was: Minor) > SparkContext.emptyRDD confuses SparkContext.union > -

[jira] [Commented] (SPARK-24260) Support for multi-statement SQL in SparkSession.sql API

2018-05-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475678#comment-16475678 ] Marco Gaido commented on SPARK-24260: - I don't think it is a good idea. Ho can you re

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474739#comment-16474739 ] Marco Gaido commented on NIFI-5191: --- Yes, exactly. So I think we have to revert the fix w

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474346#comment-16474346 ] Marco Gaido commented on NIFI-5191: --- It is a directory mounted with CIFS with some files

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474334#comment-16474334 ] Marco Gaido commented on NIFI-5191: --- [~ottobackwards] I improved the description, hope th

[jira] [Updated] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated NIFI-5191: -- Description: NIFI-4631 was done to improve performances. Unfortunately, it relies on the NIO API which h

[jira] [Commented] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/NIFI-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474312#comment-16474312 ] Marco Gaido commented on NIFI-5191: --- [~ottobackwards] it is not listing the files in the

[jira] [Created] (NIFI-5191) Revert NIFI-4631 since it causes issues with CIFS

2018-05-14 Thread Marco Gaido (JIRA)
Marco Gaido created NIFI-5191: - Summary: Revert NIFI-4631 since it causes issues with CIFS Key: NIFI-5191 URL: https://issues.apache.org/jira/browse/NIFI-5191 Project: Apache NiFi Issue Type: Bug

[jira] [Created] (SPARK-24268) DataType in error messages are not coherent

2018-05-14 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24268: --- Summary: DataType in error messages are not coherent Key: SPARK-24268 URL: https://issues.apache.org/jira/browse/SPARK-24268 Project: Spark Issue Type: Improve

[jira] [Updated] (KNOX-1274) SSO redirect for Spark2 History Server needs to be rewritten

2018-05-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated KNOX-1274: -- Attachment: KNOX-1274.patch Status: Patch Available (was: Reopened) thanks [~smore]. I uploaded

[jira] [Commented] (KNOX-1274) SSO redirect for Spark2 History Server needs to be rewritten

2018-05-11 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471693#comment-16471693 ] Marco Gaido commented on KNOX-1274: --- Hi [~smore]. Thanks for your kind answer and your wo

[jira] [Created] (SPARK-24209) 0 configuration Knox gateway support in SHS

2018-05-08 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24209: --- Summary: 0 configuration Knox gateway support in SHS Key: SPARK-24209 URL: https://issues.apache.org/jira/browse/SPARK-24209 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24189) Spark Strcutured Streaming not working with the Kafka Transactions

2018-05-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464798#comment-16464798 ] Marco Gaido commented on SPARK-24189: - You need to add also: {code} org.apache.kafk

[jira] [Commented] (SPARK-24189) Spark Strcutured Streaming not working with the Kafka Transactions

2018-05-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464788#comment-16464788 ] Marco Gaido commented on SPARK-24189: - [~abharath9] how are you submitting this appli

[jira] [Commented] (SPARK-24189) Spark Strcutured Streaming not working with the Kafka Transactions

2018-05-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464700#comment-16464700 ] Marco Gaido commented on SPARK-24189: - all kafka option should be set with the prefix

[jira] [Created] (KNOX-1295) X-Forwarded-Context contains not the full base path in SHS request

2018-05-04 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1295: - Summary: X-Forwarded-Context contains not the full base path in SHS request Key: KNOX-1295 URL: https://issues.apache.org/jira/browse/KNOX-1295 Project: Apache Knox

[jira] [Commented] (SPARK-24088) only HadoopRDD leverage HDFS Cache as preferred location

2018-05-04 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463653#comment-16463653 ] Marco Gaido commented on SPARK-24088: - [~xiaojuwu] I don't understand which problem i

[jira] [Commented] (SPARK-24177) Spark returning inconsistent rows and data in a join query when run using Spark SQL (using SQLContext.sql(...))

2018-05-04 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463616#comment-16463616 ] Marco Gaido commented on SPARK-24177: - [~ajay_monga] please may you try with a higher

[jira] [Commented] (SPARK-24170) [Spark SQL] json file format is not dropped after dropping table

2018-05-03 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462545#comment-16462545 ] Marco Gaido commented on SPARK-24170: - This is true for every datasource. This is the

[jira] [Created] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-02 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24149: --- Summary: Automatic namespaces discovery in HDFS federation Key: SPARK-24149 URL: https://issues.apache.org/jira/browse/SPARK-24149 Project: Spark Issue Type: I

[jira] [Comment Edited] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

2018-04-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456230#comment-16456230 ] Marco Gaido edited comment on SPARK-24089 at 4/27/18 11:01 AM:

[jira] [Commented] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

2018-04-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456230#comment-16456230 ] Marco Gaido commented on SPARK-24089: - [~rkrgarlapati] the problem is that you are no

[jira] [Commented] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

2018-04-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452942#comment-16452942 ] Marco Gaido commented on SPARK-24089: - Anyway, for what I can see from your post on s

[jira] [Commented] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

2018-04-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452937#comment-16452937 ] Marco Gaido commented on SPARK-24089: - Blocker can be set only by commiters, I moved

[jira] [Updated] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

2018-04-25 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24089: Priority: Critical (was: Blocker) > DataFrame.write().mode(SaveMode.Append).insertInto(TABLE) > -

[jira] [Commented] (SPARK-24051) Incorrect results for certain queries using Java and Python APIs on Spark 2.3.0

2018-04-24 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451241#comment-16451241 ] Marco Gaido commented on SPARK-24051: - [~hvanhovell] I am not sure that the analysis

[jira] [Commented] (KNOX-1274) SSO redirect for Spark2 History Server needs to be rewritten

2018-04-24 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/KNOX-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450235#comment-16450235 ] Marco Gaido commented on KNOX-1274: --- Yes [~lmc...@apache.org], this is what I am doing. S

[jira] [Created] (KNOX-1274) SSO redirect for Spark2 History Server needs to be rewritten

2018-04-24 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1274: - Summary: SSO redirect for Spark2 History Server needs to be rewritten Key: KNOX-1274 URL: https://issues.apache.org/jira/browse/KNOX-1274 Project: Apache Knox Iss

[jira] [Created] (KNOX-1273) New service for Spark2

2018-04-24 Thread Marco Gaido (JIRA)
Marco Gaido created KNOX-1273: - Summary: New service for Spark2 Key: KNOX-1273 URL: https://issues.apache.org/jira/browse/KNOX-1273 Project: Apache Knox Issue Type: New Feature Report

[jira] [Commented] (SPARK-24051) Incorrect results for certain queries using Java and Python APIs on Spark 2.3.0

2018-04-24 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449670#comment-16449670 ] Marco Gaido commented on SPARK-24051: - I was able to reproduce the issue. It is due t

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448365#comment-16448365 ] Marco Gaido commented on SPARK-24013: - [~juliuszsompolski] I have been able to reprod

[jira] [Commented] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448058#comment-16448058 ] Marco Gaido commented on SPARK-24013: - I cannot reproduce on current master. For me i

[jira] [Commented] (SPARK-24009) spark2.3.0 INSERT OVERWRITE LOCAL DIRECTORY '/home/spark/aaaaab'

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448056#comment-16448056 ] Marco Gaido commented on SPARK-24009: - I have not been able to reproduce. > spark2.3

[jira] [Commented] (SPARK-23901) Data Masking Functions

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447916#comment-16447916 ] Marco Gaido commented on SPARK-23901: - [~smilegator] [~ueshin] any comment on my ques

[jira] [Commented] (SPARK-24051) Incorrect results

2018-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447784#comment-16447784 ] Marco Gaido commented on SPARK-24051: - I am not able to reproduce on current master u

<    1   2   3   4   5   6   7   8   >