[GitHub] [spark] SparkQA commented on issue #25702: [SPARK-29001][CORE] Print events that take too long time to process

2019-09-18 Thread GitBox
SparkQA commented on issue #25702: [SPARK-29001][CORE] Print events that take too long time to process URL: https://github.com/apache/spark/pull/25702#issuecomment-532881025 **[Test build #110937 has

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532879853 > Never mind about my point if you meant starting with fresh map instead of sharing

[GitHub] [spark] HeartSaVioR commented on issue #25702: [SPARK-29001][CORE] Print events that take too long time to process

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25702: [SPARK-29001][CORE] Print events that take too long time to process URL: https://github.com/apache/spark/pull/25702#issuecomment-532879691 retest this, please This is an automated

[GitHub] [spark] HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532879327 Never mind about my point if you meant starting with fresh map instead of sharing

[GitHub] [spark] viirya commented on a change in pull request #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec

2019-09-18 Thread GitBox
viirya commented on a change in pull request #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#discussion_r325905716 ## File path:

[GitHub] [spark] viirya commented on a change in pull request #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec

2019-09-18 Thread GitBox
viirya commented on a change in pull request #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#discussion_r325905716 ## File path:

[GitHub] [spark] HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532872003 I guess we are talking based on different things, what I referred is

[GitHub] [spark] HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532872003 I guess we are talking based on different things, what I referred is

[GitHub] [spark] HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532872003 I guess we are talking based on different things, what I referred is

[GitHub] [spark] HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532872003 I guess we are talking based on different things, what I referred is SPARK-28594,

[GitHub] [spark] viirya edited a comment on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
viirya edited a comment on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826#issuecomment-532860559 > Did you ignore the merge script recommendation? The merge script will give you

[GitHub] [spark] AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532869782 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532869777 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532869782 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532869777 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
SparkQA commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532869165 **[Test build #110936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110936/testReport)** for PR 25681 at

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532869010 > additional (considerable amount of) CPU/memory consumption. Additional memory

[GitHub] [spark] zsxwing edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing edited a comment on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532869010 > additional (considerable amount of) CPU/memory consumption. Additional

[GitHub] [spark] rdblue commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
rdblue commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532868393 @brkyvz, I rebased, added Java API tests, and opened SPARK-29157 to track the PySpark API.

[GitHub] [spark] HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532867903 > I think that's a separate issue since It doesn't change how long it takes to

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532867802 By the way, I'm totally +1 on SPARK-28594. But I think without doing snapshots in

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532867328 > This seems to assume there're only "appends", but the reality is that there're also

[GitHub] [spark] SparkQA commented on issue #25670: [SPARK-28869][CORE] Roll over event log files

2019-09-18 Thread GitBox
SparkQA commented on issue #25670: [SPARK-28869][CORE] Roll over event log files URL: https://github.com/apache/spark/pull/25670#issuecomment-532866743 **[Test build #110935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110935/testReport)** for PR 25670

[GitHub] [spark] AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532866014 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532866009 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532866014 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532866009 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532800853 **[Test build #110928 has

[GitHub] [spark] SparkQA commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
SparkQA commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532865282 **[Test build #110928 has

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532864722 > I guess SPARK-28594 would be more preferred approach for streaming query in the end. I

[GitHub] [spark] SparkQA commented on issue #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec

2019-09-18 Thread GitBox
SparkQA commented on issue #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#issuecomment-532864046 **[Test build #110934 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532862279 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] HeartSaVioR commented on issue #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25830: [SPARK-29140][SQL] Handle parameters having "array" of javaType properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#issuecomment-532862764 Updated, please take a next round of review. Thanks!

[GitHub] [spark] SparkQA removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532852811 **[Test build #110932 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532862272 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532862279 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532862272 Merged build finished. Test FAILed. This

[GitHub] [spark] SparkQA commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
SparkQA commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532862249 **[Test build #110932 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532861808 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532861816 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532861816 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532861808 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API

2019-09-18 Thread GitBox
SparkQA commented on issue #25681: [SPARK-28612][SQL] Add DataFrameWriterV2 API URL: https://github.com/apache/spark/pull/25681#issuecomment-532861301 **[Test build #110933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110933/testReport)** for PR 25681 at

[GitHub] [spark] viirya commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
viirya commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826#issuecomment-532860559 > Ur, @viirya . > Did you ignore the merge script recommendation? The merge script

[GitHub] [spark] AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532859373 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532859358 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532859358 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532859373 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532859119 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532859119 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532859128 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532859128 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532858313 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] SparkQA commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
SparkQA commented on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532858226 **[Test build #110925 has

[GitHub] [spark] AmplabJenkins commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532858313 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25832: [MINOR][SS][DOCS] Adapt multiple watermark policy comment to the reality URL: https://github.com/apache/spark/pull/25832#issuecomment-532763540 **[Test build #110925 has

[GitHub] [spark] HeartSaVioR commented on issue #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#issuecomment-532858294 >> array types is more obvious than binary types? > array type is ArrayData, which doesn't

[GitHub] [spark] SparkQA removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532748724 **[Test build #110922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110922/testReport)** for

[GitHub] [spark] SparkQA commented on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-18 Thread GitBox
SparkQA commented on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-532858395 **[Test build #110922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110922/testReport)** for PR

[GitHub] [spark] AmplabJenkins commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532858302 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532858302 Merged build finished. Test FAILed.

[GitHub] [spark] SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532857595 **[Test build #110914 has

[GitHub] [spark] SparkQA removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-532698517 **[Test build #110914 has

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec

2019-09-18 Thread GitBox
HeartSaVioR commented on a change in pull request #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#discussion_r325882156 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532856353 Test FAILed. Refer to this link for build results (access rights to CI

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec

2019-09-18 Thread GitBox
HeartSaVioR commented on a change in pull request #25830: [SPARK-29140][SQL] Handle BinaryType of parameter properly in HashAggregateExec URL: https://github.com/apache/spark/pull/25830#discussion_r325881881 ## File path:

[GitHub] [spark] AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532856340 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532856353 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532856340 Merged build finished. Test FAILed.

[GitHub] [spark] SparkQA commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
SparkQA commented on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532855981 **[Test build #110930 has

[GitHub] [spark] SparkQA removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#issuecomment-532809882 **[Test build #110930 has

[GitHub] [spark] HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
HeartSaVioR commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532855117 > One of the issues I hit in the past is that it cannot render UI for a long-running

[GitHub] [spark] dongjoon-hyun edited a comment on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
dongjoon-hyun edited a comment on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826#issuecomment-532854221 Ur, @viirya . Did you ignore the merge script recommendation? The merge

[GitHub] [spark] dongjoon-hyun commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
dongjoon-hyun commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826#issuecomment-532854221 Ur, @viirya . Did you ignore the merge script recommendation? ```

[GitHub] [spark] AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532853294 Merged build finished. Test PASSed.

[GitHub] [spark] dongjoon-hyun closed pull request #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
dongjoon-hyun closed pull request #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532853300 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532853154 Merged build finished. Test FAILed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532853294 Merged build finished. Test PASSed. This

[GitHub] [spark] vanzin commented on issue #25805: [SPARK-29082][core] Skip delegation token generation if no credentials are available.

2019-09-18 Thread GitBox
vanzin commented on issue #25805: [SPARK-29082][core] Skip delegation token generation if no credentials are available. URL: https://github.com/apache/spark/pull/25805#issuecomment-532853070 Ok, no more comments, I'll merge to master.

[GitHub] [spark] AmplabJenkins commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532853158 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] vanzin closed pull request #25805: [SPARK-29082][core] Skip delegation token generation if no credentials are available.

2019-09-18 Thread GitBox
vanzin closed pull request #25805: [SPARK-29082][core] Skip delegation token generation if no credentials are available. URL: https://github.com/apache/spark/pull/25805 This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532853154 Merged build finished. Test FAILed. This is an automated message

[GitHub] [spark] dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532853227 Anyway, this Scala change looks safe to me. I'll merge this to master. Thank you, @wangyum , @srowen , @kiszk ! Please make

[GitHub] [spark] AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532853300 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-18 Thread GitBox
SparkQA commented on issue #25771: [SPARK-28970][SQL] Implement USE CATALOG/NAMESPACE for Data Source V2 URL: https://github.com/apache/spark/pull/25771#issuecomment-532852811 **[Test build #110932 has

[GitHub] [spark] dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532852922 Hmm. It's weird because it's terminated before the configuration.

[GitHub] [spark] SparkQA removed a comment on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
SparkQA removed a comment on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532691796 **[Test build #110911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110911/testReport)** for

[GitHub] [spark] SparkQA commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
SparkQA commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532852070 **[Test build #110911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110911/testReport)** for PR 25404

[GitHub] [spark] AmplabJenkins commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#issuecomment-532848025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#issuecomment-532848016 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
AmplabJenkins removed a comment on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#issuecomment-532848025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
AmplabJenkins commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#issuecomment-532848016 Merged build finished. Test PASSed. This is

[GitHub] [spark] SparkQA commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
SparkQA commented on issue #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#issuecomment-532847558 **[Test build #110931 has

[GitHub] [spark] viirya closed pull request #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
viirya closed pull request #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826 This is an automated message from the Apache

[GitHub] [spark] viirya commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-18 Thread GitBox
viirya commented on issue #25826: [SPARK-29042][Core][BRANCH-2.4] Sampling-based RDD with unordered input should be INDETERMINATE URL: https://github.com/apache/spark/pull/25826#issuecomment-532844769 Thanks! Merged to 2.4.

[GitHub] [spark] yifeih commented on a change in pull request #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API

2019-09-18 Thread GitBox
yifeih commented on a change in pull request #25823: [SPARK-28211][Core][Shuffle] Propose Shuffle Driver Components API URL: https://github.com/apache/spark/pull/25823#discussion_r325866627 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@ -524,6

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532842226 In addition, SHS needs to support multiple applications. If applications can generate

[GitHub] [spark] dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10

2019-09-18 Thread GitBox
dongjoon-hyun commented on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532842141 I checked and noticed that the PR Builder timeout `430m`. It looks safe. I was confused with the other Jenkins job configurations

[GitHub] [spark] zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer

2019-09-18 Thread GitBox
zsxwing commented on issue #25577: [WIP][CORE][SPARK-28867] InMemoryStore checkpoint to speed up replay log file in HistoryServer URL: https://github.com/apache/spark/pull/25577#issuecomment-532841153 I'm +1 on taking snapshot in driver rather than SHS. One of the issues I hit in the past

<    1   2   3   4   5   6   7   8   9   10   >