- As usual the license and signatures are OK
- No blockers, check
- 9 "Critical" bugs for 1.5.0 are listed below just for everyone's
reference (48 total issues still targeted for 1.5.0)
- Under Java 7 + Ubuntu 15, I only had one consistent test failure,
but obviously it's not failing in Jenkins
- I saw more test failures with Java 8 but the seemed like flaky
tests, so am pretending I didn't see that
Test failure
DirectKafkaStreamSuite:
- offset recovery *** FAILED ***
The code passed to eventually never returned normally. Attempted 197
times over 10.012973046 seconds. Last failure message:
strings.forall({
((elem: Any) => DirectKafkaStreamSuite.collectedData.contains(elem))
}) was false. (DirectKafkaStreamSuite.scala:249)
1.5.0 Critical Bugs
SPARK-6484 Spark Core Ganglia metrics xml reporter doesn't escape
correctly Josh Rosen
SPARK-6701 Tests, YARN Flaky test: o.a.s.deploy.yarn.YarnClusterSuite
Python application
SPARK-7420 Tests Flaky test: o.a.s.streaming.JobGeneratorSuite "Do not
clear received block data too soon" Tathagata Das
SPARK-8119 Spark Core HeartbeatReceiver should not adjust application
executor resources Andrew Or
SPARK-8414 Spark Core Ensure ContextCleaner actually triggers clean
ups Andrew Or
SPARK-8447 Shuffle Test external shuffle service with all shuffle managers
SPARK-10224 Streaming BlockGenerator may lost data in the last block
SPARK-10310 SQL [Spark SQL] All result records will be popluated into
ONE line during the script transform due to missing the correct
line/filed delimeter
SPARK-10337 SQL Views are broken
On Tue, Sep 1, 2015 at 9:41 PM, Reynold Xin <r...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.0. The vote is open until Friday, Sep 4, 2015 at 21:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
>
> The tag to be voted on is v1.5.0-rc3:
> https://github.com/apache/spark/commit/908e37bcc10132bb2aa7f80ae694a9df6e40f31a
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.0-rc3-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (published as 1.5.0-rc3) can be
> found at:
> https://repository.apache.org/content/repositories/orgapachespark-1143/
>
> The staging repository for this release (published as 1.5.0) can be found
> at:
> https://repository.apache.org/content/repositories/orgapachespark-1142/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.0-rc3-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> This vote is happening towards the end of the 1.5 QA period, so -1 votes
> should only occur for significant regressions from 1.4. Bugs already present
> in 1.4, minor regressions, or bugs related to new features will not block
> this release.
>
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.0?
> ===============================================================
> 1. It is OK for documentation patches to target 1.5.0 and still go into
> branch-1.5, since documentations will be packaged separately from the
> release.
> 2. New features for non-alpha-modules should target 1.6+.
> 3. Non-blocker bug fixes should target 1.5.1 or 1.6.0, or drop the target
> version.
>
>
> ==================================================
> Major changes to help you focus your testing
> ==================================================
>
> As of today, Spark 1.5 contains more than 1000 commits from 220+
> contributors. I've curated a list of important changes for 1.5. For the
> complete list, please refer to Apache JIRA changelog.
>
> RDD/DataFrame/SQL APIs
>
> - New UDAF interface
> - DataFrame hints for broadcast join
> - expr function for turning a SQL expression into DataFrame column
> - Improved support for NaN values
> - StructType now supports ordering
> - TimestampType precision is reduced to 1us
> - 100 new built-in expressions, including date/time, string, math
> - memory and local disk only checkpointing
>
> DataFrame/SQL Backend Execution
>
> - Code generation on by default
> - Improved join, aggregation, shuffle, sorting with cache friendly
> algorithms and external algorithms
> - Improved window function performance
> - Better metrics instrumentation and reporting for DF/SQL execution plans
>
> Data Sources, Hive, Hadoop, Mesos and Cluster Management
>
> - Dynamic allocation support in all resource managers (Mesos, YARN,
> Standalone)
> - Improved Mesos support (framework authentication, roles, dynamic
> allocation, constraints)
> - Improved YARN support (dynamic allocation with preferred locations)
> - Improved Hive support (metastore partition pruning, metastore connectivity
> to 0.13 to 1.2, internal Hive upgrade to 1.2)
> - Support persisting data in Hive compatible format in metastore
> - Support data partitioning for JSON data sources
> - Parquet improvements (upgrade to 1.7, predicate pushdown, faster metadata
> discovery and schema merging, support reading non-standard legacy Parquet
> files generated by other libraries)
> - Faster and more robust dynamic partition insert
> - DataSourceRegister interface for external data sources to specify short
> names
>
> SparkR
>
> - YARN cluster mode in R
> - GLMs with R formula, binomial/Gaussian families, and elastic-net
> regularization
> - Improved error messages
> - Aliases to make DataFrame functions more R-like
>
> Streaming
>
> - Backpressure for handling bursty input streams.
> - Improved Python support for streaming sources (Kafka offsets, Kinesis,
> MQTT, Flume)
> - Improved Python streaming machine learning algorithms (K-Means, linear
> regression, logistic regression)
> - Native reliable Kinesis stream support
> - Input metadata like Kafka offsets made visible in the batch details UI
> - Better load balancing and scheduling of receivers across cluster
> - Include streaming storage in web UI
>
> Machine Learning and Advanced Analytics
>
> - Feature transformers: CountVectorizer, Discrete Cosine transformation,
> MinMaxScaler, NGram, PCA, RFormula, StopWordsRemover, and VectorSlicer.
> - Estimators under pipeline APIs: naive Bayes, k-means, and isotonic
> regression.
> - Algorithms: multilayer perceptron classifier, PrefixSpan for sequential
> pattern mining, association rule generation, 1-sample Kolmogorov-Smirnov
> test.
> - Improvements to existing algorithms: LDA, trees/ensembles, GMMs
> - More efficient Pregel API implementation for GraphX
> - Model summary for linear and logistic regression.
> - Python API: distributed matrices, streaming k-means and linear models,
> LDA, power iteration clustering, etc.
> - Tuning and evaluation: train-validation split and multiclass
> classification evaluator.
> - Documentation: document the release version of public API methods
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
