Modified: spark/site/docs/1.5.0/api/scala/index/index-a.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/index/index-a.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/index/index-a.html (original) +++ spark/site/docs/1.5.0/api/scala/index/index-a.html Wed Sep 16 20:10:48 2015 @@ -346,16 +346,13 @@ <div class="occurrences"><a href="../org/apache/spark/sql/functions$.html" class="extype" name="org.apache.spark.sql.functions">functions</a> </div> </div><div class="entry"> <div class="name">attempt</div> - <div class="occurrences"><a href="../org/apache/spark/scheduler/TaskInfo.html" class="extype" name="org.apache.spark.scheduler.TaskInfo">TaskInfo</a> <a href="../org/apache/spark/status/api/v1/TaskData.html" class="extype" name="org.apache.spark.status.api.v1.TaskData">TaskData</a> </div> - </div><div class="entry"> - <div class="name">attemptID</div> - <div class="occurrences"><a href="../org/apache/spark/TaskCommitDenied.html" class="extype" name="org.apache.spark.TaskCommitDenied">TaskCommitDenied</a> </div> + <div class="occurrences"><strike><a href="../org/apache/spark/scheduler/TaskInfo.html" class="extype" name="org.apache.spark.scheduler.TaskInfo">TaskInfo</a></strike> <a href="../org/apache/spark/status/api/v1/TaskData.html" class="extype" name="org.apache.spark.status.api.v1.TaskData">TaskData</a> </div> </div><div class="entry"> <div class="name">attemptId</div> <div class="occurrences"><strike><a href="../org/apache/spark/TaskContext.html" class="extype" name="org.apache.spark.TaskContext">TaskContext</a></strike> <a href="../org/apache/spark/scheduler/StageInfo.html" class="extype" name="org.apache.spark.scheduler.StageInfo">StageInfo</a> <a href="../org/apache/spark/status/api/v1/ApplicationAttemptInfo.html" class="extype" name="org.apache.spark.status.api.v1.ApplicationAttemptInfo">ApplicationAttemptInfo</a> <a href="../org/apache/spark/status/api/v1/StageData.html" class="extype" name="org.apache.spark.status.api.v1.StageData">StageData</a> </div> </div><div class="entry"> <div class="name">attemptNumber</div> - <div class="occurrences"><a href="../org/apache/spark/TaskContext.html" class="extype" name="org.apache.spark.TaskContext">TaskContext</a> </div> + <div class="occurrences"><a href="../org/apache/spark/TaskCommitDenied.html" class="extype" name="org.apache.spark.TaskCommitDenied">TaskCommitDenied</a> <a href="../org/apache/spark/TaskContext.html" class="extype" name="org.apache.spark.TaskContext">TaskContext</a> <a href="../org/apache/spark/scheduler/TaskInfo.html" class="extype" name="org.apache.spark.scheduler.TaskInfo">TaskInfo</a> </div> </div><div class="entry"> <div class="name">attempts</div> <div class="occurrences"><a href="../org/apache/spark/status/api/v1/ApplicationInfo.html" class="extype" name="org.apache.spark.status.api.v1.ApplicationInfo">ApplicationInfo</a> </div>
Modified: spark/site/docs/1.5.0/api/scala/index/index-l.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/index/index-l.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/index/index-l.html (original) +++ spark/site/docs/1.5.0/api/scala/index/index-l.html Wed Sep 16 20:10:48 2015 @@ -261,6 +261,9 @@ <div class="name">localCheckpoint</div> <div class="occurrences"><a href="../org/apache/spark/rdd/RDD.html" class="extype" name="org.apache.spark.rdd.RDD">RDD</a> </div> </div><div class="entry"> + <div class="name">localProperties</div> + <div class="occurrences"><a href="../org/apache/spark/SparkContext.html" class="extype" name="org.apache.spark.SparkContext">SparkContext</a> </div> + </div><div class="entry"> <div class="name">localValue</div> <div class="occurrences"><a href="../org/apache/spark/Accumulable.html" class="extype" name="org.apache.spark.Accumulable">Accumulable</a> </div> </div><div class="entry"> Modified: spark/site/docs/1.5.0/api/scala/org/apache/spark/SparkContext.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/org/apache/spark/SparkContext.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/org/apache/spark/SparkContext.html (original) +++ spark/site/docs/1.5.0/api/scala/org/apache/spark/SparkContext.html Wed Sep 16 20:10:48 2015 @@ -1113,6 +1113,19 @@ through this method with new ones, it sh <span class="name">@<a href="annotation/DeveloperApi.html" class="extype" name="org.apache.spark.annotation.DeveloperApi">DeveloperApi</a></span><span class="args">()</span> </dd></dl></div> + </li><li name="org.apache.spark.SparkContext#localProperties" visbl="prt" data-isabs="false" fullComment="yes" group="Ungrouped"> + <a id="localProperties:InheritableThreadLocal[java.util.Properties]"></a> + <a id="localProperties:InheritableThreadLocal[Properties]"></a> + <h4 class="signature"> + <span class="modifier_kind"> + <span class="modifier"></span> + <span class="kind">val</span> + </span> + <span class="symbol"> + <span class="name">localProperties</span><span class="result">: <span class="extype" name="java.lang.InheritableThreadLocal">InheritableThreadLocal</span>[<span class="extype" name="java.util.Properties">Properties</span>]</span> + </span> + </h4> + <div class="fullcomment"><dl class="attributes block"> <dt>Attributes</dt><dd>protected[<a href="package.html" class="extype" name="org.apache.spark">org.apache.spark</a>] </dd></dl></div> </li><li name="org.apache.spark.Logging#log" visbl="prt" data-isabs="false" fullComment="yes" group="Ungrouped"> <a id="log:org.slf4j.Logger"></a> <a id="log:Logger"></a> Modified: spark/site/docs/1.5.0/api/scala/org/apache/spark/TaskCommitDenied.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/org/apache/spark/TaskCommitDenied.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/org/apache/spark/TaskCommitDenied.html (original) +++ spark/site/docs/1.5.0/api/scala/org/apache/spark/TaskCommitDenied.html Wed Sep 16 20:10:48 2015 @@ -35,7 +35,7 @@ <span class="kind">case class</span> </span> <span class="symbol"> - <span class="name">TaskCommitDenied</span><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptID">attemptID: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result"> extends <a href="TaskFailedReason.html" class="extype" name="org.apache.spark.TaskFailedReason">TaskFailedReason</a> with <span class="extype" name="scala.Product">Product</span> with <span class="extype" name="scala.Serializable">Serializable</span></span> + <span class="name">TaskCommitDenied</span><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptNumber">attemptNumber: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result"> extends <a href="TaskFailedReason.html" class="extype" name="org.apache.spark.TaskFailedReason">TaskFailedReason</a> with <span class="extype" name="scala.Product">Product</span> with <span class="extype" name="scala.Serializable">Serializable</span></span> </span> </h4> @@ -85,7 +85,7 @@ Task requested the driver to commit, but <div id="constructors" class="members"> <h3>Instance Constructors</h3> <ol><li name="org.apache.spark.TaskCommitDenied#<init>" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> - <a id="<init>(jobID:Int,partitionID:Int,attemptID:Int):org.apache.spark.TaskCommitDenied"></a> + <a id="<init>(jobID:Int,partitionID:Int,attemptNumber:Int):org.apache.spark.TaskCommitDenied"></a> <a id="<init>:TaskCommitDenied"></a> <h4 class="signature"> <span class="modifier_kind"> @@ -93,7 +93,7 @@ Task requested the driver to commit, but <span class="kind">new</span> </span> <span class="symbol"> - <span class="name">TaskCommitDenied</span><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptID">attemptID: <span class="extype" name="scala.Int">Int</span></span>)</span> + <span class="name">TaskCommitDenied</span><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptNumber">attemptNumber: <span class="extype" name="scala.Int">Int</span></span>)</span> </span> </h4> @@ -184,16 +184,16 @@ Task requested the driver to commit, but </span> </h4> <div class="fullcomment"><dl class="attributes block"> <dt>Definition Classes</dt><dd>Any</dd></dl></div> - </li><li name="org.apache.spark.TaskCommitDenied#attemptID" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> - <a id="attemptID:Int"></a> - <a id="attemptID:Int"></a> + </li><li name="org.apache.spark.TaskCommitDenied#attemptNumber" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> + <a id="attemptNumber:Int"></a> + <a id="attemptNumber:Int"></a> <h4 class="signature"> <span class="modifier_kind"> <span class="modifier"></span> <span class="kind">val</span> </span> <span class="symbol"> - <span class="name">attemptID</span><span class="result">: <span class="extype" name="scala.Int">Int</span></span> + <span class="name">attemptNumber</span><span class="result">: <span class="extype" name="scala.Int">Int</span></span> </span> </h4> Modified: spark/site/docs/1.5.0/api/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil$.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil%24.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil$.html (original) +++ spark/site/docs/1.5.0/api/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil$.html Wed Sep 16 20:10:48 2015 @@ -182,29 +182,16 @@ </span>)</span> </dd></dl></div> - </li><li name="org.apache.spark.mapred.SparkHadoopMapRedUtil#commitTask" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> - <a id="commitTask(committer:org.apache.hadoop.mapreduce.OutputCommitter,mrTaskContext:org.apache.hadoop.mapreduce.TaskAttemptContext,sparkTaskContext:org.apache.spark.TaskContext):Unit"></a> - <a id="commitTask(OutputCommitter,TaskAttemptContext,TaskContext):Unit"></a> - <h4 class="signature"> - <span class="modifier_kind"> - <span class="modifier"></span> - <span class="kind">def</span> - </span> - <span class="symbol"> - <span class="name">commitTask</span><span class="params">(<span name="committer">committer: <span class="extype" name="org.apache.hadoop.mapreduce.OutputCommitter">OutputCommitter</span></span>, <span name="mrTaskContext">mrTaskContext: <span class="extype" name="org.apache.hadoop.mapreduce.TaskAttemptContext">TaskAttemptContext</span></span>, <span name="sparkTaskContext">sparkTaskContext: <a href="../TaskContext.html" class="extype" name="org.apache.spark.TaskContext">TaskContext</a></span>)</span><span class="result">: <span class="extype" name="scala.Unit">Unit</span></span> - </span> - </h4> - </li><li name="org.apache.spark.mapred.SparkHadoopMapRedUtil#commitTask" visbl="pub" data-isabs="false" fullComment="yes" group="Ungrouped"> - <a id="commitTask(committer:org.apache.hadoop.mapreduce.OutputCommitter,mrTaskContext:org.apache.hadoop.mapreduce.TaskAttemptContext,jobId:Int,splitId:Int,attemptId:Int):Unit"></a> - <a id="commitTask(OutputCommitter,TaskAttemptContext,Int,Int,Int):Unit"></a> + <a id="commitTask(committer:org.apache.hadoop.mapreduce.OutputCommitter,mrTaskContext:org.apache.hadoop.mapreduce.TaskAttemptContext,jobId:Int,splitId:Int):Unit"></a> + <a id="commitTask(OutputCommitter,TaskAttemptContext,Int,Int):Unit"></a> <h4 class="signature"> <span class="modifier_kind"> <span class="modifier"></span> <span class="kind">def</span> </span> <span class="symbol"> - <span class="name">commitTask</span><span class="params">(<span name="committer">committer: <span class="extype" name="org.apache.hadoop.mapreduce.OutputCommitter">OutputCommitter</span></span>, <span name="mrTaskContext">mrTaskContext: <span class="extype" name="org.apache.hadoop.mapreduce.TaskAttemptContext">TaskAttemptContext</span></span>, <span name="jobId">jobId: <span class="extype" name="scala.Int">Int</span></span>, <span name="splitId">splitId: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptId">attemptId: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <span class="extype" name="scala.Unit">Unit</span></span> + <span class="name">commitTask</span><span class="params">(<span name="committer">committer: <span class="extype" name="org.apache.hadoop.mapreduce.OutputCommitter">OutputCommitter</span></span>, <span name="mrTaskContext">mrTaskContext: <span class="extype" name="org.apache.hadoop.mapreduce.TaskAttemptContext">TaskAttemptContext</span></span>, <span name="jobId">jobId: <span class="extype" name="scala.Int">Int</span></span>, <span name="splitId">splitId: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <span class="extype" name="scala.Unit">Unit</span></span> </span> </h4> <p class="shortcomment cmt">Commits a task output.</p><div class="fullcomment"><div class="comment cmt"><p>Commits a task output. Before committing the task output, we need to know whether some other Modified: spark/site/docs/1.5.0/api/scala/org/apache/spark/package.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/org/apache/spark/package.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/org/apache/spark/package.html (original) +++ spark/site/docs/1.5.0/api/scala/org/apache/spark/package.html Wed Sep 16 20:10:48 2015 @@ -554,7 +554,7 @@ including the serializer, Akka actor sys <span class="kind">case class</span> </span> <span class="symbol"> - <a href="TaskCommitDenied.html"><span class="name">TaskCommitDenied</span></a><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptID">attemptID: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result"> extends <a href="TaskFailedReason.html" class="extype" name="org.apache.spark.TaskFailedReason">TaskFailedReason</a> with <span class="extype" name="scala.Product">Product</span> with <span class="extype" name="scala.Serializable">Serializable</span></span> + <a href="TaskCommitDenied.html"><span class="name">TaskCommitDenied</span></a><span class="params">(<span name="jobID">jobID: <span class="extype" name="scala.Int">Int</span></span>, <span name="partitionID">partitionID: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptNumber">attemptNumber: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result"> extends <a href="TaskFailedReason.html" class="extype" name="org.apache.spark.TaskFailedReason">TaskFailedReason</a> with <span class="extype" name="scala.Product">Product</span> with <span class="extype" name="scala.Serializable">Serializable</span></span> </span> </h4> <p class="comment cmt">:: DeveloperApi :: Modified: spark/site/docs/1.5.0/api/scala/org/apache/spark/scheduler/TaskInfo.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/api/scala/org/apache/spark/scheduler/TaskInfo.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/api/scala/org/apache/spark/scheduler/TaskInfo.html (original) +++ spark/site/docs/1.5.0/api/scala/org/apache/spark/scheduler/TaskInfo.html Wed Sep 16 20:10:48 2015 @@ -85,7 +85,7 @@ Information about a running task attempt <div id="constructors" class="members"> <h3>Instance Constructors</h3> <ol><li name="org.apache.spark.scheduler.TaskInfo#<init>" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> - <a id="<init>(taskId:Long,index:Int,attempt:Int,launchTime:Long,executorId:String,host:String,taskLocality:org.apache.spark.scheduler.TaskLocality.TaskLocality,speculative:Boolean):org.apache.spark.scheduler.TaskInfo"></a> + <a id="<init>(taskId:Long,index:Int,attemptNumber:Int,launchTime:Long,executorId:String,host:String,taskLocality:org.apache.spark.scheduler.TaskLocality.TaskLocality,speculative:Boolean):org.apache.spark.scheduler.TaskInfo"></a> <a id="<init>:TaskInfo"></a> <h4 class="signature"> <span class="modifier_kind"> @@ -93,7 +93,7 @@ Information about a running task attempt <span class="kind">new</span> </span> <span class="symbol"> - <span class="name">TaskInfo</span><span class="params">(<span name="taskId">taskId: <span class="extype" name="scala.Long">Long</span></span>, <span name="index">index: <span class="extype" name="scala.Int">Int</span></span>, <span name="attempt">attempt: <span class="extype" name="scala.Int">Int</span></span>, <span name="launchTime">launchTime: <span class="extype" name="scala.Long">Long</span></span>, <span name="executorId">executorId: <span class="extype" name="scala.Predef.String">String</span></span>, <span name="host">host: <span class="extype" name="scala.Predef.String">String</span></span>, <span name="taskLocality">taskLocality: <a href="TaskLocality$.html#TaskLocality=org.apache.spark.scheduler.TaskLocality.Value" class="extmbr" name="org.apache.spark.scheduler.TaskLocality.TaskLocality">TaskLocality.TaskLocality</a></span>, <span name="speculative">speculative: <span class="extype" name="scala.Boolean">Boolean</span></span>)</span> + <span class="name">TaskInfo</span><span class="params">(<span name="taskId">taskId: <span class="extype" name="scala.Long">Long</span></span>, <span name="index">index: <span class="extype" name="scala.Int">Int</span></span>, <span name="attemptNumber">attemptNumber: <span class="extype" name="scala.Int">Int</span></span>, <span name="launchTime">launchTime: <span class="extype" name="scala.Long">Long</span></span>, <span name="executorId">executorId: <span class="extype" name="scala.Predef.String">String</span></span>, <span name="host">host: <span class="extype" name="scala.Predef.String">String</span></span>, <span name="taskLocality">taskLocality: <a href="TaskLocality$.html#TaskLocality=org.apache.spark.scheduler.TaskLocality.Value" class="extmbr" name="org.apache.spark.scheduler.TaskLocality.TaskLocality">TaskLocality.TaskLocality</a></span>, <span name="speculative">speculative: <span class="extype" name="scala.Boolean">Boolean</span></span>)</span> </span> </h4> @@ -200,16 +200,16 @@ same name but different IDs to exist in </span> </h4> <div class="fullcomment"><dl class="attributes block"> <dt>Definition Classes</dt><dd>Any</dd></dl></div> - </li><li name="org.apache.spark.scheduler.TaskInfo#attempt" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> - <a id="attempt:Int"></a> - <a id="attempt:Int"></a> + </li><li name="org.apache.spark.scheduler.TaskInfo#attemptNumber" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped"> + <a id="attemptNumber:Int"></a> + <a id="attemptNumber:Int"></a> <h4 class="signature"> <span class="modifier_kind"> <span class="modifier"></span> <span class="kind">val</span> </span> <span class="symbol"> - <span class="name">attempt</span><span class="result">: <span class="extype" name="scala.Int">Int</span></span> + <span class="name">attemptNumber</span><span class="result">: <span class="extype" name="scala.Int">Int</span></span> </span> </h4> @@ -668,7 +668,26 @@ IndirectTaskResult and later fetching th - + <div id="values" class="values members"> + <h3>Deprecated Value Members</h3> + <ol><li name="org.apache.spark.scheduler.TaskInfo#attempt" visbl="pub" data-isabs="false" fullComment="yes" group="Ungrouped"> + <a id="attempt:Int"></a> + <a id="attempt:Int"></a> + <h4 class="signature"> + <span class="modifier_kind"> + <span class="modifier"></span> + <span class="kind">def</span> + </span> + <span class="symbol"> + <span class="name deprecated" title="Deprecated: (Since version 1.6.0) Use attemptNumber">attempt</span><span class="result">: <span class="extype" name="scala.Int">Int</span></span> + </span> + </h4> + <div class="fullcomment"><dl class="attributes block"> <dt>Annotations</dt><dd> + <span class="name">@deprecated</span> + + </dd><dt>Deprecated</dt><dd class="cmt"><p><i>(Since version 1.6.0)</i> Use attemptNumber</p></dd></dl></div> + </li></ol> + </div> </div> <div id="inheritedMembers"> Modified: spark/site/docs/1.5.0/configuration.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/configuration.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/configuration.html (original) +++ spark/site/docs/1.5.0/configuration.html Wed Sep 16 20:10:48 2015 @@ -609,9 +609,12 @@ of the most common options to set are:</ <td><code>spark.shuffle.manager</code></td> <td>sort</td> <td> - Implementation to use for shuffling data. There are two implementations available: - <code>sort</code> and <code>hash</code>. Sort-based shuffle is more memory-efficient and is - the default option starting in 1.2. + Implementation to use for shuffling data. There are three implementations available: + <code>sort</code>, <code>hash</code> and the new (1.5+) <code>tungsten-sort</code>. + Sort-based shuffle is more memory-efficient and is the default option starting in 1.2. + Tungsten-sort is similar to the sort based shuffle, with a direct binary cache-friendly + implementation with a fall back to regular sort based shuffle if its requirements are not + met. </td> </tr> <tr> Modified: spark/site/docs/1.5.0/mllib-data-types.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/mllib-data-types.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/mllib-data-types.html (original) +++ spark/site/docs/1.5.0/mllib-data-types.html Wed Sep 16 20:10:48 2015 @@ -272,7 +272,7 @@ For multiclass classification, labels sh <span class="n">LabeledPoint</span> <span class="n">pos</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LabeledPoint</span><span class="o">(</span><span class="mf">1.0</span><span class="o">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="na">dense</span><span class="o">(</span><span class="mf">1.0</span><span class="o">,</span> <span class="mf">0.0</span><span class="o">,</span> <span class="mf">3.0</span><span class="o">));</span> <span class="c1">// Create a labeled point with a negative label and a sparse feature vector.</span> -<span class="n">LabeledPoint</span> <span class="n">neg</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LabeledPoint</span><span class="o">(</span><span class="mf">1.0</span><span class="o">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="na">sparse</span><span class="o">(</span><span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="kt">int</span><span class="o">[]</span> <span class="o">{</span><span class="mi">0</span><span class="o">,</span> <span class="mi">2</span><span class="o">},</span> <span class="k">new</span> <span class="kt">double</span><span class="o">[]</span> <span class="o">{</span><span class="mf">1.0</span><span class="o">,</span> <span class="mf">3.0</span><span class="o">}));</span></code></pre></div> +<span class="n">LabeledPoint</span> <span class="n">neg</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">LabeledPoint</span><span class="o">(</span><span class="mf">0.0</span><span class="o">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="na">sparse</span><span class="o">(</span><span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="kt">int</span><span class="o">[]</span> <span class="o">{</span><span class="mi">0</span><span class="o">,</span> <span class="mi">2</span><span class="o">},</span> <span class="k">new</span> <span class="kt">double</span><span class="o">[]</span> <span class="o">{</span><span class="mf">1.0</span><span class="o">,</span> <span class="mf">3.0</span><span class="o">}));</span></code></pre></div> </div> Modified: spark/site/docs/1.5.0/sql-programming-guide.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/sql-programming-guide.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/sql-programming-guide.html (original) +++ spark/site/docs/1.5.0/sql-programming-guide.html Wed Sep 16 20:10:48 2015 @@ -1787,7 +1787,7 @@ will compile against Hive 1.2.1 and use <tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr> <tr> <td><code>spark.sql.hive.metastore.version</code></td> - <td><code>0.13.1</code></td> + <td><code>1.2.1</code></td> <td> Version of the Hive metastore. Available options are <code>0.12.0</code> through <code>1.2.1</code>. Modified: spark/site/docs/1.5.0/submitting-applications.html URL: http://svn.apache.org/viewvc/spark/site/docs/1.5.0/submitting-applications.html?rev=1703457&r1=1703456&r2=1703457&view=diff ============================================================================== --- spark/site/docs/1.5.0/submitting-applications.html (original) +++ spark/site/docs/1.5.0/submitting-applications.html Wed Sep 16 20:10:48 2015 @@ -187,8 +187,8 @@ Mesos clusters. Currently only YARN supp and add Python <code>.zip</code>, <code>.egg</code> or <code>.py</code> files to the search path with <code>--py-files</code>.</p> <p>There are a few options available that are specific to the -<a href="#cluster-overview.html#cluster-manager-types">cluster manager</a> that is being used. -For example, with a <a href="#spark-standalone">Spark Standalone</a> cluster with <code>cluster</code> deploy mode, +<a href="cluster-overview.html#cluster-manager-types">cluster manager</a> that is being used. +For example, with a <a href="spark-standalone.html">Spark standalone cluster</a> with <code>cluster</code> deploy mode, you can also specify <code>--supervise</code> to make sure that the driver is automatically restarted if it fails with non-zero exit code. To enumerate all such options available to <code>spark-submit</code>, run it with <code>--help</code>. Here are a few examples of common options:</p> @@ -200,7 +200,7 @@ run it with <code>--help</code>. Here ar /path/to/examples.jar <span class="se">\</span> 100 -<span class="c"># Run on a Spark Standalone cluster in client deploy mode</span> +<span class="c"># Run on a Spark standalone cluster in client deploy mode</span> ./bin/spark-submit <span class="se">\</span> --class org.apache.spark.examples.SparkPi <span class="se">\</span> --master spark://207.184.161.138:7077 <span class="se">\</span> @@ -209,7 +209,7 @@ run it with <code>--help</code>. Here ar /path/to/examples.jar <span class="se">\</span> 1000 -<span class="c"># Run on a Spark Standalone cluster in cluster deploy mode with supervise</span> +<span class="c"># Run on a Spark standalone cluster in cluster deploy mode with supervise</span> ./bin/spark-submit <span class="se">\</span> --class org.apache.spark.examples.SparkPi <span class="se">\</span> --master spark://207.184.161.138:7077 <span class="se">\</span> @@ -230,7 +230,7 @@ run it with <code>--help</code>. Here ar /path/to/examples.jar <span class="se">\</span> 1000 -<span class="c"># Run a Python application on a Spark Standalone cluster</span> +<span class="c"># Run a Python application on a Spark standalone cluster</span> ./bin/spark-submit <span class="se">\</span> --master spark://207.184.161.138:7077 <span class="se">\</span> examples/src/main/python/pi.py <span class="se">\</span> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
