jsoref commented on a change in pull request #30323:
URL: https://github.com/apache/spark/pull/30323#discussion_r521049587
##########
File path: R/pkg/tests/fulltests/test_sparkSQL.R
##########
@@ -2066,7 +2066,7 @@ test_that("higher order functions", {
createDataFrame(data.frame(id = 1)),
expr("CAST(array(1.0, 2.0, -3.0, -4.0) AS array<double>) xs"),
expr("CAST(array(0.0, 3.0, 48.0) AS array<double>) ys"),
- expr("array('FAILED', 'SUCCEDED') as vs"),
+ expr("array('FAILED', 'SUCCEEDED') as vs"),
Review comment:
This worries me
##########
File path: R/CRAN_RELEASE.md
##########
@@ -25,7 +25,7 @@ To release SparkR as a package to CRAN, we would use the
`devtools` package. Ple
First, check that the `Version:` field in the `pkg/DESCRIPTION` file is
updated. Also, check for stale files not under source control.
-Note that while `run-tests.sh` runs `check-cran.sh` (which runs `R CMD
check`), it is doing so with `--no-manual --no-vignettes`, which skips a few
vignettes or PDF checks - therefore it will be preferred to run `R CMD check`
on the source package built manually before uploading a release. Also note that
for CRAN checks for pdf vignettes to success, `qpdf` tool must be there (to
install it, eg. `yum -q -y install qpdf`).
+Note that while `run-tests.sh` runs `check-cran.sh` (which runs `R CMD
check`), it is doing so with `--no-manual --no-vignettes`, which skips a few
vignettes or PDF checks - therefore it will be preferred to run `R CMD check`
on the source package built manually before uploading a release. Also note that
for CRAN checks for pdf vignettes to success, `qpdf` tool must be there (to
install it, e.g. `yum -q -y install qpdf`).
Review comment:
I'm happy to drop `e.g.` and `i.e.`
##########
File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala
##########
@@ -342,7 +342,7 @@ class WorkerSuite extends SparkFunSuite with Matchers with
BeforeAndAfter {
testWorkDirCleanupAndRemoveMetadataWithConfig(true)
}
- test("WorkdDirCleanup cleans only app dirs when" +
+ test("WorkDirCleanup cleans only app dirs when" +
Review comment:
This is suspicious
##########
File path:
core/src/main/scala/org/apache/spark/scheduler/BarrierJobAllocationFailed.scala
##########
@@ -45,10 +45,10 @@ private[spark] object BarrierJobAllocationFailed {
val ERROR_MESSAGE_RUN_BARRIER_WITH_UNSUPPORTED_RDD_CHAIN_PATTERN =
"[SPARK-24820][SPARK-24821]: Barrier execution mode does not allow the
following pattern of " +
"RDD chain within a barrier stage:\n1. Ancestor RDDs that have different
number of " +
- "partitions from the resulting RDD (eg.
union()/coalesce()/first()/take()/" +
+ "partitions from the resulting RDD (e.g.
union()/coalesce()/first()/take()/" +
Review comment:
This is some form of output change
##########
File path: dev/create-release/translate-contributors.py
##########
@@ -207,7 +207,7 @@ def generate_candidates(author, issues):
print(p)
# In interactive mode, additionally provide "custom" option and await
user response
if INTERACTIVE_MODE:
- print(" [%d] %s - Raw Github username" % (raw_index, author))
+ print(" [%d] %s - Raw GitHub username" % (raw_index, author))
Review comment:
This is user facing
##########
File path: docs/sql-ref-syntax-dml-insert-into.md
##########
@@ -100,29 +100,29 @@ SELECT * FROM students;
```sql
-- Assuming the persons table has already been created and populated.
SELECT * FROM persons;
-+-------------+-------------------------+---------+
-| name| address| ssn|
-+-------------+-------------------------+---------+
-|Dora Williams|134 Forest Ave, Melo Park|123456789|
-+-------------+-------------------------+---------+
-| Eddie Davis| 245 Market St, Milpitas|345678901|
-+-------------+-------------------------+---------+
++-------------+--------------------------+---------+
+| name| address| ssn|
++-------------+--------------------------+---------+
+|Dora Williams|134 Forest Ave, Menlo Park|123456789|
Review comment:
This address would actually exist, and the cities are all in the Silicon
Valley area, so there's no good reason to misspell this city.
##########
File path: project/MimaExcludes.scala
##########
@@ -730,7 +730,7 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment"),
// [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0
-
ProblemFilters.exclude[FinalMethodProblem]("org.apache.spark.network.util.AbstractFileRegion.transfered"),
+
ProblemFilters.exclude[FinalMethodProblem]("org.apache.spark.network.util.AbstractFileRegion.transferred"),
Review comment:
?
##########
File path:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedRleValuesReader.java
##########
@@ -486,7 +486,7 @@ public void readDoubles(
}
}
- public void readBinarys(
+ public void readBinaries(
Review comment:
this might be a frozen public api...
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
##########
@@ -422,7 +422,7 @@ class StringExpressionsSuite extends SparkFunSuite with
ExpressionEvalHelper {
checkEvaluation(SoundEx(Literal("Uhrbach")), "U612")
checkEvaluation(SoundEx(Literal("Moskowitz")), "M232")
checkEvaluation(SoundEx(Literal("Moskovitz")), "M213")
- checkEvaluation(SoundEx(Literal("relyheewsgeessg")), "R422")
+ checkEvaluation(SoundEx(Literal("reltheewsgeessg")), "R422")
Review comment:
I didn't mean to change this (will drop)
##########
File path:
common/network-common/src/test/java/org/apache/spark/network/crypto/AuthEngineSuite.java
##########
@@ -150,8 +150,8 @@ public void testEncryptedMessage() throws Exception {
ByteArrayWritableChannel channel = new
ByteArrayWritableChannel(data.length);
TransportCipher.EncryptedMessage emsg =
handler.createEncryptedMessage(buf);
- while (emsg.transfered() < emsg.count()) {
- emsg.transferTo(channel, emsg.transfered());
+ while (emsg.transferred() < emsg.count()) {
+ emsg.transferTo(channel, emsg.transferred());
Review comment:
I hope I can keep this
##########
File path:
common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
##########
@@ -303,7 +303,7 @@ public void close() {
@Override
public String toString() {
return new ToStringBuilder(this, ToStringStyle.SHORT_PREFIX_STYLE)
- .append("remoteAdress", channel.remoteAddress())
+ .append("remoteAddress", channel.remoteAddress())
Review comment:
This is probably a dangerous change
##########
File path: core/src/test/scala/org/apache/spark/rpc/netty/NettyRpcEnvSuite.scala
##########
@@ -73,7 +73,7 @@ class NettyRpcEnvSuite extends RpcEnvSuite with MockitoSugar
with TimeLimits {
val nettyEnv = env.asInstanceOf[NettyRpcEnv]
val client = mock[TransportClient]
- val senderAddress = RpcAddress("locahost", 12345)
+ val senderAddress = RpcAddress("localhost", 12345)
Review comment:
This is scary
##########
File path:
common/network-common/src/test/java/org/apache/spark/network/util/TransportFrameDecoderSuite.java
##########
@@ -98,7 +98,7 @@ public void testConsolidationPerf() throws Exception {
writtenBytes += pieceBytes;
}
logger.info("Writing 300MiB frame buf with consolidation of
threshold " + threshold
- + " took " + totalTime + " milis");
+ + " took " + totalTime + " millis");
Review comment:
This is a logging output change
##########
File path:
core/src/main/resources/org/apache/spark/ui/static/dataTables.rowsGroup.js
##########
@@ -166,7 +166,7 @@ RowsGroup.prototype = {
this._mergeColumn(newSequenceRow, (iRow-1),
columnsIndexesCopy)
},
- _toogleDirection: function(dir)
+ _toggleDirection: function(dir)
Review comment:
I try to skip resources that come from third party repos -- if this or
anything else fall into that category, please let me know.
##########
File path:
common/network-common/src/main/java/org/apache/spark/network/util/AbstractFileRegion.java
##########
@@ -24,7 +24,7 @@
@Override
@SuppressWarnings("deprecation")
- public final long transfered() {
+ public final long transferred() {
Review comment:
Oops, I should have dropped this
##########
File path: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala
##########
@@ -968,7 +968,7 @@ private[spark] object JsonProtocolSuite extends Assertions {
private val stackTrace = {
Array[StackTraceElement](
new StackTraceElement("Apollo", "Venus", "Mercury", 42),
- new StackTraceElement("Afollo", "Vemus", "Mercurry", 420),
+ new StackTraceElement("Afollo", "Vemus", "Mercury", 420),
new StackTraceElement("Ayollo", "Vesus", "Blackberry", 4200)
Review comment:
I'm not sure what the goal of this blob is (`BlackBerry` shouldn't be
written the way it is written here either). If things are intentionally
misspelled, it'd be great if there were a comment justifying the nonstandard
use.
##########
File path: dev/github_jira_sync.py
##########
@@ -157,7 +157,7 @@ def reset_pr_labels(pr_num, jira_components):
considered = considered + [pr_num]
url = pr['html_url']
- title = "[Github] Pull Request #%s (%s)" % (pr['number'],
pr['user']['login'])
+ title = "[GitHub] Pull Request #%s (%s)" % (pr['number'],
pr['user']['login'])
Review comment:
This is likely user facing (as is the block above)
##########
File path: dev/create-release/release-build.sh
##########
@@ -452,7 +452,7 @@ if [[ "$1" == "publish-release" ]]; then
if ! is_dry_run; then
nexus_upload=$NEXUS_ROOT/deployByRepositoryId/$staged_repo_id
- echo "Uplading files to $nexus_upload"
+ echo "Uploading files to $nexus_upload"
Review comment:
This is public facing for some weak definition
##########
File path: dev/appveyor-guide.md
##########
@@ -33,22 +33,22 @@ Currently, SparkR on Windows is being tested with
[AppVeyor](https://ci.appveyor
<img width="379" alt="2016-09-04 11 07 58"
src="https://cloud.githubusercontent.com/assets/6477701/18228810/2f674e5e-7299-11e6-929d-5c2dff269ddc.png";>
-- Click "Github".
+- Click "GitHub".
<img width="360" alt="2016-09-04 11 08 10"
src="https://cloud.githubusercontent.com/assets/6477701/18228811/344263a0-7299-11e6-90b7-9b1c7b6b8b01.png";>
-#### After signing up, go to profile to link Github and AppVeyor.
+#### After signing up, go to profile to link GitHub and AppVeyor.
- Click your account and then click "Profile".
<img width="204" alt="2016-09-04 11 09 43"
src="https://cloud.githubusercontent.com/assets/6477701/18228803/12a4b810-7299-11e6-9140-5cfc277297b1.png";>
-- Enable the link with GitHub via clicking "Link Github account".
+- Enable the link with GitHub via clicking "Link GitHub account".
Review comment:
I hope no one is publicly misspelling the GitHub brand.
##########
File path: core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala
##########
@@ -254,14 +254,14 @@ private[netty] class NettyRpcEnv(
val timeoutCancelable = timeoutScheduler.schedule(new Runnable {
override def run(): Unit = {
- val remoteReceAddr = if (remoteAddr == null) {
+ val remoteRecAddr = if (remoteAddr == null) {
Review comment:
This is odd
##########
File path: docs/sql-ref-syntax-dml-insert-into.md
##########
@@ -69,11 +69,11 @@ INSERT INTO students VALUES
('Amy Smith', '123 Park Ave, San Jose', 111111);
SELECT * FROM students;
-+---------+---------------------+----------+
-| name| address|student_id|
-+---------+---------------------+----------+
-|Amy Smith|123 Park Ave,San Jose| 111111|
-+---------+---------------------+----------+
++---------+----------------------+----------+
+| name| address|student_id|
++---------+----------------------+----------+
+|Amy Smith|123 Park Ave, San Jose| 111111|
Review comment:
This is odd. The `INSERT` above doesn't match the output w/o this change.
##########
File path: dev/create-release/translate-contributors.py
##########
@@ -104,12 +104,12 @@
def generate_candidates(author, issues):
candidates = []
- # First check for full name of Github user
+ # First check for full name of GitHub user
github_name = get_github_name(author, github_client)
if github_name:
- candidates.append((github_name, "Full name of Github user %s" %
author))
+ candidates.append((github_name, "Full name of GitHub user %s" %
author))
else:
- candidates.append((NOT_FOUND, "No full name found for Github user %s"
% author))
+ candidates.append((NOT_FOUND, "No full name found for GitHub user %s"
% author))
Review comment:
This is probably user facing
##########
File path: examples/src/main/python/sql/arrow.py
##########
@@ -285,7 +285,7 @@ def asof_join(l, r):
ser_to_frame_pandas_udf_example(spark)
print("Running pandas_udf example: Series to Series")
ser_to_ser_pandas_udf_example(spark)
- print("Running pandas_udf example: Iterator of Series to Iterator of
Seires")
+ print("Running pandas_udf example: Iterator of Series to Iterator of
Series")
Review comment:
??
##########
File path: docs/sql-ref-syntax-qry-select-orderby.md
##########
@@ -28,7 +28,7 @@ clause, this clause guarantees a total order in the output.
### Syntax
```sql
-ORDER BY { expression [ sort_direction | nulls_sort_oder ] [ , ... ] }
+ORDER BY { expression [ sort_direction | nulls_sort_order ] [ , ... ] }
Review comment:
This is scary
##########
File path: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java
##########
@@ -93,7 +93,7 @@
* Maximum time (in ms) to wait for a child process to connect back to the
launcher server
* when using @link{#start()}.
*/
- public static final String CHILD_CONNECTION_TIMEOUT =
"spark.launcher.childConectionTimeout";
+ public static final String CHILD_CONNECTION_TIMEOUT =
"spark.launcher.childConnectionTimeout";
Review comment:
This is a public thing...
##########
File path: licenses-binary/LICENSE-jsp-api.txt
##########
@@ -722,7 +722,7 @@ Oracle facilitates your further distribution of this
package by adding
the Classpath Exception to the necessary parts of its GPLv2 code, which
permits you to use that code in combination with other independent
modules not licensed under the GPLv2. However, note that this would
-not permit you to commingle code under an incompatible license with
+not permit you to comingle code under an incompatible license with
Review comment:
Oops. I try to avoid touching license files -- I'll drop
##########
File path: python/pyspark/context.py
##########
@@ -258,7 +258,7 @@ def _do_init(self, master, appName, sparkHome, pyFiles,
environment, batchSize,
sys.path.insert(1, filepath)
except Exception:
warnings.warn(
- "Failed to add file [%s] speficied in
'spark.submit.pyFiles' to "
+ "Failed to add file [%s] specified in
'spark.submit.pyFiles' to "
Review comment:
user facing?
##########
File path: project/MimaExcludes.scala
##########
@@ -1488,7 +1488,7 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.SparkEnv.getThreadLocal"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.rdd.RDDFunctions.treeReduce"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.rdd.RDDFunctions.treeAggregate"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.tree.configuration.Strategy.defaultStategy"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.tree.configuration.Strategy.defaultStrategy"),
Review comment:
?
##########
File path: mllib/src/main/scala/org/apache/spark/ml/feature/Selector.scala
##########
@@ -77,7 +77,7 @@ private[feature] trait SelectorParams extends Params
* @group param
*/
@Since("3.1.0")
- final val fpr = new DoubleParam(this, "fpr", "The higest p-value for
features to be kept.",
+ final val fpr = new DoubleParam(this, "fpr", "The highest p-value for
features to be kept.",
Review comment:
Is this public in some special way?
##########
File path: python/pyspark/ml/regression.py
##########
@@ -1442,7 +1442,7 @@ def setParams(self, *, featuresCol="features",
labelCol="label", predictionCol="
maxDepth=5, maxBins=32, minInstancesPerNode=1,
minInfoGain=0.0,
maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0,
checkpointInterval=10, lossType="squared", maxIter=20,
stepSize=0.1, seed=None,
- impuriy="variance", featureSubsetStrategy="all",
validationTol=0.01,
+ impurity="variance", featureSubsetStrategy="all",
validationTol=0.01,
Review comment:
api?
##########
File path:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala
##########
@@ -58,10 +58,10 @@ private[sql] class AvroDeserializer(
private lazy val decimalConversions = new DecimalConversion()
- private val dateRebaseFunc = DataSourceUtils.creteDateRebaseFuncInRead(
+ private val dateRebaseFunc = DataSourceUtils.createDateRebaseFuncInRead(
datetimeRebaseMode, "Avro")
- private val timestampRebaseFunc =
DataSourceUtils.creteTimestampRebaseFuncInRead(
+ private val timestampRebaseFunc =
DataSourceUtils.createTimestampRebaseFuncInRead(
Review comment:
I'm really uncertain about this -- maybe `crete` is a thing??
##########
File path: python/pyspark/mllib/regression.py
##########
@@ -739,7 +739,7 @@ def _validate(self, dstream):
"dstream should be a DStream object, got %s" % type(dstream))
if not self._model:
raise ValueError(
- "Model must be intialized using setInitialWeights")
+ "Model must be initialized using setInitialWeights")
Review comment:
just a note that this is en-US
##########
File path: python/pyspark/cloudpickle/cloudpickle_fast.py
##########
@@ -556,7 +556,7 @@ def dump(self, obj):
# `dispatch` attribute. Earlier versions of the protocol 5
CloudPickler
# used `CloudPickler.dispatch` as a class-level attribute storing all
# reducers implemented by cloudpickle, but the attribute name was not a
- # great choice given the meaning of `Cloudpickler.dispatch` when
+ # great choice given the meaning of `Cloudpickle.dispatch` when
Review comment:
?
##########
File path: python/pyspark/mllib/clustering.py
##########
@@ -843,7 +843,7 @@ def setInitialCenters(self, centers, weights):
@since('1.5.0')
def setRandomCenters(self, dim, weight, seed):
"""
- Set the initial centres to be random samples from
+ Set the initial centers to be random samples from
Review comment:
if this is an en-GB thing, I'll drop (I try not to enforce a particular
style; this repo appears very mixed; I think this change was done by an
automated pass)
##########
File path: python/pyspark/ml/regression.pyi
##########
@@ -477,7 +477,7 @@ class GBTRegressor(
maxIter: int = ...,
stepSize: float = ...,
seed: Optional[int] = ...,
- impuriy: str = ...,
+ impurity: str = ...,
Review comment:
api?
##########
File path: python/pyspark/sql/utils.py
##########
@@ -145,10 +145,10 @@ def toJArray(gateway, jtype, arr):
:param jtype: java type of element in array
:param arr: python type list
"""
- jarr = gateway.new_array(jtype, len(arr))
+ jarray = gateway.new_array(jtype, len(arr))
Review comment:
`jarray` is used for the same sort of thing in other files
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
##########
@@ -420,7 +420,7 @@ private[spark] object Config extends Logging {
val KUBERNETES_FILE_UPLOAD_PATH =
ConfigBuilder("spark.kubernetes.file.upload.path")
.doc("Hadoop compatible file system path where files from the local file
system " +
- "will be uploded to in cluster mode.")
+ "will be uploaded to in cluster mode.")
Review comment:
user facing
##########
File path:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtilsSuite.scala
##########
@@ -49,14 +49,14 @@ class KubernetesVolumeUtilsSuite extends SparkFunSuite {
val sparkConf = new SparkConf(false)
sparkConf.set("test.persistentVolumeClaim.volumeName.mount.path", "/path")
sparkConf.set("test.persistentVolumeClaim.volumeName.mount.readOnly",
"true")
- sparkConf.set("test.persistentVolumeClaim.volumeName.options.claimName",
"claimeName")
Review comment:
??
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
##########
@@ -180,7 +180,7 @@ private[spark] class ExecutorPodsAllocator(
// It's possible that we have outstanding pods that are outdated when
dynamic allocation
// decides to downscale the application. So check if we can release any
pending pods early
// instead of waiting for them to time out. Drop them first from the
unacknowledged list,
- // then from the pending. However, in order to prevent too frequent
frunctuation, newly
Review comment:
?????
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala
##########
@@ -566,7 +566,7 @@ object RewriteCorrelatedScalarSubquery extends
Rule[LogicalPlan] with AliasHelpe
subqueryRoot = Project(projList ++ havingInputs, subqueryRoot)
case s @ SubqueryAlias(alias, _) =>
subqueryRoot = SubqueryAlias(alias, subqueryRoot)
- case op => sys.error(s"Unexpected operator $op in corelated
subquery")
+ case op => sys.error(s"Unexpected operator $op in correlated
subquery")
Review comment:
user facing
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala
##########
@@ -165,7 +165,7 @@ object DataSourceUtils {
"Gregorian calendar.", null)
}
- def creteDateRebaseFuncInRead(
+ def createDateRebaseFuncInRead(
Review comment:
see earlier about "is `crete` a thing?"
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala
##########
@@ -81,7 +81,7 @@ class UnsupportedOperationsSuite extends SparkFunSuite {
// Commands
assertNotSupportedInStreamingPlan(
- "commmands",
+ "commands",
DummyCommand(),
outputMode = Append,
expectedMsgs = "commands" :: Nil)
Review comment:
my change here is probably wrong. would `commands_` be ok?
Often you insert a randomly uppercased letter to indicate you're expecting
an error...
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1116,7 +1116,7 @@ object SQLConf {
val CODEGEN_FACTORY_MODE = buildConf("spark.sql.codegen.factoryMode")
.doc("This config determines the fallback behavior of several codegen
generators " +
- "during tests. `FALLBACK` means trying codegen first and then
fallbacking to " +
+ "during tests. `FALLBACK` means trying codegen first and then falling
back to " +
Review comment:
user facing...
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ErrorParserSuite.scala
##########
@@ -77,7 +77,7 @@ class ErrorParserSuite extends AnalysisTest {
}
test("SPARK-21136: misleading error message due to problematic antlr
grammar") {
- intercept("select * from a left joinn b on a.id = b.id", "missing 'JOIN'
at 'joinn'")
+ intercept("select * from a left join_ b on a.id = b.id", "missing 'JOIN'
at 'join_'")
Review comment:
I'm assuming this is ok (it's in the family of the `commmands` thing
above)
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##########
@@ -1316,7 +1316,7 @@ class SQLQuerySuite extends QueryTest with
SharedSparkSession with AdaptiveSpark
)
}
- test("oder by asc by default when not specify ascending and descending") {
Review comment:
?
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/DataTypeParserSuite.scala
##########
@@ -124,8 +124,8 @@ class DataTypeParserSuite extends SparkFunSuite {
unsupported("struct<x int, y string>")
test("Do not print empty parentheses for no params") {
- assert(intercept("unkwon").getMessage.contains("unkwon is not supported"))
- assert(intercept("unkwon(1,2,3)").getMessage.contains("unkwon(1,2,3) is
not supported"))
Review comment:
does this need to be misspelled?
##########
File path:
sql/core/src/test/java/test/org/apache/spark/sql/Java8DatasetAggregatorSuite.java
##########
@@ -34,43 +34,43 @@
@Test
public void testTypedAggregationAverage() {
KeyValueGroupedDataset<String, Tuple2<String, Integer>> grouped =
generateGroupedDataset();
- Dataset<Tuple2<String, Double>> agged = grouped.agg(
+ Dataset<Tuple2<String, Double>> aggregated = grouped.agg(
Review comment:
Java doesn't have to be terse...
##########
File path:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala
##########
@@ -719,7 +719,7 @@ class HiveCompatibilitySuite extends HiveQueryFileTest with
BeforeAndAfter {
"groupby_multi_insert_common_distinct",
"groupby_multi_single_reducer2",
"groupby_multi_single_reducer3",
- "groupby_mutli_insert_common_distinct",
Review comment:
is this a reference to another repo?
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala
##########
@@ -783,7 +783,7 @@ class DDLParserSuite extends AnalysisTest with
SharedSparkSession {
"escape.delim" -> "y",
"serialization.format" -> "x",
"line.delim" -> "\n",
- "colelction.delim" -> "a", // yes, it's a typo from Hive :)
+ "collection.delim" -> "a", // yes, it's a typo from Hive :)
Review comment:
oops, will drop
##########
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveOptions.scala
##########
@@ -108,7 +108,7 @@ object HiveOptions {
"fieldDelim" -> "field.delim",
"escapeDelim" -> "escape.delim",
// The following typo is inherited from Hive...
- "collectionDelim" -> "colelction.delim",
+ "collectionDelim" -> "collection.delim",
Review comment:
will drop
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/insert.sql
##########
@@ -589,26 +589,26 @@ drop table inserttest;
-- that estate->es_result_relation_info is appropriately set/reset for each
-- routed tuple)
-- [SPARK-29718] Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in
CREATE TABLE
--- create table donothingbrtrig_test (a int, b text) partition by list (a);
--- create table donothingbrtrig_test1 (b text, a int);
--- create table donothingbrtrig_test2 (c text, b text, a int);
--- alter table donothingbrtrig_test2 drop column c;
--- create or replace function donothingbrtrig_func() returns trigger as
$$begin raise notice 'b: %', new.b; return NULL; end$$ language plpgsql;
--- create trigger donothingbrtrig1 before insert on donothingbrtrig_test1 for
each row execute procedure donothingbrtrig_func();
--- create trigger donothingbrtrig2 before insert on donothingbrtrig_test2 for
each row execute procedure donothingbrtrig_func();
--- alter table donothingbrtrig_test attach partition donothingbrtrig_test1 for
values in (1);
--- alter table donothingbrtrig_test attach partition donothingbrtrig_test2 for
values in (2);
--- insert into donothingbrtrig_test values (1, 'foo'), (2, 'bar');
+-- create table do nothingbrtrig_test (a int, b text) partition by list (a);
+-- create table do nothingbrtrig_test1 (b text, a int);
+-- create table do nothingbrtrig_test2 (c text, b text, a int);
+-- alter table do nothingbrtrig_test2 drop column c;
+-- create or replace function do nothingbrtrig_func() returns trigger as
$$begin raise notice 'b: %', new.b; return NULL; end$$ language plpgsql;
+-- create trigger do nothingbrtrig1 before insert on do nothingbrtrig_test1
for each row execute procedure do nothingbrtrig_func();
+-- create trigger do nothingbrtrig2 before insert on do nothingbrtrig_test2
for each row execute procedure do nothingbrtrig_func();
+-- alter table do nothingbrtrig_test attach partition do nothingbrtrig_test1
for values in (1);
+-- alter table do nothingbrtrig_test attach partition do nothingbrtrig_test2
for values in (2);
+-- insert into do nothingbrtrig_test values (1, 'foo'), (2, 'bar');
-- [SPARK-29386] Copy data between a file and a table
--- copy donothingbrtrig_test from stdout;
+-- copy do nothingbrtrig_test from stdout;
Review comment:
oops, didn't mean to do this (will drop)
##########
File path:
sql/hive/src/test/resources/ql/src/test/queries/clientpositive/overridden_confs.q
##########
@@ -1,4 +1,4 @@
set
hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.VerifyOverriddenConfigsHook;
-set hive.config.doesnt.exit=abc;
+set hive.config.does_not.exit=abc;
Review comment:
probably need to drop?
##########
File path:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala
##########
@@ -961,8 +961,8 @@ class HiveCompatibilitySuite extends HiveQueryFileTest with
BeforeAndAfter {
"subq2",
"subquery_exists",
"subquery_exists_having",
- "subquery_notexists",
- "subquery_notexists_having",
Review comment:
ibid?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://cloud.githubusercontent.com/assets/6477701/18228810/2f674e5e-7299-11e6-929d-5c2dff269ddc.png"](https://cloud.githubusercontent.com/assets/6477701/18228810/2f674e5e-7299-11e6-929d-5c2dff269ddc.png"){kind=link}
![https://cloud.githubusercontent.com/assets/6477701/18228811/344263a0-7299-11e6-90b7-9b1c7b6b8b01.png"](https://cloud.githubusercontent.com/assets/6477701/18228811/344263a0-7299-11e6-90b7-9b1c7b6b8b01.png"){kind=link}
![https://cloud.githubusercontent.com/assets/6477701/18228803/12a4b810-7299-11e6-9140-5cfc277297b1.png"](https://cloud.githubusercontent.com/assets/6477701/18228803/12a4b810-7299-11e6-9140-5cfc277297b1.png"){kind=link}