i see this warning when running jobs on cluster: 2016-10-12 14:46:47 WARN spark.SparkContext: Spark is not running in local mode, therefore the checkpoint directory must not be on the local filesystem. Directory '/tmp' appears to be on the local filesystem.
however the checkpoint "directory" that it warns about is a hadoop path. i use an unqualified path, which means a path on the default filesystem by hadoop convention. when running on the cluster my default filesystem is hdfs (and it correctly uses hdfs). how about if we change the method that does this check (Utils.nonLocalPaths) to be aware of the default filesystem instead of incorrectly assuming its local if not specified?
