Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/3753#discussion_r28769771
--- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala ---
@@ -41,20 +41,20 @@ class SparkHadoopWriter(@transient jobConf: JobConf)
with SparkHadoopMapRedUtil
with Serializable {
- private val now = new Date()
- private val conf = new SerializableWritable(jobConf)
+ protected val now = new Date()
+ protected val conf = new SerializableWritable(jobConf)
- private var jobID = 0
- private var splitID = 0
- private var attemptID = 0
- private var jID: SerializableWritable[JobID] = null
- private var taID: SerializableWritable[TaskAttemptID] = null
+ protected var jobID = 0
+ protected var splitID = 0
+ protected var attemptID = 0
+ protected var jID: SerializableWritable[JobID] = null
+ protected var taID: SerializableWritable[TaskAttemptID] = null
- @transient private var writer: RecordWriter[AnyRef,AnyRef] = null
- @transient private var format: OutputFormat[AnyRef,AnyRef] = null
- @transient private var committer: OutputCommitter = null
- @transient private var jobContext: JobContext = null
- @transient private var taskContext: TaskAttemptContext = null
+ @transient protected var writer: RecordWriter[AnyRef,AnyRef] = null
+ @transient protected var format: OutputFormat[AnyRef,AnyRef] = null
+ @transient protected var committer: OutputCommitter = null
+ @transient protected var jobContext: JobContext = null
+ @transient protected var taskContext: TaskAttemptContext = null
--- End diff --
I changed the scope of var/def of SparkHadoopWriter to reuse these code in
orc writting api implementation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
