Hi, I am building a data source so I can convert a custom source to dataframe. I have been going over examples such as JDBC and noticed that JDBC does the following:
val inputMetrics = context.taskMetrics().inputMetrics and whenever a new record is added: inputMetrics.incRecordsRead(1) however, incRecordsRead is spark private. I was wondering: how can I get standard metrics through a data source? Thanks, Assaf. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Adding-metrics-to-spark-datasource-tp28487.html Sent from the Apache Spark User List mailing list archive at Nabble.com.