I noticed spark has some nice memory tracking estimators in it, but they are private. We have some custom implementations of RDD and PairRDD to suit our internal needs and it’d be fantastic if we’d be able to just leverage the memory estimates that already exist in Spark.
Is there any change they can be made public inside the library or have some interface to them such that children classes can make use of them? Thanks, Stephen Carman, M.S. AI Engineer, Coldlight Solutions, LLC Cell - 267 240 0363 This e-mail is intended solely for the above-mentioned recipient and it may contain confidential or privileged information. If you have received it in error, please notify us immediately and delete the e-mail. You must not copy, distribute, disclose or take any action in reliance on it. In addition, the contents of an attachment to this e-mail may contain software viruses which could damage your own computer system. While ColdLight Solutions, LLC has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage which you sustain as a result of software viruses. You should perform your own virus checks before opening the attachment. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
