TheNeuralBit commented on a change in pull request #14961: URL: https://github.com/apache/beam/pull/14961#discussion_r647005303
########## File path: sdks/python/apache_beam/dataframe/frames.py ########## @@ -127,6 +127,16 @@ def wrapper(self, *args, **kwargs): return frame_base.with_docs_from(base)(wrapper) +# Docstring to use for head and tail (commonly used to peek at datasets) +_PEEK_METHOD_EXPLANATION = ( + "because it is `order-sensitive <https://s.apache.org/dataframe-order-sensitive-operations>`_.\n\n" + "If you'd like to use it to peek at a large dataset, interactive Beam's " + ":func:`ib.collect <apache_beam.runners.interactive.interactive_beam.collect>` " + "with ``n`` specified may be a useful alternative.\n\n" + "Also consider using :meth:`DeferredDataFrame.nlargest` if you're " + "interested in finding the top-N elements in a dataset.") Review comment: Yeah `sample` is the last operation I feel like is a must-have for GA. I've just been thinking about it so far. I'd like to discuss some of the hangups in-person. A preview: - Making `frac` work is trivial since you can just do `frac` on each partition. - I'm not sure about the right way to make `n` work though (which we should figure out, since it's the most natural alternative to head). - I think we can support `weights` by doing a little extra work to adjust the weights within each partition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
