Sure no prob. On Tue, Oct 6, 2015 at 6:35 PM, Tathagata Das <t...@databricks.com> wrote:
> Given the interest, I am also inclining towards making it a public > developer API. Maybe even experimental. Cody, mind submitting a patch? > > > On Tue, Oct 6, 2015 at 7:45 AM, Sean Owen <so...@cloudera.com> wrote: > >> For what it's worth, I also use this class in an app, but it happens >> to be from Java code where it acts as if it's public. So no problem >> for my use case, but I suppose, another small vote for the usefulness >> of this class to the caller. I end up using getLatestLeaderOffsets to >> figure out how to initialize initial offsets. >> >> On Tue, Oct 6, 2015 at 3:24 PM, Cody Koeninger <c...@koeninger.org> >> wrote: >> > I personally think KafkaCluster (or the equivalent) should be made >> public. >> > When I'm deploying spark I just sed out the private[spark] and rebuild. >> > >> > There's a general reluctance to make things public due to backwards >> > compatibility, but if enough people ask for it... ? >> > >> > On Tue, Oct 6, 2015 at 6:51 AM, Jonathan Coveney <jcove...@gmail.com> >> wrote: >> >> >> >> You can put a class in the org.apache.spark namespace to access >> anything >> >> that is private[spark]. You can then make enrichments there to access >> >> whatever you need. Just beware upgrade pain :) >> >> >> >> >> >> El martes, 6 de octubre de 2015, Erwan ALLAIN <eallain.po...@gmail.com >> > >> >> escribió: >> >>> >> >>> Hello, >> >>> >> >>> I'm currently testing spark streaming with kafka. >> >>> I'm creating DirectStream with KafkaUtils and everything's fine. >> However >> >>> I would like to use the signature where I can specify my own message >> handler >> >>> (to play with partition and offset). In this case, I need to manage >> >>> offset/partition by myself to fill fromOffsets argument. >> >>> I have found a Jira on this usecase >> >>> https://issues.apache.org/jira/browse/SPARK-6714 but it has been >> closed >> >>> telling that it's too specific. >> >>> I'm aware that it can be done using kafka api (TopicMetaDataRequest >> and >> >>> OffsetRequest) but what I have to do is almost the same as the >> KafkaCluster >> >>> which is private. >> >>> >> >>> is it possible to : >> >>> - add another signature in KafkaUtils ? >> >>> - make KafkaCluster public ? >> >>> >> >>> or do you have any other srmart solution where I don't need to >> copy/paste >> >>> KafkaCluster ? >> >>> >> >>> Thanks. >> >>> >> >>> Regards, >> >>> Erwan ALLAIN >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
