Re: sstableloader from java

2016-10-06 Thread Paul Weiss
I ended up create a utility class which combines the functionality
of CQLSSTableWriter and the existing BulkLoader class which is used by the
sstableloader.  The code is located here:

https://github.com/paul-weiss/cassandra-examples/blob/master/src/main/java/org/apache/cassandra/tools/BulkWriterLoader.java

If there is interest I could merge into the codebase, just let me know.

thanks,
-paul

On Wed, Sep 21, 2016 at 9:37 PM, J. D. Jordan <jeremiah.jor...@gmail.com>
wrote:

> Yeah. I wouldn't use that class, I was just pointing it out as an example
> of code doing a similar thing.
>
> > On Sep 21, 2016, at 8:16 PM, Paul Weiss <paulweiss@gmail.com> wrote:
> >
> > Great will try out.   I noticed the code is within the hadoop folder,  I
> am
> > using local disk so hopefully it will work without hadoop.
> >
> > Thanks,
> > -paul
> >
> > On Sep 21, 2016 8:56 PM, "Jeremiah D Jordan" <jeremiah.jor...@gmail.com>
> > wrote:
> >
> >> Yes using the SSTableLoader class.  You can see the CqlBulkRecordWriter
> >> class for an example of writing out sstables to disk and then using the
> >> SSTableLoader class to stream them to a cluster.
> >>
> >> -Jeremiah
> >>
> >>> On Sep 21, 2016, at 7:18 PM, Paul Weiss <paulweiss@gmail.com>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Is it possible to call the sstableloader from java instead using the
> >>> command line program? I have a process that uses the CQLSSTableWriter
> and
> >>> generates the sstable files but am looking for an end to end process
> that
> >>> bulk loads without any manual intervention.
> >>>
> >>> Ideally would like to avoid forking another process so I can properly
> >> check
> >>> for errors.
> >>>
> >>> Thanks,
> >>> -paul
> >>
> >>
>


Re: sstableloader from java

2016-09-21 Thread Paul Weiss
Great will try out.   I noticed the code is within the hadoop folder,  I am
using local disk so hopefully it will work without hadoop.

Thanks,
-paul

On Sep 21, 2016 8:56 PM, "Jeremiah D Jordan" <jeremiah.jor...@gmail.com>
wrote:

> Yes using the SSTableLoader class.  You can see the CqlBulkRecordWriter
> class for an example of writing out sstables to disk and then using the
> SSTableLoader class to stream them to a cluster.
>
> -Jeremiah
>
> > On Sep 21, 2016, at 7:18 PM, Paul Weiss <paulweiss@gmail.com> wrote:
> >
> > Hi,
> >
> > Is it possible to call the sstableloader from java instead using the
> > command line program? I have a process that uses the CQLSSTableWriter and
> > generates the sstable files but am looking for an end to end process that
> > bulk loads without any manual intervention.
> >
> > Ideally would like to avoid forking another process so I can properly
> check
> > for errors.
> >
> > Thanks,
> > -paul
>
>


sstableloader from java

2016-09-21 Thread Paul Weiss
Hi,

Is it possible to call the sstableloader from java instead using the
command line program? I have a process that uses the CQLSSTableWriter and
generates the sstable files but am looking for an end to end process that
bulk loads without any manual intervention.

Ideally would like to avoid forking another process so I can properly check
for errors.

Thanks,
-paul