What do you expect to see, how did you create it, and what are the weird values? Any chance your compression settings are different for writing and reading?
On Tue, Oct 25, 2011 at 7:41 AM, Gayatri Rao <[email protected]> wrote: > Thanks Dmitriy. > I was trying to implement the MyClassConverter for my custom class > and I overided and implemented the method > > @Override > public Object bytesToObject(DataByteArray dataByteArray) throws > IOException { > > MyClass o = (MyClass) ReflectionUtils.newInstance(MyClass.class, > null); > o.readFields(new DataInputStream(new > ByteArrayInputStream(dataByteArray > .get()))); > return o; > > } > > and my MyClass.readFields is as follows: > > @Override > public void readFields(DataInput in) throws IOException { > num = in.readInt(); > list = new ArrayList<String>(); > for (int i = 0; i < 3; i++) { > list.add(WritableUtils.readString(in)); > } > > } > > This puts some weird data in num and list. Any idea what I might be doing > wrong? > > > On Tue, Oct 25, 2011 at 9:03 AM, Dmitriy Ryaboy <[email protected]> wrote: > >> you can compile it with "ant -Dnothrift=true" >> >> There's also a "-Dnoprotobuf=true" option, but I just tried it and it seems >> we do require protobufs in 1 place that's not excluded when we skip >> protocol >> buffers, so you still need protoc version 2.3 >> >> D >> >> On Mon, Oct 24, 2011 at 6:52 PM, Gayatri Rao <[email protected]> wrote: >> >> > Thats great, thanks, I ll check it out. Is thrift a dependency for >> > building? >> > >> > On Mon, Oct 24, 2011 at 6:49 PM, Dmitriy Ryaboy <[email protected]> >> > wrote: >> > >> > > Correct -- it's completely rewritten. >> > > >> > > We haven't published EB to a public maven repo, though I believe we did >> > add >> > > a "maven-install" ant target to publish to your local maven repo. >> > > >> > > D >> > > >> > > On Mon, Oct 24, 2011 at 6:33 PM, Gayatri Rao <[email protected]> >> > wrote: >> > > >> > > > I have checked the SequenceFileLoader from elephantbird and it seems >> > to >> > > > use >> > > > a different SequenceFileLoader as oppose to the one there is in >> > piggybank >> > > > Is there any reason for that? >> > > > >> > > > On Mon, Oct 24, 2011 at 5:57 PM, Gayatri Rao <[email protected]> >> > > wrote: >> > > > >> > > > > Thank Dmitriy. Are the jars available in maven repository? >> > > > > >> > > > > Thanks, >> > > > > Gayatri >> > > > > >> > > > > >> > > > > On Mon, Oct 24, 2011 at 11:55 AM, Dmitriy Ryaboy < >> [email protected] >> > > > >wrote: >> > > > > >> > > > >> We have a massively improved (well, rewritten from scratch) >> > > > SequenceLoader >> > > > >> in elephantbird. Take a look here: >> > > > >> >> > > > >> >> > > > >> > > >> > >> https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/load/SequenceFileLoader.java >> > > > >> >> > > > >> No separate readme on usage, but all the related classes are >> > > > >> well-documented >> > > > >> in Javadocs. >> > > > >> >> > > > >> D >> > > > >> >> > > > >> On Mon, Oct 24, 2011 at 12:55 AM, Daniel Dai < >> [email protected] >> > > >> > > > >> wrote: >> > > > >> >> > > > >> > I think it is the >> > SequenceFileLoader.translateWritableToPigDataType >> > > > >> which >> > > > >> > does not support custom writable. Try to enhance >> > > > >> > translateWritableToPigDataType. >> > > > >> > >> > > > >> > Daniel >> > > > >> > >> > > > >> > On Mon, Oct 24, 2011 at 12:21 AM, Gayatri Rao < >> > [email protected]> >> > > > >> wrote: >> > > > >> > >> > > > >> > > Hi All, >> > > > >> > > >> > > > >> > > I am trying to use the sequence file loader in piggybank for >> my >> > > > custom >> > > > >> > > writable object. I am working with pig 0.8, It looks like it >> > does >> > > > not >> > > > >> > work >> > > > >> > > for user defined custom writables? >> > > > >> > > Any pointers on how I can write a loader for my own custom >> > > writable? >> > > > >> > > >> > > > >> > > Thanks, >> > > > >> > > Gayatri >> > > > >> > > >> > > > >> > >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >> >
