JSON XML response writer issues with short binary fields
I finally got around to looking at why short field values are returned as java.lang.Short:value. Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used: // default... for debugging only writeStr(name, val.getClass().getName() + ':' + val.toString(), true); The same thing happens when you have a binary field, since val in that case is byte[], so you get [B:[B@address of byte array Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema. This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere. -- Ken PS - any good reason why XMLWriter is a final class? I created my own fixed version of JSONResponseWriter w/o much effort because I could subclass it, but XMLWriter being final makes it hard (impossible?) to do the same, since there are numerous internal methods that take an explicit XMLWriter object as a parameter. -- Ken Krugler http://www.scaleunlimited.com custom big data solutions training Hadoop, Cascading, Mahout Solr
Re: JSON XML response writer issues with short binary fields
On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler kkrugler_li...@transpac.com wrote: I finally got around to looking at why short field values are returned as java.lang.Short:value. Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the check for (val instanceof Short), and thus this bit of code is used: // default... for debugging only writeStr(name, val.getClass().getName() + ':' + val.toString(), true); The same thing happens when you have a binary field, since val in that case is byte[], so you get [B:[B@address of byte array Has anybody else run into this? Seems odd that it's not a known issue, so I'm wondering if there's something odd about my schema. This is especially true since BinaryField has write methods for both XML and JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm wondering how normally the BinaryField.write() methods would get used, and whether the actual problem lies elsewhere. Hmmm, Ryan recently restructured some of the writer code to support the pseudo-field feature. A quick look at the code seems like FieldType.write() methods are not used anymore (the Document is transformed into a SolrDocument and writeVal is used for each value). Guess this highlights deficiencies in our tests as well surrounding short, byte and binary. Could you open an issue for this? -Yonik http://www.lucidimagination.com
Re: JSON XML response writer issues with short binary fields
-Yonik
http://www.lucidimagination.com
On Fri, Jan 13, 2012 at 4:22 PM, Yonik Seeley
yo...@lucidimagination.com wrote:
On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler
kkrugler_li...@transpac.com wrote:
I finally got around to looking at why short field values are returned as
java.lang.Short:value.
Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the
check for (val instanceof Short), and thus this bit of code is used:
// default... for debugging only
writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
The same thing happens when you have a binary field, since val in that case
is byte[], so you get [B:[B@address of byte array
Has anybody else run into this? Seems odd that it's not a known issue, so
I'm wondering if there's something odd about my schema.
This is especially true since BinaryField has write methods for both XML and
JSON (via TextResponseWriter) that handle Base64-encoding the data. So I'm
wondering how normally the BinaryField.write() methods would get used, and
whether the actual problem lies elsewhere.
Hmmm, Ryan recently restructured some of the writer code to support
the pseudo-field feature. A quick look at the code seems like
FieldType.write() methods are not used anymore (the Document is
transformed into a SolrDocument and writeVal is used for each value).
Double hmmm... I see this in writeVal()
} else if (val instanceof IndexableField) {
IndexableField f = (IndexableField)val;
SchemaField sf = schema.getFieldOrNull( f.name() );
if( sf != null ) {
sf.getType().write(this, name, f);
}
So my initial quick analysis of FieldType.write() not being used
anymore doesn't look correct.
Anyway, please do open an issue and we'll get to the bottom of it.
-Yonik
http://www.lucidimagination.com
Re: JSON XML response writer issues with short binary fields
On Jan 13, 2012, at 1:39pm, Yonik Seeley wrote:
-Yonik
http://www.lucidimagination.com
On Fri, Jan 13, 2012 at 4:22 PM, Yonik Seeley
yo...@lucidimagination.com wrote:
On Fri, Jan 13, 2012 at 4:04 PM, Ken Krugler
kkrugler_li...@transpac.com wrote:
I finally got around to looking at why short field values are returned as
java.lang.Short:value.
Both XMLWriter.writeVal() and TextResponseWriter.writeVal() are missing the
check for (val instanceof Short), and thus this bit of code is used:
// default... for debugging only
writeStr(name, val.getClass().getName() + ':' + val.toString(), true);
The same thing happens when you have a binary field, since val in that case
is byte[], so you get [B:[B@address of byte array
Has anybody else run into this? Seems odd that it's not a known issue, so
I'm wondering if there's something odd about my schema.
This is especially true since BinaryField has write methods for both XML
and JSON (via TextResponseWriter) that handle Base64-encoding the data. So
I'm wondering how normally the BinaryField.write() methods would get used,
and whether the actual problem lies elsewhere.
Hmmm, Ryan recently restructured some of the writer code to support
the pseudo-field feature. A quick look at the code seems like
FieldType.write() methods are not used anymore (the Document is
transformed into a SolrDocument and writeVal is used for each value).
Double hmmm... I see this in writeVal()
} else if (val instanceof IndexableField) {
IndexableField f = (IndexableField)val;
SchemaField sf = schema.getFieldOrNull( f.name() );
if( sf != null ) {
sf.getType().write(this, name, f);
}
So my initial quick analysis of FieldType.write() not being used
anymore doesn't look correct.
Anyway, please do open an issue and we'll get to the bottom of it.
Thanks for the fast response - I was beginning to worry that nobody read my
posts :)
See https://issues.apache.org/jira/browse/SOLR-3035
I've attached some test code to the issue, plus a simple fix for the JSON case.
Regards,
-- Ken
--
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions training
Hadoop, Cascading, Mahout Solr
