data import with transformer
Hi, I have been reading the solr book and wiki, but I cant find any
similar examples to what Im looking for.
I have a database field called category, this field needs some text
manipulation before it goes in the index
here is the java code for what im trying to do:
// categories look like this prefix category suffix
// I want to turn them into category remove prefix and suffix and
spaces before and after
public static String getPrettyCategoryName(String categoryName)
{
String result;
if (categoryName == null || categoryName.equals())
{
// nothing to do; just return what was passed in.
result = categoryName;
}
else
{
result = categoryName.toLowerCase();
if (result.startsWith(startString))
{
result = result.substring(startString.length());
}
if (result.endsWith(endString))
{
result = result.substring(0, (result.length() -
endString
.length()));
}
if (result.length() 0)
{
result = Character.toUpperCase(result.charAt(0))
+ result.substring(1);
}
}
return result;
}
Can I have a transformer call a java method?
It seems like I can, but how do I transform must one column. If
someone can point me to a complete example that transforms a column
using java or javascript im sure I can figure this out
thanks
Joel
Re: data import with transformer
I'd recommend two ways: The way I do it in my app is that I have written a
MySql function to transform the column as part of the select statement. In
this approach, your select query would like so:
select col1, col2, col3, spPrettyPrintCategory(category) as X, col4,
col5, from table where
field column=X name=category /
The field element is used to map the column X into the solr field name
which I am assuming is the same as your category name.
The second approach is to write the JavaScript transformer. The relevant
code is in the wiki:
dataConfig
script![CDATA[
function PrettyCategory(row){
//split on spaces
var pieces = row.get('category').split(' ');
// get the second element of this array...do a trim if
needed...
var catname = pieces[1];
row.remove('category');
row.put('category', catname);
return row;
}
]]/script
document
entity name=e transformer=script:PrettyCategory
query=select * from X
/entity
/document
/dataConfig
- Bill
--
From: Joel Nylund jnyl...@yahoo.com
Sent: Thursday, October 29, 2009 9:18 AM
To: solr-user@lucene.apache.org
Subject: data import with transformer
Hi, I have been reading the solr book and wiki, but I cant find any
similar examples to what Im looking for.
I have a database field called category, this field needs some text
manipulation before it goes in the index
here is the java code for what im trying to do:
// categories look like this prefix category suffix
// I want to turn them into category remove prefix and suffix and
spaces before and after
public static String getPrettyCategoryName(String categoryName)
{
String result;
if (categoryName == null || categoryName.equals())
{
// nothing to do; just return what was passed in.
result = categoryName;
}
else
{
result = categoryName.toLowerCase();
if (result.startsWith(startString))
{
result = result.substring(startString.length());
}
if (result.endsWith(endString))
{
result = result.substring(0, (result.length() -
endString
.length()));
}
if (result.length() 0)
{
result = Character.toUpperCase(result.charAt(0))
+ result.substring(1);
}
}
return result;
}
Can I have a transformer call a java method?
It seems like I can, but how do I transform must one column. If
someone can point me to a complete example that transforms a column
using java or javascript im sure I can figure this out
thanks
Joel
Re: data import with transformer
Another option is the RegexTransformer in DIH:
http://wiki.apache.org/solr/DataImportHandler?highlight=%28regex%29#RegexTransformer
Chantal
William Pierce schrieb:
I'd recommend two ways: The way I do it in my app is that I have written a
MySql function to transform the column as part of the select statement. In
this approach, your select query would like so:
select col1, col2, col3, spPrettyPrintCategory(category) as X, col4,
col5, from table where
field column=X name=category /
The field element is used to map the column X into the solr field name
which I am assuming is the same as your category name.
The second approach is to write the JavaScript transformer. The relevant
code is in the wiki:
dataConfig
script![CDATA[
function PrettyCategory(row){
//split on spaces
var pieces = row.get('category').split(' ');
// get the second element of this array...do a trim if
needed...
var catname = pieces[1];
row.remove('category');
row.put('category', catname);
return row;
}
]]/script
document
entity name=e transformer=script:PrettyCategory
query=select * from X
/entity
/document
/dataConfig
- Bill
--
From: Joel Nylund jnyl...@yahoo.com
Sent: Thursday, October 29, 2009 9:18 AM
To: solr-user@lucene.apache.org
Subject: data import with transformer
Hi, I have been reading the solr book and wiki, but I cant find any
similar examples to what Im looking for.
I have a database field called category, this field needs some text
manipulation before it goes in the index
here is the java code for what im trying to do:
// categories look like this prefix category suffix
// I want to turn them into category remove prefix and suffix and
spaces before and after
public static String getPrettyCategoryName(String categoryName)
{
String result;
if (categoryName == null || categoryName.equals())
{
// nothing to do; just return what was passed in.
result = categoryName;
}
else
{
result = categoryName.toLowerCase();
if (result.startsWith(startString))
{
result = result.substring(startString.length());
}
if (result.endsWith(endString))
{
result = result.substring(0, (result.length() -
endString
.length()));
}
if (result.length() 0)
{
result = Character.toUpperCase(result.charAt(0))
+ result.substring(1);
}
}
return result;
}
Can I have a transformer call a java method?
It seems like I can, but how do I transform must one column. If
someone can point me to a complete example that transforms a column
using java or javascript im sure I can figure this out
thanks
Joel
