[
https://issues.apache.org/jira/browse/PIG-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004751#comment-15004751
]
Rohini Palaniswamy commented on PIG-4734:
-----------------------------------------
Any udf not implementing outputSchema() method is good for reproducing. But the
UDF that hit the issue had another slight property in addition to not
implementing outputSchema() method. It does GenerateMap extends
EvalFunc<Map<String, String>> making returnType of EvalFunc to be Map<String,
String>, but can also return Map<String, Float>. It would be good to have that
in the testcase as well as EvalFunc.getReturnType() is used when outputSchema()
returns null.
DEFINE GenerateMapFloat GenerateMap('FLOAT', 'xyz');
{code}
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
public class GenerateMap extends EvalFunc<Map<String, String>>{
private ArrayList<String> map_keys = null;
private String mapType = null;
public GenerateMap(String... args ) throws Exception {
// TODO Auto-generated constructor stub
for (int i = 0; i < args.length; i++)
{
if (args[i] == null || args[i].length() <= 0)
{
System.out.println(i);
throw new Exception("Cannot have null or empty string map key");
} else if(i > 0) {
map_keys.add(args[i]);
} else {
map_keys = new ArrayList<String>();
mapType = args[i];
}
}
}
@Override
public Map exec(Tuple input) throws IOException {
if (input.size() != map_keys.size())
{
throw new IOException("tuple size of: " + input.size() + " must
match number of arguments: " + map_keys.size() + " in constructor");
}
Map pig_map;
if(mapType.equals("STRING")) {
pig_map = new HashMap<String, String>();
}
else {
pig_map = new HashMap<String, Float>();
}
for (int i = 0; i < input.size(); i++)
{
if (input.get(i) != null)
{
String value = input.get(i).toString();
if (value.length() != 0)
{
if(mapType.equals("STRING")) {
pig_map.put(map_keys.get(i), value);
} else {
pig_map.put(map_keys.get(i), Float.parseFloat(value));
}
}
}
}
return pig_map;
}
}
{code}
> TOMAP schema inferring breaks some scripts in type checking for bincond
> -----------------------------------------------------------------------
>
> Key: PIG-4734
> URL: https://issues.apache.org/jira/browse/PIG-4734
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Daniel Dai
> Fix For: 0.16.0, 0.15.1
>
> Attachments: PIG-4734-1.patch
>
>
> PIG-4674 added schema inferring for TOMAP.
> {code}
> FOREACH A GENERATE (val == 'x' ? TOMAP('key', floatfield1) : (val == 'y' ?
> GenerateFloatMap('key', floatfield2) : NULL)) as floatmap:map[float],
> {code}
> The following line fails with
> {code}
> Two inputs of BinCond must have compatible schemas. left hand side:
> #675:map(#676:float) right hand side: #801:map
> at
> org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:616)
> ... 45 more
> {code}
> GenerateFloatMap is a UDF that returns new HashMap<String, Float>(), but does
> not have outputSchema() defined. It worked earlier because TOMAP also did not
> have outputSchema() defined.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
