[
https://issues.apache.org/jira/browse/HIVE-21240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767045#comment-16767045
]
Zoltan Haindrich commented on HIVE-21240:
-----------------------------------------
Could you please explain why are you making the tests pass by "adapting" them
to your changes? ...IIRC HCat uses Lists/etc but the other world is using
standard Array-s
{code}
@@ -109,9 +109,10 @@ public void testStructNullField() throws Exception {
udf.initialize(arguments);
Object res = udf.evaluate(evalArgs("{\"a\":null}"));
- assertTrue(res instanceof Object[]);
- Object o[] = (Object[]) res;
- assertEquals(null, o[0]);
+ assertTrue(res instanceof List<?>);
+
+ List<?> o = (List<?>) res;
+ assertEquals(null, o.get(0));
}
}
{code}
I'm not sure about the whole rewrite thing:
{quote}
Use Jackson Tree parser instead of manually parsing
{quote}
actually..."manually parsing" have avoided to have the tree parsed as a tree
upfront; it worked by using a 1 token lookahead
{quote}
Current JSON parser accepts, but does not apply, custom timestamp formats in
most cases
{quote}
can't this be fixed without rewriting?
{quote}
Added cache for column-name to column-index searches, currently O\(n\) for each
row processed, for each column in the row
{quote}
I think this could be also fixed in the existing implementation; IIRC there was
a FIXME about this issue....
could you please stop reformating lines which are not modified at all?
especially comments/apidocs ?
{code}
- private static StringBuilder appendWithQuotes(StringBuilder sb, String
value) {
- return sb == null ? null :
sb.append(SerDeUtils.QUOTE).append(value).append(SerDeUtils.QUOTE);
+ private static StringBuilder appendWithQuotes(StringBuilder sb,
+ String value) {
+ return sb == null ? null
+ : sb.append(SerDeUtils.QUOTE).append(value).append(SerDeUtils.QUOTE);
- // TODO : code section copied over from SerDeUtils because of non-standard
json production there
- // should use quotes for all field names. We should fix this there, and then
remove this copy.
+ // TODO : code section copied over from SerDeUtils because of non-standard
+ // json production there
+ // should use quotes for all field names. We should fix this there, and then
+ // remove this copy.
{code}
please use the hive formatter
(https://cwiki.apache.org/confluence/display/Hive/HowToContribute)
> JSON SerDe Deserialize Re-Write
> -------------------------------
>
> Key: HIVE-21240
> URL: https://issues.apache.org/jira/browse/HIVE-21240
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
> Affects Versions: 4.0.0, 3.1.1
> Reporter: BELUGA BEHR
> Assignee: BELUGA BEHR
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21240.1.patch, HIVE-21240.1.patch,
> HIVE-21240.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The JSON SerDe has a few issues, I will link them to this JIRA.
> * Use Jackson Tree parser instead of manually parsing
> * Added support for base-64 encoded data (the expected format when using JSON)
> * Added support to skip blank lines (returns all columns as null values)
> * Current JSON parser accepts, but does not apply, custom timestamp formats
> in most cases
> * Added some unit tests
> * Added cache for column-name to column-index searches, currently O\(n\) for
> each row processed, for each column in the row
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
