Xeinn commented on issue #4104: NIFI-7159 URL: https://github.com/apache/nifi/pull/4104#issuecomment-595001978 Sorry, it's 3am here, just read first line and replied... missed rest of content on email(was on my phone) will read through in morning. Regards Chris On 5 Mar 2020 12:06 am, Mike <[email protected]> wrote: @MikeThomsen commented on this pull request. Looks pretty good, but there is definitely some scope creep that will require others' input. ________________________________ In nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r388006141&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942475012&sdata=XWa%2Fpsj%2FaDh6zjpZ%2FPkQeMcP6DdqRQhRsBz%2BowUECEQ%3D&reserved=0>: > @@ -669,6 +669,9 @@ private void writeValue(final JsonGenerator generator, final Object value, final } break; } + case DECIMAL: I think this should actually be a double because that's the maximum that Elasticsearch supports. See here: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.elastic.co%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmapping-types.html&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942475012&sdata=YHV3U6dbWF7ijTcGE3dGZ0aS9YdugbjqGz7hHH8WEUA%3D&reserved=0> ________________________________ In nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/PutHBaseRecord.java<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r388006605&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942485007&sdata=vYgcjLLPE4yK7WbHMmORO1xcAB8mOPlTsng1Usvv%2B3U%3D&reserved=0>: > @@ -341,6 +341,8 @@ protected PutFlowFile createPut(ProcessSession session, ProcessContext context, case BOOLEAN: retVal = clientService.toBytes(record.getAsBoolean(field)); break; + case DECIMAL: + // Decimal to be treated as the same as double It should be broken down into a byte array like the other types. ________________________________ In nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r388006930&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942495005&sdata=TngnYOop4kHdaO7c1PdgTSXptjYjhgo0Nnpr73IyT8o%3D&reserved=0>: > @@ -104,6 +105,10 @@ public static Object convertToORCObject(TypeInfo typeInfo, Object o, final boole if (o instanceof Double) { return new DoubleWritable((double) o); } + // Map BigDecimal to a Double type - this should be improved to map to Hive Decimal type + if (o instanceof BigDecimal) { There were some runtime issues with unit tests related to Orc. @mattyb149<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmattyb149&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942505006&sdata=Uh48wNNGrMxwezJ0oeePBSIONqX0dwztEAcpOVU%2FOEE%3D&reserved=0> @bbende<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbbende&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942514998&sdata=AbgWHeiA8yuR5lnje6Q7bf21PfGg2QKjkG0krwgLLEc%3D&reserved=0> @ijokarumawak<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fijokarumawak&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942524997&sdata=2DM3oYjo4Se9I0akbixrRG2kW4bYpw4yzWdnA6mEIFY%3D&reserved=0> do y'all have any insight into this ORC conversion? ________________________________ In nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/src/main/java/org/apache/nifi/controller/kudu/KuduLookupService.java<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r388007299&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942534986&sdata=%2Bk9ocVYWhw0TcbIHz5mCcBXnQHyhLSgPy1mive51I9E%3D&reserved=0>: > @@ -317,7 +317,7 @@ private RecordSchema kuduSchemaToNiFiSchema(Schema kuduTableSchema, List<String> case BINARY: case STRING: case DECIMAL: - fields.add(new RecordField(cs.getName(), RecordFieldType.STRING.getDataType())); + fields.add(new RecordField(cs.getName(), RecordFieldType.DECIMAL.getDataType())); I don't know anything about Kudu, so I may put this ticket on hold and ask for input from nifi-dev on a few of these other components (ex Hive) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%3Femail_source%3Dnotifications%26email_token%3DAFHXE7CHGMESCDOAGOUXIWDRF3UIRA5CNFSM4K72OIGKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCYA3SRY%23pullrequestreview-369211719&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942544983&sdata=3W2eDNRRhvMVdVOP9qOfw4pSbtEJvncNuRDHenYYGms%3D&reserved=0>, or unsubscribe<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFHXE7EJVGYFFGBPKHRJK4LRF3UIRANCNFSM4K72OIGA&data=02%7C01%7C%7Ce644906b5c724171b30c08d7c0991093%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189635942554981&sdata=SdrT1%2FeIhXaeUKASG%2BMsf9qsCyOXdnve6%2BsF1E0on68%3D&reserved=0>.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
