Hi team, We need expert input to discuss how to implement Rule engine in hive. Do you have any references available to implement rule in hive/pig.
We are migrating our Stored Procedures into Multiple Hive query but it is becoming complex in maintenance, Hive is not Procedural Language, so we could not write IF ELSE logic or any kind of procedural language. Can any one suggest us which HDP technology can be helpful for Procedural language replacement, we are thinking of PIG, Can it becomes best example to perform rule/data transformation. Our data is in Structure format table with around 250 columns, We are having Rules like Update Columns based on lookup table values, delete rows which doesn't satisfy the condition, update statement include update of multiple columns like case etc and some date conversions columns. Suggest us best way to implement this rule engine. previously we have used our SQL Server for Rule engine, now we have migrated application to Big Data, we are looking for any references available to perform this rule transformation. Some of our Finding are Make use of Hive StreamWriting PIG. We are .NET developers, Can we think of writing .EXE application and stream row wise data to .EXE and apply rules on top of each row. Will it be nicer solution or Is it better to implement in PIG, but implementing doesn't fetch me much benefit when compared with Hive. Can you please comment on above approaches please. Thanks, Mahender
