No I don't think we are restricting Malhar to just abstract classes. Whenever they are couple of use cases that we see quite often, we add concrete implementations.
For eg. FileLineInputOperator which is a concrete implementation of AbstractFileInputOperator. FSSliceReader is an example as well. In AbstractJdbcOutputOperator case there hasn't been such common insert/update query. Also if you look at the example I provided, it is very simple to provide a concrete implementation. If you would like to change JdbcPOJOOutputOperator to work for "UPDATE/MERGE" then please go ahead. Chandni On Sun, Dec 13, 2015 at 8:47 PM, Bhupesh Chawda <[email protected]> wrote: > I see. So, just to understand more, do we plan to keep Malhar restricted to > the base functionality (as in abstract classes)? And put the configuration > aspect / concrete implementations in apps that use these operators? > > Thanks. > Bhupesh > > On Sat, Dec 12, 2015 at 5:43 AM, Chandni Singh <[email protected]> > wrote: > > > Hi, > > > > Here is an example of doing Upsert with JDBC: > > > > > https://github.com/chandnisingh/Malhar/blob/examples/apps/jdbc/src/main/java/com/datatorrent/jdbc/JdbcWriter.java > > > > Thanks, > > Chandni > > > > On Fri, Dec 11, 2015 at 11:19 AM, Chandni Singh <[email protected] > > > > wrote: > > > > > The operators are under Malhar/lib/db/jdbc. > > > > > > Here is one of them: > > > > > > > > > https://github.com/apache/incubator-apex-malhar/blob/devel-3/library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcTransactionableOutputOperator.java > > > > > > They work with any kind PreparedStatement - insert or update > > > > > > Chandni > > > > > > > > > > > > On Fri, Dec 11, 2015 at 10:59 AM, Bhupesh Chawda < > > [email protected]> > > > wrote: > > > > > >> Hi Chandni, > > >> > > >> I don't see an update query being handled in the operator. Could you > > >> please > > >> point me to the appropriate class? > > >> Or did you mean that handling a update query is just a matter of > > extending > > >> the class and providing a concrete implementation? > > >> > > >> Thanks. > > >> -Bhupesh > > >> > > >> On Fri, Dec 11, 2015 at 10:42 PM, Chandni Singh < > > [email protected]> > > >> wrote: > > >> > > >> > Hi Bhupesh, > > >> > > > >> > The current abstract JDBC Output Operators in library are generic > and > > >> have > > >> > already been used in multiple POCs and applications. In fact this > > >> operator > > >> > has matured through customer use cases. It is not just an insert > > >> operator. > > >> > We have used it to perform update and inserts. > > >> > > > >> > That said, I don't think it is a good idea to introduce input format > > in > > >> > these abstract implementations. It is written to handle any type of > > >> query, > > >> > be it a procedure call (that was an actual customer use case). > > >> > > > >> > Chandni > > >> > > > >> > On Fri, Dec 11, 2015 at 2:50 AM, Bhupesh Chawda < > > >> [email protected]> > > >> > wrote: > > >> > > > >> > > Hi All, > > >> > > > > >> > > We are planning to proceed with the following approach for JDBC > > >> *update* > > >> > > operator: > > >> > > > > >> > > - *Update Query Configuration* > > >> > > > > >> > > > > >> > > - Example Update Query: *update tableName set a = ?** where b > = ? > > >> and > > >> > c > > >> > > > ?;* > > >> > > - Example JSON input array for parameter instantiations: *[{a, > > >> > > expression, INTEGER}, {b, expression, VARCHAR}, {c, expression, > > >> > DATE}]* > > >> > > > > >> > > We are also planning to change the JDBC Output Operator in Malhar > > >> Library > > >> > > which currently does just insert. We plan to make the input format > > >> > > consistent for both insert and update and hence the change to the > > >> current > > >> > > way of configuration using JSON. Following would be the config for > > >> > inserts: > > >> > > > > >> > > - *Insert Query Configuration* > > >> > > > > >> > > > > >> > > - Example Insert Query: *insert into tableName values (?, ?, > .. , > > >> ?);* > > >> > > - Example JSON input array for parameter instantiations: *[{a, > > >> > > expression, INTEGER}, {b, expression, VARCHAR}, .. , {c, > > >> expression, > > >> > > DATE}]* > > >> > > > > >> > > Please let us know your thoughts. > > >> > > > > >> > > Thanks. > > >> > > > > >> > > -Bhupesh > > >> > > > > >> > > > > >> > > > > >> > > On Wed, Dec 9, 2015 at 6:38 PM, Bhupesh Chawda < > > >> [email protected]> > > >> > > wrote: > > >> > > > > >> > > > Hi All, > > >> > > > > > >> > > > Would it be a good idea to introduce the update functionality to > > the > > >> > JDBC > > >> > > > output operator in Apache Apex Malhar library. > > >> > > > > > >> > > > The following are possible approaches: > > >> > > > > > >> > > > 1. Accept a update query from the user with place holders for > > >> > values. > > >> > > > Example: *update tableName set a = ?, b = ? where c = ? and > d > > > >> ?*. > > >> > > > Here "?" will be provided by the user as java expressions > which > > >> will > > >> > > be > > >> > > > evaluated from the incoming tuple. > > >> > > > 2. Another option is to accept in some configuration format > > >> (json / > > >> > > > xml) the following and formulate the query in the operator. > > This > > >> can > > >> > > become > > >> > > > arbitrarily complex. > > >> > > > 1. update clause columns > > >> > > > 2. update clause expressions > > >> > > > 3. where clause columns > > >> > > > 4. where clause expressions > > >> > > > > > >> > > > I am thinking about going ahead with 1. Please let me know if > any > > >> other > > >> > > > option is possible and whether such a functionality already > exists > > >> in > > >> > > some > > >> > > > other class. > > >> > > > > > >> > > > Thanks. > > >> > > > > > >> > > > -Bhupesh > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
