Oh, wonderful!
NIFI will be the golden bridge over the data lake!

Date: Fri, 4 Dec 2015 07:07:16 -0700
From: [email protected]
To: [email protected]
Subject: Re: PutSQL : feature enhancement - PutPropertyToSQL



        Bob


Please don’t feel like your question about Hive was ignored. It is a very valid 
question that took some time for us to come to terms with ;)

Yes indeed we are thinking about it as we have similar problem with JMS 
providers. The core issue here is the class path of all that is required by the 
driver and indeed Hive requires a lot.

There is an issue https://issues.apache.org/jira/browse/NIFI-981 where this is 
already been discussed. We are also considering a raw ORC sink instead of 
interacting with Hive via JDBC etc.

The bad news; As of right now we don’t have much to offer

The good news; We certainly plan to have that in 0.5.0 and I am hoping that 
even before 0.5 will have something in the snapshot that you can try.


So, you won’t need to wait for 1.0, but you may need to wait a little bit ;)


Cheers

Oleg


On Dec 1, 2015, at 6:30 PM, Bob ZHao <[hidden email]<mailto:[hidden email]>> 
wrote:


Hi Mark,


Thanks for the workaround. This ETL trick always works! Only not sure for

SQL injection.


Do you have any idea  about "configure the DBCP connection pool for hive" ?

It's painful and we can't wait until 1.0 release.


Thanks,

Bob



Mark Payne wrote

You can also potentially use the ReplaceText processor to build a SQL

statement

using the Expression Language, something like:


INSERT INTO MyTable (col1, col2) VALUES ('${attr1}', '${attr2}')


-Mark



On Dec 1, 2015, at 5:10 PM, Bryan Bende &lt;


bbende@


&gt; wrote:


Bob,


I think this is slightly different than what you are suggesting, but in

the

next release there is a new AttributesToJson processor makes it very easy

to construct a new json document from values that were extracted using

EvaluateJsonPath.


Would this makes things a bit easier for sending json to

ConvertJsonToSql?


-Bryan


On Tue, Dec 1, 2015 at 4:14 PM, Bob ZHao &lt;


bob.zhao.work@


&gt; wrote:


Hello,


I like the idea to support more on tabular data target. But it might

take a

long way to go.

I have a quick enhancement suggestion.


*COnvertJSONToSQL* only support flat JSON structure instead of the

complicated format -- this make sense. But since we can use

*EvaluateJsonPath* to filter our the target fields, could we combine

them

together for a new one *PutPropertyToSQL*? This could be very quick

implementation.


NIFI needn't to worry about the complex of Json,  user can use JsonPath

to

chop the fields and fragments for customization.

Then put the result to target table. BTW, please use a trick to save the

raw

Json - by pointing to a column.


Thanks,

Bob





--

View this message in context:

http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536.html
Sent from the Apache NiFi Developer List mailing list archive at

Nabble.com.







--

View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536p5541.html
Sent from the Apache NiFi Developer List mailing list archive at 
Nabble.com<http://Nabble.com>.





        
        
        
        

        

        
        
                If you reply to this email, your message will be added to the 
discussion below:
                
http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536p5599.html
        
        
                
                To unsubscribe from PutSQL : feature enhancement - 
PutPropertyToSQL, click here.

                NAML
                                                  



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/PutSQL-feature-enhancement-PutPropertyToSQL-tp5536p5695.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Reply via email to