Hi Vivekananthan, Can we create a Jira and follow up on this please?
On Mon, Mar 14, 2016 at 10:40 AM, Kasun Indrasiri <[email protected]> wrote: > > > On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva <[email protected]> wrote: > >> Hi Kasun, >> >> I don't think we have the same functionality of VFS sender in file >> connector. We have only focused on use cases not covered with VFS transport. >> > > Ok. I think we need to make sure things are consistent across the > transport and the file connector. Can we please include this as an > improvement for the next version of the connector. > > >> > @Vivekananthan - Please confirm this? >> >> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri <[email protected]> wrote: >> >>> This is great. Thanks a lot Malaka. >>> Also, if we use the file connector as the outbound channel, will it work >>> in the same way? >>> >>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> Please ignore my previous comments. This can be done with current >>>> implementation. >>>> >>>> if (builder instanceof DataSourceMessageBuilder && >>>> "true".equals(streaming)) { >>>> dataSource = ManagedDataSourceFactory.create(new >>>> FileObjectDataSource(file, contentType)); >>>> in = null; >>>> } else { >>>> in = new >>>> AutoCloseInputStream(file.getContent().getInputStream()); >>>> dataSource = null; >>>> } >>>> .............. >>>> OMElement documentElement; >>>> if (in != null) { >>>> documentElement = builder.processDocument(in, >>>> contentType, axis2MsgCtx); >>>> } else { >>>> documentElement = >>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource, >>>> contentType, axis2MsgCtx); >>>> } >>>> ........... >>>> if(dataSource != null) { >>>> dataSource.destroy(); >>>> } >>>> >>>> This is >>>> because >>>> message builder able to build messages from >>>> >>>> DataSource objects. >>>> >>>> >>>> DataSource >>>> by definition the data from a DataSource can be read >>>> >>>> multiple times, >>>> builders that implement >>>> this interface >>>> can >>>> avoid storing the message content in memory. >>>> >>>> If a message builder implements this interface and the >>>> file/vfs >>>> is able to provide the message payload as a data source, then the >>>> method defined by this interface should be preferred over the method >>>> defined by Builder. >>>> This >>>> helps optimizing >>>> PT >>>> with >>>> vfs/file. >>>> The builder will typically expose the data source directly or >>>> indirectly through the returned OMElement, e.g. by adding to the tree an >>>> OMText or OMDataSource node referencing the data source. >>>> >>>> I have checked this with inbound but there is a fix we need to do. I >>>> have done it in [1]. I have done several tests with and without streaming >>>> from 5mb to 1gb files. >>>> >>>> Without streaming [2] and [3] will show the memory growth. With >>>> streaming [4]. Without streaming most of the time ESB went OOM. >>>> >>>> Find the related configs and axis2 changes in [5] and [6]. >>>> >>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458 >>>> >>>> [2] >>>> >>>> [image: Inline image 1] >>>> >>>> [3] >>>> >>>> [image: Inline image 2] >>>> >>>> [4] >>>> >>>> [image: Inline image 3] >>>> >>>> [5] >>>> >>>> <inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"; name="load" >>>> sequence="request" onError="fault" protocol="file" suspend="false"> >>>> <parameters> >>>> <parameter name="interval">10000</parameter> >>>> <parameter name="coordination">true</parameter> >>>> <parameter >>>> name="transport.vfs.ContentType">application/file</parameter> >>>> <parameter >>>> name="transport.vfs.LockReleaseSameNode">false</parameter> >>>> <parameter name="transport.vfs.AutoLockRelease">false</parameter> >>>> <parameter >>>> name="transport.vfs.ActionAfterFailure">DELETE</parameter> >>>> <parameter name="transport.vfs.CreateFolder">true</parameter> >>>> <parameter name="sequential">true</parameter> >>>> <parameter >>>> name="transport.vfs.ActionAfterProcess">DELETE</parameter> >>>> <parameter >>>> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in</parameter> >>>> <parameter name="transport.vfs.DistributedLock">false</parameter> >>>> <parameter name="transport.vfs.Streaming">true</parameter> >>>> <parameter name="transport.vfs.Locking">enable</parameter> >>>> <parameter name="transport.vfs.FileSortAscending">true</parameter> >>>> <parameter name="transport.vfs.FileSortAttribute">NONE</parameter> >>>> <parameter name="transport.vfs.Build">false</parameter> >>>> </parameters> >>>> </inboundEndpoint> >>>> >>>> <endpoint name="fileout"> >>>> <address uri="vfs:file:///home/wso2/work/tmp/file/out"/> >>>> </endpoint> >>>> <sequence name="request" onError="fault"> >>>> <property name="OUT_ONLY" scope="default" type="STRING" >>>> value="true"/> >>>> <send> >>>> <endpoint key="fileout"/> >>>> </send> >>>> </sequence> >>>> >>>> [6] >>>> <messageBuilder contentType="application/file" >>>> class="org.apache.axis2.format.BinaryBuilder"/> >>>> >>>> <messageFormatter contentType="application/file" >>>> >>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/> >>>> >>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva <[email protected]> wrote: >>>> >>>>> Hi Kasun, >>>>> >>>>> Currently no OOB solution with ESB 4.9.0. File always gets build >>>>> before mediation. >>>>> >>>>> However use case mentioned can be handled with Schedule task -> File >>>>> connector search -> File connector copy >>>>> >>>>> +1 for OOB solution. >>>>> >>>>> We can do this for ESB 5.0. Also there is an option to do this as a >>>>> custom inbound. So that this can be used by previous ESB versions as well. >>>>> >>>>> So default inbound can be used, if someone needs to do a mediation on >>>>> file content and custom inbound for PT file use case. WDYT? >>>>> >>>>> >>>>> On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Malaka, >>>>>> >>>>>> Do we support the $subject? Basically we use an Inbound as the source >>>>>> and use file connector as the destination. If this is not supported yet, >>>>>> we >>>>>> got to add this to ESB 5. >>>>>> >>>>>> Thanks, >>>>>> Kasun >>>>>> >>>>>> -- >>>>>> Kasun Indrasiri >>>>>> Software Architect >>>>>> WSO2, Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> cell: +94 77 556 5206 >>>>>> Blog : http://kasunpanorama.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Best Regards, >>>>> >>>>> Malaka Silva >>>>> Senior Tech Lead >>>>> M: +94 777 219 791 >>>>> Tel : 94 11 214 5345 >>>>> Fax :94 11 2145300 >>>>> Skype : malaka.sampath.silva >>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>> >>>>> WSO2, Inc. >>>>> lean . enterprise . middleware >>>>> http://www.wso2.com/ >>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>> <http://wso2.com/about/team/malaka-silva/> >>>>> https://store.wso2.com/store/ >>>>> >>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>> this email only if it is absolutely necessary. >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Best Regards, >>>> >>>> Malaka Silva >>>> Senior Tech Lead >>>> M: +94 777 219 791 >>>> Tel : 94 11 214 5345 >>>> Fax :94 11 2145300 >>>> Skype : malaka.sampath.silva >>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>> Blog : http://mrmalakasilva.blogspot.com/ >>>> >>>> WSO2, Inc. >>>> lean . enterprise . middleware >>>> http://www.wso2.com/ >>>> http://www.wso2.com/about/team/malaka-silva/ >>>> <http://wso2.com/about/team/malaka-silva/> >>>> https://store.wso2.com/store/ >>>> >>>> Save a tree -Conserve nature & Save the world for your future. Print >>>> this email only if it is absolutely necessary. >>>> >>> >>> >>> >>> -- >>> Kasun Indrasiri >>> Software Architect >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> cell: +94 77 556 5206 >>> Blog : http://kasunpanorama.blogspot.com/ >>> >> >> >> >> -- >> >> Best Regards, >> >> Malaka Silva >> Senior Tech Lead >> M: +94 777 219 791 >> Tel : 94 11 214 5345 >> Fax :94 11 2145300 >> Skype : malaka.sampath.silva >> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >> Blog : http://mrmalakasilva.blogspot.com/ >> >> WSO2, Inc. >> lean . enterprise . middleware >> http://www.wso2.com/ >> http://www.wso2.com/about/team/malaka-silva/ >> <http://wso2.com/about/team/malaka-silva/> >> https://store.wso2.com/store/ >> >> Save a tree -Conserve nature & Save the world for your future. Print this >> email only if it is absolutely necessary. >> > > > > -- > Kasun Indrasiri > Software Architect > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > cell: +94 77 556 5206 > Blog : http://kasunpanorama.blogspot.com/ > -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ <http://wso2.com/about/team/malaka-silva/> https://store.wso2.com/store/ Save a tree -Conserve nature & Save the world for your future. Print this email only if it is absolutely necessary.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
