Hi Kasun, I don't think we need to cover any functionality that is already available with ESB. Any use case that we need the vfs sender from connector?
On Mon, Mar 14, 2016 at 10:40 AM, Kasun Indrasiri <[email protected]> wrote: > > > On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva <[email protected]> wrote: > >> Hi Kasun, >> >> I don't think we have the same functionality of VFS sender in file >> connector. We have only focused on use cases not covered with VFS transport. >> > > Ok. I think we need to make sure things are consistent across the > transport and the file connector. Can we please include this as an > improvement for the next version of the connector. > > >> > @Vivekananthan - Please confirm this? >> >> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri <[email protected]> wrote: >> >>> This is great. Thanks a lot Malaka. >>> Also, if we use the file connector as the outbound channel, will it work >>> in the same way? >>> >>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> Please ignore my previous comments. This can be done with current >>>> implementation. >>>> >>>> if (builder instanceof DataSourceMessageBuilder && >>>> "true".equals(streaming)) { >>>> dataSource = ManagedDataSourceFactory.create(new >>>> FileObjectDataSource(file, contentType)); >>>> in = null; >>>> } else { >>>> in = new >>>> AutoCloseInputStream(file.getContent().getInputStream()); >>>> dataSource = null; >>>> } >>>> .............. >>>> OMElement documentElement; >>>> if (in != null) { >>>> documentElement = builder.processDocument(in, >>>> contentType, axis2MsgCtx); >>>> } else { >>>> documentElement = >>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource, >>>> contentType, axis2MsgCtx); >>>> } >>>> ........... >>>> if(dataSource != null) { >>>> dataSource.destroy(); >>>> } >>>> >>>> This is >>>> because >>>> message builder able to build messages from >>>> >>>> DataSource objects. >>>> >>>> >>>> DataSource >>>> by definition the data from a DataSource can be read >>>> >>>> multiple times, >>>> builders that implement >>>> this interface >>>> can >>>> avoid storing the message content in memory. >>>> >>>> If a message builder implements this interface and the >>>> file/vfs >>>> is able to provide the message payload as a data source, then the >>>> method defined by this interface should be preferred over the method >>>> defined by Builder. >>>> This >>>> helps optimizing >>>> PT >>>> with >>>> vfs/file. >>>> The builder will typically expose the data source directly or >>>> indirectly through the returned OMElement, e.g. by adding to the tree an >>>> OMText or OMDataSource node referencing the data source. >>>> >>>> I have checked this with inbound but there is a fix we need to do. I >>>> have done it in [1]. I have done several tests with and without streaming >>>> from 5mb to 1gb files. >>>> >>>> Without streaming [2] and [3] will show the memory growth. With >>>> streaming [4]. Without streaming most of the time ESB went OOM. >>>> >>>> Find the related configs and axis2 changes in [5] and [6]. >>>> >>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458 >>>> >>>> [2] >>>> >>>> [image: Inline image 1] >>>> >>>> [3] >>>> >>>> [image: Inline image 2] >>>> >>>> [4] >>>> >>>> [image: Inline image 3] >>>> >>>> [5] >>>> >>>> <inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"; name="load" >>>> sequence="request" onError="fault" protocol="file" suspend="false"> >>>> <parameters> >>>> <parameter name="interval">10000</parameter> >>>> <parameter name="coordination">true</parameter> >>>> <parameter >>>> name="transport.vfs.ContentType">application/file</parameter> >>>> <parameter >>>> name="transport.vfs.LockReleaseSameNode">false</parameter> >>>> <parameter name="transport.vfs.AutoLockRelease">false</parameter> >>>> <parameter >>>> name="transport.vfs.ActionAfterFailure">DELETE</parameter> >>>> <parameter name="transport.vfs.CreateFolder">true</parameter> >>>> <parameter name="sequential">true</parameter> >>>> <parameter >>>> name="transport.vfs.ActionAfterProcess">DELETE</parameter> >>>> <parameter >>>> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in</parameter> >>>> <parameter name="transport.vfs.DistributedLock">false</parameter> >>>> <parameter name="transport.vfs.Streaming">true</parameter> >>>> <parameter name="transport.vfs.Locking">enable</parameter> >>>> <parameter name="transport.vfs.FileSortAscending">true</parameter> >>>> <parameter name="transport.vfs.FileSortAttribute">NONE</parameter> >>>> <parameter name="transport.vfs.Build">false</parameter> >>>> </parameters> >>>> </inboundEndpoint> >>>> >>>> <endpoint name="fileout"> >>>> <address uri="vfs:file:///home/wso2/work/tmp/file/out"/> >>>> </endpoint> >>>> <sequence name="request" onError="fault"> >>>> <property name="OUT_ONLY" scope="default" type="STRING" >>>> value="true"/> >>>> <send> >>>> <endpoint key="fileout"/> >>>> </send> >>>> </sequence> >>>> >>>> [6] >>>> <messageBuilder contentType="application/file" >>>> class="org.apache.axis2.format.BinaryBuilder"/> >>>> >>>> <messageFormatter contentType="application/file" >>>> >>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/> >>>> >>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva <[email protected]> wrote: >>>> >>>>> Hi Kasun, >>>>> >>>>> Currently no OOB solution with ESB 4.9.0. File always gets build >>>>> before mediation. >>>>> >>>>> However use case mentioned can be handled with Schedule task -> File >>>>> connector search -> File connector copy >>>>> >>>>> +1 for OOB solution. >>>>> >>>>> We can do this for ESB 5.0. Also there is an option to do this as a >>>>> custom inbound. So that this can be used by previous ESB versions as well. >>>>> >>>>> So default inbound can be used, if someone needs to do a mediation on >>>>> file content and custom inbound for PT file use case. WDYT? >>>>> >>>>> >>>>> On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Malaka, >>>>>> >>>>>> Do we support the $subject? Basically we use an Inbound as the source >>>>>> and use file connector as the destination. If this is not supported yet, >>>>>> we >>>>>> got to add this to ESB 5. >>>>>> >>>>>> Thanks, >>>>>> Kasun >>>>>> >>>>>> -- >>>>>> Kasun Indrasiri >>>>>> Software Architect >>>>>> WSO2, Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> cell: +94 77 556 5206 >>>>>> Blog : http://kasunpanorama.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Best Regards, >>>>> >>>>> Malaka Silva >>>>> Senior Tech Lead >>>>> M: +94 777 219 791 >>>>> Tel : 94 11 214 5345 >>>>> Fax :94 11 2145300 >>>>> Skype : malaka.sampath.silva >>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>> >>>>> WSO2, Inc. >>>>> lean . enterprise . middleware >>>>> http://www.wso2.com/ >>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>> <http://wso2.com/about/team/malaka-silva/> >>>>> https://store.wso2.com/store/ >>>>> >>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>> this email only if it is absolutely necessary. >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Best Regards, >>>> >>>> Malaka Silva >>>> Senior Tech Lead >>>> M: +94 777 219 791 >>>> Tel : 94 11 214 5345 >>>> Fax :94 11 2145300 >>>> Skype : malaka.sampath.silva >>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>> Blog : http://mrmalakasilva.blogspot.com/ >>>> >>>> WSO2, Inc. >>>> lean . enterprise . middleware >>>> http://www.wso2.com/ >>>> http://www.wso2.com/about/team/malaka-silva/ >>>> <http://wso2.com/about/team/malaka-silva/> >>>> https://store.wso2.com/store/ >>>> >>>> Save a tree -Conserve nature & Save the world for your future. Print >>>> this email only if it is absolutely necessary. >>>> >>> >>> >>> >>> -- >>> Kasun Indrasiri >>> Software Architect >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> cell: +94 77 556 5206 >>> Blog : http://kasunpanorama.blogspot.com/ >>> >> >> >> >> -- >> >> Best Regards, >> >> Malaka Silva >> Senior Tech Lead >> M: +94 777 219 791 >> Tel : 94 11 214 5345 >> Fax :94 11 2145300 >> Skype : malaka.sampath.silva >> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >> Blog : http://mrmalakasilva.blogspot.com/ >> >> WSO2, Inc. >> lean . enterprise . middleware >> http://www.wso2.com/ >> http://www.wso2.com/about/team/malaka-silva/ >> <http://wso2.com/about/team/malaka-silva/> >> https://store.wso2.com/store/ >> >> Save a tree -Conserve nature & Save the world for your future. Print this >> email only if it is absolutely necessary. >> > > > > -- > Kasun Indrasiri > Software Architect > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > cell: +94 77 556 5206 > Blog : http://kasunpanorama.blogspot.com/ > -- Best Regards, Malaka Silva Senior Tech Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware http://www.wso2.com/ http://www.wso2.com/about/team/malaka-silva/ <http://wso2.com/about/team/malaka-silva/> https://store.wso2.com/store/ Save a tree -Conserve nature & Save the world for your future. Print this email only if it is absolutely necessary.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
