Hi Malaka, I have created Jira [1].
[1] https://wso2.org/jira/browse/ESBCONNECT-86 *Thank youVivekananthan Sivanayagam* *Associate Software Engineer | WSO2* *E:[email protected] <e%[email protected]>* *M:+94752786138* On Mon, Mar 14, 2016 at 10:59 AM, Malaka Silva <[email protected]> wrote: > Hi Vivekananthan, > > Can we create a Jira and follow up on this please? > > On Mon, Mar 14, 2016 at 10:40 AM, Kasun Indrasiri <[email protected]> wrote: > >> >> >> On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva <[email protected]> wrote: >> >>> Hi Kasun, >>> >>> I don't think we have the same functionality of VFS sender in file >>> connector. We have only focused on use cases not covered with VFS transport. >>> >> >> Ok. I think we need to make sure things are consistent across the >> transport and the file connector. Can we please include this as an >> improvement for the next version of the connector. >> >> >>> >> @Vivekananthan - Please confirm this? >>> >>> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri <[email protected]> wrote: >>> >>>> This is great. Thanks a lot Malaka. >>>> Also, if we use the file connector as the outbound channel, will it >>>> work in the same way? >>>> >>>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva <[email protected]> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Please ignore my previous comments. This can be done with current >>>>> implementation. >>>>> >>>>> if (builder instanceof DataSourceMessageBuilder && >>>>> "true".equals(streaming)) { >>>>> dataSource = ManagedDataSourceFactory.create(new >>>>> FileObjectDataSource(file, contentType)); >>>>> in = null; >>>>> } else { >>>>> in = new >>>>> AutoCloseInputStream(file.getContent().getInputStream()); >>>>> dataSource = null; >>>>> } >>>>> .............. >>>>> OMElement documentElement; >>>>> if (in != null) { >>>>> documentElement = builder.processDocument(in, >>>>> contentType, axis2MsgCtx); >>>>> } else { >>>>> documentElement = >>>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource, >>>>> contentType, axis2MsgCtx); >>>>> } >>>>> ........... >>>>> if(dataSource != null) { >>>>> dataSource.destroy(); >>>>> } >>>>> >>>>> This is >>>>> because >>>>> message builder able to build messages from >>>>> >>>>> DataSource objects. >>>>> >>>>> >>>>> DataSource >>>>> by definition the data from a DataSource can be read >>>>> >>>>> multiple times, >>>>> builders that implement >>>>> this interface >>>>> can >>>>> avoid storing the message content in memory. >>>>> >>>>> If a message builder implements this interface and the >>>>> file/vfs >>>>> is able to provide the message payload as a data source, then the >>>>> method defined by this interface should be preferred over the method >>>>> defined by Builder. >>>>> This >>>>> helps optimizing >>>>> PT >>>>> with >>>>> vfs/file. >>>>> The builder will typically expose the data source directly or >>>>> indirectly through the returned OMElement, e.g. by adding to the tree an >>>>> OMText or OMDataSource node referencing the data source. >>>>> >>>>> I have checked this with inbound but there is a fix we need to do. I >>>>> have done it in [1]. I have done several tests with and without streaming >>>>> from 5mb to 1gb files. >>>>> >>>>> Without streaming [2] and [3] will show the memory growth. With >>>>> streaming [4]. Without streaming most of the time ESB went OOM. >>>>> >>>>> Find the related configs and axis2 changes in [5] and [6]. >>>>> >>>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458 >>>>> >>>>> [2] >>>>> >>>>> [image: Inline image 1] >>>>> >>>>> [3] >>>>> >>>>> [image: Inline image 2] >>>>> >>>>> [4] >>>>> >>>>> [image: Inline image 3] >>>>> >>>>> [5] >>>>> >>>>> <inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"; name="load" >>>>> sequence="request" onError="fault" protocol="file" suspend="false"> >>>>> <parameters> >>>>> <parameter name="interval">10000</parameter> >>>>> <parameter name="coordination">true</parameter> >>>>> <parameter >>>>> name="transport.vfs.ContentType">application/file</parameter> >>>>> <parameter >>>>> name="transport.vfs.LockReleaseSameNode">false</parameter> >>>>> <parameter name="transport.vfs.AutoLockRelease">false</parameter> >>>>> <parameter >>>>> name="transport.vfs.ActionAfterFailure">DELETE</parameter> >>>>> <parameter name="transport.vfs.CreateFolder">true</parameter> >>>>> <parameter name="sequential">true</parameter> >>>>> <parameter >>>>> name="transport.vfs.ActionAfterProcess">DELETE</parameter> >>>>> <parameter >>>>> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in</parameter> >>>>> <parameter name="transport.vfs.DistributedLock">false</parameter> >>>>> <parameter name="transport.vfs.Streaming">true</parameter> >>>>> <parameter name="transport.vfs.Locking">enable</parameter> >>>>> <parameter >>>>> name="transport.vfs.FileSortAscending">true</parameter> >>>>> <parameter >>>>> name="transport.vfs.FileSortAttribute">NONE</parameter> >>>>> <parameter name="transport.vfs.Build">false</parameter> >>>>> </parameters> >>>>> </inboundEndpoint> >>>>> >>>>> <endpoint name="fileout"> >>>>> <address uri="vfs:file:///home/wso2/work/tmp/file/out"/> >>>>> </endpoint> >>>>> <sequence name="request" onError="fault"> >>>>> <property name="OUT_ONLY" scope="default" type="STRING" >>>>> value="true"/> >>>>> <send> >>>>> <endpoint key="fileout"/> >>>>> </send> >>>>> </sequence> >>>>> >>>>> [6] >>>>> <messageBuilder contentType="application/file" >>>>> class="org.apache.axis2.format.BinaryBuilder"/> >>>>> >>>>> <messageFormatter contentType="application/file" >>>>> >>>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/> >>>>> >>>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Kasun, >>>>>> >>>>>> Currently no OOB solution with ESB 4.9.0. File always gets build >>>>>> before mediation. >>>>>> >>>>>> However use case mentioned can be handled with Schedule task -> File >>>>>> connector search -> File connector copy >>>>>> >>>>>> +1 for OOB solution. >>>>>> >>>>>> We can do this for ESB 5.0. Also there is an option to do this as a >>>>>> custom inbound. So that this can be used by previous ESB versions as >>>>>> well. >>>>>> >>>>>> So default inbound can be used, if someone needs to do a mediation on >>>>>> file content and custom inbound for PT file use case. WDYT? >>>>>> >>>>>> >>>>>> On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Malaka, >>>>>>> >>>>>>> Do we support the $subject? Basically we use an Inbound as the >>>>>>> source and use file connector as the destination. If this is not >>>>>>> supported >>>>>>> yet, we got to add this to ESB 5. >>>>>>> >>>>>>> Thanks, >>>>>>> Kasun >>>>>>> >>>>>>> -- >>>>>>> Kasun Indrasiri >>>>>>> Software Architect >>>>>>> WSO2, Inc.; http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> >>>>>>> cell: +94 77 556 5206 >>>>>>> Blog : http://kasunpanorama.blogspot.com/ >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Best Regards, >>>>>> >>>>>> Malaka Silva >>>>>> Senior Tech Lead >>>>>> M: +94 777 219 791 >>>>>> Tel : 94 11 214 5345 >>>>>> Fax :94 11 2145300 >>>>>> Skype : malaka.sampath.silva >>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>>> >>>>>> WSO2, Inc. >>>>>> lean . enterprise . middleware >>>>>> http://www.wso2.com/ >>>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>>> <http://wso2.com/about/team/malaka-silva/> >>>>>> https://store.wso2.com/store/ >>>>>> >>>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>>> this email only if it is absolutely necessary. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Best Regards, >>>>> >>>>> Malaka Silva >>>>> Senior Tech Lead >>>>> M: +94 777 219 791 >>>>> Tel : 94 11 214 5345 >>>>> Fax :94 11 2145300 >>>>> Skype : malaka.sampath.silva >>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>> >>>>> WSO2, Inc. >>>>> lean . enterprise . middleware >>>>> http://www.wso2.com/ >>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>> <http://wso2.com/about/team/malaka-silva/> >>>>> https://store.wso2.com/store/ >>>>> >>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>> this email only if it is absolutely necessary. >>>>> >>>> >>>> >>>> >>>> -- >>>> Kasun Indrasiri >>>> Software Architect >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> cell: +94 77 556 5206 >>>> Blog : http://kasunpanorama.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> >>> Best Regards, >>> >>> Malaka Silva >>> Senior Tech Lead >>> M: +94 777 219 791 >>> Tel : 94 11 214 5345 >>> Fax :94 11 2145300 >>> Skype : malaka.sampath.silva >>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>> Blog : http://mrmalakasilva.blogspot.com/ >>> >>> WSO2, Inc. >>> lean . enterprise . middleware >>> http://www.wso2.com/ >>> http://www.wso2.com/about/team/malaka-silva/ >>> <http://wso2.com/about/team/malaka-silva/> >>> https://store.wso2.com/store/ >>> >>> Save a tree -Conserve nature & Save the world for your future. Print >>> this email only if it is absolutely necessary. >>> >> >> >> >> -- >> Kasun Indrasiri >> Software Architect >> WSO2, Inc.; http://wso2.com >> lean.enterprise.middleware >> >> cell: +94 77 556 5206 >> Blog : http://kasunpanorama.blogspot.com/ >> > > > > -- > > Best Regards, > > Malaka Silva > Senior Tech Lead > M: +94 777 219 791 > Tel : 94 11 214 5345 > Fax :94 11 2145300 > Skype : malaka.sampath.silva > LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 > Blog : http://mrmalakasilva.blogspot.com/ > > WSO2, Inc. > lean . enterprise . middleware > http://www.wso2.com/ > http://www.wso2.com/about/team/malaka-silva/ > <http://wso2.com/about/team/malaka-silva/> > https://store.wso2.com/store/ > > Save a tree -Conserve nature & Save the world for your future. Print this > email only if it is absolutely necessary. >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
