Hi Kasun,
I don't think we have the same functionality of VFS sender in file
connector. We have only focused on use cases not covered with VFS transport.
@Vivekananthan - Please confirm this?
On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri <ka...@wso2.com> wrote:
> This is great. Thanks a lot Malaka.
> Also, if we use the file connector as the outbound channel, will it work
> in the same way?
>
> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva <mal...@wso2.com> wrote:
>
>> Hi All,
>>
>> Please ignore my previous comments. This can be done with current
>> implementation.
>>
>> if (builder instanceof DataSourceMessageBuilder &&
>> "true".equals(streaming)) {
>> dataSource = ManagedDataSourceFactory.create(new
>> FileObjectDataSource(file, contentType));
>> in = null;
>> } else {
>> in = new
>> AutoCloseInputStream(file.getContent().getInputStream());
>> dataSource = null;
>> }
>> ..............
>> OMElement documentElement;
>> if (in != null) {
>> documentElement = builder.processDocument(in,
>> contentType, axis2MsgCtx);
>> } else {
>> documentElement =
>> ((DataSourceMessageBuilder)builder).processDocument(dataSource,
>> contentType, axis2MsgCtx);
>> }
>> ...........
>> if(dataSource != null) {
>> dataSource.destroy();
>> }
>>
>> This is
>> because
>> message builder able to build messages from
>>
>> DataSource objects.
>>
>>
>> DataSource
>> by definition the data from a DataSource can be read
>>
>> multiple times,
>> builders that implement
>> this interface
>> can
>> avoid storing the message content in memory.
>>
>> If a message builder implements this interface and the
>> file/vfs
>> is able to provide the message payload as a data source, then the method
>> defined by this interface should be preferred over the method defined by
>> Builder.
>> This
>> helps optimizing
>> PT
>> with
>> vfs/file.
>> The builder will typically expose the data source directly or indirectly
>> through the returned OMElement, e.g. by adding to the tree an OMText or
>> OMDataSource node referencing the data source.
>>
>> I have checked this with inbound but there is a fix we need to do. I
>> have done it in [1]. I have done several tests with and without streaming
>> from 5mb to 1gb files.
>>
>> Without streaming [2] and [3] will show the memory growth. With streaming
>> [4]. Without streaming most of the time ESB went OOM.
>>
>> Find the related configs and axis2 changes in [5] and [6].
>>
>> [1] https://wso2.org/jira/browse/ESBJAVA-4458
>>
>> [2]
>>
>> [image: Inline image 1]
>>
>> [3]
>>
>> [image: Inline image 2]
>>
>> [4]
>>
>> [image: Inline image 3]
>>
>> [5]
>>
>> <inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"; name="load"
>> sequence="request" onError="fault" protocol="file" suspend="false">
>> <parameters>
>> <parameter name="interval">10000</parameter>
>> <parameter name="coordination">true</parameter>
>> <parameter
>> name="transport.vfs.ContentType">application/file</parameter>
>> <parameter
>> name="transport.vfs.LockReleaseSameNode">false</parameter>
>> <parameter name="transport.vfs.AutoLockRelease">false</parameter>
>> <parameter
>> name="transport.vfs.ActionAfterFailure">DELETE</parameter>
>> <parameter name="transport.vfs.CreateFolder">true</parameter>
>> <parameter name="sequential">true</parameter>
>> <parameter
>> name="transport.vfs.ActionAfterProcess">DELETE</parameter>
>> <parameter
>> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in</parameter>
>> <parameter name="transport.vfs.DistributedLock">false</parameter>
>> <parameter name="transport.vfs.Streaming">true</parameter>
>> <parameter name="transport.vfs.Locking">enable</parameter>
>> <parameter name="transport.vfs.FileSortAscending">true</parameter>
>> <parameter name="transport.vfs.FileSortAttribute">NONE</parameter>
>> <parameter name="transport.vfs.Build">false</parameter>
>> </parameters>
>> </inboundEndpoint>
>>
>> <endpoint name="fileout">
>> <address uri="vfs:file:///home/wso2/work/tmp/file/out"/>
>> </endpoint>
>> <sequence name="request" onError="fault">
>> <property name="OUT_ONLY" scope="default" type="STRING"
>> value="true"/>
>> <send>
>> <endpoint key="fileout"/>
>> </send>
>> </sequence>
>>
>> [6]
>> <messageBuilder contentType="application/file"
>> class="org.apache.axis2.format.BinaryBuilder"/>
>>
>> <messageFormatter contentType="application/file"
>>
>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
>>
>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva <mal...@wso2.com> wrote:
>>
>>> Hi Kasun,
>>>
>>> Currently no OOB solution with ESB 4.9.0. File always gets build before
>>> mediation.
>>>
>>> However use case mentioned can be handled with Schedule task -> File
>>> connector search -> File connector copy
>>>
>>> +1 for OOB solution.
>>>
>>> We can do this for ESB 5.0. Also there is an option to do this as a
>>> custom inbound. So that this can be used by previous ESB versions as well.
>>>
>>> So default inbound can be used, if someone needs to do a mediation on
>>> file content and custom inbound for PT file use case. WDYT?
>>>
>>>
>>> On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri <ka...@wso2.com>
>>> wrote:
>>>
>>>> Hi Malaka,
>>>>
>>>> Do we support the $subject? Basically we use an Inbound as the source
>>>> and use file connector as the destination. If this is not supported yet, we
>>>> got to add this to ESB 5.
>>>>
>>>> Thanks,
>>>> Kasun
>>>>
>>>> --
>>>> Kasun Indrasiri
>>>> Software Architect
>>>> WSO2, Inc.; http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> cell: +94 77 556 5206
>>>> Blog : http://kasunpanorama.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Best Regards,
>>>
>>> Malaka Silva
>>> Senior Tech Lead
>>> M: +94 777 219 791
>>> Tel : 94 11 214 5345
>>> Fax :94 11 2145300
>>> Skype : malaka.sampath.silva
>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
>>> Blog : http://mrmalakasilva.blogspot.com/
>>>
>>> WSO2, Inc.
>>> lean . enterprise . middleware
>>> http://www.wso2.com/
>>> http://www.wso2.com/about/team/malaka-silva/
>>> <http://wso2.com/about/team/malaka-silva/>
>>> https://store.wso2.com/store/
>>>
>>> Save a tree -Conserve nature & Save the world for your future. Print
>>> this email only if it is absolutely necessary.
>>>
>>
>>
>>
>> --
>>
>> Best Regards,
>>
>> Malaka Silva
>> Senior Tech Lead
>> M: +94 777 219 791
>> Tel : 94 11 214 5345
>> Fax :94 11 2145300
>> Skype : malaka.sampath.silva
>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
>> Blog : http://mrmalakasilva.blogspot.com/
>>
>> WSO2, Inc.
>> lean . enterprise . middleware
>> http://www.wso2.com/
>> http://www.wso2.com/about/team/malaka-silva/
>> <http://wso2.com/about/team/malaka-silva/>
>> https://store.wso2.com/store/
>>
>> Save a tree -Conserve nature & Save the world for your future. Print this
>> email only if it is absolutely necessary.
>>
>
>
>
> --
> Kasun Indrasiri
> Software Architect
> WSO2, Inc.; http://wso2.com
> lean.enterprise.middleware
>
> cell: +94 77 556 5206
> Blog : http://kasunpanorama.blogspot.com/
>
--
Best Regards,
Malaka Silva
Senior Tech Lead
M: +94 777 219 791
Tel : 94 11 214 5345
Fax :94 11 2145300
Skype : malaka.sampath.silva
LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
Blog : http://mrmalakasilva.blogspot.com/
WSO2, Inc.
lean . enterprise . middleware
http://www.wso2.com/
http://www.wso2.com/about/team/malaka-silva/
<http://wso2.com/about/team/malaka-silva/>
https://store.wso2.com/store/
Save a tree -Conserve nature & Save the world for your future. Print this
email only if it is absolutely necessary.
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
