Re: [Dev] [DAS] Adding a relative path in external-spark-classpath.conf doesn't work

[Supun Sethunga]

Thanks Nirmal for the fix!

I'm closing the Jira [1]

[1] https://wso2.org/jira/browse/DAS-397

On Wed, Apr 13, 2016 at 9:17 AM, Nirmal Fernando  wrote:

> Thanks, Niranda for merging.
>
> On Wed, Apr 13, 2016 at 8:48 AM, Niranda Perera  wrote:
>
>> Hi Nirmal,
>>
>> Thanks for pointing this out. I will merge the PR.
>>
>> Rgds
>> On Apr 13, 2016 08:35, "Nirmal Fernando"  wrote:
>>
>>> Hi,
>>>
>>> * Even though we have implemented the support for relative paths at
>>> repository/conf/analytics/spark/external-spark-classpath.conf file, it
>>> isn't working properly.
>>>
>>>   if (fileExists(line)) {
>>>
>>> scp = scp + separator + line;
>>>
>>> } *else if (fileExists(carbonHome + File.separator +
>>> line)) {*
>>>
>>> *scp = scp + separator + carbonHome + File.separator
>>> + line;*
>>>
>>> *} *else {
>>>
>>> throw new IOException("File not found : " + line);
>>>
>>> }
>>>
>>>   
>>>
>>>   
>>>
>>>   private static boolean fileExists(String path) {
>>>
>>> File tempFile = new File(path);
>>>
>>> return tempFile.exists() && !tempFile.isDirectory();
>>>
>>>}
>>> * We check for file.exists in order to determine the existence of the
>>> file and it'll be true even if it's a relative path in some cases (where
>>> current execution directory = carbon.home).
>>> * But Spark needs the path to be absolute.
>>> * Hence the fix was to check whether the path is absolute too.
>>>
>>> https://github.com/wso2/carbon-analytics/pull/170/files
>>>
>>> Please review and merge this PR.
>>>
>>> --
>>>
>>> Thanks & regards,
>>> Nirmal
>>>
>>> Team Lead - WSO2 Machine Learner
>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc.
>>> Mobile: +94715779733
>>> Blog: http://nirmalfdo.blogspot.com/
>>>
>>>
>>>
>
>
> --
>
> Thanks & regards,
> Nirmal
>
> Team Lead - WSO2 Machine Learner
> Associate Technical Lead - Data Technologies Team, WSO2 Inc.
> Mobile: +94715779733
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
*Supun Sethunga*
Software Engineer
WSO2, Inc.
http://wso2.com/
lean | enterprise | middleware
Mobile : +94 716546324
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DAS] Adding a relative path in external-spark-classpath.conf doesn't work

[Nirmal Fernando]

Thanks, Niranda for merging.

On Wed, Apr 13, 2016 at 8:48 AM, Niranda Perera  wrote:

> Hi Nirmal,
>
> Thanks for pointing this out. I will merge the PR.
>
> Rgds
> On Apr 13, 2016 08:35, "Nirmal Fernando"  wrote:
>
>> Hi,
>>
>> * Even though we have implemented the support for relative paths at
>> repository/conf/analytics/spark/external-spark-classpath.conf file, it
>> isn't working properly.
>>
>>   if (fileExists(line)) {
>>
>> scp = scp + separator + line;
>>
>> } *else if (fileExists(carbonHome + File.separator +
>> line)) {*
>>
>> *scp = scp + separator + carbonHome + File.separator
>> + line;*
>>
>> *} *else {
>>
>> throw new IOException("File not found : " + line);
>>
>> }
>>
>>   
>>
>>   
>>
>>   private static boolean fileExists(String path) {
>>
>> File tempFile = new File(path);
>>
>> return tempFile.exists() && !tempFile.isDirectory();
>>
>>}
>> * We check for file.exists in order to determine the existence of the
>> file and it'll be true even if it's a relative path in some cases (where
>> current execution directory = carbon.home).
>> * But Spark needs the path to be absolute.
>> * Hence the fix was to check whether the path is absolute too.
>>
>> https://github.com/wso2/carbon-analytics/pull/170/files
>>
>> Please review and merge this PR.
>>
>> --
>>
>> Thanks & regards,
>> Nirmal
>>
>> Team Lead - WSO2 Machine Learner
>> Associate Technical Lead - Data Technologies Team, WSO2 Inc.
>> Mobile: +94715779733
>> Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>


-- 

Thanks & regards,
Nirmal

Team Lead - WSO2 Machine Learner
Associate Technical Lead - Data Technologies Team, WSO2 Inc.
Mobile: +94715779733
Blog: http://nirmalfdo.blogspot.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DAS] Adding a relative path in external-spark-classpath.conf doesn't work

[Niranda Perera]

Hi Nirmal,

Thanks for pointing this out. I will merge the PR.

Rgds
On Apr 13, 2016 08:35, "Nirmal Fernando"  wrote:

> Hi,
>
> * Even though we have implemented the support for relative paths at
> repository/conf/analytics/spark/external-spark-classpath.conf file, it
> isn't working properly.
>
>   if (fileExists(line)) {
>
> scp = scp + separator + line;
>
> } *else if (fileExists(carbonHome + File.separator +
> line)) {*
>
> *scp = scp + separator + carbonHome + File.separator +
> line;*
>
> *} *else {
>
> throw new IOException("File not found : " + line);
>
> }
>
>   
>
>   
>
>   private static boolean fileExists(String path) {
>
> File tempFile = new File(path);
>
> return tempFile.exists() && !tempFile.isDirectory();
>
>}
> * We check for file.exists in order to determine the existence of the file
> and it'll be true even if it's a relative path in some cases (where current
> execution directory = carbon.home).
> * But Spark needs the path to be absolute.
> * Hence the fix was to check whether the path is absolute too.
>
> https://github.com/wso2/carbon-analytics/pull/170/files
>
> Please review and merge this PR.
>
> --
>
> Thanks & regards,
> Nirmal
>
> Team Lead - WSO2 Machine Learner
> Associate Technical Lead - Data Technologies Team, WSO2 Inc.
> Mobile: +94715779733
> Blog: http://nirmalfdo.blogspot.com/
>
>
>
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [DAS] Adding a relative path in external-spark-classpath.conf doesn't work

[Nirmal Fernando]

Hi,

* Even though we have implemented the support for relative paths at
repository/conf/analytics/spark/external-spark-classpath.conf file, it
isn't working properly.

  if (fileExists(line)) {

scp = scp + separator + line;

} *else if (fileExists(carbonHome + File.separator + line))
{*

*scp = scp + separator + carbonHome + File.separator +
line;*

*} *else {

throw new IOException("File not found : " + line);

}

  

  

  private static boolean fileExists(String path) {

File tempFile = new File(path);

return tempFile.exists() && !tempFile.isDirectory();

   }
* We check for file.exists in order to determine the existence of the file
and it'll be true even if it's a relative path in some cases (where current
execution directory = carbon.home).
* But Spark needs the path to be absolute.
* Hence the fix was to check whether the path is absolute too.

https://github.com/wso2/carbon-analytics/pull/170/files

Please review and merge this PR.

-- 

Thanks & regards,
Nirmal

Team Lead - WSO2 Machine Learner
Associate Technical Lead - Data Technologies Team, WSO2 Inc.
Mobile: +94715779733
Blog: http://nirmalfdo.blogspot.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev