[jira] [Updated] (FALCON-2224) Change return type of extension APIs to APIResult

2016-12-15 Thread Pragya Mittal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FALCON-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pragya Mittal updated FALCON-2224:
--
Summary: Change return type of extension APIs to APIResult  (was: Change 
return type of Register and Unregister API to ApiResult)

> Change return type of extension APIs to APIResult
> -
>
> Key: FALCON-2224
> URL: https://issues.apache.org/jira/browse/FALCON-2224
> Project: Falcon
>  Issue Type: Bug
>Reporter: Praveen Adlakha
>Assignee: Praveen Adlakha
>
> Hi All,
> Currently the return type of the mentioned API is String we should change it 
> to APIResult.
> Thanks
> Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (FALCON-2224) Change return type of Register and Unregister API to ApiResult

2016-12-15 Thread Praveen Adlakha (JIRA) 
Praveen Adlakha created FALCON-2224:
---

 Summary: Change return type of Register and Unregister API to 
ApiResult
 Key: FALCON-2224
 URL: https://issues.apache.org/jira/browse/FALCON-2224
 Project: Falcon
  Issue Type: Bug
Reporter: Praveen Adlakha
Assignee: Praveen Adlakha


Hi All,

Currently the return type of the mentioned API is String we should change it to 
APIResult.

Thanks
Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FALCON-2224) Change return type of extension APIs to APIResult

2016-12-15 Thread Pragya Mittal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FALCON-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pragya Mittal updated FALCON-2224:
--
Description: Currently the return type of the extension API is String we 
should change it to APIResult.  (was: Hi All,

Currently the return type of the mentioned API is String we should change it to 
APIResult.

Thanks
Praveen)

> Change return type of extension APIs to APIResult
> -
>
> Key: FALCON-2224
> URL: https://issues.apache.org/jira/browse/FALCON-2224
> Project: Falcon
>  Issue Type: Bug
>Reporter: Praveen Adlakha
>Assignee: Praveen Adlakha
>
> Currently the return type of the extension API is String we should change it 
> to APIResult.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FALCON-2222) Delete api for backlog metrics

2016-12-15 Thread Pallavi Rao (JIRA) 

[ 
https://issues.apache.org/jira/browse/FALCON-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750724#comment-15750724
 ] 

Pallavi Rao commented on FALCON-:
-

Ideally, the user should do a manual kill, which will put the instance in 
IGNORED status. And, the backlog computation should not consider IGNORED 
instances. If this is not happening now, then, it is a bug.

> Delete api for backlog metrics
> --
>
> Key: FALCON-
> URL: https://issues.apache.org/jira/browse/FALCON-
> Project: Falcon
>  Issue Type: Task
>Reporter: Praveen Adlakha
>
> Hi All,
> Currently we donot have any api to delete the backlog from the database.
> There can be instances where user knowingly doesnt want to run the killed 
> instances in that case there is no way to remove backlog except deleting and 
> resubmit the process.
> Do take care of the ACL owner while deleting the entry from db.
> Thanks
> Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FALCON-2221) Submitting an existing job fails with MySQLIntegrityConstraintViolationException

2016-12-15 Thread sandeep samudrala (JIRA) 

[ 
https://issues.apache.org/jira/browse/FALCON-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750733#comment-15750733
 ] 

sandeep samudrala commented on FALCON-2221:
---

[~pragya.mittal]: Is it happening for register of an extension too?

> Submitting an existing job fails with 
> MySQLIntegrityConstraintViolationException
> 
>
> Key: FALCON-2221
> URL: https://issues.apache.org/jira/browse/FALCON-2221
> Project: Falcon
>  Issue Type: Bug
>  Components: extensions
>Affects Versions: trunk
>Reporter: Pragya Mittal
>Assignee: sandeep samudrala
>Priority: Minor
>
> Submitting an already existing job should succeed saying 'Job already 
> submitted' (as is the case with process submission). Instead it throws the 
> followinf error :
> {noformat}
> dataqa@falcon1001:~$ falcon extension -submit -jobName sample -file abc  
> -extensionName extension-example
> Dec 15, 2016 7:15:31 AM com.sun.jersey.api.client.ClientResponse getEntity
> SEVERE: A message body reader for Java class 
> org.apache.falcon.resource.APIResult, and Java type class 
> org.apache.falcon.resource.APIResult, and MIME media type text/html; 
> charset=iso-8859-1 was not found
> Dec 15, 2016 7:15:31 AM com.sun.jersey.api.client.ClientResponse getEntity
> SEVERE: The registered message body readers compatible with the MIME media 
> type are:
> */* ->
>   com.sun.jersey.core.impl.provider.entity.FormProvider
>   com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider
>   com.sun.jersey.core.impl.provider.entity.StringProvider
>   com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
>   com.sun.jersey.core.impl.provider.entity.FileProvider
>   com.sun.jersey.core.impl.provider.entity.InputStreamProvider
>   com.sun.jersey.core.impl.provider.entity.DataSourceProvider
>   com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.ReaderProvider
>   com.sun.jersey.core.impl.provider.entity.DocumentProvider
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader
>   com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General
>   com.sun.jersey.core.impl.provider.entity.EntityHolderReader
>   com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy
> ERROR: Internal Server Error;
> 
> 
> Error 500 The transaction has been rolled back.  See the nested 
> exceptions for details on the errors that occurred.
> 
> HTTP ERROR 500
> Problem accessing /api/extension/submit/extension-example. Reason:
> The transaction has been rolled back.  See the nested exceptions for 
> details on the errors that occurred.Caused 
> by:openjpa-2.4.0-r422266:1674604 fatal store error 
> org.apache.openjpa.persistence.RollbackException: The transaction has been 
> rolled back.  See the nested exceptions for details on the errors that 
> occurred.
> FailedObject: org.apache.falcon.persistence.ExtensionJobsBean@24f6b1a0
>   at 
> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:593)
>   at 
> org.apache.falcon.extensions.jdbc.ExtensionMetaStore.commitAndCloseTransaction(ExtensionMetaStore.java:175)
>   at 
> org.apache.falcon.extensions.jdbc.ExtensionMetaStore.storeExtensionJob(ExtensionMetaStore.java:141)
>   at 
> org.apache.falcon.resource.extensions.ExtensionManager.submitEntities(ExtensionManager.java:433)
>   at 
> org.apache.falcon.resource.extensions.ExtensionManager.submit(ExtensionManager.java:318)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>   at 
>

[jira] [Updated] (FALCON-2221) Submitting an existing job fails with MySQLIntegrityConstraintViolationException

2016-12-15 Thread sandeep samudrala (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FALCON-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sandeep samudrala updated FALCON-2221:
--
Priority: Minor  (was: Major)

> Submitting an existing job fails with 
> MySQLIntegrityConstraintViolationException
> 
>
> Key: FALCON-2221
> URL: https://issues.apache.org/jira/browse/FALCON-2221
> Project: Falcon
>  Issue Type: Bug
>  Components: extensions
>Affects Versions: trunk
>Reporter: Pragya Mittal
>Priority: Minor
>
> Submitting an already existing job should succeed saying 'Job already 
> submitted' (as is the case with process submission). Instead it throws the 
> followinf error :
> {noformat}
> dataqa@falcon1001:~$ falcon extension -submit -jobName sample -file abc  
> -extensionName extension-example
> Dec 15, 2016 7:15:31 AM com.sun.jersey.api.client.ClientResponse getEntity
> SEVERE: A message body reader for Java class 
> org.apache.falcon.resource.APIResult, and Java type class 
> org.apache.falcon.resource.APIResult, and MIME media type text/html; 
> charset=iso-8859-1 was not found
> Dec 15, 2016 7:15:31 AM com.sun.jersey.api.client.ClientResponse getEntity
> SEVERE: The registered message body readers compatible with the MIME media 
> type are:
> */* ->
>   com.sun.jersey.core.impl.provider.entity.FormProvider
>   com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider
>   com.sun.jersey.core.impl.provider.entity.StringProvider
>   com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
>   com.sun.jersey.core.impl.provider.entity.FileProvider
>   com.sun.jersey.core.impl.provider.entity.InputStreamProvider
>   com.sun.jersey.core.impl.provider.entity.DataSourceProvider
>   com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.ReaderProvider
>   com.sun.jersey.core.impl.provider.entity.DocumentProvider
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader
>   com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader
>   com.sun.jersey.json.impl.provider.entity.JSONJAXBElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONArrayProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONObjectProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
>   com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General
>   com.sun.jersey.core.impl.provider.entity.EntityHolderReader
>   com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JSONListElementProvider$General
>   com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy
> ERROR: Internal Server Error;
> 
> 
> Error 500 The transaction has been rolled back.  See the nested 
> exceptions for details on the errors that occurred.
> 
> HTTP ERROR 500
> Problem accessing /api/extension/submit/extension-example. Reason:
> The transaction has been rolled back.  See the nested exceptions for 
> details on the errors that occurred.Caused 
> by:openjpa-2.4.0-r422266:1674604 fatal store error 
> org.apache.openjpa.persistence.RollbackException: The transaction has been 
> rolled back.  See the nested exceptions for details on the errors that 
> occurred.
> FailedObject: org.apache.falcon.persistence.ExtensionJobsBean@24f6b1a0
>   at 
> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:593)
>   at 
> org.apache.falcon.extensions.jdbc.ExtensionMetaStore.commitAndCloseTransaction(ExtensionMetaStore.java:175)
>   at 
> org.apache.falcon.extensions.jdbc.ExtensionMetaStore.storeExtensionJob(ExtensionMetaStore.java:141)
>   at 
> org.apache.falcon.resource.extensions.ExtensionManager.submitEntities(ExtensionManager.java:433)
>   at 
> org.apache.falcon.resource.extensions.ExtensionManager.submit(ExtensionManager.java:318)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>   at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
>   at 
>

[jira] [Created] (FALCON-2223) Distributed mode support for User Extension

2016-12-15 Thread Praveen Adlakha (JIRA) 
Praveen Adlakha created FALCON-2223:
---

 Summary: Distributed mode support for User Extension
 Key: FALCON-2223
 URL: https://issues.apache.org/jira/browse/FALCON-2223
 Project: Falcon
  Issue Type: Sub-task
Reporter: Praveen Adlakha
Assignee: Praveen Adlakha


Hi All,

This issue is to track distributed mode support for user extension.In this jira 
I will be making changes for:

register 
unregister
detail 
describe API.

as these are logically grouped together which will be running on prism only.

Thanks
Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FALCON-2223) Distributed mode support for User Extension

2016-12-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FALCON-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751190#comment-15751190
 ] 

ASF GitHub Bot commented on FALCON-2223:


GitHub user PraveenAdlakha opened a pull request:

https://github.com/apache/falcon/pull/323

FALCON-2223 Distributed mode support for User Extension



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/PraveenAdlakha/falcon extensionManagerSupport

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/falcon/pull/323.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #323


commit 8e96ec8351ddcc53677de61692c43a61d4cbf738
Author: Praveen Adlakha 
Date:   2016-12-15T11:59:46Z

FALCON-2223 Distributed mode support for User Extension




> Distributed mode support for User Extension
> ---
>
> Key: FALCON-2223
> URL: https://issues.apache.org/jira/browse/FALCON-2223
> Project: Falcon
>  Issue Type: Sub-task
>  Components: extensions
>Reporter: Praveen Adlakha
>Assignee: Praveen Adlakha
>
> Hi All,
> This issue is to track distributed mode support for user extension.In this 
> jira I will be making changes for:
> register 
> unregister
> detail 
> describe API.
> as these are logically grouped together which will be running on prism only.
> Thanks
> Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (FALCON-2207) Only the owner of the extension, should be able to unregister, disable, enable.

2016-12-15 Thread Pallavi Rao (JIRA) 

 [ 
https://issues.apache.org/jira/browse/FALCON-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pallavi Rao resolved FALCON-2207.
-
   Resolution: Fixed
Fix Version/s: trunk

Issue resolved by pull request 322
[https://github.com/apache/falcon/pull/322]

> Only the owner of the extension, should be able to unregister, disable, 
> enable.
> ---
>
> Key: FALCON-2207
> URL: https://issues.apache.org/jira/browse/FALCON-2207
> Project: Falcon
>  Issue Type: Sub-task
>  Components: extensions
>Reporter: Pallavi Rao
>Assignee: Pracheer Agarwal
> Fix For: trunk
>
>
> It is possible that owner of extension's hdfs location and the user who is 
> registering that extension might be different.
> There is a need to capture the owner of the extension (the user who 
> registered it) and not allow other users to operate (unregister, enable, 
> disable) on the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FALCON-2223) Distributed mode support for User Extension

2016-12-15 Thread Praveen Adlakha (JIRA) 

[ 
https://issues.apache.org/jira/browse/FALCON-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751213#comment-15751213
 ] 

Praveen Adlakha commented on FALCON-2223:
-

DEV testing:

{code}

In distributed mode with client pointing to Server with extension service 
running on server:

$ ./bin/falcon extension -enumerate 
Hadoop home is set, adding libraries from 
'/usr/hdp/2.2.0.0-2041/hadoop/bin/hadoop classpath' into falcon classpath
ERROR: Bad Request;Enumerate is not supported on Server.Please run your 
operation on prism.

$ ./bin/falcon extension -describe -extensionName abcd
ERROR: Bad Request;Describe is not supported on Server.Please run your 
operation on prism.

$ ./bin/falcon extension -detail -extensionName abcd
ERROR: Bad Request;Detail is not supported on Server.Please run your operation 
on prism.

$ ./bin/falcon extension -unregister -extensionName abcd
ERROR: Bad Request;Unregister is not supported on Server.Please run your 
operation on prism.

$ ./bin/falcon extension -definition -extensionName abcd
ERROR: Bad Request;Definition is not supported on Server.Please run your 
operation on prism.


In distributed mode with client pointing to Prism with extension service 
running on prism:

./bin/falcon extension -enumerate 
{
  "extensions": [
{
  "name": "abcd",
  "type": "Custom extension",
  "location": "hdfs://10.14.118.114:8020/user/dataqa/sampleextension/"
}
],
"totalResults: 1"
}   

$ ./bin/falcon extension -describe -extensionName test
ERROR: Internal Server Error;java.io.FileNotFoundException: File 
file:/usr/local/falcon/prism/extensions/test does not exist
CausedBy: File file:/usr/local/falcon/prism/extensions/test does not exist

Describe seems to be broked.

$ ./bin/falcon extension -detail -extensionName test
{
  "name": "test",
  "type": "Custom extension",
  "description": "test123",
  "location": "hdfs://localhost:8020/user/dataqa/sampleextension"
}

$ ./bin/falcon extension -unregister -extensionName abcd
Deleted extension:abcd

$ ./bin/falcon extension -register  -extensionName abcd -path 
hdfs://10.14.118.114:8020/user/dataqa/sampleextension/
Extension :abcd registered successfully.


In embedded mode :

./bin/falcon extension -enumerate 
{
  "extensions": [
{
  "name": "abcd",
  "type": "Custom extension",
  "location": "hdfs://10.14.118.114:8020/user/dataqa/sampleextension/"
}
],
"totalResults: 1"
} 

$ ./bin/falcon extension -detail -extensionName test
{
  "name": "test",
  "type": "Custom extension",
  "description": "test123",
  "location": "hdfs://localhost:8020/user/dataqa/sampleextension"
}

$ ./bin/falcon extension -unregister -extensionName abcd
Deleted extension:abcd

$ ./bin/falcon extension -register  -extensionName abcd -path 
hdfs://10.14.118.114:8020/user/dataqa/sampleextension/
Extension :abcd registered successfully.
{code}

> Distributed mode support for User Extension
> ---
>
> Key: FALCON-2223
> URL: https://issues.apache.org/jira/browse/FALCON-2223
> Project: Falcon
>  Issue Type: Sub-task
>  Components: extensions
>Reporter: Praveen Adlakha
>Assignee: Praveen Adlakha
>
> Hi All,
> This issue is to track distributed mode support for user extension.In this 
> jira I will be making changes for:
> register 
> unregister
> detail 
> describe API.
> as these are logically grouped together which will be running on prism only.
> Thanks
> Praveen



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] falcon pull request #323: FALCON-2223 Distributed mode support for User Exte...

2016-12-15 Thread PraveenAdlakha 
GitHub user PraveenAdlakha opened a pull request:

https://github.com/apache/falcon/pull/323

FALCON-2223 Distributed mode support for User Extension



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/PraveenAdlakha/falcon extensionManagerSupport

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/falcon/pull/323.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #323


commit 8e96ec8351ddcc53677de61692c43a61d4cbf738
Author: Praveen Adlakha 
Date:   2016-12-15T11:59:46Z

FALCON-2223 Distributed mode support for User Extension




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] falcon pull request #322: FALCON-2207 ACL check while deleting extensions

2016-12-15 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/falcon/pull/322


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FALCON-2207) Only the owner of the extension, should be able to unregister, disable, enable.

2016-12-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FALCON-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751251#comment-15751251
 ] 

ASF GitHub Bot commented on FALCON-2207:


Github user asfgit closed the pull request at:

https://github.com/apache/falcon/pull/322


> Only the owner of the extension, should be able to unregister, disable, 
> enable.
> ---
>
> Key: FALCON-2207
> URL: https://issues.apache.org/jira/browse/FALCON-2207
> Project: Falcon
>  Issue Type: Sub-task
>  Components: extensions
>Reporter: Pallavi Rao
>Assignee: Pracheer Agarwal
>
> It is possible that owner of extension's hdfs location and the user who is 
> registering that extension might be different.
> There is a need to capture the owner of the extension (the user who 
> registered it) and not allow other users to operate (unregister, enable, 
> disable) on the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)