[jira] [Resolved] (EAGLE-434) BUG: periodically trigger schedule in service mode
[ https://issues.apache.org/jira/browse/EAGLE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-434. -- Resolution: Fixed merged the code to upstream > BUG: periodically trigger schedule in service mode > -- > > Key: EAGLE-434 > URL: https://issues.apache.org/jira/browse/EAGLE-434 > Project: Eagle > Issue Type: Bug >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Original Estimate: 24h > Remaining Estimate: 24h > > the periodical trigger schedule is not started -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #329: EAGLE-453 Use META-INF/providers/${appPro...
Github user haoch closed the pull request at: https://github.com/apache/incubator-eagle/pull/329 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-453) Use META-INF/providers/${appProviderClassName}.xml as default metadata path
[ https://issues.apache.org/jira/browse/EAGLE-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416565#comment-15416565 ] ASF GitHub Bot commented on EAGLE-453: -- GitHub user haoch opened a pull request: https://github.com/apache/incubator-eagle/pull/329 EAGLE-453 Use META-INF/providers/${appProviderClassName}.xml as default metadata path https://issues.apache.org/jira/browse/EAGLE-453 You can merge this pull request into a Git repository by running: $ git pull https://github.com/haoch/incubator-eagle EAGLE-453 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/329.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #329 commit ee5849f7ca29898b5a38ee3942131b14971afcc4 Author: Hao ChenDate: 2016-08-11T05:15:09Z EAGLE-453 Use META-INF/providers/${appProviderClassName}.xml as default metadata path > Use META-INF/providers/${appProviderClassName}.xml as default metadata path > --- > > Key: EAGLE-453 > URL: https://issues.apache.org/jira/browse/EAGLE-453 > Project: Eagle > Issue Type: New Feature >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Fix For: v0.5.0 > > > Use META-INF/providers/${APP_PROVDER_CLASS_NAME}.xml as default metadata xml > path, for example: > META-INF/providers/org.apache.eagle.security.hbase.HBaseAuditLogAppProvider.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #329: EAGLE-453 Use META-INF/providers/${appPro...
GitHub user haoch opened a pull request: https://github.com/apache/incubator-eagle/pull/329 EAGLE-453 Use META-INF/providers/${appProviderClassName}.xml as default metadata path https://issues.apache.org/jira/browse/EAGLE-453 You can merge this pull request into a Git repository by running: $ git pull https://github.com/haoch/incubator-eagle EAGLE-453 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/329.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #329 commit ee5849f7ca29898b5a38ee3942131b14971afcc4 Author: Hao ChenDate: 2016-08-11T05:15:09Z EAGLE-453 Use META-INF/providers/${appProviderClassName}.xml as default metadata path --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (EAGLE-453) Use META-INF/providers/${appProviderClassName}.xml as default metadata path
[ https://issues.apache.org/jira/browse/EAGLE-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen updated EAGLE-453: --- Summary: Use META-INF/providers/${appProviderClassName}.xml as default metadata path (was: Use META-INF/${appProviderClassName}.xml as default metadata path) > Use META-INF/providers/${appProviderClassName}.xml as default metadata path > --- > > Key: EAGLE-453 > URL: https://issues.apache.org/jira/browse/EAGLE-453 > Project: Eagle > Issue Type: New Feature >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Fix For: v0.5.0 > > > Use META-INF/providers/${APP_PROVDER_CLASS_NAME}.xml as default metadata xml > path, for example: > META-INF/providers/org.apache.eagle.security.hbase.HBaseAuditLogAppProvider.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416532#comment-15416532 ] ASF GitHub Bot commented on EAGLE-435: -- Github user RalphSu commented on a diff in the pull request: https://github.com/apache/incubator-eagle/pull/322#discussion_r74367571 --- Diff: eagle-core/eagle-alert-parent/eagle-alert/alert-coordinator/src/main/java/org/apache/eagle/alert/coordinator/ExclusiveExecutor.java --- @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.eagle.alert.coordinator; + +import org.apache.curator.RetryPolicy; +import org.apache.curator.framework.CuratorFramework; +import org.apache.curator.framework.CuratorFrameworkFactory; +import org.apache.curator.framework.recipes.leader.LeaderSelector; +import org.apache.curator.framework.recipes.leader.LeaderSelectorListener; +import org.apache.curator.framework.recipes.leader.LeaderSelectorListenerAdapter; +import org.apache.curator.framework.state.ConnectionState; +import org.apache.curator.retry.ExponentialBackoffRetry; +import org.apache.curator.utils.CloseableUtils; +import org.apache.eagle.alert.config.ZKConfig; +import org.apache.eagle.alert.config.ZKConfigBuilder; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.typesafe.config.Config; +import com.typesafe.config.ConfigFactory; + +/** + * + * @author xiancli --- End diff -- In eagle, the convention is to remove author info > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Fix For: v0.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-451) oozie ingestion+alert engine preview
[ https://issues.apache.org/jira/browse/EAGLE-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416528#comment-15416528 ] Hao Chen commented on EAGLE-451: [~chitin] Is it similar with https://issues.apache.org/jira/browse/EAGLE-448? > oozie ingestion+alert engine preview > > > Key: EAGLE-451 > URL: https://issues.apache.org/jira/browse/EAGLE-451 > Project: Eagle > Issue Type: Improvement >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-452) Add Jersey Resource Unit Test Case
Hao Chen created EAGLE-452: -- Summary: Add Jersey Resource Unit Test Case Key: EAGLE-452 URL: https://issues.apache.org/jira/browse/EAGLE-452 Project: Eagle Issue Type: Improvement Affects Versions: v0.4.0, v0.5.0 Reporter: Hao Chen Assignee: Chang chen Priority: Minor Fix For: v0.5.0 We need to add more Unit Test Cases for Jersey Resources, referring to https://github.com/apache/incubator-eagle/blob/develop/eagle-examples/eagle-app-example/src/test/java/org/apache/eagle/app/example/ExampleApplicationProviderTest.java#L45 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-416) Add check style support
[ https://issues.apache.org/jira/browse/EAGLE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416519#comment-15416519 ] ASF GitHub Bot commented on EAGLE-416: -- Github user haoch commented on the issue: https://github.com/apache/incubator-eagle/pull/303 @r7raul1984 I am planing to merge this PR, but have little conflicts, could you please help resole it firstly? > Add check style support > --- > > Key: EAGLE-416 > URL: https://issues.apache.org/jira/browse/EAGLE-416 > Project: Eagle > Issue Type: Improvement >Reporter: JiJun Tang >Assignee: JiJun Tang >Priority: Trivial > Labels: engineering, pull-request-available > Fix For: v0.5.0 > > > Add check style when build the project. Just print warning in console. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-431) MetadataDaoFactory should use config from eagle server
[ https://issues.apache.org/jira/browse/EAGLE-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen resolved EAGLE-431. Resolution: Fixed > MetadataDaoFactory should use config from eagle server > -- > > Key: EAGLE-431 > URL: https://issues.apache.org/jira/browse/EAGLE-431 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Hao Chen > Fix For: v0.5.0 > > > org.apache.eagle.alert.metadata.impl.MetadataDaoFactory uses > ConfigFactory.load() to get configuration. We should use config object from > eagle server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle issue #303: [EAGLE-416] Add check style support
Github user haoch commented on the issue: https://github.com/apache/incubator-eagle/pull/303 @r7raul1984 I am planing to merge this PR, but have little conflicts, could you please help resole it firstly? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen resolved EAGLE-442. Resolution: Fixed > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > h1. Module Registry Interfaces > * ApplicationProvider > {code} > ApplicationProvider{ >void register(ModuleRegistry) > } > {code} > * ModuleRegistry > {code} > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > {code} > * ModuleScope > - Modules registered in scope of `GlobalScope` will be visible globally > - Modules registered in > `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only > be visible when selected MetadataStore is active (set in configuration file) > {code} > ModuleScope > |- GlobalScope > |- MetadataStore > |- MySQLMetadataStore > |- MongodbMetadataStore > |- MemoryMetadataStore > {code} > h1. Example > * Registry: in `HBaseAuditLogAppProvider` > {code} > @Override > public void register(ModuleRegistry registry) { > registry.register(MemoryMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); > } > }); > registry.register(MySQLMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); > } > }); > } > {code} > * Inject: For example, `HbaseMetadataBrowseWebResource` could get current > `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as > following, instead of using > [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) > {code} > @Inject > public HbaseMetadataBrowseWebResource(ApplicationEntityService > entityService, ISecurityMetadataDAO metadataDAO){ > this.entityService = entityService; > this.dao = metadataDAO; > } > {code} > h1. More Use Cases > * ExampleResource > * ExampleService > * ExampleApplicationProviderTest#testApplicationExtensions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416517#comment-15416517 ] ASF GitHub Bot commented on EAGLE-442: -- Github user haoch closed the pull request at: https://github.com/apache/incubator-eagle/pull/323 > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > h1. Module Registry Interfaces > * ApplicationProvider > {code} > ApplicationProvider{ >void register(ModuleRegistry) > } > {code} > * ModuleRegistry > {code} > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > {code} > * ModuleScope > - Modules registered in scope of `GlobalScope` will be visible globally > - Modules registered in > `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only > be visible when selected MetadataStore is active (set in configuration file) > {code} > ModuleScope > |- GlobalScope > |- MetadataStore > |- MySQLMetadataStore > |- MongodbMetadataStore > |- MemoryMetadataStore > {code} > h1. Example > * Registry: in `HBaseAuditLogAppProvider` > {code} > @Override > public void register(ModuleRegistry registry) { > registry.register(MemoryMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); > } > }); > registry.register(MySQLMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); > } > }); > } > {code} > * Inject: For example, `HbaseMetadataBrowseWebResource` could get current > `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as > following, instead of using > [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) > {code} > @Inject > public HbaseMetadataBrowseWebResource(ApplicationEntityService > entityService, ISecurityMetadataDAO metadataDAO){ > this.entityService = entityService; > this.dao = metadataDAO; > } > {code} > h1. More Use Cases > * ExampleResource > * ExampleService > * ExampleApplicationProviderTest#testApplicationExtensions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #323: EAGLE-442 Support ApplicationProvider to ...
Github user haoch closed the pull request at: https://github.com/apache/incubator-eagle/pull/323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416509#comment-15416509 ] ASF GitHub Bot commented on EAGLE-442: -- Github user haoch commented on the issue: https://github.com/apache/incubator-eagle/pull/323 I think ApplicationProvider is just for providing a package of static application process/metadata/extensions but not for runtime. As following typical example, for different scope of memory/mysql, `ISecurityMetadataDAO` should bind to different implementation `InMemMetadataDaoImpl`/ `JDBCSecurityMetadataDAO`, and caller just need to inject with `@Inject ISecurityMetadataDAO dao`. ~~~ @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } ~~~ While if register with current metadata store instance, then the code have to be as following, it will bind the register interface with MetadataStore and not extensible enough. ~~~ @Override public void register(MetadataStore store, ModuleRegistry registry) { if(store instance of MemoryMetadataStore){ registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); } else if (store instance of MySQLMetadataStore){ registry.register(new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } } ~~~ I think both maybe ok, and have similar implementation code, but the first implementation should be more flexible, especially in future we may be able to extend more than MetadataStore. > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > h1. Module Registry Interfaces > * ApplicationProvider > {code} > ApplicationProvider{ >void register(ModuleRegistry) > } > {code} > * ModuleRegistry > {code} > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > {code} > * ModuleScope > - Modules registered in scope of `GlobalScope` will be visible globally > - Modules registered in > `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only > be visible when selected MetadataStore is active (set in configuration file) > {code} > ModuleScope > |- GlobalScope > |- MetadataStore > |- MySQLMetadataStore > |- MongodbMetadataStore > |- MemoryMetadataStore > {code} > h1. Example > * Registry: in `HBaseAuditLogAppProvider` > {code} > @Override > public void register(ModuleRegistry registry) { > registry.register(MemoryMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); > } > }); > registry.register(MySQLMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); > } > }); > } > {code} > * Inject: For example, `HbaseMetadataBrowseWebResource` could get current > `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as > following, instead of using > [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) > {code} > @Inject > public HbaseMetadataBrowseWebResource(ApplicationEntityService > entityService,
[GitHub] incubator-eagle issue #323: EAGLE-442 Support ApplicationProvider to extend ...
Github user haoch commented on the issue: https://github.com/apache/incubator-eagle/pull/323 I think ApplicationProvider is just for providing a package of static application process/metadata/extensions but not for runtime. As following typical example, for different scope of memory/mysql, `ISecurityMetadataDAO` should bind to different implementation `InMemMetadataDaoImpl`/ `JDBCSecurityMetadataDAO`, and caller just need to inject with `@Inject ISecurityMetadataDAO dao`. ~~~ @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } ~~~ While if register with current metadata store instance, then the code have to be as following, it will bind the register interface with MetadataStore and not extensible enough. ~~~ @Override public void register(MetadataStore store, ModuleRegistry registry) { if(store instance of MemoryMetadataStore){ registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); } else if (store instance of MySQLMetadataStore){ registry.register(new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } } ~~~ I think both maybe ok, and have similar implementation code, but the first implementation should be more flexible, especially in future we may be able to extend more than MetadataStore. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-eagle pull request #322: [EAGLE-435] make coordiantor schedule ope...
Github user RalphSu commented on a diff in the pull request: https://github.com/apache/incubator-eagle/pull/322#discussion_r74366625 --- Diff: eagle-core/eagle-alert-parent/eagle-alert/alert-coordinator/src/main/java/org/apache/eagle/alert/coordinator/Coordinator.java --- @@ -98,28 +103,58 @@ public Coordinator(Config config, ConfigBusProducer producer, IMetadataServiceCl } public synchronized ScheduleState schedule(ScheduleOption option) { -Stopwatch watch = Stopwatch.createStarted(); -IScheduleContext context = new ScheduleContextBuilder(client).buildContext(); -TopologyMgmtService mgmtService = new TopologyMgmtService(); -IPolicyScheduler scheduler = PolicySchedulerFactory.createScheduler(); - -scheduler.init(context, mgmtService); -ScheduleState state = scheduler.schedule(option); - -long scheduleTime = watch.elapsed(TimeUnit.MILLISECONDS); -state.setScheduleTimeMillis((int) scheduleTime);// hardcode to integer -watch.reset(); -watch.start(); - -// persist & notify -postSchedule(client, state, producer); - -watch.stop(); -long postTime = watch.elapsed(TimeUnit.MILLISECONDS); -LOG.info("Schedule result, schedule time {} ms, post schedule time {} ms !", scheduleTime, postTime); - -currentState = state; -return state; + ScheduleZkState scheduleZkState = new ScheduleZkState(); + ExclusiveExecutor.Runnable exclusiveRunnable = new ExclusiveExecutor.Runnable() { + @Override + public void run() throws Exception { + scheduleZkState.scheduleAcquired = true; + + while (!scheduleZkState.scheduleCompleted) { + Thread.sleep(ACQUIRE_LOCK_WAIT_INTERVAL_MS); + } + } + }; + ExclusiveExecutor.execute(GREEDY_SCHEDULER_ZK_PATH, exclusiveRunnable); + int waitMaxTimes = 0; + while (waitMaxTimes < ACQUIRE_LOCK_MAX_RETRIES_TIMES) { //about 3 minutes waiting + if (!scheduleZkState.scheduleAcquired) { + waitMaxTimes ++; + try { + Thread.sleep(ACQUIRE_LOCK_WAIT_INTERVAL_MS); + } catch (InterruptedException e) {} + continue; + } + + ScheduleState state = null; + try { + Stopwatch watch = Stopwatch.createStarted(); + IScheduleContext context = new ScheduleContextBuilder(client).buildContext(); + TopologyMgmtService mgmtService = new TopologyMgmtService(); + IPolicyScheduler scheduler = PolicySchedulerFactory.createScheduler(); + + scheduler.init(context, mgmtService); + state = scheduler.schedule(option); + + long scheduleTime = watch.elapsed(TimeUnit.MILLISECONDS); + state.setScheduleTimeMillis((int) scheduleTime);// hardcode to integer + watch.reset(); + watch.start(); + + // persist & notify + postSchedule(client, state, producer); + + watch.stop(); + long postTime = watch.elapsed(TimeUnit.MILLISECONDS); + LOG.info("Schedule result, schedule time {} ms, post schedule time {} ms !", scheduleTime, postTime); + + currentState = state; + } finally { + //schedule completed + scheduleZkState.scheduleCompleted = true; + } + return state; + } + throw new LockWebApplicationException("Acquire greedy scheduler lock failed, please retry later"); --- End diff -- Just the "scheduler lock" is enough? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416507#comment-15416507 ] ASF GitHub Bot commented on EAGLE-435: -- Github user RalphSu commented on a diff in the pull request: https://github.com/apache/incubator-eagle/pull/322#discussion_r74366625 --- Diff: eagle-core/eagle-alert-parent/eagle-alert/alert-coordinator/src/main/java/org/apache/eagle/alert/coordinator/Coordinator.java --- @@ -98,28 +103,58 @@ public Coordinator(Config config, ConfigBusProducer producer, IMetadataServiceCl } public synchronized ScheduleState schedule(ScheduleOption option) { -Stopwatch watch = Stopwatch.createStarted(); -IScheduleContext context = new ScheduleContextBuilder(client).buildContext(); -TopologyMgmtService mgmtService = new TopologyMgmtService(); -IPolicyScheduler scheduler = PolicySchedulerFactory.createScheduler(); - -scheduler.init(context, mgmtService); -ScheduleState state = scheduler.schedule(option); - -long scheduleTime = watch.elapsed(TimeUnit.MILLISECONDS); -state.setScheduleTimeMillis((int) scheduleTime);// hardcode to integer -watch.reset(); -watch.start(); - -// persist & notify -postSchedule(client, state, producer); - -watch.stop(); -long postTime = watch.elapsed(TimeUnit.MILLISECONDS); -LOG.info("Schedule result, schedule time {} ms, post schedule time {} ms !", scheduleTime, postTime); - -currentState = state; -return state; + ScheduleZkState scheduleZkState = new ScheduleZkState(); + ExclusiveExecutor.Runnable exclusiveRunnable = new ExclusiveExecutor.Runnable() { + @Override + public void run() throws Exception { + scheduleZkState.scheduleAcquired = true; + + while (!scheduleZkState.scheduleCompleted) { + Thread.sleep(ACQUIRE_LOCK_WAIT_INTERVAL_MS); + } + } + }; + ExclusiveExecutor.execute(GREEDY_SCHEDULER_ZK_PATH, exclusiveRunnable); + int waitMaxTimes = 0; + while (waitMaxTimes < ACQUIRE_LOCK_MAX_RETRIES_TIMES) { //about 3 minutes waiting + if (!scheduleZkState.scheduleAcquired) { + waitMaxTimes ++; + try { + Thread.sleep(ACQUIRE_LOCK_WAIT_INTERVAL_MS); + } catch (InterruptedException e) {} + continue; + } + + ScheduleState state = null; + try { + Stopwatch watch = Stopwatch.createStarted(); + IScheduleContext context = new ScheduleContextBuilder(client).buildContext(); + TopologyMgmtService mgmtService = new TopologyMgmtService(); + IPolicyScheduler scheduler = PolicySchedulerFactory.createScheduler(); + + scheduler.init(context, mgmtService); + state = scheduler.schedule(option); + + long scheduleTime = watch.elapsed(TimeUnit.MILLISECONDS); + state.setScheduleTimeMillis((int) scheduleTime);// hardcode to integer + watch.reset(); + watch.start(); + + // persist & notify + postSchedule(client, state, producer); + + watch.stop(); + long postTime = watch.elapsed(TimeUnit.MILLISECONDS); + LOG.info("Schedule result, schedule time {} ms, post schedule time {} ms !", scheduleTime, postTime); + + currentState = state; + } finally { + //schedule completed + scheduleZkState.scheduleCompleted = true; + } + return state; + } + throw new LockWebApplicationException("Acquire greedy scheduler lock failed, please retry later"); --- End diff -- Just the "scheduler lock" is enough? > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 >
Re: [HELP] cannot upload eagle jars to maven repository
Tickets created: https://issues.apache.org/jira/browse/INFRA-12409 https://issues.apache.org/jira/browse/INFRA-12410 On Thu, Aug 11, 2016 at 10:07 AM, Michael Wuwrote: > Henry, > > Thank you very much for guiding me with those details! I'll do it right > now. > > Michael > > On Tue, Aug 9, 2016 at 2:39 AM, Henry Saputra > wrote: > >> So, we need to file 2 infra tickets: >> 1. One is to add target in Maven repository for Eagle. You can see sample >> from here: https://issues.apache.org/jira/browse/INFRA-8026 or >> https://issues.apache.org/jira/browse/INFRA-7246 >> 2. Create ticket for you to be granted karma to access Apache Nexus repo. >> >> - Henry >> >> On Mon, Aug 8, 2016 at 12:56 AM, Michael Wu wrote: >> >> > Hi Henry, >> > >> > No, eagle never pushed artifacts to ASF Nexus repository. How can we get >> > the access, please? >> > >> > Michael >> > >> > On Mon, Aug 8, 2016 at 2:21 PM, Henry Saputra >> > wrote: >> > >> > > I found Eagle in ASF release dist folder but do not see it in the >> Nexus >> > > Maven repo (https://repository.apache.org) >> > > >> > > Has Eagle pushed artifacts to ASF Nexus repository for prev releases? >> > > >> > > - Henry >> > > >> > > On Wed, Aug 3, 2016 at 8:04 PM, Michael Wu >> wrote: >> > > >> > > > Hi mentors, >> > > > >> > > > I was trying to use >> > > > >> > > > $> mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_ >> > > PASSPHRASE} >> > > > deploy >> > > > >> > > > to upload eagle's jars to >> > > > https://repository.apache.org/service/local/staging/deploy/maven2, >> but >> > > it >> > > > always return 400 error, as below fragment: >> > > > >> > > > [ERROR] Failed to execute goal >> > > > org.apache.maven.plugins:maven-deploy-plugin:2.8.1:deploy >> > > (default-deploy) >> > > > on project eagle-parent: Failed to deploy artifacts: Could not >> transfer >> > > > artifact org.apache.eagle:eagle-parent:pom:0.4.0-incubating from/to >> > > > apache.releases.https ( >> > > > https://repository.apache.org/service/local/staging/deploy/maven2): >> > > Failed >> > > > to transfer file: >> > > > https://repository.apache.org/service/local/staging/deploy/ >> > > > maven2/org/apache/eagle/eagle-parent/0.4.0-incubating/eagle- >> > > > parent-0.4.0-incubating.pom. >> > > > Return code is: 400, ReasonPhrase: Bad Request. -> [Help 1] >> > > > >> > > > >> > > > Could you instruct me, whether it's caused by the lack of access to >> the >> > > > repository and which mailing list I should get connected with to >> solve >> > > this >> > > > issue, please? Thanks. >> > > > >> > > > Michael >> > > > >> > > >> > >> > >
[GitHub] incubator-eagle pull request #326: [Eagle 443] refactor ProcessedTimeStamp
Github user wujinhu closed the pull request at: https://github.com/apache/incubator-eagle/pull/326 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-440) Alert mongodb storage refine
[ https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416431#comment-15416431 ] ASF GitHub Bot commented on EAGLE-440: -- GitHub user mizeng opened a pull request: https://github.com/apache/incubator-eagle/pull/328 EAGLE-440: Alert mongodb storage refine You can merge this pull request into a Git repository by running: $ git pull https://github.com/mizeng/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/328.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #328 commit 678437aadb5ca6e5c7256fabd40b4966e4227911 Author: mizengDate: 2016-08-10T08:04:16Z EAGLE-440: Alert mongodb storage refine > Alert mongodb storage refine > > > Key: EAGLE-440 > URL: https://issues.apache.org/jira/browse/EAGLE-440 > Project: Eagle > Issue Type: Improvement >Reporter: Zeng, Bryant >Assignee: Zeng, Bryant > > Currently, some Alert Mongo Collection is too large. Consider future > increase, need to split large collection to some small collections. > To back compatible, need to keep the interface as the same as before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #328: EAGLE-440: Alert mongodb storage refine
GitHub user mizeng opened a pull request: https://github.com/apache/incubator-eagle/pull/328 EAGLE-440: Alert mongodb storage refine You can merge this pull request into a Git repository by running: $ git pull https://github.com/mizeng/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/328.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #328 commit 678437aadb5ca6e5c7256fabd40b4966e4227911 Author: mizengDate: 2016-08-10T08:04:16Z EAGLE-440: Alert mongodb storage refine --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-440) Alert mongodb storage refine
[ https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416424#comment-15416424 ] ASF GitHub Bot commented on EAGLE-440: -- Github user mizeng closed the pull request at: https://github.com/apache/incubator-eagle/pull/324 > Alert mongodb storage refine > > > Key: EAGLE-440 > URL: https://issues.apache.org/jira/browse/EAGLE-440 > Project: Eagle > Issue Type: Improvement >Reporter: Zeng, Bryant >Assignee: Zeng, Bryant > > Currently, some Alert Mongo Collection is too large. Consider future > increase, need to split large collection to some small collections. > To back compatible, need to keep the interface as the same as before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #324: EAGLE-440: Alert mongodb storage refine
Github user mizeng closed the pull request at: https://github.com/apache/incubator-eagle/pull/324 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: [HELP] cannot upload eagle jars to maven repository
Henry, Thank you very much for guiding me with those details! I'll do it right now. Michael On Tue, Aug 9, 2016 at 2:39 AM, Henry Saputrawrote: > So, we need to file 2 infra tickets: > 1. One is to add target in Maven repository for Eagle. You can see sample > from here: https://issues.apache.org/jira/browse/INFRA-8026 or > https://issues.apache.org/jira/browse/INFRA-7246 > 2. Create ticket for you to be granted karma to access Apache Nexus repo. > > - Henry > > On Mon, Aug 8, 2016 at 12:56 AM, Michael Wu wrote: > > > Hi Henry, > > > > No, eagle never pushed artifacts to ASF Nexus repository. How can we get > > the access, please? > > > > Michael > > > > On Mon, Aug 8, 2016 at 2:21 PM, Henry Saputra > > wrote: > > > > > I found Eagle in ASF release dist folder but do not see it in the Nexus > > > Maven repo (https://repository.apache.org) > > > > > > Has Eagle pushed artifacts to ASF Nexus repository for prev releases? > > > > > > - Henry > > > > > > On Wed, Aug 3, 2016 at 8:04 PM, Michael Wu wrote: > > > > > > > Hi mentors, > > > > > > > > I was trying to use > > > > > > > > $> mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_ > > > PASSPHRASE} > > > > deploy > > > > > > > > to upload eagle's jars to > > > > https://repository.apache.org/service/local/staging/deploy/maven2, > but > > > it > > > > always return 400 error, as below fragment: > > > > > > > > [ERROR] Failed to execute goal > > > > org.apache.maven.plugins:maven-deploy-plugin:2.8.1:deploy > > > (default-deploy) > > > > on project eagle-parent: Failed to deploy artifacts: Could not > transfer > > > > artifact org.apache.eagle:eagle-parent:pom:0.4.0-incubating from/to > > > > apache.releases.https ( > > > > https://repository.apache.org/service/local/staging/deploy/maven2): > > > Failed > > > > to transfer file: > > > > https://repository.apache.org/service/local/staging/deploy/ > > > > maven2/org/apache/eagle/eagle-parent/0.4.0-incubating/eagle- > > > > parent-0.4.0-incubating.pom. > > > > Return code is: 400, ReasonPhrase: Bad Request. -> [Help 1] > > > > > > > > > > > > Could you instruct me, whether it's caused by the lack of access to > the > > > > repository and which mailing list I should get connected with to > solve > > > this > > > > issue, please? Thanks. > > > > > > > > Michael > > > > > > > > > >
[jira] [Commented] (EAGLE-440) Alert mongodb storage refine
[ https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416372#comment-15416372 ] ASF GitHub Bot commented on EAGLE-440: -- Github user RalphSu commented on a diff in the pull request: https://github.com/apache/incubator-eagle/pull/324#discussion_r74361599 --- Diff: eagle-core/eagle-alert-parent/eagle-alert/alert-metadata-parent/alert-metadata/src/main/java/org/apache/eagle/alert/metadata/impl/MongoMetadataDaoImpl.java --- @@ -285,6 +318,21 @@ public OpResult removePublishmentType(String pubType) { return remove(publishmentType, pubType); } +private OpResult addOne(MongoCollection collection, T t) { +OpResult result = new OpResult(); +try { +String json = mapper.writeValueAsString(t); +collection.insertOne(Document.parse(json)); +result.code = 200; +result.message = "add state succeed!"; +} catch (Exception e) { +result.code = 400; +result.message = e.getMessage(); +LOG.error("", e); --- End diff -- better to have meaningful message > Alert mongodb storage refine > > > Key: EAGLE-440 > URL: https://issues.apache.org/jira/browse/EAGLE-440 > Project: Eagle > Issue Type: Improvement >Reporter: Zeng, Bryant >Assignee: Zeng, Bryant > > Currently, some Alert Mongo Collection is too large. Consider future > increase, need to split large collection to some small collections. > To back compatible, need to keep the interface as the same as before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #324: EAGLE-440: Alert mongodb storage refine
Github user RalphSu commented on a diff in the pull request: https://github.com/apache/incubator-eagle/pull/324#discussion_r74361599 --- Diff: eagle-core/eagle-alert-parent/eagle-alert/alert-metadata-parent/alert-metadata/src/main/java/org/apache/eagle/alert/metadata/impl/MongoMetadataDaoImpl.java --- @@ -285,6 +318,21 @@ public OpResult removePublishmentType(String pubType) { return remove(publishmentType, pubType); } +private OpResult addOne(MongoCollection collection, T t) { +OpResult result = new OpResult(); +try { +String json = mapper.writeValueAsString(t); +collection.insertOne(Document.parse(json)); +result.code = 200; +result.message = "add state succeed!"; +} catch (Exception e) { +result.code = 400; +result.message = e.getMessage(); +LOG.error("", e); --- End diff -- better to have meaningful message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (EAGLE-451) oozie ingestion+alert engine preview
Lingang Deng created EAGLE-451: -- Summary: oozie ingestion+alert engine preview Key: EAGLE-451 URL: https://issues.apache.org/jira/browse/EAGLE-451 Project: Eagle Issue Type: Improvement Reporter: Lingang Deng Assignee: Lingang Deng Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-450) Alert check meta version conflict between runtime modules
Zeng, Bryant created EAGLE-450: -- Summary: Alert check meta version conflict between runtime modules Key: EAGLE-450 URL: https://issues.apache.org/jira/browse/EAGLE-450 Project: Eagle Issue Type: Improvement Reporter: Zeng, Bryant Assignee: Zeng, Bryant -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-449) clean inner process alert engine code
Edward Zhang created EAGLE-449: -- Summary: clean inner process alert engine code Key: EAGLE-449 URL: https://issues.apache.org/jira/browse/EAGLE-449 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Edward Zhang Assignee: Edward Zhang Fix For: v0.5.0 alert engine now is separated into a standalone storm topology, it is not necessary to keep inner process alert engine -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-448) convert eagle-oozie app to use new app framework
Edward Zhang created EAGLE-448: -- Summary: convert eagle-oozie app to use new app framework Key: EAGLE-448 URL: https://issues.apache.org/jira/browse/EAGLE-448 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Edward Zhang Assignee: Edward Zhang Fix For: v0.5.0 eagle-oozie app should use new app framework to manage its lifecycle -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-447) convert eagle-maprfs app to use new app framework
Edward Zhang created EAGLE-447: -- Summary: convert eagle-maprfs app to use new app framework Key: EAGLE-447 URL: https://issues.apache.org/jira/browse/EAGLE-447 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Edward Zhang Assignee: Edward Zhang Fix For: v0.5.0 eagle-maprfs app should use new app framework to manage its lifecycle -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-446) convert eagle-hive app to use new app framework
Edward Zhang created EAGLE-446: -- Summary: convert eagle-hive app to use new app framework Key: EAGLE-446 URL: https://issues.apache.org/jira/browse/EAGLE-446 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Edward Zhang Assignee: Edward Zhang Fix For: v0.5.0 eagle-hive should use new application framework -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-444) convert eagle-gc app to use new app framework
Edward Zhang created EAGLE-444: -- Summary: convert eagle-gc app to use new app framework Key: EAGLE-444 URL: https://issues.apache.org/jira/browse/EAGLE-444 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Edward Zhang Assignee: Edward Zhang Fix For: v0.5.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #321: EAGLE-283: Convert fid,uid in MapR's audi...
Github user DadanielZ closed the pull request at: https://github.com/apache/incubator-eagle/pull/321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-283) Convert fid,uid in MapR's audit log to FIle/folder name, user name
[ https://issues.apache.org/jira/browse/EAGLE-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416192#comment-15416192 ] ASF GitHub Bot commented on EAGLE-283: -- Github user DadanielZ closed the pull request at: https://github.com/apache/incubator-eagle/pull/321 > Convert fid,uid in MapR's audit log to FIle/folder name, user name > -- > > Key: EAGLE-283 > URL: https://issues.apache.org/jira/browse/EAGLE-283 > Project: Eagle > Issue Type: New Feature >Reporter: DanielZhou >Assignee: DanielZhou > > FIle/folder, user's names are not saved in MapR's audit log file, instead, > the audit log file keeps FID, UID. > Need a converter to resolve the relationship between ids and their actual > names. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle issue #321: EAGLE-283: Convert fid,uid in MapR's audit log t...
Github user DadanielZ commented on the issue: https://github.com/apache/incubator-eagle/pull/321 Thank you Edward. Closing this pr as it has been merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (EAGLE-426) Load app web classes and app topology classes using configured lib dir
[ https://issues.apache.org/jira/browse/EAGLE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang closed EAGLE-426. -- > Load app web classes and app topology classes using configured lib dir > -- > > Key: EAGLE-426 > URL: https://issues.apache.org/jira/browse/EAGLE-426 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang >Priority: Minor > Fix For: v0.5.0 > > > App framework has the following issues after remove hard dependency from > eagle server to individual applications > 1. load web app class through jersey package scanner by providing correct > context classloader > 2. load web topology class through correct classloader > Basically we use SPI and application.provider.dir to specify libraries and > load them -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-420) HBase audit log monitoring using new app framework
[ https://issues.apache.org/jira/browse/EAGLE-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang resolved EAGLE-420. Resolution: Fixed > HBase audit log monitoring using new app framework > -- > > Key: EAGLE-420 > URL: https://issues.apache.org/jira/browse/EAGLE-420 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > This ticket is used for converting HBase audit log monitoring to use new > monitoring application framework > New monitoring application supports the following 2 features > 1) in development time, developer can write topology and test topology > without knowing this topology will be managed by Eagle service API > 2) in runtime, operator can manage topology lifecycle using API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (EAGLE-420) HBase audit log monitoring using new app framework
[ https://issues.apache.org/jira/browse/EAGLE-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang closed EAGLE-420. -- > HBase audit log monitoring using new app framework > -- > > Key: EAGLE-420 > URL: https://issues.apache.org/jira/browse/EAGLE-420 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > This ticket is used for converting HBase audit log monitoring to use new > monitoring application framework > New monitoring application supports the following 2 features > 1) in development time, developer can write topology and test topology > without knowing this topology will be managed by Eagle service API > 2) in runtime, operator can manage topology lifecycle using API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-426) Load app web classes and app topology classes using configured lib dir
[ https://issues.apache.org/jira/browse/EAGLE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang resolved EAGLE-426. Resolution: Fixed > Load app web classes and app topology classes using configured lib dir > -- > > Key: EAGLE-426 > URL: https://issues.apache.org/jira/browse/EAGLE-426 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang >Priority: Minor > Fix For: v0.5.0 > > > App framework has the following issues after remove hard dependency from > eagle server to individual applications > 1. load web app class through jersey package scanner by providing correct > context classloader > 2. load web topology class through correct classloader > Basically we use SPI and application.provider.dir to specify libraries and > load them -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-430) explore new packaging method for eagle using new app framework
[ https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang resolved EAGLE-430. Resolution: Fixed > explore new packaging method for eagle using new app framework > -- > > Key: EAGLE-430 > URL: https://issues.apache.org/jira/browse/EAGLE-430 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > New app framework will manage application jars in central place, while > application jars include topology jars and web jars. > We need figure out Eagle 0.5 binary layout. > The goal is with only Eagle 0.5 server we can start any topology remotely. > /eagle-server > /conf (application.conf) > /bin(add class path) > /lib >/eagle-server.jar >/apps > app1.jar > app1-web.jar > app2.jar > /web > index.html > /js -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (EAGLE-430) explore new packaging method for eagle using new app framework
[ https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang closed EAGLE-430. -- > explore new packaging method for eagle using new app framework > -- > > Key: EAGLE-430 > URL: https://issues.apache.org/jira/browse/EAGLE-430 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > New app framework will manage application jars in central place, while > application jars include topology jars and web jars. > We need figure out Eagle 0.5 binary layout. > The goal is with only Eagle 0.5 server we can start any topology remotely. > /eagle-server > /conf (application.conf) > /bin(add class path) > /lib >/eagle-server.jar >/apps > app1.jar > app1-web.jar > app2.jar > /web > index.html > /js -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (EAGLE-437) create workable fat jar for starting application remotely
[ https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang closed EAGLE-437. -- > create workable fat jar for starting application remotely > - > > Key: EAGLE-437 > URL: https://issues.apache.org/jira/browse/EAGLE-437 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Explore ways to package fat jar for starting applications > 1) package all topology jars together into one single fat jar. Need fix >a. metadata.xml should be differentiated by different directory, for > example metadata.xml is under directory of > /META-INF/apps//metadata.xml >b. SPI configuration file should be removed and > ApplicationProviderSPILoader should be redesign to get all providers > 2) package fat jar on demand. > No far jar after packaging. > Need remember all dependent jars and write code to package fat jar on > demand -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-437) create workable fat jar for starting application remotely
[ https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang resolved EAGLE-437. Resolution: Fixed > create workable fat jar for starting application remotely > - > > Key: EAGLE-437 > URL: https://issues.apache.org/jira/browse/EAGLE-437 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Explore ways to package fat jar for starting applications > 1) package all topology jars together into one single fat jar. Need fix >a. metadata.xml should be differentiated by different directory, for > example metadata.xml is under directory of > /META-INF/apps//metadata.xml >b. SPI configuration file should be removed and > ApplicationProviderSPILoader should be redesign to get all providers > 2) package fat jar on demand. > No far jar after packaging. > Need remember all dependent jars and write code to package fat jar on > demand -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (EAGLE-428) Hbase monitoring's metadata browse API in new app framework
[ https://issues.apache.org/jira/browse/EAGLE-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang closed EAGLE-428. -- > Hbase monitoring's metadata browse API in new app framework > --- > > Key: EAGLE-428 > URL: https://issues.apache.org/jira/browse/EAGLE-428 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Hbase monitoring's metadata browse API is used for retrieve hbase's table, > column etc, and check against sensitivity type. > We should use the configuration from ApplicationEntity to manage metadata > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-428) Hbase monitoring's metadata browse API in new app framework
[ https://issues.apache.org/jira/browse/EAGLE-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Zhang resolved EAGLE-428. Resolution: Fixed > Hbase monitoring's metadata browse API in new app framework > --- > > Key: EAGLE-428 > URL: https://issues.apache.org/jira/browse/EAGLE-428 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Hbase monitoring's metadata browse API is used for retrieve hbase's table, > column etc, and check against sensitivity type. > We should use the configuration from ApplicationEntity to manage metadata > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle issue #326: [Eagle 443] refactor ProcessedTimeStamp
Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/326 reviewed, good to go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-283) Convert fid,uid in MapR's audit log to FIle/folder name, user name
[ https://issues.apache.org/jira/browse/EAGLE-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416146#comment-15416146 ] ASF GitHub Bot commented on EAGLE-283: -- Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/321 Thanks for contributing this. I have merged that please close this PR > Convert fid,uid in MapR's audit log to FIle/folder name, user name > -- > > Key: EAGLE-283 > URL: https://issues.apache.org/jira/browse/EAGLE-283 > Project: Eagle > Issue Type: New Feature >Reporter: DanielZhou >Assignee: DanielZhou > > FIle/folder, user's names are not saved in MapR's audit log file, instead, > the audit log file keeps FID, UID. > Need a converter to resolve the relationship between ids and their actual > names. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle issue #321: EAGLE-283: Convert fid,uid in MapR's audit log t...
Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/321 Thanks for contributing this. I have merged that please close this PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-eagle issue #323: EAGLE-442 Support ApplicationProvider to extend ...
Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/323 I have concern with module scope, application only needs know what is current metadata store type, looks there is no need for scope. Do you have other use cases where application need know about framework. But I think you can still merge, and we can watch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416129#comment-15416129 ] ASF GitHub Bot commented on EAGLE-442: -- Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/323 I have concern with module scope, application only needs know what is current metadata store type, looks there is no need for scope. Do you have other use cases where application need know about framework. But I think you can still merge, and we can watch > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > h1. Module Registry Interfaces > * ApplicationProvider > {code} > ApplicationProvider{ >void register(ModuleRegistry) > } > {code} > * ModuleRegistry > {code} > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > {code} > * ModuleScope > - Modules registered in scope of `GlobalScope` will be visible globally > - Modules registered in > `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only > be visible when selected MetadataStore is active (set in configuration file) > {code} > ModuleScope > |- GlobalScope > |- MetadataStore > |- MySQLMetadataStore > |- MongodbMetadataStore > |- MemoryMetadataStore > {code} > h1. Example > * Registry: in `HBaseAuditLogAppProvider` > {code} > @Override > public void register(ModuleRegistry registry) { > registry.register(MemoryMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); > } > }); > registry.register(MySQLMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); > } > }); > } > {code} > * Inject: For example, `HbaseMetadataBrowseWebResource` could get current > `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as > following, instead of using > [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) > {code} > @Inject > public HbaseMetadataBrowseWebResource(ApplicationEntityService > entityService, ISecurityMetadataDAO metadataDAO){ > this.entityService = entityService; > this.dao = metadataDAO; > } > {code} > h1. More Use Cases > * ExampleResource > * ExampleService > * ExampleApplicationProviderTest#testApplicationExtensions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-437) create workable fat jar for starting application remotely
[ https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416039#comment-15416039 ] ASF GitHub Bot commented on EAGLE-437: -- Github user yonzhang closed the pull request at: https://github.com/apache/incubator-eagle/pull/327 > create workable fat jar for starting application remotely > - > > Key: EAGLE-437 > URL: https://issues.apache.org/jira/browse/EAGLE-437 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Explore ways to package fat jar for starting applications > 1) package all topology jars together into one single fat jar. Need fix >a. metadata.xml should be differentiated by different directory, for > example metadata.xml is under directory of > /META-INF/apps//metadata.xml >b. SPI configuration file should be removed and > ApplicationProviderSPILoader should be redesign to get all providers > 2) package fat jar on demand. > No far jar after packaging. > Need remember all dependent jars and write code to package fat jar on > demand -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #327: Eagle assembly for new app framework
Github user yonzhang closed the pull request at: https://github.com/apache/incubator-eagle/pull/327 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-eagle issue #327: Eagle assembly for new app framework
Github user yonzhang commented on the issue: https://github.com/apache/incubator-eagle/pull/327 Already reviewed by Hao in patch 315 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-437) create workable fat jar for starting application remotely
[ https://issues.apache.org/jira/browse/EAGLE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416028#comment-15416028 ] ASF GitHub Bot commented on EAGLE-437: -- GitHub user yonzhang opened a pull request: https://github.com/apache/incubator-eagle/pull/327 Eagle assembly for new app framework https://issues.apache.org/jira/browse/EAGLE-437 https://issues.apache.org/jira/browse/EAGLE-430 You can merge this pull request into a Git repository by running: $ git pull https://github.com/yonzhang/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/327.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #327 commit feeece3f2b5af9e2c2fa17af6e8806f50a8a2930 Author: yonzhangDate: 2016-08-10T21:17:04Z eagle assembly for new app framework > create workable fat jar for starting application remotely > - > > Key: EAGLE-437 > URL: https://issues.apache.org/jira/browse/EAGLE-437 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > Explore ways to package fat jar for starting applications > 1) package all topology jars together into one single fat jar. Need fix >a. metadata.xml should be differentiated by different directory, for > example metadata.xml is under directory of > /META-INF/apps//metadata.xml >b. SPI configuration file should be removed and > ApplicationProviderSPILoader should be redesign to get all providers > 2) package fat jar on demand. > No far jar after packaging. > Need remember all dependent jars and write code to package fat jar on > demand -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #327: Eagle assembly for new app framework
GitHub user yonzhang opened a pull request: https://github.com/apache/incubator-eagle/pull/327 Eagle assembly for new app framework https://issues.apache.org/jira/browse/EAGLE-437 https://issues.apache.org/jira/browse/EAGLE-430 You can merge this pull request into a Git repository by running: $ git pull https://github.com/yonzhang/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/327.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #327 commit feeece3f2b5af9e2c2fa17af6e8806f50a8a2930 Author: yonzhangDate: 2016-08-10T21:17:04Z eagle assembly for new app framework --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-430) explore new packaging method for eagle using new app framework
[ https://issues.apache.org/jira/browse/EAGLE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416027#comment-15416027 ] ASF GitHub Bot commented on EAGLE-430: -- Github user yonzhang closed the pull request at: https://github.com/apache/incubator-eagle/pull/315 > explore new packaging method for eagle using new app framework > -- > > Key: EAGLE-430 > URL: https://issues.apache.org/jira/browse/EAGLE-430 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Edward Zhang > Fix For: v0.5.0 > > > New app framework will manage application jars in central place, while > application jars include topology jars and web jars. > We need figure out Eagle 0.5 binary layout. > The goal is with only Eagle 0.5 server we can start any topology remotely. > /eagle-server > /conf (application.conf) > /bin(add class path) > /lib >/eagle-server.jar >/apps > app1.jar > app1-web.jar > app2.jar > /web > index.html > /js -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #315: explore new packaging method for eagle us...
Github user yonzhang closed the pull request at: https://github.com/apache/incubator-eagle/pull/315 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-eagle pull request #326: [Eagle 443] refactor ProcessedTimeStamp
GitHub user wujinhu opened a pull request: https://github.com/apache/incubator-eagle/pull/326 [Eagle 443] refactor ProcessedTimeStamp You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/incubator-eagle EAGLE-443 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/326.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #326 commit 9ff8072f454d3881c4532b89f5e1243e99d72384 Author: jinhuwuDate: 2016-07-08T06:44:27Z fix build problem and remove unused files commit 4b044057a782367393e19b8c64aec443b3274c89 Author: jinhuwu Date: 2016-07-14T01:58:48Z add spark running job support commit 4bed3e38c24b3af2e0a296d0e99960e72ef7a4a9 Author: jinhuwu Date: 2016-07-14T11:50:12Z add spark running job support commit 5f68e6fdf9b8e3d50b8d0917572f6418004614a9 Author: jinhuwu Date: 2016-07-14T14:50:49Z add spark running job support commit 9b72c49f62701014f1e5c061910940389e5ea9a2 Author: jinhuwu Date: 2016-07-18T06:23:03Z refactor eagle-jpm-mr-history commit 12a81a5d18b1e377a30337243b19001c5ee83c3e Author: pkuwm Date: 2016-07-18T18:26:09Z Refactor. commit e12e4744dbf33ffdffeeeb9c03407ea8d346104b Author: jinhuwu Date: 2016-07-21T06:32:19Z refactor eagle-jpm-mr-history commit 202541db4bdbd03474a051a64f1208dbc1eee549 Author: jinhuwu Date: 2016-07-21T06:36:33Z add mr running job feeder commit cc119ad3fab65fe3fe1e49efe0780e249896b156 Author: jinhuwu Date: 2016-07-21T08:14:53Z add mr running job feeder commit 120db5a91ab83e3a782d716a7cb7f0ed343e8f31 Author: jinhuwu Date: 2016-07-21T11:27:08Z add mr running job feeder commit 23bf0db0a66efea3b2fcc44f0b0fc69d2ae7aeab Author: jinhuwu Date: 2016-07-22T05:47:47Z bug fix commit e1aa1b7e86ae8031f1e18bb989c45ebe7aec7a53 Author: pkuwm Date: 2016-07-26T18:19:58Z Parse spark event log to get jobconfig. commit f2bf638d013b912b1c575a8d014e66651519baa5 Author: pkuwm Date: 2016-07-26T21:47:20Z Merge branch 'develop' of github.com:wujinhu/incubator-eagle into develop commit 5de5c310d085d61fa2f17fc33dc443aec5b377e8 Author: pkuwm Date: 2016-07-29T21:00:03Z Format date to timestamp: dateTimeToLong() commit f392316dab5e101ba76f115640b3dc8fbbee5ce7 Author: pkuwm Date: 2016-07-29T21:34:16Z Parse history logs with .inprogress extension. commit ed1dad3fcc6c25bfbfc6b0a6fb55b2a669a11400 Author: pkuwm Date: 2016-07-29T21:38:15Z Fix: driver executor does not have timestamp. commit 0fe2dca05f293ce128a8cf6a470485184e3cb621 Author: pkuwm Date: 2016-07-29T22:11:58Z Get attempt Id for Yarn version 2.4.x commit 11a2e2103e67390672bc35fa20a292b00807ae69 Author: pkuwm Date: 2016-07-30T00:16:32Z Get clusterInfo from Yarn api. ResourceManager version is included in clusterInfo. commit 39b9bddde98cd33e4eaaf94aee88922b15fca35e Author: pkuwm Date: 2016-07-30T00:24:55Z Fix: refactoring issue. commit 8f024d1989ec23f9ac682c144f1a4222fdb4cc2e Author: jinhuwu Date: 2016-07-31T15:14:15Z bug fix commit 77f8f155a3fd2342c8b6f35ad4492b37d099b49f Author: pkuwm Date: 2016-08-01T17:34:21Z Fix: parseAndValidateJSON, json deserialization exception and BufferedReader close. commit 6fc3bcf36a3588a372f61f9f3a1d0cbc97cd94a5 Author: jinhuwu Date: 2016-08-02T09:21:11Z bug fux commit 4f4ae12f1328a13a56e75f5e8f330b7a637cda95 Author: jinhuwu Date: 2016-08-02T09:23:02Z Merge branch 'develop' of https://github.com/wujinhu/incubator-eagle into develop # Conflicts: # eagle-jpm/eagle-jpm-spark-running/src/main/java/org/apache/eagle/jpm/spark/running/parser/JHFSparkRunningJobParser.java commit 12f670878e0d1650e618af707d0bf2f562080155 Author: jinhuwu Date: 2016-08-02T09:55:42Z bug fix commit 95a2fe5878d778a1b6264ba1e936e6074da70b14 Author: jinhuwu Date: 2016-08-02T10:10:08Z bug fix commit 04e6adf58a02a6b233ad708fb3e0bcfd29237f67 Author: pkuwm Date: 2016-08-02T19:51:58Z Set attemptId = 1 for yarn-client mode. commit c6419302b0d01a989d26b6f671d067e17b591ac2 Author: pkuwm Date: 2016-08-02T22:13:38Z Use try-with-resources statement to handle inputstream and fix bugs. commit eb478c1bb6b4aefa4cb8b67ad233bd7ef8ef86da Author: pkuwm
[jira] [Commented] (EAGLE-443) refactor ProcessedTimeStamp
[ https://issues.apache.org/jira/browse/EAGLE-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415184#comment-15415184 ] ASF GitHub Bot commented on EAGLE-443: -- Github user wujinhu closed the pull request at: https://github.com/apache/incubator-eagle/pull/325 > refactor ProcessedTimeStamp > --- > > Key: EAGLE-443 > URL: https://issues.apache.org/jira/browse/EAGLE-443 > Project: Eagle > Issue Type: Improvement >Reporter: wujinhu >Assignee: wujinhu > > In the previous implementation, we use a blot to update processed job time > stamp. But we can do it in a simple way. The spout can update itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #325: [EAGLE-443] refactor ProcessedTimeStamp
Github user wujinhu closed the pull request at: https://github.com/apache/incubator-eagle/pull/325 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-eagle issue #315: explore new packaging method for eagle using new...
Github user haoch commented on the issue: https://github.com/apache/incubator-eagle/pull/315 @yonzhang it seems to fail in unit test, but never mind, i fixed in #323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (EAGLE-278) MR history jobs Feeder
[ https://issues.apache.org/jira/browse/EAGLE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu resolved EAGLE-278. --- Resolution: Resolved > MR history jobs Feeder > -- > > Key: EAGLE-278 > URL: https://issues.apache.org/jira/browse/EAGLE-278 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.4.0 >Reporter: Jing Ge >Assignee: wujinhu > Labels: JPM > Fix For: v0.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-277) Spark History jobs Feeder
[ https://issues.apache.org/jira/browse/EAGLE-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu resolved EAGLE-277. --- Resolution: Resolved > Spark History jobs Feeder > - > > Key: EAGLE-277 > URL: https://issues.apache.org/jira/browse/EAGLE-277 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.4.0 >Reporter: Jing Ge >Assignee: wujinhu > Labels: JPM > Fix For: v0.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-443) refactor ProcessedTimeStamp
[ https://issues.apache.org/jira/browse/EAGLE-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415132#comment-15415132 ] ASF GitHub Bot commented on EAGLE-443: -- GitHub user wujinhu opened a pull request: https://github.com/apache/incubator-eagle/pull/325 [EAGLE-443] refactor ProcessedTimeStamp You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/325.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #325 commit 9ff8072f454d3881c4532b89f5e1243e99d72384 Author: jinhuwuDate: 2016-07-08T06:44:27Z fix build problem and remove unused files commit 4b044057a782367393e19b8c64aec443b3274c89 Author: jinhuwu Date: 2016-07-14T01:58:48Z add spark running job support commit 4bed3e38c24b3af2e0a296d0e99960e72ef7a4a9 Author: jinhuwu Date: 2016-07-14T11:50:12Z add spark running job support commit 5f68e6fdf9b8e3d50b8d0917572f6418004614a9 Author: jinhuwu Date: 2016-07-14T14:50:49Z add spark running job support commit 9b72c49f62701014f1e5c061910940389e5ea9a2 Author: jinhuwu Date: 2016-07-18T06:23:03Z refactor eagle-jpm-mr-history commit 12a81a5d18b1e377a30337243b19001c5ee83c3e Author: pkuwm Date: 2016-07-18T18:26:09Z Refactor. commit e12e4744dbf33ffdffeeeb9c03407ea8d346104b Author: jinhuwu Date: 2016-07-21T06:32:19Z refactor eagle-jpm-mr-history commit 202541db4bdbd03474a051a64f1208dbc1eee549 Author: jinhuwu Date: 2016-07-21T06:36:33Z add mr running job feeder commit cc119ad3fab65fe3fe1e49efe0780e249896b156 Author: jinhuwu Date: 2016-07-21T08:14:53Z add mr running job feeder commit 120db5a91ab83e3a782d716a7cb7f0ed343e8f31 Author: jinhuwu Date: 2016-07-21T11:27:08Z add mr running job feeder commit 23bf0db0a66efea3b2fcc44f0b0fc69d2ae7aeab Author: jinhuwu Date: 2016-07-22T05:47:47Z bug fix commit e1aa1b7e86ae8031f1e18bb989c45ebe7aec7a53 Author: pkuwm Date: 2016-07-26T18:19:58Z Parse spark event log to get jobconfig. commit f2bf638d013b912b1c575a8d014e66651519baa5 Author: pkuwm Date: 2016-07-26T21:47:20Z Merge branch 'develop' of github.com:wujinhu/incubator-eagle into develop commit 5de5c310d085d61fa2f17fc33dc443aec5b377e8 Author: pkuwm Date: 2016-07-29T21:00:03Z Format date to timestamp: dateTimeToLong() commit f392316dab5e101ba76f115640b3dc8fbbee5ce7 Author: pkuwm Date: 2016-07-29T21:34:16Z Parse history logs with .inprogress extension. commit ed1dad3fcc6c25bfbfc6b0a6fb55b2a669a11400 Author: pkuwm Date: 2016-07-29T21:38:15Z Fix: driver executor does not have timestamp. commit 0fe2dca05f293ce128a8cf6a470485184e3cb621 Author: pkuwm Date: 2016-07-29T22:11:58Z Get attempt Id for Yarn version 2.4.x commit 11a2e2103e67390672bc35fa20a292b00807ae69 Author: pkuwm Date: 2016-07-30T00:16:32Z Get clusterInfo from Yarn api. ResourceManager version is included in clusterInfo. commit 39b9bddde98cd33e4eaaf94aee88922b15fca35e Author: pkuwm Date: 2016-07-30T00:24:55Z Fix: refactoring issue. commit 8f024d1989ec23f9ac682c144f1a4222fdb4cc2e Author: jinhuwu Date: 2016-07-31T15:14:15Z bug fix commit 77f8f155a3fd2342c8b6f35ad4492b37d099b49f Author: pkuwm Date: 2016-08-01T17:34:21Z Fix: parseAndValidateJSON, json deserialization exception and BufferedReader close. commit 6fc3bcf36a3588a372f61f9f3a1d0cbc97cd94a5 Author: jinhuwu Date: 2016-08-02T09:21:11Z bug fux commit 4f4ae12f1328a13a56e75f5e8f330b7a637cda95 Author: jinhuwu Date: 2016-08-02T09:23:02Z Merge branch 'develop' of https://github.com/wujinhu/incubator-eagle into develop # Conflicts: # eagle-jpm/eagle-jpm-spark-running/src/main/java/org/apache/eagle/jpm/spark/running/parser/JHFSparkRunningJobParser.java commit 12f670878e0d1650e618af707d0bf2f562080155 Author: jinhuwu Date: 2016-08-02T09:55:42Z bug fix commit 95a2fe5878d778a1b6264ba1e936e6074da70b14 Author: jinhuwu Date: 2016-08-02T10:10:08Z bug fix commit 04e6adf58a02a6b233ad708fb3e0bcfd29237f67 Author: pkuwm Date: 2016-08-02T19:51:58Z Set attemptId = 1 for yarn-client mode. commit
[jira] [Resolved] (EAGLE-443) refactor ProcessedTimeStamp
[ https://issues.apache.org/jira/browse/EAGLE-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu resolved EAGLE-443. --- Resolution: Resolved > refactor ProcessedTimeStamp > --- > > Key: EAGLE-443 > URL: https://issues.apache.org/jira/browse/EAGLE-443 > Project: Eagle > Issue Type: Improvement >Reporter: wujinhu >Assignee: wujinhu > > In the previous implementation, we use a blot to update processed job time > stamp. But we can do it in a simple way. The spout can update itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #325: [EAGLE-443] refactor ProcessedTimeStamp
GitHub user wujinhu opened a pull request: https://github.com/apache/incubator-eagle/pull/325 [EAGLE-443] refactor ProcessedTimeStamp You can merge this pull request into a Git repository by running: $ git pull https://github.com/wujinhu/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/325.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #325 commit 9ff8072f454d3881c4532b89f5e1243e99d72384 Author: jinhuwuDate: 2016-07-08T06:44:27Z fix build problem and remove unused files commit 4b044057a782367393e19b8c64aec443b3274c89 Author: jinhuwu Date: 2016-07-14T01:58:48Z add spark running job support commit 4bed3e38c24b3af2e0a296d0e99960e72ef7a4a9 Author: jinhuwu Date: 2016-07-14T11:50:12Z add spark running job support commit 5f68e6fdf9b8e3d50b8d0917572f6418004614a9 Author: jinhuwu Date: 2016-07-14T14:50:49Z add spark running job support commit 9b72c49f62701014f1e5c061910940389e5ea9a2 Author: jinhuwu Date: 2016-07-18T06:23:03Z refactor eagle-jpm-mr-history commit 12a81a5d18b1e377a30337243b19001c5ee83c3e Author: pkuwm Date: 2016-07-18T18:26:09Z Refactor. commit e12e4744dbf33ffdffeeeb9c03407ea8d346104b Author: jinhuwu Date: 2016-07-21T06:32:19Z refactor eagle-jpm-mr-history commit 202541db4bdbd03474a051a64f1208dbc1eee549 Author: jinhuwu Date: 2016-07-21T06:36:33Z add mr running job feeder commit cc119ad3fab65fe3fe1e49efe0780e249896b156 Author: jinhuwu Date: 2016-07-21T08:14:53Z add mr running job feeder commit 120db5a91ab83e3a782d716a7cb7f0ed343e8f31 Author: jinhuwu Date: 2016-07-21T11:27:08Z add mr running job feeder commit 23bf0db0a66efea3b2fcc44f0b0fc69d2ae7aeab Author: jinhuwu Date: 2016-07-22T05:47:47Z bug fix commit e1aa1b7e86ae8031f1e18bb989c45ebe7aec7a53 Author: pkuwm Date: 2016-07-26T18:19:58Z Parse spark event log to get jobconfig. commit f2bf638d013b912b1c575a8d014e66651519baa5 Author: pkuwm Date: 2016-07-26T21:47:20Z Merge branch 'develop' of github.com:wujinhu/incubator-eagle into develop commit 5de5c310d085d61fa2f17fc33dc443aec5b377e8 Author: pkuwm Date: 2016-07-29T21:00:03Z Format date to timestamp: dateTimeToLong() commit f392316dab5e101ba76f115640b3dc8fbbee5ce7 Author: pkuwm Date: 2016-07-29T21:34:16Z Parse history logs with .inprogress extension. commit ed1dad3fcc6c25bfbfc6b0a6fb55b2a669a11400 Author: pkuwm Date: 2016-07-29T21:38:15Z Fix: driver executor does not have timestamp. commit 0fe2dca05f293ce128a8cf6a470485184e3cb621 Author: pkuwm Date: 2016-07-29T22:11:58Z Get attempt Id for Yarn version 2.4.x commit 11a2e2103e67390672bc35fa20a292b00807ae69 Author: pkuwm Date: 2016-07-30T00:16:32Z Get clusterInfo from Yarn api. ResourceManager version is included in clusterInfo. commit 39b9bddde98cd33e4eaaf94aee88922b15fca35e Author: pkuwm Date: 2016-07-30T00:24:55Z Fix: refactoring issue. commit 8f024d1989ec23f9ac682c144f1a4222fdb4cc2e Author: jinhuwu Date: 2016-07-31T15:14:15Z bug fix commit 77f8f155a3fd2342c8b6f35ad4492b37d099b49f Author: pkuwm Date: 2016-08-01T17:34:21Z Fix: parseAndValidateJSON, json deserialization exception and BufferedReader close. commit 6fc3bcf36a3588a372f61f9f3a1d0cbc97cd94a5 Author: jinhuwu Date: 2016-08-02T09:21:11Z bug fux commit 4f4ae12f1328a13a56e75f5e8f330b7a637cda95 Author: jinhuwu Date: 2016-08-02T09:23:02Z Merge branch 'develop' of https://github.com/wujinhu/incubator-eagle into develop # Conflicts: # eagle-jpm/eagle-jpm-spark-running/src/main/java/org/apache/eagle/jpm/spark/running/parser/JHFSparkRunningJobParser.java commit 12f670878e0d1650e618af707d0bf2f562080155 Author: jinhuwu Date: 2016-08-02T09:55:42Z bug fix commit 95a2fe5878d778a1b6264ba1e936e6074da70b14 Author: jinhuwu Date: 2016-08-02T10:10:08Z bug fix commit 04e6adf58a02a6b233ad708fb3e0bcfd29237f67 Author: pkuwm Date: 2016-08-02T19:51:58Z Set attemptId = 1 for yarn-client mode. commit c6419302b0d01a989d26b6f671d067e17b591ac2 Author: pkuwm Date: 2016-08-02T22:13:38Z Use try-with-resources statement to handle inputstream and fix bugs. commit eb478c1bb6b4aefa4cb8b67ad233bd7ef8ef86da Author: pkuwm
[jira] [Created] (EAGLE-443) refactor ProcessedTimeStamp
wujinhu created EAGLE-443: - Summary: refactor ProcessedTimeStamp Key: EAGLE-443 URL: https://issues.apache.org/jira/browse/EAGLE-443 Project: Eagle Issue Type: Improvement Reporter: wujinhu Assignee: wujinhu In the previous implementation, we use a blot to update processed job time stamp. But we can do it in a simple way. The spout can update itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen updated EAGLE-442: --- Description: h1. Module Registry Interfaces * ApplicationProvider {code} ApplicationProvider{ void register(ModuleRegistry) } {code} * ModuleRegistry {code} ModuleRegistry { void register(ModuleScope scope,Module module); default void register(Module moduel) { register(GlobalScope.class, module) } } {code} * ModuleScope - Modules registered in scope of `GlobalScope` will be visible globally - Modules registered in `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only be visible when selected MetadataStore is active (set in configuration file) {code} ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore {code} h1. Example * Registry: in `HBaseAuditLogAppProvider` {code} @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } {code} * Inject: For example, `HbaseMetadataBrowseWebResource` could get current `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as following, instead of using [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) {code} @Inject public HbaseMetadataBrowseWebResource(ApplicationEntityService entityService, ISecurityMetadataDAO metadataDAO){ this.entityService = entityService; this.dao = metadataDAO; } {code} h1. More Use Cases * ExampleResource * ExampleService * ExampleApplicationProviderTest#testApplicationExtensions was: ## Module Registry Interfaces * ApplicationProvider ``` ApplicationProvider{ void register(ModuleRegistry) } ``` * ModuleRegistry ``` ModuleRegistry { void register(ModuleScope scope,Module module); default void register(Module moduel) { register(GlobalScope.class, module) } } ``` * ModuleScope - Modules registered in scope of `GlobalScope` will be visible globally - Modules registered in `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only be visible when selected MetadataStore is active (set in configuration file) ``` ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore ``` ## Example * Registry: in `HBaseAuditLogAppProvider` ``` @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } ``` * Inject: For example, `HbaseMetadataBrowseWebResource` could get current `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as following, instead of using [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) ``` @Inject public HbaseMetadataBrowseWebResource(ApplicationEntityService entityService, ISecurityMetadataDAO metadataDAO){ this.entityService = entityService; this.dao = metadataDAO; } ``` ## More Use Cases * ExampleResource * ExampleService * ExampleApplicationProviderTest#testApplicationExtensions ## JIRA * https://issues.apache.org/jira/browse/EAGLE-442 * https://issues.apache.org/jira/browse/EAGLE-431 > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > h1. Module
[jira] [Updated] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen updated EAGLE-442: --- Description: Module Registry Interfaces ApplicationProvider ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry ModuleRegistry { void register(ModuleScope scope,Module module); default void register(Module moduel) { register(GlobalScope.class, module) } } ModuleScope Modules registered in scope of GlobalScope will be visible globally Modules registered in MySQLMetadataStore/MongodbMetadataStore/MemoryMetadataStore will only be visible when selected MetadataStore is active (set in configuration file) ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore Example Registry: in HBaseAuditLogAppProvider @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } Inject: For example, HbaseMetadataBrowseWebResource could get current ISecurityMetadataDAO with @Inject ISecurityMetadataDAO metadataDAO as following, instead of using MetadataDaoFactory @Inject public HbaseMetadataBrowseWebResource(ApplicationEntityService entityService, ISecurityMetadataDAO metadataDAO){ this.entityService = entityService; this.dao = metadataDAO; } More Use Cases ExampleResource ExampleService ExampleApplicationProviderTest#testApplicationExtensions > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > Module Registry Interfaces > ApplicationProvider > ApplicationProvider{ >void register(ModuleRegistry) > } > ModuleRegistry > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > ModuleScope > Modules registered in scope of GlobalScope will be visible globally > Modules registered in > MySQLMetadataStore/MongodbMetadataStore/MemoryMetadataStore will only be > visible when selected MetadataStore is active (set in configuration file) > ModuleScope > |- GlobalScope > |- MetadataStore > |- MySQLMetadataStore > |- MongodbMetadataStore > |- MemoryMetadataStore > Example > Registry: in HBaseAuditLogAppProvider > @Override > public void register(ModuleRegistry registry) { > registry.register(MemoryMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); > } > }); > registry.register(MySQLMetadataStore.class, new AbstractModule() { > @Override > protected void configure() { > > bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); > } > }); > } > Inject: For example, HbaseMetadataBrowseWebResource could get current > ISecurityMetadataDAO with @Inject ISecurityMetadataDAO metadataDAO as > following, instead of using MetadataDaoFactory > @Inject > public HbaseMetadataBrowseWebResource(ApplicationEntityService > entityService, ISecurityMetadataDAO metadataDAO){ > this.entityService = entityService; > this.dao = metadataDAO; > } > More Use Cases > ExampleResource > ExampleService > ExampleApplicationProviderTest#testApplicationExtensions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen updated EAGLE-442: --- Description: ## Module Registry Interfaces * ApplicationProvider ``` ApplicationProvider{ void register(ModuleRegistry) } ``` * ModuleRegistry ``` ModuleRegistry { void register(ModuleScope scope,Module module); default void register(Module moduel) { register(GlobalScope.class, module) } } ``` * ModuleScope - Modules registered in scope of `GlobalScope` will be visible globally - Modules registered in `MySQLMetadataStore`/`MongodbMetadataStore`/`MemoryMetadataStore` will only be visible when selected MetadataStore is active (set in configuration file) ``` ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore ``` ## Example * Registry: in `HBaseAuditLogAppProvider` ``` @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } ``` * Inject: For example, `HbaseMetadataBrowseWebResource` could get current `ISecurityMetadataDAO` with `@Inject ISecurityMetadataDAO metadataDAO` as following, instead of using [`MetadataDaoFactory`](https://github.com/apache/incubator-eagle/blob/9b10f1b60f0f6451e71ac3396b326f9565ef/eagle-security/eagle-security-common/src/main/java/org/apache/eagle/security/service/MetadataDaoFactory.java) ``` @Inject public HbaseMetadataBrowseWebResource(ApplicationEntityService entityService, ISecurityMetadataDAO metadataDAO){ this.entityService = entityService; this.dao = metadataDAO; } ``` ## More Use Cases * ExampleResource * ExampleService * ExampleApplicationProviderTest#testApplicationExtensions ## JIRA * https://issues.apache.org/jira/browse/EAGLE-442 * https://issues.apache.org/jira/browse/EAGLE-431 was: Module Registry Interfaces ApplicationProvider ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry ModuleRegistry { void register(ModuleScope scope,Module module); default void register(Module moduel) { register(GlobalScope.class, module) } } ModuleScope Modules registered in scope of GlobalScope will be visible globally Modules registered in MySQLMetadataStore/MongodbMetadataStore/MemoryMetadataStore will only be visible when selected MetadataStore is active (set in configuration file) ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore Example Registry: in HBaseAuditLogAppProvider @Override public void register(ModuleRegistry registry) { registry.register(MemoryMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(InMemMetadataDaoImpl.class); } }); registry.register(MySQLMetadataStore.class, new AbstractModule() { @Override protected void configure() { bind(ISecurityMetadataDAO.class).to(JDBCSecurityMetadataDAO.class); } }); } Inject: For example, HbaseMetadataBrowseWebResource could get current ISecurityMetadataDAO with @Inject ISecurityMetadataDAO metadataDAO as following, instead of using MetadataDaoFactory @Inject public HbaseMetadataBrowseWebResource(ApplicationEntityService entityService, ISecurityMetadataDAO metadataDAO){ this.entityService = entityService; this.dao = metadataDAO; } More Use Cases ExampleResource ExampleService ExampleApplicationProviderTest#testApplicationExtensions > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > > ## Module Registry Interfaces > * ApplicationProvider > ``` > ApplicationProvider{ >void register(ModuleRegistry) > } > ``` > * ModuleRegistry > ``` > ModuleRegistry { > void register(ModuleScope scope,Module module); > default void register(Module moduel) { > register(GlobalScope.class, module) > } > } > ``` > *
[jira] [Commented] (EAGLE-431) MetadataDaoFactory should use config from eagle server
[ https://issues.apache.org/jira/browse/EAGLE-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415043#comment-15415043 ] Hao Chen commented on EAGLE-431: In EAGLE-442, application is able to extend metadata modules by scope (global/memory/mysql/mongod ..) in SPI, so resources/services could inject required service like Config with @Inject annotation, we should treat this as the standard practice, so that we don't need to lots of separated MetadataDaoFactory in different modules. {code} @Inject public JDBCSecurityMetadataDAO(Config config){ this.config = config; } {code} > MetadataDaoFactory should use config from eagle server > -- > > Key: EAGLE-431 > URL: https://issues.apache.org/jira/browse/EAGLE-431 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Hao Chen > Fix For: v0.5.0 > > > org.apache.eagle.alert.metadata.impl.MetadataDaoFactory uses > ConfigFactory.load() to get configuration. We should use config object from > eagle server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (EAGLE-431) MetadataDaoFactory should use config from eagle server
[ https://issues.apache.org/jira/browse/EAGLE-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen reassigned EAGLE-431: -- Assignee: Hao Chen (was: Edward Zhang) > MetadataDaoFactory should use config from eagle server > -- > > Key: EAGLE-431 > URL: https://issues.apache.org/jira/browse/EAGLE-431 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: Hao Chen > Fix For: v0.5.0 > > > org.apache.eagle.alert.metadata.impl.MetadataDaoFactory uses > ConfigFactory.load() to get configuration. We should use config object from > eagle server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #324: EAGLE-440: Alert mongodb storage refine
GitHub user mizeng opened a pull request: https://github.com/apache/incubator-eagle/pull/324 EAGLE-440: Alert mongodb storage refine You can merge this pull request into a Git repository by running: $ git pull https://github.com/mizeng/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/324.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #324 commit 1fe32803f5c8c93ef9c7f29fe68b5fdc194d5368 Author: mizengDate: 2016-08-10T08:04:16Z EAGLE-440: Alert mongodb storage refine --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-440) Alert mongodb storage refine
[ https://issues.apache.org/jira/browse/EAGLE-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414888#comment-15414888 ] ASF GitHub Bot commented on EAGLE-440: -- GitHub user mizeng opened a pull request: https://github.com/apache/incubator-eagle/pull/324 EAGLE-440: Alert mongodb storage refine You can merge this pull request into a Git repository by running: $ git pull https://github.com/mizeng/incubator-eagle develop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/324.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #324 commit 1fe32803f5c8c93ef9c7f29fe68b5fdc194d5368 Author: mizengDate: 2016-08-10T08:04:16Z EAGLE-440: Alert mongodb storage refine > Alert mongodb storage refine > > > Key: EAGLE-440 > URL: https://issues.apache.org/jira/browse/EAGLE-440 > Project: Eagle > Issue Type: Improvement >Reporter: ZengMing >Assignee: ZengMing > > Currently, some Alert Mongo Collection is too large. Consider future > increase, need to split large collection to some small collections. > To back compatible, need to keep the interface as the same as before. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (EAGLE-442) Support to extend metastore DAO modules in SPI
[ https://issues.apache.org/jira/browse/EAGLE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414886#comment-15414886 ] ASF GitHub Bot commented on EAGLE-442: -- GitHub user haoch opened a pull request: https://github.com/apache/incubator-eagle/pull/323 EAGLE-442 Allow ApplicationProvider to extend extensible guice modules ## Interfaces ``` ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry { void register(ModuleScope,Module) } ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore ``` ## Use Case * ExampleResource * ExampleService You can merge this pull request into a Git repository by running: $ git pull https://github.com/haoch/incubator-eagle EAGLE-442 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #323 commit 6b5b8c83caa421b953299a1a4d4a256fb1ef2ed3 Author: Hao ChenDate: 2016-08-10T08:06:17Z EAGLE-442 Allow ApplicationProvider to extend extensible guice modules ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry { void register(ModuleScope,Module) } ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore > Support to extend metastore DAO modules in SPI > -- > > Key: EAGLE-442 > URL: https://issues.apache.org/jira/browse/EAGLE-442 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Hao Chen >Assignee: Hao Chen > Labels: app-framework, extensibility > Fix For: v0.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #323: EAGLE-442 Allow ApplicationProvider to ex...
GitHub user haoch opened a pull request: https://github.com/apache/incubator-eagle/pull/323 EAGLE-442 Allow ApplicationProvider to extend extensible guice modules ## Interfaces ``` ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry { void register(ModuleScope,Module) } ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore ``` ## Use Case * ExampleResource * ExampleService You can merge this pull request into a Git repository by running: $ git pull https://github.com/haoch/incubator-eagle EAGLE-442 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #323 commit 6b5b8c83caa421b953299a1a4d4a256fb1ef2ed3 Author: Hao ChenDate: 2016-08-10T08:06:17Z EAGLE-442 Allow ApplicationProvider to extend extensible guice modules ApplicationProvider{ void register(ModuleRegistry) } ModuleRegistry { void register(ModuleScope,Module) } ModuleScope |- GlobalScope |- MetadataStore |- MySQLMetadataStore |- MongodbMetadataStore |- MemoryMetadataStore --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414841#comment-15414841 ] ASF GitHub Bot commented on EAGLE-435: -- GitHub user garrettlish opened a pull request: https://github.com/apache/incubator-eagle/pull/322 [EAGLE-435] make coordiantor schedule operation exclusive in distributed deployment You can merge this pull request into a Git repository by running: $ git pull https://github.com/garrettlish/incubator-eagle eagle435 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #322 commit 214bf13df09242c7035ef080cc58c433defbe7ce Author: Xiancheng LiDate: 2016-08-05T04:43:27Z [EAGLE-435] make coordiantor schedule operation exclusive in distributed deployment > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Fix For: v0.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #322: [EAGLE-435] make coordiantor schedule ope...
GitHub user garrettlish opened a pull request: https://github.com/apache/incubator-eagle/pull/322 [EAGLE-435] make coordiantor schedule operation exclusive in distributed deployment You can merge this pull request into a Git repository by running: $ git pull https://github.com/garrettlish/incubator-eagle eagle435 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-eagle/pull/322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #322 commit 214bf13df09242c7035ef080cc58c433defbe7ce Author: Xiancheng LiDate: 2016-08-05T04:43:27Z [EAGLE-435] make coordiantor schedule operation exclusive in distributed deployment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414839#comment-15414839 ] ASF GitHub Bot commented on EAGLE-435: -- Github user garrettlish closed the pull request at: https://github.com/apache/incubator-eagle/pull/317 > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Fix For: v0.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-eagle pull request #317: [EAGLE-435] make coordiantor schedule ope...
Github user garrettlish closed the pull request at: https://github.com/apache/incubator-eagle/pull/317 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---