[GitHub] nifi pull request #580: 1.0 documentation updates - overview section

[Haimo-Liu]

GitHub user Haimo-Liu opened a pull request:

https://github.com/apache/nifi/pull/580

1.0 documentation updates - overview section

updated overview section that covers zero-master clustering, multi-tenancy, 
plus some other features, and adjusted formatting

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Haimo-Liu/nifi 1.0-docs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/580.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #580


commit e5f05ab1801b0072da1fe365caffc3180eea81a9
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-23T21:08:25Z

update overview

clustering paragraph

commit d00557fceca1265697bb13db1c91da539916e079
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T03:10:30Z

Update overview.adoc

commit c256ddcdff2d055eb68d4741ca02e8dce7d0b073
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T15:19:23Z

Update overview.adoc

commit d58c8ebb1fda40f6e50f20ddaaca55cb883b9749
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T15:55:39Z

Update overview.adoc

commit c7f7a74ea47848cdf5f5b66701c03aa58176a282
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T19:32:14Z

Update overview.adoc

commit ff1e8a3e7114233c5a655034fcd813f26f1307c1
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T19:58:54Z

Update overview.adoc

commit c01d96312e942cc933bed2e56112095ff8bd3157
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T20:10:49Z

Add files via upload

commit bce6c7932dfea529528e59c727e7326483e2cf82
Author: Haimo Liu <haimoli...@gmail.com>
Date:   2016-06-24T20:12:17Z

Update overview.adoc




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Jira privileges request

[Haimo Liu]

Hi Bryan,

I would be more than glad to become a contributor too, can you help grant
me the same JIRA privilege? Username: h_liu

Thanks!
Haimo

On Tue, Jul 12, 2016 at 1:12 PM, Andrew Lim 
wrote:

> Thanks Bryan!  I think I’m all set now.
>
> -Drew
>
> > On Jul 12, 2016, at 1:09 PM, Bryan Bende  wrote:
> >
> > Drew,
> >
> > I added you the contributors role in JIRA. Let us know if you still can't
> > assign tickets.
> >
> > -Bryan
> >
> > On Tue, Jul 12, 2016 at 1:06 PM, Andrew Lim 
> > wrote:
> >
> >> There are some tickets that I would like to assist on and assign to
> >> myself.  Could I be given the privileges in Jira to do so?
> >>
> >> My username is:   andrewmlim
> >>
> >> Thanks!
> >>
> >> -Drew
>
>

Re: BUG!!!!

[Haimo Liu]

您好，您指的不一致具体是哪些API呢？能否给几个例子帮助我们分析？

另外现在的Apache document里面的API是针对nifi 1.0的。不知道您运行的local nifi是不是也是相同的version?

On Mon, Sep 5, 2016 at 6:06 AM, 妙妙╭筱Mary↗ <565109...@qq.com> wrote:

> 您好：
> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html 和
> localhost:8080/nifi/ help中的Rest-api有些地方不一致，希望您能及时更新。

Re: [ANNOUNCE] New Apache NiFi Committer Rob Moran

[Haimo Liu]

Congrats Rob

On Mon, Sep 19, 2016 at 5:51 PM, Matt Burgess  wrote:

> Congratulations Rob!! Well deserved!!
>
> -Matt
>
> On Mon, Sep 19, 2016 at 3:08 PM, Tony Kurc  wrote:
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that Rob
> > Moran has accepted the PMC's invitation to become a committer on the
> Apache
> > NiFi project. We greatly appreciate all of Rob's hard work and generous
> > contributions to the project. We look forward to his continued
> involvement
> > in the project.
> >
> > Among other things, Rob has been responsible for a lot of the look and
> feel
> > of the NiFi user interface, to include the awesome logos!
> >
> > Welcome and congratulations!
> >
> > Tony
>

Re: MINIFI-350 minifi-cpp end-to-end integration testing framework

[Haimo Liu]

great idea Andy! I can see this being extremely valuable even outside of
the MINIFI cpp context. Specifically, to migrate my dataflow from one
environment to another (DEV to QA to PROD), an integration testing
framework could be very helpful for flow validation purposes.

in addition to testing your MINIFI agents and network connectivities, have
you taken into consideration the integration testing of a potentially very
complex dataflow itself? Say I am collecting data from 50 data sources, and
ingesting to 20 different targets, may I leverage your testing framework to
spin up necessary containers (HDFS, Hbase, Oracle, etc. just different end
points) and run a docker compose script to validate my flow during
migration? Would be very nice to see your framework to be designed
extensible in a way to cover flow specific testing as well. Maybe you
already have it all sorted out :)

Thanks,
Haimo

On Thu, Jul 13, 2017 at 1:50 PM, Andy Christianson <
achristian...@hortonworks.com> wrote:

> Thanks for the feedback. I will put together a proof of concept which we
> can further evaluate/refine/merge upstream.
>
> -Andy
>
> On 7/13/17, 11:30 AM, "Kevin Doran"  wrote:
>
> Great idea, Andy! Additional types of automated tests would help the
> minifi-cpp project significantly, and I think your proposal is an
> appropriate way to add integration tests for the minifi agent. This sounds
> like a great way to verify expected behavior of processors and the system
> of components in flow combinations.
>
> I like the idea of declarative tests that are interpreted / run by a
> harness or framework as a way to allow others can contribute test cases.
>
> I've never used the Bats framework before, but it seems like a
> reasonable option for what you describe. It might require writing a fair
> amount of bash code under-the-hood to get the functionality you want
> (helper functions and such), but it looks like it would keep the test cases
> themselves and the output clean and light. Perhaps others can offer
> suggestions here.
>
> One comment, which you've probably already considered, is that we
> should keep the dependencies (if any) that get added for integration tests
> that leverage the docker target optional so they are not required for folks
> that just want to build libminifi or the agent. It would be more of a
> developer/contributor option but users could skip these tests.
>
> /docker/test/integration seems like a reasonable place to add test
> cases. Others would probably know better. I think the README.md would be a
> reasonable place to document how to run the tests with a reference to
> another document that describes how to add / contribute new test cases. I'm
> not sure where the best location for the documentation should live.
>
> Thanks,
> Kevin
>
> On 7/13/17, 10:34, "Andy Christianson" 
> wrote:
>
> Yes, I envision having a directory of declarative test cases. Each
> would include a flow yaml, one or more input files, and expected outputs.
>
> I’d like to document the convention before writing the
> implementation because if the conventions are solid, we can change out the
> actual test driver implementation later on if needed.
>
> Would it be best to document this in a section within /README.md,
> or should I add a new file such as /docs/Testing.md, or /TESTING.md?
>
> As for where the test cases would be added, I was thinking maybe
> /docker/test/integration, keeping consistent with the existing convention
> (i.e. /libminifi/test/integration).
>
> -Andy
>
> On 7/13/17, 10:14 AM, "Marc"  wrote:
>
> Hi Andy,
>I think this is a great idea to test integrating MiNiFi
> among multiple
> system components. Do you have a feel for how you will allow
> others to
> create test cases? Will you attempt to minimize the footprint
> of
> contributed tests by creating a bats based framework? I ask
> because it
> would be cool if contributors could supply a flow ( input )
> and expected
> output and we automatically run the necessary
> containers/components. Is
> this along the lines of your vision?
>
>   Thanks,
>Marc
>
> On Wed, Jul 12, 2017 at 12:26 PM, Andy Christianson <
> achristian...@hortonworks.com> wrote:
>
> > Hi All,
> >
> > I am looking at MINIFI-350 and would like to implement some
> end-to-end
> > integration tests for minifi cpp. Essentially, the tests
> would:
> >
> >
> >   1.  Stand up a new minifi cpp docker container
> >   2.  Send test data to HTTP input ports on the container
> >   3.  Run data through a minifi flow
> >   4.  Receive output data to a HTTP endpoint
> >   5.  

Re: [VOTE] Release Apache NiFi 1.2.0 (RC2)

[Haimo Liu]

+1 non-binding, release this as Apache NiFi 1.2.0

Tested full clean install on OSX

Tested RecordReader/Writer flow
Tested CDC flow (MySql DB1 as source, MySql DB2 as target)



On May 6, 2017 11:40 AM, "Joe Witt"  wrote:

+1 (binding).


Full clean build w/contrib check on OSX, Linux 3.10
Builds clean on Windows10 but there are test failures due to os
compatibility issues with the test.

Tested the application on Windows, Linux and OSX.  All looks solid.
License/Notice looks good.

Provided a detailed template flow and instructions to use the new
record reader/writer capability with internal schema registry to
acquire, filter, and convert NiFi's own provenance stream into JSON,
Avro, XML and CSV at once.
https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

This is a really impressive release with tremendous community
engagement!  Thanks all

Joe

On Fri, May 5, 2017 at 10:07 PM, Bryan Bende  wrote:
> Hello,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi nifi-1.2.0 (RC2).
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1104
>
> The Git tag is nifi-1.2.0-RC2
> The Git commit ID is 3a605af8e0ac024fb0ba67262d49dab2727b2576
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
3a605af8e0ac024fb0ba67262d49dab2727b2576
>
> Checksums of nifi-1.2.0-source-release.zip:
> MD5: 90e298a9e23a9dab65358daddd8b5990
> SHA1: e138941f576bdb1dff17df6674c19ffae3ef6719
> SHA256: c18398800c435dabff9f45ec55a450ca78c3c1aec222aa295c0e2057912feeb6
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/bbende.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 381 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
projectId=12316020=12338432
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
Release+Notes#ReleaseNotes-Version1.2.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  The please vote:
>
> [ ] +1 Release this package as nifi-1.2.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because because...