I think I'm going to give the nar-maven-plugin a try to compile the C code
and then try to build a nar with the nifi-nar-maven-plugin. I hope I can
find some other nar plugins to complete my maven plugin bingo board.
As for reasons for offheap storage, joe alluded to some of the reasons in
the ticket.
Some reasons:
1) manage dynamically allocated memory without gc pressure (why in C)
2) ability to tune data structures for best load and lookup times (the lack
of threadsafety and collision detection on insertion are examples)
3) I'm attempting to write code using C's standard library to avoid
dependency and build challenges (in conjunction with 2, which helps me
rationalize not use an existing library )
4) Have a library which is "optimized" for nifi operations ("tried and
true" factor)
5) Writing in C makes me happy.
Frankly, offheap aside, my question was more about native code. I think
having a robust way of including libs that aren't jvm friendly is probably
a good "feature" to have.
On Thu, Mar 26, 2015 at 8:55 AM, Benson Margulies <[email protected]>
wrote:
> Time for fun and confusion. The nar-maven-plugin (the other one) does
> a working job in many cases of just folding the C++ build into the
> Maven build.
>
>
> On Thu, Mar 26, 2015 at 8:50 AM, Mark Payne <[email protected]> wrote:
> > Tony,
> >
> >
> > Off the top of my head I’d vote just put it in nifi-commons. I wouldn’t
> worry about the fact that it’s C rather than Java - it’s all handled by
> Maven. But I’m not sure what exactly is involved, so if it doesn’t make
> sense then by all means do whatever you think is best in that respect.
> >
> >
> > As far as integrating it into the application: I wouldn’t integrate it
> at all. I would just make it a library that a NAR can bring in. NAR’s are
> allowed to have a “native” directory where native libraries live, so I’d
> just leave it up to the NAR to bring it in.
> >
> >
> > -Mark
> >
> >
> >
> >
> >
> >
> > From: Tony Kurc
> > Sent: Wednesday, March 25, 2015 11:53 PM
> > To: [email protected]
> >
> >
> >
> >
> >
> > I've gotten some work done on NIFI-455. I'm building a shared library in
> C,
> > and calling functions in that library using JNI. I've been working in a
> > separate repo (https://github.com/trkurc/offheap) for now as I'm
> struggling
> > a bit with where native code should live in the source tree and how it
> > should be built and distributed. I'm hoping to get some suggestions..
> I've
> > seen quite a bit of variety in the way other java projects include native
> > code such as (mixed into maven-friendly source tree, maven build with ant
> > run plugin with cmake and make) [1] (similar, but just with ant-run with
> > cmake) [2] (separate part of source tree with pom that invokes make via
> > executable plugin) [3].
> >
> > Building and putting in the source tree is one issue. The next couple
> > integration steps are getting the shared library somewhere sensible and
> > setting up the environment to load, (and doing sensible things if the
> > shared library can't be found).
> >
> > If people have thoughts, or idioms that other projects use that we may be
> > able to borrow, I'd like to know!
> >
> > Tony
> >
> > [1]
> >
> https://github.com/apache/hadoop/tree/3c9181722b05a9192f5440ea8f3f77231f84eac6/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src
> > [2]
> >
> https://github.com/apache/hbase/tree/543c64d2e9d2fc608ef65cafd485a910d4b8d480/hbase-server/src/main/native/src
> > [3]
> >
> https://github.com/apache/accumulo/tree/be4aade67b0f0711e25de9ac01552471b48a75af/server/native
> >
> > *parens included to improve groupin
>
