This is a great question. Our intension is for iota to be a SMACK Stack Not
Platform. This means it uses
1) Apache Spark
2) Apache Mesos
3) Akka
4) Apache Cassandra
5) Apache Kafka
The Fey engine is a component of the iota platform but not the entire platform.
The platform is a vertical stack that has a Mesos Cluster as it’s highest
layer. This could be in the cloud (AWS, Azure, …). The next level of the
platform might be at the device level say a RaspberryPi with sensors or an
aggregation server. The edge devices can process data locally on the device
using Fey components or/and feed them up the stack using Fey components. Large
amounts of data will arrive at the Mesos cluster. This data can be proceeded by
Fey and Fey components running as an Akka cluster on Mesos. It can also be
processed using a Spark cluster also running on the Mesos layer. Cassandra is
used to persist data at the cluster level. Kafka is the message bus for major
software component to use when communicating with each other. The SMACK stack
becomes the data pipeline and processing engine for IoT data streaming from
edge based sensors.
The first component of iota that we are making available is the Fey engine and
a selection of Fey components. For example a Fey component for Kafka that
includes Kafka producer and consumer capabilities. The iota Fey engine can run
at all levels of the iota stack. This can include a RaspberryPi on a JVM in
standalone Akka mode or on the Mesos Cluster where it runs as an Akka cluster.
I hope this sheds light on the big picture. We can discuss the role of Docker
and Spark later so let’s not worry about them right now.
This is a grand vision so where do we start? The simplest way to start is with
the Fey engine and Fey performers.
We are first releasing a version of the Fey engine and a selection of
performers. Interested parties can download and experiment with these. We hope
they will be inspired to build their own software components that they can use
and share with others. The best components can be included in the Fey component
release. A rich set of these components (called performers) will enable
powerful applications to be assembled by others providing data pipeline and
processing functionality needed in the iota platform. Here are a few examples
of performers that could be and have been written
1) A Restful performer that implements get, put and post to reach out to get
data and control “Things” on the internet. This could run on a RaspberryPi and
data could be passed on to other performers.
2) A component that ingests data from other performers and processes the data
based on some mathematical or statistical decisions and selectively outputs
that data to other performers.
3) A component that interacts get/put to an Apache Ignite in-memory system.
Data received from other performers can be put the data store or data read from
the store can be sent to other performers.
4) A component that interacts with Apache ActiveMQ similar to 3)
5) A component that receives a high level image from another component, makes
it low resolution image version and send it on to other performers.
6) A component that can read values from I2C sensors and pass the value read
to other components.
These are some examples of component that have been and can be written as Fey
Performers. They form the core of a communication and processing system that
can be used to define a component based application. Such applications play an
important role in the iota platform.
How can someone contribute to the code base?
1) Select you preferred development workflow. The following are all free
- vim and sbt (simple build tool) on the command line
- IntelliJ Community edition (sbt)
- Eclipse (sbt)
my personal preference is IntelliJ Community edition - Eclipse is also a valid
option if you prefer Eclipse.
Follow the instructions that are being placed in the iota wiki on how to
download and run the fey engine and fey performers. Think of development for
Fey in the following terms
1) You have a palette of performers with different functionalities
2) Your applications will be limited by the set of performers you have
available to you. Say you have three performers P1 that reads numbers form a
sensor at a certain frequency say 5 seconds, you have P2 that waits for values
coming from other performers and if those values are between 0 and 100 they are
set to other performers, and you have P3 that receives data from other
performers and places a message in the log file.
3) To create an application you would create the following orchestration P1 ->
P2 -> P3 (it is a json object that defines each of the components used how
they interact with one another P1->P2->P3 and any parameters need by each
performer e.g. the 5 seconds frequency for this instance of performer P1).
4) You may find yourself needing to write an application for which you do not
have a complete set of components. Say it’s the example in 3) above but you
also want to send the message on a Kafka topic? You would then have to either
find a pre-existing Kafka performer that you get from some else or you would
write your own.
So get familiar with Fey and Fey components on your own local dev machine and
write a simple Fey Application (orchestration) if you don’t have the needed
functionality write one or more performs to give you the functionality you
need.
This would be a good start. For run put Fey on a RaspberryPI or some other
Linux based ARM device (Intel Edition?) and run Fey on the device.
I hope my long winded explanation is of some help. Feel free to ask other
questions if I have not explained things well or not answered them at all.
-Tony
> On Feb 10, 2017, at 10:13 PM, Gunnar Tapper <[email protected]> wrote:
>
> Hi,
>
> I'm trying to match the code with what's being said on
> http://iota.incubator.apache.org/features.html
> <http://iota.incubator.apache.org/features.html>
>
> I'm not clear where I find; for example, docker and spark parts?
>
> What's your advise for someone who wants to contribute? It seems like a good
> idea to read up on akka but then what?
>
> I'm setting up my project to use eclipse. Is that the IDE of choice or is
> there something more appropriate?
>
>
> --
> Thanks,
>
> Gunnar
> If you think you can you can, if you think you can't you're right.
