Hi Juan!

I've just created a JIRA with support for HDFS. Great idea some kind
of examples with spark within vagrant, for testing Myriad.

https://issues.apache.org/jira/browse/MYRIAD-263
--
Javi Roman

Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info


On Thu, Apr 12, 2018 at 10:30 AM, Juan P <jpgg...@gmail.com> wrote:
> Congratulations for the work Javi Roman !!
> It seems to me a very interesting tool to accelerate and test the work of
> the contributors. Personally, I would leave HDFS installed as a service to
> make management and maintenance easier. For demonstrations and tests, I
> think it would be interesting to put a spark-shell and update the README
> with some map-reduce example. What do you think?
>
> On 9 April 2018 at 19:34, Javi Roman <jroman.espi...@gmail.com> wrote:
>
>> Hi!
>>
>> I've just created a PR [1] with the fist version of DC/OS (tested
>> v1.11.0) for Myriad development/deployment at DC/OS.
>>
>> The cluster created by default is something like that:
>>
>> $ vagrant status
>> Current machine states:
>>
>> bt                         running (libvirt)
>> m1                       running (libvirt)
>> a1                        running (libvirt)
>> a2                        running (libvirt)
>> a3                        running (libvirt)
>> a4                        running (libvirt)
>> p1                        running (libvirt)
>>
>> The bootstrap machine (with DC/OS CLI installed). One master node, 4
>> private agents, and one public.
>>
>> I haven't tested Myraid yet, because we have different options for it:
>>
>> - Deploy HDFS as a system service.
>> - Deploy HDFS using Mesosphere Universe (we have to tune this one, the
>> requirements don't fit in this small Vagrant cluster).
>> - Deploy Myriad by means of Marathon and Docker.
>>
>> We have to test different options for this deployment. One interesting
>> idea would be create a Myriad Universe with packages for HDFS tuned
>> for this Vagrant, with upstream Apache Hadoop, and packages for
>> Myriad. This could be a good point for demos.
>>
>>
>> [1] https://github.com/apache/incubator-myriad/pull/108
>> --
>> Javi Roman
>>
>> Twitter: @javiromanrh
>> GitHub: github.com/javiroman
>> Linkedin: es.linkedin.com/in/javiroman
>> Big Data Blog: dataintensive.info
>>

Reply via email to