Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-23 Thread Denis Magda
Thanks, Pavel!

Anton, Alex, we're in agreement to merge the changes in Ignite 2.9. Please
go ahead removing the Hadoop/IGFS modules from both the master and 2.9
branches.

-
Denis


On Thu, Jul 23, 2020 at 8:14 AM Pavel Tupitsyn  wrote:

> Denis,
>
> Thanks for the explanation, I don't have any objections anymore.
>
> On Wed, Jul 22, 2020 at 7:46 PM Denis Magda  wrote:
>
> > Hi,
> >
> > Send an email to this address: dev-unsubscr...@ignite.apache.org
> >
> > -
> > Denis
> >
> >
> > On Wed, Jul 22, 2020 at 9:28 AM C Ravikiran 
> > wrote:
> >
> > > Hi,
> > >
> > > Could you please help me to unsubscribe all these ignite mails.
> > >
> > > I have unsubscribed it, but still I am getting lot of emails
> > > Could you please help me..
> > >
> > > On Wed, 22 Jul, 2020, 9:45 pm Anton Kalashnikov, 
> > > wrote:
> > >
> > > > Sorry, I was mistaken, we can not leave these methods because at
> least
> > > > FileSystemConfiguration and HadoopConfiguration require corresponded
> > > > classes that were deleted. So I think we should just remove it right
> > now.
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > >
> > > >
> > > > 22.07.2020, 18:56, "Anton Kalashnikov" :
> > > > > Hi,
> > > > >
> > > > > All of these methods are from IgniteConfiguration:
> > > > > Hadoop configuration:
> > > > > - HadoopConfiguration getHadoopConfiguration()
> > > > > - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration
> > > > hadoopCfg)
> > > > >
> > > > > IGFS (Ignite In-Memory File System) configurations:
> > > > > - FileSystemConfiguration[] getFileSystemConfiguration
> > > > > - IgniteConfiguration
> > > > setFileSystemConfiguration(FileSystemConfiguration... igfsCfg)
> > > > >
> > > > > thread pool size that will be used to process outgoing IGFS
> messages:
> > > > > - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> > > > > - int getIgfsThreadPoolSize()
> > > > >
> > > > > Of course, I can leave these methods intact but they will be doing
> > > > nothing so API formally wouldn't be changed but, in fact, features
> > would
> > > be
> > > > removed. Does it make sense? I don't think so and in my opinion,
> > perhaps
> > > it
> > > > is ok to remove these methods right now if we are ready to remove
> these
> > > > features right now. (but again, if there are some concerns about it,
> I
> > > can
> > > > easily to leave these methods with empty implementation)
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Anton Kalashnikov
> > > > >
> > > > > 22.07.2020, 17:47, "Denis Magda" :
> > > > >>  Hi Alex,
> > > > >>
> > > > >>  It's been a year since we voted to discontinue this integration
> [1]
> > > > and it
> > > > >>  wasn't removed from the source code earlier only because of the
> > > > internal
> > > > >>  dependencies with the ML component. Now all the dependencies are
> > gone
> > > > and
> > > > >>  Ignite 2.9 is the right version to finish the discontinuation
> > > process.
> > > > It
> > > > >>  would make sense to wait for Ignite 3.0 only there are some
> > breaking
> > > > >>  changes in the APIs that will stay in Ignite.
> > > > >>
> > > > >>   @Anton Kalashnikov , you mentioned
> > that
> > > > you
> > > > >>  removed some methods from the configuration. Could you please
> list
> > > them
> > > > >>  here? Are they Hadoop-specific or generic?
> > > > >>
> > > > >>  [1]
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
> > > > >>
> > > > >>  -
> > > > >>  Denis
> > > > >>
> > > > >>  On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov <
> > > plehanov.a...@gmail.com
> > > > >
> > > > >>  wrote:
> > > > >>
> > > > >>>   Guys,
> > > > >>>
> > > > >>>   Any updates here? Looks like we still don't have a consensus
> > about
> > > > release
> > > > >>>   version for this patch (already mention it in the release
> > thread).
> > > > >>>   Currently, the ticket is still targeted to 2.9.
> > > > >>>
> > > > >>>   ср, 15 июл. 2020 г. в 00:40, Denis Magda :
> > > > >>>
> > > > >>>   > I don't think it's required to wait until Ignite 3.0 to make
> > this
> > > > happen.
> > > > >>>   > If I'm not mistaken, we stopped releasing Hadoop binaries and
> > > > sources a
> > > > >>>   > long time ago (at least you can't longer find them on the
> > > downloads
> > > > >>>   page).
> > > > >>>   > Also, we removed all the mentioning from the documentation
> and
> > > > website.
> > > > >>>   > Nobody complained or requested for a maintenance release
> since
> > > > that time.
> > > > >>>   > Thus, I would remove the integration in 2.9. If anybody shows
> > up
> > > > later
> > > > >>>   then
> > > > >>>   > they can use the sources in the 2.8 branch and do whatever
> they
> > > > want.
> > > > >>>   >
> > > > >>>   > -
> > > > >>>   > Denis
> > > > >>>   >
> > > > >>>   >
> > > > >>>   > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn <
> > > > ptupit...@apache.org>
> > > > >>>   > 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-23 Thread Pavel Tupitsyn
Denis,

Thanks for the explanation, I don't have any objections anymore.

On Wed, Jul 22, 2020 at 7:46 PM Denis Magda  wrote:

> Hi,
>
> Send an email to this address: dev-unsubscr...@ignite.apache.org
>
> -
> Denis
>
>
> On Wed, Jul 22, 2020 at 9:28 AM C Ravikiran 
> wrote:
>
> > Hi,
> >
> > Could you please help me to unsubscribe all these ignite mails.
> >
> > I have unsubscribed it, but still I am getting lot of emails
> > Could you please help me..
> >
> > On Wed, 22 Jul, 2020, 9:45 pm Anton Kalashnikov, 
> > wrote:
> >
> > > Sorry, I was mistaken, we can not leave these methods because at least
> > > FileSystemConfiguration and HadoopConfiguration require corresponded
> > > classes that were deleted. So I think we should just remove it right
> now.
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > >
> > >
> > > 22.07.2020, 18:56, "Anton Kalashnikov" :
> > > > Hi,
> > > >
> > > > All of these methods are from IgniteConfiguration:
> > > > Hadoop configuration:
> > > > - HadoopConfiguration getHadoopConfiguration()
> > > > - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration
> > > hadoopCfg)
> > > >
> > > > IGFS (Ignite In-Memory File System) configurations:
> > > > - FileSystemConfiguration[] getFileSystemConfiguration
> > > > - IgniteConfiguration
> > > setFileSystemConfiguration(FileSystemConfiguration... igfsCfg)
> > > >
> > > > thread pool size that will be used to process outgoing IGFS messages:
> > > > - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> > > > - int getIgfsThreadPoolSize()
> > > >
> > > > Of course, I can leave these methods intact but they will be doing
> > > nothing so API formally wouldn't be changed but, in fact, features
> would
> > be
> > > removed. Does it make sense? I don't think so and in my opinion,
> perhaps
> > it
> > > is ok to remove these methods right now if we are ready to remove these
> > > features right now. (but again, if there are some concerns about it, I
> > can
> > > easily to leave these methods with empty implementation)
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > > 22.07.2020, 17:47, "Denis Magda" :
> > > >>  Hi Alex,
> > > >>
> > > >>  It's been a year since we voted to discontinue this integration [1]
> > > and it
> > > >>  wasn't removed from the source code earlier only because of the
> > > internal
> > > >>  dependencies with the ML component. Now all the dependencies are
> gone
> > > and
> > > >>  Ignite 2.9 is the right version to finish the discontinuation
> > process.
> > > It
> > > >>  would make sense to wait for Ignite 3.0 only there are some
> breaking
> > > >>  changes in the APIs that will stay in Ignite.
> > > >>
> > > >>   @Anton Kalashnikov , you mentioned
> that
> > > you
> > > >>  removed some methods from the configuration. Could you please list
> > them
> > > >>  here? Are they Hadoop-specific or generic?
> > > >>
> > > >>  [1]
> > > >>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
> > > >>
> > > >>  -
> > > >>  Denis
> > > >>
> > > >>  On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov <
> > plehanov.a...@gmail.com
> > > >
> > > >>  wrote:
> > > >>
> > > >>>   Guys,
> > > >>>
> > > >>>   Any updates here? Looks like we still don't have a consensus
> about
> > > release
> > > >>>   version for this patch (already mention it in the release
> thread).
> > > >>>   Currently, the ticket is still targeted to 2.9.
> > > >>>
> > > >>>   ср, 15 июл. 2020 г. в 00:40, Denis Magda :
> > > >>>
> > > >>>   > I don't think it's required to wait until Ignite 3.0 to make
> this
> > > happen.
> > > >>>   > If I'm not mistaken, we stopped releasing Hadoop binaries and
> > > sources a
> > > >>>   > long time ago (at least you can't longer find them on the
> > downloads
> > > >>>   page).
> > > >>>   > Also, we removed all the mentioning from the documentation and
> > > website.
> > > >>>   > Nobody complained or requested for a maintenance release since
> > > that time.
> > > >>>   > Thus, I would remove the integration in 2.9. If anybody shows
> up
> > > later
> > > >>>   then
> > > >>>   > they can use the sources in the 2.8 branch and do whatever they
> > > want.
> > > >>>   >
> > > >>>   > -
> > > >>>   > Denis
> > > >>>   >
> > > >>>   >
> > > >>>   > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn <
> > > ptupit...@apache.org>
> > > >>>   > wrote:
> > > >>>   >
> > > >>>   > > We are breaking backwards compatibility,
> > > >>>   > > so this can be only done for Ignite 3.0, am I right?
> > > >>>   > >
> > > >>>   > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov <
> > > kaa@yandex.ru>
> > > >>>   > > wrote:
> > > >>>   > >
> > > >>>   > > > Hi everyone,
> > > >>>   > > >
> > > >>>   > > > The task of removal IGFS and Hadoop accelerator is ready to
> > > review.(
> > > >>>   > > > https://issues.apache.org/jira/browse/IGNITE-11942)
> > > >>>   > > > I've already asked some guys 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Denis Magda
Hi,

Send an email to this address: dev-unsubscr...@ignite.apache.org

-
Denis


On Wed, Jul 22, 2020 at 9:28 AM C Ravikiran 
wrote:

> Hi,
>
> Could you please help me to unsubscribe all these ignite mails.
>
> I have unsubscribed it, but still I am getting lot of emails
> Could you please help me..
>
> On Wed, 22 Jul, 2020, 9:45 pm Anton Kalashnikov, 
> wrote:
>
> > Sorry, I was mistaken, we can not leave these methods because at least
> > FileSystemConfiguration and HadoopConfiguration require corresponded
> > classes that were deleted. So I think we should just remove it right now.
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> >
> > 22.07.2020, 18:56, "Anton Kalashnikov" :
> > > Hi,
> > >
> > > All of these methods are from IgniteConfiguration:
> > > Hadoop configuration:
> > > - HadoopConfiguration getHadoopConfiguration()
> > > - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration
> > hadoopCfg)
> > >
> > > IGFS (Ignite In-Memory File System) configurations:
> > > - FileSystemConfiguration[] getFileSystemConfiguration
> > > - IgniteConfiguration
> > setFileSystemConfiguration(FileSystemConfiguration... igfsCfg)
> > >
> > > thread pool size that will be used to process outgoing IGFS messages:
> > > - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> > > - int getIgfsThreadPoolSize()
> > >
> > > Of course, I can leave these methods intact but they will be doing
> > nothing so API formally wouldn't be changed but, in fact, features would
> be
> > removed. Does it make sense? I don't think so and in my opinion, perhaps
> it
> > is ok to remove these methods right now if we are ready to remove these
> > features right now. (but again, if there are some concerns about it, I
> can
> > easily to leave these methods with empty implementation)
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > > 22.07.2020, 17:47, "Denis Magda" :
> > >>  Hi Alex,
> > >>
> > >>  It's been a year since we voted to discontinue this integration [1]
> > and it
> > >>  wasn't removed from the source code earlier only because of the
> > internal
> > >>  dependencies with the ML component. Now all the dependencies are gone
> > and
> > >>  Ignite 2.9 is the right version to finish the discontinuation
> process.
> > It
> > >>  would make sense to wait for Ignite 3.0 only there are some breaking
> > >>  changes in the APIs that will stay in Ignite.
> > >>
> > >>   @Anton Kalashnikov , you mentioned that
> > you
> > >>  removed some methods from the configuration. Could you please list
> them
> > >>  here? Are they Hadoop-specific or generic?
> > >>
> > >>  [1]
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
> > >>
> > >>  -
> > >>  Denis
> > >>
> > >>  On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov <
> plehanov.a...@gmail.com
> > >
> > >>  wrote:
> > >>
> > >>>   Guys,
> > >>>
> > >>>   Any updates here? Looks like we still don't have a consensus about
> > release
> > >>>   version for this patch (already mention it in the release thread).
> > >>>   Currently, the ticket is still targeted to 2.9.
> > >>>
> > >>>   ср, 15 июл. 2020 г. в 00:40, Denis Magda :
> > >>>
> > >>>   > I don't think it's required to wait until Ignite 3.0 to make this
> > happen.
> > >>>   > If I'm not mistaken, we stopped releasing Hadoop binaries and
> > sources a
> > >>>   > long time ago (at least you can't longer find them on the
> downloads
> > >>>   page).
> > >>>   > Also, we removed all the mentioning from the documentation and
> > website.
> > >>>   > Nobody complained or requested for a maintenance release since
> > that time.
> > >>>   > Thus, I would remove the integration in 2.9. If anybody shows up
> > later
> > >>>   then
> > >>>   > they can use the sources in the 2.8 branch and do whatever they
> > want.
> > >>>   >
> > >>>   > -
> > >>>   > Denis
> > >>>   >
> > >>>   >
> > >>>   > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn <
> > ptupit...@apache.org>
> > >>>   > wrote:
> > >>>   >
> > >>>   > > We are breaking backwards compatibility,
> > >>>   > > so this can be only done for Ignite 3.0, am I right?
> > >>>   > >
> > >>>   > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov <
> > kaa@yandex.ru>
> > >>>   > > wrote:
> > >>>   > >
> > >>>   > > > Hi everyone,
> > >>>   > > >
> > >>>   > > > The task of removal IGFS and Hadoop accelerator is ready to
> > review.(
> > >>>   > > > https://issues.apache.org/jira/browse/IGNITE-11942)
> > >>>   > > > I've already asked some guys to take a look at it but if
> > somebody
> > >>>   > > familiar
> > >>>   > > > with this part of code, feel free to take a look at the
> changes
> > >>>   > > > too(especially scripts changes).
> > >>>   > > >
> > >>>   > > > I also think it is good to decide which release it should be
> > planned
> > >>>   > on.
> > >>>   > > > This task planned for 2.9 right now but I should notice that
> > first of
> > >>>   > all
> > >>>   

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Denis Magda
The removal of these methods will not impact anybody during migration to
2.9. Unless you use the Hadoop Accelerator or IGFS in your application. In
the latter scenario, well, the integration has already been discontinued
and you need to clean your application code before moving forward to 2.9.
Anyway, it's highly unlikely that we have any users of the integrations,
otherwise, they would show up throughout the year after learning that the
integration is no longer supported and discontinued.

Pavel, do you still prefer us merging the changes in Ignite 3.0 or agree to
do that in 2.9?

-
Denis


On Wed, Jul 22, 2020 at 8:56 AM Anton Kalashnikov  wrote:

> Hi,
>
> All of these methods are from IgniteConfiguration:
> Hadoop configuration:
> - HadoopConfiguration getHadoopConfiguration()
> - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration hadoopCfg)
>
> IGFS (Ignite In-Memory File System) configurations:
> - FileSystemConfiguration[] getFileSystemConfiguration
> - IgniteConfiguration
> setFileSystemConfiguration(FileSystemConfiguration... igfsCfg)
>
> thread pool size that will be used to process outgoing IGFS messages:
> - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> - int getIgfsThreadPoolSize()
>
>
> Of course, I can leave these methods intact but they will be doing nothing
> so API formally wouldn't be changed but, in fact, features would be
> removed. Does it make sense? I don't think so and in my opinion, perhaps it
> is ok to remove these methods right now if we are ready to remove these
> features right now. (but again, if there are some concerns about it, I can
> easily to leave these methods with empty implementation)
>
> --
> Best regards,
> Anton Kalashnikov
>
>
>
> 22.07.2020, 17:47, "Denis Magda" :
> > Hi Alex,
> >
> > It's been a year since we voted to discontinue this integration [1] and
> it
> > wasn't removed from the source code earlier only because of the internal
> > dependencies with the ML component. Now all the dependencies are gone and
> > Ignite 2.9 is the right version to finish the discontinuation process. It
> > would make sense to wait for Ignite 3.0 only there are some breaking
> > changes in the APIs that will stay in Ignite.
> >
> >  @Anton Kalashnikov , you mentioned that you
> > removed some methods from the configuration. Could you please list them
> > here? Are they Hadoop-specific or generic?
> >
> > [1]
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
> >
> > -
> > Denis
> >
> > On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov 
> > wrote:
> >
> >>  Guys,
> >>
> >>  Any updates here? Looks like we still don't have a consensus about
> release
> >>  version for this patch (already mention it in the release thread).
> >>  Currently, the ticket is still targeted to 2.9.
> >>
> >>  ср, 15 июл. 2020 г. в 00:40, Denis Magda :
> >>
> >>  > I don't think it's required to wait until Ignite 3.0 to make this
> happen.
> >>  > If I'm not mistaken, we stopped releasing Hadoop binaries and
> sources a
> >>  > long time ago (at least you can't longer find them on the downloads
> >>  page).
> >>  > Also, we removed all the mentioning from the documentation and
> website.
> >>  > Nobody complained or requested for a maintenance release since that
> time.
> >>  > Thus, I would remove the integration in 2.9. If anybody shows up
> later
> >>  then
> >>  > they can use the sources in the 2.8 branch and do whatever they want.
> >>  >
> >>  > -
> >>  > Denis
> >>  >
> >>  >
> >>  > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn 
> >>  > wrote:
> >>  >
> >>  > > We are breaking backwards compatibility,
> >>  > > so this can be only done for Ignite 3.0, am I right?
> >>  > >
> >>  > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov <
> kaa@yandex.ru>
> >>  > > wrote:
> >>  > >
> >>  > > > Hi everyone,
> >>  > > >
> >>  > > > The task of removal IGFS and Hadoop accelerator is ready to
> review.(
> >>  > > > https://issues.apache.org/jira/browse/IGNITE-11942)
> >>  > > > I've already asked some guys to take a look at it but if somebody
> >>  > > familiar
> >>  > > > with this part of code, feel free to take a look at the changes
> >>  > > > too(especially scripts changes).
> >>  > > >
> >>  > > > I also think it is good to decide which release it should be
> planned
> >>  > on.
> >>  > > > This task planned for 2.9 right now but I should notice that
> first of
> >>  > all
> >>  > > > there are a lot of changes and secondly there are some changes in
> >>  > public
> >>  > > > API(removed some methods from configuration). So maybe it makes
> sense
> >>  > to
> >>  > > > move this ticket to the next release. What do you think?
> >>  > > >
> >>  > > > --
> >>  > > > Best regards,
> >>  > > > Anton Kalashnikov
> >>  > > >
> >>  > > >
> >>  > > > 10.02.2020, 15:45, "Alexey Zinoviev" :
> >>  > > > > Thank you so you much! Will wait:)
> >>  > > > >
> >>  > > > > пн, 10 февр. 2020 г. в 15:13, Alexey 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread C Ravikiran
Hi,

Could you please help me to unsubscribe all these ignite mails.

I have unsubscribed it, but still I am getting lot of emails
Could you please help me..

On Wed, 22 Jul, 2020, 9:45 pm Anton Kalashnikov,  wrote:

> Sorry, I was mistaken, we can not leave these methods because at least
> FileSystemConfiguration and HadoopConfiguration require corresponded
> classes that were deleted. So I think we should just remove it right now.
>
> --
> Best regards,
> Anton Kalashnikov
>
>
>
> 22.07.2020, 18:56, "Anton Kalashnikov" :
> > Hi,
> >
> > All of these methods are from IgniteConfiguration:
> > Hadoop configuration:
> > - HadoopConfiguration getHadoopConfiguration()
> > - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration
> hadoopCfg)
> >
> > IGFS (Ignite In-Memory File System) configurations:
> > - FileSystemConfiguration[] getFileSystemConfiguration
> > - IgniteConfiguration
> setFileSystemConfiguration(FileSystemConfiguration... igfsCfg)
> >
> > thread pool size that will be used to process outgoing IGFS messages:
> > - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> > - int getIgfsThreadPoolSize()
> >
> > Of course, I can leave these methods intact but they will be doing
> nothing so API formally wouldn't be changed but, in fact, features would be
> removed. Does it make sense? I don't think so and in my opinion, perhaps it
> is ok to remove these methods right now if we are ready to remove these
> features right now. (but again, if there are some concerns about it, I can
> easily to leave these methods with empty implementation)
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> > 22.07.2020, 17:47, "Denis Magda" :
> >>  Hi Alex,
> >>
> >>  It's been a year since we voted to discontinue this integration [1]
> and it
> >>  wasn't removed from the source code earlier only because of the
> internal
> >>  dependencies with the ML component. Now all the dependencies are gone
> and
> >>  Ignite 2.9 is the right version to finish the discontinuation process.
> It
> >>  would make sense to wait for Ignite 3.0 only there are some breaking
> >>  changes in the APIs that will stay in Ignite.
> >>
> >>   @Anton Kalashnikov , you mentioned that
> you
> >>  removed some methods from the configuration. Could you please list them
> >>  here? Are they Hadoop-specific or generic?
> >>
> >>  [1]
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
> >>
> >>  -
> >>  Denis
> >>
> >>  On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov  >
> >>  wrote:
> >>
> >>>   Guys,
> >>>
> >>>   Any updates here? Looks like we still don't have a consensus about
> release
> >>>   version for this patch (already mention it in the release thread).
> >>>   Currently, the ticket is still targeted to 2.9.
> >>>
> >>>   ср, 15 июл. 2020 г. в 00:40, Denis Magda :
> >>>
> >>>   > I don't think it's required to wait until Ignite 3.0 to make this
> happen.
> >>>   > If I'm not mistaken, we stopped releasing Hadoop binaries and
> sources a
> >>>   > long time ago (at least you can't longer find them on the downloads
> >>>   page).
> >>>   > Also, we removed all the mentioning from the documentation and
> website.
> >>>   > Nobody complained or requested for a maintenance release since
> that time.
> >>>   > Thus, I would remove the integration in 2.9. If anybody shows up
> later
> >>>   then
> >>>   > they can use the sources in the 2.8 branch and do whatever they
> want.
> >>>   >
> >>>   > -
> >>>   > Denis
> >>>   >
> >>>   >
> >>>   > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn <
> ptupit...@apache.org>
> >>>   > wrote:
> >>>   >
> >>>   > > We are breaking backwards compatibility,
> >>>   > > so this can be only done for Ignite 3.0, am I right?
> >>>   > >
> >>>   > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov <
> kaa@yandex.ru>
> >>>   > > wrote:
> >>>   > >
> >>>   > > > Hi everyone,
> >>>   > > >
> >>>   > > > The task of removal IGFS and Hadoop accelerator is ready to
> review.(
> >>>   > > > https://issues.apache.org/jira/browse/IGNITE-11942)
> >>>   > > > I've already asked some guys to take a look at it but if
> somebody
> >>>   > > familiar
> >>>   > > > with this part of code, feel free to take a look at the changes
> >>>   > > > too(especially scripts changes).
> >>>   > > >
> >>>   > > > I also think it is good to decide which release it should be
> planned
> >>>   > on.
> >>>   > > > This task planned for 2.9 right now but I should notice that
> first of
> >>>   > all
> >>>   > > > there are a lot of changes and secondly there are some changes
> in
> >>>   > public
> >>>   > > > API(removed some methods from configuration). So maybe it
> makes sense
> >>>   > to
> >>>   > > > move this ticket to the next release. What do you think?
> >>>   > > >
> >>>   > > > --
> >>>   > > > Best regards,
> >>>   > > > Anton Kalashnikov
> >>>   > > >
> >>>   > > >
> >>>   > > > 10.02.2020, 15:45, "Alexey Zinoviev" :
> >>>   > > > > 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Anton Kalashnikov
Sorry, I was mistaken, we can not leave these methods because at least 
FileSystemConfiguration and HadoopConfiguration require corresponded classes 
that were deleted. So I think we should just remove it right now.

-- 
Best regards,
Anton Kalashnikov



22.07.2020, 18:56, "Anton Kalashnikov" :
> Hi,
>
> All of these methods are from IgniteConfiguration:
> Hadoop configuration:
> - HadoopConfiguration getHadoopConfiguration()
> - IgniteConfiguration setHadoopConfiguration(HadoopConfiguration hadoopCfg)
>
> IGFS (Ignite In-Memory File System) configurations:
> - FileSystemConfiguration[] getFileSystemConfiguration
> - IgniteConfiguration setFileSystemConfiguration(FileSystemConfiguration... 
> igfsCfg)
>
> thread pool size that will be used to process outgoing IGFS messages:
> - IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
> - int getIgfsThreadPoolSize()
>
> Of course, I can leave these methods intact but they will be doing nothing so 
> API formally wouldn't be changed but, in fact, features would be removed. 
> Does it make sense? I don't think so and in my opinion, perhaps it is ok to 
> remove these methods right now if we are ready to remove these features right 
> now. (but again, if there are some concerns about it, I can easily to leave 
> these methods with empty implementation)
>
> --
> Best regards,
> Anton Kalashnikov
>
> 22.07.2020, 17:47, "Denis Magda" :
>>  Hi Alex,
>>
>>  It's been a year since we voted to discontinue this integration [1] and it
>>  wasn't removed from the source code earlier only because of the internal
>>  dependencies with the ML component. Now all the dependencies are gone and
>>  Ignite 2.9 is the right version to finish the discontinuation process. It
>>  would make sense to wait for Ignite 3.0 only there are some breaking
>>  changes in the APIs that will stay in Ignite.
>>
>>   @Anton Kalashnikov , you mentioned that you
>>  removed some methods from the configuration. Could you please list them
>>  here? Are they Hadoop-specific or generic?
>>
>>  [1]
>>  
>> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
>>
>>  -
>>  Denis
>>
>>  On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov 
>>  wrote:
>>
>>>   Guys,
>>>
>>>   Any updates here? Looks like we still don't have a consensus about release
>>>   version for this patch (already mention it in the release thread).
>>>   Currently, the ticket is still targeted to 2.9.
>>>
>>>   ср, 15 июл. 2020 г. в 00:40, Denis Magda :
>>>
>>>   > I don't think it's required to wait until Ignite 3.0 to make this 
>>> happen.
>>>   > If I'm not mistaken, we stopped releasing Hadoop binaries and sources a
>>>   > long time ago (at least you can't longer find them on the downloads
>>>   page).
>>>   > Also, we removed all the mentioning from the documentation and website.
>>>   > Nobody complained or requested for a maintenance release since that 
>>> time.
>>>   > Thus, I would remove the integration in 2.9. If anybody shows up later
>>>   then
>>>   > they can use the sources in the 2.8 branch and do whatever they want.
>>>   >
>>>   > -
>>>   > Denis
>>>   >
>>>   >
>>>   > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn 
>>>   > wrote:
>>>   >
>>>   > > We are breaking backwards compatibility,
>>>   > > so this can be only done for Ignite 3.0, am I right?
>>>   > >
>>>   > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov 
>>>   > > wrote:
>>>   > >
>>>   > > > Hi everyone,
>>>   > > >
>>>   > > > The task of removal IGFS and Hadoop accelerator is ready to review.(
>>>   > > > https://issues.apache.org/jira/browse/IGNITE-11942)
>>>   > > > I've already asked some guys to take a look at it but if somebody
>>>   > > familiar
>>>   > > > with this part of code, feel free to take a look at the changes
>>>   > > > too(especially scripts changes).
>>>   > > >
>>>   > > > I also think it is good to decide which release it should be planned
>>>   > on.
>>>   > > > This task planned for 2.9 right now but I should notice that first 
>>> of
>>>   > all
>>>   > > > there are a lot of changes and secondly there are some changes in
>>>   > public
>>>   > > > API(removed some methods from configuration). So maybe it makes 
>>> sense
>>>   > to
>>>   > > > move this ticket to the next release. What do you think?
>>>   > > >
>>>   > > > --
>>>   > > > Best regards,
>>>   > > > Anton Kalashnikov
>>>   > > >
>>>   > > >
>>>   > > > 10.02.2020, 15:45, "Alexey Zinoviev" :
>>>   > > > > Thank you so you much! Will wait:)
>>>   > > > >
>>>   > > > > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
>>>   > > > alexey.goncha...@gmail.com>:
>>>   > > > >
>>>   > > > >> Got it, then no need to rush, let's wait for the TF-IGFS
>>>   > decoupling.
>>>   > > > >>
>>>   > > > >> пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <
>>>   > > zaleslaw@gmail.com
>>>   > > > >:
>>>   > > > >>
>>>   > > > >> > Tensorflow integration uses IGFS, if you have any idea how to
>>>   > 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Anton Kalashnikov
Hi,

All of these methods are from IgniteConfiguration:
Hadoop configuration:
- HadoopConfiguration getHadoopConfiguration()
- IgniteConfiguration setHadoopConfiguration(HadoopConfiguration hadoopCfg)

IGFS (Ignite In-Memory File System) configurations:
- FileSystemConfiguration[] getFileSystemConfiguration
- IgniteConfiguration setFileSystemConfiguration(FileSystemConfiguration... 
igfsCfg)

thread pool size that will be used to process outgoing IGFS messages:
- IgniteConfiguration setIgfsThreadPoolSize(int poolSize)
- int getIgfsThreadPoolSize()


Of course, I can leave these methods intact but they will be doing nothing so 
API formally wouldn't be changed but, in fact, features would be removed. Does 
it make sense? I don't think so and in my opinion, perhaps it is ok to remove 
these methods right now if we are ready to remove these features right now. 
(but again, if there are some concerns about it, I can easily to leave these 
methods with empty implementation)

-- 
Best regards,
Anton Kalashnikov



22.07.2020, 17:47, "Denis Magda" :
> Hi Alex,
>
> It's been a year since we voted to discontinue this integration [1] and it
> wasn't removed from the source code earlier only because of the internal
> dependencies with the ML component. Now all the dependencies are gone and
> Ignite 2.9 is the right version to finish the discontinuation process. It
> would make sense to wait for Ignite 3.0 only there are some breaking
> changes in the APIs that will stay in Ignite.
>
>  @Anton Kalashnikov , you mentioned that you
> removed some methods from the configuration. Could you please list them
> here? Are they Hadoop-specific or generic?
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html
>
> -
> Denis
>
> On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov 
> wrote:
>
>>  Guys,
>>
>>  Any updates here? Looks like we still don't have a consensus about release
>>  version for this patch (already mention it in the release thread).
>>  Currently, the ticket is still targeted to 2.9.
>>
>>  ср, 15 июл. 2020 г. в 00:40, Denis Magda :
>>
>>  > I don't think it's required to wait until Ignite 3.0 to make this happen.
>>  > If I'm not mistaken, we stopped releasing Hadoop binaries and sources a
>>  > long time ago (at least you can't longer find them on the downloads
>>  page).
>>  > Also, we removed all the mentioning from the documentation and website.
>>  > Nobody complained or requested for a maintenance release since that time.
>>  > Thus, I would remove the integration in 2.9. If anybody shows up later
>>  then
>>  > they can use the sources in the 2.8 branch and do whatever they want.
>>  >
>>  > -
>>  > Denis
>>  >
>>  >
>>  > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn 
>>  > wrote:
>>  >
>>  > > We are breaking backwards compatibility,
>>  > > so this can be only done for Ignite 3.0, am I right?
>>  > >
>>  > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov 
>>  > > wrote:
>>  > >
>>  > > > Hi everyone,
>>  > > >
>>  > > > The task of removal IGFS and Hadoop accelerator is ready to review.(
>>  > > > https://issues.apache.org/jira/browse/IGNITE-11942)
>>  > > > I've already asked some guys to take a look at it but if somebody
>>  > > familiar
>>  > > > with this part of code, feel free to take a look at the changes
>>  > > > too(especially scripts changes).
>>  > > >
>>  > > > I also think it is good to decide which release it should be planned
>>  > on.
>>  > > > This task planned for 2.9 right now but I should notice that first of
>>  > all
>>  > > > there are a lot of changes and secondly there are some changes in
>>  > public
>>  > > > API(removed some methods from configuration). So maybe it makes sense
>>  > to
>>  > > > move this ticket to the next release. What do you think?
>>  > > >
>>  > > > --
>>  > > > Best regards,
>>  > > > Anton Kalashnikov
>>  > > >
>>  > > >
>>  > > > 10.02.2020, 15:45, "Alexey Zinoviev" :
>>  > > > > Thank you so you much! Will wait:)
>>  > > > >
>>  > > > > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
>>  > > > alexey.goncha...@gmail.com>:
>>  > > > >
>>  > > > >> Got it, then no need to rush, let's wait for the TF-IGFS
>>  > decoupling.
>>  > > > >>
>>  > > > >> пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <
>>  > > zaleslaw@gmail.com
>>  > > > >:
>>  > > > >>
>>  > > > >> > Tensorflow integration uses IGFS, if you have any idea how to
>>  > store
>>  > > > files
>>  > > > >> > in memory by another way, please suggest something.
>>  > > > >> > I hope to decouple Ignite-TF integration to the separate
>>  > repository
>>  > > > >> before
>>  > > > >> > release 2.9 with its own file system over Ignite Caches
>>  > > > >> >
>>  > > > >> > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <
>>  > vololo...@gmail.com
>>  > > >:
>>  > > > >> >
>>  > > > >> > > Is not it blocked by
>>  > > > >> > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated
>>  > in
>>  > > > JIRA?
>>  > > 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Denis Magda
Hi Alex,

It's been a year since we voted to discontinue this integration [1] and it
wasn't removed from the source code earlier only because of the internal
dependencies with the ML component. Now all the dependencies are gone and
Ignite 2.9 is the right version to finish the discontinuation process. It
would make sense to wait for Ignite 3.0 only there are some breaking
changes in the APIs that will stay in Ignite.

 @Anton Kalashnikov , you mentioned that you
removed some methods from the configuration. Could you please list them
here? Are they Hadoop-specific or generic?

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html


-
Denis


On Wed, Jul 22, 2020 at 1:52 AM Alex Plehanov 
wrote:

> Guys,
>
> Any updates here? Looks like we still don't have a consensus about release
> version for this patch (already mention it in the release thread).
> Currently, the ticket is still targeted to 2.9.
>
> ср, 15 июл. 2020 г. в 00:40, Denis Magda :
>
> > I don't think it's required to wait until Ignite 3.0 to make this happen.
> > If I'm not mistaken, we stopped releasing Hadoop binaries and sources a
> > long time ago (at least you can't longer find them on the downloads
> page).
> > Also, we removed all the mentioning from the documentation and website.
> > Nobody complained or requested for a maintenance release since that time.
> > Thus, I would remove the integration in 2.9. If anybody shows up later
> then
> > they can use the sources in the 2.8 branch and do whatever they want.
> >
> > -
> > Denis
> >
> >
> > On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn 
> > wrote:
> >
> > > We are breaking backwards compatibility,
> > > so this can be only done for Ignite 3.0, am I right?
> > >
> > > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov 
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > The task of removal IGFS and Hadoop accelerator is ready to review.(
> > > > https://issues.apache.org/jira/browse/IGNITE-11942)
> > > > I've already asked some guys to take a look at it but if somebody
> > > familiar
> > > > with this part of code, feel free to take a look at the changes
> > > > too(especially scripts changes).
> > > >
> > > > I also think it is good to decide which release it should be planned
> > on.
> > > > This task planned for 2.9 right now but I should notice that first of
> > all
> > > > there are a lot of changes and secondly there are some changes in
> > public
> > > > API(removed some methods from configuration). So maybe it makes sense
> > to
> > > > move this ticket to the next release. What do you think?
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > >
> > > > 10.02.2020, 15:45, "Alexey Zinoviev" :
> > > > > Thank you so you much! Will wait:)
> > > > >
> > > > > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com>:
> > > > >
> > > > >>  Got it, then no need to rush, let's wait for the TF-IGFS
> > decoupling.
> > > > >>
> > > > >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <
> > > zaleslaw@gmail.com
> > > > >:
> > > > >>
> > > > >>  > Tensorflow integration uses IGFS, if you have any idea how to
> > store
> > > > files
> > > > >>  > in memory by another way, please suggest something.
> > > > >>  > I hope to decouple Ignite-TF integration to the separate
> > repository
> > > > >>  before
> > > > >>  > release 2.9 with its own file system over Ignite Caches
> > > > >>  >
> > > > >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <
> > vololo...@gmail.com
> > > >:
> > > > >>  >
> > > > >>  > > Is not it blocked by
> > > > >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated
> > in
> > > > JIRA?
> > > > >>  > >
> > > > >>  > > @Alex Zinoviev could you please shed some light on this?
> > > > >>  > >
> > > > >>  > > Best regards,
> > > > >>  > > Ivan Pavlukhin
> > > > >>  > >
> > > > >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <
> > > kaa@yandex.ru
> > > > >:
> > > > >>  > >
> > > > >>  > > >
> > > > >>  > > > I found the correct ticket for such activity -
> > > > >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> > > > >>  > > >
> > > > >>  > > > --
> > > > >>  > > > Best regards,
> > > > >>  > > > Anton Kalashnikov
> > > > >>  > > >
> > > > >>  > > >
> > > > >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov"  >:
> > > > >>  > > > > Hello.
> > > > >>  > > > >
> > > > >>  > > > > I created a ticket for this activity -
> > > > >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if
> we
> > > are
> > > > >>  still
> > > > >>  > > in consensus I'll do it at the nearest time(I've already had
> > the
> > > > >>  prepared
> > > > >>  > > code).
> > > > >>  > > > >
> > > > >>  > > > > --
> > > > >>  > > > > Best regards,
> > > > >>  > > > > Anton Kalashnikov
> > > > >>  > > > >
> > > > >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> > > > alexey.goncha...@gmail.com
> > > > >>  >:
> > > > >>  > > 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-22 Thread Alex Plehanov
Guys,

Any updates here? Looks like we still don't have a consensus about release
version for this patch (already mention it in the release thread).
Currently, the ticket is still targeted to 2.9.

ср, 15 июл. 2020 г. в 00:40, Denis Magda :

> I don't think it's required to wait until Ignite 3.0 to make this happen.
> If I'm not mistaken, we stopped releasing Hadoop binaries and sources a
> long time ago (at least you can't longer find them on the downloads page).
> Also, we removed all the mentioning from the documentation and website.
> Nobody complained or requested for a maintenance release since that time.
> Thus, I would remove the integration in 2.9. If anybody shows up later then
> they can use the sources in the 2.8 branch and do whatever they want.
>
> -
> Denis
>
>
> On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn 
> wrote:
>
> > We are breaking backwards compatibility,
> > so this can be only done for Ignite 3.0, am I right?
> >
> > On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov 
> > wrote:
> >
> > > Hi everyone,
> > >
> > > The task of removal IGFS and Hadoop accelerator is ready to review.(
> > > https://issues.apache.org/jira/browse/IGNITE-11942)
> > > I've already asked some guys to take a look at it but if somebody
> > familiar
> > > with this part of code, feel free to take a look at the changes
> > > too(especially scripts changes).
> > >
> > > I also think it is good to decide which release it should be planned
> on.
> > > This task planned for 2.9 right now but I should notice that first of
> all
> > > there are a lot of changes and secondly there are some changes in
> public
> > > API(removed some methods from configuration). So maybe it makes sense
> to
> > > move this ticket to the next release. What do you think?
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > >
> > > 10.02.2020, 15:45, "Alexey Zinoviev" :
> > > > Thank you so you much! Will wait:)
> > > >
> > > > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com>:
> > > >
> > > >>  Got it, then no need to rush, let's wait for the TF-IGFS
> decoupling.
> > > >>
> > > >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <
> > zaleslaw@gmail.com
> > > >:
> > > >>
> > > >>  > Tensorflow integration uses IGFS, if you have any idea how to
> store
> > > files
> > > >>  > in memory by another way, please suggest something.
> > > >>  > I hope to decouple Ignite-TF integration to the separate
> repository
> > > >>  before
> > > >>  > release 2.9 with its own file system over Ignite Caches
> > > >>  >
> > > >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin <
> vololo...@gmail.com
> > >:
> > > >>  >
> > > >>  > > Is not it blocked by
> > > >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated
> in
> > > JIRA?
> > > >>  > >
> > > >>  > > @Alex Zinoviev could you please shed some light on this?
> > > >>  > >
> > > >>  > > Best regards,
> > > >>  > > Ivan Pavlukhin
> > > >>  > >
> > > >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <
> > kaa@yandex.ru
> > > >:
> > > >>  > >
> > > >>  > > >
> > > >>  > > > I found the correct ticket for such activity -
> > > >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> > > >>  > > >
> > > >>  > > > --
> > > >>  > > > Best regards,
> > > >>  > > > Anton Kalashnikov
> > > >>  > > >
> > > >>  > > >
> > > >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> > > >>  > > > > Hello.
> > > >>  > > > >
> > > >>  > > > > I created a ticket for this activity -
> > > >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we
> > are
> > > >>  still
> > > >>  > > in consensus I'll do it at the nearest time(I've already had
> the
> > > >>  prepared
> > > >>  > > code).
> > > >>  > > > >
> > > >>  > > > > --
> > > >>  > > > > Best regards,
> > > >>  > > > > Anton Kalashnikov
> > > >>  > > > >
> > > >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> > > alexey.goncha...@gmail.com
> > > >>  >:
> > > >>  > > > >> Folks,
> > > >>  > > > >>
> > > >>  > > > >> I think there is a consensus here, but we did not remove
> > IGFS
> > > >>  > > neither in
> > > >>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding
> > > ticket
> > > >>  > > for 2.9?
> > > >>  > >
> > > >>  >
> > >
> >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-14 Thread Denis Magda
I don't think it's required to wait until Ignite 3.0 to make this happen.
If I'm not mistaken, we stopped releasing Hadoop binaries and sources a
long time ago (at least you can't longer find them on the downloads page).
Also, we removed all the mentioning from the documentation and website.
Nobody complained or requested for a maintenance release since that time.
Thus, I would remove the integration in 2.9. If anybody shows up later then
they can use the sources in the 2.8 branch and do whatever they want.

-
Denis


On Thu, Jul 9, 2020 at 3:52 AM Pavel Tupitsyn  wrote:

> We are breaking backwards compatibility,
> so this can be only done for Ignite 3.0, am I right?
>
> On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov 
> wrote:
>
> > Hi everyone,
> >
> > The task of removal IGFS and Hadoop accelerator is ready to review.(
> > https://issues.apache.org/jira/browse/IGNITE-11942)
> > I've already asked some guys to take a look at it but if somebody
> familiar
> > with this part of code, feel free to take a look at the changes
> > too(especially scripts changes).
> >
> > I also think it is good to decide which release it should be planned on.
> > This task planned for 2.9 right now but I should notice that first of all
> > there are a lot of changes and secondly there are some changes in public
> > API(removed some methods from configuration). So maybe it makes sense to
> > move this ticket to the next release. What do you think?
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> > 10.02.2020, 15:45, "Alexey Zinoviev" :
> > > Thank you so you much! Will wait:)
> > >
> > > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> > alexey.goncha...@gmail.com>:
> > >
> > >>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
> > >>
> > >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev <
> zaleslaw@gmail.com
> > >:
> > >>
> > >>  > Tensorflow integration uses IGFS, if you have any idea how to store
> > files
> > >>  > in memory by another way, please suggest something.
> > >>  > I hope to decouple Ignite-TF integration to the separate repository
> > >>  before
> > >>  > release 2.9 with its own file system over Ignite Caches
> > >>  >
> > >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin  >:
> > >>  >
> > >>  > > Is not it blocked by
> > >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in
> > JIRA?
> > >>  > >
> > >>  > > @Alex Zinoviev could you please shed some light on this?
> > >>  > >
> > >>  > > Best regards,
> > >>  > > Ivan Pavlukhin
> > >>  > >
> > >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov <
> kaa@yandex.ru
> > >:
> > >>  > >
> > >>  > > >
> > >>  > > > I found the correct ticket for such activity -
> > >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> > >>  > > >
> > >>  > > > --
> > >>  > > > Best regards,
> > >>  > > > Anton Kalashnikov
> > >>  > > >
> > >>  > > >
> > >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> > >>  > > > > Hello.
> > >>  > > > >
> > >>  > > > > I created a ticket for this activity -
> > >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we
> are
> > >>  still
> > >>  > > in consensus I'll do it at the nearest time(I've already had the
> > >>  prepared
> > >>  > > code).
> > >>  > > > >
> > >>  > > > > --
> > >>  > > > > Best regards,
> > >>  > > > > Anton Kalashnikov
> > >>  > > > >
> > >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> > alexey.goncha...@gmail.com
> > >>  >:
> > >>  > > > >> Folks,
> > >>  > > > >>
> > >>  > > > >> I think there is a consensus here, but we did not remove
> IGFS
> > >>  > > neither in
> > >>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding
> > ticket
> > >>  > > for 2.9?
> > >>  > >
> > >>  >
> >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-09 Thread Pavel Tupitsyn
We are breaking backwards compatibility,
so this can be only done for Ignite 3.0, am I right?

On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov  wrote:

> Hi everyone,
>
> The task of removal IGFS and Hadoop accelerator is ready to review.(
> https://issues.apache.org/jira/browse/IGNITE-11942)
> I've already asked some guys to take a look at it but if somebody familiar
> with this part of code, feel free to take a look at the changes
> too(especially scripts changes).
>
> I also think it is good to decide which release it should be planned on.
> This task planned for 2.9 right now but I should notice that first of all
> there are a lot of changes and secondly there are some changes in public
> API(removed some methods from configuration). So maybe it makes sense to
> move this ticket to the next release. What do you think?
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 10.02.2020, 15:45, "Alexey Zinoviev" :
> > Thank you so you much! Will wait:)
> >
> > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> alexey.goncha...@gmail.com>:
> >
> >>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
> >>
> >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev  >:
> >>
> >>  > Tensorflow integration uses IGFS, if you have any idea how to store
> files
> >>  > in memory by another way, please suggest something.
> >>  > I hope to decouple Ignite-TF integration to the separate repository
> >>  before
> >>  > release 2.9 with its own file system over Ignite Caches
> >>  >
> >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
> >>  >
> >>  > > Is not it blocked by
> >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in
> JIRA?
> >>  > >
> >>  > > @Alex Zinoviev could you please shed some light on this?
> >>  > >
> >>  > > Best regards,
> >>  > > Ivan Pavlukhin
> >>  > >
> >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov  >:
> >>  > >
> >>  > > >
> >>  > > > I found the correct ticket for such activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> >>  > > >
> >>  > > > --
> >>  > > > Best regards,
> >>  > > > Anton Kalashnikov
> >>  > > >
> >>  > > >
> >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> >>  > > > > Hello.
> >>  > > > >
> >>  > > > > I created a ticket for this activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
> >>  still
> >>  > > in consensus I'll do it at the nearest time(I've already had the
> >>  prepared
> >>  > > code).
> >>  > > > >
> >>  > > > > --
> >>  > > > > Best regards,
> >>  > > > > Anton Kalashnikov
> >>  > > > >
> >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> alexey.goncha...@gmail.com
> >>  >:
> >>  > > > >> Folks,
> >>  > > > >>
> >>  > > > >> I think there is a consensus here, but we did not remove IGFS
> >>  > > neither in
> >>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding
> ticket
> >>  > > for 2.9?
> >>  > >
> >>  >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-09 Thread Anton Kalashnikov
Hi everyone,

The task of removal IGFS and Hadoop accelerator is ready to 
review.(https://issues.apache.org/jira/browse/IGNITE-11942)
I've already asked some guys to take a look at it but if somebody familiar with 
this part of code, feel free to take a look at the changes too(especially 
scripts changes).

I also think it is good to decide which release it should be planned on. This 
task planned for 2.9 right now but I should notice that first of all there are 
a lot of changes and secondly there are some changes in public API(removed some 
methods from configuration). So maybe it makes sense to move this ticket to the 
next release. What do you think?

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 15:45, "Alexey Zinoviev" :
> Thank you so you much! Will wait:)
>
> пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk :
>
>>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
>>
>>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev :
>>
>>  > Tensorflow integration uses IGFS, if you have any idea how to store files
>>  > in memory by another way, please suggest something.
>>  > I hope to decouple Ignite-TF integration to the separate repository
>>  before
>>  > release 2.9 with its own file system over Ignite Caches
>>  >
>>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
>>  >
>>  > > Is not it blocked by
>>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
>>  > >
>>  > > @Alex Zinoviev could you please shed some light on this?
>>  > >
>>  > > Best regards,
>>  > > Ivan Pavlukhin
>>  > >
>>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :
>>  > >
>>  > > >
>>  > > > I found the correct ticket for such activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-11942
>>  > > >
>>  > > > --
>>  > > > Best regards,
>>  > > > Anton Kalashnikov
>>  > > >
>>  > > >
>>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
>>  > > > > Hello.
>>  > > > >
>>  > > > > I created a ticket for this activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
>>  still
>>  > > in consensus I'll do it at the nearest time(I've already had the
>>  prepared
>>  > > code).
>>  > > > >
>>  > > > > --
>>  > > > > Best regards,
>>  > > > > Anton Kalashnikov
>>  > > > >
>>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" >  >:
>>  > > > >> Folks,
>>  > > > >>
>>  > > > >> I think there is a consensus here, but we did not remove IGFS
>>  > > neither in
>>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
>>  > > for 2.9?
>>  > >
>>  >


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Alexey Zinoviev
Thank you so you much! Will wait:)

пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk :

> Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
>
> пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev :
>
> > Tensorflow integration uses IGFS, if you have any idea how to store files
> > in memory by another way, please suggest something.
> > I hope to decouple Ignite-TF integration to the separate repository
> before
> > release 2.9 with its own file system over Ignite Caches
> >
> > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
> >
> > > Is not it blocked by
> > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
> > >
> > > @Alex Zinoviev could you please shed some light on this?
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :
> > >
> > > >
> > > > I found the correct ticket for such activity -
> > > https://issues.apache.org/jira/browse/IGNITE-11942
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > >
> > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> > > > > Hello.
> > > > >
> > > > > I created a ticket for this activity -
> > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
> still
> > > in consensus I'll do it at the nearest time(I've already had the
> prepared
> > > code).
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Anton Kalashnikov
> > > > >
> > > > > 10.02.2020, 12:07, "Alexey Goncharuk"  >:
> > > > >>  Folks,
> > > > >>
> > > > >>  I think there is a consensus here, but we did not remove IGFS
> > > neither in
> > > > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> > > for 2.9?
> > >
> >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Alexey Goncharuk
Got it, then no need to rush, let's wait for the TF-IGFS decoupling.

пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev :

> Tensorflow integration uses IGFS, if you have any idea how to store files
> in memory by another way, please suggest something.
> I hope to decouple Ignite-TF integration to the separate repository before
> release 2.9 with its own file system over Ignite Caches
>
> пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
>
> > Is not it blocked by
> > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
> >
> > @Alex Zinoviev could you please shed some light on this?
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :
> >
> > >
> > > I found the correct ticket for such activity -
> > https://issues.apache.org/jira/browse/IGNITE-11942
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > >
> > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> > > > Hello.
> > > >
> > > > I created a ticket for this activity -
> > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still
> > in consensus I'll do it at the nearest time(I've already had the prepared
> > code).
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > > 10.02.2020, 12:07, "Alexey Goncharuk" :
> > > >>  Folks,
> > > >>
> > > >>  I think there is a consensus here, but we did not remove IGFS
> > neither in
> > > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> > for 2.9?
> >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Alexey Zinoviev
Tensorflow integration uses IGFS, if you have any idea how to store files
in memory by another way, please suggest something.
I hope to decouple Ignite-TF integration to the separate repository before
release 2.9 with its own file system over Ignite Caches

пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :

> Is not it blocked by
> https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
>
> @Alex Zinoviev could you please shed some light on this?
>
> Best regards,
> Ivan Pavlukhin
>
> пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :
>
> >
> > I found the correct ticket for such activity -
> https://issues.apache.org/jira/browse/IGNITE-11942
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> > 10.02.2020, 12:16, "Anton Kalashnikov" :
> > > Hello.
> > >
> > > I created a ticket for this activity -
> https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still
> in consensus I'll do it at the nearest time(I've already had the prepared
> code).
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > > 10.02.2020, 12:07, "Alexey Goncharuk" :
> > >>  Folks,
> > >>
> > >>  I think there is a consensus here, but we did not remove IGFS
> neither in
> > >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
> for 2.9?
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Ivan Pavlukhin
Is not it blocked by
https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?

@Alex Zinoviev could you please shed some light on this?

Best regards,
Ivan Pavlukhin

пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :

>
> I found the correct ticket for such activity - 
> https://issues.apache.org/jira/browse/IGNITE-11942
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 10.02.2020, 12:16, "Anton Kalashnikov" :
> > Hello.
> >
> > I created a ticket for this activity - 
> > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in 
> > consensus I'll do it at the nearest time(I've already had the prepared 
> > code).
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> > 10.02.2020, 12:07, "Alexey Goncharuk" :
> >>  Folks,
> >>
> >>  I think there is a consensus here, but we did not remove IGFS neither in
> >>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Anton Kalashnikov
I found the correct ticket for such activity - 
https://issues.apache.org/jira/browse/IGNITE-11942

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 12:16, "Anton Kalashnikov" :
> Hello.
>
> I created a ticket for this activity - 
> https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in 
> consensus I'll do it at the nearest time(I've already had the prepared code).
>
> --
> Best regards,
> Anton Kalashnikov
>
> 10.02.2020, 12:07, "Alexey Goncharuk" :
>>  Folks,
>>
>>  I think there is a consensus here, but we did not remove IGFS neither in
>>  2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Anton Kalashnikov
Hello.

I created a ticket for this activity - 
https://issues.apache.org/jira/browse/IGNITE-12647. And if we are still in 
consensus I'll do it at the nearest time(I've already had the prepared code). 

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 12:07, "Alexey Goncharuk" :
> Folks,
>
> I think there is a consensus here, but we did not remove IGFS neither in
> 2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-02-10 Thread Alexey Goncharuk
Folks,

I think there is a consensus here, but we did not remove IGFS neither in
2.7 nor in 2.8, did we? Should we schedule a corresponding ticket for 2.9?


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-18 Thread Denis Magda
Dmitriy,

I like the first approach with applying Maxim's idea of creating a branch
> named -igfs-Hadoop (not release, but current master state).


+1 for this approach. Any other opinions before we finalize this discussion?

-
Denis


On Tue, Jun 18, 2019 at 11:28 AM Dmitriy Pavlov  wrote:

> Hi Denis,
>
> I like the first approach with applying Maxim's idea of creating a branch
> named -igfs-Hadoop (not release, but current master state).
> 2nd) 3rd party repo can be Apache repo just like ignite-release. But it's
> true it is time-consuming to move code.
> 3rd) Attic is for projects, I hope no one here wants to Ignite to be there
> :) I'm not sure it is possible to move just one component there. But if it
> is possible, we should anyway start from 2nd option and create standalone
> repo ignite-igfs-hadoop (and it will become later
> attic-ignite-igfs-hadoop).
>
> But if someone could stand up and say he/she wants to do migration from one
> repo to another (option 2), I like it as well.
>
> Sincerely
>
> вт, 18 июн. 2019 г. в 21:05, Denis Magda :
>
> > Igniters,
> >
> > Thanks a lot for sharing your opinion. As I see, there is a consensus
> that
> > IGFS and Hadoop Accelerator are to be discontinued and no longer
> supported
> > by the community.
> >
> > As for the source code, if the community prefers moving the source code
> to
> > another repository rather than removing it, then let's do it. I see 3
> > solutions here:
> >
> >- The simplest - just point out to the latest Ignite release branch
> that
> >has the source code. This should be Ignite 2.6.0. Remove from Ignite
> > master.
> >- Decouple from the master and move to a 3rd party Github repo. More
> >complicated and time-consuming.
> >- See if we should move the component to Apache Attic (
> >http://attic.apache.org): the Attic is designed for projects to be
> >retired but not for the components. Thus, that might be not an option.
> >
> > Personally, I'm for the first approach. Does it sound reasonable?
> >
> > -
> > Denis
> >
> >
> > On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov 
> > wrote:
> >
> > > Folks,
> > >
> > > +1 to reduce the number of supported features.
> > >
> > > Probably, the best solution will be removing IGFS from core module and
> > > making it as an Ignite plugin (will require some efforts to do this).
> > > I've also think we can move IGFS to the separate branch (from the
> > > master one) if someone will decide merge to latest changes from the
> > > master branch to build Ignite from scratch with IGFS feature.
> > >
> > > On Mon, 17 Jun 2019 at 22:42, Denis Magda  wrote:
> > > >
> > > > >
> > > > > +1 from me to reduce supported feature list.
> > > > > Guys, are we talking about Ignite 3.0?
> > > >
> > > >
> > > > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the
> next
> > > > release? As for other features and integrations, 3.0 should be
> > considered
> > > > as a version.
> > > >
> > > >
> > > > > +1 from me provided that we move sources to some supplementary
> > > repository.
> > > > > If someone later would like to maintain/fix this module, he/she
> > should
> > > be
> > > > > able to access sources with current state of the master.
> > > >
> > > >
> > > > Dmitry, are you suggesting to move the sources to Github and abandon
> > them
> > > > there? Sort of legacy code cemetery.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov  >
> > > wrote:
> > > >
> > > > > +1 from me to reduce supported feature list.
> > > > >
> > > > > Guys, are we talking about Ignite 3.0?
> > > > >
> > > > >
> > > > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > > > Denis,
> > > > > >
> > > > > > I fully support this idea.
> > > > > >
> > > > > > First, looking back, I do not think it was a good design in the
> > first
> > > > > place
> > > > > > to build IGFS on top of Ignite caches. Second, I have never seen
> a
> > > case
> > > > > > where IGFS provided significant performance boost. Usually it's
> > > either
> > > > > all
> > > > > > data already fits buffer cache, and IGFS caching is not needed;
> or
> > > data
> > > > > > does not fit buffer cache, and access pattern is close to full
> scan
> > > and
> > > > > > additional caching in IGFS does not make sense.
> > > > > >
> > > > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван  >:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> > > appear
> > > > > > > from time to time in questions on a user mailing list. So, it
> > seems
> > > > > > > that there is a practical need for such solutions.
> > > > > > >
> > > > > > > But of course it does not mean that we should continue a
> support
> > of
> > > > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> > > well
> > > > > > > common use cases then we should discontinue it. If any of them
> is
> > > very
> > > > > > > good for it's 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-18 Thread Dmitriy Pavlov
Hi Denis,

I like the first approach with applying Maxim's idea of creating a branch
named -igfs-Hadoop (not release, but current master state).
2nd) 3rd party repo can be Apache repo just like ignite-release. But it's
true it is time-consuming to move code.
3rd) Attic is for projects, I hope no one here wants to Ignite to be there
:) I'm not sure it is possible to move just one component there. But if it
is possible, we should anyway start from 2nd option and create standalone
repo ignite-igfs-hadoop (and it will become later attic-ignite-igfs-hadoop).

But if someone could stand up and say he/she wants to do migration from one
repo to another (option 2), I like it as well.

Sincerely

вт, 18 июн. 2019 г. в 21:05, Denis Magda :

> Igniters,
>
> Thanks a lot for sharing your opinion. As I see, there is a consensus that
> IGFS and Hadoop Accelerator are to be discontinued and no longer supported
> by the community.
>
> As for the source code, if the community prefers moving the source code to
> another repository rather than removing it, then let's do it. I see 3
> solutions here:
>
>- The simplest - just point out to the latest Ignite release branch that
>has the source code. This should be Ignite 2.6.0. Remove from Ignite
> master.
>- Decouple from the master and move to a 3rd party Github repo. More
>complicated and time-consuming.
>- See if we should move the component to Apache Attic (
>http://attic.apache.org): the Attic is designed for projects to be
>retired but not for the components. Thus, that might be not an option.
>
> Personally, I'm for the first approach. Does it sound reasonable?
>
> -
> Denis
>
>
> On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov 
> wrote:
>
> > Folks,
> >
> > +1 to reduce the number of supported features.
> >
> > Probably, the best solution will be removing IGFS from core module and
> > making it as an Ignite plugin (will require some efforts to do this).
> > I've also think we can move IGFS to the separate branch (from the
> > master one) if someone will decide merge to latest changes from the
> > master branch to build Ignite from scratch with IGFS feature.
> >
> > On Mon, 17 Jun 2019 at 22:42, Denis Magda  wrote:
> > >
> > > >
> > > > +1 from me to reduce supported feature list.
> > > > Guys, are we talking about Ignite 3.0?
> > >
> > >
> > > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> > > release? As for other features and integrations, 3.0 should be
> considered
> > > as a version.
> > >
> > >
> > > > +1 from me provided that we move sources to some supplementary
> > repository.
> > > > If someone later would like to maintain/fix this module, he/she
> should
> > be
> > > > able to access sources with current state of the master.
> > >
> > >
> > > Dmitry, are you suggesting to move the sources to Github and abandon
> them
> > > there? Sort of legacy code cemetery.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov 
> > wrote:
> > >
> > > > +1 from me to reduce supported feature list.
> > > >
> > > > Guys, are we talking about Ignite 3.0?
> > > >
> > > >
> > > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > > Denis,
> > > > >
> > > > > I fully support this idea.
> > > > >
> > > > > First, looking back, I do not think it was a good design in the
> first
> > > > place
> > > > > to build IGFS on top of Ignite caches. Second, I have never seen a
> > case
> > > > > where IGFS provided significant performance boost. Usually it's
> > either
> > > > all
> > > > > data already fits buffer cache, and IGFS caching is not needed; or
> > data
> > > > > does not fit buffer cache, and access pattern is close to full scan
> > and
> > > > > additional caching in IGFS does not make sense.
> > > > >
> > > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> > appear
> > > > > > from time to time in questions on a user mailing list. So, it
> seems
> > > > > > that there is a practical need for such solutions.
> > > > > >
> > > > > > But of course it does not mean that we should continue a support
> of
> > > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> > well
> > > > > > common use cases then we should discontinue it. If any of them is
> > very
> > > > > > good for it's purposes but we do not have a capacity to support
> it
> > > > > > without sacrificing main Ignite goals then we still should
> > discontinue
> > > > > > it in my mind.
> > > > > >
> > > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > > > responsibility and integrations. And I suppose there are other
> > Ignite
> > > > > > features which could be dropped.
> > > > > >
> > > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > > > > >
> > > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I'd like us to move on and finish our 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-18 Thread Denis Magda
Igniters,

Thanks a lot for sharing your opinion. As I see, there is a consensus that
IGFS and Hadoop Accelerator are to be discontinued and no longer supported
by the community.

As for the source code, if the community prefers moving the source code to
another repository rather than removing it, then let's do it. I see 3
solutions here:

   - The simplest - just point out to the latest Ignite release branch that
   has the source code. This should be Ignite 2.6.0. Remove from Ignite master.
   - Decouple from the master and move to a 3rd party Github repo. More
   complicated and time-consuming.
   - See if we should move the component to Apache Attic (
   http://attic.apache.org): the Attic is designed for projects to be
   retired but not for the components. Thus, that might be not an option.

Personally, I'm for the first approach. Does it sound reasonable?

-
Denis


On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov  wrote:

> Folks,
>
> +1 to reduce the number of supported features.
>
> Probably, the best solution will be removing IGFS from core module and
> making it as an Ignite plugin (will require some efforts to do this).
> I've also think we can move IGFS to the separate branch (from the
> master one) if someone will decide merge to latest changes from the
> master branch to build Ignite from scratch with IGFS feature.
>
> On Mon, 17 Jun 2019 at 22:42, Denis Magda  wrote:
> >
> > >
> > > +1 from me to reduce supported feature list.
> > > Guys, are we talking about Ignite 3.0?
> >
> >
> > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> > release? As for other features and integrations, 3.0 should be considered
> > as a version.
> >
> >
> > > +1 from me provided that we move sources to some supplementary
> repository.
> > > If someone later would like to maintain/fix this module, he/she should
> be
> > > able to access sources with current state of the master.
> >
> >
> > Dmitry, are you suggesting to move the sources to Github and abandon them
> > there? Sort of legacy code cemetery.
> >
> > -
> > Denis
> >
> >
> > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov 
> wrote:
> >
> > > +1 from me to reduce supported feature list.
> > >
> > > Guys, are we talking about Ignite 3.0?
> > >
> > >
> > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > Denis,
> > > >
> > > > I fully support this idea.
> > > >
> > > > First, looking back, I do not think it was a good design in the first
> > > place
> > > > to build IGFS on top of Ignite caches. Second, I have never seen a
> case
> > > > where IGFS provided significant performance boost. Usually it's
> either
> > > all
> > > > data already fits buffer cache, and IGFS caching is not needed; or
> data
> > > > does not fit buffer cache, and access pattern is close to full scan
> and
> > > > additional caching in IGFS does not make sense.
> > > >
> > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> > > >
> > > > > Denis,
> > > > >
> > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> appear
> > > > > from time to time in questions on a user mailing list. So, it seems
> > > > > that there is a practical need for such solutions.
> > > > >
> > > > > But of course it does not mean that we should continue a support of
> > > > > IGFS and Hadoop Accelerator. If both are not solutions that fit
> well
> > > > > common use cases then we should discontinue it. If any of them is
> very
> > > > > good for it's purposes but we do not have a capacity to support it
> > > > > without sacrificing main Ignite goals then we still should
> discontinue
> > > > > it in my mind.
> > > > >
> > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > > responsibility and integrations. And I suppose there are other
> Ignite
> > > > > features which could be dropped.
> > > > >
> > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > > > >
> > > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I'd like us to move on and finish our conversation on the IGFS
> [1]
> > > and
> > > > > > Hadoop Accelerator [2] support.
> > > > > >
> > > > > > To my knowledge, there is no single committer who maintains the
> > > > > > integrations; they are no longer tested and, even more, the
> community
> > > > > > stopped providing the binaries since Ignite 2.6.0 release (look
> for
> > > > > > In-Memory Hadoop Accelerator table [3]).
> > > > > >
> > > > > > Why all of that happened? Because of a little value, something
> > > succeeds
> > > > > > while something fails. Does it mean that Ignite cannot be used
> for
> > > Hadoop
> > > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > > but a
> > > > > > solution is different. Have Ignite with native persistence
> deployed
> > > close
> > > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > > >
> > > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator
> from
> > > our
> > > > > > master repository and rework 

Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-18 Thread Maxim Muzafarov
Folks,

+1 to reduce the number of supported features.

Probably, the best solution will be removing IGFS from core module and
making it as an Ignite plugin (will require some efforts to do this).
I've also think we can move IGFS to the separate branch (from the
master one) if someone will decide merge to latest changes from the
master branch to build Ignite from scratch with IGFS feature.

On Mon, 17 Jun 2019 at 22:42, Denis Magda  wrote:
>
> >
> > +1 from me to reduce supported feature list.
> > Guys, are we talking about Ignite 3.0?
>
>
> Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> release? As for other features and integrations, 3.0 should be considered
> as a version.
>
>
> > +1 from me provided that we move sources to some supplementary repository.
> > If someone later would like to maintain/fix this module, he/she should be
> > able to access sources with current state of the master.
>
>
> Dmitry, are you suggesting to move the sources to Github and abandon them
> there? Sort of legacy code cemetery.
>
> -
> Denis
>
>
> On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov  wrote:
>
> > +1 from me to reduce supported feature list.
> >
> > Guys, are we talking about Ignite 3.0?
> >
> >
> > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > Denis,
> > >
> > > I fully support this idea.
> > >
> > > First, looking back, I do not think it was a good design in the first
> > place
> > > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > > where IGFS provided significant performance boost. Usually it's either
> > all
> > > data already fits buffer cache, and IGFS caching is not needed; or data
> > > does not fit buffer cache, and access pattern is close to full scan and
> > > additional caching in IGFS does not make sense.
> > >
> > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> > >
> > > > Denis,
> > > >
> > > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > > from time to time in questions on a user mailing list. So, it seems
> > > > that there is a practical need for such solutions.
> > > >
> > > > But of course it does not mean that we should continue a support of
> > > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > > common use cases then we should discontinue it. If any of them is very
> > > > good for it's purposes but we do not have a capacity to support it
> > > > without sacrificing main Ignite goals then we still should discontinue
> > > > it in my mind.
> > > >
> > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > responsibility and integrations. And I suppose there are other Ignite
> > > > features which could be dropped.
> > > >
> > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > > >
> > > > >
> > > > > Igniters,
> > > > >
> > > > > I'd like us to move on and finish our conversation on the IGFS [1]
> > and
> > > > > Hadoop Accelerator [2] support.
> > > > >
> > > > > To my knowledge, there is no single committer who maintains the
> > > > > integrations; they are no longer tested and, even more, the community
> > > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > > In-Memory Hadoop Accelerator table [3]).
> > > > >
> > > > > Why all of that happened? Because of a little value, something
> > succeeds
> > > > > while something fails. Does it mean that Ignite cannot be used for
> > Hadoop
> > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > but a
> > > > > solution is different. Have Ignite with native persistence deployed
> > close
> > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > >
> > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> > our
> > > > > master repository and rework existing public documentation showing
> > how to
> > > > > achieve the acceleration with Ignite.
> > > > >
> > > > > Any supporters or objections?
> > > > >
> > > > >
> > > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > [4]
> > > > >
> > > >
> > > >
> > https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > >
> > > > > -
> > > > > Denis
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Denis Magda
>
> +1 from me to reduce supported feature list.
> Guys, are we talking about Ignite 3.0?


Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
release? As for other features and integrations, 3.0 should be considered
as a version.


> +1 from me provided that we move sources to some supplementary repository.
> If someone later would like to maintain/fix this module, he/she should be
> able to access sources with current state of the master.


Dmitry, are you suggesting to move the sources to Github and abandon them
there? Sort of legacy code cemetery.

-
Denis


On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov  wrote:

> +1 from me to reduce supported feature list.
>
> Guys, are we talking about Ignite 3.0?
>
>
> В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > Denis,
> >
> > I fully support this idea.
> >
> > First, looking back, I do not think it was a good design in the first
> place
> > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > where IGFS provided significant performance boost. Usually it's either
> all
> > data already fits buffer cache, and IGFS caching is not needed; or data
> > does not fit buffer cache, and access pattern is close to full scan and
> > additional caching in IGFS does not make sense.
> >
> > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> >
> > > Denis,
> > >
> > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > from time to time in questions on a user mailing list. So, it seems
> > > that there is a practical need for such solutions.
> > >
> > > But of course it does not mean that we should continue a support of
> > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > common use cases then we should discontinue it. If any of them is very
> > > good for it's purposes but we do not have a capacity to support it
> > > without sacrificing main Ignite goals then we still should discontinue
> > > it in my mind.
> > >
> > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > responsibility and integrations. And I suppose there are other Ignite
> > > features which could be dropped.
> > >
> > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > >
> > > >
> > > > Igniters,
> > > >
> > > > I'd like us to move on and finish our conversation on the IGFS [1]
> and
> > > > Hadoop Accelerator [2] support.
> > > >
> > > > To my knowledge, there is no single committer who maintains the
> > > > integrations; they are no longer tested and, even more, the community
> > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > In-Memory Hadoop Accelerator table [3]).
> > > >
> > > > Why all of that happened? Because of a little value, something
> succeeds
> > > > while something fails. Does it mean that Ignite cannot be used for
> Hadoop
> > > > acceleration, in general? No, quite the opposite, it CAN be used,
> but a
> > > > solution is different. Have Ignite with native persistence deployed
> close
> > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > >
> > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> our
> > > > master repository and rework existing public documentation showing
> how to
> > > > achieve the acceleration with Ignite.
> > > >
> > > > Any supporters or objections?
> > > >
> > > >
> > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > [4]
> > > >
> > >
> > >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > >
> > > > -
> > > > Denis
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Dmitriy Pavlov
Hi Igniters,

+1 from me provided that we move sources to some supplementary repository.
If someone later would like to maintain/fix this module, he/she should be
able to access sources with current state of the master.

Sincerely,
Dmitriy Pavlov

пн, 17 июн. 2019 г. в 18:25, Igor Sapego :

> +1 from me. Hadoop Accelerator seems like an outdated solution,
> and I believe IGFS was only added to support Hadoop Accelerator.
>
> Best Regards,
> Igor
>
>
> On Mon, Jun 17, 2019 at 12:00 PM Nikolay Izhikov 
> wrote:
>
> > +1 from me to reduce supported feature list.
> >
> > Guys, are we talking about Ignite 3.0?
> >
> >
> > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > Denis,
> > >
> > > I fully support this idea.
> > >
> > > First, looking back, I do not think it was a good design in the first
> > place
> > > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > > where IGFS provided significant performance boost. Usually it's either
> > all
> > > data already fits buffer cache, and IGFS caching is not needed; or data
> > > does not fit buffer cache, and access pattern is close to full scan and
> > > additional caching in IGFS does not make sense.
> > >
> > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> > >
> > > > Denis,
> > > >
> > > > I must say that aforementioned solutions for a Hadoop ecosystem
> appear
> > > > from time to time in questions on a user mailing list. So, it seems
> > > > that there is a practical need for such solutions.
> > > >
> > > > But of course it does not mean that we should continue a support of
> > > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > > common use cases then we should discontinue it. If any of them is
> very
> > > > good for it's purposes but we do not have a capacity to support it
> > > > without sacrificing main Ignite goals then we still should
> discontinue
> > > > it in my mind.
> > > >
> > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > responsibility and integrations. And I suppose there are other Ignite
> > > > features which could be dropped.
> > > >
> > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > > >
> > > > >
> > > > > Igniters,
> > > > >
> > > > > I'd like us to move on and finish our conversation on the IGFS [1]
> > and
> > > > > Hadoop Accelerator [2] support.
> > > > >
> > > > > To my knowledge, there is no single committer who maintains the
> > > > > integrations; they are no longer tested and, even more, the
> community
> > > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > > In-Memory Hadoop Accelerator table [3]).
> > > > >
> > > > > Why all of that happened? Because of a little value, something
> > succeeds
> > > > > while something fails. Does it mean that Ignite cannot be used for
> > Hadoop
> > > > > acceleration, in general? No, quite the opposite, it CAN be used,
> > but a
> > > > > solution is different. Have Ignite with native persistence deployed
> > close
> > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > >
> > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> > our
> > > > > master repository and rework existing public documentation showing
> > how to
> > > > > achieve the acceleration with Ignite.
> > > > >
> > > > > Any supporters or objections?
> > > > >
> > > > >
> > > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > [4]
> > > > >
> > > >
> > > >
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > >
> > > > > -
> > > > > Denis
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Igor Sapego
+1 from me. Hadoop Accelerator seems like an outdated solution,
and I believe IGFS was only added to support Hadoop Accelerator.

Best Regards,
Igor


On Mon, Jun 17, 2019 at 12:00 PM Nikolay Izhikov 
wrote:

> +1 from me to reduce supported feature list.
>
> Guys, are we talking about Ignite 3.0?
>
>
> В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > Denis,
> >
> > I fully support this idea.
> >
> > First, looking back, I do not think it was a good design in the first
> place
> > to build IGFS on top of Ignite caches. Second, I have never seen a case
> > where IGFS provided significant performance boost. Usually it's either
> all
> > data already fits buffer cache, and IGFS caching is not needed; or data
> > does not fit buffer cache, and access pattern is close to full scan and
> > additional caching in IGFS does not make sense.
> >
> > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> >
> > > Denis,
> > >
> > > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > > from time to time in questions on a user mailing list. So, it seems
> > > that there is a practical need for such solutions.
> > >
> > > But of course it does not mean that we should continue a support of
> > > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > > common use cases then we should discontinue it. If any of them is very
> > > good for it's purposes but we do not have a capacity to support it
> > > without sacrificing main Ignite goals then we still should discontinue
> > > it in my mind.
> > >
> > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > responsibility and integrations. And I suppose there are other Ignite
> > > features which could be dropped.
> > >
> > > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > >
> > > >
> > > > Igniters,
> > > >
> > > > I'd like us to move on and finish our conversation on the IGFS [1]
> and
> > > > Hadoop Accelerator [2] support.
> > > >
> > > > To my knowledge, there is no single committer who maintains the
> > > > integrations; they are no longer tested and, even more, the community
> > > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > > In-Memory Hadoop Accelerator table [3]).
> > > >
> > > > Why all of that happened? Because of a little value, something
> succeeds
> > > > while something fails. Does it mean that Ignite cannot be used for
> Hadoop
> > > > acceleration, in general? No, quite the opposite, it CAN be used,
> but a
> > > > solution is different. Have Ignite with native persistence deployed
> close
> > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > >
> > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from
> our
> > > > master repository and rework existing public documentation showing
> how to
> > > > achieve the acceleration with Ignite.
> > > >
> > > > Any supporters or objections?
> > > >
> > > >
> > > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > [4]
> > > >
> > >
> > >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > >
> > > > -
> > > > Denis
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Nikolay Izhikov
+1 from me to reduce supported feature list.

Guys, are we talking about Ignite 3.0?


В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> Denis,
> 
> I fully support this idea.
> 
> First, looking back, I do not think it was a good design in the first place
> to build IGFS on top of Ignite caches. Second, I have never seen a case
> where IGFS provided significant performance boost. Usually it's either all
> data already fits buffer cache, and IGFS caching is not needed; or data
> does not fit buffer cache, and access pattern is close to full scan and
> additional caching in IGFS does not make sense.
> 
> пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :
> 
> > Denis,
> > 
> > I must say that aforementioned solutions for a Hadoop ecosystem appear
> > from time to time in questions on a user mailing list. So, it seems
> > that there is a practical need for such solutions.
> > 
> > But of course it does not mean that we should continue a support of
> > IGFS and Hadoop Accelerator. If both are not solutions that fit well
> > common use cases then we should discontinue it. If any of them is very
> > good for it's purposes but we do not have a capacity to support it
> > without sacrificing main Ignite goals then we still should discontinue
> > it in my mind.
> > 
> > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > responsibility and integrations. And I suppose there are other Ignite
> > features which could be dropped.
> > 
> > ср, 12 июн. 2019 г. в 21:04, Denis Magda :
> > 
> > > 
> > > Igniters,
> > > 
> > > I'd like us to move on and finish our conversation on the IGFS [1] and
> > > Hadoop Accelerator [2] support.
> > > 
> > > To my knowledge, there is no single committer who maintains the
> > > integrations; they are no longer tested and, even more, the community
> > > stopped providing the binaries since Ignite 2.6.0 release (look for
> > > In-Memory Hadoop Accelerator table [3]).
> > > 
> > > Why all of that happened? Because of a little value, something succeeds
> > > while something fails. Does it mean that Ignite cannot be used for Hadoop
> > > acceleration, in general? No, quite the opposite, it CAN be used, but a
> > > solution is different. Have Ignite with native persistence deployed close
> > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > 
> > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> > > master repository and rework existing public documentation showing how to
> > > achieve the acceleration with Ignite.
> > > 
> > > Any supporters or objections?
> > > 
> > > 
> > > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > [3] https://ignite.apache.org/download.cgi#binaries
> > > [4]
> > > 
> > 
> > https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > 
> > > -
> > > Denis
> > 
> > 
> > 
> > --
> > Best regards,
> > Ivan Pavlukhin
> > 


signature.asc
Description: This is a digitally signed message part


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Alexey Goncharuk
Denis,

I fully support this idea.

First, looking back, I do not think it was a good design in the first place
to build IGFS on top of Ignite caches. Second, I have never seen a case
where IGFS provided significant performance boost. Usually it's either all
data already fits buffer cache, and IGFS caching is not needed; or data
does not fit buffer cache, and access pattern is close to full scan and
additional caching in IGFS does not make sense.

пн, 17 июн. 2019 г. в 11:28, Павлухин Иван :

> Denis,
>
> I must say that aforementioned solutions for a Hadoop ecosystem appear
> from time to time in questions on a user mailing list. So, it seems
> that there is a practical need for such solutions.
>
> But of course it does not mean that we should continue a support of
> IGFS and Hadoop Accelerator. If both are not solutions that fit well
> common use cases then we should discontinue it. If any of them is very
> good for it's purposes but we do not have a capacity to support it
> without sacrificing main Ignite goals then we still should discontinue
> it in my mind.
>
> P.S. Personally I am a fan of UNIX way. I like ideas of a single
> responsibility and integrations. And I suppose there are other Ignite
> features which could be dropped.
>
> ср, 12 июн. 2019 г. в 21:04, Denis Magda :
>
> >
> > Igniters,
> >
> > I'd like us to move on and finish our conversation on the IGFS [1] and
> > Hadoop Accelerator [2] support.
> >
> > To my knowledge, there is no single committer who maintains the
> > integrations; they are no longer tested and, even more, the community
> > stopped providing the binaries since Ignite 2.6.0 release (look for
> > In-Memory Hadoop Accelerator table [3]).
> >
> > Why all of that happened? Because of a little value, something succeeds
> > while something fails. Does it mean that Ignite cannot be used for Hadoop
> > acceleration, in general? No, quite the opposite, it CAN be used, but a
> > solution is different. Have Ignite with native persistence deployed close
> > to your Hadoop cluster (replace GridGain with Ignite) [4].
> >
> > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> > master repository and rework existing public documentation showing how to
> > achieve the acceleration with Ignite.
> >
> > Any supporters or objections?
> >
> >
> > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > [3] https://ignite.apache.org/download.cgi#binaries
> > [4]
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> >
> > -
> > Denis
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-17 Thread Павлухин Иван
Denis,

I must say that aforementioned solutions for a Hadoop ecosystem appear
from time to time in questions on a user mailing list. So, it seems
that there is a practical need for such solutions.

But of course it does not mean that we should continue a support of
IGFS and Hadoop Accelerator. If both are not solutions that fit well
common use cases then we should discontinue it. If any of them is very
good for it's purposes but we do not have a capacity to support it
without sacrificing main Ignite goals then we still should discontinue
it in my mind.

P.S. Personally I am a fan of UNIX way. I like ideas of a single
responsibility and integrations. And I suppose there are other Ignite
features which could be dropped.

ср, 12 июн. 2019 г. в 21:04, Denis Magda :

>
> Igniters,
>
> I'd like us to move on and finish our conversation on the IGFS [1] and
> Hadoop Accelerator [2] support.
>
> To my knowledge, there is no single committer who maintains the
> integrations; they are no longer tested and, even more, the community
> stopped providing the binaries since Ignite 2.6.0 release (look for
> In-Memory Hadoop Accelerator table [3]).
>
> Why all of that happened? Because of a little value, something succeeds
> while something fails. Does it mean that Ignite cannot be used for Hadoop
> acceleration, in general? No, quite the opposite, it CAN be used, but a
> solution is different. Have Ignite with native persistence deployed close
> to your Hadoop cluster (replace GridGain with Ignite) [4].
>
> So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> master repository and rework existing public documentation showing how to
> achieve the acceleration with Ignite.
>
> Any supporters or objections?
>
>
> [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> [3] https://ignite.apache.org/download.cgi#binaries
> [4]
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
>
> -
> Denis



--
Best regards,
Ivan Pavlukhin


[DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2019-06-12 Thread Denis Magda
Igniters,

I'd like us to move on and finish our conversation on the IGFS [1] and
Hadoop Accelerator [2] support.

To my knowledge, there is no single committer who maintains the
integrations; they are no longer tested and, even more, the community
stopped providing the binaries since Ignite 2.6.0 release (look for
In-Memory Hadoop Accelerator table [3]).

Why all of that happened? Because of a little value, something succeeds
while something fails. Does it mean that Ignite cannot be used for Hadoop
acceleration, in general? No, quite the opposite, it CAN be used, but a
solution is different. Have Ignite with native persistence deployed close
to your Hadoop cluster (replace GridGain with Ignite) [4].

So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
master repository and rework existing public documentation showing how to
achieve the acceleration with Ignite.

Any supporters or objections?


[1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
[2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
[3] https://ignite.apache.org/download.cgi#binaries
[4]
https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator

-
Denis