Re: Retirement steps

Hi @all, I complete all steps except 10. and 11. Two questions: 1) For step 3. I think we settled this a while ago but don't see a date on the status page. Does someone remember the details? 2) Step 8. b) can we still use the Apache GitHub for Crail or do we have to move if we don't want the

Re: Retirement steps

Hi Julian. Thank you for all your help. I'm out this week but can start with going through the steps next week. Jonas On Sun, 12 Jun 2022 11:09:57 -0700 Julian Hyde wrote: The following are the steps to retire Crail, based on

Re: [VOTE] Retire Crail from Incubator

+1 On Wed, 1 Jun 2022 12:28:07 -0700 Julian Hyde wrote: Crail has been in incubation since late 2017 [1]. For the last couple of years, activity has been low [2], and the chances of attracting new committers and building a vibrant community seem very slim at this point. I therefore

Re: [DISCUSS] Retire Crail from the Incubator

Julian, Thanks for starting the discussion. As previously indicated, given the low activity I think it makes sense to retire the project. Jonas On Mon, 30 May 2022 11:10:12 -0700 Julian Hyde wrote: Is it time to retire Crail? Please read the Guide to Retirement [1] to understand the

Re: Podling Crail Report Reminder - May 2022

Thanks Bernard. LGTM @Julian What would be the process of retiring Crail as an Apache incubator? Jonas On Wed, 4 May 2022 10:47:50 +0200 bernard metzler wrote: On 28/04/2022 17:38, Julian Hyde wrote: Can someone volunteer to write this report? I wrote the report for May. @all, please

Re: Board reports

Hi Julian, all, Bernard previously indicated no/low activity and possible implication for the future of Crail with Apache. I personally think we are at a point where there is little chance of Crail ever getting out of incubating status. With this in mind, maybe it is time to off-board from

Re: [MENTORS] Download pages and links to podling releases

Thanks for pointing this out. Bernard and I are looking into it. Jonas On Thu, 16 Sep 2021 00:12:33 -0700 Julian Hyde wrote: Download policy has changed and the downloads page has a few links that need to be updated. I logged a bug to track: https://issues.apache.org/jira/browse/CRAIL-117

Re: HDFS adapter

Laurent d'Orazio wrote: Hi Jonas, I have just checked, but I do not find it (even in my spams). Can you forward it to me? Thanks again and sorry to bother you with that :( Best, Laurent Le 27 avr. 2021 à 14:51, Jonas Pfefferle a écrit : Hi Laurent, The question has been answered by Patrick

Re: HDFS adapter

Hi Laurent, The question has been answered by Patrick already. Check your history... Best, Jonas On Mon, 26 Apr 2021 18:21:01 +0200 Laurent d'Orazio wrote: Dear all, A gentle remind. Thanks again. Best, Laurent Le 19 avr. 2021 à 18:07, Laurent d'Orazio a écrit : Dear all, My deepest

Re: Crail draft podling report June 2020

Thanks for the report Bernard. I added it to the wiki. Thanks, Jonas On Tue, 2 Jun 2020 11:04:00 +0200 bernard metzler wrote: Please find below a draft podling report for Crail, subject to discussion, suggestions, changes, etc. Initially I wanted to directly put that in at

Re: Podling Crail Report Reminder - March 2020

Bernard wrote the new podling report and I just added it. Thanks, Jonas On Sun, 1 Mar 2020 17:54:39 -0800 Julian Hyde wrote: Can someone volunteer to write this? In order to submit by Thursday, let’s get a draft onto dev by Monday or Tuesday. Julian On Mar 1, 2020, at 12:10,

Re: [VOTE] Release of Apache Crail-1.2-incubating rc2

Thanks for preparing the release Adrian! I checked the following items: * checksums are correct * signature good * source tarball compiles and shows right version in assembly * binary tarball executes (TCP) Please use Gitbox or GitHub links from now on

Re: [GitHub] [incubator-crail] PepperJo opened a new pull request #82: [NVMf] Make keepalive thread a daemon thread

reInputStream.java:108) at org.apache.crail.core.CoreDataStore.closeFileSystem(CoreDataStore.java:515) at org.apache.crail.CrailStore.close(CrailStore.java:52) ... 4 more Regards, David C: 714-476-2692 ____ From: Jonas Pfefferle Sent: Friday, August 16, 2

Re: [GitHub] [incubator-crail] PepperJo opened a new pull request #82: [NVMf] Make keepalive thread a daemon thread

it a shot. Thanks and enjoy your weekend… your a lot closer to it than me ! Regards, David From: Jonas Pfefferle Sent: Friday, August 16, 2019 12:24:30 AM To: dev@crail.apache.org ; David Crespi Subject: Re: [GitHub] [incubator-crail] PepperJo opened

Re: Clarifying questions...

Hi David, Good to hear things work now. 1) Technically, you can use the RdmaStorageTier "directly" with a SSD since it allocates its data in "datapath" (and then mmaps it). Now this path is typically a hugetlbfs but it can be a standard mount point. However, there are a few drawbacks with

Re: Setting up storage class 1 and 2

ics, freeBlocks 4096 NVMf datanode is showing 1TB. 19/07/01 17:23:57 INFO crail: datanode statistics, freeBlocks 1048576 Regards, David From: David Crespi Sent: Monday, July 1, 2019 3:57:42 PM To: Jonas Pfefferle; dev@crail.apache.org Subject: RE

Re: Setting up storage class 1 and 2

, David From: David Crespi Sent: Friday, June 28, 2019 7:49:51 AM To: Jonas Pfefferle; dev@crail.apache.org Subject: RE: Setting up storage class 1 and 2 Oh, and while I’m thinking about it Jonas, when I added the patches you provided the other day, I

Re: Setting up storage class 1 and 2

Hi David, At the moment, it is possible to add a NVMf datanode even if only the RDMA storage type is specified in the config. As you have seen this will go wrong as soon as a client tries to connect to the datanode. Make sure to start the RDMA datanode with the appropriate classname, see:

Re: Question on crail type-1 with terasort shuffle data

On Jun 26, 2019, at 09:17, Jonas Pfefferle wrote: There can only be 1 NQN in the whole system. For multiple storage classes just use different ip/port combinations with the same NQN. e.g. start a NVMf server on node 1 with storage class 1 with NQN X and start a server on node 2 with storage

Re: Question on crail type-1 with terasort shuffle data

with the namenode. So this is the correct way of using it, but it’s having a problem determining which controller to identify? Regards, David From: Jonas Pfefferle<mailto:peppe...@japf.ch> Sent: Wednesday, June 26, 2019 9:01 AM To: dev@crail.apache.org<mailto:dev@crail.a

Re: Question on crail type-1 with terasort shuffle data

. Regards, Jonas On Wed, 26 Jun 2019 09:17:40 +0200 "Jonas Pfefferle" wrote: Hi David, This is currently a limitation of the NVMf storage tier. You need to use the same NQN on all datanodes. The port can be different however. I will clarify this in the documentation. Rega

Re: Question on wildcard used with spark-default.conf

mentioned before, I’m not rally that familiar with java. Regards, David C: 714-476-2692 ____ From: Jonas Pfefferle Sent: Friday, June 21, 2019 12:47:34 AM To: David Crespi Subject: Re: Question on wildcard used with spark-default.conf That probably

Re: Crail-Spark Shuffle Manager config error

, the fix is shortly ready. Regards Adrian On 6/19/19 09:21, Jonas Pfefferle wrote: Hi David, I assume you are running with latest Crail master. We just pushed a change to the CrailConfiguration initialization which we have not adapted yet in the shuffle plugin (Should be a one line fix

Re: Cannot allocate memory error when using RDMA

Hi Jeongyoon, This looks like a user limit problem. Can you check if you have "max locked memory" set to a high enough value or unlimited (use "ulimit -l"). You can set memlock in /etc/security/limits.conf e.g.: *           soft    memlock        unlimited *           hard    memlock     

Re: Question on wildcard used with spark-default.conf

Hi David, the wildcard works for me without problem. However I do not use the environment variable. Can you try putting the actual Crail path instead of the env variable. Jonas On Wed, 19 Jun 2019 22:33:09 + David Crespi wrote: A question on what is documented on the crail website

Re: Crail-Spark Shuffle Manager config error

Hi David, I assume you are running with latest Crail master. We just pushed a change to the CrailConfiguration initialization which we have not adapted yet in the shuffle plugin (Should be a one line fix). @Adrian Can you take a look. Regards, Jonas On Tue, 18 Jun 2019 23:24:48 +

Source of Python -> C/C++ call overhead benchmark available

Hi @all I just put up the source code for the Python -> C/C++ call overhead blog, so everyone can try to reproduce the numbers in the blog: https://github.com/zrlio/Python-c-benchmark For those who have not read the blog post: http://crail.incubator.apache.org/blog/2019/01/python.html

Re: Unreleased maven artifacts

Let me take a look. Thanks, Jonas On Thu, 20 Dec 2018 12:56:33 -0800 Julian Hyde wrote: I was just in Nexus [1] and noticed that there are quite a few Crail repositories. I believe that they correspond to the failed release candidates (rc1, rc2 etc.) As release manager, be sure to drop

Re: [VOTE] Release of Apache Crail-1.1-incubating [rc6]

.0/jar On Nov 17, 2018, at 3:13 AM, Patrick Stuedi wrote: +1 + Compiles from source + Runs from binaries Thanks! -Patrick On Thu, Nov 15, 2018 at 5:16 PM Jonas Pfefferle wrote: Hi all For another round, we prepared a new release to address the issues found in rc5. This is a call t

[VOTE] Release of Apache Crail-1.1-incubating [rc5]

Hi all, We prepared a new release to address the issues found in rc4. This is a call for a vote on releasing Apache Crail 1.1-incubating, release candidate 5. The following issues has been addressed: * Add licenses of all dependencies * Update DiSNI and DaRPC version (they now also contain

Licenses of dependencies

Hi all, Please take a look at this pull request: https://github.com/apache/incubator-crail/pull/64 which adds licenses and copyright notices for all dependencies. If there are no comments we will merge the pull request and start another vote. Cheers, Jonas

[CANCEL][VOTE] Release of Apache Crail-1.1-incubating [rc4]

from the dependencies. > > On Mon, Nov 5, 2018 at 7:37 AM Jonas Pfefferle wrote: > > > > Hi all, > > > > > > We prepared a new release to address the issues found in rc3. This is a call > > for a vote on releasing Apache Crail 1.1-incubating, relea

Re: [VOTE] Release of Apache Crail-1.1-incubating [rc4]

Nevermind, looking at the other projects they don't have extra files for the binaries but just put everything in one NOTICE resp. LICENSE file... Any tools to help with this process or check that all licenses are covered? Thanks, Jonas On Tue, 06 Nov 2018 16:39:00 +0100 "Jonas Pfef

Re: [VOTE] Release of Apache Crail-1.1-incubating [rc4]

with the distribution. Here is a good example: https://svn.apache.org/repos/asf/tuscany/sca-java-2.x/trunk/distribution/all/src/main/release/bin/LICENSE You need to check what needs to be listed in the NOTICE file as well, from the dependencies. On Mon, Nov 5, 2018 at 7:37 AM Jonas Pfefferle wrote

[VOTE] Release of Apache Crail-1.1-incubating [rc4]

Hi all, We prepared a new release to address the issues found in rc3. This is a call for a vote on releasing Apache Crail 1.1-incubating, release candidate 4. The following issues has been addressed: * Append filename in checksum files to allow checking checksums with "sha512sum -c" * Add

[RESULT][VOTE] Release of Apache Crail-1.1-incubating [rc3]

From: "Jonas Pfefferle" To: "dev@crail.apache.org" Date: 10/26/2018 01:15 PM Subject:[VOTE] Release of Apache Crail-1.1-incubating [rc3] Hi all, We prepared a new release to address the issues found in rc2. This is a call for a vote on releasing Apache

[VOTE] Release of Apache Crail-1.1-incubating [rc3]

Hi all, We prepared a new release to address the issues found in rc2. This is a call for a vote on releasing Apache Crail 1.1-incubating, release candidate rc3. The following issues has been addressed: * add license to documentation and remove exclude from rat check * Improve build process

Jenkins access

@Mentors Can you please give me access to the Apache jenkins servers (https://builds.apache.org/). I want to setup some automated build tests. Here is a how to give access: https://cwiki.apache.org/confluence/display/INFRA/Jenkins Thanks, Jonas

[CANCEL][VOTE] Release of Apache Crail-1.1-incubating [rc2]

All, I'm canceling the vote for Apache Crail 1.1-incubating release candidate 2, due to found issues (see below). I will prepare a new release candidate. Regards, Jonas On Wed, 24 Oct 2018 09:26:34 +0200 "Jonas Pfefferle" wrote: Hi Julian, Comments inline. Thanks, Jonas

Re: [VOTE] Release of Apache Crail-1.1-incubating [rc2]

te! I built from source and installed and test-ran Crail both from built source and binary download. Looks all pretty good to me! +1 from my side Cheers, Bernard. On 10/22/2018 15:27, Jonas Pfefferle wrote: Hi all, This is a call for a vote on releasing Apache Crail 1.1-incubating, release candida

[VOTE] Release of Apache Crail-1.1-incubating [rc2]

Hi all, This is a call for a vote on releasing Apache Crail 1.1-incubating, release candidate rc2. The source and binary tarball, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/crail/1.1-rc2/ The commit to be voted upon:

[ANNOUNCE] Apache Crail 1.0-incubating released

The Apache Crail community is pleased to announce the first source release of Apache Crail version 1.0-incubating. Crail is a high-performance distributed data store designed for fast sharing of ephemeral data in distributed data processing workloads. You can read more about Crail on the

Re: Draft podling report available for review

db64fd. I think you should create a git tag for the exact commit that the release was based upon. Julian [1] https://github.com/apache/incubator-crail/releases/tag/v1.0 <https://github.com/apache/incubator-crail/releases/tag/v1.0> On Jun 7, 2018, at 12:49 AM, Jonas Pfefferle wrote:

Re: Draft podling report available for review

Hi Julian * We don't have a "Download" page yet because we had some discussion about the design (see Jira) but we will add this ASAP. * Regarding the download link in the news: I didn't want to link each version + the link to get the checksum files. I will change the link as soon as we have

Re: [VOTE] Crail v1.0-rc3 source release

h> wrote: +1 Thanks Jonas for addressing the issues from the last IPMC vote! I just built and ran Crail from the source tar ball https://dist.apache.org/repos/dist/dev/incubator/crail/1.0-rc3/apache-crail-1.0-rc3-incubating-src.tar.gz Thanks, Bernard. On 5/17/2018 9:31, Jonas Pfefferle wrote

Re: Release Next Steps, was Re: [VOTE] Crail v1.0-rc2 source release

ick On Mon, Apr 30, 2018 at 2:58 PM, Jonas Pfefferle <peppe...@japf.ch> wrote: I looked at the example of Toree. How can I stage artifacts at https://repository.apache.org/content/repositories ? I assume this is for maven central? Thanks, Jonas On Fri, 27 Apr 2018 12:03:49 -0700 Lucian

Re: Release Next Steps, was Re: [VOTE] Crail v1.0-rc2 source release

:57 AM, Jonas Pfefferle <peppe...@japf.ch> wrote: I'm closing the PPMC vote now: 5 + votes 0 - votes @Julian, Luciano How do we kick of the IPMC vote? Thanks Jonas On Thu, 26 Apr 2018 17:19:03 +0200 "Jonas Pfefferle" <peppe...@japf.ch> wrote: Thanks for the clarificatio

Re: [VOTE] Crail v1.0-rc2 source release

I'm closing the PPMC vote now: 5 + votes 0 - votes @Julian, Luciano How do we kick of the IPMC vote? Thanks Jonas On Thu, 26 Apr 2018 17:19:03 +0200 "Jonas Pfefferle" <peppe...@japf.ch> wrote: Thanks for the clarification. Let's wait until tomorrow to close the voti

Re: [VOTE] Crail v1.0-rc2 source release

el - did you by any chance found the time to have Crail RC2 evaluated..? Thanks very much for your support! Cheers, Bernard. On 4/26/2018 9:47, Jonas Pfefferle wrote: Hi Julian, Thanks for the vote. I will link the key into the web of trust thing and I just fixed the KEYS file (The "HOWTO&quo

Re: [VOTE] Crail v1.0-rc2 source release

to-wot> [2] https://dist.apache.org/repos/dist/release/incubator/crail/KEYS <https://dist.apache.org/repos/dist/release/incubator/crail/KEYS> On Apr 23, 2018, at 4:40 AM, Jonas Pfefferle <peppe...@japf.ch> wrote: Hi all, I fixed all the comments from rc1 and created rc2: * Add

[VOTE] Crail v1.0-rc2 source release

Hi all, I fixed all the comments from rc1 and created rc2: * Add all three source download options in README (@Julian requirements were already there) * Fix NOTICE file to include actual year for Apache Software Foundation * Fix signatures * Created KEYS file

[CANCEL][VOTE] Crail v1.0-rc1 source release

/dist/release/incubator/crail/KEYS. Might as well create it now. Will do. Julian On Fri, Apr 20, 2018 at 8:29 AM, bernard metzler <bmetz...@gmx.ch> wrote: +1 (non-binding) from me. I am convinced we fixed the issues from last RC. I wanted to keep the voting procedure for RC1 alive

Re: [VOTE] Crail v1.0-rc1 source release

14:38:06 +0200 "Jonas Pfefferle" <peppe...@japf.ch> wrote: Hi all, I fixed all the comments from rc0 and created rc1. Thanks to everyone who has contributed to this release. You can read the release notes here: https://github.com/apache/incubator-crail/blob/master/HISTOR

[VOTE] Crail v1.0-rc1 source release

Hi all, I fixed all the comments from rc0 and created rc1. Thanks to everyone who has contributed to this release. You can read the release notes here: https://github.com/apache/incubator-crail/blob/master/HISTORY.md The commit to be voted upon:

Re: [CANCEL] [VOTE] Crail v1.0-rc0 source release

All, I updated the pull request: * Removed the copyright notice and moved it to NOTICE file * Added license header to README.md and HISTORY.md I will create a new rc as soon as someone approves the changes. Thanks, Jonas On Mon, 16 Apr 2018 10:08:37 +0200 "Jonas Pfefferle" <pep

[CANCEL] [VOTE] Crail v1.0-rc0 source release

a release is viable, and may cancel the vote early if it is clear that another iteration is needed.) On Fri, Apr 13, 2018 at 8:55 AM, Luciano Resende <luckbr1...@gmail.com> wrote: On Fri, Apr 13, 2018 at 4:03 AM, Jonas Pfefferle <peppe...@japf.ch> wrote: Luciano Thanks for rev

Re: [VOTE] Crail v1.0-rc0 source release

to its fragilities. On Thu, Apr 12, 2018 at 4:33 AM, Jonas Pfefferle <peppe...@japf.ch> wrote: Hi all, I packaged the source and updated the history for our first source release. Thanks to everyone who has contributed to this release. You can read the release notes here: https://github.com/apac

[VOTE] Crail v1.0-rc0 source release

Hi all, I packaged the source and updated the history for our first source release. Thanks to everyone who has contributed to this release. You can read the release notes here: https://github.com/apache/incubator-crail/blob/master/HISTORY.md The commit to be voted upon:

Re: preparing for a first release

Hi Julian Thanks for the hints I will look into it. Jonas On Tue, 10 Apr 2018 16:46:26 -0700 Julian Hyde wrote: Reviewing the repo at a particular commit is a good first step. The next step will be to create a release candidate (a tar ball and checksum files and

Crail github not in sync

Hi @all, It looks like the apache git is not syncing with the github repository anymore. I still don't seem to have elevated access in JIRA and like Adrian I also have the problem that I cannot use my key to push to the Apache git. I verified that the key is correct and updated in the Apache

Key access git

Hi @all, Is there a way to use key access for the apache git? Thanks, Jonas

Re: Importing Crail repository, was :Re: incubator-crail git commit: Add license file

ip-us.apache.org/repos/asf/incubator-crail/diff/fff061f0 Branch: refs/heads/master Commit: fff061f09d1714b20425bf3a0cf602bf319556f0 Parents: Author: Jonas Pfefferle <j...@zurich.ibm.com> Authored: Wed Jan 17 11:42:12 2018 +0100 Committer: Jonas Pfefferle <j...@zurich.ibm.com> Committ

Re: Kicking off incubation

Hi @all, I'm Jonas Pfefferle. I work for IBM Research at the Zurich research lab. I started at IBM with RDMA virtualization work and since a while I have been working on Crail, e.g. adding NVMf support, performance optimizations, etc. I also have some other open source project involvement