t level for faster
>> scans and inserts, and making testing simpler and more consistent
>> would make things easier to test across C++ and Python.
>>
>> - Wes
>>
>> On Mon, Aug 28, 2017 at 2:22 AM, Adar Lieber-Dembo <a...@cloudera.com>
>> wrote:
&g
-1
I agree with Hao's assessment that KUDU-2131 should be considered a
blocker for 1.5. Below is some more color:
Before the fix for KUDU-1726, tablet copies fsynced more than
necessary but reliably kept their sessions alive by amortizing the
cost of the fsync in each downloaded block.
With the
Over the weekend I've been hacking on an idea that's been discussed in
the past: replacing our language-specific mini cluster support with
the more robust and featureful C++ mini cluster. Besides reducing the
maintenance burden for non-C++ language support, this has the added
benefit of providing
The Python patch wasn't as much as work as I thought, so I ended up
doing it myself: https://gerrit.cloudera.org/c/8236.
On Fri, Oct 6, 2017 at 3:53 PM, Adar Lieber-Dembo <a...@cloudera.com> wrote:
> To close the loop on this, I just merged a new action for the Kudu CLI
> which
Do you have an existing account in Apache JIRA? I looked for your
first name and your e-mail address but I couldn't find any accounts.
If you don't have an account yet, you can create one here:
https://issues.apache.org/jira/secure/Signup!default.jspa.
Once you have an account created, please
Thanks for volunteering, Grant.
Here are a couple of other patches to consider. None of them will
affect production use cases but improve the robustness of various test
or test-running scripts:
3d73405 (build-and-test.sh: unbreak Python 2.6 based builds): needed
for build-and-test.sh to pass on
I just merged a fix for KUDU-1961, which enables ccache in conjunction
with devtoolset-3 when building in RHEL and CentOS 6 environments. I
don't expect it to cause any problems, but let me know if you
encounter unusual build failures on el6.
Thanks for starting a discussion on this topic.
> There've been a few discussions recently regarding changes to Kudu's
> metadata storage. There are a number of areas that could benefit from
> improving this layer, and I've been coalescing some of these ideas to lock
> down what changes make
That's usually a sign that download-thirdparty.sh (invoked by
build-if-necessary.sh) was interrupted the first time it ran, perhaps
via Ctrl-C. Run 'git clean -xdf thirdparty' to clear out your
thirdparty build output and try again.
If the issue persists, please reply and attach the full output
-1, because commit 038c3eb has messed up deadline-based waiting on
macOS. I don't know the full extent of the breakage but Will and
Alexey reported seeing abnormally high CPU usage in Kudu servers.
I published a fix in http://gerrit.cloudera.org:8080/9695 and, once it
is merged to master, I
> I do think the 'NTP in kudu' could help a bit here, especially if it were
> only used as a "backup" in case the kernel is unsynchronized. I'm a little
> nervous about the impact on NTP servers, though, in our minicluster based
> tests where we might start and stop tens of thousands of times in
The clock sync errors do seem to have increased over the past few
months. If we could just fix those, I think we'd be left with almost
entirely "known" flakies. Any ideas as to what's going on? Think it's
something that could be addressed with your NTP-client-in-Kudu patch
series?
On Fri, Mar 23,
+1
- Built in TSAN mode on Ubuntu 16.04
- Ran tests. Most passed except master-test (filed KUDU-2361). Then my
computer crashed, taking down the 1.7 build output (which was in /tmp)
with it.
On Tue, Mar 20, 2018 at 10:09 AM, William Berkeley wrote:
> +1
>
> - Built in
+1
Built and ran DEBUG C++ tests in slow mode on CentOS 6.6. All passed.
Built and ran Java tests using DEBUG Kudu binaries on CentOS 6.6. All passed.
Reviewed the release notes.
On Mon, Oct 22, 2018 at 5:17 PM Dan Burkert wrote:
>
> +1
>
> Built and ran C++ tests on MacOS. Had three failures
Thanks for doing this, Attila.
I've added release notes for my patches, Todd's patches (since he's
currently away), and for all Kudu contributors who aren't committers.
On Wed, Oct 3, 2018 at 2:07 PM Attila Bukor wrote:
>
> Hi Devs,
>
> I've created a Google doc to collect the release notes:
>
Hi Zoltan,
I'm not feeling brave enough to continue cc'ing a bunch of other
Apache projects just yet, but I did leave a comment in your gdoc
describing what Kudu currently supports and what we could offer in the
future.
It wasn't clear if you wanted to carry out this discussion over gdoc
+1
Thanks for volunteering to RM, Andrew.
On Tue, Jan 22, 2019 at 9:35 PM Andrew Wong wrote:
>
> Hello Kudu developers!
>
> It's been just around three months since we released 1.8.0. In that time,
> we've accrued some important improvements and bug fixes (around 201 commits
> since Kudu
Do we need to update any configuration in gerrit.cloudera.org or
jenkins.kudu.apache.org?
On Thu, Jan 3, 2019 at 6:18 AM Attila Bukor wrote:
> Hi,
>
> I submitted 2 patches to Gerrit to change our tooling to use the gitbox
> URLs:
>
> - https://gerrit.cloudera.org/c/12150/
> -
I just merged a patch to master that drops JDK 7 support from Kudu's
Java artifacts (i.e. the Java client, Spark bindings, etc.). The
oldest supported JDK version is now JDK 8.
JDK 7 support has been deprecated since Kudu 1.5.0 (about a year and a
half ago) so there's been ample notice that this
I merged a series of patches that will cause Java test failures to be
reported to the flaky test dashboard
(http://dist-test.cloudera.org:8080). Over the next week or so we
should start to see failures percolate in and learn more about which
tests are flaky.
Of note: we used to retry every Java
A recent Slack discussion revealed that we may not have consensus on
how our project handles backports to older branches. This e-mail is an
attempt to build that consensus; please weigh in if you have an
opinion.
Perhaps it's best to start from first principles: when release x.y is
the latest
+1
I ran the C++ tests (DEBUG build) in slow mode on Ubuntu 18, CentOS
6.6, and CentOS 7.3. I saw a couple failures but I'm pretty sure
they're just flakes; filed KUDU-2718 and KUDU-2719 for them.
On Wed, Feb 27, 2019 at 1:51 AM Andrew Wong wrote:
>
> Hello Kudu devs!
>
> As suggested on the
+1
I ran all C++ tests with KUDU_ALLOW_SLOW_TESTS=1 in DEBUG mode on
Ubuntu 18.04 and CentOS 6.6. All passed except for a known Ubuntu 18
failure (a variant of KUDU-2641).
On Thu, Feb 21, 2019 at 5:54 PM helifu wrote:
>
> +1
> * All C++ tests passed in debug/release/tsan mode on Debian8.9,
> > I'm an Impala dev working on replacing Thrift with krpc. One issue that
> > recently came up is that we would like to have a simple way of simulating
> > different types of failures of rpcs for testing purposes, and I was
> > wondering if krpc already has anything like this built in, or if
build), with and without my patch.
>
> ________
> From: Adar Lieber-Dembo
> Sent: 14 April 2019 19:59:45
> To: dev@kudu.apache.org
> Subject: Re: java unit tests failing when starting MiniKuduCluster
>
> It might be related to your local changes. This caught
Congrats, Yingchun! Thank you for all of your hard work on Kudu.
On Wed, Jun 5, 2019 at 11:25 AM Todd Lipcon wrote:
>
> Hi Kudu community,
>
> I'm happy to announce that the Kudu PMC has voted to add Yingchun Lai as a
> new committer and PMC member.
>
> Yingchun has been contributing to Kudu for
It might be related to your local changes. This caught my eye:
extra_master_flags: "--rpc_max_message_size=3860733440"
extra_tserver_flags: "--rpc_max_message_size=3860733440"
Why did you have to add this configuration? What happens if you remove
it? On a related note, do the tests still
Today I finished merging a patch series that switches on dist-test
when running tests that power the flaky test dashboard (previously
they were run on a CentOS 6.6 VM). This wasn't done because we need
these tests to finish faster; it's because an increasing number of
tests are flaky in one
(+user, -dev, as this is more appropriate for the users list)
The Kudu master currently keeps a record of all tables and partitions,
including those that have been deleted. With a high enough rate of
table deletion it's theoretically possible for that to consume a lot
of disk space or memory. In
Hi Kudu community,
I'm happy to announce that the Kudu PMC has voted to add Lifu He, Yao
Xu, and Yao Zhang as new committers and PMC members.
Lifu has worked on a variety of patches, from dead container deletion
in the block manager, to metric aggregation in the master, to several
performance
+1
On Tue, Oct 1, 2019 at 11:44 AM Alexey Serbin
wrote:
>
> Hello Kudu developers!
>
> It's been a bit more than three months since we released 1.10.0. In that
> time,
> we've accrued some important improvements and bug fixes (around 189 commits
> since Kudu 1.10.0). By our usual three-month
My two cents:
- The presence of 1.11.0 on the download page means that 1.11.0 has
been de facto released, announcement or no announcement. The
announcement doesn't add any additional hurt, so I think we should
move forward with it.
- Separately, let's also announce the licensing issue and say that
Hi devs,
The Apache Impala project has this neat integration set up between gitbox
and JIRA where if someone pushes a new commit to gitbox and that commit has
a JIRA associated with it, the push generates a JIRA comment describing the
commit that was just pushed. You can see an example of this
:
> > > >>
> > > >> SGTM. My first thought was that the issues mailing list might get more
> > > >> traffic, but I'm ok with that, since contributors today often try to
> > > post
> > > >> an equivalent comment anyways.
UDU-2990 and the rest of notes for
> 1.11.0 and 1.10.0 correspondingly?
>
>
> Thanks,
>
> Alexey
>
> On Thu, Nov 7, 2019 at 5:07 AM Adar Lieber-Dembo
> wrote:
>
> > KUDU-2990 has been fixed on master, so 1.12.0 will be conformant when
> > it is released in s
removing it from Kudu.
> >>
> >>
> >> I am okay with that. I will post a patch to the project and share it
> >> here.
> >>
> >> On Wed, Sep 18, 2019 at 2:05 PM Adar Lieber-Dembo
> >> wrote:
> >>
> >>>
+1
On Ubuntu 18.04 I built RELEASE with NO_TESTS=1 and verified that
there were no numa or memkind dependencies or symbols in any binaries.
I also grepped for numa and memkind across the codebase and didn't see
any unexpected dependencies.
I did see memkind erroneously listed in
+1
I ran DEBUG tests in slow mode on Ubuntu 18.04 and CentOS 6.6. All passed.
On Sat, Nov 16, 2019 at 1:38 PM Attila Bukor wrote:
>
> +1
>
> - Verified checksum, the signature and that the archive was created from the
> tag
> - Successfully built Kudu in release mode on CentOS 7.3 and macOS
+1
On Ubuntu 18.04 I built RELEASE with NO_TESTS=1 and verified that
there were no numa or memkind dependencies or symbols in any binaries.
I also grepped for numa and memkind across the codebase and didn't see
any unexpected dependencies.
Also on Ubuntu 18.04 I built DEBUG and verified that all
u.apache.org/docs/known_issues.html#_other_known_issues
> >
> >
> > Best regards,
> >
> > Alexey
> >
> > On Fri, Nov 1, 2019 at 4:20 PM Adar Lieber-Dembo > >
> > wrote:
> >
> > > My two cents:
> > > - The presence of 1
o me.
> >
> > I don't feel particularly strongly for this approach vs the one laid out by
> > Alexey, but thought I'd bring it up since in general it seems like external
> > mini cluster tests are meant to simulate as real of an environment as
> > possible (modulo a
+1
I ran all tests in slow DEBUG mode on CentOS 6.6 and Ubuntu 18.04. All
of them passed.
I'm surprised that live row count still has issues given the late
breaking fixes that Yingchun Lai merged into 1.11.0. His last fix was
expressly intended to address the problem where a table with a mix of
-1
I built and ran all DEBUG tests in slow mode on Ubuntu 18.04 and
CentOS 6.6. Notable failures included:
- memory_gc-itest: looks like it's broken for most DEBUG builds, even
after Yingchun's deflake patch
(http://dist-test.cloudera.org:8080/test_drilldown?test_name=memory_gc-itest).
-
+0
Built on Ubuntu 18.04 and CentOS 6.6. Ran DEBUG tests in slow mode. No failures.
I do wish the release incorporated the fix for KUDU-2980, but that
hasn't even been published to CR yet. Plus it won't break backwards
compatibility if we fix it later.
On the other hand,
Thanks for the detailed summary.
I agree with the goal of using the same time source across all tests
if possible. Originally, I was hoping this would be 'builtin', but
that'd require deploying MiniChronyd too, both to avoid a dependency
on Internet connectivity and to ensure that clock
Hi Kudu community,
I'm happy to announce that the Kudu PMC has voted to add Yifan Zhang
as a new committer and PMC member.
Yifan's contributions have included:
- Filtering tables and tablets in ksck.
- Tooling to remove or alter columns.
- Adding a mean gauge to the metrics subsystem.
- The
nd code.
> >- This can be done after the next release.
> > - Though the tests are removed and we use Hive 3 we still technically
> > work with Hive 2 and Sentry. We are no longer testing/validating
> > it though.
> >
> > Please let me know if you
I just merged some changes that modify include-what-you-use (IWYU) to
run against the libc++ in thirdparty rather than against the system's
libstdc++. Coupled with some other changes, I expect that running IWYU
on any Linux system should yield the exact same set of recommendations
as when IWYU is
I share the two concerns you highlighted, Alexey.
1. This would be a backwards incompatible change.
2. The default NTP server list could be unreachable, or could be a
poorer choice than whatever the cluster is currently using. Grant's
suggestion could mitigate that somewhat, but it's sort of weird
49 matches
Mail list logo