Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-08 Thread Nihal Jain
I have created sub tasks with necessary details in the umbrella jira. Will take them up in coming days. Also will add more sub tasks later if needed. Regards Nihal On Sat, 9 Mar 2024, 11:53 Istvan Toth, wrote: > Thank you Nihal. > I'm not very familiar with the tools in the test code, so you

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-08 Thread Istvan Toth
Thank you Nihal. I'm not very familiar with the tools in the test code, so you can probably plan that work better. I just have some generic steps in mind: * Identify all the tools / scripts in the test jars * Identify and analyze their dependencies (compared to the current runtime deps) * Decide

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-08 Thread Nihal Jain
Sure I will be able to take up. Please create tasks with necessary details or let me know if you want me to create. On Fri, 8 Mar 2024, 12:45 Istvan Toth, wrote: > Thanks for volunteering, Nihal. > > I could work on the Hadoop-less, and assemblies, and you could work on > cleaning up the test

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-07 Thread Istvan Toth
Thanks for volunteering, Nihal. I could work on the Hadoop-less, and assemblies, and you could work on cleaning up the test jars. Would that work for you ? I know that I'm picking the smaller part, but it turns out that I won't have as much time to work on this as I hoped. (Unless there are

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-06 Thread Istvan Toth
We seem to be in agreement in principle, however the devil is in the details. The first step should be moving the diagnostic tools out of the test jars. Are there any tools we don't want to move out ? Do the diagnostic tools pull in extra dependencies compared to the current runtime JARs, and if

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Bryan Beaudreault
I'm +0 on hbase-examples, but +100 on any improvements we can make to ltt/pe/chaos/minicluster/etc. It's extremely frustrating how much reliance we have on test jars both generally but also specifically around these core test executables. Unfortunately I haven't had time to dedicate to these

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Nihal Jain
Thank you for bringing this up. +1 for this change. In fact, some time back, we had faced similar problem. Security scans found that we were bundling some vulnerable hadoop test jar. To deal with that we had to make a change in our internal HBase fork to exclude all HBase and Hadoop test jars

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Nick Dimiduk
This would be great cleanup, big +1 from me for all three of these adjustments, including the promotion of pe, ltt, and friends out of the test scope. I believe that we included hbase test jars because we used to freely mix classes needed for minicluster between runtime and test jars, which in

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Istvan Toth
I agree, we don't want to omit those from the binary distro. We should identify what those tools are. (Should be easy based on the presence of main() or the Tool interface) Such tools could either be moved into a new module, like hbase-tools, or simply moved to the runtime JARs. Istvan On Tue,

Re: [DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Duo Zhang
There are some tools in the tests jar, such as PerformanceEvaluation. But anyway, maybe they should be moved to main... Istvan Toth 于2024年3月5日周二 16:14写道: > > DISCLAIMER: I don't have a patch ready, or even an elegant way mapped out > to achieve this, this is about discussing whether we even

[DISCUSS] Removing tests and/or Hadoop from the binary assemblies

2024-03-05 Thread Istvan Toth
DISCLAIMER: I don't have a patch ready, or even an elegant way mapped out to achieve this, this is about discussing whether we even want to make these changes. These are also substantial changes, but they could be targeted for HBase 3.0. One issue I have noticed is that we ship test jars and test