[
https://issues.apache.org/jira/browse/SOLR-14660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484938#comment-17484938
]
ASF subversion and git services commented on SOLR-14660:
--------------------------------------------------------
Commit f2cedab7ecd8e6501fb63e285d857f73016911e1 in solr's branch
refs/heads/branch_9x from Istvan Farkas
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=f2cedab ]
SOLR-14660: Move HDFS support to a new HDFS module
This commit
- moves the code implementing HDFS support to a new HDFS module which has its
own build and tests.
- moves block caching implementation to the HDFS module
- removes direct references from Solr Core to HDFS module
- moves test classes which are required both by Core and the HDFS module to the
test framework
- updates the documentation link to refer the HdfsDirectoryFactory from the new
location
- removes the deprecated flags from HDFS classes
- removes the deprecated notice from the reference guide
After ensuring that the module is on the classpath, HDFS support can be used as
before,
no changes need to be done to the configuration files.
Some specific changes
- Remove DirectoryFactory.LOCK_TYPE_HDFS
- updateHandler will use the update log instance returned by the directory
factory if no class is specified in solrconfig.xml.
- test resource files are now copied from core upon build to avoid having to
maintain 2 set of the same files in git.
- HdfsCollectionsAPIDistributedZkTest extends CollectionsAPIDistributedZkTest -
introduced AbstractCollectionsAPIDistributedZkTestBase
- HdfsUnloadDistributedZkTest extends UnloadDistributedZkTest - introduced
AbstractUnloadDistributedZkTestBase
- HdfsTlogReplayBufferedWhileIndexingTest extends
TlogReplayBufferedWhileIndexingTest - introduced
AbstractTlogReplayBufferedWhileIndexingTestBase
- HDFSRestartWhileUpdatingTest extends RestartWhileUpdatingTest - introduced
AbstractRestartWhileUpdatingTestBase
- MoveReplicaHDFSTest extends MoveReplicaTest - introduced
AbstractMoveReplicaTestBase
- HdfsRecoveryZkTest extends RecoveryZkTest - introduced
AbstractRecoveryZkTestBase
- HdfsBasicDistributedZk2Test extends BasicDistributedZk2Test - separated by
introducing AbstractBasicDistributedZk2TestBase
- HdfsChaosMonkeySafeLeaderTest extends ChaosMonkeySafeLeaderTest - separated
by introducing AbstractChaosMonkeySafeLeaderTestBase
- HdfsNNFailoverTest extends BasicDistributedZkTest. Separated it to
AbstractBasicDistributedZkTestBase and BasicDistributedZkTest.
- Also changed the following classes to extend
AbstractBasicDistributedZkTestBase:
- HdfsBasicDistributedZkTest
- HdfsWriteToMultipleCollectionsTest
- StressHdfsTest
- TestHdfsBackupRestoreCore uses BackupRestoreUtils and BackupStatusChecker,
these were moved to test framework (these also use log4j classes)
- Moved the fetchRestoreStatus method out of TestRestoreCore to a util class as
it is shared by hdfs tests and core tests
- HdfsSyncSliceTest was extending SyncSliceTest
- HdfsChaosMonkeyNothingIsSafeTest was extending
ChaosMonkeyNothingIsSafeTest.java, however since there is no depenency possible
between test projects, it had to be split up
- Created AbstractChaosMonkeyNotheingIsSafeTestBase as a common ancestor. Class
is placed to the test framework because it has junit depenencies and needs to
be shared
- Moved the login in the test() method to doTest() which is now invoked from
the child classes
- Moved FullThrottleStoppableIndexingThread to the test framework as it needs
to be shared
Closes PR #324
Co-authored-by: Istvan Farkas <[email protected]>
Co-authored-by: Kevin Risden <[email protected]>
> Migrating HDFS into a module
> ----------------------------
>
> Key: SOLR-14660
> URL: https://issues.apache.org/jira/browse/SOLR-14660
> Project: Solr
> Issue Type: Improvement
> Components: Hadoop Integration, Plugin system
> Reporter: Ishan Chattopadhyaya
> Assignee: Kevin Risden
> Priority: Blocker
> Labels: package, packagemanager
> Fix For: 9.0
>
> Time Spent: 11h 20m
> Remaining Estimate: 0h
>
> Following up on the deprecation of HDFS (SOLR-14021), we need to work on
> isolating it away from Solr core and making a package for this. This issue is
> to track the efforts for that.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
