Andrew Kyle Purtell created PHOENIX-8039:
--------------------------------------------
Summary: Vector index test harness docker cluster
Key: PHOENIX-8039
URL: https://issues.apache.org/jira/browse/PHOENIX-8039
Project: Phoenix
Issue Type: Sub-task
Reporter: Andrew Kyle Purtell
Assignee: Andrew Kyle Purtell
The vector harness is a client framework responsible for ingesting large scale
vector corpora into Phoenix tables, constructing secondary vector indexes,
computing brute force ground truth, executing multidimensional query and
mutation sweeps, introducing operational disruptions, and recording execution
manifests alongside metric time series. The harness exercises the two index
architectures implemented as PHOENIX-7998 and PHOENIX-8009.
For developer convenience in testing these features in a distributed topology
(albeit single host), this work provides a Docker-based distributed cluster
topology:
- A coordination and storage tier consisting of a standalone ZooKeeper daemon,
an HDFS NameNode, a SecondaryNameNode, and three independent DataNodes.
- A distributed compute tier featuring a YARN ResourceManager, three YARN
NodeManagers, and a dedicated MapReduce JobHistoryServer.
- A database tier consisting of an HBase Master and three RegionServers hosting
Phoenix coprocessors.
- Deterministic initialization, readiness, and health check orchestration.
- Client containers providing environments for harness workload phases and
administrative maintenance utilities, including IndexFsckTool, IndexTool, the
HBase shell, and Hadoop command line interfaces.
All containers execute within the Docker virtual machine network namespace,
mapping all network listeners to localhost while isolating daemon instances via
deterministic port allocations.
---
Design doc: https://gist.github.com/apurtell/5b038f7e8a89b6d6c6dbc4b73c564e07
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
