Kasia Kucharczyk created BEAM-4413:
--------------------------------------
Summary: Create infrastructure configuration (docker/k8s) for
HCatalogIOIT
Key: BEAM-4413
URL: https://issues.apache.org/jira/browse/BEAM-4413
Project: Beam
Issue Type: Test
Components: testing
Reporter: Kasia Kucharczyk
Assignee: Jason Kuster
Currently the HCatalogIOIT test is working +only+ with provided infrastructure
(e.g. DataProc). It is caused by lack of docker image which would meet test
requirements.
* First of all some of Hive images are custom made and does not contain
HCatalog.
* Secondly some of images are problematic in terms of network configuration.
It was impossible to make all components visible for each other (datanodes,
namenodes, etc.), what was observed with following error:
{code:java}
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File could only be
replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s)
running and 1 node(s) are excluded in this operation.{code}
Here is related stackoverflow issue:
[https://stackoverflow.com/questions/26743656/hadoop-2-5-0-failure-to-write-file-remotely]
For a moment there is no solution.
What seems to be a next reasonable point to check here is creating own docker
image, publish it and test it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
