[
https://issues.apache.org/jira/browse/FLINK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Felix seibert updated FLINK-12550:
----------------------------------
Description:
LocatableInputSplitAssigner (in package api.common.io) fails to assign local
input splits to hosts whose hostname contains a dot ("."). To reproduce add the
following test to LocatableSplitAssignerTest and execute it. It will always
fail. In my mind, this is contrary to the expected behaviour, which is that the
host should obtain the one split that is stored on the very same machine.
{code:java}
@Test
public void testLocalSplitAssignmentForHostWithDomainName() {
try {
String hostNameWithDot = "testhost.testdomain";
// load one split
Set<LocatableInputSplit> splits = new HashSet<LocatableInputSplit>();
splits.add(new LocatableInputSplit(0, hostNameWithDot));
// get next split for the host
LocatableInputSplitAssigner ia = new LocatableInputSplitAssigner(splits);
InputSplit is = null;
ia.getNextInputSplit(hostNameWithDot, 0);
// there should be exactly zero remote and one local assignment
assertEquals(0, ia.getNumberOfRemoteAssignments());
assertEquals(1, ia.getNumberOfLocalAssignments());
}
catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
{code}
I also experienced this error in practice, and will later today open a pull
request to fix it.
Note: I'm not sure if I selected the correct component category.
was:
LocatableInputSplitAssigner (in package api.common.io) fails to assign local
input splits to hosts whose hostname contains a dot ("."). To reproduce add the
following test to LocatableSplitAssignerTest and execute it. It will always
fail. In my mind, this is contrary to the expected behaviour, which is that the
host should obtain the one split that is stored on the very same machine.
{code:java}
@Test
public void testLocalSplitAssignmentForHostWithDomainName() {
try {
String hostNameWithDot = "testhost.testdomain";
// load one split
Set<LocatableInputSplit> splits = new HashSet<LocatableInputSplit>();
splits.add(new LocatableInputSplit(0, hostNameWithDot));
// get all available splits
LocatableInputSplitAssigner ia = new LocatableInputSplitAssigner(splits);
InputSplit is = null;
ia.getNextInputSplit(hostNameWithDot, 0);
assertEquals(0, ia.getNumberOfRemoteAssignments());
assertEquals(1, ia.getNumberOfLocalAssignments());
}
catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
{code}
I also experienced this error in practice, and will later today open a pull
request to fix it.
Note: I'm not sure if I selected the correct component category.
> hostnames with a dot never receive local input splits
> -----------------------------------------------------
>
> Key: FLINK-12550
> URL: https://issues.apache.org/jira/browse/FLINK-12550
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
> Affects Versions: 1.8.0
> Reporter: Felix seibert
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> LocatableInputSplitAssigner (in package api.common.io) fails to assign local
> input splits to hosts whose hostname contains a dot ("."). To reproduce add
> the following test to LocatableSplitAssignerTest and execute it. It will
> always fail. In my mind, this is contrary to the expected behaviour, which is
> that the host should obtain the one split that is stored on the very same
> machine.
>
> {code:java}
> @Test
> public void testLocalSplitAssignmentForHostWithDomainName() {
> try {
> String hostNameWithDot = "testhost.testdomain";
> // load one split
> Set<LocatableInputSplit> splits = new HashSet<LocatableInputSplit>();
> splits.add(new LocatableInputSplit(0, hostNameWithDot));
> // get next split for the host
> LocatableInputSplitAssigner ia = new
> LocatableInputSplitAssigner(splits);
> InputSplit is = null;
> ia.getNextInputSplit(hostNameWithDot, 0);
> // there should be exactly zero remote and one local assignment
> assertEquals(0, ia.getNumberOfRemoteAssignments());
> assertEquals(1, ia.getNumberOfLocalAssignments());
> }
> catch (Exception e) {
> e.printStackTrace();
> fail(e.getMessage());
> }
> }
> {code}
> I also experienced this error in practice, and will later today open a pull
> request to fix it.
>
> Note: I'm not sure if I selected the correct component category.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
