[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3100 ---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r232267315
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
@patricker sorry - thanks for pinging! We don't know that we are trying to
avoid `@Ignore`d tests in general necessarily. What we are trying to avoid is
marking tests as `@Ignore` as a "solution" to tests that intermittently fail.
Instead, those tests need to be fixed or deleted. Having a test that is
intended to be run manually to test performance before & after a change is a
common pattern that we follow throughout the codebase. Some people also have
marked these types of tests as Integration Tests instead of unit tests, but I
think that's a bad idea, as it's not truly an integration test. I'm sure there
are better ways to handle this, but for the time being I would assert that it
is common practice and don't believe it's unfavorable.
re: the presence of the RepeatingInputStream. I do understand your
temptation to remove it. The reason that I suggest we keep it, though, is that
there have been many times that it would have been helpful but was not
available so I used only a small bit of data in tests. I do think there are
others tests in the codebase that could be updated to use this, as well. I'm
not hell-bent on this, though, so if you still feel it should be removed, say
the word and i'll push a commit that removes it.
Thanks for the review!
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r231534519
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
@markap14 Thoughts?
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r229750511
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
It would probably be best to remove it from this specific PR and include it
in a future one when needed.
Generally, aren't we trying to avoid more `@ignore` tests? Otherwise I'd
say just include your previous tests and call it good.
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r229685801
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
I am however tempted to leave the class in, as I've considered writing such
a thing a few times for measuring performance of some piece of code.
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r229685528
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
Interesting. I wrote this to use in some unit tests so that I could easily
test performance, running over several GB of data. The unit tests don't
reference it now... I intended to mark those test as @Ignore, indicating that
they were only useful for manual testing before/after changes to determine the
performance. But apparently I removed the tests all together instead of
@Ignore'ing them.
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
Github user patricker commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3100#discussion_r229500969
--- Diff:
nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/RepeatingInputStream.java
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.stream.io;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Objects;
+
+public class RepeatingInputStream extends InputStream {
--- End diff --
Where is this class being used? I can't find any reference to it.
---
[GitHub] nifi pull request #3100: NIFI-5718: Implemented LineDemarcator and removed N...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3100 NIFI-5718: Implemented LineDemarcator and removed NLKBufferedReader i⦠â¦n order to improve performance Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5718 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3100.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3100 ---
