Sounds good, Mark. On Mon, Dec 7, 2015 at 3:11 PM, Mark Payne <[email protected]> wrote:
> I think the correct way to move forward is to go ahead and keep the file > separator, > as it is consistent with how the GetFile processor works and probably most > intuitive > for windows users. However, I do think that using a forward-slash instead > does provide > benefit as it is portable across operating systems. > > At this time, we cannot really change GetFile's behavior, though, as it > could certainly break > existing flows that depend on the existing style of slashes. > > With the given reviews around the ticket indicating that builds are > successful now on OS X, > Linux, and Windows and given that this is basically the only ticket > preventing > the 0.4.0 release from occurring, I would like to go ahead and merge in > the patch, and we should open > up a bigger discussion for "How should NiFi handle path separators" for > the 1.0.0 release, where we have > more flexibility in changing these things since it is a major release. > > Thanks > -Mark > > > > On Dec 7, 2015, at 11:32 AM, Tony Kurc <[email protected]> wrote: > > > > Probably want a function to go from dos to unix filenames. That replace > is > > a little dangerous. Although is this seriously a use case? > > On Dec 7, 2015 10:44 AM, "Mark Payne" <[email protected]> wrote: > > > >> Mike, > >> > >> That is accurate that Linux will not recognize \ as a path separator. It > >> would have no > >> problem reading the attribute but would not be able to write to a > >> directory/file with > >> that name. If you wanted to use that as a path in linux you could > >> certainly reference it > >> via: > >> > >> ${path:replace("\\", "/")} > >> > >> However, this problem already exists today with GetFile - it is using > the > >> File.separator > >> so that if you have a path like "subDir1/subDir2/myFile.txt" in Windows, > >> the path attribute > >> will be "subDir1\subDir2/" with both a forward slash and a backslash. > I'm > >> suggesting that > >> with ListFile instead of using "subDir1\subDir2/" as the attribute it > >> should be "subDir1\subDir2\" > >> > >> Thanks > >> -Mark > >> > >> > >>> On Dec 7, 2015, at 10:30 AM, Michael Moser <[email protected]> wrote: > >>> > >>> I disagree Mark, mainly due to the flowfile attributes such as 'path' > >> that > >>> processors like GetFile (and I assume ListFile) create. Windows > supports > >>> the forward slash but Unixes do not support the back slash > >> File.separator. > >>> What happens when a flowfile created on a Windows NiFi is sent over to > a > >>> Unix NiFi? Do Expression Language expressions that expect a forward > >> slash > >>> begin to fail when reading a 'path' attribute that contains > backslashes? > >>> I'm concerned ... > >>> > >>> -- Mike > >>> > >>> > >>> On Mon, Dec 7, 2015 at 10:12 AM, Mark Payne <[email protected]> > >> wrote: > >>> > >>>> Errr, actually, after thinking about it more - operators who are > running > >>>> on Windows should probably expect \ to be used instead of / > >>>> > >>>> So I think the better solution is to get rid of the "/" anywhere in > the > >>>> processor and tests and always use File.separator. > >>>> > >>>> Will do so and create a patch, if nobody objects to that. > >>>> > >>>> Thanks > >>>> -Mark > >>>> > >>>> > >>>>> On Dec 7, 2015, at 9:59 AM, Mark Payne <[email protected]> wrote: > >>>>> > >>>>> Tony, > >>>>> > >>>>> I think there are really two possible solutions to this: > >>>>> > >>>>> 1) Always use / in the path attributes instead of \ -- i generally > >>>> prefer this approach, as windows has worked with forward slashes since > >> Win > >>>> 98 (I believe?). > >>>>> 2) Have unit test look for file.separator -- benefit here is that it > is > >>>> consistent with the way that GetFile works, and I'd not want to change > >> that > >>>> because it's quite likely that some people are routing based on the > >> 'path' > >>>> attribute. > >>>>> > >>>>> Normally I would tend to make consistency a high priority. However, I > >>>> see ListFile / FetchFile largely as a replacement for GetFile and am > >>>> guessing that in the future GetFile will be deprecated and removed. So > >> I am > >>>> less inclined to stay consistent between the 'old generation' and 'new > >>>> generation' of processors. So personally I'd prefer to go the first > >> route. > >>>>> > >>>>> -Mark > >>>>> > >>>>> > >>>>>> On Dec 7, 2015, at 12:35 AM, Tony Kurc <[email protected]> wrote: > >>>>>> > >>>>>> I submitted a patch to get the test to pass (NIFI-1261). Seems a bit > >>>> icky, > >>>>>> but I'll defer to Joe Skora and Mark Payne for correct behavior. > >>>>>> > >>>>>> On Mon, Dec 7, 2015 at 12:11 AM, Tony Kurc <[email protected]> > wrote: > >>>>>> > >>>>>>> Joe - I'm putting a ticket in for a fix. Looks like it was > introduced > >>>> by > >>>>>>> the NIFI-1246 patch. > >>>>>>> > >>>>>>> On Sun, Dec 6, 2015 at 11:36 PM, Joe Percivall < > >>>>>>> [email protected]> wrote: > >>>>>>> > >>>>>>>> Yup I saw the same behavior. > >>>>>>>> > >>>>>>>> On the second try (doing mvn clean install -rf > >>>> :nifi-standard-processors) > >>>>>>>> the tailfile error went away. The listFile error still occurred > >>>> though. > >>>>>>>> > >>>>>>>> Joe > >>>>>>>> - - - - - - > >>>>>>>> Joseph Percivall > >>>>>>>> linkedin.com/in/Percivall > >>>>>>>> e: [email protected] > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Sunday, December 6, 2015 11:32 PM, Tony Kurc <[email protected] > > > >>>> wrote: > >>>>>>>> Er, just the tailfile error > >>>>>>>> > >>>>>>>> On Dec 6, 2015 11:31 PM, "Tony Kurc" <[email protected]> wrote: > >>>>>>>> > >>>>>>>>> Joe, I had this happen and it worked on a second try. > >>>>>>>>> On Dec 6, 2015 11:23 PM, "Joe Percivall" > >>>> <[email protected] > >>>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Windows 8 build fails with maven 3.3.3 and Java 1.8.0_65. > >>>>>>>>>> > >>>>>>>>>> I get these error messages: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> TestListFile.testRecurse:441 expected:<subdir1[/]subdir2/> but > >>>>>>>>>> was:<subdir1[\]subdir2/> > >>>>>>>>>> > >>>>>>>> > >>>> > >> > TestTailFile.testMultipleRolloversAfterHavingReadAllDataWhileStillRunning:381 > >>>>>>>>>> expected:<[world]> but was:<[abc > >>>>>>>>>> > >>>>>>>>>> These were not any of the same errors I saw last time testing on > >>>>>>>> Windows > >>>>>>>>>> a couple weeks ago. > >>>>>>>>>> > >>>>>>>>>> Joe > >>>>>>>>>> - - - - - - > >>>>>>>>>> Joseph Percivall > >>>>>>>>>> linkedin.com/in/Percivall > >>>>>>>>>> e: [email protected] > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Sunday, December 6, 2015 9:05 PM, Tony Kurc < > [email protected]> > >>>>>>>> wrote: > >>>>>>>>>> I've gotten confirmation of CentOS 7.1.1503 x86_64, Oracle JDK > >> 8u66 > >>>>>>>>>> working > >>>>>>>>>> fine. and Fedora 23 not working with the same error that Andre > >>>>>>>> reported. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Sun, Dec 6, 2015 at 5:50 PM, Tony Kurc <[email protected]> > >> wrote: > >>>>>>>>>> > >>>>>>>>>>> I'll also try it on windows 10 (again x64_64) > >>>>>>>>>>> > >>>>>>>>>>> On Sun, Dec 6, 2015 at 5:36 PM, <[email protected] > > > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> I can run it on Windows 8 tonight if no one else has. > >>>>>>>>>>>> > >>>>>>>>>>>> Joe > >>>>>>>>>>>> > >>>>>>>>>>>> Sent from my phone > >>>>>>>>>>>> > >>>>>>>>>>>>> On Dec 6, 2015, at 4:09 PM, Tony Kurc <[email protected]> > >> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Signatures and hashes look good. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Built fine on Ubuntu 14.04 x86_64. I even cursed a little bit > >>>>>>>> less at > >>>>>>>>>>>>> TestJdbcHugeStream! > >>>>>>>>>>>>> > >>>>>>>>>>>>> LICENSE, NOTICE and README look good. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Docs look good. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Binary ran successfully. > >>>>>>>>>>>>> > >>>>>>>>>>>>> +1 > >>>>>>>>>>>>> > >>>>>>>>>>>>> Did anyone try building on windows? > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Sat, Dec 5, 2015 at 11:46 PM, Aldrin Piri < > >>>>>>>> [email protected]> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Followed helper provided by Joe. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Keys good. > >>>>>>>>>>>>>> Signatures good. > >>>>>>>>>>>>>> Hashes good. > >>>>>>>>>>>>>> Source release builds and passes contrib > >>>>>>>>>>>>>> Required docs present and look correct. > >>>>>>>>>>>>>> Checked out copy of repo for specified commit hash and > diff'd > >>>>>>>>>> against > >>>>>>>>>>>>>> source bundle. Commit is as anticipated. > >>>>>>>>>>>>>> Ran convenience binary with varying templates all > >> successfully. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Release notes and upgrade/migration guides look good. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Kudos to the community on all the efforts involved with this > >>>>>>>>>> release. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> +1, Release this package as Apache NiFi 0.4.0 > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Sat, Dec 5, 2015 at 10:32 PM, Joe Witt < > >> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hello NiFi Community, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I am pleased to be calling this vote for the source release > >> of > >>>>>>>>>> Apache > >>>>>>>>>>>>>>> NiFi 0.4.0. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The source zip, including signatures, digests, and > associated > >>>>>>>>>>>>>>> convenience binaries can be found at: > >>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/nifi/nifi-0.4.0/ > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The Git tag is nifi-0.4.0-RC1 > >>>>>>>>>>>>>>> The Git commit ID is > 191a56f54e3ec178f9f29e1287f23ba66dbf9e43 > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>> > >> > https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=191a56f54e3ec178f9f29e1287f23ba66dbf9e43 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Checksums of NiFi 0.4.0 Source Release: > >>>>>>>>>>>>>>> MD5: b69fd7ec632d7569906e20508058556b > >>>>>>>>>>>>>>> SHA1: 31d88ec7a8431ba5935370eb09be7a343c46411c > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Release artifacts are signed with the following key: > >>>>>>>>>>>>>>> https://people.apache.org/keys/committer/joewitt.asc > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> KEYS file available here: > >>>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/nifi/KEYS > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 152 issues were closed/resolved for this release: > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333070 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Release note highlights: > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.4.0 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Migration/Upgrade guidance: > >>>>>>>>>>>>>>> > >>>>>>>>>> > >> https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance > >>>>>>>>>>>>>>> > >>>>>>>> https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The vote will be open for 72 hours. > >>>>>>>>>>>>>>> Please download the release candidate and evaluate the > >>>> necessary > >>>>>>>>>> items > >>>>>>>>>>>>>>> including checking hashes, signatures, build from source, > and > >>>>>>>> test. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Then please vote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> [ ] +1 Release this package as Apache NiFi 0.4.0 > >>>>>>>>>>>>>>> [ ] +0 no opinion > >>>>>>>>>>>>>>> [ ] -1 Do not release this package because... > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> > >> > >> > >
