The sender/output stream can only ever be in use by one thread since it acquires it from the pool at the beginning of onTrigger, which makes it unavailable to any other threads until it is placed back in the pool.
The flush is there just because now it is writing to an output stream and we may be leaving the sender/output stream open depending if "connection per flow file" is true or not. In the case where you do produce the issue, where is NiFi running? Is it sending from NiFi on your Mac to the device on Linux? On Wed, Sep 20, 2017 at 3:45 PM, Davy De Waele <ddewa...@gmail.com> wrote: > Don't get it either... new code looks more or less the same. been trying > various things all day. Glad I got it working with a "patched" PutTCP. > Better from what I got but still not ideal. > > Can it be a threading issue (we interact with multiple modules) in the input > / output stream handling ? Also noticed a flush that wasn't there before. > Are there other places where charsets can be set ? > > Been thinking about what could cause the tcp server to react this way but no > idea. Can reproduce it easily. Will check if I can enable additional logging > on the moxa. > > On Wed, 20 Sep 2017 at 21:36, Bryan Bende <bbe...@gmail.com> wrote: >> >> For what it is worth, I can't reproduce this on my Mac laptop either... >> >> Setup GenerateFlowFile with same sample text your provided, followed >> by PutTCP with the \r\n delimiter, this sends to ListenTCP in the same >> NiFi instance. >> >> If I look at the content of the flow files coming out of ListenTCP, >> and I put the content viewer in Hex mode, I see: >> >> 0x00000000 0D 40 43 4F 4D 4D 41 4E 44 0D 0D .@COMMAND.. >> >> The 0D 40 43 4F 4D 4D 41 4E 44 0D 0D lines up with \r@COMMAND\r which >> would make sense since ListenTCP read-off the \r\n at the end. >> >> >> On Wed, Sep 20, 2017 at 1:20 PM, Bryan Bende <bbe...@gmail.com> wrote: >> > I'll keep looking at this, but just to summarize the changes that were >> > made in case something jumps out to someone else... >> > >> > The changes were to avoid reading the entire content of the flow file >> > into memory. >> > >> > Previous Behavior: >> > - Create a ByteArrayOuputStream >> > - Use an InputStreamCallback to read entire bytes of flow file, and >> > write them to above output stream >> > - Get value of the outgoing delimiter property >> > - If delimiter is not null, then get the bytes of the delimiter using >> > the charset configured in the processor, and write the delimiter bytes >> > to the above output stream >> > - Get byte array from the ByteArrayOutputStream and pass it to the >> > channel sender, which in turns calls write on a SocketChannel >> > >> > New behavior: >> > - Get the direct SockeOutputStream from the sender >> > - Get the InputStream of the flow file >> > - Call IOUtils.copy to copy the flow file input stream to the socket >> > output stream in chunks >> > - Perform exact same delimiter logic as above >> > >> > >> > >> > >> > On Wed, Sep 20, 2017 at 1:13 PM, Davy De Waele <ddewa...@gmail.com> >> > wrote: >> >> Correct about the delimiter. >> >> Think character set is (was) automatically filled in with utf-8. (Never >> >> noticed a default option) >> >> >> >> On Wed, 20 Sep 2017 at 19:11, Bryan Bende <bbe...@gmail.com> wrote: >> >>> >> >>> Thanks for the info and for digging into this.. >> >>> >> >>> Just to confirm, in your PutTCP configuration, you are saying that you >> >>> have "Outgoing Message Delimiter" set to \r\n ? >> >>> >> >>> And do you have the "Character Set" property set to the default of >> >>> "UTF-8" >> >>> ? >> >>> >> >>> On Wed, Sep 20, 2017 at 12:59 PM, ddewaele <ddewa...@gmail.com> wrote: >> >>> > I've put up a git repo that I use to test the PutTCP behavior. >> >>> > >> >>> > >> >>> > >> >>> > (https://github.com/ddewaele/com.ixortalk.tcp/compare/dev_puttcp_1.4?expand=1) >> >>> > >> >>> > What I've done. >> >>> > >> >>> > - Built Nifi 1.4.0-SNAPSHOT + your PR and used that as my base nifi >> >>> > installation >> >>> > - Noticed weird characters in the tcp responses >> >>> > - Switched to 1.3.0. No more weird characters >> >>> > - Isolated the PutTCP processor by creating a custom PutTCP (see >> >>> > github >> >>> > repo >> >>> > below) to experiment (instead of the built-in PutTCP) >> >>> > - Repo has 2 branches. master contains an "older" PutTCP processor >> >>> > from >> >>> > the >> >>> > Nifi code base. With that one I don't have the weird characters >> >>> > (even on >> >>> > a >> >>> > nifi 1.4.0-SNAPSHOT + your PR). >> >>> > - If I use the dev_puttcp_1.4 branch on my nifi I get the weird >> >>> > characters. >> >>> > >> >>> > >> >>> > Some info on my setup : >> >>> > >> >>> > I'm sending commands to PutTCP using a GenerateFlowFile processor. >> >>> > payload = ${literal('\r')}@COMMAND${literal('\r')} >> >>> > PutTCP adds a delimier (\r\n) >> >>> > >> >>> > Regarding provenance. PutTCP adds the delimiter to the payload >> >>> > before >> >>> > sending it to the TCP socket. That delimiter is not in the flowfile >> >>> > that is >> >>> > put in provenance I think. >> >>> > >> >>> > Cannot reproduce the issue on my mac with my mock tcp server. But on >> >>> > the >> >>> > actual environment (Linux CentOS) with the Moxa devices, there is a >> >>> > clear >> >>> > difference in behaviour. Will see if I can tunnel my through the >> >>> > devices >> >>> > >> >>> > All the info I have right now .... >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
