Yes, it allows the VIO to know when it is done. We only set it when we have read all the data from the input. I suspect if you continually set it the VIO might think it is done and send a WRITE_COMPLETE but you actually still have more to write.
Shaun On Fri, Mar 15, 2013 at 5:20 PM, Owens, Steve <[email protected]>wrote: > Shaun, > > Looking at the code for TSVIONBytesSet we have > > *void*281 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#281>TSVIONBytesSet > <http://code.metager.de/source/s?refs=TSVIONBytesSet&project=apache>(TSVIO > <http://code.metager.de/source/s?defs=TSVIO&project=apache> viop > <http://code.metager.de/source/s?refs=viop&project=apache>, int64_t > <http://code.metager.de/source/s?defs=int64_t&project=apache> nbytes > <http://code.metager.de/source/s?refs=nbytes&project=apache>)282 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#282>{283 > > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#283> > sdk_assert > <http://code.metager.de/source/s?defs=sdk_assert&project=apache>(sdk_sanity_check_iocore_structure > > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#sdk_sanity_check_iocore_structure>(viop > <http://code.metager.de/source/s?defs=viop&project=apache>) == TS_SUCCESS > <http://code.metager.de/source/s?defs=TS_SUCCESS&project=apache>);284 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#284> > sdk_assert > <http://code.metager.de/source/s?defs=sdk_assert&project=apache>(nbytes > <http://code.metager.de/source/s?defs=nbytes&project=apache> >= 0);285 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#285>286 > > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#286> > VIO <http://code.metager.de/source/s?defs=VIO&project=apache> *vio > <http://code.metager.de/source/s?refs=vio&project=apache> = (VIO > <http://code.metager.de/source/s?defs=VIO&project=apache> *)viop > <http://code.metager.de/source/s?defs=viop&project=apache>;287 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#287> > vio <http://code.metager.de/source/s?defs=vio&project=apache>->nbytes > <http://code.metager.de/source/s?defs=nbytes&project=apache> = nbytes > <http://code.metager.de/source/s?defs=nbytes&project=apache>;288 > <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#288>} > > It looks like this method updates the nbytes of the VIO. But it doesn't > explain what the implications of changing this value are. I am presuming > that when I called TSVConnWrite(output_conn, contp, data->output_reader, > INT64_MAX); That the value of vio->nbytes got set to INT64_MAX. Since my > transform may or may not know how much upstream data it needs to process I > don't see any way for it to update this value to anything meaningful and it > makes me wonder why I would want to. > > In my STATE_EXIT processing I am calling: TSContCall(TSVIOContGet(input_vio), > TS_EVENT_VCONN_WRITE_COMPLETE, input_vio); But I don't call: > TSVConnShutdown(TSTransformOutputVConnGet(contp), 0, 1); until I receive the > TS_EVENT_VCONN_WRITE_COMPLETE event in my callbackHandler. > > What would happen if I continuously update TSVIONBytesSet(output_vio, > total_written) to indicate how much I have written thus far? > Steve Owens > > > From: Shaun mcginnity <[email protected]> > Reply-To: "[email protected]" <[email protected] > > > Date: Fri, 15 Mar 2013 09:48:05 -0700 > > To: "[email protected]" <[email protected]> > Subject: Re: API Question > > Hi Steve, > > I believe that in a transform you should only call TSVConnWrite once. > > You say you are shutting down the VIOs. Are you doing this when you > receive a WRITE_COMPLETE event? If not you may be closing the VIO early. > You can tell the output VIO that all the data has been written to it using > TSVIONBytesSet(output_vio, total_written); > > Regards, > > Shaun > > > On Fri, Mar 15, 2013 at 4:25 PM, Owens, Steve <[email protected]>wrote: > >> Shaun, >> >> This is extremely helpful. In fact it would make a great addition to the >> traffic server on line documentation. Personally I prefer to understand >> what the methods I am calling do rather than not. >> >> The one question that remains unanswered is whether or not this method >> needs to be called more than once. >> >> The reason for my question is this: >> >> I have a transform which essentially prepends and postpends content to >> the upstream content being served. It works in states >> >> STATE_START = write the prefix content downstream and shift to >> STATE_MIDDLE. >> STATE_MIDDLE = copy upstream content to downstream until there is >> no more content and then shift to STATE_END. >> STATE_END = write the suffix content downstream and shift to >> STATE_EXIT. >> STATE_EXIT = clean up the transaction and shut down the VIO's >> because were done. >> >> This plugin works with smaller upstream content as expected. However >> when the upstream content gets larger than about 30K, what I am seeing is >> that from the transform plugin's perspective it is working as expected. >> The expected amount of data is being copied downstream and the suffix is >> being written. I can tell this because I am logging the values of >> upstream_avail, upstream_todo, bytesWritten etc. And everything adds up to >> all content being delivered. However the client is receiving truncated >> content and I am baffled as to why this my be happening. I thought that >> maybe I was not properly using the TSVConnWrite method but from your >> explanation of the method below, it would seem that this is possibly not >> the case. >> >> At any rate, >> >> Thank you for the explanation. >> >> Best Regards, >> >> Steve Owens >> >> From: Shaun mcginnity <[email protected]> >> Reply-To: "[email protected]" < >> [email protected]> >> Date: Fri, 15 Mar 2013 07:09:12 -0700 >> To: "[email protected]" <[email protected]> >> Subject: Re: API Question >> >> Hi Steve, >> >> TSVConnWrite initiates a write operation on the transform's downstream >> VConnection. The last parameter is the total amount of data you are going >> to write. INT64_MAX is effectively unlimited. >> >> When you have data to write you get the VIO's buffer >> (TSVIOBufferGet(output_vio)), and copy the data into it. Then the VIO is >> reenabled. Reenabling the VIO causes an IMMEDIATE event to be sent to the >> VIO's VConnection. When the VConnection's event handler has finished with >> the data it will call your continuation back. If it expects you to send >> more data it will pass a WRITE_READY event. If you have given the VIO the >> total amount of data as set in TSVConnWrite it will call you back with a >> WRITE_COMPLETE event. Thisbehaviour is the same as what the transform >> should do to its input VIO. >> >> Since the downstream VConnection is the client and / or cache, if you do >> a finite write, i.e. not INT64_MAX, then the HTTP transaction will close >> when the limit is reached. Issuing another write in this case will probably >> not work. You can do multiple writes on a NetVConnection though, e.g. as >> provided by TSNetConnect. >> >> Hope that helps. >> >> Shaun >> >> >> >> -----Original Message----- >> >> From: James Peach [mailto:[email protected] <[email protected]>] >> >> Sent: Friday, March 15, 2013 3:12 AM >> >> To: [email protected] >> >> Subject: Re: API Question >> >> >> >> On 14/03/2013, at 5:13 PM, "Owens, Steve" <[email protected]> wrote: >> >> >> >> > In looking at the following documentation: >> http://trafficserver.apache.org/docs/trunk/sdk/io-guide/transformations.en.html >> >> > >> >> > It would seem that the only mention of TSVConnWrite is to a single >> parameter method. >> >> > >> >> > Yet in my plugin I am using >> >> > >> >> > data->output_vio = TSVConnWrite(output_conn, contp, data->output_reader, >> >> > INT64_MAX); >> >> > >> >> > What has changed about this method? >> >> > >> >> > What does it actually do? >> >> >> >> It schedules data to be written to the VConnection. >> >> >> >> > >> >> > Do I need to call it every time the plugin is re-enabled? Or can I >> just call it once? >> >> >> >> I don't know that I really understand the question, but if the length is >> INT64_MAX you would only call it once. If you want you cancall it a >> multiple times with shorter lengths. >> >> >> >> J >> > >
