Re: problem for plugin example: server-transform.c
On 3.1 I have to use the ntohs function for the port number when using TsNetConnect. Have you tried this? On 12 September 2011 02:33, steven liu stevenli...@gmail.com wrote: Thanks. Yes. I am setting port using hons() using following codes. struct sockaddr_in ip4addr; ip4addr.sin_family = AF_INET; ip4addr.sin_port = htons(server_port); ip4addr.sin_addr.s_addr = inet_addr((const char *)(127.0.0.1)); action = TSNetConnect(contp, (struct sockaddr const*)ip4addr); On Mon, Sep 12, 2011 at 6:34 AM, Alan M. Carroll a...@network-geographics.com wrote: Are you setting the port in the sockaddr? Note that in a sockaddr, all data is in network order (including the port). Sunday, September 11, 2011, 11:24:25 AM, you wrote: Dear All, I am tring to run a plugin example: server-transform.c under V3.0.0. But I found that it does not work out of the box. Firstly, it cannot be compiled properly due to TSNetConnect(contp, server_ip, server_port); In V3.0.0, the parameter of this function has been changed to TSNetConnect(TSCont contp, sockaddr const* addr); I have tried to change to the new function. However, it does not work properly. This transform plugin actually sends received buffer to a local socket server (echo server), which will send the content back to the plugin. I found that even TSNetConnect() returns SUCCESS, but it does not send anything out to the socket server. I have tried to use TCPdump to capture packets with port = 7 (echo port). BUt I did not captured any such packets. It looks like TSNetConnect() has some problems. Many thanks for any help. Rgds, Steve
Re: problem for plugin example: server-transform.c
The server port comes from a URL, e.g. http://10.44.10.1:80, and I just atoi the port. I have checked with a network trace that the correct port is being connected to. With previous versions of Traffic Server I never had to use ntohs and just passed in the atoied number. On 12 September 2011 15:57, Alan M. Carroll a...@network-geographics.comwrote: Actually, Chris, you should use htons since this is going from host to network order. However on all modern systems htons() and ntohs() are the same function. Some of us old timers remember systems where that wasn't the case but those have long since faded to memories. I think TSNetConnect gets tested during regression, so this seems a rather unexpected failure. From where do you get server_port? atoi and its ilk, hard wired constant, or some other network connection? Can you do a netstat -a -n --tcp and verify that the listening process is listening on the port you expect? Monday, September 12, 2011, 9:20:26 AM, you wrote: On 3.1 I have to use the ntohs function for the port number when using TsNetConnect. Have you tried this? On 12 September 2011 02:33, steven liu stevenli...@gmail.com wrote: Thanks. Yes. I am setting port using hons() using following codes. struct sockaddr_in ip4addr; ip4addr.sin_family = AF_INET; ip4addr.sin_port = htons(server_port); ip4addr.sin_addr.s_addr = inet_addr((const char *)(127.0.0.1)); action = TSNetConnect(contp, (struct sockaddr const*)ip4addr);
Problems with using transformation API with caching enabled
Hi, I have written a plugin that scans data using transformations. It all works fine until caching is enabled. I have found two problems and I have supplied modified bnull_transform code to demonstrate them. The only changes I have made to the code are: basic logging, adding hooks for READ_CACHE_HDR_HOOK and reading the cache headers instead of the server headers where appropriate. The problems are: 1. It seems that when data is transformed from the cache that another duplicate copy of it is written to the cache. To re-create this I use an Apache server with the expires module enabled and then: a. stop Traffic Server b. clear the cache - traffic_server --Cclear c. restart Traffic Server - running the modified bnull plugin d. wget a 1MB file through the proxy - bnull logging shows that this comes from the server e. wait a while then use 'traffic_line -r proxy.process.cache_bytes_used' to see that 1081344 bytes have been stored in the cache f. perform exactly the same wget again - bnull logging shows that this comes from the cache g. use 'traffic_line -r proxy.process.cache_bytes_used' to see that 2162688 bytes have been stored in the cache - exactly double h. keep using wget and 1081344 bytes keeps on being added to the cache I have got around this in my own plugin by not writing to the OutputVConn connection when the headers have come from the cache. I do not know whether this is the best solution though. 2. This problem is a bit more complicated. It seems that sometimes if a client sends a request with a 'if-modified-since' header to Traffic Server with caching enabled but the response does not come from the cache then the created transformation is not called so I cannot scan the data. The modified bnull plugin shows this - in this case the cache header code path is never used because the headers always come from the server. I have done some network traces which I have attached which show the problem. To test this I use a Eurogamer site URL where the response has been cached in the browser causing it to add a 'if-modified-since' header to subsequent requests. The URL I use is: http://www.eurogamer.net/styles/Autocomplete.css?v10.20-21670. Normally this occurs (the 200sentback.cap shows this): a. Go to the URL so that the response is stored in the browser cache. b. Restart the browser. c. Enter the URL again - browser sends request with 'if-modified-since' header - this is shown in the capture. d. The capture shows the request being sent to the server with the 'if-modified-since' header stripped. e. The server returns 200. f. The proxy returns 200 and the transformation fires as normal - the logging shows this: 20110909.17h38m12s transform_plugin - from server 20110909.17h38m12s transformable 1 20110909.17h38m12s bnull_transform 1 20110909.17h38m12s handle_transform 20110909.17h38m12s bnull_transform 1 20110909.17h38m12s handle_transform 20110909.17h38m12s bnull_transform 103 20110909.17h38m12s bnull_transform 1 When the problem occurs (the 304sentback.cap shows this): a. Go to the URL so that the response is stored in the browser cache. b. Restart the browser. c. Enter the URL again - browser sends request with 'if-modified-since' header - this is shown in the capture. d. The capture shows the request being sent to the server with the 'if-modified-since' header stripped. e. The server returns 200. f. The proxy returns 304 and closes the connection to the server. When this occurs the transaction is closed down and the transformation is never called. The logging shows this: 20110909.17h39m28s transform_plugin - from server 20110909.17h39m28s transformable 1 20110909.17h39m28s bnull_transform 1 As the logging shows 'handle_transform' is never called. Note that it can take a number of attempts to re-create the problem. I used Firefox and opened a new tab for each attempt. Its odd that for 90% of the requests Traffic Server returns a 200 response but for the rest (which are identical) it returns a 304. I guess that the 304 is causing the transformation not to fire because it thinks there is no data. I cannot think of a workaround to get past this issue. Can anyone shine any light on this? Cheers, Chris Reynolds. /** @file A brief file description @section license License Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the License); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing
Re: benchmarking 3.1.0 and 3.0.1
To clarify when you say logging off do you mean: CONFIG proxy.config.log.logging_enabled INT 0 Thanks, Chris Reynolds. On 9 September 2011 18:36, Leif Hedstrom zw...@apache.org wrote: On 09/09/2011 10:56 AM, Bryan Call wrote: I ran some benchmarks on the 3.0.1 release and the 3.1.0 release with different options on and off. This benchmark was done with 1 client, 1 proxy, and 3 origin servers. The servers are 12 core (24 hyper-threaded) boxes and are the beefiest I could get a hold of. The response was only a 1 byte response, so that I wouldn't make the NIC a bottleneck. There are a couple things to note in the benchmark results: 1. There is a huge impact in performance in 3.0.1 and 3.1.0 having logging on and only proxying to the origin server. Turning off logging increases the performance by 185%. I think there can be some large performance gains to be won here. 2. There was a huge performance decrease from 3.0.1 to 3.1.0 with just turning off the cache and only proxying to the origin. 3.1.0 is only performing at 14% of what 3.0.1 is. Turning off shared sessions removes this decrease in performance, so some of the work that Leif is doing might resolve this. I am going to benchmark his patch today. I'm wondering if we should back out the two-line fix from TS-880, and instead do the proposed fix from TS-924 ? Thoughts? -- leif
Re: [VOTE] Release Apache Traffic Server v3.1.0
+1 from me as well. Running Polygraph performance (on a custom Linux from scratch 32-bit system) and it seems to be stable. Unfortunately I do not have any results from previous builds so I cannot tell if the performance has changed. On 25 August 2011 13:03, Jack Quinlin jquin...@yahoo.com wrote: +1 from me. That candidate, url is http://people.apache.org/~zwoop/rel-candidates/ , works well with CentOS-6.0-x86_64, and on Amazon EC2 linux instances 64bit. If needed I can install it on other OS as I have most pltaforms on hand or can easily commandeer a server. -- *From:* Leif Hedstrom zw...@apache.org *To:* dev@trafficserver.apache.org; 'us...@trafficserver.apache.org' us...@trafficserver.apache.org *Sent:* Monday, August 22, 2011 10:15 AM *Subject:* [VOTE] Release Apache Traffic Server v3.1.0 Hi all, I've prepared a package for a v3.1.0 release. Please take a look at the artifacts, check the STATUS/README/CHANGES files, and do builds and tests. After finishing your examination of the release candidate, please cast your ±/0 votes, I will call the vote on 8/25/2011. The src tar-ball and signatures are available in http://people.apache.org/~zwoop/rel-candidates/ The relevant files are minotaur (15:07) 263/0 $ ls -lrt total 2440 -rw-r--r-- 1 zwoop zwoop 2439617 Aug 22 00:50 trafficserver-3.1.0-unstable.tar.bz2 -rw-r--r-- 1 zwoop zwoop 836 Aug 22 00:50 trafficserver-3.1.0-unstable.tar.bz2.asc -rw-r--r-- 1 zwoop zwoop 71 Aug 22 00:50 trafficserver-3.1.0-unstable.tar.bz2.md5 -rw-r--r-- 1 zwoop zwoop 79 Aug 22 00:50 trafficserver-3.1.0-unstable.tar.bz2.sha1 Checksums are MD5: 39899ebbfffeb02a6be2afd63765588f *trafficserver-3.1.0 unstable.tar.bz2 SHA1: c83c26d65dda28ccd130a0d3dc624a8ef15c3af9 *trafficserver-3.1.0-unstable.tar.bz2 This is primarily a large number of bug fixes over the 3.0.0 release, but it also adds a foundation for finishing the IPv6 support (to origin servers). Over 80 bugs / improvements were closed for this release. Thanks, -- The Apache TS developer community
Re: Obtaining the client's host name
Thank-you for the suggestion - I will just use gethostbyaddr_r for the moment. On 8 August 2011 16:02, Leif Hedstrom zw...@apache.org wrote: On 08/08/2011 08:09 AM, Theo Schlossnagle wrote: I would think we'd need to provide a convenience function that uses the DNS subsystem to do PTR record lookups. Agreed. I think we should provide the equivalent of TSHostLookup(), but for PTR lookups. -- Leif
Re: HTTP responses slower after first attempt
Thank-you. The trunk build does seem to have fixed the problem. On 22 August 2011 10:45, ming@gmail.com ming@gmail.com wrote: we have fixed the problem in trunk, please test with latest trunk and report back if it is fixed in you situation. be aware, that v3.0.1 is not fixed too, you should test the svn trunk. you may get the information from: https://issues.apache.org/jira/browse/TS-880 thanks 在 2011-08-22一的 09:48 +0100，Chris Reynolds写道： Hi, I am performing some basic performance tests with Traffic Server with and without a plugin I have written. I have noticed that if you download (without any plugin enabled) a file then download another file that the second download is a lot slower that the first one - over 10 times as slow. If you then wait for about 10-20 seconds it is back to the original speed. For example downloading an 8MB file using wget takes about 0.09s, but downloading the same or another 8 MB file on the same server again it is 1.3s. If I bypass the proxy then the files always downloads at about 0.07s. This seems to be affecting the overall performance of Traffic Server. Note I am using version 3.0.0 with no caching enabled. Has any one got any ideas? Thank-you, Chris Reynolds.
TSNetConnect now takes host byte ordered port number
Hi, The trunk code version of TSNetConnect now seems to take a host byte ordered port number. Previously (i.e. 3.0.0) took the network byte order. I.e. I now have to use the ntohs function on the port number. Is this by design? Thank-you, Chris Reynolds.
Re: HTTP responses slower after first attempt
Hi, I have managed to resolve this problem. If the proxy.config.http.share_server_sessions config value is set to 0 then this problem does not occur - the default is 1. When running performance tests I was using the same server. Is this a bug? I would have thought that sharing server sessions would make the performance faster not slower. Chris Reynolds. On 22 August 2011 09:48, Chris Reynolds shann...@gmail.com wrote: Hi, I am performing some basic performance tests with Traffic Server with and without a plugin I have written. I have noticed that if you download (without any plugin enabled) a file then download another file that the second download is a lot slower that the first one - over 10 times as slow. If you then wait for about 10-20 seconds it is back to the original speed. For example downloading an 8MB file using wget takes about 0.09s, but downloading the same or another 8 MB file on the same server again it is 1.3s. If I bypass the proxy then the files always downloads at about 0.07s. This seems to be affecting the overall performance of Traffic Server. Note I am using version 3.0.0 with no caching enabled. Has any one got any ideas? Thank-you, Chris Reynolds.
Obtaining the client's host name
Hi, Is it possible using the API to obtain the client's host name - i.e. a reverse DNS lookup? If not I guess it is safe to use gethostbyaddr_r to look it up from the client's IP address. Thank-you, Chris Reynolds.
Obtaining proxy port inside plugin
Hi, Is it possible to obtain the proxy's port number from inside a plugin? I want to use more than one listening port but I need to know which port the client connected to in my plugin code. Note - I am using Traffic Server 3.0.0. Thank-you, Chris Reynolds.
