Re: Cant compile squid 3.2.3 with ssl-crtd (FEDORA 17 x64)
tis 2012-11-27 klockan 04:44 +0200 skrev Eliezer Croitoru: I tried to build squid 3.2.3 on fedora 17 x64 and got an error. compiling options -g -O2 -c -o certificate_db.o certificate_db.cc certificate_db.cc: In member function גvoid Ssl::CertificateDb::load()ג: certificate_db.cc:460:76: error: גindex_serial_hash_LHASH_HASHג was not declared in this scope certificate_db.cc:460:106: error: גindex_serial_cmp_LHASH_COMPג was not [...]$ So it's something with the crtd code but I am unable to understand the meaning of it. Fedora 15-17 have borked openssl headers lying about the OpenSSL version, caused by a slightly misguided patch in OpenSSL which they haven't been able to get rid of due to ABI breakage. This causes problems for our crtd which tries do adapt to the detected OpenSSL version but can't on Fedora. Supposedly fixed in Fedora 18 but have not verified yet. Regards Henrik
Re: [PATCH] HelperReply upgrade stage 2
On 27/11/2012 8:43 a.m., Alex Rousskov wrote: On 11/24/2012 07:47 AM, Amos Jeffries wrote: Stage 2: This stage of the helper reply protocol upgrade adds kv-pair support to the HelperReply object. It uses the new Notes objects to hold any key=value syntax fields received from the helper after the result code field. The values section for these kv-pair may be token or quoted string. Helpers which previously rfc1738 encoded the values may still do so but it is no longer mandatory. The response syntax for all helpers becomes: [channel-ID SP ] result [ SP key-pair ...] [ SP other] EOL The parser for HelperReply is also updated to map the old AF and NA NTLM/Negotiate response fields into the HelperReply notes. * token= is added to supply the NTLM and Negotiate server blob / token field. * user= is added to supply the user label field. The relevant callback handlers are updated for these helpers to make use of these new keys. The bundled Digest authentication helpers are all upgraded to send the new format responses. They now use ERR for failed lookup, BH for internal errors, and OK with ha1= key added to supply a HA1 response. The handler for Digest authentication is updated to process the new HelperReply fields with failover the old format on Unknown result codes. The external ACL handler is updated to pull its key=value pairs out of the Notes list. The old parser loop becomes useless with this and is removed. Taking with it support for several long deprecated keys login=, passwd=, and error=. Any other keys MAY be sent on any response. However at this stage 2 patch they are ignored. As are repeated / secondary values for the expected key names, only the first instance sent in the response is used. Amos +// list of key=value pairs the helper produced +Notes responseKeys; We store both keys and their values here. And we are already inside the reply class. Please rename to something like notes or annotations. Sure. We do now have notes floating out our ears in the same sort of way as conn. +// the value may be a quoted string or a token +const bool urlDecode = (*p != ''); // check before moving p. +char *v = strwordtok(NULL, p); +if (v != NULL urlDecode (p-v) 2) // 1-octet %-escaped requires 3 bytes +rfc1738_unescape(v); +String value = v; If I am reading this correctly, v may be NULL here. If so, value becomes an undefined String. Undefined Strings are not a problem in this particular code, but may be a problem with stage 3 patch reviewed separately in the next email. Fixed. Updated to pass when v is NULL so we completely avoid the undefined String issues. Please make value const or remove it completely. Tried. + +public: +/** + * Adds a note key and value to the notes list. + * If the key name already exists in list, add the given value to its set of values. + */ +void add(const String noteKey, const String noteValue); + +/** + * Looks up a note by key name and returns a Note::Pointer to it + */ +Note::Pointer findByName(const String noteKey) const; Please move up into the existing public section, above the existing data members. Done. +/** + * Looks up a note by key name and returns a Note::Pointer to it + */ +Note::Pointer findByName(const String noteKey) const; It may be a good idea to rename this to findByKey() or simply find(). For the description, consider a more complete Returns a pointer to an existing Note with a given key or nil. Done. +lm_request-server_blob = xstrdup(tokenNote-values[0]-value.termedBuf()); +CvtBin(ha1Note-values[0]-value.termedBuf(), digest_user-HA1); + digest_request-setDenyMessage(msgNote-values[0]-value.termedBuf()); + auth_user_request-user()-username(userLabel-values[0]-value.termedBuf()); Lots of code seems to assume that either a Note has only one value or that the first value is the most important one. I suggest adding Note::singleValue() or a similar method returning values[0]-value. That method will make the above code much simpler and easier to read _and_ it can, for example, assert() or warn if there are actually multiple values stored in the Note. Added a convenience method firstValue() which returns a termedBuf() or . That gets rid of most of them. Made it also perform and return the termedBuf() used almost everywhere at present so it can check for undefined String and return . I am not sure how to deal with the char* vs. String needs cleanly. That will need cleaning up when StringNG goes in but is not urgent before then. At present the external ACL interface is the only one needing String, and only to fill a set of variables which I hope to replace with a Notes::Pointer soon anyway. So have left those using the String API from values[0]-value. + * key-pair := OWS token '=' ( quoted-string | token ) I think you are
Re: [PATCH] HelperReply upgrade stage 2
Preview of the changes requested for review while I build test them.
Unless there are any objections to this in the next ~10hrs or the build
tests fail these will be added to stage2 and the whole lot merged.
Amos
=== modified file 'src/HelperReply.cc'
--- src/HelperReply.cc 2012-11-11 05:10:59 +
+++ src/HelperReply.cc 2012-11-27 10:19:52 +
@@ -54,11 +54,11 @@
MemBuf authToken;
authToken.init();
authToken.append(w1, strlen(w1));
-responseKeys.add(token,authToken.content());
+notes.add(token,authToken.content());
} else {
// token field is mandatory on this response code
result = HelperReply::BrokenHelper;
-responseKeys.add(message,Missing 'token' data);
+notes.add(message,Missing 'token' data);
}
} else if (!strncmp(p,AF ,3)) {
@@ -75,19 +75,19 @@
MemBuf authToken;
authToken.init();
authToken.append(w1, strlen(w1));
-responseKeys.add(token,authToken.content());
+notes.add(token,authToken.content());
MemBuf user;
user.init();
user.append(w2,strlen(w2));
-responseKeys.add(user,user.content());
+notes.add(user,user.content());
} else if (w1 != NULL) {
// NTLM user
MemBuf user;
user.init();
user.append(w1,strlen(w1));
-responseKeys.add(user,user.content());
+notes.add(user,user.content());
}
} else if (!strncmp(p,NA ,3)) {
// NTLM fail-closed ERR response
@@ -109,7 +109,7 @@
// Hack for backward-compatibility: BH used to be a text message...
if (other().hasContent() result == HelperReply::BrokenHelper) {
-responseKeys.add(message,other().content());
+notes.add(message,other().content());
modifiableOther().clean();
}
}
@@ -139,9 +139,9 @@
char *v = strwordtok(NULL, p);
if (v != NULL urlDecode (p-v) 2) // 1-octet %-escaped requires
3 bytes
rfc1738_unescape(v);
-String value = v;
+const String value(v?v:); // value can be empty, but must not be NULL
-responseKeys.add(key, value);
+notes.add(key, value);
modifiableOther().consume(p - other().content());
modifiableOther().consumeWhitespacePrefix();
@@ -171,9 +171,9 @@
}
// dump the helper key=pair notes list
-if (r.responseKeys.notes.size() 0) {
+if (r.notes.notes.size() 0) {
os , notes={;
-for (Notes::NotesList::const_iterator m =
r.responseKeys.notes.begin(); m != r.responseKeys.notes.end(); ++m) {
+for (Notes::NotesList::const_iterator m = r.notes.notes.begin(); m !=
r.notes.notes.end(); ++m) {
for (Note::Values::iterator v = (*m)-values.begin(); v !=
(*m)-values.end(); ++v) {
os ',' (*m)-key '='
ConfigParser::QuoteString((*v)-value);
}
=== modified file 'src/HelperReply.h'
--- src/HelperReply.h 2012-11-10 06:40:21 +
+++ src/HelperReply.h 2012-11-27 10:25:27 +
@@ -24,7 +24,7 @@
HelperReply operator =(const HelperReply r);
public:
-HelperReply() : result(HelperReply::Unknown), responseKeys(),
whichServer(NULL) {
+HelperReply() : result(HelperReply::Unknown), notes(), whichServer(NULL) {
other_.init(1,1);
other_.terminate();
}
@@ -42,8 +42,8 @@
MemBuf modifiableOther() const { return *const_castMemBuf*(other_); }
/** parse a helper response line format:
- * line := [ result ] *#( key-pair )
- * key-pair := OWS token '=' ( quoted-string | token )
+ * line := [ result ] *#( kv-pair )
+ * kv-pair := OWS token '=' ( quoted-string | token )
*
* token are URL-decoded.
* quoted-string are \-escape decoded and the quotes are stripped.
@@ -60,12 +60,12 @@
BrokenHelper, // BH indicating failure due to helper internal
problems.
// result codes for backward compatibility with NTLM/Negotiate
-// TODO: migrate to a variant of the above results with key-pair
parameters
+// TODO: migrate to a variant of the above results with kv-pair
parameters
TT
} result;
// list of key=value pairs the helper produced
-Notes responseKeys;
+Notes notes;
/// for stateful replies the responding helper 'server' needs to be
preserved across callbacks
CbcPointerhelper_stateful_server whichServer;
=== modified file 'src/Notes.cc'
--- src/Notes.cc2012-11-06 22:32:56 +
+++ src/Notes.cc2012-11-27 11:42:55 +
@@ -73,7 +73,7 @@
}
Note::Pointer
-Notes::findByName(const String noteKey) const
+Notes::find(const String noteKey) const
{
typedef
Re: Cant compile squid 3.2.3 with ssl-crtd (FEDORA 17 x64)
On 11/27/2012 06:44 AM, Eliezer Croitoru wrote: On 11/27/2012 8:25 AM, Alex Rousskov wrote: On 11/26/2012 07:44 PM, Eliezer Croitoru wrote: I tried to build squid 3.2.3 on fedora 17 x64 and got an error. SNIP So it's something with the crtd code but I am unable to understand the meaning of it. Without the ssl-crtd there is not a problem in compilation. Is it a dependency ? or a code problem? Maybe related to bug 3232: http://bugs.squid-cache.org/show_bug.cgi?id=3232 What is your OpenSSL version? Alex. Thanks Alex, Yes indeed that is the problem with fedora. What solved the problem was updating openssl and the dependencies. I upgraded using the RPM's from fedora 19 which will be installed fine manually and not from 17 repository. The packages I needed to install openSSL 1.0.1c are: glibc-2.16.90-28.fc19.x86_64.rpm glibc-headers-2.16.90-28.fc19.x86_64.rpm openssl-libs-1.0.1c-8.fc19.x86_64.rpm glibc-common-2.16.90-28.fc19.x86_64.rpm openssl-1.0.1c-8.fc19.x86_64.rpm glibc-devel-2.16.90-28.fc19.x86_64.rpm openssl-devel-1.0.1c-8.fc19.x86_64.rpm Hope it will help someone Not unless you respond to the mailing list Alex. To the mailing list it is :D (somehow slipped under my finger)
Re: [PATCH] HelperReply upgrade stage 2
On 28.11.2012 01:01, Amos Jeffries wrote: Preview of the changes requested for review while I build test them. Unless there are any objections to this in the next ~10hrs or the build tests fail these will be added to stage2 and the whole lot merged. Amos Merged as trunk rev.12490, now onto stage 3... Amos
[PATCH] Refactoring url to original url (storeurl step 1)
The patch is refactoring of two variables:
mem_object-url into mem_obj-original_url
Store_entry-url into Store_entry-originalUrl
Both are named inline with other variables in their scope.
This is the first step towards Store_url completion.
Eliezer
--
Eliezer Croitoru
https://www1.ngtech.co.il
sip:ngt...@sip2sip.info
IT consulting for Nonprofit organizations
eliezer at ngtech.co.il
=== modified file 'src/MemObject.cc'
--- src/MemObject.cc2012-09-04 09:10:20 +
+++ src/MemObject.cc2012-11-28 06:02:17 +
@@ -77,10 +77,10 @@
void
MemObject::resetUrls(char const *aUrl, char const *aLog_url)
{
-safe_free(url);
+safe_free(original_url);
safe_free(log_url);/* XXX account log_url */
log_url = xstrdup(aLog_url);
-url = xstrdup(aUrl);
+original_url = xstrdup(aUrl);
}
MemObject::MemObject(char const *aUrl, char const *aLog_url)
@@ -89,11 +89,11 @@
HttpReply *rep = new HttpReply;
_reply = HTTPMSGLOCK(rep);
-url = xstrdup(aUrl);
+original_url = xstrdup(aUrl);
#if URL_CHECKSUM_DEBUG
-chksum = url_checksum(url);
+chksum = url_checksum(original_url);
#endif
@@ -109,10 +109,10 @@
MemObject::~MemObject()
{
debugs(20, 3, HERE del MemObject this);
-const Ctx ctx = ctx_enter(url);
+const Ctx ctx = ctx_enter(original_url);
#if URL_CHECKSUM_DEBUG
-assert(chksum == url_checksum(url));
+assert(chksum == url_checksum(original_url));
#endif
if (!shutting_down)
@@ -135,7 +135,7 @@
ctx_exit(ctx); /* must exit before we free mem-url */
-safe_free(url);
+safe_free(original_url);
safe_free(log_url);/* XXX account log_url */
@@ -320,7 +320,7 @@
void
MemObject::checkUrlChecksum () const
{
-assert(chksum == url_checksum(url));
+assert(chksum == url_checksum(original_url));
}
#endif
=== modified file 'src/MemObject.h'
--- src/MemObject.h 2012-09-22 20:07:31 +
+++ src/MemObject.h 2012-11-28 06:02:17 +
@@ -99,7 +99,7 @@
#endif
HttpRequestMethod method;
-char *url;
+char *original_url;
mem_hdr data_hdr;
int64_t inmem_lo;
dlink_list clients;
=== modified file 'src/Store.h'
--- src/Store.h 2012-10-29 04:59:58 +
+++ src/Store.h 2012-11-28 05:52:40 +
@@ -113,7 +113,7 @@
/// whether we are in the process of writing this entry to disk
bool swappingOut() const { return swap_status == SWAPOUT_WRITING; }
void swapOutFileClose(int how);
-const char *url() const;
+const char *originalUrl() const;
int checkCachable();
int checkNegativeHit() const;
int locked() const;
=== modified file 'src/StoreMetaURL.cc'
--- src/StoreMetaURL.cc 2012-09-01 14:38:36 +
+++ src/StoreMetaURL.cc 2012-11-28 06:30:57 +
@@ -40,13 +40,17 @@
StoreMetaURL::checkConsistency(StoreEntry *e) const
{
assert (getType() == STORE_META_URL);
-
-if (!e-mem_obj-url)
+/*
+ * TODO: add here or store_url_rewritted condition.
+ * this is to prevent url-mismatch when swapping out an object
+ * when using store_url feature.
+ */
+if (!e-mem_obj-original_url)
return true;
-if (strcasecmp(e-mem_obj-url, (char *)value)) {
+if (strcasecmp(e-mem_obj-original_url, (char *)value)) {
debugs(20, DBG_IMPORTANT, storeClientReadHeader: URL mismatch);
-debugs(20, DBG_IMPORTANT, \t{ (char *) value } != {
e-mem_obj-url });
+debugs(20, DBG_IMPORTANT, \t{ (char *) value } != {
e-mem_obj-original_url });
return false;
}
=== modified file 'src/acl/Asn.cc'
--- src/acl/Asn.cc 2012-11-15 07:35:32 +
+++ src/acl/Asn.cc 2012-11-28 05:52:39 +
@@ -290,7 +290,7 @@
}
if (result.length == 0 asState-dataRead) {
-debugs(53, 3, asHandleReply: Done: e-url() );
+debugs(53, 3, asHandleReply: Done: e-originalUrl() );
asStateFree(asState);
return;
} else if (result.flags.error) {
@@ -355,7 +355,7 @@
debugs(53, 3, asState-offset = asState-offset);
if (e-store_status == STORE_PENDING) {
-debugs(53, 3, asHandleReply: store_status == STORE_PENDING:
e-url() );
+debugs(53, 3, asHandleReply: store_status == STORE_PENDING:
e-originalUrl() );
StoreIOBuffer tempBuffer (AS_REQBUF_SZ - asState-reqofs,
asState-offset,
asState-reqbuf + asState-reqofs);
@@ -366,7 +366,7 @@
asState);
} else {
StoreIOBuffer tempBuffer;
-debugs(53, 3, asHandleReply: store complete, but data received
e-url() );
+debugs(53, 3, asHandleReply: store complete, but data received
e-originalUrl() );
tempBuffer.offset = asState-offset;
tempBuffer.length = AS_REQBUF_SZ - asState-reqofs;
tempBuffer.data = asState-reqbuf + asState-reqofs;
@@ -382,7 +382,7 @@
asStateFree(void *data)
{
ASState *asState =
