Re: removing incr0 from Rivet from quattuor

[Karl Lehenbauer]

Yeah, that is exactly why. -k

Sent from my iPhone

> On Jan 12, 2020, at 09:03, Massimo Manghi  wrote:
> 
> I always conjectured that command "incr0 " had been introduced in 
> Rivet as an alternative to 'incr'. In Tcl <= 8.4 this command failed if the 
> variable whose name is passed as argument is not set and incr0 simply creates 
> the variable in this case
> 
> Starting with Tcl 8.5 command native 'incr' works in pretty the same way and 
> since we are supporting Tcl >= 8.6 I think we can safely remove it at least 
> from quattuor
> 
> -- Massimo
> 
> -
> To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
> For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
> 

-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: nws on rivet?

[Karl Lehenbauer]

Hi Horace,

Good to hear from you!

NeoWebScript is dead at this point, and we've thrown our efforts behind Rivet 
since the early 2000s.

Rivet is functionally pretty close but it does a lot more, now, and the Tcl is 
more straight-ahead.  The NWS support code for tables and stuff should work 
with minimal changes.

I would recommend recoding the application for Rivet and getting on modern 
Apache, Rivet, and Tcl.

Karl


On 1/22/19, 11:56 AM, "Georgios Petasis"  wrote:

Στις 22/1/2019 16:26, ο Massimo Manghi έγραψε:
> Hi Horace
>
> On 1/22/19 6:10 AM, Horace 'Kicker' Vallas wrote:
>> Hi guys - long time  ooba oooba!!
>>
>> LOL ... question: how can I get nws running on rivet (if that even 
>> makes sense).
>>
>> Brad's site (backbenimble.com) is still running nws but apparent 
>> changes on the way from Authorize.net looks like they'll be dropping 
>> md5 in favor of sha-512 and the only sha-512 tcl stuff I've found 
>> requires a newer version of tcl than we're running - and Troy tells 
>> me that he can't find a newer tcl for the system on brad's server.
>>
>> Anway - hi - hope everyone from the old NWS gang are doing well HA 
>> HA! ... Oh yeah - I'm freakin' BALD now HA HA HA
>>
>
> I wasn't in the NWS gang so I can't provide a definite answer, but how 
> big and complex was NWS's API? Is a compatibility layer so difficult 
> to write? I guess that in many cases there must be a one-to-one 
> correspondence between rivet and NWS calls, at least for the very 
> fundamental features (HTML entities conversions, Urlencoded and form 
> data sent with the POST method, HTTP header manipulation ...)
>
>  -- Massimo
>
What is NWS?

George



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Tcl random seed in each Apache child can be non-unique when running SeparateVirtualInterps no

[Karl Lehenbauer]

On May 14, 2017, at 11:27 AM, Massimo Manghi 
<man...@biol.unipr.it<mailto:man...@biol.unipr.it>> wrote:



On 05/14/2017 03:45 AM, Karl Lehenbauer wrote:
This is kind of cool.  If the mod_rivet master interpreter has gotten
its random seed initialized, say by some Tcl code requesting a random
number, then when Apache forks off children they will inherit the same
random seed and hence produce the same sequence.

This can be fixed from the Rivet config by forcing children to re-seed
the RNG, something like:

RivetServerConf ChildInitScript "expr {srand([clock clicks] + [pid])}”

It might be a good idea to do this in C in Rivet whenever running
SeparateVirtualInterps off.



Did you actually try also the case with separate interpreters? In mod_rivet 2.3 
these interpreters are slaves of a single root interpreter. How deeply are 
slave interpreters initialized wrt their parent?

In mod_rivet 3.0 the problem should be confined in the prefork bridge when 
SeparateVirtualInterps Off, otherwise separate interpreters are interpreters in 
their own right, not slaves

-- Massimo

Good catch, Massimo...  I had not tried it.  However from the command line it 
looks like new interpreters are seeded uniquely, even safe interpreters...

 $ tclsh

 % info patchlevel
8.6.6

 % interp create
interp0
 % interp0 eval "expr rand()"
0.06173784474923175

 % interp create
interp1
 % interp1 eval "expr rand()"
0.40116244712898624

 % interp create -safe
interp2
 % interp2 eval "expr rand()"
0.2504010522972797

 % interp create -safe
interp3
 % interp3 eval "expr rand()"
0.27850173519854515

Tcl random seed in each Apache child can be non-unique when running SeparateVirtualInterps no

[Karl Lehenbauer]

This is kind of cool.  If the mod_rivet master interpreter has gotten its 
random seed initialized, say by some Tcl code requesting a random number, then 
when Apache forks off children they will inherit the same random seed and hence 
produce the same sequence.

This can be fixed from the Rivet config by forcing children to re-seed the RNG, 
something like:

RivetServerConf ChildInitScript "expr {srand([clock clicks] + [pid])}”

It might be a good idea to do this in C in Rivet whenever running 
SeparateVirtualInterps off.

Re: Rivet's FormBroker package

[Karl Lehenbauer]

I haven’t dug into it in depth yet, but this is looking pretty good.  I think a 
nice addition would be to allow specification of a default value.  If a var is 
defined in your FormBroker DDL-type specification, a default value isn’t 
defined, and the value is not present in the response, the broker would raise 
an error.  Otherwise if the default value is provided but the value isn’t in 
the response, the default would be provided.

Another thing… at work we have been leaning toward copying values as they are 
checked and possibly transformed from the response array into a new array.  
That way there’s no risk that a value in the response array gets used 
downstream without having been explicitly called out.  An alternative approach 
would be to remove any value from the response array that isn’t in the broker 
specification (and possibly treat the presence of something unexpected as a 
fatal error).  One way or another I think it is important that a response value 
not be able to be used downstream without being explicitly called out in the 
broker specification.

On 10/17/16, 10:29 AM, "Massimo Manghi"  wrote:


An initial prototype of the Rivet's FormBroker is now available from

trunk/rivet/packages/formbroker.tcl

The code is inspired by Karl's original code but it goes further ahead 
trying to become a form definition repository. The overall style of the 
package has an OOP flavor even though no one of the OOP environments 
available for Tcl was used. It's just 'namespace ensemble' based. I will 
henceforth use the word 'object' meaning any instance of form descriptor 
created by the FormBroker package.

Forms definition objects are referenced through commands generated by 
the FormBroker package with the 'create' call

set fbobj [::FormBroker create \
{var1 string bounds 10 constrain quote} \
{var2 email} \
{var3 integer bounds 10} \
{var4 unsigned bounds {10 100} constrain}]


  which is quite similar to the original form broker: each element in 
the argument list is a list in its own right in which the first and 
second element must be the form variable name and type. At the moment 
supported types are 'string', 'integer', 'unsigned', 'email'. Each of 
them has its own validation procedure. The supported variable types can 
be extended easily, but non portably: I mean that writing a validator 
requires explicit manipulation of the dictionary that provides a form 
variable internal representation. (As such it's a design flaw, at the 
moment). The keyword 'constrain' means that, when possible, a value is 
brought within its assigned bounds. For a 'string' it means the string 
has to be truncated to be n characters when longer.

  The namespace ensemble offers also a 'creategc' method that demands a 
Tcl variable name as first argument. The command name is returned and 
also stored in the named Tcl variable. When this Tcl variable is 
destroyed the unset trace that ensues triggers the object destructor to 
be called, causing the form internal representation to be garbage 
collected...

  A form response is then checked calling

  $fbobj validate response

where 'response' is the usual array of variables made by 
::rivet::load_response. This method returns 'true' if the array 
validates. If the validation fails the method

$fbobj failing

returns a list of variable names with the validation error codes.

Variables can be quoted and the quoting function can be customized (the 
internal function just puts a variable value between single quotes). A 
custom function for quoting must have the very basic form

set quoted_string [ $orig_string]

the need for quotation can be variable specific (in that case 'validate' 
quotes in the 'response' array only the variables eligible to be 
quoted). Overall quoting can be forced by calling

$fbobj validate -forcequote response

There is more to say but I don't want to bother you any further. I will 
answer your questions with pleasure. The namespace ensemble API is open 
to be amended if you have some strong idea on how to redesign it. I 
won't set out writing the documentation any soon: I'm going to allow 
more time to see if the design settle down using the package in regular 
development (which I still have to do!). If you're interested to write a 
specific data type validator I will show you how to (there's an example 
in trunk/contrib/validate_mac.tcl which shows how to validate a mac 
address)

  -- Massimo

P.S. I will certainly remove the 'namespace export *' line from the 
package in order to keep private all the methods not intended for 

Re: Form data validator

[Karl Lehenbauer]

Yeah we too have been working on this once again.  Basically take an array 
created by load_response and invoke procs to validate, sanitize, etc.  The 
procs generate errors like if any of a specified list of fields that are 
supposed to be integer, aren’t, and Tcl 8.6’s try…trap is handy for wrapping 
them all together into one try body, handling any error with the trap arm.  
Also they can be used unguarded, let the error trace back to Rivet, and have 
the Rivet error handler recognize the errorCode and do something appropriate.



I’ve pasted a chunk of that stuff below, but we are instead going to make 
something similar that validates, copies, sanitizes stuff from the response 
array into a second array that’ll be used by the code to do whatever.



#

# response_security_error - issue an error with errorCode

#   set appropriate -- we expect the rivet error handler

#   to catch this and do the right thing

#

proc response_security_error {type message} {

error $message "" [list RIVET SECURITY $type $message]

}



#

# require_response_vars - error if any of the specified are not in the response

#

proc require_response_vars {_response args} {

upvar $_response response



foreach var $args {

if {![info exists response($var)]} {

response_security_error MISSING_VAR "var $var not present in 
$_response"

}

}

}



#

# force_response_integers - error if any of named vars in response doesn't exist

#   or isn't an integer

#

proc force_response_integers {_response args} {

upvar $_response response



require_response_vars response {*}$args



foreach var $args {

if {![regexp {[0-9-]*} response($var)]} {

response_security_error NOT_INTEGER "illegal content in $var"

}



if {![scan $response($var) %d response($var)]} {

response_security_error NOT_INTEGER "illegal content in $var"

}

}

}



#

# force_response_integer_in_range - error if var in response isn't an integer

#   or if it isn't in range

#

proc force_response_integer_in_range {_response var lowest highest} {

upvar $_response response



force_response_integers response $var



if {$response($var) < $lowest || $response($var) > $highest} {

response_security_error "OUT_OF_RANGE" "$var out of range"

}

}



#

# force_quote_response_strings - sanitize and pg_quote all the specified 
strings in the array

#

proc force_quote_response_strings {_response args} {

upvar $_response response

force_sanitize_response_strings response {*}$args



foreach var $args {

set response($var) [pg_quote $response($var)]

}

}



#

# force_quote_response_unfilteredstrings - rewrite named response

#   elements pg_quoted

#

proc force_quote_response_unfilteredstrings {_response args} {

upvar $_response response

require_response_vars response {*}$args



foreach var $args {

set response($var) [pg_quote $response($var)]

}

}





On 9/23/16, 11:20 AM, "Massimo Manghi"  wrote:



Years ago Karl proposed to develop a form data validator he called

'response broker'. (The message is at [1]) Damon answered quite rightly

"haven't we all written something like this at some point?"



I'm now working on a refactoring of an application and I'm striving to

redesign it in a way to have an emphasis on object collaboration rather

than long and complicated methods. (I want to convince someone to adopt

it by enticing him into working with a set of easy to read and

understand classes)



Form data validation became soon one of the sore points: I can't imagine

a method for validation that can match safety and completeness without

being cumbersome, repetitive or, in a word, boring to work with



Much more exciting would be working on a real validator that solves to

some extent the general problem.



Has anyone worked on Karl's idea? Am I inexorably about to set out to

reinvent the wheel?



  -- Massimo



[1]


http://mail-archives.apache.org/mod_mbox/tcl-rivet-dev/201301.mbox/%3CDA2C5384-232C-4377-B35C-A91E8FA348C6%40gmail.com%3E







-

To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org

For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: a new script execution model for mod_rivet

[Karl Lehenbauer]

I think this is a good idea.  All the C code this would replace probably 
provides very little in terms of performance.

On 9/21/16, 4:03 PM, "Massimo Manghi"  wrote:

I will carry on my analysis on this proposal and I will try as much as 
possible to isolate into a single file the code to be changed for having 
a working version of the model (unless fundamental incompatibilities 
emerge). If this is possible I will create a branch with it.

I have to set it aside for a while now

  -- Massimo

On 09/19/2016 03:09 PM, Massimo Manghi wrote:
> I've been thinking lately of a rather radical revision of the Tcl code
> execution as it's done currently in mod_rivet. I haven't checked the
> feasibility of every detail but I don't see fundamental and forbidding
> obstacles to the implementation of such scheme.
>
> Let's imagine to have mod_rivet replace the current C language machinery
> of script evaluation by simply Tcl_Evaluating this script
>
>   try {
> 
>   } trap {RIVET ABORTPAGE} {
> 
>   } trap {RIVET THREAD_EXIT} {
> 
>   } on error {::rivet::error_code ::rivet::error_options} {
> 
>   } finally {
> 
>   }
>
> The script could be composed in memory from configuration information,
> for example omitting the  trap or the 'finally' branch if
> such handlers have not been set up. Also the error-script handler
> determination could be done during the configuration stage and the
> branch would be always existing al least with the default handler
>
> The content generation script should by default be made by the
> concatenation of the sequence
>
> ::Rivet initialize_request
> eval 
> namespace eval ::request {  }
> eval 
>
> which reproduces the current execution sequence. The script referenced
> in the URL could be stored in the module status and in case accessed by
> introducing a new command, something like
>
> eval [::rivet::url_script]
>
> the command should in case fetch the script from the cache. So far I can
> see only problems impacting the performance, as I expect the C language
> machinery to be faster, but in this case not dramatically faster than
> the 'tryfinally' construct (after all most requests are successfully
> processed without errors. At least we hope so). Of course there are
> possible optimizations like chaining the before-url-after scripts in one
> script requiring a single call to Tcl_EvalObjEx
>
> Perhaps I've been influenced by years of mod_rewrite usage where the
> local path is analyzed and the transformation output blended into the
> list of argument-value pairs to become the final URL query arguments. I
> ended up having virtual hosts running websites with a single empty root
> index.rvt and every template is parsed from the main application script.
> Furthermore the content generation script is shared between different
> applications and by no means resides in any directory tree pointed by
> DocumentRoot directives. The bottom line is that the segmentation in 3
> scripts has no meaning to me (..and for what I need also the URL
> referenced script has no interest. But that's my approach...).
>
> My point is that we could take a further step ahead and make the default
> request processing script shown above entirely replaceable by a user
> defined script (either based on try...on error...finally... or on any
> other solutions the user would deem fit for their applications). Some
> optimization could be attained by introducing some mechanism for
> building such script by inclusion of the code, not simply sourcing or
> parsing other files. The script could be kept in a special cache
> providing some methods for determining its depends on the files it was
> built from, in case one of them is modified requiring the buffer to be
> refreshed.
>
> opinions and criticism are welcome (...well, actually the latter are
> less welcome, but don't be afraid ;-))
>
> regards
>
>  -- Massimo
>
> -
> To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
> For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
>
>

-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Tcl exit command in mod_rivet

[Karl Lehenbauer]

Hi Massimo,

I want to affirm your comments from the 11th that there needs to be a way to to 
terminate the child process or thread if code detects that things have gotten 
wonky.  We do this, for instance, on failure to obtain a database connection 
during page processing startup and on any uncaught traceback within the guts of 
our top level traceback-handling code.

Karl

> On Sep 16, 2015, at 9:09 AM, Massimo Manghi  wrote:
> 
> Hi Harald
> 
> On 09/16/2015 09:29 AM, Harald Oehlmann wrote:
>> 
>> The later thinking gives arguments to define exit as "abort thread or
>> process".
>> 
>> It is also interesting what happens to exit calls on the C level,
>> libraries may use: Tcl_Exit, Tcl_ExitThread
>> 
>> Thank you,
>> Harald
>> 
> 
> I agree Harald, I think we have to find the best possible way to drive the 
> behavior of 'exit' and let it fit into Rivet ecosystem in the most gentle way.
> 
> AFAIK Tcl_ExitThread is the proper way at C-language level to terminate a 
> thread created by the Tcl Thread library. It's equivalent to apr_thread_exit, 
> that I used having followed in the worker bridge (and in general in 
> mod_rivet) the design of having APR controlling everything not very Tcl 
> specific, avoiding as much as possible intersections between the 2 libraries. 
> In other words, I called Tcl_* from a certain level up and stuck to APR for 
> anything from that level down towards the Apache framework...
> 
> -- Massimo
> 
> 
> -
> To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
> For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
> 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Tcl exit command in mod_rivet

[Karl Lehenbauer]

On 9/16/15, 11:58 AM, "Massimo Manghi" <massimo.man...@unipr.it> wrote:

>
>
>On 09/16/2015 04:37 PM, Karl Lehenbauer wrote:
>> Hi Massimo,
>>
>> I want to affirm your comments from the 11th that there needs to be a
>> way to to terminate the child process or thread if code detects that
>> things have gotten wonky.  We do this, for instance, on failure to
>> obtain a database connection during page processing startup and on
>> any uncaught traceback within the guts of our top level
>> traceback-handling code.
>>
>> Karl
>>
>
>So Karl, you're in favor of the idea of terminating the process anyway
>without configuration option to enable/disable this behavior and without
>going to the AbortScript stage. Is it correct?
>
>  -- Massimo

I don¹t have an opinion about how it should be implemented or whether or
not it should be configurable, I just think it¹s important for Tcl code to
be able to say ³I have lost faith in the viability of this child process
or thread and I want it to terminate.²

It doesn¹t have to be ³exit², either; it could be ³terminate_child² or
whatever.



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Tcl exit command in mod_rivet

[Karl Lehenbauer]

> On Sep 11, 2015, at 4:18 AM, Massimo Manghi  wrote:
> 
> As no answer or suggestion followed this proposal I will implement it and 
> commit it in trunk. Then, in order to mitigate the possible interpreter 
> termination/initialization storm that might ensue an inappropriate usage of 
> the 'exit' command (we don't delete interpreters anymore but child exit 
> scripts must run anyway), I will implement a two states machine in the 
> threads of the rivet_worker_mpm bridge. Interpreters will be created and 
> initialized only when the first request for a specific virtual host comes in.

Creating interpreters only when the first request for specific virtual host 
comes in is a great idea!  (I presume this is specifically only when running 
with separate virtual interpreters.)

Not deleting the interpreter on process exit is a nice if small performance win 
for sites (like us) that have httpd children exit after handling a few thousand 
requests.

I haven’t really followed the threading stuff closely because for our 
production stuff the approach of loading up the master interpreter in the httpd 
parent with the hundreds of packages and stuff that we use has forked children 
inherit an interpreter already loaded up rather than having each child load all 
the packages themselves is such a huge win.   With the threaded approach each 
thread will have to load up and when hundreds are starting up simultaneously 
and repeatedly reading the directories along the search path and sourcing in 
hundreds or thousands of files each, it’s a big hit.

In a hybrid model if the master could load up a bunch of threads and forked 
children can somehow activate those threads where they don’t have to load up 
and the hybrid model is a performance win over forking or something, that’d be 
pretty cool.

Anyway, your plan of with threads having exit just exit the thread I think is 
really good.


> 
> -- Massimo
> 
> On 09/07/2015 12:15 AM, Massimo Manghi wrote:
>> Hi everyone
>> 
>> I'm CC'ing also Alexandre Ferrieux as his opinion on the subject is
>> authoritative.
>> 
>> In the wake of the ticket opened by Anton on core.tcl.tk we are facing
>> the need to change the way we handle child process (prefork MPM) and
>> thread (worker MPM) terminations. We are now recommended to avoid to
>> delete explicitly an interpreter when a child process goes away: it's OK
>> with the prefork bridge (and in the 2.2 module) but there are
>> implications for the threaded model.
>> 
>> With the worker MPM we implement our own exit command which shadows the
>> Tcl core command. This is needed by the sheer existence of the command
>> which I gather eventually calls Tcl_Exit which in turn tears the whole
>> child process down. In a threaded bridge like rivet_worker_mpm you would
>> have all the threads abruptly interrupted just because somewhere in a
>> different thread, likely running a different application, a programmer
>> placed an exit command. Hence our exit code simply tells the thread from
>> where it was called to exit, to notify the supervisor and clean
>> everything to avoid leaks (Tcl interpreter firstly)
>> 
>> If we want homogeneity between the two MPM models we should consider to
>> tell all the threads to gently exit and then let the child process exit
>> as well, I don't see any other sensible approach, even though the
>> overhead of having many threads exit and then restarted just as a side
>> effect of invoking exit is kind of disturbing to me
>> 
>> 
>>  -- Massimo
>> 
>> 
>> -
>> To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
>> For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
> 
> -
> To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
> For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
> 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: advise on migrating from neoweb to rivet

[Karl Lehenbauer]

Hi John,

That’s pretty awesome that you’re running Neowebscript.  I pretty much invented 
it and did a lot of the work on NWS back in the mid-1990s, so it’s a pleasure 
and a little bit of a surprise to see that someone is still using it.

The biggest difference between Rivet and Neowebscript is that Neowebscript was 
really designed for shared hosting while Rivet is more oriented toward a site 
that owns all of its content.

Rivet, however, is considerably more efficient, powerful, and evolved, though 
it still can support multiple independent sites on a single machine through its 
“separate virtual interpreters” mechanism.

By powerful, the difference is that the full power of Tcl is available to 
webpage authors, not the restricted version that NWS provides.  For instance, 
you can read or write any file in the system that you have permission to, 
require any package, open sockets and pipes, etc.

By efficient, the difference is that Tcl interpreters survive past the 
generation of a webpage and are reused again and again, reducing the overhead 
of generating a page.

By evolved, all of our efforts have gone toward Rivet for many years and so, 
you know, Rivet does more out of the box than Neowebscript did.

The trickiest part, in my opinion, is getting Rivet configured and running 
properly.  From the viewpoint of the webpage writer I think you’ll find that 
the principles are pretty much the same.

I haven’t looked at NWS in quite a while but I think a number of capabilities 
it provided were not ported to Rivet, like the key-value store, so you’ll need 
to do some rewriting or recreate the capabilities in Rivet.  Overall, though, 
once you get your head wrapped around Rivet I think you’ll agree it’s 
considerably improved.

Good luck and let us know how it goes.  I’ll be happy to answer more questions, 
although not always necessarily quickly.

Regards,

Karl



On Feb 26, 2014, at 4:27 AM, John j...@generic.ie wrote:

 Hi,
 
 
 I have a number of sites that are running Neowebscript on our apache servers 
 (freebsd) and wanted to look at migrating these to rivet. The clients are 
 quite happy with the current system and I have hardened the server as best we 
 can, but would like some pointers on this and did anyone else have any issues 
 with migration?
 
 
 Regards
 
 John
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

a load_response transaction-type thing

[Karl Lehenbauer]

This talk of new capabilities for a new release of Rivet has had me revisiting 
something I've been thinking about for quite a while...

It's really easy for developers to use forms in a way that they shouldn't, 
putting stuff in hidden fields, etc, that could easily be monkeyed with by an 
attacker.  That data needs to be kept server side and accessed with an opaque, 
unguessable session ID cookie or equivalent.  Yes, yes… clear, but not simple, 
and that friction helps to cause the observed problem.

It's also super common for developers to accept input from forms with 
insufficient checks on the validity of the input data, the source of SQL 
injection attacks and so forth.

I'm thinking of what you might call a response broker.  (This isn't an original 
idea -- I've read about stuff like this.)

Rather than doing a load_response, the page handling the response would 
explicitly name the fields you expect to get from the form, you invoke a proc 
specifying the fields you expect to find, their data types, optional code to 
validate them, and an array to stuff the validated fields into.

set wantVarList {{username string} {id integer} {uid check_routine 
validate_uid} {password check_routine validate_password} {hash base64} {email 
email}}

set status [response_broker $wantVarList response]

Since every page using the response broker would need to check for response 
broker parse failures it would probably be nice to be able to specify a general 
handler routine that would run and then abort the page, removing the need to 
check the return.

Now if I run…

response_broker $wantVarList response

…if the page continues then the response array would contain validated fields 
found in the form for the variables named in wantVarList and no others.  You 
might want the presence of unexpected fields to also blow out, but that would 
be likely to bite you pretty often when the reasons are harmless.  Maybe log 
them and include a {field ignore} option that will inhibit logging for 
expected-but-ignored fields.

We've done a form package at FlightAware that has a specific look and feel that 
allows you to specify both Tcl and Javascript validation code.  Of course you 
still need the Tcl code on the server but it's nice in the modern era to also 
provide Javascript validation on the browser.  We could probably open source if 
its appearance was genericized or something, but I mention it mainly to point 
out the usefulness of such an approach both for the developer and the users and 
a likely need to push Rivet into providing more stuff to support developers 
making modern websites.

Re: Scripts output oddity *SOLVED*

[Karl Lehenbauer]

Good work, Massimo.  Thank you.

On Nov 20, 2011, at 4:32 AM, Massimo Manghi wrote:

 I solved this problem with a quick review of the svn logs pertaining the
 code involved in the Rivet channel handling and rvt template parsing
 
 It was instant clear that none had changed the code in years
 and so I turned my attention to the rvt files in the test suite.
 It turned out they were supposed to have the final eol stripped off,
 whereas my editor reintroduced the eol when I changed the command
 names into their fully qualified form.
 The problem was tiny, the reason why and when an eol slid into those
 files is rather obscured in the darkness of svn, since there is no
 sign of modification in the logs.
 
 Only virtualhost.test seems to be in a bad state with a substantial
 mismatch between actual and expected outputs
 
 -- Massimo
 
 
 On 18.11.2011 11:49, Massimo Manghi wrote:
 I realized yesterday night that tests in rivet-namespace failed
 because of a spurious carriage return showed up in the channel
 output buffer.
 
 I verified the same character appears also with code in trunk but
 it cannot be a change in Tcl.
 
 Rivet scripts are made out of the concatenation of the BeforeScript,
 plus the script referenced in the URL, plus the AfterScript.
 
 The unexpected CR comes just after the output of the URL script, so
 either I accept it as is or I have to debug the Tcl channel internal
 buffer. Suggestions before I dive into this?
 
 -- Massimo
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Tcl_Mutex call

[Karl Lehenbauer]

I know Tcl's thread model is one interpreter per thread.  That seems like
a pretty good fit.  Is it even relevant?

On 7/29/11 1:16 PM, Damon Courtney da...@tclhome.com wrote:

I don't recall being the one to addd that code, but I would imagine we
need to go with Apache's model here.  We run inside Apache's world, for
the most part, so we should let that be the record of authority in cases
like these.

Damon


On Jul 29, 2011, at 12:39 PM, Massimo Manghi wrote:

 Hi guys
 
 I'm reviewing mod_rivet.c to understand what
 making it mpm-worker compliant is about. Currently mod_rivet
 fails  miserably when running with the threaded mpm,
 still this is an issue we should try to
 deal with.
 
 It seems someone (Damon?) laid down the code
 setting up Rivet to support threads by putting the
 content generation within a pair of
 Tcl_MutexLock-Tcl_MutexUnlock calls.
 I don't see any other purpose for this and there is
 no comment explaining the need for this. I
 
 This is not enought to let it work and anyway
 Rivet should use apr_* call to better suit the Apache
 threading model.
 
 Anyone can recall why these calls made it into the code?
 
 -- Massimo
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: abort_page, AbortScript and ForceAfterScript

[Karl Lehenbauer]

On Feb 15, 2011, at 5:18 PM, Massimo Manghi wrote:
 I thought for a while about the possibility of doing recursive handling of the
 abort_page generated error, but I thought in most cases the abort code passed
 to abort_page could help to cope with what required an interruption and there
 wasn't a compelling need for nested interruptions. Maybe I had it wrong, we'll
 see in the practical usage. Furthermore, coding it the way it is required
 little changes

I completely agree with your approach and I don't think it's necessary to try 
to do anything about recursive abort_page.  It's up to the Rivet developer to 
make sure their before and after scripts, rivet_error proc, etc, either work 
properly or reasonably handle when they don't.

Where Rivet callsTcl_Eval* in mod_rivet.c, if an error is returned, it either 
does special handling in the case of page processing (i.e. 
Rivet_ExecuteAndCheck) such as handling abort_page, invoking the error script, 
etc, or it logs the error in every other case.  I think that's prudent and 
appropriate and really about as much as we should try to do.

Beyond that, it's up to the developer to test and develop reasonable before and 
after scripts, etc.  FlightAware has a pretty sophisticated ErrorScript proc.  
It inserts uncaught tracebacks into a rivet_errors table in the database, along 
with other stuff like the user ID if logged in, the URL, webserver host, etc.  
But of course a lot of things can go wrong with that, like if things are really 
screwed up we might not even have a database connection.  So we have a dead 
simple error proc that wraps the fancy one in a catch and does the minimum if 
that dies.  In fact, here it is:

## NOTE ALSO: If flightaware_error_guts traces back, we exit the child process!
##
proc flightaware_error {} {
if {[catch flightaware_error_guts catchResult] == 1} {
catch {::bsd::syslog log error flightaware_error: TRACEBACK IN 
flightaware_error!!! - $::errorInfo}
catch {puts stderr TRACEBACK in flightaware_error! - $::errorInfo - 
exiting the child!}
exit 244
}
}

It might be a little radical to exit the Apache child if rivet_error traces 
back but our sense (and experience) is that, once it was debugged, a failing 
flightaware_error_guts meant things were screwed up enough that we didn't want 
Rivet trying to use the interpreter anymore.

Re: new scripts and variables for 'abort_page'

[Karl Lehenbauer]

It looks reasonable to me, Massimo.  I've circulated your mail on the
company dev list, soliciting feedback, and will follow up.

On 2/8/11 5:15 PM, Massimo Manghi massimo.man...@unipr.it wrote:


I'm doing some analysis on the scheme proposed by Karl for a new way to
handle
conditions raised by 'abort_page',

My proposal is to add 2 new configuration script and extend the globals
with
2 new status variables:

 - AbortScript: script that gets run if abort_page is called by the
content
generator or by AfterScript.
 
 - ForceAfterScript (or FinalAfterScript or whatever): it's run anyway
regardless the completion code of the script that ran before in the
request
process

 - Command abort_page will have an internal status flag 'aborting' to be
set
in the module globals. This flag is set to 0 at the very beginning of the
request processing (Rivet_SendContent). 'abort_page' tests this flag, if
0 it
gets set as 1 and the command returns TCL_ERROR setting the usual error
info
that are trapped in Rivet_ExecuteAndCheck.

Any subsequent call to the command will return TCL_OK disabling its
effects
within ForceAfterScript (just making sure that it cannot be interrupted in
this way).

- 'abort_page' will accept an argument to be later retrived with a new
command 'abort_condition'. AbortScript may check this condition variable
to
branch into different blocks of code.

Basically, the code in Rivet_ExecuteAndCheck where the abort condition is
checked

if (strcmp (Tcl_GetString (errorCodeElementObj), RIVET) == 0) {
 ...

}

should test the definition of a new conf variable rivet_abort_script and
execute it if defined. The whole function has to also check for the new
conf
variable rivet_force_script and run it anyway before printing the headers
and
flushing the channel.

thoughts?

-- Massimo


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

abort_page bypasses AfterScript...

[Karl Lehenbauer]

Hi Everyone...

We just noticed that AfterScript gets tagged onto the text of the webpage such 
that, if something in the page does an abort_page, there's no Rivet config 
script that's going to get a crack at the interpreter before it sits waiting 
for the next request to invoke it.

How abort_page works is its invokes a Tcl error with an errorCode of RIVET 
ABORTPAGE.  If the mod_rivet.c code that invokes Tcl_EvalObjEx then it bypasses 
the error handler and continues (Rivet_ExecuteAndCheck) with the flush and 
release.

It's also valid for the AfterScript itself to invoke abort_page, IMO.  We do 
that.

One possibility would be to invoke the AfterScript using Tcl_Eval in the 
abort_page path.  But that would break the AfterScript being able to invoke 
abort_page.  Also it would change the behavior every page that does an 
abort_page in a possibly bad way.  

No, I think AfterScript should stay the way it is.

If people agree that it should be possible to specify code in the 
configuration, an additional after script, that gets executed whether the page 
aborted or not.  Maybe it would be AbortScript (that's pretty simple).  But the 
code in the page shouldn't be able to do an explicit return and bypass this 
script either.  (I think a bare return in the page will bypass the 
AfterScript).  So it's a second After script, the after script of last resorts 
that's always guaranteed to execute whereas AfterScript is not.  
ForceAfterScript?  AlwaysAfterScript?



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: reintegrating branches/master-interp

[Karl Lehenbauer]

On 1/31/11 11:34 AM, Damon Courtney da...@tclhome.com wrote:

If Karl says it's ready and he's already using it in production, that's
good enough for me.  Push it.

Completely solid.

Also, I've been thinking maybe we need to enable SeparateVirtualInterps
by default.  I realize it's my own stupidity, but when developing in a
dev vs. production environment (usually on different virtual hosts), this
really hosed me up the other night until I figure it out.  Is there
really a good reason someone would WANT all the interps to serve every
domain?

443 / 80 needn't be handled by different interpreters and a big site might
have several names, like we also serve flightaware.co.uk, but the typical
user of Rivet with more than one vhost would probably want separate
interpreters and if they were big enough to need the other style they'd
figure it out by then.  Short answer, then, is it's OK with me.



I'm sure there are reasons why (multiple domains all hosting the same
site and content), but should that really be the default case?




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: reintegrating branches/master-interp

[Karl Lehenbauer]

On 1/31/11 1:36 PM, Damon Courtney da...@tclhome.com wrote:
Although, I'm curious.  I don't know exactly what the code says about
this, but does it handle virtual hosts based on the actual name of the
requested host or based on some Apache internals?  For example, if you
have:

VirtualHost *:80
ServerName foo.com
ServerAlias www.foo.com bar.com www.bar.com
/VirtualHost

Does that all show up to Rivet as a single virtual host, or is Rivet
determining that each of those domain names is a separate virtual host?
I would think they should all show up as a single virtual host and get
served the same interp, but I don't know.

Pretty sure with separate virtual interpreters you get one interpreter per
VirtualHost directive.



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Speaking of Tcl-US 2011

[Karl Lehenbauer]

On 1/27/11 10:56 AM, Damon Courtney da...@tclhome.com wrote:


On Jan 27, 2011, at 9:12 AM, Clif Flynt wrote:

 
 Would it be reasonable to do a compare-and-contrast panel at the Tcl
 conference?  Would anyone be willing to step up and defend their turf?

I'm hoping to go this year and drag Karl along.  He's quite a busy guy
though, so I don't know if he can leave his commercial endeavors for 3
days and do something academic. 0-]  If I can make it this year, I'd be
happy to sit in on a panel as Rivet rep.

I am quite interested.  In fact I think there are only about two broken
things in Tcl that if they were fixed it would be strong for many years to
come.  (And it's not coroutines and tail recursion.)  I would like to come
try to cause something around that.

I use Rivet for the big projects.  I like having Apache there where it
really counts.  I know it will get the job done and that despite problems
with my Tcl or maybe even problems in Rivet, Apache will run like a champ
under tremendous load and never let me down.  As for Rivet itself, the
nice thing about it is that it's pretty small when you look at it.

Yeah, Tcl / PostgreSQL / Rivet / Apache is industrial-strength.  No doubt
about it.  Add varnish in front of your Apache server and your capacity
for serving pages goes way up (things that can be cached such as images,
pages to non-logged-in users, etc, can be served repeated by varnish with
no heavyweight Rivet-enabled, database-enabled Apache backend involvement.
 Then hook up with a content delivery network and start generating URLs to
them referencing your cached content.  Now you can do millions of pages a
day.  Get a load balancer and start adding webservers.  Cluster your
database with Slony (and soon, native PostgreSQL 9) and start adding
database servers.  It scales.

That would probably be my paper... Building and running a Top 1000 Website
using Tcl


With all the work Massimo has done on the builds, Rivet compiles with
ease now.  The same could not be said of previous versions.  And, because
it's built on top of Tcl, you have powerful packages available for just
about everything you need.  Whenever I compile PHP, I always have to
figure out which of the 1000 --enable-X options to enable to get the
support I need built into the module.

Massimo has made a huge difference.  M



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Rivet namespace and Rivet package

[Karl Lehenbauer]

There is code at SF, available via svn co 
https://atwf.svn.sourceforge.net/svnroot/atwf atwf

But there is some problem with the repo…

svn: URL 
'https://atwf.svn.sourceforge.net/svnroot/atwf/application/modules/Module1' of 
existing directory 'atwf/application/modules/module1' does not match expected 
URL 'https://atwf.svn.sourceforge.net/svnroot/atwf/application/modules/module1'

http://sourceforge.net/projects/atwf/

last commit 46 days ago...

On Jan 27, 2011, at 8:12 AM, Massimo Manghi wrote:

 Arnulf's work is the only serious attempt I know to endow Tcl programming 
 with a consistent framework for web development. Still, the SF site has no 
 download to offer. Do you know if Arnulf moved it on to some other resource?
 
 -- Massimo
 
 
 On 01/27/2011 02:31 PM, Harald Oehlmann wrote:
 
 Maybe ATWF, the port of the zend framework to Rivet by Arnulf Wiedemann
 may be interesting?
 See:
 http://wiki.tcl.tk/25821
 or the presentation at TC2010.
 
 Harald
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 
   
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 

Re: Rivet namespace and Rivet package

[Karl Lehenbauer]

I like the name.  We are down with jQuery.  But it would need to support
PostgreSQL to get any traction at FlightAware.

You might have a look at yajl-tcl for generating valid JSON really
quickly.  The library includes returning PostgreSQL query results as JSON.
 It can parse, too, but the results are left-to-right so it's gotta be
taken further to be useful.

On 1/27/11 10:49 AM, Damon Courtney da...@tclhome.com wrote:

With this latest project of mine I actually started work on a framework
built solely around Rivet.  I'm not shooting for something that works
across web servers or something that is portable, I'm writing something
that takes advantage of all the things Rivet has to offer.  I call it
Skyscraper.

It kinda sucks right now because I've only really just started it, but
it's coming along.  It's based on many of the ideas used in Rails but in
a Tcl way.  When I looked at Ruby and Rails once before, there was a lot
to like about it.  Right now it's sort of hardwired for MySQL and jQuery
as the external components, but that's purely because it's what I'm
using.  The ties to jQuery at this point are not much more than just
including the tags to include it, and MOST of the SQL code is generic
with only a few MySQL-specific things.

It uses TDBC as the database backend, so it can easily support any
database Tcl does, and the rest is all straight Tcl or Rivet.  It doesn't
use TclOO though it probably could if someone had a strong feeling about
it.  I found when using Rails that the object stuff felt kinda' forced.
Like you didn't really need it, but because you're in Ruby, that's just
the way you do it.  Do you really need a new object for each request?
Isn't that what Rivet's ::request namespace is doing for you?

I'm open to hearing what people have to say on this topic.  I needed a
good MVC framework for writing Rivet apps, so I made one.  I'm up for
taking suggestions as I write it if anyone has any.  I hope to include it
as part of Rivet one day.  Unlike TclOO that was supposedly made to build
other OO frameworks on top of, I'd like to see Rivet ship with one
framework to rule them all and right out of the box.

D


On Jan 27, 2011, at 8:12 AM, Massimo Manghi wrote:

 Arnulf's work is the only serious attempt I know to endow Tcl
programming with a consistent framework for web development. Still, the
SF site has no download to offer. Do you know if Arnulf moved it on to
some other resource?
 
 -- Massimo
 
 
 On 01/27/2011 02:31 PM, Harald Oehlmann wrote:
 
 Maybe ATWF, the port of the zend framework to Rivet by Arnulf Wiedemann
 may be interesting?
 See:
 http://wiki.tcl.tk/25821
 or the presentation at TC2010.
 
 Harald
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 
   
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: master interpreter

[Karl Lehenbauer]

On Jan 23, 2011, at 6:09 PM, Massimo Manghi wrote:

 The code in branches/master-interp is now holding the code I modified with the
 master interpreter creation and initialization through the undocumented
 ServerInitScript directive. Children interps are supposed to be clones of the
 master interp and shouldn't need further explicit additional initialization
 (didn't test it in a complex environment though)

Hello Massimo,

This is fantastic news.  We will test it, starting tomorrow.  I am really 
excited about this.

Karl


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Rivet wish list

[Karl Lehenbauer]

On Jan 20, 2011, at 5:42 PM, Massimo Manghi wrote:
 A fork call copies data, stack and heap from the parent to the child process, 
 so in principle an interpreter state could be copied into the new process 
 (text pages are shared). What is missing in this picture for having a real 
 cloning?

I've looked at this a little further.  I think we would create the parent 
interpreter in the httpd parent process in Rivet_InitHandler(), which is 
invoked during http initialization because it was passed to 
ap_hook_post_config() by rivet_register_hooks().

At this point, the interpreter in the parent httpd process, we'd initialize it 
with the normal Tcl_Init-type stuff and then execute any global init scripts.  
Again, this all happens within Rivet_InitHandler when there is only the one 
httpd process running.

As the parent process forks off all the httpd children, each child has 
Rivet_ChildInit() called, as before, because Rivet_ChildInit was passed to 
ap_hook_child_init() by rivet_register_hooks().

Rivet_ChildInit() would be modified to use the interpreter created by 
Rivet_InitHandler(), unless separate virtual interpreters was defined, and then 
instead of doing all that interpreter creation stuff in Rivet_InitTclStuff(), 
like it does now, it would do the Apache stuff it needs to do and also execute 
the child init scripts on the interpreter it inherited from the parent when the 
parent forked it off.

Per-page processing would continue unmodified.



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Rivet wish list

[Karl Lehenbauer]

Greetings, Programs!

Here are a couple-three things I think would make a big difference for using 
Rivet to build and deploy big websites.

  1.  Named separate virtual interpreters.

We use separate virtual interpreters for development (for each developer we 
diddle auto_path to give them source-controlled private copies of all of our 
packages) and it's really badass.  The problem comes in the all-or-nothing 
approach of SVI.  We set up a virtual host for each developer, for both port 80 
(well really 8080 + varnish on 80) and 443.  This results in 18 interpreters in 
each httpd process on our development machine, making for very large httpd 
processes and slow startup time after a graceful.

If we could name the virtual interpreters, we could cut the number of 
interpreters from 18 to 9 in this case.

2. Virtual interpreter restarts without a graceful

Right now if one of our developers changes a package, private to them, they 
still have to do an apachectl graceful to pick up the change.  This restarts 
all of the httpd processes and reinitializes all of the interpreters.  Our 
interpreter initialization is intense.  Each FlightAware httpd process loads 
468 packages.

I'd like to be able to cause only one vhost's Tcl interpreters to be reloaded 
by a Tcl_DeleteInterp / Tcl_CreateInterp / Rivet initialization process.  
Instead of a graceful, you'd be able to specify something like a trigger file 
for each vhost.  Every time a vhost (with separate virtual interpreters) serves 
a page, it gets the mtime of the trigger file.  If the mtime of the trigger 
file has changed since the last time the interpreter served a page, Rivet 
deletes the virtual host's interpreter, creates and initializes a new one, and 
then handles the page.  [I tried to write this but kind of lost control of it 
and was not successful.]

This way, developers could totally reload all their libraries without any httpd 
processes being stopped or started.  Also this will lower overall overhead 
because a lot of times a httpd process won't have ever handled a page in its 
lifetime for many to most of the virtual interpreters.

An additional improvement would be to create the ability to not even initialize 
a vhost's separate virtual interpreter until the first time it is needed.

3. Apache children inheriting a preloaded interpreter from the parent Apache 
process

This is unrelated to separate virtual interpreters -- this is for the 
production website.  When we graceful a production webserver we take it out of 
the load balancer first.  When we start 200 httpd processes and each one loads 
468 packages, it's not pretty.  There is a lot of lock contention in FreeBSD 
while these processes all bang on the same package directories, something we 
don't totally understand.  (This is much faster with ZFS, once tuned, btw.)  It 
takes minutes to settle down before our stuff can put the server back into the 
webserver.  What would be incredibly cool would be to be able to load up the 
468 packages in the parent httpd process one time and then have each child 
process use the same Tcl interpreter already loaded with the packages.  This 
would be a way more than 200-fold improvement in Apache startup time for us 
(because it would eliminate all the contention.)  [I don't even know if this is 
possible.]

I offer this because it's like a possible direction to take Rivet development.  
I can help, definitely, and provide production load :-), if anyone is 
interested in taking any of these ideas to further design, code, etc, please 
let me know.

Karl

Re: form.tcl -method get|post?

[Karl Lehenbauer]

If the request is simply to weigh in, then I've got some rhymes for the house.

Regarding the supposed Itcl compatibility of Tcl 8.6, having to invoke methods 
in the same object with either $this or [self] is a non-starter for us 
(FlightAware) -- we have way too much Itcl code for that plus it just smells.  
Surely it's temporary.

Object systems let you build bigger programs without losing control of them.  I 
think it's a really natural idea -- code and data are defined together and you 
can have multiple instances of something with all the data for each instance 
nicely grouped and easily accessed.  Upvar is heavily (ab)used as a sort of 
pseudo-object system.  Look at tcllib for lots of examples.  For example with 
the http package, code accesses instances of data using upvar.  It works but 
it's better to support it in a first order and non-kludgey way, i.e. built into 
the language.

I think object-oriented typeless langauges (like Itcl, etc) work well.  You 
don't have to write different methods for all the possible datatypes like you 
do for C++ or Objective C.

I know form can do GET and POST.

On Oct 27, 2010, at 8:34 PM, Damon Courtney wrote:

 Just jumping in here.  I'm working pretty heavily in Rivet for the next few 
 months, so I will be updating little things and making changes a lot.  The 
 form package is due for some updating and a bit of an overhaul, I think.  I 
 know that Karl said they have a version that returns the form elements 
 instead of outputting them, and that's probably a good change.
 
 As far as I know, passing -method should accept anything.  There's not an 
 actual check on what is supported, and what is supported in the form tag is 
 up to your browser.  I know I ran into this recently.  I would do something 
 like:
 
 form #auto -method PUT
 
 And in viewing the source, it would say method=PUT, but when it was 
 submitted back to the server, PUT had been turned to POST (POST is the 
 default, by the way).  After some reading, I think this was the browser doing 
 it.  I couldn't find a better explanation.  So, I was trying to figure out 
 how the Rails guys do it since they use PUT and DELETE as methods in their 
 RESTful interfaces.  Turns out they fake it.  They just include a hidden 
 field that passes the method, and the Rails code on the backend knows to look 
 for this variable and set the method to that.  So, I went with that. 0-]
 
 I think all of our packages should be converted to TclOO if we can.  Though 
 the latest Itcl IS built on top of TclOO, so we're really just talking syntax 
 here.  I don't particularly care as long as we're talking TclOO in the 
 future.  I'm all for moving things forward to 8.6 with a newer version.  Not 
 necessarily dropping support for 8.5, just using 8.6 where we can.  That 
 would be one advantage to using Itcl syntax.  It remains backward compatible 
 with 8.5 and the old Itcl.
 
 Anyone else wanna' weigh in here? 0-]
 
 D
 
 
 On Oct 27, 2010, at 7:16 PM, Clif Flynt wrote:
 
 On Thu, Oct 28, 2010 at 12:38:29AM +0100, Massimo Manghi wrote:
 
 wouldn't '$this methodName' work out a solution?
 
 I won't swear to it, but I think I tried that first, and 'my cmd',
 and neither of them was happy with me.
 
 I've got a kludged version of form.tcl that works (to the extent
 that I've tested it) with the new 8.6.
 
 Is anyone interested in this code?
 
 
 yes, I am. 
 
 I've attached it to this email.  If attachments get scrubbed, let
 me know, and I'll mail it direct.
 
 It's something we should talk about. Tcl 8.6 is getting momentum among the
 Tcl'ers and its new introductions will probably set new standards in the
 language history.
 
 I'm pretty bullish on TclOO.  I think it's a nice mix of providing
 structure while still giving the developer the freedom to rework 
 things as his understanding of the problem matures.  (How's that for
 buzz-word compliance?)
 
 I worked in C++ for 5 or 6 years and did a little with itcl.  I ended
 up unthrilled with OO.  TclOO has revived my excitement.
 
 This construct in the form.tcl code was the only one that caused
 me some dismay at a quick-N-dirty translation to TclOO:
 
   public variable defaults  {
   upvar 1 $defaults array
   array set DefaultValues [array get array]
   }
 
 I think this can be implemented in TclOO with a variable trace and a
 new object method, but I didn't have time right then to play with it.
 
 Clif
 
 -- 
 ... Clif Flynt ... http://www.cwflynt.com ... c...@cflynt.com ...
 .. Tcl/Tk: A Developer's Guide (2nd edition) - Morgan Kauffman ..
 .. 17'th Annual Tcl/Tk Conference:  2010,   Oak Brook, IL  USA ..
 .  http://www.tcl.tk/community/tcl2010/  
 
 
 
 
 
 form8.6.tcl
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 
 
 

Re: New member

[Karl Lehenbauer]

Hi Clif,

Welcome to Rivet!

FlightAware uses Rivet extensively.  We use a CDN for performance and we also 
use Varnish to front-end our Apache servers, including using ESI includes where 
part of the content of a page can be cached by Varnish.  Behind it all sit four 
reasonably powerful 1U webservers behind a load balancer, all running Rivet, 
using a PostgreSQL cluster for the backend.  We serve something like 90 million 
pages a month to about 2.5 million users.

We made a slight variant of the forms package that returns the form rather than 
emitting it.  This is useful for, you know, assembling complex pages.

Come on in, the water's fine!

Karl

On Oct 26, 2010, at 11:40 AM, Clif Flynt wrote:

 Hi,
  I'm starting a fairly major project and intending to use Apache2,
 Rivet and the 8.6 tdbc and TclOO.  We're anticipating 1000+
 simultaneous users once the system goes live.
 
  I'll be beating on the form.tcl package fairly hard (already
 had one go-to with it).  I've got a version of form.tcl that's 
 happy (partly) with the 8.6 out-of-the-box version of Itcl.  This
 seemed to want a few more of the TclOO constructs (using [self] to
 call object internal methods.)
 
  Suggestions and comments are welcome.
 
  Clif
 
 -- 
 ... Clif Flynt ... http://www.cwflynt.com ... c...@cflynt.com ...
 .. Tcl/Tk: A Developer's Guide (2nd edition) - Morgan Kauffman ..
 .. 17'th Annual Tcl/Tk Conference:  2010,   Oak Brook, IL  USA ..
 .  http://www.tcl.tk/community/tcl2010/  
 
 
 
 
 
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: global nonsense

[Karl Lehenbauer]

Absolutely you can leave truly global variables laying around, it's valuable, 
and this is an important capability of Rivet.  We get around the problem you're 
talking about by using toplevel namespaces for the procs and variables of our 
packages.

We load up tons of truly global variables in our Rivet child init script, 
arrays that function as a cache for airport names, city names, aircraft types, 
all sorts of stuff.  We've moved to message catalogs for a lot of that stuff 
now, using speedtables and PostgreSQL, but we did it that way for years and 
still have lots of global stuff that pages expect and rely on.

Anyway...

So if you package require foo then the foo package is wrapped in a namespace 
eval ::foo {} and it won't recompile any compiled procs for subsequent pages.  
We don't put procs in the .rvt pages, that is we try to avoid it if there is 
any reusability possible at all, it goes into a package.

When we want real global variables we use the ::var notation or we invoke 
::global instead of global.

I think it needs to stay the way it works now and the way it works is fine and 
you just need to scale up your approach by loading all your reusable stuff 
outside of ::request, even pulled in in your child init script but that's not 
required.

As long as you use packages and the stuff is in namespaces, it's all money.  
And tcllib and all that stuff work that way, most stuff works that way, so you 
don't get screwed by third party packages very much either.

There's also that rivet cache dealie where you can say oh cache the last 200 
compiled pages, so you might look at that too.  Also I bet you can define your 
procs globally with something like proc ::foo if you want to just shotgun it.

On Sep 9, 2010, at 7:06 PM, Damon Courtney wrote:

 So, way back when we first wrote Rivet, we had this bright idea to make a 
 ::request::global command that automatically forced global variables into 
 the ::request namespace so that users could say
 
 global foo
 
 And get what was EFFECTIVELY a global variable for their request but without 
 actually setting something in the global namespace that would hang around 
 between requests.  It seemed like a good idea at the time. 0-]  Now that I'm 
 doing more Rivet development, this is looking like maybe a bad idea.  Not 
 that we need to take out what is already there (it's already been in use for 
 a while), but I think we need something better and a little more specific to 
 what we're doing.
 
 My problem is this.  I have a source file of code that is getting sourced 
 every time a request comes in, and it's re-compiling my procs every time 
 because the request namespace got blown away from the last request.  Instead, 
 I want to make my procs in the global namespace but with the ability to scope 
 variables from the ::request namespace so that the procs exist but never 
 actually operate at #0.
 
 I could easily do something like:
 
 variable ::request::foo
 
 within my procs, but that is assuming knowledge of Rivet and knowing that 
 we're operating within a namespace called ::request.  I would prefer not to 
 do that.  So, my proposal is either to rename Tcl's global out of the way and 
 just make our global hack be the default global, or we create a new command.  
 Something like reqvar (for a request variable) or something.
 
 I'm perfectly happy to just rename global to something like ::tcl_global and 
 be done with it.  In reality, when the hell would you EVER want a truly 
 global variable to be set from your request?  There's absolutely NO guarantee 
 you're ever going to see that variable again, so it seems silly to even allow 
 it.  If you REALLY had a need for it, you could just as easily use ::foo and 
 get a globally-scoped variable without the command.
 
 Any thoughts or opinions?
 
 D
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: bugs in session package postgres support

[Karl Lehenbauer]

Looks good, Massimo.  Is that all that's needed?

On Jul 10, 2010, at 12:39 PM, Massimo Manghi wrote:

 Even though a vote has been held on the pmc list, with a general consensus to 
 release Rivet 2.0.1 I think these two bugs should make it into the new 
 release, as their impact is such that Session package wouldn't work for a 
 Postgres installation. I will ask the pmc list members to swiftly express a 
 new vote and then release this time. These are the bugs
 
 
 1) the rivet_session_cache were created with key_ and package_ columns still 
 named 'key' and 'package'. These columns had been renamed because  Mysql and 
 Oracle (iirc) consider these as keywords,therefore they are reserved words. 
 session-create.sql needed to be changed accordingly
 
 2) Suprisingly also rivet/packages/dio/dio_Postgresql.tcl had a typo in a 
 variable name causing method 'store' to fail
 
 Index: rivet/packages/session/session-create.sql
 ===
 --- rivet/packages/session/session-create.sql   (revision 961366)
 +++ rivet/packages/session/session-create.sql   (working copy)
 @@ -16,8 +16,8 @@
 
 create table rivet_session_cache(
 session_id varchar REFERENCES rivet_session(session_id) ON DELETE 
 CASCADE,
 -packagevarchar,
 -key varchar,
 +package_   varchar,
 +key_varchar,
 datavarchar,
 
 UNIQUE( session_id, package, key )
 Index: rivet/packages/dio/dio_Postgresql.tcl
 ===
 --- rivet/packages/dio/dio_Postgresql.tcl   (revision 961366)
 +++ rivet/packages/dio/dio_Postgresql.tcl   (working copy)
 @@ -138,7 +138,7 @@
# did, put that in in place of numrows, hiding a postgresql
# idiosyncracy from DIO
if {$numrows == 0} {
 -   set cmdrows [pg_result $resultId -cmdTuples]
 +   set cmdrows [pg_result $resultid -cmdTuples]
if {$cmdrows != } {
set numrows $cmdrows
}
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Rivet 2.0.0

[Karl Lehenbauer]

That's awesome, Arnulf!  Congratulations…

On May 2, 2010, at 1:21 PM, Arnulf Wiedemann wrote:

 Am Sonntag 02 Mai 2010 14:33:57 schrieb Arnulf Wiedemann:
 Hello,
 I have downloaded Apache 2.2.15 and Rivet 2.0.0 and installed them on my
 system (SUSE Linux 11.2)
 
 Thanks for finally releasing 2.0.0
 
 In general that works together. I am using Tcl 8.6b1.2 and Itcl 4.0b4 cvs
 version.
 
 I can do a few statements including creating an itcl class and object and
 calling it, but when using a package with more classes and objects I get a
 segmentation violation. Calling the same package on the command line with
 tclsh8.6 makes no problems.
 
 Has anybody else the same problem and how can I debug apache with rivet?
 
 Best regards
 Arnulf Wiedemann (apw on tcl wiki)
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 
 
 
 I could fix the problem myself after finding out the gdb httpd with: run -X 
 allows to debug apache.
 
 Problem was a bug in itclng package, which I could fix and now the 
 combination 
 is working as expected.
 
 Arnulf
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

RPM created for Rivet -- some questions

[Karl Lehenbauer]

Hey Gang,

Jeff Lawson aka Cow has made an RPM of Rivet and has 
rpmforge.nethttp://rpmforge.net, the largest third-party RPM repo for 
CentOS/Fedora/RHEL (and other RHEL-based) distros, willing to carry it.  The 
issue is right now we only have the snapshots.  They need tarballs to stay 
around longer... weeks, at least.

I'm not familiar with the resources that are available to the project or much 
of the release engineering.  Is there an official place we can lodge tarballs 
to solve this issue?

Regards,
-karl

http://lists.rpmforge.net/pipermail/packagers/2010-April/001177.html

http://svn.rpmforge.net/viewvc/rpmforge/trunk/rpms/mod_rivet/

my apache creds don't work

[Karl Lehenbauer]

I know there was a breakin and it's been a while since I've done anything.  Do 
any of you guys know what the score is with our creds or what we need to do to 
get back online with them?


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: RPM created for Rivet -- some questions

[Karl Lehenbauer]

Yeah we're on 2.2.

On Apr 15, 2010, at 3:15 PM, David Welton wrote:

 It's been so long that I've forgotten how to go about things
 officially.  I know we have to vote on it...
 
 I'd still be a little bit concerned about the 2.X support.  Are you
 guys using that at flightaware?
 
 -- 
 David N. Welton
 
 http://www.welton.it/davidw/
 
 http://www.dedasys.com/
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: Who maintains Rivet

[Karl Lehenbauer]

There are a few of us.  Every single page on flightaware.com is generated with 
Rivet, and we are growing and a top 250 website.

On Apr 12, 2010, at 1:21 PM, Thomas Lloyd wrote:

 Hi List,
 
 
 Who currently maintains Rivet if anyone. Do people have an opinion of
 how sustainable Rivet is into the future?
 
 Tom
 
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: More..

[Karl Lehenbauer]

OK... you need to get the thing to dump core.  What operating system are you 
using?  After you get it to dump core you can fire up gdb on httpd and the 
coredump and execute a bt (backtrace) command and paste the contents in an 
email and I'll probably be able to figure it out without too much trouble.

Here are my notes on how to get Apache to dump core:


Edit this page
Create a new page
Delete this page
Getting Coredumps From Apache
Configure Apache to Allow Coredumps
It is occasionally desirable to get Apache to dump core when it gets something 
like a segmentation fault.  (By default it does not dump core.)  By getting it 
to dump core, gdb can usually be used to quickly determine the cause of the 
fault.

To tell Apache to save coredumps, use the CoreDumpDirectory directive in the 
httpd.conf.

CoreDumpDirectory /tmp/apache

The directory must exist and the Apache process must be able to write to it, so 
if you make it a subdirectory of /tmp, make sure to create it and to chmod a+w 
on it.

Note that if every child is trapping then /tmp can fill up pretty fast.  Watch 
for that and after you get a few, do an apachectl stop or something.

Configure FreeBSD to Allow SUID/SGID Programs to Dump Core
BUT it still doesn't work?

Well, FreeBSD has this uh feature (no, it makes sense, but it's a pain) where 
it won't by default generate coredumps for processes that are set user ID 
and/or set group ID...  You have to tell the kernel to allow that, as follows:

sysctl kern.sugid_coredump=1

Configure FreeBSD to Add Process ID to Coredump Filenames
Next problem is you only get one httpd.core file since all of the httpds that 
are dying are writing to that file.  You can tell FreeBSD to include the 
process ID in the coredump filenames...
sysctl kern.corefile=%N.%P.core


On Mar 11, 2010, at 9:17 AM, Tom Lloyd wrote:

 There are numerous messages like this in my apache error log. 
 
 [Wed Mar 10 22:33:37 2010] [notice] child pid 23855 exit signal Segmentation 
 fault (11)
 
 I am a bit lost ;)
 
 -
 To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
 For additional commands, e-mail: rivet-dev-h...@tcl.apache.org
 

trouble producing a working configure script

[Karl Lehenbauer]

I am having a hellacious time getting a working configure script for  
Rivet.  If I run aclocal (1.10), it gives a ton of errors complaining  
about stuff in tclconfig/libtool.m4.


OK, so libtool.m4 is old.  I copied a newer version out of libtool.

I then ran aclocal, and libtoolize --force --copy to install new  
config.guess, config.sub and ltmain.sh in tclconfig dir.


Ran autoheader, automake, autoconf.  Cool.

Then try to run configure, it barfs...

./configure: line 2747: syntax error near unexpected token  
`rivet_config.h,rivet'

./configure: line 2747: `AX_PREFIX_CONFIG_H(rivet_config.h,rivet)'

This is because AX_PREFIX_CONFIG_H isn't defined.

It's defined in m4/ax_prefix_config_h.m4 but I haven't found anywhere  
where that file is getting pulled in.


I have also pulled in newer Tcl Extension Architecture (TEA) support  
files into tclconfig.sh and edited configure.ac to reference the  
slightly newer version of TEA (3.7 versus 3.6).


Mostly probably I just need to get that AX_PREFIX_CONFIG_H macro  
pulled in somehow.


Any suggestions?

-karl


-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: coredumps on shutdown with separate virtual interps

[Karl Lehenbauer]

Massimo,

Apology accepted and no worries.  What was hard about it was trying to  
debug the channel driver malfunction that was innocent but was  
implicated as a coincidence of trying to execute something while  
tearing down an interpreter that was already being torn down.


The bug itself was actually pretty cool, and I learned a lot about  
Rivet while debugging it, so I didn't feel like the time was wasted.


Karl

On May 8, 2009, at 10:30 AM, Massimo Manghi wrote:


I read the log of your last commit and it looks like I introduced
this bug because I missed the implications of having the SVI
switch turned on. I apologize.

-- Massimo

Karl Lehenbauer wrote:
Separate virtual interpreters are amazingly cool.  Using them,  
fiddling with auto_path, and using git we're able to give each  
developer their own Rivet interpreters with a private library path  
that gets searched first, their own webpages, and so forth.  It's  
great.


But we're having a problem that when we do an apachectl stop all of  
the children that have served webpages dump core.  The problem is  
that outputproc is getting called by the flush activities triggered  
by the call to Tcl_Finalize() as part of the shutdown.   There  
isn't really a request structure so it quickly dereferences a null  
pointer and that triggers a coredump.


Doing some debugging, I've determined that the Tcl channel that  
Rivet maps to Apache's ap_rwrite and such, its close routine is  
getting called prior to the attempt to flush this data.


What's weird is that the data that it's trying to write is the same  
as the last webpage the child generated.


I'm continuing to investigate because I want to know why it's  
happening, although it's also reasonable to make the closeproc set  
a closed flag that the outputproc would honor.  The outputproc  
could also very easily detect that the request structure isn't  
real, but the big question is why is it getting called in the first  
place.


I mention it here in the hopes that someone will have a flash of  
insight and/or knows the answer.


Karl

-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org






-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

Re: coredumps on shutdown with separate virtual interps

[Karl Lehenbauer]

Correct.  It is fixed and we're using separate virtual interpreters to  
give each developer their own webroot and packages.  It's all  
integrated with git and we're really digging it.


On May 7, 2009, at 4:31 PM, David Welton wrote:


Looks like you found a solution, according to the commit logs?

--
David N. Welton

http://www.welton.it/davidw/

http://www.dedasys.com/
Sent from Padova, PD, Italy

-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org




-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

coredumps on shutdown with separate virtual interps

[Karl Lehenbauer]

Separate virtual interpreters are amazingly cool.  Using them,  
fiddling with auto_path, and using git we're able to give each  
developer their own Rivet interpreters with a private library path  
that gets searched first, their own webpages, and so forth.  It's great.


But we're having a problem that when we do an apachectl stop all of  
the children that have served webpages dump core.  The problem is that  
outputproc is getting called by the flush activities triggered by the  
call to Tcl_Finalize() as part of the shutdown.   There isn't really a  
request structure so it quickly dereferences a null pointer and that  
triggers a coredump.


Doing some debugging, I've determined that the Tcl channel that Rivet  
maps to Apache's ap_rwrite and such, its close routine is getting  
called prior to the attempt to flush this data.


What's weird is that the data that it's trying to write is the same as  
the last webpage the child generated.


I'm continuing to investigate because I want to know why it's  
happening, although it's also reasonable to make the closeproc set a  
closed flag that the outputproc would honor.  The outputproc could  
also very easily detect that the request structure isn't real, but the  
big question is why is it getting called in the first place.


I mention it here in the hopes that someone will have a flash of  
insight and/or knows the answer.


Karl

-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

interpreter-per-vhost problem; Rivet_Panic problem

[Karl Lehenbauer]

I'm trying to turn on SeparateVirtualInterps and getting a coredump.   
Debugging that.  Getting a Tcl_Panic that's calling out to Rivet_Panic  
due to attempt to append to a shared object.


The Rivet_Panic is coredumping because it's dereferencing globalrr,  
which is null because it hasn't served a request yet.  I think the  
init stuff needs to set a global server rec pointer but am looking at  
what it would use for a pool.  Anyway this needs to be fixed so that  
panics in module initialization will cause error messages rather than  
coredumps.  I expect to have something soon.






-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

interesting problem in apache-2/mod_rivet.c when .rvt file doesn't exist

[Karl Lehenbauer]

In Apache 1.3, if a .rvt file doesn't exist, it's a 404 plain and  
simple.


But in Apache 2, if a .rvt file doesn't exist, module keeps going.   
The problem in in src/apache-2/mod_rivet.c, in Rivet_SendContent.  It  
checks for r-finfo.st_mode in 1, r-finfo.protection in 2.  It  
assumes protection will be 0 if the file doesn't exist, and it appears  
to be documented that way in Apache, but if I log the protection value  
right before the check, it's not 0, and this causes the file-not-found  
path to not get followed.


Still looking...



-
To unsubscribe, e-mail: rivet-dev-unsubscr...@tcl.apache.org
For additional commands, e-mail: rivet-dev-h...@tcl.apache.org

upload data and upload channel problem solved / fixed for Apache 2

[Karl Lehenbauer]

This was tricky.

Apache has its own portable I/O library that apparently is just a FILE  
* in Apache 1.3 but is an Apache-defined C structure in 2.2.  Rivet  
made some assumptions about it that it shouldn't have, like that it  
could use the fileno() library call to get the UNIX file descriptor.


I've got a fix working such that upload channel works with Apache 2  
and upload data which has not been working in Apache 1.3 or 2.  I've  
also added a new subcommand, upload tempname, to get the name of the  
temporary file that contains the uploaded file for those who want to  
roll their own.


In addition to working, it is now system independent without any  
special code, like it'll work on NT and stuff.


The cost is that it opens the temp file that contains the uploaded  
file, rather than grabbing the file handle out of the Apache portable  
runtime file structure.  There appears to be no legal way to grab  
the file descriptor out of the apr_file_t structure, so I think it's a  
small cost in overhead for something that's intrinsically relatively  
high overhead anyway (uploading files), and IMHO it's worth it to use  
the Apache file API in a standard manner.


With your agreement I'd like to commit these changes.

By the way, the raw_post command doesn't seem to work.

Apache Notes table, Apache tables in general, and Rivet

[Karl Lehenbauer]

There's a key-value Apache table called notes that's part of  
request_rec structure.


It's used to communicate various stuff between modules.

We came across a need to be able to interface to it from Tcl.  We want  
to stuff the authenticated username into Apache logs using the % 
{varname}n option of CustomLog formatting.  To do that we'd stuff a  
note with the username and then invoke that note's name in the log.


There are various bits of code to interface to certain tables defined  
in the request_rec: headers_in, headers_out, and err_headers_out.  Not  
currently used are subprocess_env and notes.


I propose adding a single new C command to Rivet/Tcl to provide  
interfaces to all these tables for getting, setting and inventorying  
their contents.


Something like

   table get tablename key
   table set tablename key value
   table unset tablename key
   table list tablename

I know there is some ability to support multiple entries with the same  
key.  Right now I'm punting on that but I suppose it might need that.   
Perhaps


   table add tablename key value

Anybody have any thoughts, concerns, objections, accolades, remarks,  
hot buttons, flames, etc?


Really I only need an interface to notes but I figured subprocess_env  
could be a good one to be able to mess with as well, at least.   
Conceivably we could eventually simplify by using a more uniform  
interface at the C level as well.  I've got it coded but haven't  
tested it yet.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: DIO buglet

[Karl Lehenbauer]

You're right.  There's thing thing about DIO where it does an

upvar $arrayName $arrayName $arrayName array

...and I'm not totally sure why... it's some sort of Damon magic.

But yeah, that's the problem, any array name that's also a variable  
inside the method or a public variable of the class and it's gonna fail.


I've been working on DIO.  It's fairly high overhead, creating an  
object for every result, and it pulls several result variables across  
to the result object whether they're requested or not... stuff like  
that.  I've been toying with the idea of trying to reuse result  
objects due to the overhead of Itcl object creation.  Also  
occasionally studying Xotcl since that's the forthcoming official  
object system.


One thing that makes it difficult is I'm a PostgreSQL guy and not a  
MySQL/SQLite guy.  A recent case in point, I want the DIO update  
method to return the number of rows affected, but it's a different  
result method, pgresult -cmdTuples rather than pgresult -numTuples,  
and I don't know how the others handle that.


On Nov 9, 2005, at 12:53 PM, David N. Welton wrote:


package require DIO

set dbh [DIO::handle Postgresql dbh -db shopping]

$dbh array select id, code from codes res

puts $res(code)

This barfs because 'res' is a magic name.  I don't mind picking  
another

name, but it's a minor annoyance.

Ciao,
--
David N. Welton
- http://www.dedasys.com/davidw/

Linux, Open Source Consulting
- http://www.dedasys.com/

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: macosx build

[Karl Lehenbauer]

On Sep 23, 2005, at 6:42 AM, David N. Welton wrote:I saw you reverted the MacOS X build instructions to use aardvark... Youdon't think it is possible to make it work 'correctly' ?Oh it's possible, it just doesn't build mod_rivet with the correct flags, and I felt like it was better to have instructions there that work rather than ones that say it's normal and then have it not work.It needs to use "cc -dynamiclib -undefined suppress -flat_namespace" -- maybe libtool can't be coaxed into doing that, I don't know.  Maybe apxs can shed some light on the situation.Preparing for the hurricane today (just got back from NYC last night -- took an earlier flight which turned out to be smart as they're closing the airport today at noon)... deciding whether to evacuate or not, moving stuff around at the office, etc.Karl

Rivet version number

[Karl Lehenbauer]

IMHO there is zero reason for Rivet to have a release number  1.0.   
Considering how solid it is and how heavily my company and some  
nonprofits I support are using it, I think a 2.0 number is in order.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]