Streaming files
In Java servlets, I can stream a file back to the browser one chunk at a time. This has 2 benefits which interest me. 1) Files can be stored outside the web root so users cannot download them unless they are logged in, even if they know the path. 2) Large files can be streamed back to the client without having the entire file loaded into memory at the same time How would you recommend achieving similar functionality in mod_perl? -- John Dunlap *CTO | Lariat * *Direct:* *j...@lariat.co * *Customer Service:* 877.268.6667 supp...@lariat.co
Re: Streaming files
One fairly straight-forward approach would be to write a script that serves as the path for downloads, then have it parse the filename to use as a key in determining which content to send to the user. (The AcceptPathInfo directive could be helpful for this, especially if you want to the script to appear as a subdirectory to the user.) That script could perform all the necessary security checks that you need, and read portions of the file to be streamed as needed (the streaming protocol implementation details would also need to be handled by your script, and you might find it helpful to look at what's available in CPAN for the streaming protocol you want to use). > In Java servlets, I can stream a file back to the browser one chunk at a > time. This has 2 benefits which interest me. > 1) Files can be stored outside the web root so users cannot download them > unless they are logged in, even if they know the path. > 2) Large files can be streamed back to the client without having the entire > file loaded into memory at the same time > > How would you recommend achieving similar functionality in mod_perl? > > -- > John Dunlap > *CTO | Lariat * > > *Direct:* > *j...@lariat.co * > > *Customer Service:* > 877.268.6667 > supp...@lariat.co > Randolf Richardson - rand...@inter-corporate.com Inter-Corporate Computer & Network Services, Inc. Beautiful British Columbia, Canada https://www.inter-corporate.com/
Re: Streaming files
You could put the files behind Nginx and use your mod_perl web app as an external authentication service. I've never done it but it doesn't look too hard. On Fri, Aug 16, 2019, 12:25 PM John Dunlap wrote: > In Java servlets, I can stream a file back to the browser one chunk at a > time. This has 2 benefits which interest me. > 1) Files can be stored outside the web root so users cannot download them > unless they are logged in, even if they know the path. > 2) Large files can be streamed back to the client without having the > entire file loaded into memory at the same time > > How would you recommend achieving similar functionality in mod_perl? > > -- > John Dunlap > *CTO | Lariat * > > *Direct:* > *j...@lariat.co * > > *Customer Service:* > 877.268.6667 > supp...@lariat.co >
Re: Streaming files
1. Streaming in chunks has nothing to do with security - you should have proper authentication and authorization layer wrapping these requests. 2. Handling chunks and large files are things which apache has perfected and we usually let apache do the nitty gritty. After all the validations are complete I just do return $r->lookup_file($file)->run(); - lookup_file is provided by Apache2::SubRequest On Fri, Aug 16, 2019 at 1:37 PM Dan Ullom wrote: > You could put the files behind Nginx and use your mod_perl web app as an > external authentication service. I've never done it but it doesn't look too > hard. > > On Fri, Aug 16, 2019, 12:25 PM John Dunlap wrote: > >> In Java servlets, I can stream a file back to the browser one chunk at a >> time. This has 2 benefits which interest me. >> 1) Files can be stored outside the web root so users cannot download them >> unless they are logged in, even if they know the path. >> 2) Large files can be streamed back to the client without having the >> entire file loaded into memory at the same time >> >> How would you recommend achieving similar functionality in mod_perl? >> >> -- >> John Dunlap >> *CTO | Lariat * >> >> *Direct:* >> *j...@lariat.co * >> >> *Customer Service:* >> 877.268.6667 >> supp...@lariat.co >> >
Weird issues using DBI + mod_perl
So I’ve built a site using mod_perl in CGI mode, with DBI and Oracle as my database back end. Everything works fine for a while but then I get intermittent weird errors that start happening for every page load, which is resulting in non-functional pages and error messages “Something happened contact your system administrator” "DBD::Oracle::st execute failed: ORA-01008: not all variables bound (DBD ERROR: OCIStmtExecute) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual”]” at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 267, line 522.\n There are no bound variables in the query... Another one: [error] DBD::Oracle::st execute failed: ORA-01007: variable not in select list (DBD ERROR: OCIStmtExecute) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual"] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 267, line 522.\n Other errors that happen are : DBD::Oracle::st execute failed: ORA-01007: variable not in select list (DBD ERROR: OCIStmtExecute) [for Statement "select pid, cn, email, afflist, bldg_id, start_task from login_info where cookie =?" with ParamValues: :p1='NS9DdciuH7XPeSVygRyUjoviZ’] DBD::Oracle::db prepare failed: ORA-01002: fetch out of sequence (DBD ERROR: OCIStmtExecute/Describe) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual"] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 266, line 522.\n DBD::Oracle::st execute failed: ORA-03106: fatal two-task communication protocol error (DBD ERROR: OCIStmtExecute) [for Statement "select pid, cn, email, afflist, bldg_id, start_task from login_info where cookie =?" with ParamValues: :p1='v2bpY8jnoPw5yf0x71I4wZPx6'] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 307 Most of these errors (that cause the DBI stuff to break) are happening in this defaults module, which IS referenced more than once per page load as the pages have ajax calls to other scripts that reference this module. restarting httpd fixes the problem, for a while. This isn’t a very heavily loaded site, right now it’s the middle of the afternoon and I'm seeing 'accesses per minute' kind of traffic. And clues as to where I should start to look? -- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group Institutions do not have opinions, merely customs
AW: Weird issues using DBI + mod_perl
Hi Bruce, I'm just guessing, but this sounds much like reusing or double using of Oracle db handles, context handles or statement handles. You have to have a look at the bookkeeping of the handles per request. Also forking of the childs may byte you. You must ensure that db handles are ONLY opened in the child process doing the work and get closed properly. With DBI in use you can use the variable DBI_TRACE to get extensive log. Google for that in combination with mod_perl. Just some hints. Best regards Andreas -Ursprüngliche Nachricht- Von: Bruce Johnson Gesendet: Freitag, 16. August 2019 23:36 An: dbi users ; mod_perl list Betreff: Weird issues using DBI + mod_perl So I’ve built a site using mod_perl in CGI mode, with DBI and Oracle as my database back end. Everything works fine for a while but then I get intermittent weird errors that start happening for every page load, which is resulting in non-functional pages and error messages “Something happened contact your system administrator” "DBD::Oracle::st execute failed: ORA-01008: not all variables bound (DBD ERROR: OCIStmtExecute) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual”]” at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 267, line 522.\n There are no bound variables in the query... Another one: [error] DBD::Oracle::st execute failed: ORA-01007: variable not in select list (DBD ERROR: OCIStmtExecute) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual"] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 267, line 522.\n Other errors that happen are : DBD::Oracle::st execute failed: ORA-01007: variable not in select list (DBD ERROR: OCIStmtExecute) [for Statement "select pid, cn, email, afflist, bldg_id, start_task from login_info where cookie =?" with ParamValues: :p1='NS9DdciuH7XPeSVygRyUjoviZ’] DBD::Oracle::db prepare failed: ORA-01002: fetch out of sequence (DBD ERROR: OCIStmtExecute/Describe) [for Statement "select to_char(add_months(sysdate, 24), 'MM/DD/'), to_char(add_months(sysdate, 24), 'J') from dual"] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 266, line 522.\n DBD::Oracle::st execute failed: ORA-03106: fatal two-task communication protocol error (DBD ERROR: OCIStmtExecute) [for Statement "select pid, cn, email, afflist, bldg_id, start_task from login_info where cookie =?" with ParamValues: :p1='v2bpY8jnoPw5yf0x71I4wZPx6'] at /home/allwebfiles/perl/LocalModules/Cal4Defaults.pm line 307 Most of these errors (that cause the DBI stuff to break) are happening in this defaults module, which IS referenced more than once per page load as the pages have ajax calls to other scripts that reference this module. restarting httpd fixes the problem, for a while. This isn’t a very heavily loaded site, right now it’s the middle of the afternoon and I'm seeing 'accesses per minute' kind of traffic. And clues as to where I should start to look? -- Bruce Johnson University of Arizona College of Pharmacy Information Technology Group Institutions do not have opinions, merely customs
