Re: [racket-users] Handin Issues

[mattsap]

On Tuesday, October 11, 2016 at 12:42:44 PM UTC-4, Eli Barzilay wrote:
> On Tue, Oct 11, 2016 at 9:35 AM,   wrote:
> > Thanks for your detailed response!
Thanks! If I get any progress, I'll try to update later.

> >
> > I do have the following additions at the top of the file:
> > (add-header-line! (get-submission-timestamp))
> > (add-report-line! (get-submission-timestamp))
> >
> > So would this mean there isn't an issue with back-end but rather my
> > checker.rkt file?
> 
> Yeah: you cannot add header lines when the submissions are not converted
> to text, because there's no functionality to modify the rich media file.
> But it would be better if the checker would complain about an invalid
> configuration instead of failing when the text file is not there --
> which is the change that I suggested.  This way you'd get a readable
> error message telling you what's wrong instead of tracking the problem
> youself.
> 
> 
> > Would it be possible to link some example handin servers on a
> > github/bitbucket/or even the racket website (particularly a
> > checker.rkt file)?  I think my main issue is there are alot of
> > configurations that require particular details and I may miss some
> > important documentation. I could help contribute a checker.rkt for my
> > purposes after I get it fully working... [...]
> 
> (Unfortunately, most of your *real* issues are more high level problems
> with how to handle submissions with images, how to test them, etc -- and
> that's something that personally I have no experience with...)
> 
> -- 
>((x=>x(x))(x=>x(x)))  Eli Barzilay:
>http://barzilay.org/  Maze is Life!

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Handin Issues

[Eli Barzilay]

On Tue, Oct 11, 2016 at 9:35 AM,   wrote:
> Thanks for your detailed response!
>
> I do have the following additions at the top of the file:
> (add-header-line! (get-submission-timestamp))
> (add-report-line! (get-submission-timestamp))
>
> So would this mean there isn't an issue with back-end but rather my
> checker.rkt file?

Yeah: you cannot add header lines when the submissions are not converted
to text, because there's no functionality to modify the rich media file.
But it would be better if the checker would complain about an invalid
configuration instead of failing when the text file is not there --
which is the change that I suggested.  This way you'd get a readable
error message telling you what's wrong instead of tracking the problem
youself.


> Would it be possible to link some example handin servers on a
> github/bitbucket/or even the racket website (particularly a
> checker.rkt file)?  I think my main issue is there are alot of
> configurations that require particular details and I may miss some
> important documentation. I could help contribute a checker.rkt for my
> purposes after I get it fully working... [...]

(Unfortunately, most of your *real* issues are more high level problems
with how to handle submissions with images, how to test them, etc -- and
that's something that personally I have no experience with...)

-- 
   ((x=>x(x))(x=>x(x)))  Eli Barzilay:
   http://barzilay.org/  Maze is Life!

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Handin Issues

[mattsap]

Thanks for your detailed response!

I do have the following additions at the top of the file:
(add-header-line! (get-submission-timestamp))
(add-report-line! (get-submission-timestamp))

So would this mean there isn't an issue with back-end but rather my checker.rkt 
file?

Would it be possible to link some example handin servers on a 
github/bitbucket/or even the racket website (particularly a checker.rkt file)?  
I think my main issue is there are alot of configurations that require 
particular details and I may miss some important documentation. I could help 
contribute a checker.rkt for my purposes after I get it fully working...

Particularly, after getting the server up and running, I'm having issues with 
comparing students that submit functions that return images. Primarily, I've 
ran a test such as the following that fails:

(@test "pinwheel case 1"
  "Incorrect pinwheel function"
   (above  (rectangle 10 30 "solid" "blue") (rectangle 10 30 "solid" 
"blue"))
   (above  (rectangle 10 30 "solid" "blue") (rectangle 10 30 "solid" 
"blue"))
 1)

A brilliant Professor suggested that instead of comparing the images, I should 
convert the images to a list of colors and compare those. Is there a better 
solution to this? I think if there were more examples, this is one of the 
things that would help make this setting up handin much easier.

Thanks for your help! I really appreciate this!






On Tuesday, October 11, 2016 at 1:45:26 AM UTC-4, Eli Barzilay wrote:
> On Thu, Oct 6, 2016 at 10:08 AM,   wrote:
> > So, I've been playing around with the handin server these days.
> >
> > My purposes of the handin require students to submit images using the
> > htdp image and universe libraries.
> >
> > First, I would like to note that for the images, you must turn off
> > textualization/create-text in your checker:
> >
> > (check:
> >; Get timestamp of the submission and add it to header and report
> 
> ([*] Is there some code that does the addition described here?)
> 
> 
> >:language 'racket
> >:requires '(bsl-plus-image)
> >:create-text? #f
> >:textualize? #f
> >
> > My issues follow:
> >
> > 1) Initially, there is a conflict with one of the image functions and
> >[...])
> 
> (I don't know about these issues...)
> 
> 
> > 2) Afterwards, there is a permission denied error:
> > [2|2016-10-06T09:49:28] ERROR: Error in your code --
> > [2|2016-10-06T09:49:28] file-exists?: `exists' access denied for 
> > libobj.dylib
> > [...]
> > As of now, I expose my path. I'm not sure how to create a better
> > solution than the one mentioned stating to add the following lines at
> > the top of your checker.rkt
> 
> The "/" in the additional setting is a path, and the `read` allows
> reading any file under that path.  So instead of "/", just use a path
> that includes only stuff that should be OK to read, including the
> "libobj.dylib" file.  (I don't use OSX so I don't know where it is.)
> 
> 
> > 3) Finally, I found a very serious issue in the collection's handin
> >server code. students are not able to submit because of the server
> >was trying to write to a grading/text.rkt even though I specified
> >not to! I've traced the issue to the collection's handin server.
> >
> > From (collection-path "handin-server"),
> >
> > /Library/Racket/6.6/pkgs/handin/handin-server/checker.rkt:
> >
> > Starting around line 467 the below code is the culprit!
> > Essentially, at the bottom of the below segment, regardless if you
> > checked if you were creating a text file in your checker.rkt, it will
> > attempt to write the text with the below code:
> >  (when (thread-cell-ref added-lines) (write-text))
> >
> > Essentially, to fix this issue, you may replace it with
> >  (when (and (thread-cell-ref added-lines) create-text?) (write-text))
> 
> That does look like a bug, but I think that a proper solution would make
> `add-header-line!` throw an error if there is no textualization
> happenning, except that getting the textualization flag there might be
> tricky, so a bit hackier solution is to make the code that you're
> talking about throw an error, so instead of
> 
> (when (thread-cell-ref added-lines) (write-text))
> 
> it would be something like (untested):
> 
> (when (thread-cell-ref added-lines)
>   (unless create-text?
> (error* "bad checker: no text created, but header lines added"))
>   (write-text))
> 
> This is better than what you're suggesting, which would silently ignore
> the added lines.
> 
> If you do have some code that adds header lines (my [*] question at the
> top) then that's the problem, and the error should be a configuration
> error.  (The thing is that additional headers are written to a generated
> text file -- there's no code that edits the student's media-format
> submission file.)
> 
> -- 
>((x=>x(x))(x=>x(x)))  Eli Barzilay:
>http://barzilay.org/

Re: [racket-users] Handin Issues

[Eli Barzilay]

On Thu, Oct 6, 2016 at 10:08 AM,   wrote:
> So, I've been playing around with the handin server these days.
>
> My purposes of the handin require students to submit images using the
> htdp image and universe libraries.
>
> First, I would like to note that for the images, you must turn off
> textualization/create-text in your checker:
>
> (check:
>; Get timestamp of the submission and add it to header and report

([*] Is there some code that does the addition described here?)


>:language 'racket
>:requires '(bsl-plus-image)
>:create-text? #f
>:textualize? #f
>
> My issues follow:
>
> 1) Initially, there is a conflict with one of the image functions and
>[...])

(I don't know about these issues...)


> 2) Afterwards, there is a permission denied error:
> [2|2016-10-06T09:49:28] ERROR: Error in your code --
> [2|2016-10-06T09:49:28] file-exists?: `exists' access denied for libobj.dylib
> [...]
> As of now, I expose my path. I'm not sure how to create a better
> solution than the one mentioned stating to add the following lines at
> the top of your checker.rkt

The "/" in the additional setting is a path, and the `read` allows
reading any file under that path.  So instead of "/", just use a path
that includes only stuff that should be OK to read, including the
"libobj.dylib" file.  (I don't use OSX so I don't know where it is.)


> 3) Finally, I found a very serious issue in the collection's handin
>server code. students are not able to submit because of the server
>was trying to write to a grading/text.rkt even though I specified
>not to! I've traced the issue to the collection's handin server.
>
> From (collection-path "handin-server"),
>
> /Library/Racket/6.6/pkgs/handin/handin-server/checker.rkt:
>
> Starting around line 467 the below code is the culprit!
> Essentially, at the bottom of the below segment, regardless if you
> checked if you were creating a text file in your checker.rkt, it will
> attempt to write the text with the below code:
>  (when (thread-cell-ref added-lines) (write-text))
>
> Essentially, to fix this issue, you may replace it with
>  (when (and (thread-cell-ref added-lines) create-text?) (write-text))

That does look like a bug, but I think that a proper solution would make
`add-header-line!` throw an error if there is no textualization
happenning, except that getting the textualization flag there might be
tricky, so a bit hackier solution is to make the code that you're
talking about throw an error, so instead of

(when (thread-cell-ref added-lines) (write-text))

it would be something like (untested):

(when (thread-cell-ref added-lines)
  (unless create-text?
(error* "bad checker: no text created, but header lines added"))
  (write-text))

This is better than what you're suggesting, which would silently ignore
the added lines.

If you do have some code that adds header lines (my [*] question at the
top) then that's the problem, and the error should be a configuration
error.  (The thing is that additional headers are written to a generated
text file -- there's no code that edits the student's media-format
submission file.)

-- 
   ((x=>x(x))(x=>x(x)))  Eli Barzilay:
   http://barzilay.org/  Maze is Life!

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Handin Issues

[mattsap]

So, I've been playing around with the handin server these days.

My purposes of the handin require students to submit images using the htdp 
image and universe libraries.

First, I would like to note that for the images, you must turn off 
textualization/create-text in your checker:

(check:
   ; Get timestamp of the submission and add it to header and report
   :language 'racket
   :requires '(bsl-plus-image)  
   
   :create-text? #f
   :textualize? #f

My issues follow:

1) Initially, there is a conflict with one of the image functions and another 
function provided by the language. I fixed this by adding the package as 
suggested in the other thread. We're still waiting to see if the checker 
sandbox has updated in order to see if there's been an update. (see here for 
more detail: https://groups.google.com/forum/#!topic/racket-users/twViz38AL7M)

2) Afterwards, there is a permission denied error:
[2|2016-10-06T09:49:28] ERROR: Error in your code --
[2|2016-10-06T09:49:28] file-exists?: `exists' access denied for libobj.dylib

This can be solved using this hacky solution in this thread:
https://groups.google.com/forum/?fromgroups#!searchin/racket-users/libobj%7Csort:relevance/racket-users/W_-4aN7kPyI/cwgwz2xIBAAJ

As of now, I expose my path. I'm not sure how to create a better solution than 
the one mentioned stating to add the following lines at the top of your 
checker.rkt
(module checker handin-server/checker
  (require handin-server/grading-utils)

  (require handin-server/sandbox)
  (sandbox-path-permissions
   (cons
(list 'read "/")
(sandbox-path-permissions)))

3) Finally, I found a very serious issue in the collection's handin server 
code. students are not able to submit because of the server was trying to write 
to a grading/text.rkt even though I specified not to! I've traced the issue to 
the collection's handin server.

>From (collection-path "handin-server"), 

/Library/Racket/6.6/pkgs/handin/handin-server/checker.rkt:

Starting around line 467 the below code is the culprit!
Essentially, at the bottom of the below segment, regardless if you checked if 
you were creating a text file in your checker.rkt, it will attempt to write the 
text with the below code:
 (when (thread-cell-ref added-lines) (write-text))

Essentially, to fix this issue, you may replace it with
 (when (and (thread-cell-ref added-lines) create-text?) (write-text))

;; 
 ;; convert to text, evaluate, check
 (define (check users submission)
   (define text-file (format "grading/text.~a" suffix))
   (define (prefix-line str)
 (printf "~a~a\n" markup-prefix str))
   (define generic-substs `(("submission" . ,submission-dir)))
   (define (prefix-line/substs str)
 (prefix-line (subst str generic-substs)))
   (define (write-text)
 (set-run-status "creating text file")
 (with-output-to-file text-file #:exists 'truncate
   (lambda ()
 (for ([user (in-list users)])
   (prefix-line (user-substs user student-line)))
 (for-each prefix-line/substs extra-lines)
 (for-each prefix-line/substs
   (cond [(thread-cell-ref added-lines)
  => unbox]
 [else '()]))
 (display submission-text
   (define submission-text
 (and create-text?
  (begin (set-run-status "reading submission")
 ((if multi-file
(unpack-multifile-submission
 names-checker output-file)
submission->bytes)
  submission maxwidth textualize? untabify?
  markup-prefix prefix-re
   (define (uem-handler e)
 (let ([m (if (exn? e) (exn-message e) (format "~a" e))])
   (cond
 [(procedure? uem) (uem m)]
 [(not (string? uem))
  (error* "badly configured user-error-message")]
 [(regexp-match? #rx"~[aesvAESV]" uem) (error* uem m)]
 [else (error* "~a" uem)])))
   (when create-text? (make-directory "grading") (write-text))
   (when value-printer (current-value-printer value-printer))
   (when coverage? (sandbox-coverage-enabled #t))
   (set-run-status "checking submission")
   (cond
 [(not eval?) (let () body ...)]