13.10 Sandboxed Evaluation
The bindings documented in this section are provided by the scheme/sandbox library, not scheme/base or scheme.
The scheme/sandbox module provides utilities for creating “sandboxed” evaluators, which are configured in a particular way and can have restricted resources (memory and time), filesystem access, and network access.
| ||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||
input-program : any/c | ||||||||||||||||||||||||||||
requires : (listof (or/c module-path? path?)) | ||||||||||||||||||||||||||||
allow : (listof (or/c module-path? path?)) | ||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||
allow : (listof (or/c module-path? path?)) |
The make-evaluator function creates an evaluator with a language and requires specification, and starts evaluating the given input-programs. The make-module-evaluator function creates an evaluator that works in the context of a given module. The result in either case is a function for further evaluation.
The returned evaluator operates in an isolated and limited environment. In particular, filesystem access is restricted. The allow argument extends the set of files that are readable by the evaluator to include the specified modules and their imports (transitively). When language is a module path and when requires is provided, the indicated modules are implicitly included in the allow list.
Each input-program or module-decl argument provides a program in one of the following forms:
an input port used to read the program;
a string or a byte string holding the complete input;
a path that names a file holding the input; or
an S-expression or a syntax object, which is evaluated as with eval (see also get-uncovered-expressions).
In the first three cases above, the program is read using sandbox-reader, with line-counting enabled for sensible error messages, and with 'program as the source (used for testing coverage). In the last case, the input is expected to be the complete program, and is converted to a syntax object (using 'program as the source), unless it already is a syntax object.
The returned evaluator function accepts additional expressions (each time it is called) in essentially the same form: a string or byte string holding a sequence of expressions, a path for a file holding expressions, an S-expression, or a syntax object. If the evaluator receives an eof value, it is terminated and raises errors thereafter. See also kill-evaluator, which terminates the evaluator without raising an exception.
For make-evaluator, multiple input-programs are effectively concatenated to form a single program. The way that the input-programs are evaluated depends on the language argument:
The language argument can be a module path (i.e., a datum that matches the grammar for module-path of require).
In this case, the input-programs are automatically wrapped in a module, and the resulting evaluator works within the resulting module’s namespace.
The language argument can be a list starting with 'special, which indicates a built-in language with special input configuration. The possible values are '(special r5rs) or a value indicating a teaching language: '(special beginner), '(special beginner-abbr), '(special intermediate), '(special intermediate-lambda), or '(special advanced).
In this case, the input-programs are automatically wrapped in a module, and the resulting evaluator works within the resulting module’s namespace. In addition, certain parameters (such as such as read-accept-infix-dot) are set to customize reading programs from strings and ports.
This option is provided mainly for older test systems. Using make-module-evaluator with input starting with #lang is generally better.
Finally, language can be a list whose first element is 'begin.
In this case, a new namespace is created using sandbox-namespace-specs, which by default creates a new namespace using make-base-namespace or make-gui-namespace (depending on gui?).
In the new namespace, language is evaluated as an expression to further initialize the namespace.
The requires list adds additional imports to the module or namespace for the input-programs, even in the case that require is not made available through the language.
The following examples illustrate the difference between an evaluator that puts the program in a module and one that merely initializes a top-level namespace:
| |||
program:1:0: compile: unbound variable in module in: later | |||
| |||
> (base-module-eval '(f)) | |||
5 | |||
| |||
> (base-top-eval '(+ 1 2)) | |||
3 | |||
> (base-top-eval '(define later 5)) | |||
> (base-top-eval '(f)) | |||
5 |
The make-module-evaluator function is essentially a restriction of make-evaluator, where the program must be a module, and all imports are part of the program:
(define base-module-eval2 |
; equivalent to base-module-eval: |
(make-module-evaluator '(module m scheme/base |
(define (f) later) |
(define later 5)))) |
In all cases, the evaluator operates in an isolated and limited environment:
It uses a new custodian and namespace. When gui? is true, it is also runs in its own eventspace.
The evaluator works under the sandbox-security-guard, which restricts file system and network access.
Each evaluation is wrapped in a call-with-limits; see also sandbox-eval-limits and set-eval-limits.
Evaluation can also be instrumented to track coverage information when sandbox-coverage-enabled is set. Exceptions (both syntax and run-time) are propagated as usual to the caller of the evaluation function (i.e., catch it with with-handlers). However, note that a sandboxed evaluator is convenient for testing, since all exceptions happen in the same way, so you don’t need special code to catch syntax errors.
Finally, the fact that a sandboxed evaluator accept syntax objects makes it usable as the value for "current-eval", which means that you can easily start a sandboxed read-eval-print-loop. For example, here is a quick implementation of a networked REPL:
(define e |
(make-module-evaluator '(module m scheme/base))) |
(let-values ([(i o) (tcp-accept (tcp-listen 9999))]) |
[current-eval a]) |
(fprintf o "\nBye...\n") |
(close-output-port o))) |
13.10.1 Customizing Evaluators
The evaluators that make-evaluator creates can be customized via several parameters. These parameters affect newly created evaluators; changing them has no effect on already-running evaluators.
(sandbox-init-hook) → (-> any) |
(sandbox-init-hook thunk) → void? |
A parameter that determines a thunk to be called for initializing a new evaluator. The hook is called just before the program is evaluated in a newly-created evaluator context. It can be used to setup environment parameters related to reading, writing, evaluation, and so on. Certain languages ('(special r5rs) and the teaching languages) have initializations specific to the language; the hook is used after that initialization, so it can override settings.
(sandbox-reader) → (any/c . -> . any) |
(sandbox-reader proc) → void? |
A parameter that determines a function to reads all expressions from (current-input-port). The function is used to read program source for an evaluator when a string. byte string, or port is supplies. The reader function receives a value to be used as input source (i.e., the first argument to read-syntax), and it should return a list of syntax objects. The default reader calls read-syntax, accumulating results in a list until it receives eof.
| |||||||||||
(sandbox-input in) → void? | |||||||||||
|
A parameter that determines the initial current-input-port setting for a newly created evaluator. It defaults to #f, which creates an empty port. The following other values are allowed:
a string or byte string, which is converted to a port using open-input-string or open-input-bytes;
an input port;
the symbol 'pipe, which triggers the creation of a pipe, where put-input can return the output end of the pipe or write directly to it;
a thunk, which is called to obtain a port (e.g., using current-input-port means that the evaluator input is the same as the calling context’s input).
| ||||||||||
(sandbox-output in) → void? | ||||||||||
|
A parameter that determines the initial current-output-port setting for a newly created evaluator. It defaults to #f, which creates a port that discrds all data. The following other values are allowed:
an output port, which is used as-is;
the symbol 'bytes, which causes get-output to return the complete output as a byte string;
the symbol 'string, which is similar to 'bytes, but makes get-output produce a string;
the symbol 'pipe, which triggers the creation of a pipe, where get-output returns the input end of the pipe;
a thunk, which is called to obtain a port (e.g., using current-output-port means that the evaluator output is not diverted).
| ||||||||||
(sandbox-error-output in) → void? | ||||||||||
|
Like sandbox-output, but for the initial current-error-port value. An evaluator’s error output is set after its output, so using current-output-port (the parameter itself, not its value) for this parameter value means that the error port is the same as the evaluator’s initial output port.
The default is (lambda () (dup-output-port (current-error-port))), which means that the error output of the generated evaluator goes to the calling context’s error port.
(sandbox-coverage-enabled enabled?) → void? |
enabled? : any/c |
A parameter that controls whether syntactic coverage information is collected by sandbox evaluators. Use get-uncovered-expressions to retrieve coverage information.
(sandbox-propagate-breaks propagate?) → void? |
propagate? : any/c |
When this boolean parameter is true, breaking while an evaluator is running evaluator propagates the break signal to the sandboxed context. This makes the sandboxed evaluator break, typically, but beware that sandboxed evaluation can capture and avoid the breaks (so if safe execution of code is your goal, make sure you use it with a time limit). The default is #t.
| ||||||||
(sandbox-namespace-specs spec) → void? | ||||||||
|
A parameter that holds a list of values that specify how to create a namespace for evaluation in make-evaluator or make-module-evaluator. The first item in the list is a thunk that creates the namespace, and the rest are module paths for modules that to be attached to the created namespace using namespace-attach-module.
The default is (list make-base-namespace) if gui? is #f, (list make-gui-namespace) if gui? is #t.
The module paths are needed for sharing module instantiations between the sandbox and the caller. For example, sandbox code that returns posn values (from the lang/posn module) will not be recognized as such by your own code by default, since the sandbox will have its own instance of lang/posn and thus its own struct type for posns. To be able to use such values, include 'lang/posn in the list of module paths.
When testing code that uses a teaching language, the following piece of code can be helpful:
(let ([specs (sandbox-namespace-specs)]) |
`(,(car specs) |
,@(cdr specs) |
lang/posn |
(sandbox-override-collection-paths paths) → void? |
paths : (listof path-string?) |
A parameter that determines a list of collection directories to prefix current-library-collection-paths in an evaluator. This parameter is useful for cases when you want to test code using an alternate, test-friendly version of a collection, for example, testing code that uses a GUI (like the htdp/world teachpack) can be done using a fake library that provides the same interface but no actual interaction. The default is null.
(sandbox-security-guard guard) → void? |
guard : security-guard? |
A parameter that determines the initial (current-security-guard) for sandboxed evaluations. The default forbids all filesystem I/O except for things in sandbox-path-permissions, and it uses sandbox-network-guard for network connections.
| |||||||||
(sandbox-path-permissions perms) → void? | |||||||||
|
A parameter that configures the behavior of the default sandbox security guard by listing paths and access modes that are allowed for them. The contents of this parameter is a list of specifications, each is an access mode and a byte-regexp for paths that are granted this access.
The access mode symbol is one of: 'execute, 'write, 'delete, 'read, or 'exists. These symbols are in decreasing order: each implies access for the following modes too (e.g., 'read allows reading or checking for existence).
The path regexp is used to identify paths that are granted access. It can also be given as a path (or a string or a byte string), which is (made into a complete path, cleansed, simplified, and then) converted to a regexp that allows the path and sub-directories; e.g., "/foo/bar" applies to "/foo/bar/baz".
The default value is null, but when an evaluator is created, it is augmented by 'read permissions that make it possible to use collection libraries (including sandbox-override-collection-paths). See make-evalautor for more information.
| |||||||||||
(sandbox-network-guard proc) → void? | |||||||||||
|
A parameter that specifieds a procedure to be used (as is) by the default sandbox-security-guard. The default forbids all network connection.
| |||||||||
(sandbox-eval-limits limits) → void? | |||||||||
|
A parameter that determines the default limits on each use of a make-evaluator function, including the initial evaluation of the input program. Its value should be a list of two numbers, the first is a timeout value in seconds, and the second is a memory limit in megabytes. Either one can be #f for disabling the corresponding limit; alternately, the parameter can be set to #f to disable all limits (in case more are available in future versions). The default is (list 30 20).
When limits are set, call-with-limits (see below) is wrapped around each use of the evaluator, so consuming too much time or memory results in an exception. Change the limits of a running evaluator using set-eval-limits.
(sandbox-make-inspector make) → void? |
make : (-> inspector?) |
A parameter that determines the procedure used to create the inspector for sandboxed evaluation. The procedure is called when initializing an evaluator, and the default parameter value is make-inspector.
(sandbox-make-logger) → (-> logger?) |
(sandbox-make-logger make) → void? |
A parameter that determines the procedure used to create the logger for sandboxed evaluation. The procedure is called when initializing an evaluator, and the default parameter value is current-logger.
13.10.2 Interacting with Evaluators
The following functions are used to interact with a sandboxed evaluator in addition to using it to evaluate code.
(kill-evaluator evaluator) → void? |
Releases the resources that are held by evaluator by shutting down the evaluator’s custodian. Attempting to use an evaluator after killing raises an exception, and attempts to kill a dead evaluator are ignored.
Killing an evaluator is similar to sending an eof value to the evaluator, except that an eof value will raise an error immediately.
(break-evaluator evaluator) → void? |
Sends a break to the running evaluator. The effect of this is as if Ctrl-C was typed when the evaluator is currently executing, which propagates the break to the evaluator’s context.
(set-eval-limits evaluator secs mb) → void? |
secs : (or/c exact-nonnegative-integer? false/c) |
Changes the per-expression limits that evaluator uses to sec seconds and mb megabytes (either one can be #f, indicating no limit).
This procedure should be used to modify an existing evaluator limits, because changing the sandbox-eval-limits parameter does not affect existing evaluators. See also call-with-limits.
(put-input evaluator) → output-port? |
i/o : (or/c bytes? string? eof-object?) |
If (sandbox-input) is 'pipe when an evaluator is created, then this procedure can be used to retrieve the output port end of the pipe (when used with no arguments), or to add a string or a byte string into the pipe. It can also be used with eof, which closes the pipe.
(get-output evaluator) → (or/c input-port? bytes? string?) |
(get-error-output evaluator) |
→ (or/c input-port? bytes? string?) |
Returns the output or error-output of the evaluator, in a way that depends on the setting of (sandbox-output) or (sandbox-error-output) when the evaluator was created:
if it was 'pipe, then get-output returns the input port end of the created pipe;
if it was 'bytes or 'string, then the result is the accumulated output, and the output is directed to a new output string or byte string (so each call returns a different piece of the evaluator’s output);
otherwise, it returns #f.
| |||||||||||||||||||||
prog? : any/c = #t | |||||||||||||||||||||
src : any/c = 'program |
Retrieves uncovered expression from an evaluator, as longs as the sandbox-coverage-enabled parameter had a true value when the evaluator was created. Otherwise, and exception is raised to indicate that no coverage information is available.
The prog? argument specifies whether to obtain expressions that were uncovered after only the original input program was evaluated (#t) or after all later uses of the evaluator (#f). Using #t retrieves a list that is saved after the input program is evaluated, and before the evaluator is used, so the result is always the same.
A #t value of prog? is useful for testing student programs to find out whether a submission has sufficient test coverage built in. A #f value is useful for writing test suites for a program to ensure that your tests cover the whole code.
The second optional argument, src, specifies that the result should be filtered to hold only syntax objects whose source matches src. The default, 'program, is the source associated with the input program by the default sandbox-reader – which provides only syntax objects from the input program (and not from required modules or expressions that were passed to the evaluator). A #f avoids filtering.
The resulting list of syntax objects has at most one expression for each position and span. Thus, the contents may be unreliable, but the position information is reliable (i.e., it always indicates source code that would be painted red in DrScheme when coverage information is used).
Note that if the input program is a sequence of syntax values, either make sure that they have 'program as the source field, or use the src argument. Using a sequence of S-expressions (not syntax objects) for an input program leads to unreliable coverage results, since each expression may be assigned a single source location.
13.10.3 Miscellaneous
True if the scheme/gui module can be used, #f otherwise; see gui-available?.
Various aspects of the scheme/sandbox library change when the GUI library is available, such as using a new eventspace for each evaluator.
(call-with-limits secs mb thunk) → any |
secs : (or/c exact-nonnegative-integer? false/c) |
Executes the given thunk with memory and time restrictions: if execution consumes more than mb megabytes or more than sec seconds, then the computation is aborted and the exn:fail:resource exception is raised. Otherwise the result of the thunk is returned as usual (a value, multiple values, or an exception). Each of the two limits can be #f to indicate the absence of a limit. See also custodian-limit-memory for information on memory limits.
Sandboxed evaluators use call-with-limits, according to the sandbox-eval-limits setting and uses of set-eval-limits: each expression evaluation is protected from timeouts and memory problems. Use call-with-limits directly only to limit a whole testing session, instead of each expression.
(with-limits mb-expr body-expr body ) |
A macro version of call-with-limits.
(exn:fail:resource? v) → boolean? |
v : any/c |
(exn:fail:resource-resource exn) → (one-of/c 'time 'memory) |
exn : exn:fail:resource? |
A predicate and accessor for exceptions that are raised by call-with-limits. The resource field holds a symbol, either 'time or 'memory.