Macros¶
The term macro, as used in computer programming, originally stood for macro-instruction, meaning an instruction that represented a sequence of several machine (or micro) instructions. Over time, the term has evolved to mean any word or phrase that stands for another phrase (usually longer, but built of simpler components). Macros can be used for abbreviation, abstraction, simplification, or structuring. Many application programs, such as word processors or spreadsheets, offer a macro language for writing scripts or subroutines that bundle a number of simpler actions into one command.
Many computer languages support a macro facility for creating shorthand notations for commonly used, longer phrases. They range from simple, text-based abbreviations to full languages, permitting computed replacements. Macros are processed before the program is compiled by expanding each macro into its replacement phrase as that macro is encountered until there are no more macros. You can use macros to extend the base language by defining more sophisticated phrases in terms of simpler, built-in phrases.
The primary use of macros in programming languages is to extend or adapt the language to allow a more concise or readable solution for a particular problem domain. A simple program rarely needs macros. More complicated programs, including the implementation of a Dylan compiler and run-time system, will use macros often. Macros have no visible run-time cost or effect — they are transformations that take place during the compilation of a program (hence, they can increase compilation time). Although macros may take the form of function calls, they are not functions — they cannot be passed as functional arguments, and they cannot be invoked in a run-time image as a function can. Although macros may have parameters, they do not take arguments the way functions do. The arguments to a macro are not evaluated; they are simply program phrases that can be substituted in the replacement phrase.
Dylan provides a macro facility that is based on pattern matching and template substitution. This facility is more powerful than is a simple textual substitution facility, but is simpler than a procedural-macro facility, which allows arbitrary computations to construct replacement phrases. Dylan’s macro facility is closely integrated with the Dylan language syntax, and permits most macro needs to be satisfied. Dylan designers have also planned for a full procedural macro capability, so that it can be added compatibly at a later time if there is sufficient demand.
Patterns and templates¶
A Dylan macro consists of a set of rules. Each rule has two basic parts: a pattern that is matched against a fragment of code, and a template that is substituted for the matched fragment, perhaps including pieces of the original fragment. When a macro is invoked, each rule is tried in order until a matching pattern is found. When a match is found, the macro is replaced by the matching template. If no match can be found, an error occurs.
Dylan macros are recognized by the compiler because they fit one of three possible formats: the function macro, the statement macro, and the defining macro. The macro format determines the overall fragment that is matched against the macro’s rules at each macro invocation.
The simplest macro format that the compiler can match is that of a function call. A function macro is invoked in exactly the same way that a function is invoked. The name of the macro is a module variable that can be used anywhere a function call can occur. Typically, it is simply the name followed by a parenthesized list of arguments, but recall that slot-style abbreviations and unary and binary operators are also function calls.
The most important use of function macros is to rearrange or delay evaluation of arguments. The fragment that is matched against the function macro’s rules is the phrase that represents a function’s arguments. The function macro can then rearrange the function arguments, perhaps adding code. When a macro rearranges its arguments, its action has the effect of delaying the evaluation of the arguments (as opposed to a function call, where the argument expressions are evaluated and then passed to the function).
One simple use of delaying evaluation is to write a function-like
construct similar in spirit to C’s ?: operator:
define macro if-else
{ if-else (?test:expression, ?true:expression, ?false:expression) }
=> { if (?test) ?true else ?false end }
end macro if-else;
We could not write if-else as a function, because both the true and
false expressions would be evaluated before the function was even
called:
? define variable *x* = 0;
? define variable *y* = 0;
? *y* := if-else(*y* == 0, *x* := 1, *x* := -1);
=> 1
? *y*;
=> 1
? *x*;
=> 1
If we had defined if-else as a function, *x* would have been -1,
rather than 1, because both assignments to *x* would have been
evaluated, before if-else was called. When a macro is used, the
assignments are just substituted into the template if, which
evaluates the first clause only when the condition is true.
Looking at the macro definition of if-else, we can infer basic ideas
about macros. A macro is introduced by define macro, followed by the
macro name — in this case, if-else. The definition of the macro
is a rule that has two parts: a pattern enclosed in braces, {},
that mimics the fragment that it is to match, and a replacement.
Macro parameters, called pattern variables, are introduced in the
pattern by ?. They match fragments with particular constraints — in
this case, :expression. They are delimited by punctuation — in this
case, the open and close parentheses, (), and the comma, ,.
The replacement part of the rule, the expansion, is indicated by =>
and is defined by a template, also enclosed in braces. The template
is in the form of a code fragment, where pattern variables are used to
substitute in the fragments they matched in the pattern. Note that
matching and replacement are language based, so required and optional
whitespace is treated exactly as in Dylan. We have used optional
whitespace to improve the legibility of the macro definitions presented
here.
Most Dylan development environments provide a way to view code after all macros have been expanded. This view can be helpful in debugging macros that you write. For example, showing the expanded view of an expression like
*y* := if-else(*y* == 0, *x* := 1, *x* := -1);
might yield
*y* := if (*y* == 0) *x* := 1 else *x* := -1 end;
The exact format of the expanded view of the macro depends on the
particular development environment. Here, we show the code that comes
from the macro template in underlined italic, whereas the fragments
matched by the pattern variables and substituted into the template are
presented in our conventional code font. Note that the if-else
macro we have defined is just syntactic sugar — Dylan’s built-in if
statement is perfectly sufficient for the job.
Another reason to delay evaluation is to change the value of an argument
— for example, to implement an operator similar in spirit to C’s ++
and += operators:
define macro inc!
{ inc! (?place:expression, ?by:expression) }
=> { ?place := ?place + ?by; }
{ inc! (?place:expression) }
=> { ?place := ?place + 1; }
end macro inc!;
This macro might be used as follows:
? define variable *x* = 0;
? inc!(*x*, 3);
=> 3
? *x*;
=> 3
? inc!(*x*);
4
? *x*;
4
In this macro, it is important to delay the evaluation of the first argument because we want to be able to assign to the variable or slot it is stored in, rather than simply to manipulate the value of the variable or slot.
The inc! macro demonstrates the use of multiple rules in a macro. They
are tried in order until an appropriate match is found. This allows the
inc! macro to have two forms. The one-argument form increments the
argument by 1. The two-argument form allows the increment amount to be
specified.
Macro hygiene¶
Displaying the code fragments inserted by the macro in underlined italics both helps to show exactly what the macro has done to our code, and draws attention to an important feature of Dylan macros — they are hygienic macros. A hygienic or referentially transparent macro system is one that prevents accidental collisions of macro variables with program variables of the same name. Consider the following macro, which is used to exchange the values of two variables:
define macro swap!
{ swap! (?place1:expression, ?place2:expression) }
=> { let value = ?place1;
?place1 := ?place2;
?place2 := value
}
end macro swap!;
The local variable value is created by the macro. There is a
possibility that this variable could conflict with another variable in
the surrounding code. Consider what might happen if we were to expand
swap!(value, x):
let value = value;
value := x;
x := value
With simple textual substitutions, swap! would have no effect in this
case. Dylan’s hygienic macros solve this problem by differentiating
between the value introduced by the macro and any other value
that might appear in the original code.
Evaluation in macros¶
Dylan’s template macros do no evaluation. In particular, the pattern variables of a macro are unlike function parameters. They name fragments of code, rather than naming the result of the evaluation of a fragment of code.
If we were trying to write an operation like C’s || (one that would
evaluate expressions and would return the value of the first nonzero
expression without evaluating any subsequent expressions), we could not
write it as a function:
define method or-int (arg1, arg2) if (arg1 ~= 0) arg1 else arg2 end end;
When a function is invoked, all its arguments are evaluated first, which defeats our purpose. If we model our macro on our function idea, however, we will not get the ideal result either:
define macro or-int
{ or-int (?arg1:expression, ?arg2:expression) } =>
{ if (?arg1 ~= 0) ?arg1 else ?arg2 end }
end macro or-int;
The expansion of or-int (x := x + 1, y := y - 1) is probably not what
we want:
if (x := x + 1 ~= 0) x := x + 1 else y := y - 1 end
We see a common macro error — the expression x := x + 1 will be
evaluated twice when the resulting substitution is evaluated, leaving
x with an incorrect (or at least unexpected) value. There is no magic
technique for avoiding this error — you just have to be careful about
repeating a pattern variable in a template. Most often, if you are
repeating a pattern variable, you should be using a local variable
instead, so that the fragment that the pattern represents is evaluated
only once:
define macro or-int
{ or-int (?arg1:expression, ?arg2:expression) }
=> {
let arg1 = ?arg1;
if(arg1 ~= 0) arg1 else ?arg2 end
}
end macro or-int;
Another potential pitfall arises if the pattern variables appear in an order in the template different from the one in which they appear in the pattern. In this case, unexpected results can occur if a side effect in one fragment affects the meaning of other fragments. In this case, you would again want to use local variables to ensure that the fragments were evaluated in their natural order.
These rules are not hard and fast: The power of macros is due in a large part to the ability of macros to manipulate code fragments without evaluating those fragments, but that power must be used judiciously. If you are designing macros for use by other people, those people may expect function-like behavior, and may be surprised if there are multiple or out-of-order evaluations of macro parameters.
Constraints¶
So far, in our macros, we have seen the constraint expression used for the pattern variables. Except for a few unusual cases, pattern variables must always have a constraint associated with them. Constraints serve two purposes: they limit the fragment that the pattern variable will match, and they define the meaning of the pattern variable when it is substituted. As an example, consider the following statement macro, which we might find useful for manipulating the decoded parts of seconds:
define macro with-decoded-seconds
{
with-decoded-seconds
(?max:variable, ?min:variable, ?sec:variable = ?time:expression)
?:body
end
}
=> {
let (?max, ?min, ?sec) = decode-total-seconds(?time);
?body
}
end macro;
The preceding macro might be used as follows:
define method say (time :: <time>)
with-decoded-seconds(hours, minutes, seconds = time)
format-out("%d:%s%d",
hours, if (minutes < 10) "0" else "" end, minutes);
end;
end method say;
A statement macro can appear anywhere that a begin / end; block can
appear. A statement macro introduces a new begin word — in this case,
with-decoded-seconds — and is matched against a fragment that extends
up to the matching end.
The pattern and the constraints on the pattern variables limit what the
macro will match; they define the syntax of this particular statement.
In the case of with-decoded-seconds, the syntax of this statement
begins with a parenthesized list of
Three variable expressions (that is,
name :: <type>, where the type is optional)The literal token
=An expression (any Dylan expression yielding a value)
After the parenthesized list comes a body (any sequence of expressions
separated by ;, just as would be valid in a begin / end; block).
Note the use of the abbreviation ?:body, to mean ?body:body (a
pattern variable, body, with the constraint body).
The constraints are similar to type declarations on variables: They limit the acceptable values of the pattern variables, and they help to document the interface of the macro. The constraints also serve a second purpose: Once the compiler has recognized a fragment under a particular constraint, it will ensure the correct behavior of that fragment when that fragment is substituted in a template. For example, suppose that we define a function macro:
define macro times
{ times (?arg1:expression, ?arg2:expression ) } =>
{ ?arg1 * ?arg2 }
end macro times;
We might use the macro as follows:
times(1 + 3, 2 + 5);
Here is the expanded macro:
1 + 3 * 2 + 5
We can see that, if the macro were a simple text-substitution macro, the result would be 12, rather than the 28 we were expecting. But because, in Dylan, the constraint is maintained when a pattern variable is substituted (that is, the expression that makes up each of the pattern variables remains a single expression), the result is as though the macro automatically inserted parentheses, and the expansion were
(1 + 3) * (2 + 5)
Some development environments may display the implicit parentheses of an expression constraint. Thus, the macro will yield the expected result of 28.
More complex rules¶
The macros shown so far have all been simple: a single pattern transformed into a single template. To get a flavor of the full power of the Dylan macro system, consider this defining macro:
define macro aircraft-definer
{ define aircraft ?identifier:name (?type:name) ?flights end }
=> { register-aircraft(make("<" ## ?type ## ">", id: ?#"identifier"));
register-flights(?#"identifier", ?flights) }
flights:
{ }
=> { }
{ ?flight; ... }
=> { ?flight, ... }
flight:
{ flight ?id:name, #rest ?options:expression }
=> { make(<flight>, id: ?#"id", ?options) }
end macro aircraft-definer;
We might use the macro define aircraft as follows:
define aircraft UA4906H (DC10)
flight UA11, from: #"BOS", to: #"SFO";
flight UA12, from: #"SFO", to: #"BOS";
end aircraft UA4906H;
This macro shows a number of the more esoteric features of Dylan macros.
First, notice the pattern variable ?flights, which has no constraint,
but rather is called out as an auxiliary rule. When the compiler
matches this macro, it will try each of the auxiliary rule’s patterns
listed under flights: for a match. When it finds a match, it will
assign the pattern variable ?flights to the fragment resulting from
the matching pattern’s template substitution. In effect, auxiliary rules
give a way of writing new constraints, combined with the effect of a
subroutine for matching and substitution.
In this particular case, we use the auxiliary rule to map yet another
auxiliary rule, flight, over a sequence of flight descriptions that
look similar to the slot descriptions in a class. The mapping is
signaled by the points of ellipsis (...) which means that the rule
should be applied recursively (that is, the current rule is matched
again to the fragment that matches ...). Note that flights must
have a rule to cover the case of there being no flight; that rule also
handles the end of the recursion when the final flight has been matched.
The flight rule simply converts each flight name and its options into
the appropriate call to make, to create the flight. We could extend
this rule to allow a more natural specification for flight origin,
destination, and time.
We do the work of defining an aircraft by calling the helper functions
register-aircraft and register-flights (which are not given here),
but the macro takes care of getting the arguments in order. The
substitution "<" ## ?type ## ">" turns the name DC10 into the name
<DC10> by using concatenation, allowing a more concise format for our
definer while maintaining our convention for naming types. The substitution
?#"identifier" turns the name UA1306 into the symbol #"UA1306" by
using coercion; the program can use the symbol #"UA1306" to look up
an aircraft in the registry by name. The template for flights collects
all the individual flights into a comma-separated list that is passed to
register-flights as a #rest argument.
More hygiene¶
We shall make one more note about hygiene: In a textual substitution
macro, there is a chance that the global variables that the macro uses
(in this case, the helper function register-aircraft) could be confused
with a surrounding local variable of the same name where the macro is
called. This confusion does not happen in a Dylan macro. The global
variables used in a Dylan macro always denote what they denoted at the
time that the macro was defined, rather than at the time that the macro
is called. It is as though the variables were automatically renamed so
that conflicts will be avoided.
You will also notice this feature if you export a macro from a module. Only the macro needs to be exported. Its global references still refer to the proper (module-private) values that they had at the time the macro was defined, just as occurs when a function exported from a module calls module-private subroutines.
Occasionally, you will want to circumvent macro hygiene. You may want to
define a macro that creates a variable that is visible at the macro
call. Here is a simple statement macro that repeats its body until you
ask it to stop!:
define macro repeat
{ repeat ?:body end }
=> { block (?=stop!)
local method again() ?body; again() end;
again();
end }
end macro repeat;
The term ?=stop! says that the local variable stop!, which is the
block exit variable, will be visible when the macro is called exactly as
stop!; there will be no hygienic renaming. Here is an example that
uses the macro to count to 100:
begin
let i = 0;
repeat
if (i == 100) stop!() end;
i := i + 1;
end;
end;
Note that the body constraint invokes the Dylan parser to match the
code properly between the repeat and the corresponding end. It is
not confused by the end of the if statement, as a text-based macro
might be. The expanded view of the preceding code might look like this:
begin
let i = 0;
block (stop!)
local method again()
if (i == 100) stop!() end;
i := i + 1;
again()
end;
again();
end;
end;
Note that we have shown the local variable stop! introduced by the
macro block in code font rather than in underline italic, because
it is visible to the body and is exactly the stop! called in the if
to stop the repetition. The local variable again, on the other hand,
is not visible to the body code. We could use again instead of i as
our repetition count without a problem.
Note that we would have to document how repeat works for other users,
or they might be surprised if they tried to use stop! instead of i
in the example.
Auxiliary macros¶
One difficulty with the aircraft macro that we defined in More complex rules is this: suppose that we want each flight object to know the type of equipment used, rather than our having to look up the type in the aircraft registry. What looks like the obvious approach does not work:
define macro aircraft-definer
{ define aircraft ?identifier:name (?type:name) ?flights end }
=> { register-aircraft(make("<" ## ?type ## ">", id: ?#"identifier"));
register-flights(?#"identifier", ?flights) }
flights:
{ }
=> { }
{ ?flight; ... }
=> { ?flight, ... }
flight:
{ }
=> { }
{ flight ?id:name, #rest ?options:expression }
=> { make(<flight>, equipment: ?"type", id: ?#"id", ?options) }
end macro aircraft-definer;
When we are processing the flight auxiliary rules, we would like to be
able to reference the pattern variable ?type (coercing it to a string)
from the main rules, but it is not in scope — it is inaccessible
to the auxiliary rules. We could have register-flights set the
equipment slot after the flight is created, but we would prefer to
initialize the slot at the time we create the <flight> object. There
is a workaround, an auxiliary macro:
define macro aircraft-definer
{ define aircraft ?identifier:name (?type:name) ?flights:* end }
=> { register-aircraft (make("<" ## ?type ## ">", id: ?#"identifier"));
define flights (?#"identifier", ?"type")
?flights
end }
end macro aircraft-definer;
define macro flights-definer
{ define flights (?craft:name, ?equipment:name) end }
=> { }
{ define flights (?craft:name, ?equipment:name) ?flight ; ?more:* end
}
=> { register-flights
(?craft, make(<flight>, equipment: ?equipment, ?flight)) ;
define flights (?craft, ?equipment) ?more end }
flight:
{ }
=> { }
{ flight ?id:name, #rest ?options:expression }
=> { id: ?#"id", ?options }
end macro flights-definer;
Here, we have essentially broken out the work that used to be done by
the auxiliary rule flights into a separate definition macro. Where
flights used points of ellipsis to walk over each flight, the
definition macro uses a wildcard constraint ?more:*, explicitly
calling itself again (that is, the macro appears in the substitution,
and will be expanded again), as long as there are more flights to be
processed.
Here is an example use of the flights-definer macro:
define aircraft UA4906H (DC10)
flight UA11 from: #"BOS", to: #"SFO";
flight UA12 from: #"SFO", to: #"BOS";
end aircraft UA4906H;
Expanding that code would result in the following:
register-aircraft (make(<DC10>, id: #"UA4096H"));
register-flights (#"UA4096H",
make(<flight>, equipment: "DC10",
id: #"UA11" from: #"BOS", to: #"SFO");
register-flights (#"UA4096H",
make(<flight>, equipment: "DC10",
id: #"UA12" from: #"SFO", to: #"BOS");
(Note that this example is a hypothetical one used to illustrate macro
expansion. The define aircraft statement cannot be compiled in the
airport example.)
Summary¶
In this chapter, we introduced macros by explaining their purpose as a language-extension tool, and by showing a range of Dylan macros. Macros can be useful when you want to tailor the language to express a particular problem domain more concisely.
Pattern constraints. summarizes how constraints control pattern-variable matches.
Constraint |
Matches |
|---|---|
|
a lexeme (a Dylan word), including literal strings, symbols, and numbers and punctuation |
|
a Dylan identifier, including reserved identifiers, such as |
|
either |
|
a well-formed Dylan expression –- a constant, such as |
|
a well-formed Dylan body — a sequence of semicolon-separated constituents, each constituent being either a definition, local declaration, or expression |
|
a Dylan |
|
any sequence of Dylan tokens and parsed forms |