Changeset 22654 for docs

Timestamp:

10/19/08 19:55:59 (3 months ago)

Author:

moritz

Message:

[docs/tutorial] (aka book) many clean ups and enhancements to ch07_grammars.pod.
Also other small fixes.

Location:

docs/tutorial

Files:

• ch01_overview.pod (modified) (1 diff)
• ch05_subroutines.pod (modified) (1 diff)
• ch07_grammars.pod (modified) (26 diffs)

docs/tutorial/ch01_overview.pod

@@ -253 +253 @@
 catalyst for his thoughts.
 
+=head3 The test suite
+
+Z<CHP-1-SECT-3.1.4>
+
+X<Tang, Audrey>
+X<Pugs>
+X<Test suite>
+
+The design documents describe the Perl 6 language in prose, and the test
+suite is intended to translate that specification into code.
+
+In 2005 Audrey Tang started a Perl 6 compiler named I<Pugs>. It is
+written in Haskell, and moved very fast. The test suite began both as
+regression tests and as a feature wish list, and is now slowly being
+translated into an implementation agnostic, offical test suite that can
+be used by all implementations.
+
+Once it is done, every compiler that passes the test suite may name
+itself I<Perl 6>.
+
 =cut
+
+# vim: sw=3 ts=3 expandtab tw=72

docs/tutorial/ch05_subroutines.pod

@@ -923 +923 @@
 
    macro funky (Str $whatever) 
-      is parsed (/:w like (\w+), you know/)
+      is parsed (/:s like (\w+), you know/)
   {
       return { plain($whatever); };

docs/tutorial/ch07_grammars.pod

@@ -8 +8 @@
 TODO: This chapter is outdated in some ways
 
-  * It uses the old word "rule" instead of "regex" in far too many places
+  * It should be explained when we use "rule" and when "regex", and what
+    a "subrule" is.
   * The interpolation rules are outdated
-  * <'...'> and <"..."> are now '...' and "..."
   * some of the assertion syntax has changed, for example <foo()> means
     something different now
-  * Modifiers: explain :sigspace and :ratchet modifiers
-  * Modifiers: :u1, :u2... are now :bytes, :codes, :graphemes etc.
-  * Other missing modifiers: :overlap, :ignoreaccent
-  * The match object needs explanation
+  * Modifiers: explain :ratchet modifier
+  * The match object needs more explanation
 
 Z<CHP-7>
@@ -131 +129 @@
   regex digits {\d+}
 
-There are two more keywords that defines regexes similarly to C<regex>, but
-they imply different modifiers. C<token> introduces a regex that does
+There are two more keywords that defines regexes similarly to C<regex>, which
+imply slightly different behavior. C<token> introduces a regex that does
 not backtrack,N<technically it implies the C<:ratchet> modifier> (more details
 on that below; for now it's enough to know that it matches simple regexes
@@ -272 +270 @@
 The "word"-like metacharacters are C<.>, C<^>, C<^^>, C<$>, C<$$>. The
 C<.> matches any single character, even a newline character. Actually,
-what it matches by default is a Unicode grapheme, but you can change
-that behavior with a pragma in your code, or a modifier on the regex.
+Perl 6 has a the notion of a Unicode level, which determines if string
+manipulation happens on the byte, codepoint or grapheme level. C<.>
+matches a character in the current level, which defaults to grapheme.
+The Unicode level can be adjusted with a pragma or with modifiers.
 We'll talk more about modifiers in A<CHP-7-SECT-2.5>"Modifiers" later
 in this chapter. The C<^> and C<$> metacharacters are zero-width
@@ -416 +416 @@
 =cell Match an assertion.
 
+=row
+
+=cell C<< <( >>
+
+=cell Begin of capturing
+
+=row
+
+=cell C<< )> >>
+
+=cell End of capturring
+
 =end table
 
@@ -438 +450 @@
 X<regexes;variable interpolation>
 Note that since an ordinary variable now interpolates as a literal
-string by default, the C<\Q> escape is rarely needed.
+string by default, the C<\Q> escape is rarely needed. An interpolated
+array is interpreted as an alternation of all array elements.
 
 A<CHP-7-TABLE-3>Table 7-3 shows the escape sequences for regexes. 
@@ -455 +468 @@
 
 =bodyrows 
+
+=row
+
+=cell  C<'...'>
+
+=cell Tread everyhing between the quotes literally, except the backslash C<\>
+and  single quotes C<'>
+
+=row
+
+=cell  C<"...">
+
+=cell Like C<'...'>, but backslash escape sequences and variable interpolation
+are enabled
 
 =row 
@@ -704 +731 @@
 =row 
 
-=cell C**n>
+=cell C<**n>
 
 =cell C<**?n>
@@ -712 +739 @@
 =row 
 
-=cell C<**{n..m}>
-=cell C<**?{n..m}>
+=cell C<**n..m>
+
+=cell C<**?n..m>
 
 =cell Match at least R<n> and no more than R<m> times.
@@ -719 +747 @@
 =row 
 
-=cell C<E<lt>>R<n>C<...E<gt>>
-
-=cell C<E<lt>>R<n>C<...E<gt>?>
+=cell C<**n..*>
+
+=cell C<**?n..*>
 
 =cell Match at least R<n> times.
@@ -802 +830 @@
 =cell Negate any assertion.
 
+=cell
+
+=row C<< <.rule> >>
+
+=row Match named rule, wihtout capturing.
+
 =row 
 
@@ -810 +844 @@
 =row 
 
-=cell C<E<lt>[...]E<gt>>
+=cell C<E<lt>[...]E<gt>>, C<< <+[...]> >>
 
 =cell Match an enumerated character class.
@@ -819 +853 @@
 
 =cell Complement a character class (named or enumerated).
-
-=row 
-
-=cell C<E<lt>"..."E<gt>>
-
-=cell Match a literal string (interpolated at match time).
-
-=row 
-
-=cell C<E<lt>'...'E<gt>>
-
-=cell Match a literal string (not interpolated).
-
-=row 
-
-=cell C<E<lt>(...)E<gt>>
-
-=cell Boolean assertion. Execute a closure and match if it returns a true
-result.
 
 =row 
@@ -860 +875 @@
 =row 
 
-=cell C<E<lt>E<amp>sub()E<gt>>
-
-=cell Match an anonymous rule returned by a sub.
+=cell C<< <rule(...)> >>
+
+=cell Call a named rule with arguments.
 
 =row 
@@ -890 +905 @@
 cannot attach to the outside of a bare C</.../>. For example:
 
-  m:i/marvin/ # case insensitive
+  m:i /marvin/ # case insensitive
   rule names :i { marvin | ford | arthur }
 
-The single-character modifiers can be grouped, but the others must be
-separated by a colon:
-
-  m:wig/ zaphod /                        # OK
-  m:words:ignorecase:globally / zaphod / # OK
-  m:wordsignorecaseglobally / zaphod /   # Not OK
+Multiple modifiers can be chained, short and long names can
+be mixed:
+
+  m:s :i :g/ zaphod /
+  m:sigspace :i :global / zaphod /
+
+Modifiers can be negated with the C<:!pair> notation, so C<:!i> forces
+case-sensitive matching.
 
 Most of the modifiers can also go inside the rule, attached to the
@@ -905 +922 @@
 alteration of the pattern:
 
-  m/:w I saw [:i zaphod] / # only 'zaphod' is case insensitive
-
-The repetition modifiers (C<:R<N>x>, C<:R<N>th>, C<:once>,
-C<:globally>, and C<:exhaustive>) and the continue modifier (C<:cont>)
+  m/:s I saw [:i zaphod] / # only 'zaphod' is case insensitive
+
+The repetition modifiers (C<:R<N>x>, C<:R<N>th>,
+C<:global>, and C<:exhaustive>) and the continue modifier (C<:cont>)
 can't be lexically scoped, because they alter the return value of the
 entire rule.
@@ -917 +934 @@
 of the number. 
 
-The C<:once> modifier on a rule only allows it to match once. The rule
-will not match again until the you call the C<.reset> method on the rule
-object.
-
-The C<:globally> modifier matches as many times as possible. The
+The C<:global> modifier matches as many times as possible. The
 C<:exhaustive> modifier also matches as many times as possible, but in
 as many different ways as possible.
@@ -935 +948 @@
 
 By default, rules ignore literal whitespace within the pattern.  The
-C<:w> modifier makes rules sensitive to literal whitespace, but in an
-intelligent way. Any cluster of literal whitespace acts like an explicit
-C<\s+> when it separates two identifiers and C<\s*> everywhere else.
+C<:s> or C<:sigspace> modifier makes rules sensitive to literal whitespace,
+but in an intelligent way. Any cluster of literal whitespace acts like an
+explicit C<\s+> when it separates two identifiers and C<\s*> everywhere else.
+
+More specifically any literal whitespace in the regex is translated to
+an implict call to C<E<lt>.wsE<gt>>, where the C<ws> rule matches as
+mentioned above, but can also be overridden by the user.
 
 There are no modifiers to alter whether the matched string is treated as
@@ -970 +987 @@
 =cell Case-insensitive match.
 
-=row 
-
-=cell C<:I>
-
-=cell 
-
-=cell Case-sensitive match (on by default).
-
-=row 
-
-=cell C<:c>
-
-=cell C<:cont>
-
-=cell Continue where the previous match on the string left off.
-
-=row 
-
-=cell C<:w>
-
-=cell C<:words>
+=row
+
+=cell C<:a>
+
+=cell C<:ignoreaccent>
+
+=cell Ignore accents and other markings on characters.
+
+=row 
+
+=cell C<:c($pos)>
+
+=cell C<:continue($pos)>
+
+=cell Match at position C<$pos> or later. If C<$pos> is ommited, start where
+
+=row
+
+=cell C<:p>
+
+=cell C<:pos>
+
+=cell Match anchored at position C<$pos>. If C<$pos> is ommited, start where
+the previous match left off.
+
+=row 
+
+=cell C<:s>
+
+=cell C<:sigspace>
 
 =cell Literal whitespace in the pattern matches as C<\s+>
@@ -1013 +1039 @@
 =row 
 
-=cell 
-
-=cell C<:once>
-
-=cell Only match the pattern once.
-
-=row 
-
 =cell C<:g>
 
-=cell C<:globally>
+=cell C<:global>
 
 =cell Match the pattern as many times as possible, but only possibilities
 that don't overlap.
 
-=row 
-
-=cell C<:e>
+=row
+
+=cell C<:ov>
+
+=cell C<:overlap>
+
+=cell Match the pattern as many timies as possible, and allow overlapping
+matches, but only one match per starting position.
+
+=row 
+
+=cell C<:ex>
 
 =cell C<:exhaustive>
@@ -1041 +1068 @@
 =cell 
 
-=cell C<:u0>
-
-=cell . is a byte.
+=cell C<:bytes>
+
+=cell C<.> is a byte.
 
 =row 
@@ -1049 +1076 @@
 =cell 
 
-=cell C<:u1>
-
-=cell . is a Unicode codepoint.
+=cell C<:codes>
+
+=cell C<.> is a Unicode codepoint.
 
 =row 
@@ -1057 +1084 @@
 =cell 
 
-=cell C<:u2>
-
-=cell . is a Unicode grapheme.
+=cell C<:graphs>
+
+=cell C<.> is a Unicode grapheme.
 
 =row 
@@ -1065 +1092 @@
 =cell 
 
-=cell C<:u3>
-
-=cell . is language dependent.
+=cell C<:chars>
+
+=cell C<.> matches whatever the current Unicode level corresponds to
+(this is the default).
+
+=row
+
+=cell
+
+=cell C<:ratchet>
+
+=cell Imply a C<:> after each atom (see "Backtracking Control" below).
 
 =row 
@@ -1079 +1115 @@
 =end table
 
+=head2 Substition Modifiers
+
+Special modifiers are available for substitions that do not make sense
+on normal matches.
+
+The C<:samecase>, or short C<:ii> modifier implies the C<:ignorecase>
+modifier, but also carries the case information on a
+character-by-character base
+
+   my $s = 'The Quick Brown Fox';
+   $s ~~ s:ii/brown/blue/;
+   say $s;           # The Quick Blue Fox
+
+If the C<:sigspace> modifier is also present, a slightly more
+intelligent algorithm is used. If the source string follows one of the
+case patterns in $table (XXX: make that a proper cross-link),
+that pattern is recognized and applied onto the
+substitution string.
+
+   $_ = 'All Words Capialized';
+   s:s:ii/.*/other words/;
+   .say;             # Other Words
+
+There's a shortcut for C<s:s> named C<ss>, so you could have written the
+example above aswidth="348" height="300"
+ C<ss:ii/.*/other words/>.
+
+=begin table picture Case patterns for the :samecase modifier
+
+=headrow
+
+=cell Pattern
+
+=cell Corresponding code
+
+=bodyrows
+
+=row
+
+=cell ALL UPPERCASE
+
+=cell C<.uc>
+
+=row
+
+=cell all lowercase
+
+=cell C<.lc>
+
+=row
+
+=cell Every Word Capitalized
+
+=cell C<.lc.capitalize>
+
+=row
+
+=cell First letter upper, rest lower
+
+=cell C<.lc.ucfirst>
+
+=row
+
+=cell fIRST LETTER LOWER, REST UPPER
+
+=cell C<.uc.lcfirst>
+
+=end table
+
+A similar modifier is C<:sameaccent> (short C<:aa>). Instead of carrying
+case information, it carries accent and marking information.
+
+   my $stuff = 'Möhre';
+   $stuff ~~ s:aa/a/o/;
+   say $stuff;          # Mähre
+
+The third substitution modifier is C<:samespace>, short C<:ss>. It preserves
+whitespace that is matched by implicit C<E<lt>.wsE<gt>> rules:
+
+   my $s = "Some   white\t\n spaces";
+   $s ~~ s:ss/\w+ \w+ \w+/Completely different text/;
+   # $s is now "Completely   different\t\n text"
 
 =head1 Built-in Rules
@@ -1160 +1278 @@
 =cell Zero-width lookbehind. Assert that you're I<after> a pattern.
 
-=row 
-
-=cell C<E<lt>prop ...E<gt>>
-
-=cell Match any character with the named property.
-
-=row 
-
-=cell C<E<lt>replace(...)E<gt>>
-
-=cell Replace everything matched so far in the rule or subrule with the
-given string (under consideration).
-
 =end table
 
@@ -1245 +1350 @@
 =end table
 
-=head1 Hypothetical Variables
+The C<:ratchet> modifier, which is implied by regexes declared with the
+C<token> or C<rule> keyword, disables backtracking in the subrule, which
+is the same as adding a C<:> after every atom.
+
+=head1 The Match Object
 
 Z<CHP-7-SECT-5>
 
-X<variables;hypothetical>
-X<hypothetical variables>
+X<object;match>
 X<rules;captures>
-Hypothetical variables are a powerful way of building up data structures
-from within a match. Ordinary captures with C<()> store the result of
-the captures in C<$0>, C<$1>, etc. The values stored in these variables
-will be kept if the match is successful, but thrown away if the match
-fails (hence the term "hypothetical"). The numbered capture variables
-are accessible outside the match, but only within the immediate
-surrounding lexical scope:
-
-  "Zaphod Beeblebrox" ~~ m:w/ (\w+) (\w+) /;
-  
-  print $0; # prints Zaphod
-
-You can also capture into any user-defined variable with the binding
-operator C<:=>. These variables must already be defined in the lexical
-scope surrounding the rule:
-
-  my $person;
-  "Zaphod's just this guy." ~~ / ^ $person := (\w+) /;
-  print $person; # prints Zaphod
-
-Repeated matches can be captured into an array:
-
-  my @words;
-  "feefifofum" ~~ / @words := (f<-[f]>+)* /;
-  # @words contains ("fee", "fi", "fo", "fum")
-
-Pairs of repeated matches can be captured into a hash:
-
-  my %customers;
-  $records ~~ m:w/ %customers := [ E<lt>idE<gt> = E<lt>nameE<gt> \n]* /;
-
-If you don't need the captured value outside the rule, use a C<$?>
-variable instead. These are only directly accessible within the rule:
-
-  "Zaphod saw Zaphod" ~~ m:w/ $?name := (\w+) \w+ $?name/;
-
-A match of a named rule stores the result in a C<$?> variable with the
-same name as the rule. These variables are also accessible only within
-the rule:
-
-  "Zaphod saw Zaphod" ~~ m:w/ E<lt>nameE<gt> \w+ $?name /;
-
+
+A regex match produces a I<Match> object, which contains all information
+about the match, including start and end position, matched string, and all
+captures.
+
+The match object is returned from a regex match, and is also stored in
+the special variable C<$/>.
+
+   my $match = 'Zaphod Beeblebrox' ~~ m/\w+/;   
+   say $match;    # prints Zaphod
+
+In string context it evaluates to the text of the matched part of the
+string.
+
+Table A<CHP-7-TABLE-Match> summarises the properties of the match object.
+
+The variables C<$0>, C<$1>, C<$2> etc. are aliases to C<$/[0]>,
+C<$/[1]>, C<$/[2]>, and C<$E<lt>nameE<gt>> is an alias to
+C<$/E<lt>nameE<gt>>. Likewise an empty C<@()> is the same as C<@($/)>,
+and C<%()> stands for C<%($/)>.
+
+Match variables can also store a different scalar object. A closure in a
+regex can store such an object by calling C<make>, and can be accessed
+by forcing scalar context with C<$( $/ )>:
+
+   regex herd :i :s {
+         (\d+)
+         (\w+)s?
+         {
+            make Herd.new(
+                  animal => $1.capitalize
+                  count  => $0,
+                 );
+         }
+   }
+   'Yesterday we saw 4 mooses' ~~ m/ <herd> /;
+   # now $($<herd>) contains the new Herd object
+
+This can be used to build object trees directly from regex matches.
+
+=begin table picture Properties of the Match object
+
+Z<CHP-7-TABLE-Match>
+
+=headrow
+
+=cell Property 
+
+=cell Description
+
+=bodyrows
+
+=row
+
+=cell C<?$/>
+
+=cell True if the match was successful.
+
+=row
+
+=cell C<$/.text>
+
+=cell The matched part of the string.
+
+=row
+
+=cell C<$/.from>
+
+=cell Start position of the match.
+
+=row
+
+=cell C<$/.to>
+
+=cell End position of the match.
+
+=row
+
+=cell C<@( $/ )>
+
+=cell List of all positional captures.
+
+=row 
+
+=cell C<%( $/ )>
+
+=cell Hash of all named captures.
+
+=row
+
+=cell C<$/[$n]>
+
+=cell C<$n>th positional capture.
+
+=row
+
+=cell C<$/E<lt>nameE<gt>>
+
+=cell Access to particular named capture.
+
+=end table
+
+Capture variables are always match objects, and contain the information
+of their respective sub matches.
+
+   m/ ( a ( geek ) ( passes ) )  ( many tests ) /
+      |   |      | |        | |  |            |
+      |   $/[0][0] $/[0][1]-+ |  |            |
+      |                       |  |            |
+      $/[0]-------------------+  $/[1] -------+
+
+If a capturing group is quantified, it automatically becomes an array of
+match objects. Subsequent matches are not renumbered:
+
+   '12 45 books' ~~ m:s/ ( \d+ )+ (\w+) /
+   say $0[0];     # 12
+   say $0[1];     # 45
+   say $1;        # books
+
+When a subrule is called with the C<E<lt>subruleE<gt>> syntax, it
+produces a named capture of name C<subrule>. That named can be
+changed with the C<E<lt>newname=subruleE<gt>> syntax.
+
+   token identifier { \w+ }
+   token number     { \d+ }
+   $_ = '24 hours'
+   if m:s/<number> <unit=identifier> / {
+      say "Number: $<number>. Unit: $<unit>";
+   }
+
+These variables are also available iin the regex itself:
+
+  "Zaphod saw Zaphod" ~~ m:s/ E<lt>nameE<gt> \w+ $/<name> /;
 
 
 =cut
+
+# vim: sw=3 ts=3 expandtab ft=pod tw=72