1  /*!
2  This crate provides a library for parsing, compiling, and executing regular
3  expressions. Its syntax is similar to Perl-style regular expressions, but lacks
4  a few features like look around and backreferences. In exchange, all searches
5  execute in linear time with respect to the size of the regular expression and
6  search text.
7  
8  This crate's documentation provides some simple examples, describes
9  [Unicode support](#unicode) and exhaustively lists the
10  [supported syntax](#syntax).
11  
12  For more specific details on the API for regular expressions, please see the
13  documentation for the [`Regex`](struct.Regex.html) type.
14  
15  # Usage
16  
17  This crate is [on crates.io](https://crates.io/crates/regex) and can be
18  used by adding `regex` to your dependencies in your project's `Cargo.toml`.
19  
20  ```toml
21  [dependencies]
22  regex = "1"
23  ```
24  
25  # Example: find a date
26  
27  General use of regular expressions in this package involves compiling an
28  expression and then using it to search, split or replace text. For example,
29  to confirm that some text resembles a date:
30  
31  ```rust
32  use regex::Regex;
33  let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
34  assert!(re.is_match("2014-01-01"));
35  ```
36  
37  Notice the use of the `^` and `$` anchors. In this crate, every expression
38  is executed with an implicit `.*?` at the beginning and end, which allows
39  it to match anywhere in the text. Anchors can be used to ensure that the
40  full text matches an expression.
41  
42  This example also demonstrates the utility of
43  [raw strings](https://doc.rust-lang.org/stable/reference/tokens.html#raw-string-literals)
44  in Rust, which
45  are just like regular strings except they are prefixed with an `r` and do
46  not process any escape sequences. For example, `"\\d"` is the same
47  expression as `r"\d"`.
48  
49  # Example: Avoid compiling the same regex in a loop
50  
51  It is an anti-pattern to compile the same regular expression in a loop
52  since compilation is typically expensive. (It takes anywhere from a few
53  microseconds to a few **milliseconds** depending on the size of the
54  regex.) Not only is compilation itself expensive, but this also prevents
55  optimizations that reuse allocations internally to the matching engines.
56  
57  In Rust, it can sometimes be a pain to pass regular expressions around if
58  they're used from inside a helper function. Instead, we recommend using the
59  [`lazy_static`](https://crates.io/crates/lazy_static) crate to ensure that
60  regular expressions are compiled exactly once.
61  
62  For example:
63  
64  ```rust
65  use lazy_static::lazy_static;
66  use regex::Regex;
67  
68  fn some_helper_function(text: &str) -> bool {
69      lazy_static! {
70          static ref RE: Regex = Regex::new("...").unwrap();
71      }
72      RE.is_match(text)
73  }
74  
75  fn main() {}
76  ```
77  
78  Specifically, in this example, the regex will be compiled when it is used for
79  the first time. On subsequent uses, it will reuse the previous compilation.
80  
81  # Example: iterating over capture groups
82  
83  This crate provides convenient iterators for matching an expression
84  repeatedly against a search string to find successive non-overlapping
85  matches. For example, to find all dates in a string and be able to access
86  them by their component pieces:
87  
88  ```rust
89  # use regex::Regex;
90  # fn main() {
91  let re = Regex::new(r"(\d{4})-(\d{2})-(\d{2})").unwrap();
92  let text = "2012-03-14, 2013-01-01 and 2014-07-05";
93  for cap in re.captures_iter(text) {
94      println!("Month: {} Day: {} Year: {}", &cap[2], &cap[3], &cap[1]);
95  }
96  // Output:
97  // Month: 03 Day: 14 Year: 2012
98  // Month: 01 Day: 01 Year: 2013
99  // Month: 07 Day: 05 Year: 2014
100  # }
101  ```
102  
103  Notice that the year is in the capture group indexed at `1`. This is
104  because the *entire match* is stored in the capture group at index `0`.
105  
106  # Example: replacement with named capture groups
107  
108  Building on the previous example, perhaps we'd like to rearrange the date
109  formats. This can be done with text replacement. But to make the code
110  clearer, we can *name*  our capture groups and use those names as variables
111  in our replacement text:
112  
113  ```rust
114  # use regex::Regex;
115  # fn main() {
116  let re = Regex::new(r"(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})").unwrap();
117  let before = "2012-03-14, 2013-01-01 and 2014-07-05";
118  let after = re.replace_all(before, "$m/$d/$y");
119  assert_eq!(after, "03/14/2012, 01/01/2013 and 07/05/2014");
120  # }
121  ```
122  
123  The `replace` methods are actually polymorphic in the replacement, which
124  provides more flexibility than is seen here. (See the documentation for
125  `Regex::replace` for more details.)
126  
127  Note that if your regex gets complicated, you can use the `x` flag to
128  enable insignificant whitespace mode, which also lets you write comments:
129  
130  ```rust
131  # use regex::Regex;
132  # fn main() {
133  let re = Regex::new(r"(?x)
134    (?P<y>\d{4}) # the year
135    -
136    (?P<m>\d{2}) # the month
137    -
138    (?P<d>\d{2}) # the day
139  ").unwrap();
140  let before = "2012-03-14, 2013-01-01 and 2014-07-05";
141  let after = re.replace_all(before, "$m/$d/$y");
142  assert_eq!(after, "03/14/2012, 01/01/2013 and 07/05/2014");
143  # }
144  ```
145  
146  If you wish to match against whitespace in this mode, you can still use `\s`,
147  `\n`, `\t`, etc. For escaping a single space character, you can escape it
148  directly with `\ `, use its hex character code `\x20` or temporarily disable
149  the `x` flag, e.g., `(?-x: )`.
150  
151  # Example: match multiple regular expressions simultaneously
152  
153  This demonstrates how to use a `RegexSet` to match multiple (possibly
154  overlapping) regular expressions in a single scan of the search text:
155  
156  ```rust
157  use regex::RegexSet;
158  
159  let set = RegexSet::new(&[
160      r"\w+",
161      r"\d+",
162      r"\pL+",
163      r"foo",
164      r"bar",
165      r"barfoo",
166      r"foobar",
167  ]).unwrap();
168  
169  // Iterate over and collect all of the matches.
170  let matches: Vec<_> = set.matches("foobar").into_iter().collect();
171  assert_eq!(matches, vec![0, 2, 3, 4, 6]);
172  
173  // You can also test whether a particular regex matched:
174  let matches = set.matches("foobar");
175  assert!(!matches.matched(5));
176  assert!(matches.matched(6));
177  ```
178  
179  # Pay for what you use
180  
181  With respect to searching text with a regular expression, there are three
182  questions that can be asked:
183  
184  1. Does the text match this expression?
185  2. If so, where does it match?
186  3. Where did the capturing groups match?
187  
188  Generally speaking, this crate could provide a function to answer only #3,
189  which would subsume #1 and #2 automatically. However, it can be significantly
190  more expensive to compute the location of capturing group matches, so it's best
191  not to do it if you don't need to.
192  
193  Therefore, only use what you need. For example, don't use `find` if you
194  only need to test if an expression matches a string. (Use `is_match`
195  instead.)
196  
197  # Unicode
198  
199  This implementation executes regular expressions **only** on valid UTF-8
200  while exposing match locations as byte indices into the search string. (To
201  relax this restriction, use the [`bytes`](bytes/index.html) sub-module.)
202  
203  Only simple case folding is supported. Namely, when matching
204  case-insensitively, the characters are first mapped using the "simple" case
205  folding rules defined by Unicode.
206  
207  Regular expressions themselves are **only** interpreted as a sequence of
208  Unicode scalar values. This means you can use Unicode characters directly
209  in your expression:
210  
211  ```rust
212  # use regex::Regex;
213  # fn main() {
214  let re = Regex::new(r"(?i)Δ+").unwrap();
215  let mat = re.find("ΔδΔ").unwrap();
216  assert_eq!((mat.start(), mat.end()), (0, 6));
217  # }
218  ```
219  
220  Most features of the regular expressions in this crate are Unicode aware. Here
221  are some examples:
222  
223  * `.` will match any valid UTF-8 encoded Unicode scalar value except for `\n`.
224    (To also match `\n`, enable the `s` flag, e.g., `(?s:.)`.)
225  * `\w`, `\d` and `\s` are Unicode aware. For example, `\s` will match all forms
226    of whitespace categorized by Unicode.
227  * `\b` matches a Unicode word boundary.
228  * Negated character classes like `[^a]` match all Unicode scalar values except
229    for `a`.
230  * `^` and `$` are **not** Unicode aware in multi-line mode. Namely, they only
231    recognize `\n` and not any of the other forms of line terminators defined
232    by Unicode.
233  
234  Unicode general categories, scripts, script extensions, ages and a smattering
235  of boolean properties are available as character classes. For example, you can
236  match a sequence of numerals, Greek or Cherokee letters:
237  
238  ```rust
239  # use regex::Regex;
240  # fn main() {
241  let re = Regex::new(r"[\pN\p{Greek}\p{Cherokee}]+").unwrap();
242  let mat = re.find("abcΔᎠβⅠᏴγδⅡxyz").unwrap();
243  assert_eq!((mat.start(), mat.end()), (3, 23));
244  # }
245  ```
246  
247  For a more detailed breakdown of Unicode support with respect to
248  [UTS#18](https://unicode.org/reports/tr18/),
249  please see the
250  [UNICODE](https://github.com/rust-lang/regex/blob/master/UNICODE.md)
251  document in the root of the regex repository.
252  
253  # Opt out of Unicode support
254  
255  The `bytes` sub-module provides a `Regex` type that can be used to match
256  on `&[u8]`. By default, text is interpreted as UTF-8 just like it is with
257  the main `Regex` type. However, this behavior can be disabled by turning
258  off the `u` flag, even if doing so could result in matching invalid UTF-8.
259  For example, when the `u` flag is disabled, `.` will match any byte instead
260  of any Unicode scalar value.
261  
262  Disabling the `u` flag is also possible with the standard `&str`-based `Regex`
263  type, but it is only allowed where the UTF-8 invariant is maintained. For
264  example, `(?-u:\w)` is an ASCII-only `\w` character class and is legal in an
265  `&str`-based `Regex`, but `(?-u:\xFF)` will attempt to match the raw byte
266  `\xFF`, which is invalid UTF-8 and therefore is illegal in `&str`-based
267  regexes.
268  
269  Finally, since Unicode support requires bundling large Unicode data
270  tables, this crate exposes knobs to disable the compilation of those
271  data tables, which can be useful for shrinking binary size and reducing
272  compilation times. For details on how to do that, see the section on [crate
273  features](#crate-features).
274  
275  # Syntax
276  
277  The syntax supported in this crate is documented below.
278  
279  Note that the regular expression parser and abstract syntax are exposed in
280  a separate crate, [`regex-syntax`](https://docs.rs/regex-syntax).
281  
282  ## Matching one character
283  
284  <pre class="rust">
285  .             any character except new line (includes new line with s flag)
286  \d            digit (\p{Nd})
287  \D            not digit
288  \pN           One-letter name Unicode character class
289  \p{Greek}     Unicode character class (general category or script)
290  \PN           Negated one-letter name Unicode character class
291  \P{Greek}     negated Unicode character class (general category or script)
292  </pre>
293  
294  ### Character classes
295  
296  <pre class="rust">
297  [xyz]         A character class matching either x, y or z (union).
298  [^xyz]        A character class matching any character except x, y and z.
299  [a-z]         A character class matching any character in range a-z.
300  [[:alpha:]]   ASCII character class ([A-Za-z])
301  [[:^alpha:]]  Negated ASCII character class ([^A-Za-z])
302  [x[^xyz]]     Nested/grouping character class (matching any character except y and z)
303  [a-y&&xyz]    Intersection (matching x or y)
304  [0-9&&[^4]]   Subtraction using intersection and negation (matching 0-9 except 4)
305  [0-9--4]      Direct subtraction (matching 0-9 except 4)
306  [a-g~~b-h]    Symmetric difference (matching `a` and `h` only)
307  [\[\]]        Escaping in character classes (matching [ or ])
308  </pre>
309  
310  Any named character class may appear inside a bracketed `[...]` character
311  class. For example, `[\p{Greek}[:digit:]]` matches any Greek or ASCII
312  digit. `[\p{Greek}&&\pL]` matches Greek letters.
313  
314  Precedence in character classes, from most binding to least:
315  
316  1. Ranges: `a-cd` == `[a-c]d`
317  2. Union: `ab&&bc` == `[ab]&&[bc]`
318  3. Intersection: `^a-z&&b` == `^[a-z&&b]`
319  4. Negation
320  
321  ## Composites
322  
323  <pre class="rust">
324  xy    concatenation (x followed by y)
325  x|y   alternation (x or y, prefer x)
326  </pre>
327  
328  ## Repetitions
329  
330  <pre class="rust">
331  x*        zero or more of x (greedy)
332  x+        one or more of x (greedy)
333  x?        zero or one of x (greedy)
334  x*?       zero or more of x (ungreedy/lazy)
335  x+?       one or more of x (ungreedy/lazy)
336  x??       zero or one of x (ungreedy/lazy)
337  x{n,m}    at least n x and at most m x (greedy)
338  x{n,}     at least n x (greedy)
339  x{n}      exactly n x
340  x{n,m}?   at least n x and at most m x (ungreedy/lazy)
341  x{n,}?    at least n x (ungreedy/lazy)
342  x{n}?     exactly n x
343  </pre>
344  
345  ## Empty matches
346  
347  <pre class="rust">
348  ^     the beginning of text (or start-of-line with multi-line mode)
349  $     the end of text (or end-of-line with multi-line mode)
350  \A    only the beginning of text (even with multi-line mode enabled)
351  \z    only the end of text (even with multi-line mode enabled)
352  \b    a Unicode word boundary (\w on one side and \W, \A, or \z on other)
353  \B    not a Unicode word boundary
354  </pre>
355  
356  The empty regex is valid and matches the empty string. For example, the empty
357  regex matches `abc` at positions `0`, `1`, `2` and `3`.
358  
359  ## Grouping and flags
360  
361  <pre class="rust">
362  (exp)          numbered capture group (indexed by opening parenthesis)
363  (?P&lt;name&gt;exp)  named (also numbered) capture group (allowed chars: [_0-9a-zA-Z.\[\]])
364  (?:exp)        non-capturing group
365  (?flags)       set flags within current group
366  (?flags:exp)   set flags for exp (non-capturing)
367  </pre>
368  
369  Flags are each a single character. For example, `(?x)` sets the flag `x`
370  and `(?-x)` clears the flag `x`. Multiple flags can be set or cleared at
371  the same time: `(?xy)` sets both the `x` and `y` flags and `(?x-y)` sets
372  the `x` flag and clears the `y` flag.
373  
374  All flags are by default disabled unless stated otherwise. They are:
375  
376  <pre class="rust">
377  i     case-insensitive: letters match both upper and lower case
378  m     multi-line mode: ^ and $ match begin/end of line
379  s     allow . to match \n
380  U     swap the meaning of x* and x*?
381  u     Unicode support (enabled by default)
382  x     ignore whitespace and allow line comments (starting with `#`)
383  </pre>
384  
385  Flags can be toggled within a pattern. Here's an example that matches
386  case-insensitively for the first part but case-sensitively for the second part:
387  
388  ```rust
389  # use regex::Regex;
390  # fn main() {
391  let re = Regex::new(r"(?i)a+(?-i)b+").unwrap();
392  let cap = re.captures("AaAaAbbBBBb").unwrap();
393  assert_eq!(&cap[0], "AaAaAbb");
394  # }
395  ```
396  
397  Notice that the `a+` matches either `a` or `A`, but the `b+` only matches
398  `b`.
399  
400  Multi-line mode means `^` and `$` no longer match just at the beginning/end of
401  the input, but at the beginning/end of lines:
402  
403  ```
404  # use regex::Regex;
405  let re = Regex::new(r"(?m)^line \d+").unwrap();
406  let m = re.find("line one\nline 2\n").unwrap();
407  assert_eq!(m.as_str(), "line 2");
408  ```
409  
410  Note that `^` matches after new lines, even at the end of input:
411  
412  ```
413  # use regex::Regex;
414  let re = Regex::new(r"(?m)^").unwrap();
415  let m = re.find_iter("test\n").last().unwrap();
416  assert_eq!((m.start(), m.end()), (5, 5));
417  ```
418  
419  Here is an example that uses an ASCII word boundary instead of a Unicode
420  word boundary:
421  
422  ```rust
423  # use regex::Regex;
424  # fn main() {
425  let re = Regex::new(r"(?-u:\b).+(?-u:\b)").unwrap();
426  let cap = re.captures("$$abc$$").unwrap();
427  assert_eq!(&cap[0], "abc");
428  # }
429  ```
430  
431  ## Escape sequences
432  
433  <pre class="rust">
434  \*          literal *, works for any punctuation character: \.+*?()|[]{}^$
435  \a          bell (\x07)
436  \f          form feed (\x0C)
437  \t          horizontal tab
438  \n          new line
439  \r          carriage return
440  \v          vertical tab (\x0B)
441  \123        octal character code (up to three digits) (when enabled)
442  \x7F        hex character code (exactly two digits)
443  \x{10FFFF}  any hex character code corresponding to a Unicode code point
444  \u007F      hex character code (exactly four digits)
445  \u{7F}      any hex character code corresponding to a Unicode code point
446  \U0000007F  hex character code (exactly eight digits)
447  \U{7F}      any hex character code corresponding to a Unicode code point
448  </pre>
449  
450  ## Perl character classes (Unicode friendly)
451  
452  These classes are based on the definitions provided in
453  [UTS#18](https://www.unicode.org/reports/tr18/#Compatibility_Properties):
454  
455  <pre class="rust">
456  \d     digit (\p{Nd})
457  \D     not digit
458  \s     whitespace (\p{White_Space})
459  \S     not whitespace
460  \w     word character (\p{Alphabetic} + \p{M} + \d + \p{Pc} + \p{Join_Control})
461  \W     not word character
462  </pre>
463  
464  ## ASCII character classes
465  
466  <pre class="rust">
467  [[:alnum:]]    alphanumeric ([0-9A-Za-z])
468  [[:alpha:]]    alphabetic ([A-Za-z])
469  [[:ascii:]]    ASCII ([\x00-\x7F])
470  [[:blank:]]    blank ([\t ])
471  [[:cntrl:]]    control ([\x00-\x1F\x7F])
472  [[:digit:]]    digits ([0-9])
473  [[:graph:]]    graphical ([!-~])
474  [[:lower:]]    lower case ([a-z])
475  [[:print:]]    printable ([ -~])
476  [[:punct:]]    punctuation ([!-/:-@\[-`{-~])
477  [[:space:]]    whitespace ([\t\n\v\f\r ])
478  [[:upper:]]    upper case ([A-Z])
479  [[:word:]]     word characters ([0-9A-Za-z_])
480  [[:xdigit:]]   hex digit ([0-9A-Fa-f])
481  </pre>
482  
483  # Crate features
484  
485  By default, this crate tries pretty hard to make regex matching both as fast
486  as possible and as correct as it can be, within reason. This means that there
487  is a lot of code dedicated to performance, the handling of Unicode data and the
488  Unicode data itself. Overall, this leads to more dependencies, larger binaries
489  and longer compile times.  This trade off may not be appropriate in all cases,
490  and indeed, even when all Unicode and performance features are disabled, one
491  is still left with a perfectly serviceable regex engine that will work well
492  in many cases.
493  
494  This crate exposes a number of features for controlling that trade off. Some
495  of these features are strictly performance oriented, such that disabling them
496  won't result in a loss of functionality, but may result in worse performance.
497  Other features, such as the ones controlling the presence or absence of Unicode
498  data, can result in a loss of functionality. For example, if one disables the
499  `unicode-case` feature (described below), then compiling the regex `(?i)a`
500  will fail since Unicode case insensitivity is enabled by default. Instead,
501  callers must use `(?i-u)a` instead to disable Unicode case folding. Stated
502  differently, enabling or disabling any of the features below can only add or
503  subtract from the total set of valid regular expressions. Enabling or disabling
504  a feature will never modify the match semantics of a regular expression.
505  
506  All features below are enabled by default.
507  
508  ### Ecosystem features
509  
510  * **std** -
511    When enabled, this will cause `regex` to use the standard library. Currently,
512    disabling this feature will always result in a compilation error. It is
513    intended to add `alloc`-only support to regex in the future.
514  
515  ### Performance features
516  
517  * **perf** -
518    Enables all performance related features. This feature is enabled by default
519    and will always cover all features that improve performance, even if more
520    are added in the future.
521  * **perf-dfa** -
522    Enables the use of a lazy DFA for matching. The lazy DFA is used to compile
523    portions of a regex to a very fast DFA on an as-needed basis. This can
524    result in substantial speedups, usually by an order of magnitude on large
525    haystacks. The lazy DFA does not bring in any new dependencies, but it can
526    make compile times longer.
527  * **perf-inline** -
528    Enables the use of aggressive inlining inside match routines. This reduces
529    the overhead of each match. The aggressive inlining, however, increases
530    compile times and binary size.
531  * **perf-literal** -
532    Enables the use of literal optimizations for speeding up matches. In some
533    cases, literal optimizations can result in speedups of _several_ orders of
534    magnitude. Disabling this drops the `aho-corasick` and `memchr` dependencies.
535  * **perf-cache** -
536    This feature used to enable a faster internal cache at the cost of using
537    additional dependencies, but this is no longer an option. A fast internal
538    cache is now used unconditionally with no additional dependencies. This may
539    change in the future.
540  
541  ### Unicode features
542  
543  * **unicode** -
544    Enables all Unicode features. This feature is enabled by default, and will
545    always cover all Unicode features, even if more are added in the future.
546  * **unicode-age** -
547    Provide the data for the
548    [Unicode `Age` property](https://www.unicode.org/reports/tr44/tr44-24.html#Character_Age).
549    This makes it possible to use classes like `\p{Age:6.0}` to refer to all
550    codepoints first introduced in Unicode 6.0
551  * **unicode-bool** -
552    Provide the data for numerous Unicode boolean properties. The full list
553    is not included here, but contains properties like `Alphabetic`, `Emoji`,
554    `Lowercase`, `Math`, `Uppercase` and `White_Space`.
555  * **unicode-case** -
556    Provide the data for case insensitive matching using
557    [Unicode's "simple loose matches" specification](https://www.unicode.org/reports/tr18/#Simple_Loose_Matches).
558  * **unicode-gencat** -
559    Provide the data for
560    [Unicode general categories](https://www.unicode.org/reports/tr44/tr44-24.html#General_Category_Values).
561    This includes, but is not limited to, `Decimal_Number`, `Letter`,
562    `Math_Symbol`, `Number` and `Punctuation`.
563  * **unicode-perl** -
564    Provide the data for supporting the Unicode-aware Perl character classes,
565    corresponding to `\w`, `\s` and `\d`. This is also necessary for using
566    Unicode-aware word boundary assertions. Note that if this feature is
567    disabled, the `\s` and `\d` character classes are still available if the
568    `unicode-bool` and `unicode-gencat` features are enabled, respectively.
569  * **unicode-script** -
570    Provide the data for
571    [Unicode scripts and script extensions](https://www.unicode.org/reports/tr24/).
572    This includes, but is not limited to, `Arabic`, `Cyrillic`, `Hebrew`,
573    `Latin` and `Thai`.
574  * **unicode-segment** -
575    Provide the data necessary to provide the properties used to implement the
576    [Unicode text segmentation algorithms](https://www.unicode.org/reports/tr29/).
577    This enables using classes like `\p{gcb=Extend}`, `\p{wb=Katakana}` and
578    `\p{sb=ATerm}`.
579  
580  
581  # Untrusted input
582  
583  This crate can handle both untrusted regular expressions and untrusted
584  search text.
585  
586  Untrusted regular expressions are handled by capping the size of a compiled
587  regular expression.
588  (See [`RegexBuilder::size_limit`](struct.RegexBuilder.html#method.size_limit).)
589  Without this, it would be trivial for an attacker to exhaust your system's
590  memory with expressions like `a{100}{100}{100}`.
591  
592  Untrusted search text is allowed because the matching engine(s) in this
593  crate have time complexity `O(mn)` (with `m ~ regex` and `n ~ search
594  text`), which means there's no way to cause exponential blow-up like with
595  some other regular expression engines. (We pay for this by disallowing
596  features like arbitrary look-ahead and backreferences.)
597  
598  When a DFA is used, pathological cases with exponential state blow-up are
599  avoided by constructing the DFA lazily or in an "online" manner. Therefore,
600  at most one new state can be created for each byte of input. This satisfies
601  our time complexity guarantees, but can lead to memory growth
602  proportional to the size of the input. As a stopgap, the DFA is only
603  allowed to store a fixed number of states. When the limit is reached, its
604  states are wiped and continues on, possibly duplicating previous work. If
605  the limit is reached too frequently, it gives up and hands control off to
606  another matching engine with fixed memory requirements.
607  (The DFA size limit can also be tweaked. See
608  [`RegexBuilder::dfa_size_limit`](struct.RegexBuilder.html#method.dfa_size_limit).)
609  */
610  
611  #![deny(missing_docs)]
612  #![cfg_attr(feature = "pattern", feature(pattern))]
613  #![warn(missing_debug_implementations)]
614  
615  #[cfg(not(feature = "std"))]
616  compile_error!("`std` feature is currently required to build this crate");
617  
618  // To check README's example
619  // TODO: Re-enable this once the MSRV is 1.43 or greater.
620  // See: https://github.com/rust-lang/regex/issues/684
621  // See: https://github.com/rust-lang/regex/issues/685
622  // #[cfg(doctest)]
623  // doc_comment::doctest!("../README.md");
624  
625  #[cfg(feature = "std")]
626  pub use crate::error::Error;
627  #[cfg(feature = "std")]
628  pub use crate::re_builder::set_unicode::*;
629  #[cfg(feature = "std")]
630  pub use crate::re_builder::unicode::*;
631  #[cfg(feature = "std")]
632  pub use crate::re_set::unicode::*;
633  #[cfg(feature = "std")]
634  pub use crate::re_unicode::{
635      escape, CaptureLocations, CaptureMatches, CaptureNames, Captures,
636      Locations, Match, Matches, NoExpand, Regex, Replacer, ReplacerRef, Split,
637      SplitN, SubCaptureMatches,
638  };
639  
640  /**
641  Match regular expressions on arbitrary bytes.
642  
643  This module provides a nearly identical API to the one found in the
644  top-level of this crate. There are two important differences:
645  
646  1. Matching is done on `&[u8]` instead of `&str`. Additionally, `Vec<u8>`
647  is used where `String` would have been used.
648  2. Unicode support can be disabled even when disabling it would result in
649  matching invalid UTF-8 bytes.
650  
651  # Example: match null terminated string
652  
653  This shows how to find all null-terminated strings in a slice of bytes:
654  
655  ```rust
656  # use regex::bytes::Regex;
657  let re = Regex::new(r"(?-u)(?P<cstr>[^\x00]+)\x00").unwrap();
658  let text = b"foo\x00bar\x00baz\x00";
659  
660  // Extract all of the strings without the null terminator from each match.
661  // The unwrap is OK here since a match requires the `cstr` capture to match.
662  let cstrs: Vec<&[u8]> =
663      re.captures_iter(text)
664        .map(|c| c.name("cstr").unwrap().as_bytes())
665        .collect();
666  assert_eq!(vec![&b"foo"[..], &b"bar"[..], &b"baz"[..]], cstrs);
667  ```
668  
669  # Example: selectively enable Unicode support
670  
671  This shows how to match an arbitrary byte pattern followed by a UTF-8 encoded
672  string (e.g., to extract a title from a Matroska file):
673  
674  ```rust
675  # use std::str;
676  # use regex::bytes::Regex;
677  let re = Regex::new(
678      r"(?-u)\x7b\xa9(?:[\x80-\xfe]|[\x40-\xff].)(?u:(.*))"
679  ).unwrap();
680  let text = b"\x12\xd0\x3b\x5f\x7b\xa9\x85\xe2\x98\x83\x80\x98\x54\x76\x68\x65";
681  let caps = re.captures(text).unwrap();
682  
683  // Notice that despite the `.*` at the end, it will only match valid UTF-8
684  // because Unicode mode was enabled with the `u` flag. Without the `u` flag,
685  // the `.*` would match the rest of the bytes.
686  let mat = caps.get(1).unwrap();
687  assert_eq!((7, 10), (mat.start(), mat.end()));
688  
689  // If there was a match, Unicode mode guarantees that `title` is valid UTF-8.
690  let title = str::from_utf8(&caps[1]).unwrap();
691  assert_eq!("☃", title);
692  ```
693  
694  In general, if the Unicode flag is enabled in a capture group and that capture
695  is part of the overall match, then the capture is *guaranteed* to be valid
696  UTF-8.
697  
698  # Syntax
699  
700  The supported syntax is pretty much the same as the syntax for Unicode
701  regular expressions with a few changes that make sense for matching arbitrary
702  bytes:
703  
704  1. The `u` flag can be disabled even when disabling it might cause the regex to
705  match invalid UTF-8. When the `u` flag is disabled, the regex is said to be in
706  "ASCII compatible" mode.
707  2. In ASCII compatible mode, neither Unicode scalar values nor Unicode
708  character classes are allowed.
709  3. In ASCII compatible mode, Perl character classes (`\w`, `\d` and `\s`)
710  revert to their typical ASCII definition. `\w` maps to `[[:word:]]`, `\d` maps
711  to `[[:digit:]]` and `\s` maps to `[[:space:]]`.
712  4. In ASCII compatible mode, word boundaries use the ASCII compatible `\w` to
713  determine whether a byte is a word byte or not.
714  5. Hexadecimal notation can be used to specify arbitrary bytes instead of
715  Unicode codepoints. For example, in ASCII compatible mode, `\xFF` matches the
716  literal byte `\xFF`, while in Unicode mode, `\xFF` is a Unicode codepoint that
717  matches its UTF-8 encoding of `\xC3\xBF`. Similarly for octal notation when
718  enabled.
719  6. In ASCII compatible mode, `.` matches any *byte* except for `\n`. When the
720  `s` flag is additionally enabled, `.` matches any byte.
721  
722  # Performance
723  
724  In general, one should expect performance on `&[u8]` to be roughly similar to
725  performance on `&str`.
726  */
727  #[cfg(feature = "std")]
728  pub mod bytes {
729      pub use crate::re_builder::bytes::*;
730      pub use crate::re_builder::set_bytes::*;
731      pub use crate::re_bytes::*;
732      pub use crate::re_set::bytes::*;
733  }
734  
735  mod backtrack;
736  mod compile;
737  #[cfg(feature = "perf-dfa")]
738  mod dfa;
739  mod error;
740  mod exec;
741  mod expand;
742  mod find_byte;
743  mod input;
744  mod literal;
745  #[cfg(feature = "pattern")]
746  mod pattern;
747  mod pikevm;
748  mod pool;
749  mod prog;
750  mod re_builder;
751  mod re_bytes;
752  mod re_set;
753  mod re_trait;
754  mod re_unicode;
755  mod sparse;
756  mod utf8;
757  
758  /// The `internal` module exists to support suspicious activity, such as
759  /// testing different matching engines and supporting the `regex-debug` CLI
760  /// utility.
761  #[doc(hidden)]
762  #[cfg(feature = "std")]
763  pub mod internal {
764      pub use crate::compile::Compiler;
765      pub use crate::exec::{Exec, ExecBuilder};
766      pub use crate::input::{Char, CharInput, Input, InputAt};
767      pub use crate::literal::LiteralSearcher;
768      pub use crate::prog::{EmptyLook, Inst, InstRanges, Program};
769  }
770