Unicode character properties
Since 5.1.0, three additional escape sequences to match generic character types are available when UTF-8 mode is selected. They are:
- \p{xx}
- a character with the xx property
- \P{xx}
- a character without the xx property
- \X
- an extended Unicode sequence
The property names represented by xx above are limited to the Unicode general category properties. Each character has exactly one such property, specified by a two-letter abbreviation. For compatibility with Perl, negation can be specified by including a circumflex between the opening brace and the property name. For example, \p{^Lu} is the same as \P{Lu}.
If only one letter is specified with \p or \P, it includes all the properties that start with that letter. In this case, in the absence of negation, the curly brackets in the escape sequence are optional; these two examples have the same effect:
\p{L} \pL
Property | Matches | Notes |
---|---|---|
C | Other | |
Cc | Control | |
Cf | Format | |
Cn | Unassigned | |
Co | Private use | |
Cs | Surrogate | |
L | Letter | Includes the following properties: Ll, Lm, Lo, Lt and Lu. |
Ll | Lower case letter | |
Lm | Modifier letter | |
Lo | Other letter | |
Lt | Title case letter | |
Lu | Upper case letter | |
M | Mark | |
Mc | Spacing mark | |
Me | Enclosing mark | |
Mn | Non-spacing mark | |
N | Number | |
Nd | Decimal number | |
Nl | Letter number | |
No | Other number | |
P | Punctuation | |
Pc | Connector punctuation | |
Pd | Dash punctuation | |
Pe | Close punctuation | |
Pf | Final punctuation | |
Pi | Initial punctuation | |
Po | Other punctuation | |
Ps | Open punctuation | |
S | Symbol | |
Sc | Currency symbol | |
Sk | Modifier symbol | |
Sm | Mathematical symbol | |
So | Other symbol | |
Z | Separator | |
Zl | Line separator | |
Zp | Paragraph separator | |
Zs | Space separator |
Extended properties such as InMusicalSymbols are not supported by PCRE.
Specifying case-insensitive (caseless) matching does not affect these escape sequences. For example, \p{Lu} always matches only upper case letters.
Sets of Unicode characters are defined as belonging to certain scripts. A character from one of these sets can be matched using a script name. For example:
- \p{Greek}
- \P{Han}
Those that are not part of an identified script are lumped together as Common. The current list of scripts is:
Arabic | Armenian | Avestan | Balinese | Bamum | |
Batak | Bengali | Bopomofo | Brahmi | Braille | |
Buginese | Buhid | Canadian_Aboriginal | Carian | Chakma | |
Cham | Cherokee | Common | Coptic | Cuneiform | |
Cypriot | Cyrillic | Deseret | Devanagari | Egyptian_Hieroglyphs | |
Ethiopic | Georgian | Glagolitic | Gothic | Greek | |
Gujarati | Gurmukhi | Han | Hangul | Hanunoo | |
Hebrew | Hiragana | Imperial_Aramaic | Inherited | Inscriptional_Pahlavi | |
Inscriptional_Parthian | Javanese | Kaithi | Kannada | Katakana | |
Kayah_Li | Kharoshthi | Khmer | Lao | Latin | |
Lepcha | Limbu | Linear_B | Lisu | Lycian | |
Lydian | Malayalam | Mandaic | Meetei_Mayek | Meroitic_Cursive | |
Meroitic_Hieroglyphs | Miao | Mongolian | Myanmar | New_Tai_Lue | |
Nko | Ogham | Old_Italic | Old_Persian | Old_South_Arabian | |
Old_Turkic | Ol_Chiki | Oriya | Osmanya | Phags_Pa | |
Phoenician | Rejang | Runic | Samaritan | Saurashtra | |
Sharada | Shavian | Sinhala | Sora_Sompeng | Sundanese | |
Syloti_Nagri | Syriac | Tagalog | Tagbanwa | Tai_Le | |
Tai_Tham | Tai_Viet | Takri | Tamil | Telugu | |
Thaana | Thai | Tibetan | Tifinagh | Ugaritic | |
Vai | Yi |
The \X escape matches a Unicode extended grapheme cluster. An extended grapheme cluster is one or more Unicode characters that combine to form a single glyph. In effect, this can be thought of as the Unicode equivalent of . as it will match one composed character, regardless of how many individual characters are actually used to render it.
In versions of PCRE older than 8.32 (which corresponds to PHP versions before 5.4.14 when using the bundled PCRE library), \X is equivalent to (?>\PM\pM*). That is, it matches a character without the "mark" property, followed by zero or more characters with the "mark" property, and treats the sequence as an atomic group (see below). Characters with the "mark" property are typically accents that affect the preceding character.
Matching characters by Unicode property is not fast, because PCRE has to search a structure that contains data for over fifteen thousand characters. That is why the traditional escape sequences such as \d and \w do not use Unicode properties in PCRE.
- PHP Руководство
- Функции по категориям
- Индекс функций
- Справочник функций
- Обработка текста
- Функции для работы с регулярными выражениями (Perl-совместимые)
- Регулярные выражения PCRE
- Вступление
- Разделители
- Метасимволы
- Экранирующие последовательности
- Свойства Unicode-символов
- Якоря
- Метасимвол точка
- Символьные классы
- Альтернативный выбор
- Установка внутренних опций
- Подмаски
- Повторение
- Обратные ссылки
- Утверждения
- Однократные подмаски
- Условные подмаски
- Комментарии
- Рекурсивные шаблоны
- Производительность
Коментарии
these properties are usualy only available if PCRE is compiled with "--enable-unicode-properties"
if you want to match any word but want to provide a fallback, you can do something like that:
<?php
if(@preg_match_all('/\p{L}+/u', $str, $arr) {
// fallback goes here
// for example just '/\w+/u' for a less acurate match
}
?>
An excellent article explaining all these properties can be found here: http://www.regular-expressions.info/unicode.html
For those who wonder: 'letter_titlecase' applies to digraphs/trigraphs, where capitalization involves only the first letter.
For example, there are three codepoints for the "LJ" digraph in Unicode:
(*) uppercase "LJ": U+01C7
(*) titlecase "Lj": U+01C8
(*) lowercase "lj": U+01C9
My country, Vietnam, have our own alphabet table:
http://en.wikipedia.org/wiki/Vietnamese_alphabet
I hope PHP will support better than in Vietnamese.
To select UTF-8 mode for the additional escape sequences (\p{xx}, \P{xx}, and \X) , use the "u" modifier (see reference.pcre.pattern.modifiers).
I wondered why a German sharp S (ß) was marked as a control character by \p{Cc} and it took me a while to properly read the first sentence: "Since 5.1.0, three additional escape sequences to match generic character types are available when UTF-8 mode is selected. " :-$ and then to find out how to do so.
Not made clear in the top of page explanation, but these escaped character classes can be included within square brackets to make a broader character class. For example:
<?php preg_match( '/[\p{N}\p{L}]+/', $data ) ?>
Will match any combination of letters and numbers.
Examples are always useful! See https://unicodeplus.com/category for more.
C Other
Cc Control (Unicode code points in the ranges U+0000-U+001F and U+007F-U+009F)
Cf Format (Soft hyphen (U+00AD), zero width space (U+200B), etc.)
Cn Unassigned (Any code point that is not in the Unicode table)
Co Private use
Cs Surrogate (Characters in the range U+D800 to U+DFFF, which are invalid in utf-8)
L Letter
Ll Lower case letter (a-z, µßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ and more)
Lm Modifier letter (Letter-like characters that are usually combined with others, but here they stand alone:
ʰʱʲʳʴʵʶʷʸʹʺʻʼʽʾʿˀˁˆˇˈˉˊˋˌˍˎˏːˑˠˡˢˣˤˬˮʹͺՙ and more)
Lo Other letter (ªºƻǀǁǂǃʔ and many more ideographs and letters from unicase alphabets)
Lt Title case letter (DžLjNjDzᾈᾉᾊᾋᾌᾍᾎᾏᾘᾙᾚᾛᾜᾝᾞᾟᾨᾩᾪᾫᾬᾭᾮᾯᾼῌῼ)
Lu Upper case letter (A-Z, ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞ and more)
L& Ordinary letter (Any character that has the Lu, Ll, or Lt property)
M Mark
Mc Spacing mark (None in latin scripts)
Me Enclosing mark (Combining enclosing square (U+20DE) like in a⃞ , combining enclosing circle backslash (U+20E0) like in a⃠)
Mn Non-spacing mark (Combining diacritical marks U+0300-U+036f, like the accents on this letter a: áâãāa̅ăȧäảåa̋ǎa̍a̎ȁa̐ȃ)
N Number
Nd Decimal number (0123456789, ٠١٢٣٤٥٦٧٨٩ and digits in many other scripts.)
Nl Letter number (ⅠⅡⅢⅣⅤⅥⅦⅧⅨⅩⅪⅫⅬⅭⅮⅯⅰⅱⅲⅳⅴⅵⅶⅷⅸⅹⅺⅻⅼⅽⅾⅿ and some more)
No Other number (⁰¹²³⁴⁵⁶⁷⁸⁹ ₀₁₂₃₄₅₆₇₈₉ ½⅓⅔¼¾⅕⅖⅗⅘⅙⅚⅐⅛⅜⅝⅞⅑⅒ ①②③④⑤⑥⑦⑧⑨⑩⑪⑫⑬⑭⑮⑯⑰⑱⑲⑳, etc.)
P Punctuation
Pc Connector punctuation (_ underscore (U+005F), ‿ undertie U+203F, ⁀ character tie (U+2040), etc.)
Pd Dash punctuation (- hyphen-minus (U+002D), ‐ hyphen (U+2010), ‑ non-breaking hyphen (U+2011), ‒ figure dash (U+2012),
– en dash (U+2013), — em dash (U+2014), ― horizontal bar (U+2015), etc.)
Pe Close punctuation (right parenthesis, bracket, or brace: `)` (U+0029), `]` (U+005D), `}` (U+007D), etc.)
Pf Final punctuation (right quotation marks: » (U+00BB), ’ (U+2019), ” (U+201D), etc.)
Pi Initial punctuation (left quotation marks: « (U+00AB), ‘ (U+2018), “ (U+201C), etc.)
Po Other punctuation (!"#%&'*,./:;?@\¡§¶·¿)
Ps Open punctuation (left parenthesis, bracket, or brace: `(` (U+0028), `[` (U+005B), `{` (U+007B), etc.)
S Symbol
Sc Currency symbol ($¢£¤¥, ₠ ₡ ₢ ₣ ₤ ₥ ₦ ₧ ₨ ₩ ₪ ₫ € ₭ ₮ ₯ ₰ ₱ ₲ ₳ ₴ ₵ ₶ ₷ ₸ ₹ ₺ ₻ ₼ ₽ ₾ ₿ (U+20A0-U+20BF), etc.)
Sk Modifier symbol (Symbol-like characters that are usually combined with others, but here they stand alone:
^`¨¯´¸ and more)
Sm Mathematical symbol (+<=>|~¬±×÷϶ and many more)
So Other symbol (¦ broken bar (U+00A6), © copyright sign (U+00A9), ® registered sign (U+00AE), ° degree sign (U+00B0);
arrows, signs, emojis and many many more)
Z Separator
Zl Line separator (line separator (U+2028))
Zp Paragraph separator (paragraph separator (U+2029))
Zs Space separator (space, no-break space, en quad, em quad, en space, em space, figure space, thin space, hair space, etc.)