Skip to content

C++ regex parser: align with std::regex ECMAScript grammar and fix literal-location mapping#22227

Draft
jketema with Copilot wants to merge 26 commits into
mainfrom
copilot/port-ruby-regex-to-cpp-again
Draft

C++ regex parser: align with std::regex ECMAScript grammar and fix literal-location mapping#22227
jketema with Copilot wants to merge 26 commits into
mainfrom
copilot/port-ruby-regex-to-cpp-again

Conversation

Copilot AI commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

This series ports the imported Ruby-oriented C++ regex parsing logic to the ECMAScript grammar used by std::regex, while preserving the required commit-by-commit migration shape. It also fixes incorrect term locations for prefixed/raw string literal forms and expands coverage for ECMAScript-specific edge cases.

  • Location mapping correctness for C++ string literal spellings

    • Added regex-location coverage for plain, prefixed, raw, and custom-delimiter literals.
    • Replaced the hardcoded location offset with a content-start offset derived from StringLiteral.getValueText() (handles L/u/U/u8, raw introducer, and raw delimiters).
  • Grammar narrowing: remove Ruby-only constructs

    • Removed support for Ruby-only anchors and escapes: \A \Z \z \G, \h \H.
    • Removed Ruby-only property/group/backref/comment constructs: \p{...} \P{...}, (?#...), (?'name'...), (?<name>...), \k<name>, and lookbehind (?<=...) / (?<!...).
    • Updated corpus and learned expectations to match the narrowed grammar.
  • Grammar completion: add missing ECMAScript forms

    • Implemented control escapes \cX.
    • Implemented NUL escape \0 (only when not followed by a digit).
    • Implemented POSIX collating symbols [[.a.]] and equivalence classes [[=a=]] inside character classes, including delimiter accounting and tree-view exposure.
  • Corpus cleanup and targeted coverage

    • Replaced non-ECMAScript \u{9879} corpus case with valid \u0061.
    • Added coverage for already-supported lazy quantifiers (*?, +?, ??, {2,3}?) and escapes \f, \v.
    • Added [\b] verification; parser treats it as an in-class escaped character (backspace semantics), not RegExpWordBoundary.
  • Change note

    • Added unreleased minorAnalysis note stating the parser now targets std::regex ECMAScript grammar.

Example of the core location fix pattern:

valueText = re.getValueText() and
(
  valueText.regexpMatch("^\\s*(u8|L|u|U)?R\".*") and
  ... // prefix + delimiter aware raw literal offset
  or
  valueText.regexpMatch("^\\s*(u8|L|u|U)?\".*") and
  ... // prefixed normal literal offset
) and
startcolumn = re_start + contentStartOffset + start

Copilot AI and others added 25 commits July 23, 2026 09:58
Note that we are currently still implementing what Ruby thinks a regex is.
This is not correct as C++ by default uses a variant of ECMAScript regexes.
We will address this in the follow-up commits.
Copilot AI changed the title [WIP] Port Ruby regex library to C++ with ECMAScript grammar C++ regex parser: align with std::regex ECMAScript grammar and fix literal-location mapping Jul 23, 2026
Copilot AI requested a review from jketema July 23, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants