Macros 2.0 (v0.3) - Replacing the existing Macro System with a new Macro Engine (#4820)

* Chevrotain lib and env setup

* First draft of the macro lexer

* fix ESLint types loading for chevrotain

* Cleaner lexer modes

* Readme link to Chevrotain & license

* Add jsconfig to tests folder

- Add jsconfig.json to tests folder, to prevent IDE errors on dynamic imports inside the page.evaluate execution.

* Slight improvements on lexer & first tests

* Add more lexer tests

* More edge cases tests

* Reorder tests

* Add macro execution modifiers + more tests

- Added macro flags (execution modifiers) to lexer
- Fixed some lexing issues
- Expanded lexer tests
- Treat lexer errors as failed test

* enable eslint for tests and run it

* Fix lexing unknown flags - treat as error

* Rewrote lexer modes/tokes to capture errors better

* Add lexing for output modifiers

* Clearer names for lexer tokens

* Increase tests default timeout

* Restructure lexer error testcases

* Allow legacy underscores in macro identifiers

* Test case for legacy single-colon syntax

* Improve lexer, removing warnings

* Basic setup for MacroParser + initial tests

* Make parser errors testable

* Add macros stuff to SillyTavern.getContext

* macros test case naming + lint

* Parser consumes basic macros

- Fix lexer mode names
- Add basic macro parsing (identifier, and arguments)
- Tests: basic macro parsing tests
- Tests: simplifyCstNode supports ignoring nodes, or flattening nodes to just plaintext

* Improve macro argument parsing to allow colons in values

Enhances separator handling by fixing separator type detection and enabling colon characters within argument values
Updates validation to require at least one argument component and adds error cases for empty arguments
Includes expanded test coverage for mixed separator scenarios and edge cases

* More nested macro tests

Add error case tests to enforce macro start position requirements
Include nested macro parsing scenarios and invalid syntax checks
Ensures parser correctly handles edge cases with embedded macros

* Unvendor chevrotain

* Add document rule

* Implement visitor, switch built-ins to new type

* Puppeteer -> playwright

* Revert "Implement visitor, switch built-ins to new type"

This reverts commit 706a94b4de62129df6bd6c25e2c6dec692d12226.

* Converted puppeteeer tests

* File rename

* chore: reduce Playwright worker count to 4 for performance/stability

* test: add comprehensive legacy macro parser test suite

- Added 13 test cases covering legacy macro formats (roll, reverse, comment, datetime, time_UTC, banned, setvar)
- Documented parser limitations with TODO comments for whitespace separators, special characters, and empty arguments
- Tests validate parsing of various separator styles (space, colon, +/-) and argument formats (quoted, numeric, empty)

* fix: handle legacy macro syntax with colon or whitespace separator

- Modified arguments rule to support both double-colon (::) and single-colon (:) separators
- Made single-colon separator optional to allow whitespace-separated legacy macros
- Removed TODO comments as parser now correctly handles legacy macro formats

* feat: support space-separated quoted arguments in macro parser

- Added parsing support for equals signs and quotes as valid argument tokens
- Removed TODO comments for legacy macro parsing with quoted arguments

* fix: improves macro argument parsing with colon handling

Enhances parser to correctly handle double colons within legacy single-colon arguments
Introduces separate parsing rules for arguments with different colon constraints
Adds test coverage for arguments containing double colons in legacy format

* fix: allow empty macro arguments after double-colon separator

- Changed argument rule from AT_LEAST_ONE to MANY to permit zero-length arguments
- Updated tests to verify empty argument parsing (e.g., `{{something::}}`)
- Enhanced simplifyCstNode helper with default flatten/ignore keys and improved null handling

* refactor: improve test documentation with inline macro display

- Added inline comments showing the actual macro syntax being tested for better readability
- Removed duplicate comment in error test case
- Cleaned up extra whitespace in legacy macro tests

* feat: add legacy macro preprocessing for time offset format

- Implemented preProcessFixLegacyMacros method to convert {{time_UTC±N}} to {{time::UTC±N}} format
- Updated tests to use new preprocessing step for legacy time macro parsing
- Added runPreProcessFix option to test helper functions for controlled legacy macro handling

* feat: add support for comment macros with double-slash syntax

- Added `DoubleSlash` token to lexer to recognize `//` as a valid macro identifier
- Updated parser to accept either `//` or standard identifiers as macro names
- Enhanced test suite with comprehensive comment macro test cases including multiline support

* feat: implement macro evaluation engine with CST walking and registry integration

(I'm tired, let's just throw this in right now)

- Added CST walker and macro registry to engine initialization
- Enhanced parseDocument to handle empty input, legacy macro preprocessing, and error collection
- Implemented async evaluate method with full macro resolution pipeline
- Added resolveMacro callback to handle unknown macros and registry execution
- Integrated lexing/parsing error handling with console warnings
- Added support for preserving unknown macro

* refactor: improve type safety and code clarity in macro evaluation system

- Simplified typedef imports to use correct Chevrotain types (CstNode, IToken)
- Added TokenRange typedef for consistent offset handling
- Enhanced error messages with context-specific prefixes
- Replaced verbose type casts with inline JSDoc annotations
- Condensed singleton pattern declarations to single lines
- Improved null safety with optional chaining and nullish coalescing
- Extracted resolveMacro logic into private root function

* fix test macro whitespace arguments onls accepting one argument

* Fix OpenRouter embeddings URL
#4736

* Fix: Prevent data loss on bulk regex move to scoped scripts (#4760)

* Fix: Prevent data loss in regex bulk move

* prevents moving to scoped scripts with group selected for bulkedit

* Refactor: make whitelist validation a bit more robust (#4757)

* refactor: extract IP whitelist validation into helper function

- Added isIPInWhitelist helper with error handling for individual whitelist entry checks
- Replaced inline whitelist matching logic with reusable function calls
- Added JSDoc type annotations and error logging for failed IP matching attempts

* refactor: simplify whitelist validation with upfront filtering

- Moved IP validation to startup time instead of per-request checking
- Extracted validateWhitelist function to filter invalid entries once at initialization
- Simplified isIPInWhitelist by removing redundant error handling after validation

* fix: correct IP whitelist matching to use parsed CIDR notation

* Feat: Improve multiline input handling in popups (#4756)

* feat: improve multiline input handling in popups

- Added Ctrl+Enter requirement for submission in multiline input popups to prevent accidental sends
- Exported PopupUtils class for external use

* refactor: remove redundant higher/different rows from input popups

- Removed rows: 2 from callGenericPopup calls where default behavior is sufficient
- Increased rows from 2 to 4 in caption extension for better multiline input experience

* Fix npm audit

* feat: add max-height and scrolling to world entry key input fields (#4769)

- Added 160px max-height to select2 multiple selection, primary key, and secondary key text areas in world entries
- Enabled vertical scrolling with hidden horizontal overflow for better UX with long content

* Bump anti-troll tags limit
Closes #4763

* OpenAI: Add Sora 2 API (#4748)

* OpenAI: Add Sora 2 API

* Add duration control

* Support client generation abort

* Reduce poll log amount

* Simplify selector

* Simplify model-specific control handling

* Gemini: Pass non-success response content to frontend

* Vertex: Add Vertex AI-specific safety setting (#4770)

Closes #4455

* Feature: allows sorting tags by most used (#4768)

* add sorting tags by most used

* Fix whitespaces

* Code review updates

* Remove commented code

* Fix capitalization in comment

* Apply review suggestion

* Simplify template init

* Reformat

* Add documentation for appendViewTagToList and printViewTagList functions

* Reprint renamed tags regardless of sorting mode

---------

Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>

* Move img.swipe notice block up

* Preserve media playback state when running appendMediaToMessage (#4771)

* Preserve media playback state when running appendMediaToMessage

* Better selector specificity

* Fix local variable name

* Move typedef to global.d.ts

* Check for readyState on save/restore

* Only check for currentSrc on restoration callback

* gpt-5.1

* A 500 billion dollar startup can’t filter API payload fields

* Refactor macro argument validation to use requiredArgs and list pattern

- Replace minArgs/maxArgs/enforceArity with requiredArgs and list specification
- Add MacroListSpec typedef for flexible list argument constraints (min/max)
- Rename enforceArity to strictArgs with inverted default behavior
- Update validation logic to separately track required vs. list arguments
- Add requiredArgs and list arrays to MacroExecutionContext for easier access
- Improve error messages to clearly distinguish between required and list

* Refactor MacroRegistry.registerMacro to use options object pattern

- Move handler function into options object as required property
- Improve validation with detailed error messages for all option fields
- Add explicit type checking for requiredArgs, list, strictArgs, and description
- Consolidate name normalization and validation logic
- Simplify list option parsing with clearer conditional structure
- Update all builtin macro registrations to use new signature

* Add e2e tests for macro arity validation errors

- Test ping macro rejects calls with unexpected arguments
- Test upper macro rejects calls without required arguments
- Verify macros remain unresolved and log warnings on arity violations

* re-implement core macros as registered macros

- Create shallow frozen copy of env object before passing to macro handlers
- Add comprehensive MacroEnv typedef with nested types for names, character, system, and extra fields
- Update MacroDefinitionOptions typedef to clarify default values and mark handler as required
- Export variable manipulation functions for external use
- Fix whitespace in ifCallback JSDoc comment

* Defer slash command autocomplete initialization to firstLoadInit

- Move slash command autocomplete setup from module-level to initSlashCommandAutoComplete function
- Add null check for sendTextarea.value before accessing first character
- Import and call initSlashCommandAutoComplete in firstLoadInit sequence
- Export registerCoreMacros, registerInstructMacros, and registerVariableMacros using named export syntax
- Call registerCoreMacros from initMacros with

* Add variable manipulation functions to SillyTavern context API

- Import deleteGlobalVariable, deleteLocalVariable, addGlobalVariable, addLocalVariable, incrementGlobalVariable, incrementLocalVariable, decrementGlobalVariable, and decrementLocalVariable from variables.js
- Expose del, add, inc, and dec methods on both context.variables.local and context.variables.global objects
- Provide consistent API for variable deletion, addition, increment, and decrement operations

* Extract test setup utilities and fix trailing comma in core-macros.js

- Add testSetup utility object with goST and awaitST helper functions for Playwright tests
- Replace duplicated beforeEach setup code in MacroLexer, MacroParser, and MacroRegistry tests with testSetup.goST
- Add explanatory comments indicating tests currently run without ST context
- Fix missing trailing comma in input macro handler registration

* Refactor MacroEngine e2e tests to use real core macros instead of prototypes

- Replace prototype macros (ping, echo, upper, wrap, first) with actual core macros (newline, reverse, setvar, getvar, addvar, roll)
- Use testSetup.awaitST helper from utils.js for beforeEach setup
- Increase test timeout from 10s to 20s due to additional setup requirements
- Remove registerPrototypeMacros import and registration logic from evaluateWithEngine
- Add page.waitForTimeout(1000) to ensure macros are fully initialize
- Remove now obsolete MacroBuiltins.js

* Fix audit in tests

* Rename frontend test utils file to frontent-test-utils.js and update imports

- Rename tests/frontend/utils.js to tests/frontend/frontent-test-utils.js
- Update testSetup imports across MacroEngine, MacroLexer, MacroParser, and MacroRegistry e2e tests

* Refactor MacroRegistry for improved readability and consistency

- Reorder MacroDefinition typedef fields to match registration order (handler moved to end)
- Add inline JSDoc comments to MacroDefinitionOptions properties for clarity
- Simplify name validation logic in unregisterMacro, hasMacro, and getMacro methods using early returns
- Extract argument validation logic into dedicated isArgsValid helper function
- Refactor executeMacro to be async and use Promise.resolve for consistent promise handling

* Add returns field to MacroDefinitionOptions for documenting macro return values

- Add returns property to MacroDefinitionOptions typedef with string type and null default
- Include returns field in MacroDefinition typedef
- Store returns value in macro definition object for documentation purposes

* Add error handling to macro execution and refactor core macro handlers

- Add catch handler in MacroRegistry.executeMacro to log errors and return empty string on handler failures
- Update macro descriptions to use "index" instead of "ID" for message position macros
- Simplify reverse macro description and remove redundant null coalescing
- Add strictArgs: false to comment macro to ensure it's always removed
- Enhance time macro description with UTC offset examples
- Remove unnecessary null check

* Refactor instruct-macros.js for consistency and reduce code duplication

- Rename enabled helper to instEnabled and add sysEnabled helper for clarity
- Consolidate defaultSystemPrompt, instructSystem, and instructSystemPrompt into single registerSimple call
- Simplify systemPrompt handler by removing unnecessary null coalescing and intermediate variables
- Convert chatSeparator and chatStart macros to use registerSimple helper
- Fix JSDoc comment formatting for registerSimple helper function

* Add normalize helper to MacroExecutionContext and refactor variable macros for consistency

- Add normalizeMacroResult method to MacroEngine for converting macro results to strings
- Include normalize function in MacroExecutionContext for handler use
- Update normalizeMacroResult to handle arrays explicitly alongside objects
- Add returns field to variable macros that produce side-effects only
- Refactor variable macro handlers to use destructuring and normalize helper
- Remove redundant null checks an

* chore: Remove duplicate getglobalvar macro registration from variable-macros.js

- Remove redundant getglobalvar macro definition (already registered earlier in the file)
- Import MacroEngine in MacroRegistry for normalizeMacroResult access
- Add fallback binding for normalize in MacroExecutionContext when not provided by caller
- Update executeMacro to use executionContext.normalize instead of standalone normalizeMacroResult
- Remove normalizeMacroResult helper function from MacroRegistry (now handled by MacroEngine)

* Add core environment macros for names, character fields, system info, and deterministic pick

- Add lastGenerationType tracking with event listeners for GENERATION_STARTED and CHAT_CHANGED
- Add ensureLastGenerationTypeTracking helper to initialize event listeners once
- Register name macros: user, char, group, groupNotMuted, notChar, charIfNotGroup
- Register character card field macros: charPrompt, charInstruction, description, personality, scenario, persona, mesExamplesRaw, charDepthPrompt, cre

* Refactor core macros into separate modules by category (env, state, chat, time)

- Move name and character card field macros to new env-macros.js module
- Move system/device/runtime state macros to new state-macros.js module
- Move chat inspection macros (lastMessage, lastMessageId, etc.) to new chat-macros.js module
- Move time/date macros (time, date, weekday, isotime, etc.) to new time-macros.js module
- Remove lastGenerationType tracking logic and helper functions from core-macros.js (moved to state

* Reorganize macro system into engine and definitions directories

- Move macro engine components (MacroEngine, MacroRegistry, MacroLexer, MacroParser, MacroCstWalker) to macros/engine/ subdirectory
- Move macro definition modules (core-macros, env-macros, state-macros, chat-macros, time-macros, variable-macros, instruct-macros) to macros/definitions/ subdirectory
- Create macro-system.js as central entry point that exports engine singletons and initRegisterMacros function
- Refactor variable-macros.js to use S

* Export getGeneratingModel function and add MacroEnvBuilder to macro system exports

- Export getGeneratingModel function from script.js for external use
- Import MacroEnvBuilder in macro-system.js
- Add envBuilder singleton to macros export object alongside existing engine components

* Extract MacroEnv typedefs into separate MacroEnv.types.js file

- Create MacroEnv.types.js with MacroEnv, MacroEnvNames, MacroEnvCharacter, and MacroEnvSystem typedefs
- Remove MacroEnv typedef definitions from MacroRegistry.js
- Update MacroRegistry.js, MacroEngine.js, MacroEnvBuilder.js, and env-macros.js to import MacroEnv from MacroEnv.types.js
- Change MacroEngine.evaluate env parameter type from any to optional MacroEnv

* Add comprehensive e2e tests for MacroEnvBuilder

- Create MacroEnvBuilder.e2e.js with 13 test cases validating environment construction
- Test name override precedence (overrides vs global fallback)
- Test character field population based on replaceCharacterCard flag
- Test original value one-shot helper function behavior
- Test group override string propagation to group/group

* Add substituteParamsAsync function with experimental macro engine support

- Create substituteParamsAsync function in script.js as async alternative to substituteParams
- Use object destructuring pattern for function parameters following RO-RO convention
- Add experimental_macro_engine flag to power_user settings for feature gating
- Add MacroEnvFunctions typedef with original and postProcess function types to MacroEnv.types.js
- Add functions property to MacroEnv typedef for one-shot helpers an

* Add dynamic macro support to MacroEngine with environment-based override and postProcess execution

- Add defOverride parameter to MacroRegistry.executeMacro for temporary macro definitions
- Check env.dynamicMacros in MacroEngine#resolveMacro and create temporary macro definition when found
- Set strictArgs to true for dynamic macros to fail if called with arguments
- Execute env.functions.postProcess on macro results in MacroEngine#resolveMacro with error handling
- Update MacroEnv typedef to make

* Refactor macro handlers to assume non-null env and remove optional chaining

- Remove optional chaining (?.) from env property access in env-macros.js and instruct-macros.js
- Change MacroExecutionContext env property from optional to required in MacroRegistry.js
- Update executeMacro context parameter from optional to required
- Simplify original macro handler to call env.functions.original() directly without try-catch
- Update MacroHandler typedef to use arrow function syntax for consistency
- Remove redundant null checks an

* Refactor MacroRegistry.executeMacro to accept MacroCall object and make MacroExecutionContext properties non-optional

- Change executeMacro to accept MacroCall object instead of separate name and context parameters
- Move normalize function from context parameter to options object in executeMacro
- Construct MacroExecutionContext from MacroCall properties within executeMacro
- Set namedArgs to null in executionContext (currently unused)
- Update MacroEngine.resolveMacro to pass MacroCall directly

* Remove normalize parameter from MacroRegistry.executeMacro and bind normalizeMacroResult directly in execution context

- Remove normalize parameter from executeMacro options object in MacroRegistry.js
- Bind MacroEngine.normalizeMacroResult directly in executionContext instead of accepting override
- Remove normalize option from MacroEngine.resolveMacro call to executeMacro
- Add missing name fields (group, groupNotMuted, notChar) to MacroEnvBuilder default env object

* Make MacroEnvRawContext properties optional and change original function from required to optional in MacroEnvFunctions

- Change MacroEnvRawContext properties from required to optional with null defaults
- Change original function from required to optional in MacroEnvFunctions typedef
- Remove original function from default env object in MacroEnvBuilder (only include when provided)

* Refactor MacroEnvBuilder tests to use optional properties and add macro arity validation tests

- Remove explicit undefined/false assignments from MacroEnvRawContext test objects (now optional with defaults)
- Rename additionalMacro to dynamicMacros in MacroEnvBuilder tests and update property access from env.extra to env.dynamicMacros
- Move original function from env.extra to env.functions in test assertions
- Remove test for additionalMacro overriding original helper (no longer applicable with

* Improve macro error handling with dedicated diagnostics and runtime error propagation

- Import and use logMacroInternalError and logMacroRuntimeWarning from MacroDiagnostics in MacroEngine
- Wrap MacroRegistry.executeMacro call in try-catch to distinguish runtime vs internal errors
- Nest postProcess execution in inner try-catch with dedicated error logging
- Return raw macro syntax on execution failure instead of empty string
- Replace console.warn with logMacroRuntimeWarning for argument count

* Add macro argument type validation with positional argument definitions and runtime type checking

- Add MacroArgType and MacroPositionalArgDef typedefs for argument metadata
- Change requiredArgs option to accept number or MacroPositionalArgDef[] array
- Add requiredArgDefs property to MacroDefinition to store normalized argument definitions
- Validate requiredArgs array elements during macro registration (name, description, type fields)
- Generate default argument definitions when requiredArgs is a

* Add e2e tests for macro type validation and dynamic macro strict arity enforcement

- Add test verifying strict typed macros fail resolution when argument type is invalid
- Add test verifying non-strict typed macros execute with invalid types but log warnings
- Add test verifying dynamic macros reject arguments due to strictArgs enforcement
- Capture and assert runtime warning messages for type validation and arity violations
- Register test macros with integer type requirements and varying strict

* Add mesExamples macro with instruct mode formatting support

- Register mesExamples macro in env-macros.js to format dialogue examples
- Import parseMesExamples, main_api, power_user, and formatInstructModeExamples
- Check instruct mode enabled state and main_api to determine formatting path
- Parse raw examples using parseMesExamples with instruct mode flag
- Return empty string when raw examples are missing or parsed result is empty
- Format examples using formatInstructModeExamples when instruct mode is active

* Add URL navigation wait in test setup and increase Playwright worker count to 4

- Add waitForURL check after user selection in awaitST to ensure navigation completes before preloader check
- Increase Playwright workers from 1 to 4 for parallel test execution

* Add e2e tests for multi-line macro arguments and comment macro functionality

- Add test verifying reverse macro handles multi-line arguments with newline characters
- Add test verifying comment macro removes single-line comments with simple body
- Add test verifying comment macro accepts non-word characters immediately after //
- Add test verifying comment macro ignores additional // sequences inside comment body
- Add test verifying comment macro supports multi-line comment bodies

* Standardize JSDoc type annotations to use explicit null defaults and union types instead of nullable shorthand

* Remove individual test timeout configurations from frontend macro test files

* Add positional argument definitions with sample values and descriptions to core, env, and time macros

- Add sampleValue field to MacroPositionalArgDef typedef (optional string)
- Replace numeric requiredArgs with positional argument definition arrays in roll, banned, outlet, datetimeformat, and timeDiff macros
- Include name, sampleValue, description, and type fields for each positional argument
- Update timeDiff description to clarify absolute difference calculation
- Remove unnecessary blank lines in

* Remove async/await from macro engine evaluation and convert all macro handlers to synchronous execution

Makes me sad, but such is life

* Refactor substituteParams to use options object signature with backward compatibility for legacy positional arguments

- Rename original substituteParams to substituteParamsLegacy with unchanged positional argument signature
- Rename substituteParamsNew to substituteParams as the new primary function
- Add automatic detection and routing of legacy positional argument calls to substituteParamsLegacy
- Update substituteParamsExtended to use new options object signature and mark as deprecated

* forgor

* fix missing import, and package-lock, finally. Maybe.

* Add experimental macro engine toggle to UI settings

- Add experimental_macro_engine setting to default settings.json
- Move experimental_macro_engine property to correct position in power_user object (with other experimental settings)
- Add checkbox UI control in settings panel with flask icon and tooltip explaining nested macro resolution and logical replacement order
- Wire up checkbox event handler to save experimental_macro_engine setting
- Load experimental_macro_engine state on settings initialization

* Refactor macro pre/post-processing from parser to engine and improve error handling

- Move legacy macro pre-processing (time_UTC format) from MacroParser to MacroEngine #runPreProcessors
- Move trim macro post-processing to MacroEngine #runPostProcessors to handle cross-boundary behavior
- Remove MacroLexer import from MacroEngine (now handled by MacroParser.parseDocument)
- Update MacroParser.parseDocument to return separate lexingErrors and parserErrors arrays

* Add {{trim}} macro placeholder that defers to post-processing for cross-boundary whitespace handling

* Add eslint-plugin-playwright

* Add logMacroSyntaxWarning function for structured lexer/parser error reporting with compact human-readable payload

* Small code review fixes

- import event types from events.js
- Add chained fallback to {{input}}, just to be safe
- switched variables.js to export-per-method
- minor text adjustments in registered macro docs

* fix lexer not capturing linebreaks correctly & simplify plaintext token

- Replace alternation-based pattern with negated character class approach
- Use `(?:[^{]|\{(?!\{))+` to match non-brace chars or single braces not followed by another brace
- Add unicode flag for consistency
- Update comment to clarify intent: consume anything that is not the start of a macro '{{'

* Skip macro processing on char/group fields for env build (performance)

- Add returnRaw parameter to getCharacterCardFields to optionally return raw values without baseChatReplace
- Add returnRaw parameter to getGroupCharacterCards for consistent raw value handling
- Replace direct baseChatReplace calls with conditional transform function based on returnRaw flag
- Apply collapseNewlines when returnRaw is true and collapse_newlines setting is enabled
- Update MacroEnvBuilder to use the returnRaw as true

* Fix lexer failing to handle literal '{' before macro openers

- Add PlaintextOpenBrace token to lexer with pattern `/\{(?=\{\{)/` to match single '{' immediately before '{{'
- Update lexer mode definition to consume PlaintextOpenBrace before attempting macro start
- Update parser document rule to handle PlaintextOpenBrace as alternative to Plaintext
- Update MacroCstWalker to collect both Plaintext and Plaintext.OpenBrace tokens when building document items

* Add legacy non-curly marker pre-processing (<USER>, <BOT>, <CHAR>, <GROUP>, <CHARIFNOTGROUP>)

- Add pre-processing step in MacroEngine to rewrite legacy angle-bracket markers into their curly-brace macro equivalents
- Map <USER> → {{user}}, <BOT> → {{char}}, <CHAR> → {{char}}, <GROUP> → {{group}}, <CHARIFNOTGROUP> → {{charIfNotGroup}}
- Add e2e tests verifying legacy marker resolution through the engine pipeline
- Tests cover <USER>, <BOT>/<CHAR>, and <GROUP>/<CHARIFNOTGROUP> markers

* Add env.content to MacroEnv and use it for deterministic {{pick}} hashing

- Add content property to MacroEnv type definition and MacroEnvBuilder to expose the full original input string
- Update {{pick}} macro handler to use env.content hash instead of rawListString hash for seed generation
- Ensures deterministic behavior when the same prompt position contains different list items across evaluations
- Prevents {{pick}} from returning different values when nested macros resolve to different intermediate

* add e2e tests for env.content exposure and deterministic {{pick}} behavior

- Rename getChatIdHashCore → getChatIdHash in core-macros.js for consistency
- Add e2e test verifying env.content is exposed to macro handlers
- Add e2e test confirming {{pick}} returns stable results for same chat and content
- Tests verify deterministic behavior by comparing multiple evaluations with fixed chat_id_hash

* Rename chat macro helper functions to remove 'Core' suffix for consistency

* Add MacrosParser deprecation warnings and bridge to new macro engine when experimental flag enabled

- Import macroSystem and power_user for experimental macro engine integration
- Add @deprecated JSDoc tag to MacrosParser class
- Add #logDeprecated helper to warn about deprecated MacrosParser methods
- Add #registerMacroInNewEngine to bridge legacy macro registrations into new engine
- Add #unregisterMacroInNewEngine to bridge legacy macro unregistrations
- Log deprecation warnings in get, has, register

* Fix lint in tests

* Add comprehensive bracket handling tests and improve macro lexer/parser resilience to invalid syntax

- Add logMacroGeneralError function for non-macro-specific error logging
- Add CST validation check in MacroEngine to return original input if parser produces invalid CST
- Wrap MacroCstWalker.evaluateDocument in try-catch to gracefully handle evaluation failures
- Update MacroLexer Unknown token pattern to capture single closing braces not followed by another closing brace
- Add fallback mode exit

* Add error recovery for incomplete macros by flattening them to plaintext while preserving nested complete macros

- Enable Chevrotain error recovery in MacroParser constructor
- Add #isRecoveryToken helper to detect tokens inserted during error recovery
- Add #flattenIncompleteMacro to recursively convert incomplete macro nodes into plaintext items
- Update #collectDocumentItems to detect recovery-inserted Macro.End tokens and flatten incomplete macros
- Simplify plaintext token collection to use

* Add e2e test verifying nested macros resolve even when outer macro has invalid argument count

* Remove unused macro CST node caching mechanism from MacroCstWalker

* Add JSDoc type re-exports and register shorthand to macro-system.js for improved DX

- Re-export commonly used JSDoc types from MacroRegistry and MacroEnv modules for easier consumption by external code
- Add macros.register shorthand function bound to MacroRegistry.registerMacro for convenient macro registration
- Includes MacroDefinitionOptions, MacroHandler, MacroEnv, and related type definitions

* Migrate legacy macro registrations to new macro system API across multiple modules

- Replace MacrosParser.registerMacro calls with macros.register using object-based configuration
- Update imports from './macros.js' to './macros/macro-system.js'
- Extract macro registration into dedicated registerAuthorsNoteMacros function in authors-note.js
- Add descriptions to all macro registrations for better documentation
- Update MacrosParser iterator to yield from new registry when experimental engine is enabled

* Add MacroCategory to all macro registrations across codebase for improved organization and discoverability

- Import MacroCategory from macro-system.js in authors-note.js, memory/index.js, stable-diffusion/index.js, and macros.js
- Add category property to all macro registrations using appropriate MacroCategory values
- Assign 'legacy' category to MacrosParser auto-registered macros
- Categorize macros across chat, character, prompts, utility, random, state, time, and names categories

* fix lint

* Add MacroBrowser UI component and integrate macro help system into chat interface

- Add implementation of MacroBrowser
- Add macros.css stylesheet link to index.html
- Update core-macros.js ban macro to return empty string instead of 'Empty string' description
- Add sampleValue property to positional argument definitions in MacroRegistry
- Generate default sampleValue for numeric requiredArgs using 'arg{n}' pattern
- Set default description to '<no description>' and returns to '<empty string>' when empty

* Reorder JSDoc typedef declarations in MacroRegistry.js to have the most relevant first

* fix missing category in legacy initMacros

* Add displayOverride and exampleUsage properties to macro registration system

- Add displayOverride property to MacroDefinitionOptions for custom signature display
- Add exampleUsage property to MacroDefinitionOptions for documentation examples
- Update MacroRegistry.registerMacro to validate and process displayOverride and exampleUsage
- Add logMacroRegisterWarning function to MacroDiagnostics for registration-time warnings
- Auto-wrap displayOverride and exampleUsage in curly braces if missing

* Add error handling to MacroRegistry.registerMacro and logMacroRegisterError diagnostic function

- Add logMacroRegisterError function to MacroDiagnostics for registration failures
- Wrap MacroRegistry.registerMacro body in try-catch to handle registration errors gracefully
- Change registerMacro return type from MacroDefinition to MacroDefinition|null
- Log registration errors and return null instead of throwing, preventing macro registration failures from breaking the application

* Optimize character card field access with lazy evaluation to improve macro execution performance

* Add brace unescaping to MacroEngine post-processing to support literal curly braces in macro output

- Add regex replacement to unescape \{ and \} to { and } after macro execution
- Allows users to output literal braces by escaping them with backslashes
- Escaped sequences like \{\{ don't match MacroStart pattern and pass through as plain text

* Add alias system to macro registry with UI support for displaying and navigating macro aliases

- Add `aliases` property to MacroDefinitionOptions for defining alternative macro names
- Add `aliasOf` and `aliasVisible` properties to MacroDefinition to track alias relationships
- Update MacroRegistry.registerMacro to create alias entries pointing to primary definitions
- Add `getPrimaryMacro` method to retrieve primary definition from alias names
- Add filtering options to `getAllMacros` to exclude aliases

* Strip curly braces from MacroBrowser search query to match macro name format in search definitions

* Add enhanced macro autocomplete with argument hints and context-aware suggestions

- Add comprehensive CSS styling for enhanced macro autocomplete items with flex layout
- Add argument hint banner styling with gradient background and border
- Add current argument highlighting in details panel
- Export formatMacroSignature, createSourceIndicator, createAliasIndicator, and createTypeBadge from MacroBrowser for reuse

* Fix macro autocomplete not showing details/arguments because of trailing colons

* Change macro details CSS selectors from `.macroBrowser` to `.macro-details` for better reusability and update autocomplete to use macro enum icon

- Replace all `.macroBrowser` selectors with `.macro-details` in macros.css to allow macro details panel styling to work outside MacroBrowser context
- Change autocomplete option icon from hardcoded '{}' to `enumIcons.macro` for consistency
- Remove redundant `showCategory: false` option from renderMacroDetails call in autocomplete (now handled by default

* Add MacroArgType enum to replace string literal type union for macro argument types

- Add MacroArgType enum with STRING, INTEGER, NUMBER, and BOOLEAN values
- Replace MacroArgType typedef string literal union with enum reference
- Export MacroArgType from macro-system.js alongside MacroCategory
- Remove MacroArgType typedef re-export (now an enum, not a type)

* Add support for multiple argument types in macro definitions

- Update createTypeBadge to handle both single type and array of types, displaying as "type1 | type2" with tooltip
- Add JSDoc comments to MacroArgType enum values explaining each type
- Update MacroArgDefinition typedef to allow type property to be single MacroArgType or array
- Update MacroRegistry.registerMacro to validate array of types and default empty arrays to 'string'
- Update validateArgTypes to check if argument value matches

* Add returnType property to macro definitions with automatic type badge display in macro details panel

- Rename MacroArgType enum to MacroValueType to reflect dual use for arguments and return types
- Add returnType property to MacroDefinitionOptions (defaults to MacroValueType.STRING)
- Add returnType validation in MacroRegistry.registerMacro to ensure valid type values
- Update renderMacroDetails to always show Returns section with type badge
- Add macro-returns-content CSS class with flex layout for type

* Add detailed argument definitions, return types, and example usage to all variable macros

- Replace numeric requiredArgs with detailed argument definition objects including name, type, and description
- Add returnType property to all macros that return values (inc/dec/get variants)
- Add returns property descriptions to all macros
- Add exampleUsage arrays demonstrating typical usage patterns for each macro
- Apply changes consistently

* Add returns descriptions and return types to core macro definitions with improved documentation

* Add returns descriptions, display overrides, and example usage to time macros with improved documentation

* fix lint

* Add returns descriptions and return types to chat, environment, instruct, and state macros with improved documentation

* Add missing properties to dynamic macro definition override to match MacroDefinitionOptions structure

* Replace console logging with MacroDiagnostics logging in MacroEnvBuilder and MacroRegistry

* Add support for array-based argument types in macro autocomplete with union type display and tooltip

* Rename requiredArgs to unnamedArgs and add support for optional unnamed arguments with bracket notation in macro signatures and hints

* Add optional offset argument definition to time macro with type, sample value, and description

* Simplify example usage for random and pick macros by removing surrounding context text

* Add default value display for optional macro arguments in autocomplete and browser documentation

* Fix macro args defaultValue not being converted into the normalized values for register and display

* Allow STscript macro auto completion to still show up when typing closing braces

* Add space macro with optional count argument for inserting multiple spaces

* Add optional count argument to newline macro for inserting multiple newlines with default value of 1

* Remove MutationObserver and CTRL+F keyboard event handling from MacroBrowser

* fix lint

* Register `{{summary}}` macro for both old and new macro engines based on experimental flag

* Register `{{charPrefix}}` and `{{charNegativePrefix}}` macros for both old and new macro engines based on experimental flag

* Update hidden alias badge text to indicate deprecation status

* Simplify macro name validation and error handling in MacroRegistry

Remove redundant `macroName` variable by normalizing `name` parameter early and reusing it throughout the registration flow. Consolidate trim checks in validation condition.

* Remove exp macro engine flag checks from auto complete and help, and remove legacy macro template

Move macro registration to always use new engine regardless of experimental flag. Remove conditional logic for `experimental_macro_engine` in MacroBrowser, system messages, and slash command parser. Delete legacy `macros.html` template and associated static macro help generation. Always use MacroBrowser for macro documentation display.

* Rename macros to camelCase and add backward-compatible aliases

Rename `description`, `personality`, `scenario`, `creatorNotes` to `charDescription`, `charPersonality`, `charScenario`, `charCreatorNotes` respectively. Rename `idle_duration` to `idleDuration`. Add old names as aliases for backward compatibility. Add `comment` as visible alias for `//` macro. Mark `idle_duration` alias as hidden.

* fix `random` and `pick` macros by using list parameter directly instead of raw string

Remove `raw` parameter from `random` and `pick` macro handlers. Simplify legacy comma-separated list handling by using `list[0]` directly instead of `rawListString`. Rename `items` variable to `list` in `pick` macro for consistency.

* fix `random` and `pick` not handling all colon-separated lists as before

Extract `readSingleArgsRandomList` helper function to handle legacy comma-separated and double-colon list parsing. Reuse this helper in both `random` and `pick` macro handlers to eliminate duplicated list parsing logic.

---------

Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>
Co-authored-by: bmen25124 <bmen25124@gmail.com>
This commit is contained in:
Wolfsblvt
2025-12-15 20:29:46 +01:00
committed by GitHub
parent 4f9aeac7ac
commit 6f8b6b098e
45 changed files with 8505 additions and 275 deletions
+1
View File
@@ -79,6 +79,7 @@ GNU Affero General Public License for more details.**
* Portions of CncAnon's TavernAITurbo mod used with permission
* Visual Novel Mode inspired by the work of PepperTaco (<https://github.com/peppertaco/Tavern/>)
* Noto Sans font by Google (OFL license)
* Lexer/Parser by Chevrotain (Apache-2.0 license) <https://github.com/chevrotain/chevrotain>
* Icon theme by Font Awesome <https://fontawesome.com> (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License)
* Default content by @OtisAlejandro (Seraphina character and lorebook) and @kallmeflocc (10K Discord Users Celebratory Background)
* Docker guide by [@mrguymiah](https://github.com/mrguymiah) and [@Bronya-Rand](https://github.com/Bronya-Rand)
+1
View File
@@ -191,6 +191,7 @@
"custom_stopping_strings_macro": true,
"fuzzy_search": true,
"encode_tags": false,
"experimental_macro_engine": false,
"enableLabMode": false,
"enableZenSliders": false,
"ui_mode": 1,
+61
View File
@@ -45,6 +45,7 @@
"bowser": "^2.12.1",
"bytes": "^3.1.2",
"chalk": "^5.6.0",
"chevrotain": "^11.0.3",
"command-exists": "^1.2.9",
"compression": "^1.8.1",
"cookie-parser": "^1.4.6",
@@ -105,6 +106,7 @@
"sillytavern": "src/server-global.js"
},
"devDependencies": {
"@chevrotain/types": "^11.0.3",
"@types/archiver": "^6.0.3",
"@types/bytes": "^3.1.5",
"@types/command-exists": "^1.2.3",
@@ -171,6 +173,45 @@
"integrity": "sha512-KlmTftToTtmb6aLVdne4NluS+POWputPF5J8v25UN/EQS+K9vahWEIe1NPRSFqBQclObkqHaj7JOnFrmnSm5MA==",
"license": "Apache-2.0"
},
"node_modules/@chevrotain/cst-dts-gen": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/@chevrotain/cst-dts-gen/-/cst-dts-gen-11.0.3.tgz",
"integrity": "sha512-BvIKpRLeS/8UbfxXxgC33xOumsacaeCKAjAeLyOn7Pcp95HiRbrpl14S+9vaZLolnbssPIUuiUd8IvgkRyt6NQ==",
"license": "Apache-2.0",
"dependencies": {
"@chevrotain/gast": "11.0.3",
"@chevrotain/types": "11.0.3",
"lodash-es": "4.17.21"
}
},
"node_modules/@chevrotain/gast": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/@chevrotain/gast/-/gast-11.0.3.tgz",
"integrity": "sha512-+qNfcoNk70PyS/uxmj3li5NiECO+2YKZZQMbmjTqRI3Qchu8Hig/Q9vgkHpI3alNjr7M+a2St5pw5w5F6NL5/Q==",
"license": "Apache-2.0",
"dependencies": {
"@chevrotain/types": "11.0.3",
"lodash-es": "4.17.21"
}
},
"node_modules/@chevrotain/regexp-to-ast": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/@chevrotain/regexp-to-ast/-/regexp-to-ast-11.0.3.tgz",
"integrity": "sha512-1fMHaBZxLFvWI067AVbGJav1eRY7N8DDvYCTwGBiE/ytKBgP8azTdgyrKyWZ9Mfh09eHWb5PgTSO8wi7U824RA==",
"license": "Apache-2.0"
},
"node_modules/@chevrotain/types": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/@chevrotain/types/-/types-11.0.3.tgz",
"integrity": "sha512-gsiM3G8b58kZC2HaWR50gu6Y1440cHiJ+i3JUvcp/35JchYejb2+5MVeJK0iKThYpAa/P2PYFV4hoi44HD+aHQ==",
"license": "Apache-2.0"
},
"node_modules/@chevrotain/utils": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/@chevrotain/utils/-/utils-11.0.3.tgz",
"integrity": "sha512-YslZMgtJUyuMbZ+aKvfF3x1f5liK4mWNxghFRv7jqRR9C3R3fAOGTTKvxXDa2Y1s9zSbcpuO0cAxDYsc9SrXoQ==",
"license": "Apache-2.0"
},
"node_modules/@es-joy/jsdoccomment": {
"version": "0.46.0",
"resolved": "https://registry.npmjs.org/@es-joy/jsdoccomment/-/jsdoccomment-0.46.0.tgz",
@@ -3381,6 +3422,20 @@
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/chevrotain": {
"version": "11.0.3",
"resolved": "https://registry.npmjs.org/chevrotain/-/chevrotain-11.0.3.tgz",
"integrity": "sha512-ci2iJH6LeIkvP9eJW6gpueU8cnZhv85ELY8w8WiFtNjMHA5ad6pQLaJo9mEly/9qUyCpvqX8/POVUTf18/HFdw==",
"license": "Apache-2.0",
"dependencies": {
"@chevrotain/cst-dts-gen": "11.0.3",
"@chevrotain/gast": "11.0.3",
"@chevrotain/regexp-to-ast": "11.0.3",
"@chevrotain/types": "11.0.3",
"@chevrotain/utils": "11.0.3",
"lodash-es": "4.17.21"
}
},
"node_modules/chrome-trace-event": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/chrome-trace-event/-/chrome-trace-event-1.0.4.tgz",
@@ -6308,6 +6363,12 @@
"integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==",
"license": "MIT"
},
"node_modules/lodash-es": {
"version": "4.17.21",
"resolved": "https://registry.npmjs.org/lodash-es/-/lodash-es-4.17.21.tgz",
"integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==",
"license": "MIT"
},
"node_modules/lodash.get": {
"version": "4.4.2",
"resolved": "https://registry.npmjs.org/lodash.get/-/lodash.get-4.4.2.tgz",
+2
View File
@@ -35,6 +35,7 @@
"bowser": "^2.12.1",
"bytes": "^3.1.2",
"chalk": "^5.6.0",
"chevrotain": "^11.0.3",
"command-exists": "^1.2.9",
"compression": "^1.8.1",
"cookie-parser": "^1.4.6",
@@ -136,6 +137,7 @@
},
"main": "server.js",
"devDependencies": {
"@chevrotain/types": "^11.0.3",
"@types/archiver": "^6.0.3",
"@types/bytes": "^3.1.5",
"@types/command-exists": "^1.2.3",
+515
View File
@@ -0,0 +1,515 @@
/**
* Macro System Styles
* Styles for the macro browser, documentation, and related UI components.
*/
/* =============================================================================
MACRO BROWSER
Dynamic documentation browser for macros, similar to SlashCommandBrowser.
============================================================================= */
.macroBrowser {
display: flex;
flex-direction: column;
gap: 0.5em;
}
/* -----------------------------------------------------------------------------
Toolbar (Search + Sort)
----------------------------------------------------------------------------- */
.macroBrowser .macro-toolbar {
display: flex;
gap: 1em;
align-items: center;
flex-wrap: wrap;
}
.macroBrowser .macro-search-label {
flex: 1 1 auto;
display: flex;
gap: 0.5em;
align-items: center;
min-width: 200px;
white-space: nowrap;
}
.macroBrowser .macro-search-input {
flex: 1 1 auto;
min-width: 100px;
}
.macroBrowser .macro-sort-btn {
flex: 0 0 auto;
white-space: nowrap;
}
.macroBrowser .macro-sort-btn.active {
background-color: var(--SmartThemeQuoteColor);
}
/* -----------------------------------------------------------------------------
Container (List + Details panels)
----------------------------------------------------------------------------- */
.macroBrowser .macro-container {
display: flex;
gap: 1em;
align-items: flex-start;
container-type: inline-size;
}
.macroBrowser .macro-list-panel {
flex: 1 1 60%;
display: flex;
flex-direction: column;
gap: 2px;
max-height: 60vh;
overflow-y: auto;
}
.macroBrowser .macro-details-panel {
flex: 0 0 40%;
position: sticky;
top: 0;
max-height: 60vh;
overflow-y: auto;
background: var(--SmartThemeBlurTintColor);
border-radius: 10px;
padding: 1em;
}
.macroBrowser .macro-details-placeholder {
opacity: 0.6;
text-align: center;
padding: 2em;
}
/* -----------------------------------------------------------------------------
Category Headers
----------------------------------------------------------------------------- */
.macroBrowser .macro-category-header {
font-weight: bold;
padding: 0.75em 0.5em 0.25em;
margin-top: 0.5em;
border-bottom: 1px solid var(--SmartThemeBorderColor);
border-radius: 10px 10px 0 0;
color: var(--SmartThemeQuoteColor);
position: sticky;
top: 0;
background: var(--SmartThemeChatTintColor);
z-index: 1;
}
.macroBrowser .macro-category-header:first-child {
margin-top: 0;
}
.macroBrowser .macro-category-header.isFiltered {
display: none;
}
/* -----------------------------------------------------------------------------
Macro List Items
Layout: [signature] [description (shrinks)] [source icon]
----------------------------------------------------------------------------- */
.macroBrowser .macro-item {
display: flex;
align-items: center;
gap: 0.5em;
padding: 0.5em;
border-radius: 4px;
cursor: pointer;
transition: background-color 0.15s;
min-width: 0; /* Allow children to shrink */
}
.macroBrowser .macro-item:hover {
background: var(--black30a);
}
.macroBrowser .macro-item.selected {
background: var(--SmartThemeQuoteColor);
}
.macroBrowser .macro-item.isFiltered {
display: none;
}
.macroBrowser .macro-signature {
padding: 0.2em 0;
font-family: var(--monoFontFamily);
font-size: 0.9em;
color: var(--SmartThemeQuoteColor);
flex: 0 1 auto; /* Can shrink, but prefers not to */
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
min-width: 3em; /* Minimum before fully hidden */
}
.macroBrowser .macro-desc-preview {
flex: 1 1 0; /* Shrinks first, starts at 0 basis */
opacity: 0.7;
font-size: 0.9em;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
min-width: 0; /* Allow shrinking to nothing */
}
.macroBrowser .macro-source {
flex: 0 0 auto; /* Never shrinks */
font-size: 0.8em;
opacity: 0.7;
}
.macroBrowser .macro-source.isExtension.isThirdParty {
color: #f0a030;
}
.macroBrowser .macro-source.isExtension.isCore {
color: #30a0f0;
}
.macroBrowser .macro-source.isCore:not(.isExtension) {
color: #50c050;
}
/* -----------------------------------------------------------------------------
Macro Details Panel
----------------------------------------------------------------------------- */
.macro-details .macro-details-header {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 0.5em;
margin-bottom: 0.75em;
}
.macro-details .macro-details-name {
font-family: var(--monoFontFamily);
font-size: 1.1em;
color: var(--SmartThemeQuoteColor);
background: var(--black30a);
padding: 0.15em 0.4em 0.3em;
border-radius: 4px;
word-break: break-all;
display: inline-block; /* Shrink-wrap to content */
}
.macro-details .macro-details-header > .macro-source {
flex: 0 0 auto;
font-size: 1em;
margin-top: 0.15em; /* Align with name top padding */
}
.macro-details .macro-category-badge {
display: inline-block;
font-size: 0.75em;
padding: 0.2em 0.6em;
border-radius: 10px;
background: color-mix(in srgb, var(--SmartThemeQuoteColor) 50%, transparent);
color: var(--SmartThemeBodyColor);
margin-bottom: 0.75em;
}
.macro-details .macro-details-section {
margin-bottom: 1em;
}
.macro-details .macro-details-label {
font-weight: bold;
font-size: 0.85em;
opacity: 0.8;
margin-bottom: 0.25em;
text-transform: uppercase;
letter-spacing: 0.05em;
}
.macro-details .macro-details-text {
line-height: 1.4;
}
.macro-details .macro-returns-content {
display: flex;
align-items: baseline;
gap: 0.5em;
flex-wrap: wrap;
}
/* -----------------------------------------------------------------------------
Arguments List (in details panel)
----------------------------------------------------------------------------- */
.macro-details .macro-args-list {
list-style: none;
padding: 0;
margin: 0;
}
.macro-details .macro-arg-item {
padding: 0.4em 0;
border-bottom: 1px solid var(--black30a);
display: flex;
flex-wrap: wrap;
align-items: baseline;
gap: 0.5em;
}
.macro-details .macro-arg-item:last-child {
border-bottom: none;
}
.macro-details .macro-arg-name {
font-family: var(--monoFontFamily);
font-weight: bold;
color: var(--SmartThemeQuoteColor);
}
.macro-details .macro-arg-type {
font-size: 0.75em;
padding: 0.1em 0.4em;
border-radius: 3px;
background: var(--black30a);
color: var(--SmartThemeEmColor);
}
.macro-details .macro-arg-required {
font-size: 0.8em;
color: var(--warning-color, #e8a97f);
}
.macro-details .macro-arg-desc {
flex: 1 1 100%;
font-size: 0.9em;
opacity: 0.8;
}
.macro-details .macro-arg-sample {
font-size: 0.85em;
opacity: 0.6;
font-style: italic;
}
.macro-details .macro-arg-list-info {
font-size: 0.85em;
opacity: 0.8;
}
.macro-details .macro-example-list {
list-style: disc;
padding-left: 1.5em;
margin: 0.25em 0;
}
.macro-details .macro-example-list li {
padding: 0.2em 0;
}
.macro-details .macro-example-list code {
font-family: var(--monoFontFamily);
background: var(--black30a);
padding: 0.1em 0.3em;
border-radius: 3px;
}
/* Alias indicator icon in list items */
.macro-details .macro-alias-indicator {
flex: 0 0 auto;
font-size: 0.8em;
opacity: 0.6;
margin-left: 0.5em;
transform: rotate(90deg);
}
.macro-details .macro-item.isAlias .macro-signature {
opacity: 0.85;
}
/* Alias of indicator in details panel */
.macro-details .macro-alias-of {
display: flex;
align-items: center;
gap: 0.5em;
margin: 0.5em 0;
padding: 0.4em 0.6em;
background: var(--black30a);
border-radius: 4px;
font-size: 0.9em;
opacity: 0.9;
}
.macro-details .macro-alias-of i {
transform: rotate(90deg);
font-size: 0.85em;
opacity: 0.7;
}
.macro-details .macro-alias-of code {
font-family: var(--monoFontFamily);
background: var(--black30a);
padding: 0.1em 0.4em;
border-radius: 3px;
}
/* Aliases list in details panel */
.macro-details .macro-alias-list {
list-style: none;
padding: 0;
margin: 0.25em 0;
display: flex;
flex-wrap: wrap;
gap: 0.5em;
}
.macro-details .macro-alias-item {
display: inline-flex;
align-items: center;
gap: 0.3em;
}
.macro-details .macro-alias-item code {
font-family: var(--monoFontFamily);
background: var(--black30a);
padding: 0.2em 0.5em;
border-radius: 3px;
}
.macro-details .macro-alias-item.isHidden {
opacity: 0.5;
}
.macro-details .macro-alias-item.isHidden code {
text-decoration: line-through;
text-decoration-style: dashed;
}
.macro-details .macro-alias-hidden-badge {
font-size: 0.75em;
opacity: 0.7;
font-style: italic;
}
/* ============================================
Enhanced Macro Autocomplete Styles
============================================ */
/* Fix macro items in autocomplete list - override the grid display: contents */
.autoComplete > .item.macro-ac-item {
display: flex !important;
flex-wrap: nowrap;
align-items: center;
gap: 0.5em;
grid-column: 1 / -1;
padding: 0.25em 0.5em;
min-height: 1.8em;
}
.autoComplete > .item.macro-ac-item > .type {
flex: 0 0 auto;
width: 2em;
text-align: center;
}
.autoComplete > .item.macro-ac-item > .specs {
flex: 0 0 auto;
max-width: 40%;
}
.autoComplete > .item.macro-ac-item > .specs > .name {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.autoComplete > .item.macro-ac-item > .stopgap {
flex: 0 0 0.5em;
display: block !important;
}
.autoComplete > .item.macro-ac-item > .help {
flex: 1 1 auto;
min-width: 0;
overflow: hidden;
}
.autoComplete > .item.macro-ac-item > .help > .helpContent {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
display: block;
}
/* Indicator icons (alias and source) at the end */
.autoComplete > .item.macro-ac-item > .macro-ac-indicator {
flex: 0 0 auto;
font-size: 0.85em;
opacity: 0.6;
margin-left: auto;
}
.autoComplete > .item.macro-ac-item > .macro-ac-indicator:first-of-type {
margin-left: auto;
}
.autoComplete > .item.macro-ac-item > .macro-ac-indicator + .macro-ac-indicator {
margin-left: 0.25em;
}
/* Third-party source indicator */
.autoComplete > .item.macro-ac-item > .macro-source.isThirdParty {
color: #F89406;
}
/* Current argument hint banner in details */
.macro-ac-arg-hint {
display: flex;
align-items: baseline;
gap: 0.5em;
padding: 0.5em 0.75em;
background: linear-gradient(90deg, var(--ac-color-selectedBackground, var(--SmartThemeQuoteColor)), transparent);
border-left: 3px solid var(--ac-color-matchedText, var(--SmartThemeBorderColor));
border-radius: 0 4px 4px 0;
margin-bottom: 0.5em;
font-size: 0.9em;
}
.macro-ac-arg-hint i {
color: var(--ac-color-matchedText, var(--SmartThemeBorderColor));
font-size: 0.8em;
}
.macro-ac-hint-type {
font-family: var(--monoFontFamily);
font-size: 0.85em;
padding: 0.1em 0.3em;
background: var(--ac-color-hoveredBackground, var(--black30a));
border-radius: 3px;
margin-left: 0.25em;
}
.macro-ac-hint-desc {
opacity: 0.8;
}
.macro-ac-hint-sample {
opacity: 0.6;
font-style: italic;
}
/* Details panel using MacroBrowser styles - just add autocomplete context adjustments */
.autoComplete-details .macro-ac-details {
padding: 0.5em;
}
/* Highlight current argument in arguments list */
.autoComplete-details .macro-arg-item.current {
background: var(--ac-color-selectedBackground, var(--SmartThemeQuoteColor));
padding: 0.25em 0.5em;
margin: 0 -0.5em;
border-radius: 4px;
}
+6
View File
@@ -42,6 +42,7 @@
<link rel="stylesheet" type="text/css" href="css/extensions-panel.css">
<link rel="stylesheet" type="text/css" href="css/select2-overrides.css">
<link rel="stylesheet" type="text/css" href="css/mobile-styles.css">
<link rel="stylesheet" type="text/css" href="css/macros.css">
<link rel="stylesheet" type="text/css" href="css/user.css">
<link rel="icon" type="image/x-icon" href="favicon.ico">
<!-- Scripts are loaded at the end of the body to improve page load speed -->
@@ -5258,6 +5259,11 @@
<input id="encode_tags" type="checkbox" />
<small data-i18n="Show tags in responses">Show &lt;tags&gt; in responses</small>
</label>
<label class="checkbox_label" for="experimental_macro_engine" title="Experimental new Macro Engine.&NewLine;&NewLine;Allows nested macros to be resolved correctly and has a dedicated, logical replacement order.&NewLine;The new engine is designed to cleanly replace the old regex-based macro system.">
<input id="experimental_macro_engine" type="checkbox" />
<small data-i18n="Experimental Macro Engine">Experimental Macro Engine</small>
<i class="fa-solid fa-flask" title="Experimental feature. Currently in development to test." data-i18n="[title]Experimental feature. Currently in development to test."></i>
</label>
<label class="checkbox_label" for="disable_group_trimming" title="Allow AI messages in groups to contain lines spoken by other group members." data-i18n="[title]Allow AI messages in groups to contain lines spoken by other group members">
<input id="disable_group_trimming" type="checkbox" />
<small data-i18n="Relax message trim in Groups">Relax message trim in Groups</small>
+3
View File
@@ -22,6 +22,7 @@ import morphdom from 'morphdom';
import { toggle as slideToggle } from 'slidetoggle';
import chalk from 'chalk';
import yaml from 'yaml';
import * as chevrotain from 'chevrotain';
/**
* Expose the libraries to the 'window' object.
@@ -100,6 +101,7 @@ export default {
slideToggle,
chalk,
yaml,
chevrotain,
};
export {
@@ -124,4 +126,5 @@ export {
slideToggle,
chalk,
yaml,
chevrotain,
};
+178 -56
View File
@@ -69,7 +69,7 @@ import {
renameGroupChat,
importGroupChat,
getGroupBlock,
getGroupCharacterCards,
getGroupCharacterCardsLazy,
getGroupDepthPrompts,
} from './scripts/group-chats.js';
@@ -187,7 +187,7 @@ import {
import { debounce_timeout, GENERATION_TYPE_TRIGGERS, IGNORE_SYMBOL, inject_ids, MEDIA_DISPLAY, MEDIA_SOURCE, MEDIA_TYPE, OVERSWIPE_BEHAVIOR, SCROLL_BEHAVIOR, SWIPE_DIRECTION, SWIPE_SOURCE, SWIPE_STATE } from './scripts/constants.js';
import { cancelDebouncedMetadataSave, doDailyExtensionUpdatesCheck, extension_settings, initExtensions, loadExtensionSettings, runGenerationInterceptors } from './scripts/extensions.js';
import { COMMENT_NAME_DEFAULT, CONNECT_API_MAP, executeSlashCommandsOnChatInput, initDefaultSlashCommands, isExecutingCommandsFromChatInput, pauseScriptExecution, stopScriptExecution, UNIQUE_APIS } from './scripts/slash-commands.js';
import { COMMENT_NAME_DEFAULT, CONNECT_API_MAP, executeSlashCommandsOnChatInput, initDefaultSlashCommands, initSlashCommandAutoComplete, isExecutingCommandsFromChatInput, pauseScriptExecution, stopScriptExecution, UNIQUE_APIS } from './scripts/slash-commands.js';
import {
tag_map,
tags,
@@ -279,6 +279,8 @@ import { applyStreamFadeIn } from './scripts/util/stream-fadein.js';
import { initDomHandlers } from './scripts/dom-handlers.js';
import { SimpleMutex } from './scripts/util/SimpleMutex.js';
import { AudioPlayer } from './scripts/audio-player.js';
import { MacroEnvBuilder } from './scripts/macros/engine/MacroEnvBuilder.js';
import { MacroEngine } from './scripts/macros/engine/MacroEngine.js';
import { addChatBackupsBrowser } from './scripts/chat-backups.js';
// API OBJECT FOR EXTERNAL WIRING
@@ -707,6 +709,7 @@ async function firstLoadInit() {
initBackgrounds();
initAuthorsNote();
await initPersonas();
await initSlashCommandAutoComplete();
initWorldInfo();
initHorde();
initRossMods();
@@ -2661,14 +2664,13 @@ export function scrollChatToBottom({ waitForFrame } = {}) {
}
/**
* @deprecated Function is not needed anymore, as the new signature of substituteParams is more flexible.
*
* Substitutes {{macro}} parameters in a string.
* @param {string} content - The string to substitute parameters in.
* @param {Record<string,any>} additionalMacro - Additional environment variables for substitution.
* @param {(x: string) => string} [postProcessFn] - Post-processing function for each substituted macro.
* @returns {string} The string with substituted parameters.
*/
export function substituteParamsExtended(content, additionalMacro = {}, postProcessFn = (x) => x) {
return substituteParams(content, undefined, undefined, undefined, undefined, true, additionalMacro, postProcessFn);
return substituteParams(content, { dynamicMacros: additionalMacro, postProcessFn });
}
/**
@@ -2683,11 +2685,24 @@ export function substituteParamsExtended(content, additionalMacro = {}, postProc
* @param {(x: string) => string} [postProcessFn] - Post-processing function for each substituted macro.
* @returns {string} The string with substituted parameters.
*/
export function substituteParams(content, _name1, _name2, _original, _group, _replaceCharacterCard = true, additionalMacro = {}, postProcessFn = (x) => x) {
export function substituteParamsLegacy(content, _name1, _name2, _original, _group, _replaceCharacterCard = true, additionalMacro = {}, postProcessFn = (x) => x) {
if (!content) {
return '';
}
// If experimental macro engine is enabled, use it. This code will be cleaned up in the future.
if (power_user?.experimental_macro_engine) {
return substituteParams(content, {
name1Override: _name1,
name2Override: _name2,
original: _original,
groupOverride: _group,
replaceCharacterCard: _replaceCharacterCard ?? true,
dynamicMacros: additionalMacro ?? {},
postProcessFn: postProcessFn ?? ((x) => x),
});
}
const environment = {};
if (typeof _original === 'string') {
@@ -2787,6 +2802,55 @@ export function substituteParams(content, _name1, _name2, _original, _group, _re
return evaluateMacros(content, environment, postProcessFn);
}
/** @typedef {import('./scripts/macros/engine/MacroRegistry.js').MacroHandler} MacroHandler */
/**
* Substitutes {{macros}} in a string using the new macro engine.
*
* This will replace all registered macros and dynamic additional macros as environment context.
*
* @param {string} content - The string to substitute parameters in.
* @param {Object} [options={}] - Options for the substitution.
* @param {string} [options.name1Override] - The name of the user. Uses global name1 if not provided.
* @param {string} [options.name2Override] - The name of the character. Uses global name2 if not provided.
* @param {string} [options.original] - The original message for {{original}} substitution.
* @param {string} [options.groupOverride] - The group members list for {{group}} substitution.
* @param {boolean} [options.replaceCharacterCard=true] - Whether to replace character card macros.
* @param {Record<string,string|MacroHandler>} [options.dynamicMacros={}] - Additional environment variables as dynamic macros for substitution. Registered as macro functions.
* @param {(x: string) => string} [options.postProcessFn=(x) => x] - Post-processing function for each substituted macro.
* @returns {string} The string with substituted parameters.
*/
export function substituteParams(content, options = {}) {
if (!content) return '';
// Handle legacy signature calls to substituteParams
// We'll simply re-route them to a temporary legacy function. In the future, we'll remove this and cleanly build the options object ourselves.
const isOptionsObject = options && typeof options === 'object' && !Array.isArray(options);
if (!isOptionsObject) {
return substituteParamsLegacy.call(this, ...arguments);
}
// Keep the new macro engine behind a feature switch for now
if (!power_user?.experimental_macro_engine) {
return substituteParamsLegacy(content, options.name1Override, options.name2Override, options.original, options.groupOverride, options.replaceCharacterCard, options.dynamicMacros, options.postProcessFn);
}
const ctx = /** @type {import('./scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */ ({
content,
name1Override: options.name1Override,
name2Override: options.name2Override,
original: options.original,
groupOverride: options.groupOverride,
replaceCharacterCard: options.replaceCharacterCard ?? true,
dynamicMacros: options.dynamicMacros ?? {},
postProcessFn: options.postProcessFn ?? ((x) => x),
});
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
const result = MacroEngine.evaluate(content, env);
return result;
}
/**
* Gets stopping sequences for the prompt.
@@ -3110,11 +3174,7 @@ export function baseChatReplace(value, name1, name2) {
}
/**
* Returns the character card fields for the current character.
* @param {object} [options]
* @param {number} [options.chid] Optional character index
*
* @typedef {object} CharacterCardFields
* @typedef {Object} CharacterCardFields
* @property {string} system System prompt
* @property {string} mesExamples Message examples
* @property {string} description Description
@@ -3125,57 +3185,119 @@ export function baseChatReplace(value, name1, name2) {
* @property {string} version Character version
* @property {string} charDepthPrompt Character depth note
* @property {string} creatorNotes Character creator notes
* @returns {CharacterCardFields} Character card fields
*/
export function getCharacterCardFields({ chid = null } = {}) {
/**
* Helper to create an object with lazy, memoized getters from a map of field resolvers.
* @param {Record<string, () => string>} resolvers Map of field names to resolver functions
* @returns {CharacterCardFields} Object with lazy getters
*/
export function createLazyFields(resolvers) {
const result = /** @type {CharacterCardFields} */ ({});
for (const [key, resolver] of Object.entries(resolvers)) {
let cached;
let resolved = false;
Object.defineProperty(result, key, {
get() {
if (!resolved) {
cached = resolver();
resolved = true;
}
return cached;
},
enumerable: true,
configurable: true,
});
}
return result;
}
/**
* Returns the character card fields for the current character as lazy getters.
* Each field is only processed (baseChatReplace) when first accessed.
* @param {Object} [options={}]
* @param {number} [options.chid] Optional character index
* @returns {CharacterCardFields} Character card fields with lazy evaluation
*/
export function getCharacterCardFieldsLazy({ chid = undefined } = {}) {
const currentChid = chid ?? this_chid;
const result = {
system: '',
mesExamples: '',
description: '',
personality: '',
persona: '',
scenario: '',
jailbreak: '',
version: '',
charDepthPrompt: '',
creatorNotes: '',
};
result.persona = baseChatReplace(power_user.persona_description?.trim(), name1, name2);
const character = characters[currentChid];
if (!character) {
return result;
}
// For group chats, we need to check if group cards should be used
const useGroupCards = selected_group && character;
const groupCardsLazy = useGroupCards ? getGroupCharacterCardsLazy(selected_group, Number(currentChid)) : null;
const scenarioText = chat_metadata['scenario'] || character.scenario || '';
const exampleDialog = chat_metadata['mes_example'] || character.mes_example || '';
const systemPrompt = chat_metadata['system_prompt'] || character.data?.system_prompt || '';
/** @type {Record<string, () => string>} */
const resolvers = {
persona: () => baseChatReplace(power_user.persona_description?.trim(), name1, name2),
system: () => {
if (!character) return '';
const systemPrompt = chat_metadata['system_prompt'] || character.data?.system_prompt || '';
return power_user.prefer_character_prompt ? baseChatReplace(systemPrompt.trim(), name1, name2) : '';
},
jailbreak: () => {
if (!character) return '';
return power_user.prefer_character_jailbreak ? baseChatReplace(character.data?.post_history_instructions?.trim(), name1, name2) : '';
},
version: () => character?.data?.character_version ?? '',
charDepthPrompt: () => {
if (!character) return '';
return baseChatReplace(character.data?.extensions?.depth_prompt?.prompt?.trim(), name1, name2);
},
creatorNotes: () => {
if (!character) return '';
return baseChatReplace(character.data?.creator_notes?.trim(), name1, name2);
},
// These four fields may be overridden by group cards
description: () => {
if (groupCardsLazy) return groupCardsLazy.description;
if (!character) return '';
return baseChatReplace(character.description?.trim(), name1, name2);
},
personality: () => {
if (groupCardsLazy) return groupCardsLazy.personality;
if (!character) return '';
return baseChatReplace(character.personality?.trim(), name1, name2);
},
scenario: () => {
if (groupCardsLazy) return groupCardsLazy.scenario;
if (!character) return '';
const scenarioText = chat_metadata['scenario'] || character.scenario || '';
return baseChatReplace(scenarioText.trim(), name1, name2);
},
mesExamples: () => {
if (groupCardsLazy) return groupCardsLazy.mesExamples;
if (!character) return '';
const exampleDialog = chat_metadata['mes_example'] || character.mes_example || '';
return baseChatReplace(exampleDialog.trim(), name1, name2);
},
};
result.description = baseChatReplace(character.description?.trim(), name1, name2);
result.personality = baseChatReplace(character.personality?.trim(), name1, name2);
result.scenario = baseChatReplace(scenarioText.trim(), name1, name2);
result.mesExamples = baseChatReplace(exampleDialog.trim(), name1, name2);
result.system = power_user.prefer_character_prompt ? baseChatReplace(systemPrompt.trim(), name1, name2) : '';
result.jailbreak = power_user.prefer_character_jailbreak ? baseChatReplace(character.data?.post_history_instructions?.trim(), name1, name2) : '';
result.version = character.data?.character_version ?? '';
result.charDepthPrompt = baseChatReplace(character.data?.extensions?.depth_prompt?.prompt?.trim(), name1, name2);
result.creatorNotes = baseChatReplace(character.data?.creator_notes?.trim(), name1, name2);
return createLazyFields(resolvers);
}
if (selected_group) {
const groupCards = getGroupCharacterCards(selected_group, Number(currentChid));
/**
* Returns the character card fields for the current character.
* @param {Object} [options={}]
* @param {number} [options.chid] Optional character index
* @returns {CharacterCardFields} Character card fields
*/
export function getCharacterCardFields({ chid = undefined } = {}) {
const lazy = getCharacterCardFieldsLazy({ chid });
if (groupCards) {
result.description = groupCards.description;
result.personality = groupCards.personality;
result.scenario = groupCards.scenario;
result.mesExamples = groupCards.mesExamples;
}
}
return result;
// Resolve all lazy fields into a plain object
return {
system: lazy.system,
mesExamples: lazy.mesExamples,
description: lazy.description,
personality: lazy.personality,
persona: lazy.persona,
scenario: lazy.scenario,
jailbreak: lazy.jailbreak,
version: lazy.version,
charDepthPrompt: lazy.charDepthPrompt,
creatorNotes: lazy.creatorNotes,
};
}
/**
@@ -6610,7 +6732,7 @@ export function getGeneratingApi() {
}
}
function getGeneratingModel(mes) {
export function getGeneratingModel(mes) {
let model = '';
switch (main_api) {
case 'kobold':
+21 -4
View File
@@ -19,7 +19,7 @@ import { SlashCommand } from './slash-commands/SlashCommand.js';
import { ARGUMENT_TYPE, SlashCommandArgument } from './slash-commands/SlashCommandArgument.js';
export { MODULE_NAME as NOTE_MODULE_NAME };
import { t } from './i18n.js';
import { MacrosParser } from './macros.js';
import { macros, MacroCategory } from './macros/macro-system.js';
const MODULE_NAME = '2_floating_prompt'; // <= Deliberate, for sorting lower than memory
@@ -581,7 +581,24 @@ export function initAuthorsNote() {
}));
eventSource.on(event_types.CHAT_CHANGED, onChatChanged);
MacrosParser.registerMacro('authorsNote', () => chat_metadata[metadata_keys.prompt] ?? '', t`The contents of the Author's Note`);
MacrosParser.registerMacro('charAuthorsNote', () => this_chid !== undefined ? (extension_settings.note.chara.find((e) => e.name === getCharaFilename())?.prompt ?? '') : '', t`The contents of the Character Author's Note`);
MacrosParser.registerMacro('defaultAuthorsNote', () => extension_settings.note.default ?? '', t`The contents of the Default Author's Note`);
registerAuthorsNoteMacros();
}
function registerAuthorsNoteMacros() {
macros.register('authorsNote', {
category: MacroCategory.PROMPTS,
description: t`The contents of the Author's Note`,
handler: () => chat_metadata[metadata_keys.prompt] ?? '',
});
macros.register('charAuthorsNote', {
category: MacroCategory.CHARACTER,
description: t`The contents of the Character Author's Note`,
handler: () => this_chid !== undefined ? (extension_settings.note.chara.find((e) => e.name === getCharaFilename())?.prompt ?? '') : '',
});
macros.register('defaultAuthorsNote', {
category: MacroCategory.PROMPTS,
description: t`The contents of the Default Author's Note`,
handler: () => extension_settings.note.default ?? '',
});
}
@@ -0,0 +1,260 @@
/**
* Enhanced macro autocomplete option for the new MacroRegistry-based system.
* Reuses rendering logic from MacroBrowser for consistency and DRY.
*/
import { AutoCompleteOption } from './AutoCompleteOption.js';
import {
formatMacroSignature,
createSourceIndicator,
createAliasIndicator,
renderMacroDetails,
} from '../macros/MacroBrowser.js';
import { enumIcons } from '../slash-commands/SlashCommandCommonEnumsProvider.js';
/** @typedef {import('../macros/engine/MacroRegistry.js').MacroDefinition} MacroDefinition */
/**
* Macro context passed from the parser to provide cursor position info.
* @typedef {Object} MacroAutoCompleteContext
* @property {string} fullText - The full macro text being typed (without {{ }}).
* @property {number} cursorOffset - Cursor position within the macro text.
* @property {string} identifier - The macro identifier (name).
* @property {string[]} args - Array of arguments typed so far.
* @property {number} currentArgIndex - Index of the argument being typed (-1 if on identifier).
*/
export class EnhancedMacroAutoCompleteOption extends AutoCompleteOption {
/** @type {MacroDefinition} */
#macro;
/** @type {MacroAutoCompleteContext|null} */
#context = null;
/**
* @param {MacroDefinition} macro - The macro definition from MacroRegistry.
* @param {MacroAutoCompleteContext} [context] - Optional context for argument hints.
*/
constructor(macro, context = null) {
// Use the macro name as the autocomplete key
super(macro.name, enumIcons.macro);
this.#macro = macro;
this.#context = context;
// nameOffset = 2 to skip the {{ prefix in the display (formatMacroSignature includes braces)
this.nameOffset = 2;
}
/** @returns {MacroDefinition} */
get macro() {
return this.#macro;
}
/**
* Renders the list item for the autocomplete dropdown.
* Tight display: [icon] [signature] [description] [alias icon?] [source icon]
* @returns {HTMLElement}
*/
renderItem() {
const li = document.createElement('li');
li.classList.add('item', 'macro-ac-item');
li.setAttribute('data-name', this.name);
li.setAttribute('data-option-type', 'macro');
// Type icon
const type = document.createElement('span');
type.classList.add('type', 'monospace');
type.textContent = '{}';
li.append(type);
// Specs container (for fuzzy highlight compatibility)
const specs = document.createElement('span');
specs.classList.add('specs');
// Name with character spans for fuzzy highlighting
const nameEl = document.createElement('span');
nameEl.classList.add('name', 'monospace');
// Build signature with individual character spans (includes {{ }})
const sigText = formatMacroSignature(this.#macro);
for (const char of sigText) {
const span = document.createElement('span');
span.textContent = char;
nameEl.append(span);
}
specs.append(nameEl);
li.append(specs);
// Stopgap (spacer for flex layout)
const stopgap = document.createElement('span');
stopgap.classList.add('stopgap');
li.append(stopgap);
// Help text (description)
const help = document.createElement('span');
help.classList.add('help');
const content = document.createElement('span');
content.classList.add('helpContent');
content.textContent = this.#macro.description || '';
help.append(content);
li.append(help);
// Alias indicator icon (if this is an alias)
const aliasIcon = createAliasIndicator(this.#macro);
if (aliasIcon) {
aliasIcon.classList.add('macro-ac-indicator');
li.append(aliasIcon);
}
// Source indicator icon
const sourceIcon = createSourceIndicator(this.#macro);
sourceIcon.classList.add('macro-ac-indicator');
li.append(sourceIcon);
return li;
}
/**
* Renders the details panel content.
* Reuses renderMacroDetails from MacroBrowser with autocomplete-specific options.
* @returns {DocumentFragment}
*/
renderDetails() {
const frag = document.createDocumentFragment();
// Determine current argument index for highlighting
const currentArgIndex = this.#context?.currentArgIndex ?? -1;
// Render argument hint banner if we're typing an argument
if (currentArgIndex >= 0) {
const hint = this.#renderArgumentHint();
if (hint) frag.append(hint);
}
// Reuse MacroBrowser's renderMacroDetails with options
const details = renderMacroDetails(this.#macro, { currentArgIndex });
// Add class for autocomplete-specific styling overrides
details.classList.add('macro-ac-details');
frag.append(details);
return frag;
}
/**
* Renders the current argument hint banner.
* @returns {HTMLElement|null}
*/
#renderArgumentHint() {
if (!this.#context || this.#context.currentArgIndex < 0) return null;
const argIndex = this.#context.currentArgIndex;
const isListArg = argIndex >= this.#macro.maxArgs;
// If we're beyond unnamed args and there's no list, no hint
if (isListArg && !this.#macro.list) return null;
const hint = document.createElement('div');
hint.classList.add('macro-ac-arg-hint');
const icon = document.createElement('i');
icon.classList.add('fa-solid', 'fa-arrow-right');
hint.append(icon);
if (isListArg) {
// List argument hint
const listIndex = argIndex - this.#macro.maxArgs + 1;
const text = document.createElement('span');
text.innerHTML = `<strong>List item ${listIndex}</strong>`;
hint.append(text);
} else {
// Unnamed argument hint (required or optional)
const argDef = this.#macro.unnamedArgDefs[argIndex];
let optionalLabel = '';
if (argDef?.optional) {
optionalLabel = argDef.defaultValue !== undefined
? ` <em>(optional, default: ${argDef.defaultValue === '' ? '<empty string>' : argDef.defaultValue})</em>`
: ' <em>(optional)</em>';
}
const text = document.createElement('span');
text.innerHTML = `<strong>${argDef?.name || `Argument ${argIndex + 1}`}</strong>${optionalLabel}`;
if (argDef?.type) {
const typeSpan = document.createElement('code');
typeSpan.classList.add('macro-ac-hint-type');
if (Array.isArray(argDef.type)) {
typeSpan.textContent = argDef.type.join(' | ');
typeSpan.title = `Accepts: ${argDef.type.join(', ')}`;
} else {
typeSpan.textContent = argDef.type;
}
text.append(' ', typeSpan);
}
hint.append(text);
if (argDef?.description) {
const descSpan = document.createElement('span');
descSpan.classList.add('macro-ac-hint-desc');
descSpan.textContent = `${argDef.description}`;
hint.append(descSpan);
}
if (argDef?.sampleValue) {
const sampleSpan = document.createElement('span');
sampleSpan.classList.add('macro-ac-hint-sample');
sampleSpan.textContent = ` (e.g. ${argDef.sampleValue})`;
hint.append(sampleSpan);
}
}
return hint;
}
}
/**
* Parses the macro text to determine current argument context.
* @param {string} macroText - The text inside {{ }}, e.g., "roll::1d20" or "random::a::b".
* @param {number} cursorOffset - Cursor position within macroText.
* @returns {MacroAutoCompleteContext}
*/
export function parseMacroContext(macroText, cursorOffset) {
const parts = [];
let currentPart = '';
let partStart = 0;
let i = 0;
while (i < macroText.length) {
if (macroText[i] === ':' && macroText[i + 1] === ':') {
parts.push({ text: currentPart, start: partStart, end: i });
currentPart = '';
i += 2;
partStart = i;
} else {
currentPart += macroText[i];
i++;
}
}
// Push the last part
parts.push({ text: currentPart, start: partStart, end: macroText.length });
// Determine which part the cursor is in
let currentArgIndex = -1;
for (let idx = 0; idx < parts.length; idx++) {
const part = parts[idx];
if (cursorOffset >= part.start && cursorOffset <= part.end) {
currentArgIndex = idx - 1; // -1 because first part is identifier
break;
}
}
// If cursor is after all parts (at the end), we're in the last arg
if (currentArgIndex === -1 && cursorOffset >= parts[parts.length - 1].end) {
currentArgIndex = parts.length - 1;
}
return {
fullText: macroText,
cursorOffset,
identifier: parts[0]?.text.trim() || '',
args: parts.slice(1).map(p => p.text),
currentArgIndex,
};
}
+15 -3
View File
@@ -19,17 +19,18 @@ import {
animation_easing,
} from '../../../script.js';
import { is_group_generating, selected_group } from '../../group-chats.js';
import { loadMovingUIState } from '../../power-user.js';
import { loadMovingUIState, power_user } from '../../power-user.js';
import { dragElement } from '../../RossAscends-mods.js';
import { getTextTokens, getTokenCountAsync, tokenizers } from '../../tokenizers.js';
import { debounce_timeout } from '../../constants.js';
import { SlashCommandParser } from '../../slash-commands/SlashCommandParser.js';
import { SlashCommand } from '../../slash-commands/SlashCommand.js';
import { ARGUMENT_TYPE, SlashCommandArgument, SlashCommandNamedArgument } from '../../slash-commands/SlashCommandArgument.js';
import { MacrosParser } from '../../macros.js';
import { macros, MacroCategory } from '../../macros/macro-system.js';
import { countWebLlmTokens, generateWebLlmChatPrompt, getWebLlmContextSize, isWebLlmSupported } from '../shared.js';
import { commonEnumProviders } from '../../slash-commands/SlashCommandCommonEnumsProvider.js';
import { removeReasoningFromString } from '../../reasoning.js';
import { MacrosParser } from '/scripts/macros.js';
export { MODULE_NAME };
const MODULE_NAME = '1_memory';
@@ -1094,5 +1095,16 @@ jQuery(async function () {
returns: ARGUMENT_TYPE.STRING,
}));
MacrosParser.registerMacro('summary', () => getLatestMemoryFromChat(getContext().chat));
if (power_user.experimental_macro_engine) {
macros.register('summary', {
category: MacroCategory.CHAT,
description: 'Returns the latest memory/summary from the current chat.',
handler: () => getLatestMemoryFromChat(getContext().chat),
});
} else {
// TODO: Remove this when the experimental macro engine is replacing the old macro engine
MacrosParser.registerMacro('summary',
() => getLatestMemoryFromChat(getContext().chat),
'Returns the latest memory/summary from the current chat.');
}
});
@@ -57,9 +57,11 @@ import { SlashCommandEnumValue } from '../../slash-commands/SlashCommandEnumValu
import { callGenericPopup, Popup, POPUP_TYPE } from '../../popup.js';
import { commonEnumProviders } from '../../slash-commands/SlashCommandCommonEnumsProvider.js';
import { ToolManager } from '../../tool-calling.js';
import { MacrosParser } from '../../macros.js';
import { macros, MacroCategory } from '../../macros/macro-system.js';
import { t, translate } from '../../i18n.js';
import { oai_settings } from '../../openai.js';
import { power_user } from '/scripts/power-user.js';
import { MacrosParser } from '/scripts/macros.js';
export { MODULE_NAME };
@@ -5145,6 +5147,25 @@ jQuery(async () => {
return isNegative ? negativePrompt : characterPrompt;
};
MacrosParser.registerMacro('charPrefix', () => getMacroValue({ isNegative: false }), t`Character's positive positive Image Generation prompt prefix`);
MacrosParser.registerMacro('charNegativePrefix', () => getMacroValue({ isNegative: true }), t`Character's negative Image Generation prompt prefix`);
if (power_user.experimental_macro_engine) {
macros.register('charPrefix', {
category: MacroCategory.PROMPTS,
description: t`Character's positive Image Generation prompt prefix`,
handler: () => getMacroValue({ isNegative: false }),
});
macros.register('charNegativePrefix', {
category: MacroCategory.PROMPTS,
description: t`Character's negative Image Generation prompt prefix`,
handler: () => getMacroValue({ isNegative: true }),
});
} else {
MacrosParser.registerMacro('charPrefix',
() => getMacroValue({ isNegative: false }),
t`Character's positive Image Generation prompt prefix`,
);
MacrosParser.registerMacro('charNegativePrefix',
() => getMacroValue({ isNegative: true }),
t`Character's negative Image Generation prompt prefix`,
);
}
});
+60 -52
View File
@@ -73,6 +73,7 @@ import {
isChatSaving,
setExternalAbortController,
baseChatReplace,
createLazyFields,
depth_prompt_depth_default,
loadItemizedPrompts,
animation_duration,
@@ -477,26 +478,43 @@ export function getGroupDepthPrompts(groupId, characterId) {
* @returns {{description: string, personality: string, scenario: string, mesExamples: string}} Group character cards combined
*/
export function getGroupCharacterCards(groupId, characterId) {
const lazy = getGroupCharacterCardsLazy(groupId, characterId);
if (!lazy) return null;
// Resolve all lazy fields into a plain object
return {
description: lazy.description,
personality: lazy.personality,
scenario: lazy.scenario,
mesExamples: lazy.mesExamples,
};
}
/**
* Returns group character cards with lazy evaluation.
* Each field is only processed when first accessed.
* @param {string} groupId Group ID
* @param {number} characterId Current Character ID
* @returns {{description: string, personality: string, scenario: string, mesExamples: string}} Group character cards with lazy getters
*/
export function getGroupCharacterCardsLazy(groupId, characterId) {
const group = groups.find(x => x.id === groupId);
// If no group cards should be generated, return null so caller knows to fall back
if (!group || !group?.generation_mode || !Array.isArray(group.members) || !group.members.length) {
return null;
}
/**
* Runs the macro engine on a text, with custom <FIELDNAME> replace
* Runs baseChatReplace on a text, with custom <FIELDNAME> replace
* @param {string} value Value to replace
* @param {string} fieldName Name of the field
* @param {string} characterName Name of the character
* @param {boolean} trim Whether to trim the value
* @returns {string} Replaced text
* */
function customBaseChatReplace(value, fieldName, characterName, trim) {
if (!value) {
return '';
}
// We should do the custom field name replacement first, and then run it through the normal macro engine with provided names
*/
function customTransform(value, fieldName, characterName, trim) {
if (!value) return '';
value = value.replace(/<FIELDNAME>/gi, fieldName);
value = trim ? value.trim() : value;
return baseChatReplace(value, name1, characterName);
@@ -509,60 +527,50 @@ export function getGroupCharacterCards(groupId, characterId) {
* @param {string} fieldName Name of the field
* @param {function(string): string} [preprocess] Preprocess function
* @returns {string} Prepared text
* */
*/
function replaceAndPrepareForJoin(value, characterName, fieldName, preprocess = null) {
value = value.trim();
if (!value) {
return '';
}
// Run preprocess function
value = value?.trim() ?? '';
if (!value) return '';
if (typeof preprocess === 'function') {
value = preprocess(value);
}
// Prepare and replace prefixes
const prefix = customBaseChatReplace(group.generation_mode_join_prefix, fieldName, characterName, false);
const suffix = customBaseChatReplace(group.generation_mode_join_suffix, fieldName, characterName, false);
// Also run the macro replacement on the actual content
value = customBaseChatReplace(value, fieldName, characterName, true);
const prefix = customTransform(group.generation_mode_join_prefix, fieldName, characterName, false);
const suffix = customTransform(group.generation_mode_join_suffix, fieldName, characterName, false);
value = customTransform(value, fieldName, characterName, true);
return `${prefix}${value}${suffix}`;
}
/**
* Collects and joins field values from all group members
* @param {string} fieldName Display name of the field
* @param {function(import('../script.js').Character): string} getter Function to get field value from character
* @param {function(string): string} [preprocess] Optional preprocess function
* @returns {string} Combined field values
*/
function collectField(fieldName, getter, preprocess = null) {
const values = [];
for (const member of group.members) {
const index = characters.findIndex(x => x.avatar === member);
const character = characters[index];
if (index === -1 || !character) continue;
if (group.disabled_members.includes(member) && characterId !== index && group.generation_mode !== group_generation_mode.APPEND_DISABLED) {
continue;
}
values.push(replaceAndPrepareForJoin(getter(character), character.name, fieldName, preprocess));
}
return values.filter(x => x.length).join('\n');
}
const scenarioOverride = String(chat_metadata['scenario'] || '');
const mesExamplesOverride = String(chat_metadata['mes_example'] || '');
let descriptions = [];
let personalities = [];
let scenarios = [];
let mesExamplesArray = [];
for (const member of group.members) {
const index = characters.findIndex(x => x.avatar === member);
const character = characters[index];
if (index === -1 || !character) {
console.debug(`Skipping missing member: ${member}`);
continue;
}
if (group.disabled_members.includes(member) && characterId !== index && group.generation_mode !== group_generation_mode.APPEND_DISABLED) {
continue;
}
descriptions.push(replaceAndPrepareForJoin(character.description, character.name, 'Description'));
personalities.push(replaceAndPrepareForJoin(character.personality, character.name, 'Personality'));
scenarios.push(replaceAndPrepareForJoin(character.scenario, character.name, 'Scenario'));
mesExamplesArray.push(replaceAndPrepareForJoin(character.mes_example, character.name, 'Example Messages', (x) => !x.startsWith('<START>') ? `<START>\n${x}` : x));
}
const description = descriptions.filter(x => x.length).join('\n');
const personality = personalities.filter(x => x.length).join('\n');
const scenario = baseChatReplace(scenarioOverride?.trim(), name1, name2) || scenarios.filter(x => x.length).join('\n');
const mesExamples = baseChatReplace(mesExamplesOverride?.trim(), name1, name2) || mesExamplesArray.filter(x => x.length).join('\n');
return { description, personality, scenario, mesExamples };
return createLazyFields({
description: () => collectField('Description', c => c.description),
personality: () => collectField('Personality', c => c.personality),
scenario: () => baseChatReplace(scenarioOverride?.trim(), name1, name2) || collectField('Scenario', c => c.scenario),
mesExamples: () => baseChatReplace(mesExamplesOverride?.trim(), name1, name2) ||
collectField('Example Messages', c => c.mes_example, x => !x.startsWith('<START>') ? `<START>\n${x}` : x),
});
}
/**
+135 -12
View File
@@ -6,6 +6,8 @@ import { getInstructMacros } from './instruct-mode.js';
import { getVariableMacros } from './variables.js';
import { isMobile } from './RossAscends-mods.js';
import { inject_ids } from './constants.js';
import { initRegisterMacros, macros as macroSystem } from './macros/macro-system.js';
import { power_user } from './power-user.js';
/**
* @typedef Macro
@@ -33,6 +35,10 @@ Handlebars.registerHelper('helperMissing', function () {
* @property {string} description - Optional description of the macro
*/
/**
* @deprecated Use macros.registry.registerMacro (from scripts/macros/macro-system.js)
* or substituteParams({ dynamicMacros }) with the new macro engine.
*/
export class MacrosParser {
/**
* A map of registered macros.
@@ -46,11 +52,98 @@ export class MacrosParser {
*/
static #descriptions = new Map();
/**
* Logs a deprecation warning for MacrosParser APIs, pointing callers to
* the new macro engine registration surface.
*
* @param {string} method
* @param {string} replacement
* @returns {void}
*/
static #logDeprecated(method, replacement) {
console.warn(`[DEPRECATED] MacrosParser.${method} is deprecated and will be removed in a future version. Use ${replacement} instead.`);
}
/**
* Bridges a legacy MacrosParser macro registration into the new macro
* engine when the experimental macro engine flag is enabled.
*
* This mirrors the simple "{{key}}" replacement behavior by registering
* a 0-arg macro in MacroRegistry that does not take arguments and returns
* the sanitized value from the legacy registry.
*
* @param {string} key
* @param {string|MacroFunction} value
* @param {string} description
* @returns {void}
*/
static #registerMacroInNewEngine(key, value, description) {
if (!power_user.experimental_macro_engine) {
return;
}
// Like the old MacrosParser, we explicitly allow overriding macros, and only warn
if (macroSystem.registry.hasMacro(key)) {
console.warn(`Macro ${key} is already registered`);
}
const legacyValue = value;
macroSystem.registry.registerMacro(key, {
// Legacy MacrosParser macros never took arguments; keep the
// contract that only {{key}} without arguments is valid.
category: 'legacy',
description: typeof description === 'string' ? description : 'Automatically registered macro from MacrosParser',
handler: () => {
/** @type {string|MacroFunction|undefined} */
let stored = legacyValue;
if (typeof stored === 'function') {
try {
const nonce = uuidv4();
stored = stored(nonce);
} catch (e) {
console.warn(`Macro "${key}" function threw an error.`, e);
stored = '';
}
}
// Let the new macro engine's normalizeMacroResult handle type
// normalization for the returned value.
return stored;
},
});
}
/**
* Bridges a legacy MacrosParser macro unregistration into the new macro
* engine when the experimental macro engine flag is enabled.
*
* @param {string} key
* @returns {void}
*/
static #unregisterMacroInNewEngine(key) {
if (!power_user.experimental_macro_engine) {
return;
}
macroSystem.registry.unregisterMacro(key);
}
/**
* Returns an iterator over all registered macros.
* @returns {IterableIterator<CustomMacro>}
*/
static [Symbol.iterator] = function* () {
// When experimental macro engine is active, yield from the new registry
if (power_user.experimental_macro_engine) {
// Exclude hidden aliases for consistency with autocomplete behavior
for (const def of macroSystem.registry.getAllMacros({ excludeHiddenAliases: true })) {
yield { key: def.name, description: def.description || '' };
}
return;
}
for (const macro of MacrosParser.#macros.keys()) {
yield { key: macro, description: MacrosParser.#descriptions.get(macro) };
}
@@ -62,6 +155,7 @@ export class MacrosParser {
* @returns {string|MacroFunction|undefined} The macro value
*/
static get(key) {
MacrosParser.#logDeprecated('get', 'macros.registry.getMacro (from scripts/macros/macro-system.js)');
return MacrosParser.#macros.get(key);
}
@@ -71,6 +165,11 @@ export class MacrosParser {
* @returns {boolean} True if the macro is registered, false otherwise
*/
static has(key) {
MacrosParser.#logDeprecated('has', 'macros.registry.hasMacro (from scripts/macros/macro-system.js)');
if (power_user.experimental_macro_engine) {
return macroSystem.registry.hasMacro(key);
}
return MacrosParser.#macros.has(key);
}
@@ -81,6 +180,7 @@ export class MacrosParser {
* @param {string} [description] Optional description of the macro
*/
static registerMacro(key, value, description = '') {
MacrosParser.#logDeprecated('registerMacro', 'macros.registry.registerMacro (from scripts/macros/macro-system.js) or substituteParams({ dynamicMacros })');
if (typeof key !== 'string') {
throw new Error('Macro key must be a string');
}
@@ -101,6 +201,11 @@ export class MacrosParser {
value = this.sanitizeMacroValue(value);
}
MacrosParser.#registerMacroInNewEngine(key, value, description);
if (power_user.experimental_macro_engine) {
return;
}
if (this.#macros.has(key)) {
console.warn(`Macro ${key} is already registered`);
}
@@ -118,6 +223,7 @@ export class MacrosParser {
* @param {string} key Macro name (key)
*/
static unregisterMacro(key) {
MacrosParser.#logDeprecated('unregisterMacro', 'macros.registry.unregisterMacro (from scripts/macros/macro-system.js)');
if (typeof key !== 'string') {
throw new Error('Macro key must be a string');
}
@@ -129,6 +235,11 @@ export class MacrosParser {
throw new Error('Macro key must not be empty or whitespace only');
}
if (power_user.experimental_macro_engine) {
MacrosParser.#unregisterMacroInNewEngine(key);
return;
}
const deleted = this.#macros.delete(key);
if (!deleted) {
@@ -597,21 +708,33 @@ export function evaluateMacros(content, env, postProcessFn) {
}
export function initMacros() {
function initLastGenerationType() {
let lastGenerationType = '';
// Only manually register those is new macro engine is not on. In the new one, they are already registered automatically
if (!power_user.experimental_macro_engine) {
function initLastGenerationType() {
let lastGenerationType = '';
MacrosParser.registerMacro('lastGenerationType', () => lastGenerationType);
MacrosParser.registerMacro('lastGenerationType',
() => lastGenerationType,
'Returns the type of the last generation (e.g., "normal", "swipe", "continue", "impersonate", "quiet").',
);
eventSource.on(event_types.GENERATION_STARTED, (type, _params, isDryRun) => {
if (isDryRun) return;
lastGenerationType = type || 'normal';
});
eventSource.on(event_types.GENERATION_STARTED, (type, _params, isDryRun) => {
if (isDryRun) return;
lastGenerationType = type || 'normal';
});
eventSource.on(event_types.CHAT_CHANGED, () => {
lastGenerationType = '';
});
eventSource.on(event_types.CHAT_CHANGED, () => {
lastGenerationType = '';
});
}
MacrosParser.registerMacro('isMobile',
() => String(isMobile()),
'Returns "true" if the user is on a mobile device, "false" otherwise.',
);
initLastGenerationType();
}
MacrosParser.registerMacro('isMobile', () => String(isMobile()));
initLastGenerationType();
// TODO: Needs to be moved once old macros are deprecated and removed
initRegisterMacros();
}
+680
View File
@@ -0,0 +1,680 @@
/**
* MacroBrowser - Dynamic documentation browser for macros.
* Similar to SlashCommandBrowser but for the macro system.
*/
import { MacroRegistry, MacroCategory } from './engine/MacroRegistry.js';
import { performFuzzySearch } from '../power-user.js';
/** @typedef {import('./engine/MacroRegistry.js').MacroDefinition} MacroDefinition */
/** @typedef {import('./engine/MacroRegistry.js').MacroValueType} MacroValueType */
/**
* Category display names and order for documentation.
* @type {Record<string, { label: string, order: number }>}
*/
const CATEGORY_CONFIG = {
[MacroCategory.NAMES]: { label: 'Names & Participants', order: 1 },
[MacroCategory.UTILITY]: { label: 'Utilities', order: 2 },
[MacroCategory.RANDOM]: { label: 'Randomization', order: 3 },
[MacroCategory.TIME]: { label: 'Date & Time', order: 4 },
[MacroCategory.VARIABLE]: { label: 'Variables', order: 5 },
[MacroCategory.STATE]: { label: 'Runtime State', order: 6 },
[MacroCategory.CHARACTER]: { label: 'Character Card & Persona Fields', order: 7 },
[MacroCategory.CHAT]: { label: 'Chat History & Messages', order: 8 },
[MacroCategory.PROMPTS]: { label: 'Prompt Templates', order: 9 },
[MacroCategory.MISC]: { label: 'Miscellaneous', order: 10 },
};
/**
* MacroBrowser class for displaying searchable macro documentation.
*/
export class MacroBrowser {
/** @type {Map<string, MacroDefinition[]>} */
macrosByCategory = new Map();
/** @type {HTMLElement} */
dom;
/** @type {HTMLInputElement} */
searchInput;
/** @type {HTMLElement} */
detailsPanel;
/** @type {Map<string, HTMLElement>} */
itemMap = new Map();
/** @type {boolean} */
isSorted = false;
/**
* Groups macros by category in registration order.
* Excludes hidden aliases from the list.
*/
#loadMacros() {
this.macrosByCategory.clear();
// Exclude hidden aliases - they won't show in the list
const allMacros = MacroRegistry.getAllMacros({ excludeHiddenAliases: true });
for (const macro of allMacros) {
const category = macro.category || MacroCategory.MISC;
if (!this.macrosByCategory.has(category)) {
this.macrosByCategory.set(category, []);
}
this.macrosByCategory.get(category).push(macro);
}
}
/**
* Sorts macros within each category alphabetically.
*/
#sortMacros() {
for (const [, macros] of this.macrosByCategory) {
macros.sort((a, b) => a.name.localeCompare(b.name));
}
}
/**
* Gets categories sorted by their configured order.
* @returns {string[]}
*/
#getSortedCategories() {
return Array.from(this.macrosByCategory.keys())
.sort((a, b) => getCategoryConfig(a).order - getCategoryConfig(b).order);
}
/**
* Renders the browser into a parent element.
* @param {HTMLElement} parent
* @returns {HTMLElement}
*/
renderInto(parent) {
this.#loadMacros();
const root = document.createElement('div');
root.classList.add('macroBrowser');
this.dom = root;
// Search bar and sort button
const toolbar = document.createElement('div');
toolbar.classList.add('macro-toolbar');
const searchLabel = document.createElement('label');
searchLabel.classList.add('macro-search-label');
searchLabel.textContent = 'Search: ';
const searchInput = document.createElement('input');
searchInput.type = 'search';
searchInput.classList.add('macro-search-input', 'text_pole');
searchInput.placeholder = 'Search macros by name or description...';
searchInput.addEventListener('input', () => this.#handleSearch(searchInput.value));
this.searchInput = searchInput;
searchLabel.appendChild(searchInput);
toolbar.appendChild(searchLabel);
const sortBtn = document.createElement('button');
sortBtn.classList.add('macro-sort-btn', 'menu_button');
sortBtn.innerHTML = '<i class="fa-solid fa-arrow-down-a-z"></i> Sort A-Z';
sortBtn.title = 'Sort macros alphabetically within each category';
sortBtn.addEventListener('click', () => this.#toggleSort());
toolbar.appendChild(sortBtn);
root.appendChild(toolbar);
// Container for list and details
const container = document.createElement('div');
container.classList.add('macro-container');
// Macro list
const listPanel = document.createElement('div');
listPanel.classList.add('macro-list-panel');
this.#renderList(listPanel);
container.appendChild(listPanel);
// Details panel
const detailsPanel = document.createElement('div');
detailsPanel.classList.add('macro-details-panel');
detailsPanel.innerHTML = '<div class="macro-details-placeholder">Select a macro to view details</div>';
this.detailsPanel = detailsPanel;
container.appendChild(detailsPanel);
root.appendChild(container);
parent.appendChild(root);
return root;
}
/**
* Renders the macro list grouped by category.
* @param {HTMLElement} listPanel
*/
#renderList(listPanel) {
listPanel.innerHTML = '';
this.itemMap.clear();
for (const category of this.#getSortedCategories()) {
const macros = this.macrosByCategory.get(category);
if (!macros || macros.length === 0) continue;
// Category header
const categoryHeader = document.createElement('div');
categoryHeader.classList.add('macro-category-header');
categoryHeader.textContent = getCategoryConfig(category).label;
categoryHeader.dataset.category = category;
listPanel.appendChild(categoryHeader);
// Macro items
for (const macro of macros) {
const item = renderMacroItem(macro);
item.addEventListener('click', () => this.#showDetails(macro, item));
this.itemMap.set(macro.name, item);
listPanel.appendChild(item);
}
}
}
/**
* Shows details for a selected macro.
* @param {MacroDefinition} macro
* @param {HTMLElement} item
*/
#showDetails(macro, item) {
// Clear previous selection
this.dom.querySelectorAll('.macro-item.selected').forEach(el => el.classList.remove('selected'));
item.classList.add('selected');
// Render details
this.detailsPanel.innerHTML = '';
this.detailsPanel.appendChild(renderMacroDetails(macro));
}
/**
* Handles search input using fuzzy search.
* @param {string} query
*/
#handleSearch(query) {
query = query.trim();
// Clear details on search
this.detailsPanel.innerHTML = '<div class="macro-details-placeholder">Select a macro to view details</div>';
this.dom.querySelectorAll('.macro-item.selected').forEach(el => el.classList.remove('selected'));
// If empty query, show all
if (!query) {
for (const item of this.itemMap.values()) {
item.classList.remove('isFiltered');
}
this.dom.querySelectorAll('.macro-category-header').forEach(h => h.classList.remove('isFiltered'));
return;
}
// Trim query of braces, as we don't have them in the macro names of the search definitions
query = query.replace(/[{}]/g, '');
// Build searchable data array from all macros
const allMacros = MacroRegistry.getAllMacros();
const searchData = allMacros.map(macro => ({
name: macro.name,
aliases: macro.aliases?.map(a => a.alias).join(' '),
description: macro.description || '',
category: getCategoryConfig(macro.category).label,
argNames: macro.unnamedArgDefs.map(d => d.name).join(' '),
argDescriptions: macro.unnamedArgDefs.map(d => d.description || '').join(' '),
}));
// Fuzzy search with weighted keys
const keys = [
{ name: 'name', weight: 10 },
{ name: 'aliases', weight: 1 }, // No need to rank those high, if they are important (visible) they have their own entry
{ name: 'description', weight: 5 },
{ name: 'category', weight: 3 },
{ name: 'argNames', weight: 2 },
{ name: 'argDescriptions', weight: 1 },
];
const results = performFuzzySearch('macro-browser', searchData, keys, query);
const matchedNames = new Set(results.map(r => r.item.name));
// Filter items based on fuzzy results
for (const [name, item] of this.itemMap) {
item.classList.toggle('isFiltered', !matchedNames.has(name));
}
// Hide empty category headers
this.dom.querySelectorAll('.macro-category-header').forEach(header => {
if (!(header instanceof HTMLElement)) return;
const category = header.dataset.category;
const hasVisible = Array.from(this.itemMap.values())
.filter(item => item.dataset.macroName)
.some(item => {
const macro = MacroRegistry.getMacro(item.dataset.macroName);
return macro?.category === category && !item.classList.contains('isFiltered');
});
header.classList.toggle('isFiltered', !hasVisible);
});
}
/**
* Toggles alphabetical sorting.
*/
#toggleSort() {
this.isSorted = !this.isSorted;
if (this.isSorted) {
this.#sortMacros();
} else {
this.#loadMacros(); // Reload to restore registration order
}
const listPanel = this.dom.querySelector('.macro-list-panel');
if (!(listPanel instanceof HTMLElement)) return;
this.#renderList(listPanel);
// Re-apply current search filter
if (this.searchInput?.value) {
this.#handleSearch(this.searchInput.value);
}
// Update button state
const sortBtn = this.dom.querySelector('.macro-sort-btn');
sortBtn?.classList.toggle('active', this.isSorted);
}
/**
* Handles keyboard shortcuts.
* @param {KeyboardEvent} evt
*/
#handleKeyDown(evt) {
if (!evt.shiftKey && !evt.altKey && evt.ctrlKey && evt.key.toLowerCase() === 'f') {
if (!this.dom.closest('body')) return;
if (this.dom.closest('.mes') && !this.dom.closest('.last_mes')) return;
evt.preventDefault();
evt.stopPropagation();
evt.stopImmediatePropagation();
this.searchInput?.focus();
}
}
}
/**
* Gets the macro help content.
* If experimental_macro_engine is enabled, returns a placeholder for the browser.
* Otherwise returns the static template content.
*
* @returns {string} HTML string for help content
*/
export function getMacrosHelp() {
// Return a placeholder that will be replaced with the browser
return '<div class="macroHelp"><i class="fa-solid fa-spinner fa-spin"></i> Loading macro documentation...</div>';
}
/**
* Gets display config for a category.
* @param {string} category
* @returns {{ label: string, order: number }}
*/
function getCategoryConfig(category) {
return CATEGORY_CONFIG[category] ?? { label: category, order: 100 };
}
/**
* Formats a macro signature with its arguments.
* Uses displayOverride if available, otherwise auto-generates from args.
* Optional args are shown in [brackets].
* @param {MacroDefinition} macro
* @returns {string}
*/
export function formatMacroSignature(macro) {
// Use displayOverride if provided
if (macro.displayOverride) {
return macro.displayOverride;
}
const parts = [macro.name];
// Add all unnamed args (required + optional)
for (let i = 0; i < macro.unnamedArgDefs.length; i++) {
const argDef = macro.unnamedArgDefs[i];
const argName = argDef?.sampleValue || argDef?.name || `arg${i + 1}`;
// Wrap optional args in brackets
parts.push(argDef?.optional ? `[${argName}]` : argName);
}
// Add list args indicator
if (macro.list) {
const hasMin = macro.list.min > 0;
const hasMax = macro.list.max !== null;
if (hasMin && hasMax && macro.list.min === macro.list.max) {
// Fixed number of list items
for (let i = 0; i < macro.list.min; i++) {
parts.push(`item${i + 1}`);
}
} else {
// Variable list
parts.push('item1', 'item2', '...');
}
}
return `{{${parts.join('::')}}}`;
}
/**
* Creates a DOM element for a macro's source indicator (extension/third-party icons).
* @param {MacroDefinition} macro
* @returns {HTMLElement}
*/
export function createSourceIndicator(macro) {
const src = document.createElement('span');
src.classList.add('macro-source', 'fa-solid');
if (macro.source.isExtension) {
src.classList.add('isExtension', 'fa-cubes');
src.classList.add(macro.source.isThirdParty ? 'isThirdParty' : 'isCore');
} else {
src.classList.add('isCore', 'fa-star-of-life');
}
const titleParts = [
macro.source.isExtension ? 'Extension' : 'Core',
macro.source.isThirdParty ? 'Third Party' : (macro.source.isExtension ? 'Built-in' : null),
macro.source.name,
].filter(Boolean);
src.title = titleParts.join('\n');
return src;
}
/**
* Creates a DOM element for alias indicator icon.
* @param {MacroDefinition} macro
* @returns {HTMLElement|null}
*/
export function createAliasIndicator(macro) {
if (!macro.aliasOf) return null;
const icon = document.createElement('span');
icon.classList.add('macro-alias-indicator', 'fa-solid', 'fa-arrow-turn-up');
icon.title = `Alias of {{${macro.aliasOf}}}`;
return icon;
}
/**
* Creates a type badge element. Supports single type or array of types.
* @param {MacroValueType|MacroValueType[]} type - Single type or array of accepted types.
* @returns {HTMLElement}
*/
export function createTypeBadge(type) {
const badge = document.createElement('span');
badge.classList.add('macro-arg-type');
if (Array.isArray(type)) {
badge.textContent = type.join(' | ');
badge.title = `Accepts: ${type.join(', ')}`;
} else {
badge.textContent = type;
}
return badge;
}
/**
* Renders a single macro item for the list.
* Order: [signature] [description (shrinks)] [alias icon?] [source icon]
* @param {MacroDefinition} macro
* @returns {HTMLElement}
*/
function renderMacroItem(macro) {
const item = document.createElement('div');
item.classList.add('macro-item');
if (macro.aliasOf) item.classList.add('isAlias');
item.dataset.macroName = macro.name;
// Signature (fixed width, truncates if too long)
const signature = document.createElement('code');
signature.classList.add('macro-signature');
signature.textContent = formatMacroSignature(macro);
item.appendChild(signature);
// Description preview (shrinks to fit, truncates)
const desc = document.createElement('span');
desc.classList.add('macro-desc-preview');
desc.textContent = macro.description || '<no description>';
item.appendChild(desc);
// Alias indicator (if this is an alias entry)
const aliasIcon = createAliasIndicator(macro);
if (aliasIcon) item.appendChild(aliasIcon);
// Source indicator (fixed, stays at right edge)
item.appendChild(createSourceIndicator(macro));
return item;
}
/**
* Renders detailed information for a macro.
* Can optionally highlight the current argument being typed.
* @param {MacroDefinition} macro
* @param {Object} [options]
* @param {number} [options.currentArgIndex=-1] - Index of argument to highlight (-1 for none).
* @param {boolean} [options.showCategory=true] - Whether to show category badge.
* @returns {HTMLElement}
*/
export function renderMacroDetails(macro, options = {}) {
const { currentArgIndex = -1, showCategory = true } = options;
const details = document.createElement('div');
details.classList.add('macro-details');
// Header with name and source
const header = document.createElement('div');
header.classList.add('macro-details-header');
const nameEl = document.createElement('code');
nameEl.classList.add('macro-details-name');
nameEl.textContent = formatMacroSignature(macro);
header.appendChild(nameEl);
header.appendChild(createSourceIndicator(macro));
details.appendChild(header);
// Category badge (optional)
if (showCategory) {
const categoryBadge = document.createElement('span');
categoryBadge.classList.add('macro-category-badge');
categoryBadge.textContent = getCategoryConfig(macro.category).label;
details.appendChild(categoryBadge);
}
// If this is an alias, show what it's an alias of
if (macro.aliasOf) {
const aliasOfSection = document.createElement('div');
aliasOfSection.classList.add('macro-alias-of');
aliasOfSection.innerHTML = `<i class="fa-solid fa-arrow-turn-up"></i> Alias of <code>{{${macro.aliasOf}}}</code>`;
details.appendChild(aliasOfSection);
}
// Description
const descSection = document.createElement('div');
descSection.classList.add('macro-details-section');
const descLabel = document.createElement('div');
descLabel.classList.add('macro-details-label');
descLabel.textContent = 'Description';
descSection.appendChild(descLabel);
const descText = document.createElement('div');
descText.classList.add('macro-details-text');
descText.textContent = macro.description || '<no description>';
descSection.appendChild(descText);
details.appendChild(descSection);
// Arguments section (if any)
if (macro.unnamedArgDefs.length > 0 || macro.list) {
const argsSection = document.createElement('div');
argsSection.classList.add('macro-details-section');
const argsLabel = document.createElement('div');
argsLabel.classList.add('macro-details-label');
argsLabel.textContent = 'Arguments';
argsSection.appendChild(argsLabel);
const argsList = document.createElement('ul');
argsList.classList.add('macro-args-list');
// Unnamed args (required + optional)
for (let i = 0; i < macro.unnamedArgDefs.length; i++) {
const argDef = macro.unnamedArgDefs[i];
const argItem = document.createElement('li');
argItem.classList.add('macro-arg-item');
if (argDef?.optional) argItem.classList.add('isOptional');
if (currentArgIndex === i) argItem.classList.add('current');
const argName = document.createElement('code');
argName.classList.add('macro-arg-name');
argName.textContent = argDef?.name || `arg${i + 1}`;
argItem.appendChild(argName);
argItem.appendChild(createTypeBadge(argDef.type ?? 'string'));
const argRequiredLabel = document.createElement('span');
argRequiredLabel.classList.add(argDef?.optional ? 'macro-arg-optional' : 'macro-arg-required');
if (argDef?.optional && argDef.defaultValue !== undefined) {
argRequiredLabel.textContent = `(optional, default: ${argDef.defaultValue === '' ? '<empty string>' : argDef.defaultValue})`;
} else {
argRequiredLabel.textContent = argDef?.optional ? '(optional)' : '(required)';
}
argItem.appendChild(argRequiredLabel);
if (argDef?.description) {
const argDesc = document.createElement('span');
argDesc.classList.add('macro-arg-desc');
argDesc.textContent = `${argDef.description}`;
argItem.appendChild(argDesc);
}
if (argDef?.sampleValue) {
const sample = document.createElement('span');
sample.classList.add('macro-arg-sample');
sample.textContent = ` (e.g. ${argDef.sampleValue})`;
argItem.appendChild(sample);
}
argsList.appendChild(argItem);
}
// List args
if (macro.list) {
const listItem = document.createElement('li');
listItem.classList.add('macro-arg-item', 'macro-arg-list');
if (currentArgIndex >= macro.maxArgs) listItem.classList.add('current');
const listName = document.createElement('code');
listName.classList.add('macro-arg-name');
listName.textContent = 'item1::item2::...';
listItem.appendChild(listName);
const listInfo = document.createElement('span');
listInfo.classList.add('macro-arg-list-info');
const minMax = [];
if (macro.list.min > 0) minMax.push(`min: ${macro.list.min}`);
if (macro.list.max !== null) minMax.push(`max: ${macro.list.max}`);
if (minMax.length > 0) {
listInfo.textContent = ` (list, ${minMax.join(', ')})`;
} else {
listInfo.textContent = ' (variable-length list)';
}
listItem.appendChild(listInfo);
argsList.appendChild(listItem);
}
argsSection.appendChild(argsList);
details.appendChild(argsSection);
}
// Returns section (always show - at minimum shows the type)
{
const returnsSection = document.createElement('div');
returnsSection.classList.add('macro-details-section');
const returnsLabel = document.createElement('div');
returnsLabel.classList.add('macro-details-label');
returnsLabel.textContent = 'Returns';
returnsSection.appendChild(returnsLabel);
const returnsContent = document.createElement('div');
returnsContent.classList.add('macro-returns-content');
// Add return type badge
const returnTypeBadge = createTypeBadge(macro.returnType);
returnsContent.appendChild(returnTypeBadge);
// Add description text if provided
if (macro.returns) {
const returnsText = document.createElement('span');
returnsText.classList.add('macro-details-text');
returnsText.textContent = macro.returns;
returnsContent.appendChild(returnsText);
}
returnsSection.appendChild(returnsContent);
details.appendChild(returnsSection);
}
// Example usage section (if any)
if (macro.exampleUsage && macro.exampleUsage.length > 0) {
const exampleSection = document.createElement('div');
exampleSection.classList.add('macro-details-section');
const exampleLabel = document.createElement('div');
exampleLabel.classList.add('macro-details-label');
exampleLabel.textContent = 'Example Usage';
exampleSection.appendChild(exampleLabel);
const exampleList = document.createElement('ul');
exampleList.classList.add('macro-example-list');
for (const example of macro.exampleUsage) {
const li = document.createElement('li');
const code = document.createElement('code');
code.textContent = example;
li.appendChild(code);
exampleList.appendChild(li);
}
exampleSection.appendChild(exampleList);
details.appendChild(exampleSection);
}
// Aliases section (if this macro has aliases)
if (macro.aliases && macro.aliases.length > 0) {
const aliasSection = document.createElement('div');
aliasSection.classList.add('macro-details-section');
const aliasLabel = document.createElement('div');
aliasLabel.classList.add('macro-details-label');
aliasLabel.textContent = 'Aliases';
aliasSection.appendChild(aliasLabel);
const aliasList = document.createElement('ul');
aliasList.classList.add('macro-alias-list');
for (const { alias, visible } of macro.aliases) {
const li = document.createElement('li');
li.classList.add('macro-alias-item');
if (!visible) li.classList.add('isHidden');
const code = document.createElement('code');
code.textContent = `{{${alias}}}`;
li.appendChild(code);
if (!visible) {
const hiddenBadge = document.createElement('span');
hiddenBadge.classList.add('macro-alias-hidden-badge');
hiddenBadge.textContent = '(deprecated)';
hiddenBadge.title = 'This alias is deprecated and will not be shown in documentation or autocomplete';
li.appendChild(hiddenBadge);
}
aliasList.appendChild(li);
}
aliasSection.appendChild(aliasList);
details.appendChild(aliasSection);
}
return details;
}
@@ -0,0 +1,136 @@
import { MacroRegistry, MacroCategory, MacroValueType } from '../engine/MacroRegistry.js';
import { chat, chat_metadata } from '../../../script.js';
/**
* Registers macros that inspect the current chat log and swipe state
* (message texts, indices, swipes, and context boundaries).
*/
export function registerChatMacros() {
MacroRegistry.registerMacro('lastMessage', {
category: MacroCategory.CHAT,
description: 'Last message in the chat.',
returns: 'Last message in the chat.',
handler: () => String(getLastMessage() ?? ''),
});
MacroRegistry.registerMacro('lastMessageId', {
category: MacroCategory.CHAT,
description: 'Index of the last message in the chat.',
returns: 'Index of the last message in the chat.',
returnType: MacroValueType.INTEGER,
handler: () => String(getLastMessageId() ?? ''),
});
MacroRegistry.registerMacro('lastUserMessage', {
category: MacroCategory.CHAT,
description: 'Last user message in the chat.',
returns: 'Last user message in the chat.',
handler: () => String(getLastUserMessage() ?? ''),
});
MacroRegistry.registerMacro('lastCharMessage', {
category: MacroCategory.CHAT,
description: 'Last character/bot message in the chat.',
returns: 'Last character/bot message in the chat.',
handler: () => String(getLastCharMessage() ?? ''),
});
MacroRegistry.registerMacro('firstIncludedMessageId', {
category: MacroCategory.CHAT,
description: 'Index of the first message included in the current context.',
returns: 'Index of the first message included in the context.',
returnType: MacroValueType.INTEGER,
handler: () => String(getFirstIncludedMessageId() ?? ''),
});
MacroRegistry.registerMacro('firstDisplayedMessageId', {
category: MacroCategory.CHAT,
description: 'Index of the first displayed message in the chat.',
returns: 'Index of the first displayed message in the chat.',
returnType: MacroValueType.INTEGER,
handler: () => String(getFirstDisplayedMessageId() ?? ''),
});
MacroRegistry.registerMacro('lastSwipeId', {
category: MacroCategory.CHAT,
description: '1-based index of the last swipe for the last message.',
returns: '1-based index of the last swipe.',
returnType: MacroValueType.INTEGER,
handler: () => String(getLastSwipeId() ?? ''),
});
MacroRegistry.registerMacro('currentSwipeId', {
category: MacroCategory.CHAT,
description: '1-based index of the current swipe.',
returns: '1-based index of the current swipe.',
returnType: MacroValueType.INTEGER,
handler: () => String(getCurrentSwipeId() ?? ''),
});
}
function getLastMessageId({ exclude_swipe_in_propress = true, filter = null } = {}) {
if (!Array.isArray(chat) || chat.length === 0) {
return null;
}
for (let i = chat.length - 1; i >= 0; i--) {
const message = chat[i];
if (exclude_swipe_in_propress && message.swipes && message.swipe_id >= message.swipes.length) {
continue;
}
if (!filter || filter(message)) {
return i;
}
}
return null;
}
function getLastMessage() {
const mid = getLastMessageId();
return typeof mid === 'number' ? (chat[mid]?.mes ?? '') : '';
}
function getLastUserMessage() {
const mid = getLastMessageId({ filter: m => m.is_user && !m.is_system });
return typeof mid === 'number' ? (chat[mid]?.mes ?? '') : '';
}
function getLastCharMessage() {
const mid = getLastMessageId({ filter: m => !m.is_user && !m.is_system });
return typeof mid === 'number' ? (chat[mid]?.mes ?? '') : '';
}
function getFirstIncludedMessageId() {
const value = chat_metadata['lastInContextMessageId'];
return typeof value === 'number' ? value : null;
}
function getFirstDisplayedMessageId() {
const mesElement = document.querySelector('#chat .mes');
const mesId = Number(mesElement?.getAttribute('mesid'));
if (!Number.isNaN(mesId) && mesId >= 0) {
return mesId;
}
return null;
}
function getLastSwipeId() {
const mid = getLastMessageId({ exclude_swipe_in_propress: false });
if (typeof mid !== 'number') {
return null;
}
const swipes = chat[mid]?.swipes;
return Array.isArray(swipes) ? swipes.length : null;
}
function getCurrentSwipeId() {
const mid = getLastMessageId({ exclude_swipe_in_propress: false });
if (typeof mid !== 'number') {
return null;
}
const swipeId = chat[mid]?.swipe_id;
return typeof swipeId === 'number' ? swipeId + 1 : null;
}
@@ -0,0 +1,280 @@
import { seedrandom, droll } from '../../../lib.js';
import { chat_metadata, main_api, getMaxContextSize, extension_prompts, getCurrentChatId } from '../../../script.js';
import { getStringHash } from '../../utils.js';
import { textgenerationwebui_banned_in_macros } from '../../textgen-settings.js';
import { inject_ids } from '../../constants.js';
import { MacroRegistry, MacroCategory, MacroValueType } from '../engine/MacroRegistry.js';
/**
* Registers SillyTavern's core built-in macros in the MacroRegistry.
*
* These macros correspond to the main {{...}} macros that are available
* in prompts (time/date/chat info, utility macros, etc.). They are
* intended to preserve the behavior of the existing regex-based macros
* in macros.js while using the new MacroRegistry/MacroEngine pipeline.
*/
export function registerCoreMacros() {
// {{space}} -> ' '
MacroRegistry.registerMacro('space', {
category: MacroCategory.UTILITY,
unnamedArgs: [
{
name: 'count',
optional: true,
defaultValue: '1',
type: MacroValueType.INTEGER,
description: 'Number of spaces to insert.',
},
],
description: 'Returns one or more spaces. One space by default, more if the count argument is specified.',
returns: 'One or more spaces.',
exampleUsage: ['{{space}}', '{{space::4}}'],
handler: ({ unnamedArgs: [count] }) => ' '.repeat(Number(count ?? 1)),
});
// {{newline}} -> '\n'
MacroRegistry.registerMacro('newline', {
category: MacroCategory.UTILITY,
unnamedArgs: [
{
name: 'count',
optional: true,
defaultValue: '1',
type: MacroValueType.INTEGER,
description: 'Number of newlines to insert.',
},
],
description: 'Inserts one or more newlines. One newline by default, more if the count argument is specified.',
returns: 'One or more \\n.',
exampleUsage: ['{{newline}}', '{{newline::2}}'],
handler: ({ unnamedArgs: [count] }) => '\n'.repeat(Number(count ?? 1)),
});
// {{noop}} -> ''
MacroRegistry.registerMacro('noop', {
category: MacroCategory.UTILITY,
description: 'Does nothing and produces an empty string.',
returns: '',
handler: () => '',
});
// {{trim}} -> macro will currently replace itself with itself. Trimming is handled in post-processing.
MacroRegistry.registerMacro('trim', {
category: MacroCategory.UTILITY,
description: 'Trims all whitespaces around the trim macro.',
returns: '',
handler: () => '{{trim}}',
});
// {{input}} -> current textarea content
MacroRegistry.registerMacro('input', {
category: MacroCategory.UTILITY,
description: 'Current text from the send textarea.',
returns: 'Current text from the send textarea.',
handler: () => (/** @type {HTMLTextAreaElement} */(document.querySelector('#send_textarea')))?.value ?? '',
});
// {{maxPrompt}} -> max context size
MacroRegistry.registerMacro('maxPrompt', {
category: MacroCategory.STATE,
description: 'Maximum prompt context size.',
returns: 'Maximum prompt context size.',
returnType: MacroValueType.INTEGER,
handler: () => String(getMaxContextSize()),
});
// String utilities
MacroRegistry.registerMacro('reverse', {
category: MacroCategory.UTILITY,
unnamedArgs: [
{
name: 'value',
type: MacroValueType.STRING,
description: 'The string to reverse.',
},
],
description: 'Reverses the characters of the argument provided.',
returns: 'Reversed string.',
exampleUsage: ['{{reverse::I am Lana}}'],
handler: ({ unnamedArgs: [value] }) => Array.from(value).reverse().join(''),
});
// Comment macro: {{// ...}} -> '' (consumes any arguments)
MacroRegistry.registerMacro('//', {
aliases: [{ alias: 'comment' }],
category: MacroCategory.UTILITY,
list: true, // We consume any arguments as if this is a list, but we'll ignore them in the handler anyway
strictArgs: false, // and we also always remove it, even if the parsing might say it's invalid
description: 'Comment macro that produces an empty string. Can be used for writing into prompt definitions, without being passed to the context.',
returns: '',
displayOverride: '{{// ...}}',
exampleUsage: ['{{// This is a comment}}'],
handler: () => '',
});
// Time and date macros
// Dice roll macro: {{roll 1d6}} or {{roll: 1d6}}
MacroRegistry.registerMacro('roll', {
category: MacroCategory.RANDOM,
unnamedArgs: [
{
name: 'formula',
sampleValue: '1d20',
description: 'Dice roll formula using droll syntax (e.g. 1d20).',
type: 'string',
},
],
description: 'Rolls dice using droll syntax (e.g. {{roll 1d20}}).',
returns: 'Dice roll result.',
returnType: MacroValueType.INTEGER,
exampleUsage: [
'{{roll::1d20}}',
'{{roll::6}}',
'{{roll::3d6+4}}',
],
handler: ({ unnamedArgs: [formula] }) => {
// If only digits were provided, treat it as `1dX`.
if (/^\d+$/.test(formula)) {
formula = `1d${formula}`;
}
const isValid = droll.validate(formula);
if (!isValid) {
console.debug(`Invalid roll formula: ${formula}`);
return '';
}
const result = droll.roll(formula);
if (result === false) return '';
return String(result.total);
},
});
// Random choice macro: {{random::a::b}} or {{random a,b}}
MacroRegistry.registerMacro('random', {
category: MacroCategory.RANDOM,
list: true,
description: 'Picks a random item from a list. Will be re-rolled every time macros are resolved.',
returns: 'Randomly selected item from the list.',
exampleUsage: ['{{random::blonde::brown::red::black::blue}}'],
handler: ({ list }) => {
// Handle old legacy cases, where we have to split the list manually
if (list.length === 1) {
list = readSingleArgsRandomList(list[0]);
}
if (list.length === 0) {
return '';
}
const rng = seedrandom('added entropy.', { entropy: true });
const randomIndex = Math.floor(rng() * list.length);
return list[randomIndex];
},
});
// Deterministic choice macro: {{pick::a::b}} or {{pick a,b}}
MacroRegistry.registerMacro('pick', {
category: MacroCategory.RANDOM,
list: true,
description: 'Picks a random item from a list, but keeps the choice stable for a given chat and macro position.',
returns: 'Stable randomly selected item from the list.',
exampleUsage: ['{{pick::blonde::brown::red::black::blue}}'],
handler: ({ list, range, env }) => {
// Handle old legacy cases, where we have to split the list manually
if (list.length === 1) {
list = readSingleArgsRandomList(list[0]);
}
if (!list.length) {
return '';
}
const chatIdHash = getChatIdHash();
// Use the full original input string for deterministic behavior
const rawContentHash = getStringHash(env.content);
const offset = typeof range?.startOffset === 'number' ? range.startOffset : 0;
const combinedSeedString = `${chatIdHash}-${rawContentHash}-${offset}`;
const finalSeed = getStringHash(combinedSeedString);
const rng = seedrandom(String(finalSeed));
const randomIndex = Math.floor(rng() * list.length);
return list[randomIndex];
},
});
/** @param {string} listString @return {string[]} */
function readSingleArgsRandomList(listString) {
// If it contains double colons, those will have precedence over comma-seperated lists.
// This can only happen if the macro only had a single colon to introduce the list...
// like, {{random:a::b::c}}
if (listString.includes('::')) {
return listString.split('::').map((/** @type {string} */ item) => item.trim());
}
// Otherwise, we fall back and split by commas that may be present
return listString
.replace(/\\,/g, '##COMMA##')
.split(',')
.map((/** @type {string} */ item) => item.trim().replace(/##COMMA##/g, ','));
}
// Banned words macro: {{banned "word"}}
MacroRegistry.registerMacro('banned', {
category: MacroCategory.UTILITY,
unnamedArgs: [
{
name: 'word',
sampleValue: 'word',
description: 'Word to ban for textgenerationwebui backend.',
type: 'string',
},
],
description: 'Bans a word for textgenerationwebui backend. (Strips quotes surrounding the banned word, if present)',
returns: '',
exampleUsage: ['{{banned::delve}}'],
handler: ({ unnamedArgs: [bannedWord] }) => {
// Strip quotes via regex, which were allowed in legacy syntax
bannedWord = bannedWord.replace(/^"|"$/g, '');
if (main_api === 'textgenerationwebui') {
console.log('Found banned word in macros: ' + bannedWord);
textgenerationwebui_banned_in_macros.push(bannedWord);
}
return '';
},
});
// Outlet macro: {{outlet::key}}
MacroRegistry.registerMacro('outlet', {
category: MacroCategory.UTILITY,
unnamedArgs: [
{
name: 'key',
sampleValue: 'my-outlet-key',
description: 'Outlet key.',
type: 'string',
},
],
description: 'Returns the world info outlet prompt for a given outlet key.',
returns: 'World info outlet prompt.',
exampleUsage: ['{{outlet::character-achievements}}'],
handler: ({ unnamedArgs: [outlet] }) => {
if (!outlet) return '';
const value = extension_prompts[inject_ids.CUSTOM_WI_OUTLET(outlet)]?.value;
return value || '';
},
});
}
function getChatIdHash() {
const cachedIdHash = chat_metadata['chat_id_hash'];
if (typeof cachedIdHash === 'number') {
return cachedIdHash;
}
const chatId = chat_metadata['main_chat'] ?? getCurrentChatId();
const chatIdHash = getStringHash(chatId);
chat_metadata['chat_id_hash'] = chatIdHash;
return chatIdHash;
}
@@ -0,0 +1,192 @@
import { MacroRegistry, MacroCategory, MacroValueType } from '../engine/MacroRegistry.js';
import { isMobile } from '../../RossAscends-mods.js';
import { parseMesExamples, main_api } from '../../../script.js';
import { power_user } from '../../power-user.js';
import { formatInstructModeExamples } from '../../instruct-mode.js';
/** @typedef {import('../engine/MacroEnv.types.js').MacroEnv} MacroEnv */
/**
* Registers macros that mostly act as simple accessors to MacroEnv fields
* (names, character card fields, system metadata, extras) or basic
* environment flags.
*/
export function registerEnvMacros() {
// Names and participant macros (from MacroEnv.names)
MacroRegistry.registerMacro('user', {
category: MacroCategory.NAMES,
description: 'Your current Persona username.',
returns: 'Persona username.',
handler: ({ env }) => env.names.user,
});
MacroRegistry.registerMacro('char', {
category: MacroCategory.NAMES,
description: 'The character\'s name.',
returns: 'Character name.',
handler: ({ env }) => env.names.char,
});
MacroRegistry.registerMacro('group', {
aliases: [{ alias: 'charIfNotGroup', visible: false }],
category: MacroCategory.NAMES,
description: 'Comma-separated list of group member names (including muted) or the character name in solo chats.',
returns: 'List of group member names.',
handler: ({ env }) => env.names.group ?? '',
});
MacroRegistry.registerMacro('groupNotMuted', {
category: MacroCategory.NAMES,
description: 'Comma-separated list of group member names excluding muted members.',
returns: 'List of group member names excluding muted members.',
handler: ({ env }) => env.names.groupNotMuted ?? '',
});
MacroRegistry.registerMacro('notChar', {
category: MacroCategory.NAMES,
description: 'Comma-separated list of all participants except the current speaker.',
returns: 'List of all participants except the current speaker.',
handler: ({ env }) => env.names.notChar ?? '',
});
// Character card field macros (from MacroEnv.character)
MacroRegistry.registerMacro('charPrompt', {
category: MacroCategory.CHARACTER,
description: 'The character\'s Main Prompt override.',
returns: 'Character Main Prompt override.',
handler: ({ env }) => env.character.charPrompt ?? '',
});
MacroRegistry.registerMacro('charInstruction', {
category: MacroCategory.CHARACTER,
description: 'The character\'s Post-History Instructions override.',
returns: 'Character Post-History Instructions override.',
handler: ({ env }) => env.character.charInstruction ?? '',
});
MacroRegistry.registerMacro('charDescription', {
aliases: [{ alias: 'description' }],
category: MacroCategory.CHARACTER,
description: 'The character\'s description.',
returns: 'Character description.',
handler: ({ env }) => env.character.description ?? '',
});
MacroRegistry.registerMacro('charPersonality', {
aliases: [{ alias: 'personality' }],
category: MacroCategory.CHARACTER,
description: 'The character\'s personality.',
returns: 'Character personality.',
handler: ({ env }) => env.character.personality ?? '',
});
MacroRegistry.registerMacro('charScenario', {
aliases: [{ alias: 'scenario' }],
category: MacroCategory.CHARACTER,
description: 'The character\'s scenario.',
returns: 'Character scenario.',
handler: ({ env }) => env.character.scenario ?? '',
});
MacroRegistry.registerMacro('persona', {
category: MacroCategory.CHARACTER,
description: 'Your current Persona description.',
returns: 'Persona description.',
handler: ({ env }) => env.character.persona ?? '',
});
MacroRegistry.registerMacro('mesExamplesRaw', {
category: MacroCategory.CHARACTER,
description: 'Unformatted dialogue examples from the character card.',
returns: 'Unformatted dialogue examples.',
handler: ({ env }) => env.character.mesExamplesRaw ?? '',
});
MacroRegistry.registerMacro('mesExamples', {
category: MacroCategory.CHARACTER,
description: 'The character\'s dialogue examples, formatted for instruct mode when enabled.',
returns: 'Formatted dialogue examples.',
handler: ({ env }) => {
const raw = env.character.mesExamplesRaw ?? '';
if (!raw) return '';
const isInstruct = !!power_user?.instruct?.enabled && main_api !== 'openai';
const parsed = parseMesExamples(raw, isInstruct);
if (!Array.isArray(parsed) || parsed.length === 0) {
return '';
}
if (!isInstruct) {
return parsed.join('');
}
const formatted = formatInstructModeExamples(parsed, env.names.user, env.names.char);
return Array.isArray(formatted) ? formatted.join('') : '';
},
});
MacroRegistry.registerMacro('charDepthPrompt', {
category: MacroCategory.CHARACTER,
description: 'The character\'s @ Depth Note.',
returns: 'Character @ Depth Note.',
handler: ({ env }) => env.character.charDepthPrompt ?? '',
});
MacroRegistry.registerMacro('charCreatorNotes', {
aliases: [{ alias: 'creatorNotes' }],
category: MacroCategory.CHARACTER,
description: 'Creator notes from the character card.',
returns: 'Creator notes.',
handler: ({ env }) => env.character.creatorNotes ?? '',
});
// Character version macros (legacy variants and documented {{charVersion}})
MacroRegistry.registerMacro('charVersion', {
aliases: [
{ alias: 'version', visible: false }, // Legacy alias
{ alias: 'char_version', visible: false }, // Legacy underscore variant
],
category: MacroCategory.CHARACTER,
description: 'The character\'s version number.',
returns: 'Character version number.',
handler: ({ env }) => env.character.version ?? '',
});
// System / env extras macros (from MacroEnv.system / MacroEnv.extra)
MacroRegistry.registerMacro('model', {
category: MacroCategory.STATE,
description: 'Model name for the currently selected API (Chat Completion or Chat Completion).',
returns: 'Model name.',
handler: ({ env }) => env.system.model,
});
// TODO: Move this to the summary extension, where it belongs
MacroRegistry.registerMacro('summary', {
category: MacroCategory.CHAT,
description: 'Latest chat summary from the "Summarize" extension (when available).',
returns: 'Latest chat summary.',
handler: ({ env }) => {
const value = /** @type {any} */ (env.extra).summary;
return value == null ? '' : String(value);
},
});
MacroRegistry.registerMacro('original', {
category: MacroCategory.CHARACTER,
description: 'Original message content for {{original}} substitution in in character prompt overrides.',
returns: 'Original message content.',
handler: ({ env }) => {
const value = env.functions.original();
return value;
},
});
// Device / environment macros
MacroRegistry.registerMacro('isMobile', {
category: MacroCategory.STATE,
description: '"true" if currently running in a mobile environment, "false" otherwise.',
returns: 'Whether the environment is mobile.',
returnType: MacroValueType.BOOLEAN,
handler: () => String(isMobile()),
});
}
@@ -0,0 +1,76 @@
import { MacroRegistry, MacroCategory } from '../engine/MacroRegistry.js';
import { power_user } from '../../power-user.js';
/**
* Registers instruct-mode related {{...}} macros (instruct* and system
* prompt/context macros) in the MacroRegistry.
*/
export function registerInstructMacros() {
/**
* Helper to register macros that just expose a value from power_user.instruct.
* The first name is the primary, subsequent names become visible aliases.
* @param {string[]} names - First is primary, rest are aliases.
* @param {() => string} getValue
* @param {() => boolean} isEnabled
* @param {string} description
* @param {string} [category=MacroCategory.PROMPTS]
*/
function registerSimple(names, getValue, isEnabled, description, category = MacroCategory.PROMPTS) {
const [primary, ...aliasNames] = names;
const aliases = aliasNames.map(alias => ({ alias }));
MacroRegistry.registerMacro(primary, {
category,
description,
aliases: aliases.length > 0 ? aliases : undefined,
handler: () => (isEnabled() ? (getValue() ?? '') : ''),
});
}
const instEnabled = () => !!power_user.instruct.enabled;
const sysEnabled = () => !!power_user.sysprompt.enabled;
// Instruct template macros
registerSimple(['instructStoryStringPrefix'], () => power_user.instruct.story_string_prefix, instEnabled, 'Instruct story string prefix.');
registerSimple(['instructStoryStringSuffix'], () => power_user.instruct.story_string_suffix, instEnabled, 'Instruct story string suffix.');
registerSimple(['instructUserPrefix', 'instructInput'], () => power_user.instruct.input_sequence, instEnabled, 'Instruct input / user prefix sequence.');
registerSimple(['instructUserSuffix'], () => power_user.instruct.input_suffix, instEnabled, 'Instruct input / user suffix sequence.');
registerSimple(['instructAssistantPrefix', 'instructOutput'], () => power_user.instruct.output_sequence, instEnabled, 'Instruct output / assistant prefix sequence.');
registerSimple(['instructAssistantSuffix', 'instructSeparator'], () => power_user.instruct.output_suffix, instEnabled, 'Instruct output / assistant suffix sequence.');
registerSimple(['instructSystemPrefix'], () => power_user.instruct.system_sequence, instEnabled, 'Instruct system prefix sequence.');
registerSimple(['instructSystemSuffix'], () => power_user.instruct.system_suffix, instEnabled, 'Instruct system suffix sequence.');
registerSimple(['instructFirstAssistantPrefix', 'instructFirstOutputPrefix'], () => power_user.instruct.first_output_sequence || power_user.instruct.output_sequence, instEnabled, 'Instruct first assistant / output prefix sequence');
registerSimple(['instructLastAssistantPrefix', 'instructLastOutputPrefix'], () => power_user.instruct.last_output_sequence || power_user.instruct.output_sequence, instEnabled, 'Instruct last assistant / output prefix sequence.');
registerSimple(['instructStop'], () => power_user.instruct.stop_sequence, instEnabled, 'Instruct stop sequence.');
registerSimple(['instructUserFiller'], () => power_user.instruct.user_alignment_message, instEnabled, 'Instruct user alignment filler.');
registerSimple(['instructSystemInstructionPrefix'], () => power_user.instruct.last_system_sequence, instEnabled, 'Instruct system instruction prefix sequence.');
registerSimple(['instructFirstUserPrefix', 'instructFirstInput'], () => power_user.instruct.first_input_sequence || power_user.instruct.input_sequence, instEnabled, 'Instruct first user / input prefix sequence.');
registerSimple(['instructLastUserPrefix', 'instructLastInput'], () => power_user.instruct.last_input_sequence || power_user.instruct.input_sequence, instEnabled, 'Instruct last user / input prefix sequence.');
// System prompt macros
registerSimple(['defaultSystemPrompt', 'instructSystem', 'instructSystemPrompt'], () => power_user.sysprompt.content, sysEnabled, 'Default system prompt.');
MacroRegistry.registerMacro('systemPrompt', {
category: MacroCategory.PROMPTS,
description: 'Active system prompt text (optionally overridden by character prompt)',
handler: ({ env }) => {
const isEnabled = !!power_user.sysprompt.enabled;
if (!isEnabled) return '';
if (power_user.prefer_character_prompt && env.character.charPrompt) {
return env.character.charPrompt;
}
return power_user.sysprompt.content ?? '';
},
});
// Context template macros
registerSimple(['exampleSeparator', 'chatSeparator'], () => power_user.context.example_separator, () => true, 'Separator used between example chat blocks in text completion prompts.');
registerSimple(['chatStart'], () => power_user.context.chat_start, () => true, 'Chat start marker used in text completion prompts.');
}
@@ -0,0 +1,40 @@
import { MacroRegistry, MacroCategory } from '../engine/MacroRegistry.js';
import { eventSource, event_types } from '../../events.js';
let lastGenerationTypeValue = '';
let lastGenerationTypeTrackingInitialized = false;
function ensureLastGenerationTypeTracking() {
if (lastGenerationTypeTrackingInitialized) {
return;
}
lastGenerationTypeTrackingInitialized = true;
try {
eventSource?.on?.(event_types.GENERATION_STARTED, (type, _params, isDryRun) => {
if (isDryRun) return;
lastGenerationTypeValue = type || 'normal';
});
eventSource?.on?.(event_types.CHAT_CHANGED, () => {
lastGenerationTypeValue = '';
});
} catch {
// In non-runtime environments (tests), eventSource may be undefined or not fully initialized.
}
}
/**
* Registers macros that depend on runtime application state or event tracking
* rather than static environment fields.
*/
export function registerStateMacros() {
ensureLastGenerationTypeTracking();
MacroRegistry.registerMacro('lastGenerationType', {
category: MacroCategory.STATE,
description: 'Type of the last queued generation request (e.g. "normal", "impersonate", "regenerate", "quiet", "swipe", "continue"). Empty if none yet or chat was switched.',
returns: 'Type of the last queued generation request.',
handler: () => lastGenerationTypeValue,
});
}
@@ -0,0 +1,151 @@
import { moment } from '../../../lib.js';
import { chat } from '../../../script.js';
import { timestampToMoment } from '../../utils.js';
import { MacroRegistry, MacroCategory, MacroValueType } from '../engine/MacroRegistry.js';
/**
* Registers time/date related macros and utilities.
*/
export function registerTimeMacros() {
// Time and date macros
MacroRegistry.registerMacro('time', {
category: MacroCategory.TIME,
// Optional single list argument: UTC offset, e.g. {{time::UTC+2}}
unnamedArgs: [
{
name: 'offset',
optional: true,
defaultValue: 'null',
type: MacroValueType.STRING,
sampleValue: 'UTC+2',
description: 'UTC offset in the format UTC±(offset).',
},
],
description: 'Current local time, or UTC offset when called as {{time::UTC±(offset)}}',
returns: 'A time string in the format HH:mm.',
displayOverride: '{{time::[UTC±(offset)]}}',
exampleUsage: ['{{time}}', '{{time::UTC+2}}', '{{time::UTC-7}}'],
handler: ({ unnamedArgs: [offsetSpec] }) => {
if (!offsetSpec) return moment().format('LT');
const match = /^UTC([+-]\d+)$/.exec(offsetSpec);
if (!match) return moment().format('LT');
const offset = Number.parseInt(match[1], 10);
if (Number.isNaN(offset)) return moment().format('LT');
return moment().utc().utcOffset(offset).format('LT');
},
});
MacroRegistry.registerMacro('date', {
category: MacroCategory.TIME,
description: 'Current local date as a string in the local short format.',
returns: 'Current local date in local short format.',
handler: () => moment().format('LL'),
});
MacroRegistry.registerMacro('weekday', {
category: MacroCategory.TIME,
description: 'Current weekday name.',
returns: 'Current weekday name.',
handler: () => moment().format('dddd'),
});
MacroRegistry.registerMacro('isotime', {
category: MacroCategory.TIME,
description: 'Current time in HH:mm format.',
returns: 'Current time in HH:mm format.',
handler: () => moment().format('HH:mm'),
});
MacroRegistry.registerMacro('isodate', {
category: MacroCategory.TIME,
description: 'Current date in YYYY-MM-DD format.',
returns: 'Current date in YYYY-MM-DD format.',
handler: () => moment().format('YYYY-MM-DD'),
});
MacroRegistry.registerMacro('datetimeformat', {
category: MacroCategory.TIME,
unnamedArgs: [
{
name: 'format',
sampleValue: 'YYYY-MM-DD HH:mm:ss',
description: 'Moment.js format string.',
type: 'string',
},
],
description: 'Formats the current date/time using the given moment.js format string.',
returns: 'Formatted date/time string.',
exampleUsage: ['{{datetimeformat::YYYY-MM-DD HH:mm:ss}}', '{{datetimeformat::LLLL}}'],
handler: ({ unnamedArgs: [format] }) => moment().format(format),
});
MacroRegistry.registerMacro('idleDuration', {
aliases: [{ alias: 'idle_duration', visible: false }],
category: MacroCategory.TIME,
description: 'Human-readable duration since the last user message.',
returns: 'Human-readable duration since the last user message.',
handler: () => getTimeSinceLastMessage(),
});
// Time difference between two values
MacroRegistry.registerMacro('timeDiff', {
category: MacroCategory.TIME,
unnamedArgs: [
{
name: 'left',
sampleValue: '2023-01-01 12:00:00',
description: 'Left time value.',
type: 'string',
},
{
name: 'right',
sampleValue: '2023-01-01 15:00:00',
description: 'Right time value.',
type: 'string',
},
],
description: 'Human-readable difference between two times. Order of times does not matter, it will return the absolute difference.',
returns: 'Human-readable difference between two times.',
displayOverride: '{{timeDiff::left::right}}', // Shorten this, otherwise it's too long. Full dates don't really help for understanding the macro.
exampleUsage: ['{{ timeDiff :: 2023-01-01 12:00:00 :: 2023-01-01 15:00:00 }}'],
handler: ({ unnamedArgs: [left, right] }) => {
const diff = moment.duration(moment(left).diff(moment(right)));
return diff.humanize(true);
},
});
}
function getTimeSinceLastMessage() {
const now = moment();
if (Array.isArray(chat) && chat.length > 0) {
let lastMessage;
let takeNext = false;
for (let i = chat.length - 1; i >= 0; i--) {
const message = chat[i];
if (message.is_system) {
continue;
}
if (message.is_user && takeNext) {
lastMessage = message;
break;
}
takeNext = true;
}
if (lastMessage?.send_date) {
const lastMessageDate = timestampToMoment(lastMessage.send_date);
const duration = moment.duration(now.diff(lastMessageDate));
return duration.humanize();
}
}
return 'just now';
}
@@ -0,0 +1,224 @@
import { MacroRegistry, MacroCategory, MacroValueType } from '../engine/MacroRegistry.js';
/**
* Registers variable-related {{...}} macros that operate on local and global
* variables (e.g. {{setvar}}, {{getvar}}, {{incvar}}, etc.).
*/
export function registerVariableMacros() {
const ctx = SillyTavern.getContext();
// {{setvar::name::value}} -> '' (side-effect on local variable)
MacroRegistry.registerMacro('setvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the local variable to set.',
},
{
name: 'value',
type: [MacroValueType.STRING, MacroValueType.NUMBER],
description: 'The value to set the local variable to.',
},
],
description: 'Sets a local variable to the given value.',
returns: '',
exampleUsage: ['{{setvar::myvar::foo}}', '{{setvar::myintvar::3}}'],
handler: ({ unnamedArgs: [name, value] }) => {
ctx.variables.local.set(name, value);
return '';
},
});
// {{addvar::name::value}} -> '' (side-effect via addLocalVariable)
MacroRegistry.registerMacro('addvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the local variable to add to.',
},
{
name: 'value',
type: [MacroValueType.STRING, MacroValueType.NUMBER],
description: 'The value to add to the local variable.',
},
],
description: 'Adds a value to an existing local variable (numeric or string append). If the variable does not exist, it will be created.',
returns: '',
exampleUsage: ['{{addvar::mystrvar::foo}}', '{{addvar::myintvar::3}}'],
handler: ({ unnamedArgs: [name, value] }) => {
ctx.variables.local.add(name, value);
return '';
},
});
// {{incvar::name}} -> returns new value
MacroRegistry.registerMacro('incvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the local variable to increment.',
},
],
description: 'Increments a local variable by 1 and returns the new value. If the variable does not exist, it will be created.',
returns: 'The new value of the local variable.',
returnType: MacroValueType.NUMBER,
exampleUsage: ['{{incvar::myintvar}}'],
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.local.inc(name);
return normalize(result);
},
});
// {{decvar::name}} -> returns new value
MacroRegistry.registerMacro('decvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the local variable to decrement.',
},
],
description: 'Decrements a local variable by 1 and returns the new value. If the variable does not exist, it will be created.',
returns: 'The new value of the local variable.',
returnType: MacroValueType.NUMBER,
exampleUsage: ['{{decvar::myintvar}}'],
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.local.dec(name);
return normalize(result);
},
});
// {{getvar::name}} -> returns current value
MacroRegistry.registerMacro('getvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the local variable to get.',
},
],
description: 'Gets the value of a local variable.',
returns: 'The value of the local variable.',
returnType: [MacroValueType.STRING, MacroValueType.NUMBER],
exampleUsage: ['{{getvar::myvar}}', '{{getvar::myintvar}}'],
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.local.get(name);
return normalize(result);
},
});
// {{setglobalvar::name::value}} -> ''
MacroRegistry.registerMacro('setglobalvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the global variable to set.',
},
{
name: 'value',
type: [MacroValueType.STRING, MacroValueType.NUMBER],
description: 'The value to set the global variable to.',
},
],
description: 'Sets a global variable to the given value.',
returns: '',
exampleUsage: ['{{setglobalvar::myvar::foo}}', '{{setglobalvar::myintvar::3}}'],
handler: ({ unnamedArgs: [name, value] }) => {
ctx.variables.global.set(name, value);
return '';
},
});
// {{addglobalvar::name::value}} -> ''
MacroRegistry.registerMacro('addglobalvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the global variable to add to.',
},
{
name: 'value',
type: [MacroValueType.STRING, MacroValueType.NUMBER],
description: 'The value to add to the global variable.',
},
],
description: 'Adds a value to an existing global variable (numeric or string append). If the variable does not exist, it will be created.',
returns: '',
exampleUsage: ['{{addglobalvar::mystrvar::foo}}', '{{addglobalvar::myintvar::3}}'],
handler: ({ unnamedArgs: [name, value] }) => {
ctx.variables.global.add(name, value);
return '';
},
});
// {{incglobalvar::name}} -> returns new value
MacroRegistry.registerMacro('incglobalvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the global variable to increment.',
},
],
description: 'Increments a global variable by 1 and returns the new value. If the variable does not exist, it will be created.',
returns: 'The new value of the global variable.',
returnType: MacroValueType.NUMBER,
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.global.inc(name);
return normalize(result);
},
});
// {{decglobalvar::name}} -> returns new value
MacroRegistry.registerMacro('decglobalvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the global variable to decrement.',
},
],
description: 'Decrements a global variable by 1 and returns the new value. If the variable does not exist, it will be created.',
returns: 'The new value of the global variable.',
returnType: MacroValueType.NUMBER,
exampleUsage: ['{{decglobalvar::myintvar}}'],
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.global.dec(name);
return normalize(result);
},
});
// {{getglobalvar::name}} -> returns current value
MacroRegistry.registerMacro('getglobalvar', {
category: MacroCategory.VARIABLE,
unnamedArgs: [
{
name: 'name',
type: MacroValueType.STRING,
description: 'The name of the global variable to get.',
},
],
description: 'Gets the value of a global variable.',
returns: 'The value of the global variable.',
returnType: [MacroValueType.STRING, MacroValueType.NUMBER],
exampleUsage: ['{{getglobalvar::myvar}}', '{{getglobalvar::myintvar}}'],
handler: ({ unnamedArgs: [name], normalize }) => {
const result = ctx.variables.global.get(name);
return normalize(result);
},
});
}
@@ -0,0 +1,433 @@
/** @typedef {import('chevrotain').CstNode} CstNode */
/** @typedef {import('chevrotain').IToken} IToken */
/** @typedef {import('./MacroEnv.types.js').MacroEnv} MacroEnv */
/**
* @typedef {Object} MacroCall
* @property {string} name
* @property {string[]} args
* @property {MacroEnv} env
* @property {string} rawInner
* @property {string} rawWithBraces
* @property {{ startOffset: number, endOffset: number }} range
* @property {CstNode} cstNode
*/
/**
* @typedef {Object} EvaluationContext
* @property {string} text
* @property {MacroEnv} env
* @property {(call: MacroCall) => string} resolveMacro
*/
/**
* @typedef {Object} TokenRange
* @property {number} startOffset
* @property {number} endOffset
*/
/**
* The singleton instance of the MacroCstWalker.
*
* @type {MacroCstWalker}
*/
let instance;
export { instance as MacroCstWalker };
class MacroCstWalker {
/** @type {MacroCstWalker} */ static #instance;
/** @type {MacroCstWalker} */ static get instance() { return MacroCstWalker.#instance ?? (MacroCstWalker.#instance = new MacroCstWalker()); }
constructor() { }
/**
* Evaluates a full document CST into a resolved string.
*
* @param {EvaluationContext & { cst: CstNode }} options
* @returns {string}
*/
evaluateDocument(options) {
const { text, cst, env, resolveMacro } = options;
if (typeof text !== 'string') {
throw new Error('MacroCstWalker.evaluateDocument: text must be a string');
}
if (!cst || typeof cst !== 'object' || !cst.children) {
throw new Error('MacroCstWalker.evaluateDocument: cst must be a CstNode');
}
if (typeof resolveMacro !== 'function') {
throw new Error('MacroCstWalker.evaluateDocument: resolveMacro must be a function');
}
/** @type {EvaluationContext} */
const context = { text, env, resolveMacro };
const items = this.#collectDocumentItems(cst);
if (items.length === 0) {
return text;
}
let result = '';
let cursor = 0;
// Iterate over all items in the document. Evaluate any macro being found, and keep them in the exact same place.
for (const item of items) {
if (item.startOffset > cursor) {
result += text.slice(cursor, item.startOffset);
}
// Items can be either plaintext or macro nodes
if (item.type === 'plaintext') {
result += text.slice(item.startOffset, item.endOffset + 1);
} else {
result += this.#evaluateMacroNode(item.node, context);
}
cursor = item.endOffset + 1;
}
if (cursor < text.length) {
result += text.slice(cursor);
}
return result;
}
/** @typedef {{ type: 'plaintext', startOffset: number, endOffset: number, token: IToken }} DocumentItemPlaintext */
/** @typedef {{ type: 'macro', startOffset: number, endOffset: number, node: CstNode }} DocumentItemMacro */
/** @typedef {DocumentItemPlaintext | DocumentItemMacro} DocumentItem */
/**
* Collects top-level plaintext tokens and macro nodes from the document CST.
*
* @param {CstNode} cst
* @returns {Array<DocumentItem>}
*/
#collectDocumentItems(cst) {
const plaintextTokens = /** @type {IToken[]} */ (cst.children.plaintext || []);
const macroNodes = /** @type {CstNode[]} */ (cst.children.macro || []);
/** @type {Array<DocumentItem>} */
const items = [];
for (const token of plaintextTokens) {
if (typeof token.startOffset !== 'number' || typeof token.endOffset !== 'number') {
continue;
}
items.push({
type: 'plaintext',
startOffset: token.startOffset,
endOffset: token.endOffset,
token,
});
}
for (const macroNode of macroNodes) {
const children = macroNode.children || {};
const endToken = /** @type {IToken?} */ ((children['Macro.End'] || [])[0]);
// If the end token was inserted during error recovery, treat this macro as plaintext
if (this.#isRecoveryToken(endToken)) {
// Flatten the incomplete macro: collect its tokens as plaintext but keep nested macros
this.#flattenIncompleteMacro(macroNode, endToken, items);
continue;
}
const range = this.#getMacroRange(macroNode);
items.push({
type: 'macro',
startOffset: range.startOffset,
endOffset: range.endOffset,
node: macroNode,
});
}
items.sort((a, b) => {
if (a.startOffset !== b.startOffset) {
return a.startOffset - b.startOffset;
}
return a.endOffset - b.endOffset;
});
return items;
}
/**
* Evaluates a single macro CST node, resolving any nested macros first.
*
* @param {CstNode} macroNode
* @param {EvaluationContext} context
* @returns {string}
*/
#evaluateMacroNode(macroNode, context) {
const { text, env, resolveMacro } = context;
const children = macroNode.children || {};
const identifierTokens = /** @type {IToken[]} */ (children['Macro.identifier'] || []);
const name = identifierTokens[0]?.image || '';
const range = this.#getMacroRange(macroNode);
const startToken = /** @type {IToken?} */ ((children['Macro.Start'] || [])[0]);
const endToken = /** @type {IToken?} */ ((children['Macro.End'] || [])[0]);
const innerStart = startToken ? startToken.endOffset + 1 : range.startOffset;
const innerEnd = endToken ? endToken.startOffset - 1 : range.endOffset;
// Extract argument nodes from the "arguments" rule (if present)
const argumentsNode = /** @type {CstNode?} */ ((children.arguments || [])[0]);
const argumentNodes = /** @type {CstNode[]} */ (argumentsNode?.children?.argument || []);
/** @type {string[]} */
const args = [];
/** @type {({ value: string } & TokenRange)[]} */
const evaluatedArguments = [];
for (const argNode of argumentNodes) {
const argValue = this.#evaluateArgumentNode(argNode, context);
args.push(argValue);
const location = this.#getArgumentLocation(argNode);
if (location) {
evaluatedArguments.push({
value: argValue,
...location,
});
}
}
evaluatedArguments.sort((a, b) => a.startOffset - b.startOffset);
// Build the inner raw string between the braces, with nested macros resolved.
// This uses the already evaluated argument strings and preserves any text
// between arguments (such as separators or whitespace).
let rawInner = '';
if (innerStart <= innerEnd) {
let cursor = innerStart;
for (const entry of evaluatedArguments) {
if (entry.startOffset > cursor) {
rawInner += text.slice(cursor, entry.startOffset);
}
rawInner += entry.value;
cursor = entry.endOffset + 1;
}
if (cursor <= innerEnd) {
rawInner += text.slice(cursor, innerEnd + 1);
}
}
/** @type {MacroCall} */
const call = {
name,
args,
rawInner,
rawWithBraces: text.slice(range.startOffset, range.endOffset + 1),
range,
cstNode: macroNode,
env,
};
const value = resolveMacro(call);
const stringValue = typeof value === 'string' ? value : String(value ?? '');
return stringValue;
}
/**
* Evaluates a single argument node by resolving nested macros and reconstructing
* the original argument text.
*
* @param {CstNode} argNode
* @param {EvaluationContext} context
* @returns {string}
*/
#evaluateArgumentNode(argNode, context) {
const location = this.#getArgumentLocation(argNode);
if (!location) {
return '';
}
const { text } = context;
const nestedMacros = /** @type {CstNode[]} */ ((argNode.children || {}).macro || []);
// If there are no nested macros, we can just return the original text
if (nestedMacros.length === 0) {
return text.slice(location.startOffset, location.endOffset + 1);
}
// If there are macros, evaluate them one by one in appearing order, inside the argument, before we return the resolved argument
const nestedWithRange = nestedMacros.map(node => ({
node,
range: this.#getMacroRange(node),
}));
nestedWithRange.sort((a, b) => a.range.startOffset - b.range.startOffset);
let result = '';
let cursor = location.startOffset;
for (const entry of nestedWithRange) {
if (entry.range.startOffset < cursor) {
continue;
}
result += text.slice(cursor, entry.range.startOffset);
result += this.#evaluateMacroNode(entry.node, context);
cursor = entry.range.endOffset + 1;
}
if (cursor <= location.endOffset) {
result += text.slice(cursor, location.endOffset + 1);
}
return result;
}
/**
* Computes the character range of a macro node based on its start/end tokens
* or its own location if those are not available.
*
* @param {CstNode} macroNode
* @returns {TokenRange}
*/
#getMacroRange(macroNode) {
const startToken = /** @type {IToken?} */ (((macroNode.children || {})['Macro.Start'] || [])[0]);
const endToken = /** @type {IToken?} */ (((macroNode.children || {})['Macro.End'] || [])[0]);
if (startToken && endToken) {
return { startOffset: startToken.startOffset, endOffset: endToken.endOffset };
}
if (macroNode.location) {
return { startOffset: macroNode.location.startOffset, endOffset: macroNode.location.endOffset };
}
return { startOffset: 0, endOffset: 0 };
}
/**
* Flattens an incomplete macro node into document items.
* Tokens from the incomplete macro become plaintext, but nested complete macros are preserved.
*
* @param {CstNode} macroNode
* @param {IToken} excludeToken - The recovery-inserted token to exclude
* @param {Array<DocumentItem>} items - The items array to add to
*/
#flattenIncompleteMacro(macroNode, excludeToken, items) {
const children = macroNode.children || {};
for (const key of Object.keys(children)) {
for (const element of children[key] || []) {
// Skip the recovery-inserted token
if (element === excludeToken) continue;
// Handle IToken - add as plaintext
if ('startOffset' in element && typeof element.startOffset === 'number') {
items.push({
type: 'plaintext',
startOffset: element.startOffset,
endOffset: element.endOffset ?? element.startOffset,
token: element,
});
}
// Handle nested CstNode (macro or argument)
else if ('children' in element) {
const nestedChildren = element.children || {};
const nestedEnd = /** @type {IToken?} */ ((nestedChildren['Macro.End'] || [])[0]);
const nestedStart = /** @type {IToken?} */ ((nestedChildren['Macro.Start'] || [])[0]);
// Check if this is a complete macro node
if (nestedStart && nestedEnd) {
if (!this.#isRecoveryToken(nestedEnd)) {
// Complete nested macro - add as macro item
const range = this.#getMacroRange(element);
items.push({
type: 'macro',
startOffset: range.startOffset,
endOffset: range.endOffset,
node: element,
});
} else {
// Another incomplete nested macro - recurse
this.#flattenIncompleteMacro(element, nestedEnd, items);
}
} else {
// Not a macro node (e.g., arguments, argument) - recurse into it
this.#flattenIncompleteMacro(element, excludeToken, items);
}
}
}
}
}
/**
* Checks if a token was inserted during Chevrotain's error recovery.
* Recovery tokens have `isInsertedInRecovery=true` or invalid offset values.
*
* @param {IToken|null|undefined} token
* @returns {boolean}
*/
#isRecoveryToken(token) {
return token?.isInsertedInRecovery === true
|| typeof token?.startOffset !== 'number'
|| Number.isNaN(token?.startOffset);
}
/**
* Computes the character range of an argument node based on all its child
* tokens and nested macros.
*
* @param {CstNode} argNode
* @returns {TokenRange|null}
*/
#getArgumentLocation(argNode) {
const children = argNode.children || {};
let startOffset = Number.POSITIVE_INFINITY;
let endOffset = Number.NEGATIVE_INFINITY;
for (const key of Object.keys(children)) {
for (const element of children[key] || []) {
if (this.#isCstNode(element)) {
const location = element.location;
if (!location) {
continue;
}
if (location.startOffset < startOffset) {
startOffset = location.startOffset;
}
if (location.endOffset > endOffset) {
endOffset = location.endOffset;
}
} else if (element) {
if (element.startOffset < startOffset) {
startOffset = element.startOffset;
}
if (element.endOffset > endOffset) {
endOffset = element.endOffset;
}
}
}
}
if (!Number.isFinite(startOffset) || !Number.isFinite(endOffset)) {
return null;
}
return { startOffset, endOffset };
}
/**
* Determines whether the given value is a CST node.
*
* @param {any} value
* @returns {value is CstNode}
*/
#isCstNode(value) {
return !!value && typeof value === 'object' && 'name' in value && 'children' in value;
}
}
instance = MacroCstWalker.instance;
@@ -0,0 +1,197 @@
/** @typedef {import('./MacroCstWalker.js').MacroCall} MacroCall */
/** @typedef {import('./MacroRegistry.js').MacroDefinition} MacroDefinition */
/** @typedef {import('chevrotain').ILexingError} ILexingError */
/** @typedef {import('chevrotain').IRecognitionException} IRecognitionException */
/**
* @typedef {Object} MacroErrorContext
* @property {string} [macroName]
* @property {MacroCall} [call]
* @property {MacroDefinition} [def]
*/
/**
* Options for creating a macro runtime error.
*
* @typedef {MacroErrorContext & { message: string }} MacroRuntimeErrorOptions
*/
/**
* Options for logging macro warnings or errors.
*
* @typedef {MacroErrorContext & { message: string, error?: any }} MacroLogOptions
*/
/**
* Creates an error representing a runtime macro invocation problem (such as
* arity or type mismatches). These errors are intended to be caught by the
* MacroEngine, which will log them as runtime warnings and leave the macro
* raw in the evaluated text.
*
* @param {MacroRuntimeErrorOptions} options
* @returns {Error}
*/
export function createMacroRuntimeError({ message, call, def, macroName }) {
const inferredName = inferMacroName(call, def, macroName);
const error = new Error(message);
error.name = 'MacroRuntimeError';
// @ts-ignore - custom tagging for downstream classification
error.isMacroRuntimeError = true;
// @ts-ignore - helpful metadata for debugging
error.macroName = inferredName;
// @ts-ignore - best-effort location information
error.macroRange = call && call.range ? call.range : null;
// @ts-ignore - attach raw call/definition for convenience
if (call) error.macroCall = call;
// @ts-ignore
if (def) error.macroDefinition = def;
return error;
}
/**
* Logs a macro runtime warning with consistent, helpful context. These
* correspond to issues in how a macro was written in the text (e.g. invalid
* arguments), not bugs in macro definitions or the engine itself.
*
* @param {MacroLogOptions} options
*/
export function logMacroRuntimeWarning({ message, call, def, macroName, error }) {
const payload = buildMacroPayload({ call, def, macroName, error });
console.warn('[Macro] Warning:', message, payload);
}
/**
* Logs an internal macro error (definition or engine bug) with a consistent
* schema. These are surfaced as red errors in the console.
*
* @param {MacroLogOptions} options
*/
export function logMacroInternalError({ message, call, macroName, error }) {
const payload = buildMacroPayload({ call, def: undefined, macroName, error });
console.error('[Macro] Error:', message, payload);
}
/**
* Logs a warning during macro registration.
*
* @param {{ message: string, macroName?: string, error?: any }} options
*/
export function logMacroRegisterWarning({ message, macroName, error = undefined }) {
const payload = buildMacroPayload({ macroName, error });
console.warn('[Macro] Warning:', message, payload);
}
/**
* Logs an error during macro registration. Used when registration fails
* and the macro will not be available.
*
* @param {{ message: string, macroName?: string, error?: any }} options
*/
export function logMacroRegisterError({ message, macroName, error = undefined }) {
const payload = buildMacroPayload({ macroName, error });
console.error('[Macro] Registration Error:', message, payload);
}
/**
* Logs a macro error with a consistent schema.
*
* @param {{ message: string, error?: any }} options
*/
export function logMacroGeneralError({ message, error }) {
console.error('[Macro] Error:', message, error);
}
/**
* Logs lexer/parser syntax warnings for the macro engine with a compact,
* human-readable payload.
*
* @param {{ phase: 'lexing', input: string, errors: ILexingError[] }|{ phase: 'parsing', input: string, errors: IRecognitionException[] }} options
*/
export function logMacroSyntaxWarning({ phase, input, errors }) {
if (!errors || errors.length === 0) {
return;
}
/** @type {{ message: string, line: number|null, column: number|null, length: number|null }[]} */
const issues = errors.map((err) => {
const hasOwnLine = typeof err.line === 'number';
const hasOwnColumn = typeof err.column === 'number';
const token = /** @type {{ startLine?: number, startColumn?: number, startOffset?: number, endOffset?: number }|undefined} */ (err.token);
const line = hasOwnLine ? err.line : (token && typeof token.startLine === 'number' ? token.startLine : null);
const column = hasOwnColumn ? err.column : (token && typeof token.startColumn === 'number' ? token.startColumn : null);
/** @type {number|null} */
let length = null;
if (typeof err.length === 'number') {
length = err.length;
} else if (token && typeof token.startOffset === 'number' && typeof token.endOffset === 'number') {
length = token.endOffset - token.startOffset + 1;
}
return {
message: err.message,
line,
column,
length,
};
});
const label = phase === 'lexing' ? 'Lexing' : 'Parsing';
/** @type {Record<string, any>} */
const payload = {
phase,
count: issues.length,
issues,
input,
};
console.warn('[Macro] Warning:', `${label} errors detected`, payload);
}
/**
* Builds a structured payload for macro logging.
*
* @param {MacroErrorContext & { error?: any }} ctx
*/
function buildMacroPayload({ call, def, macroName, error }) {
const inferredName = inferMacroName(call, def, macroName);
/** @type {Record<string, any>} */
const payload = {
macroName: inferredName,
};
if (call && call.range) payload.range = call.range;
if (call && typeof call.rawInner === 'string') payload.raw = call.rawInner;
if (call) payload.call = call;
if (def) payload.def = def;
if (error) payload.error = error;
return payload;
}
/**
* Infers the most appropriate macro name from the available context.
*
* @param {MacroCall} [call]
* @param {MacroDefinition} [def]
* @param {string} [explicit]
* @returns {string}
*/
function inferMacroName(call, def, explicit) {
if (typeof explicit === 'string' && explicit.trim()) {
return explicit.trim();
}
if (call && typeof call.name === 'string' && call.name.trim()) {
return call.name.trim();
}
if (def && typeof def.name === 'string' && def.name.trim()) {
return def.name.trim();
}
return 'unknown';
}
+212
View File
@@ -0,0 +1,212 @@
import { MacroParser } from './MacroParser.js';
import { MacroCstWalker } from './MacroCstWalker.js';
import { MacroRegistry } from './MacroRegistry.js';
import { logMacroGeneralError, logMacroInternalError, logMacroRuntimeWarning, logMacroSyntaxWarning } from './MacroDiagnostics.js';
/** @typedef {import('./MacroCstWalker.js').MacroCall} MacroCall */
/** @typedef {import('./MacroEnv.types.js').MacroEnv} MacroEnv */
/** @typedef {import('./MacroRegistry.js').MacroDefinition} MacroDefinition */
/**
* The singleton instance of the MacroEngine.
*
* @type {MacroEngine}
*/
let instance;
export { instance as MacroEngine };
class MacroEngine {
/** @type {MacroEngine} */ static #instance;
/** @type {MacroEngine} */ static get instance() { return MacroEngine.#instance ?? (MacroEngine.#instance = new MacroEngine()); }
constructor() { }
/**
* Evaluates a string containing macros and resolves them.
*
* @param {string} input - The input string to evaluate.
* @param {MacroEnv} env - The environment to pass to the macro handler.
* @returns {string} The resolved string.
*/
evaluate(input, env) {
if (!input) {
return '';
}
const safeEnv = Object.freeze({ ...env });
const preProcessed = this.#runPreProcessors(input, safeEnv);
const { cst, lexingErrors, parserErrors } = MacroParser.parseDocument(preProcessed);
// For now, we log and still try to process what we can.
if (lexingErrors && lexingErrors.length > 0) {
logMacroSyntaxWarning({ phase: 'lexing', input, errors: lexingErrors });
}
if (parserErrors && parserErrors.length > 0) {
logMacroSyntaxWarning({ phase: 'parsing', input, errors: parserErrors });
}
// If the parser did not produce a valid CST, fall back to the original input.
if (!cst || typeof cst !== 'object' || !cst.children) {
logMacroGeneralError({ message: 'Macro parser produced an invalid CST. Returning original input.', error: { input, lexingErrors, parserErrors } });
return input;
}
let evaluated;
try {
evaluated = MacroCstWalker.evaluateDocument({
text: preProcessed,
cst,
env: safeEnv,
resolveMacro: this.#resolveMacro.bind(this),
});
} catch (error) {
logMacroGeneralError({ message: 'Macro evaluation failed. Returning original input.', error: { input, error } });
return input;
}
const result = this.#runPostProcessors(evaluated, safeEnv);
return result;
}
/**
* Resolves a macro call.
*
* @param {MacroCall} call - The macro call to resolve.
* @returns {string} The resolved macro.
*/
#resolveMacro(call) {
const { name, env } = call;
const raw = `{{${call.rawInner}}}`;
if (!name) return raw;
// First check if this is a dynamic macro to use. If so, we will create a temporary macro definition for it and use that over any registered macro.
/** @type {MacroDefinition?} */
let defOverride = null;
if (Object.hasOwn(env.dynamicMacros, name)) {
const impl = env.dynamicMacros[name];
defOverride = {
name,
aliases: [],
category: 'dynamic',
description: 'Dynamic macro',
minArgs: 0,
maxArgs: 0,
unnamedArgDefs: [],
list: null,
strictArgs: true, // Fail dynamic macros if they are called with arguments
returns: null,
returnType: 'string',
displayOverride: null,
exampleUsage: [],
source: { name: 'dynamic', isExtension: false, isThirdParty: false },
aliasOf: null,
aliasVisible: null,
handler: typeof impl === 'function' ? impl : () => impl,
};
}
// If not, check if the macro exists and is registered
if (!defOverride && !MacroRegistry.hasMacro(name)) {
return raw; // Unknown macro: keep macro syntax, but nested macros inside rawInner are already resolved.
}
try {
const result = MacroRegistry.executeMacro(call, { defOverride });
try {
return call.env.functions.postProcess(result);
} catch (error) {
logMacroInternalError({ message: `Macro "${name}" postProcess function failed.`, call, error });
return result;
}
} catch (error) {
const isRuntimeError = !!(error && (error.name === 'MacroRuntimeError' || error.isMacroRuntimeError));
if (isRuntimeError) {
logMacroRuntimeWarning({ message: (error.message || `Macro "${name}" execution failed.`), call, error });
} else {
logMacroInternalError({ message: `Macro "${name}" internal execution error.`, call, error });
}
return raw;
}
}
/**
* Runs pre-processors on the input text, before the engine processes the input.
*
* @param {string} text - The input text to process.
* @param {MacroEnv} env - The environment to pass to the macro handler.
* @returns {string} The processed text.
*/
#runPreProcessors(text, env) {
let result = text;
// This legacy macro will not be supported by the new macro parser, but rather regex-replaced beforehand
// {{time_UTC-10}} => {{time::UTC-10}}
result = result.replace(/{{time_(UTC[+-]\d+)}}/gi, (_match, utcOffset) => {
return `{{time::${utcOffset}}}`;
});
// Legacy non-curly markers like <USER>, <BOT>, <GROUP>, etc.
// These are rewritten into their equivalent macro forms so they go through the normal engine pipeline.
result = result.replace(/<USER>/gi, '{{user}}');
result = result.replace(/<BOT>/gi, '{{char}}');
result = result.replace(/<CHAR>/gi, '{{char}}');
result = result.replace(/<GROUP>/gi, '{{group}}');
result = result.replace(/<CHARIFNOTGROUP>/gi, '{{charIfNotGroup}}');
return result;
}
/**
* Runs post-processors on the input text, after the engine finished processing the input.
*
* @param {string} text - The input text to process.
* @param {MacroEnv} env - The environment to pass to the macro handler.
* @returns {string} The processed text.
*/
#runPostProcessors(text, env) {
let result = text;
// Unescape braces: \{ → { and \} → }
// Since \{\{ doesn't match {{ (MacroStart), it passes through as plain text.
// We only need to remove the backslashes in post-processing.
result = result.replace(/\\([{}])/g, '$1');
// The original trim macro is reaching over the boundaries of the defined macro. This is not something the engine supports.
// To treat {{trim}} as it was before, we won't process it by the engine itself,
// but doing a regex replace on {{trim}} and the surrounding area, after all other macros have been processed.
result = result.replace(/(?:\r?\n)*{{trim}}(?:\r?\n)*/gi, '');
return result;
}
/**
* Normalizes macro results into a string.
* This mirrors the behavior of the legacy macro system in a simplified way.
*
* @param {any} value
* @returns {string}
*/
normalizeMacroResult(value) {
if (value === null || value === undefined) {
return '';
}
if (value instanceof Date) {
return value.toISOString();
}
if (typeof value === 'object' || Array.isArray(value)) {
try {
return JSON.stringify(value);
} catch (_error) {
return String(value);
}
}
return String(value);
}
}
instance = MacroEngine.instance;
@@ -0,0 +1,56 @@
/**
* Shared typedefs for the structured macro environment object (MacroEnv)
* used by the macro engine, registry, env builder, and macro definition
* modules. This file intentionally only contains JSDoc typedefs so that
* it can be imported purely for type information from multiple modules
* without creating runtime dependencies.
*/
/** @typedef {import('./MacroRegistry.js').MacroHandler} MacroHandler */
/**
* @typedef {Object} MacroEnvNames
* @property {string} user
* @property {string} char
* @property {string} group
* @property {string} groupNotMuted
* @property {string} notChar
*/
/**
* @typedef {Object} MacroEnvCharacter
* @property {string} [description]
* @property {string} [personality]
* @property {string} [scenario]
* @property {string} [persona]
* @property {string} [charPrompt]
* @property {string} [charInstruction]
* @property {string} [mesExamplesRaw]
* @property {string} [charDepthPrompt]
* @property {string} [creatorNotes]
* @property {string} [version]
*/
/**
* @typedef {Object} MacroEnvSystem
* @property {string} model
*/
/**
* @typedef {Object} MacroEnvFunctions
* @property {() => string} [original]
* @property {(text: string) => string} postProcess
*/
/**
* @typedef {Object} MacroEnv
* @property {string} content - The full original input string that is being processed by the macro engine. This is the same value as substituteParams "content" and is provided so macros can build deterministic behavior based on the whole prompt when needed.
* @property {MacroEnvNames} names
* @property {MacroEnvCharacter} character
* @property {MacroEnvSystem} system
* @property {MacroEnvFunctions} functions
* @property {Object<string, string|MacroHandler>} dynamicMacros
* @property {Record<string, unknown>} extra
*/
export {};
@@ -0,0 +1,197 @@
import { name1, name2, characters, getCharacterCardFieldsLazy, getGeneratingModel } from '../../../script.js';
import { groups, selected_group } from '../../../scripts/group-chats.js';
import { logMacroGeneralError } from './MacroDiagnostics.js';
/**
* MacroEnvBuilder is responsible for constructing the MacroEnv object
* that is passed to macro handlers.
*
* It does **not** depend on the legacy regex macro system. Instead, it
* works from the same raw inputs that substituteParams receives plus a
* small bundle of global helpers, so it can eventually replace the
* environment-building block in substituteParams.
*/
/** @typedef {import('./MacroEnv.types.js').MacroEnv} MacroEnv */
/**
* @typedef {Object} MacroEnvRawContext
* @property {string} content
* @property {string|null} [name1Override]
* @property {string|null} [name2Override]
* @property {string|null} [original]
* @property {string|null} [groupOverride]
* @property {boolean} [replaceCharacterCard]
* @property {Record<string, any>|null} [dynamicMacros]
* @property {(value: string) => string} [postProcessFn]
*/
/**
* @typedef {(env: MacroEnv, ctx: MacroEnvRawContext) => void} MacroEnvProvider
*/
/**
* @enum {number} Exposed ordering buckets for providers. Callers can use envBuilder.providerOrder.* when registering providers.
*/
export const env_provider_order = {
EARLIEST: 0,
EARLY: 10,
NORMAL: 50,
LATE: 90,
LATEST: 100,
};
/** @type {MacroEnvBuilder} */
let instance;
export { instance as MacroEnvBuilder };
class MacroEnvBuilder {
/** @type {MacroEnvBuilder} */ static #instance;
/** @type {MacroEnvBuilder} */ static get instance() { return MacroEnvBuilder.#instance ?? (MacroEnvBuilder.#instance = new MacroEnvBuilder()); }
/** @type {{ fn: MacroEnvProvider, order: env_provider_order }[]} */
#providers;
constructor() {
this.#providers = [];
}
/**
* Registers a provider that can augment the MacroEnv with additional
* data (for extensions, extra context, etc.).
*
* Should be called once during initialization.
*
* @param {MacroEnvProvider} provider
* @param {env_provider_order} [order=env_provider_order.NORMAL]
* @returns {void}
*/
registerProvider(provider, order = env_provider_order.NORMAL) {
if (typeof provider !== 'function') throw new Error('Provider must be a function');
this.#providers.push({ fn: provider, order });
}
/**
* Builds a MacroEnv from the raw arguments that are conceptually the
* same as substituteParams receives, plus a bundle of global helpers.
*
* @param {MacroEnvRawContext} ctx
* @returns {MacroEnv}
*/
buildFromRawEnv(ctx) {
// Create the env first, we will populate it step by step.
// Some fields are marked as required, so we have to fill them with dummy fields here
/** @type {MacroEnv} */
const env = {
content: ctx.content,
names: { user: '', char: '', group: '', groupNotMuted: '', notChar: '' },
character: {},
system: { model: '' },
functions: { postProcess: (x) => x },
dynamicMacros: {},
extra: {},
};
if (ctx.replaceCharacterCard) {
// Use lazy fields - each property is only resolved when accessed
const fields = getCharacterCardFieldsLazy();
if (fields) {
// Define lazy getters on env.character that delegate to fields
const fieldMappings = /** @type {const} */ ([
['charPrompt', 'system'],
['charInstruction', 'jailbreak'],
['description', 'description'],
['personality', 'personality'],
['scenario', 'scenario'],
['persona', 'persona'],
['mesExamplesRaw', 'mesExamples'],
['version', 'version'],
['charDepthPrompt', 'charDepthPrompt'],
['creatorNotes', 'creatorNotes'],
]);
for (const [envKey, fieldKey] of fieldMappings) {
Object.defineProperty(env.character, envKey, {
get() { return fields[fieldKey] || ''; },
enumerable: true,
configurable: true,
});
}
}
}
// Names
env.names.user = ctx.name1Override ?? name1 ?? '';
env.names.char = ctx.name2Override ?? name2 ?? '';
env.names.group = getGroupValue(ctx, { currentChar: env.names.char, includeMuted: true });
env.names.groupNotMuted = getGroupValue(ctx, { currentChar: env.names.char, includeMuted: false });
env.names.notChar = getGroupValue(ctx, { currentChar: env.names.char, filterOutChar: true, includeUser: env.names.user });
// System
env.system.model = getGeneratingModel();
// Functions
// original (one-shot) and arbitrary additional values
if (typeof ctx.original === 'string') {
let originalSubstituted = false;
env.functions.original = () => {
if (originalSubstituted) return '';
originalSubstituted = true;
return ctx.original;
};
}
env.functions.postProcess = typeof ctx.postProcessFn === 'function' ? ctx.postProcessFn : (x) => x;
// Dynamic, per-call macros that should be visible only for this evaluation run.
if (ctx.dynamicMacros && typeof ctx.dynamicMacros === 'object') {
env.dynamicMacros = { ...ctx.dynamicMacros };
}
// Let providers augment the env, if any are registered. Apply them in order,
// so callers can influence when their provider runs relative to others.
const orderedProviders = this.#providers.slice().sort((a, b) => a.order - b.order);
for (const { fn } of orderedProviders) {
try {
fn(env, ctx);
} catch (e) {
// Provider errors should not break macro evaluation
logMacroGeneralError({ message: 'MacroEnvBuilder: Provider error', error: e });
}
}
return env;
}
}
instance = MacroEnvBuilder.instance;
/**
* @param {MacroEnvRawContext} ctx
* @param {Object} options
* @param {string} [options.currentChar=null]
* @param {boolean} [options.includeMuted=false]
* @param {boolean} [options.filterOutChar=false]
* @param {string|null} [options.includeUser=null]
* @returns {string}
*/
function getGroupValue(ctx, { currentChar = null, includeMuted = false, filterOutChar = false, includeUser = null }) {
if (typeof ctx.groupOverride === 'string') {
return ctx.groupOverride;
}
if (!selected_group) return filterOutChar ? (includeUser || '') : (currentChar ?? '');
const groupEntry = Array.isArray(groups) ? groups.find(x => x && x.id === selected_group) : null;
const members = /** @type {string[]} */ (groupEntry?.members ?? []);
const disabledMembers = /** @type {string[]} */ (groupEntry?.disabled_members ?? []);
const names = Array.isArray(members)
? members
.filter(((id) => includeMuted ? true : !disabledMembers.includes(id)))
.map(m => Array.isArray(characters) ? characters.find(c => c && c.avatar === m) : null)
.filter(c => !!c && typeof c.name === 'string')
.filter(c => !filterOutChar || c.name !== currentChar)
.map(c => c.name)
.join(', ')
: '';
return names;
}
+239
View File
@@ -0,0 +1,239 @@
import { chevrotain } from '../../../lib.js';
const { createToken, Lexer } = chevrotain;
/** @typedef {import('chevrotain').TokenType} TokenType */
/** @enum {string} */
const modes = {
plaintext: 'plaintext_mode',
macro_def: 'macro_def_mode',
macro_identifier_end: 'macro_identifier_end_mode',
macro_args: 'macro_args_mode',
macro_filter_modifer: 'macro_filter_modifer_mode',
macro_filter_modifier_end: 'macro_filter_modifier_end_mode',
};
/** @readonly */
const Tokens = {
// General capture-all plaintext without macros. Consumes any character that is not the first '{' of a macro opener '{{'.
Plaintext: createToken({ name: 'Plaintext', pattern: /(?:[^{]|\{(?!\{))+/u, line_breaks: true }),
// Single literal '{' that appears immediately before a macro opener '{{'.
PlaintextOpenBrace: createToken({ name: 'Plaintext.OpenBrace', pattern: /\{(?=\{\{)/ }),
// General macro capture
Macro: {
Start: createToken({ name: 'Macro.Start', pattern: /\{\{/ }),
// Separate macro identifier needed, that is similar to the global indentifier, but captures the actual macro "name"
// We need this, because this token is going to switch lexer mode, while the general identifier does not.
Flags: createToken({ name: 'Macro.Flag', pattern: /[!?#~/.$]/ }),
DoubleSlash: createToken({ name: 'Macro.DoubleSlash', pattern: /\/\// }),
Identifier: createToken({ name: 'Macro.Identifier', pattern: /[a-zA-Z][\w-_]*/ }),
// At the end of an identifier, there has to be whitspace, or must be directly followed by colon/double-colon separator, output modifier or closing braces
EndOfIdentifier: createToken({ name: 'Macro.EndOfIdentifier', pattern: /(?:\s+|(?=:{1,2})|(?=[|}]))/, group: Lexer.SKIPPED }),
BeforeEnd: createToken({ name: 'Macro.BeforeEnd', pattern: /(?=\}\})/, group: Lexer.SKIPPED }),
End: createToken({ name: 'Macro.End', pattern: /\}\}/ }),
},
// Captures that only appear inside arguments
Args: {
DoubleColon: createToken({ name: 'Args.DoubleColon', pattern: /::/ }),
Colon: createToken({ name: 'Args.Colon', pattern: /:/ }),
Equals: createToken({ name: 'Args.Equals', pattern: /=/ }),
Quote: createToken({ name: 'Args.Quote', pattern: /"/ }),
},
Filter: {
EscapedPipe: createToken({ name: 'Filter.EscapedPipe', pattern: /\\\|/ }),
Pipe: createToken({ name: 'Filter.Pipe', pattern: /\|/ }),
Identifier: createToken({ name: 'Filter.Identifier', pattern: /[a-zA-Z][\w-_]*/ }),
// At the end of an identifier, there has to be whitspace, or must be directly followed by colon/double-colon separator, output modifier or closing braces
EndOfIdentifier: createToken({ name: 'Filter.EndOfIdentifier', pattern: /(?:\s+|(?=:{1,2})|(?=[|}]))/, group: Lexer.SKIPPED }),
},
// All tokens that can be captured inside a macro
Identifier: createToken({ name: 'Identifier', pattern: /[a-zA-Z][\w-_]*/ }),
WhiteSpace: createToken({ name: 'WhiteSpace', pattern: /\s+/, group: Lexer.SKIPPED }),
// Capture unknown characters one by one, to still allow other tokens being matched once they are there.
// This includes any possible braces that is not the double closing braces as MacroEnd.
Unknown: createToken({ name: 'Unknown', pattern: /([^}]|\}(?!\}))/ }),
// TODO: Capture-all rest for now, that is not the macro end or opening of a new macro. Might be replaced later down the line.
Text: createToken({ name: 'Text', pattern: /.+(?=\}\}|\{\{)/, line_breaks: true }),
// DANGER ZONE: Careful with this token. This is used as a way to pop the current mode, if no other token matches.
// Can be used in modes that don't have a "defined" end really, like when capturing a single argument, argument list, etc.
// Has to ALWAYS be the last token.
ModePopper: createToken({ name: 'ModePopper', pattern: () => [''], line_breaks: false, group: Lexer.SKIPPED }),
};
/** @type {Map<string,string>} Saves all token definitions that are marked as entering modes */
const enterModesMap = new Map();
const Def = {
modes: {
[modes.plaintext]: [
using(Tokens.Plaintext),
using(Tokens.PlaintextOpenBrace),
enter(Tokens.Macro.Start, modes.macro_def),
],
[modes.macro_def]: [
exits(Tokens.Macro.End, modes.macro_def),
// An explicit double-slash will be treated above flags to consume, as it'll introduce a comment macro. Directly following is the args then.
enter(Tokens.Macro.DoubleSlash, modes.macro_args),
using(Tokens.Macro.Flags),
// We allow whitspaces inbetween flags or in front of the modifier
using(Tokens.WhiteSpace),
// Inside a macro, we will match the identifier
// Enter 'macro_identifier_end' mode automatically at the end of the identifier, so we don't match more than one identifier
enter(Tokens.Macro.Identifier, modes.macro_identifier_end),
// If none of the tokens above are found, this is an invalid macro at runtime.
// We still need to exit the mode to prevent lexer errors
exits(Tokens.ModePopper, modes.macro_def),
],
[modes.macro_identifier_end]: [
// Valid options after a macro identifier: whitespace, colon/double-colon (captured), macro end braces, or output modifier pipe.
exits(Tokens.Macro.BeforeEnd, modes.macro_identifier_end),
enter(Tokens.Macro.EndOfIdentifier, modes.macro_args, { andExits: modes.macro_identifier_end }),
],
[modes.macro_args]: [
// Macro args allow nested macros
enter(Tokens.Macro.Start, modes.macro_def),
// We allow escaped pipes to not start output modifiers. We need to capture this first, before the pipe
using(Tokens.Filter.EscapedPipe),
// If at any place during args writing there is a pipe, we lex it as an output identifier, and then continue with lex its args
enter(Tokens.Filter.Pipe, modes.macro_filter_modifer),
using(Tokens.Args.DoubleColon),
using(Tokens.Args.Colon),
using(Tokens.Args.Equals),
using(Tokens.Args.Quote),
using(Tokens.Identifier),
using(Tokens.WhiteSpace),
// Last fallback, before we need to exit the mode, as we might have characters we (wrongly) haven't defined yet
using(Tokens.Unknown),
// Args are optional, and we don't know how long, so exit the mode to be able to capture the actual macro end
exits(Tokens.ModePopper, modes.macro_args),
],
[modes.macro_filter_modifer]: [
using(Tokens.WhiteSpace),
enter(Tokens.Filter.Identifier, modes.macro_filter_modifier_end, { andExits: modes.macro_filter_modifer }),
],
[modes.macro_filter_modifier_end]: [
// Valid options after a filter itenfier: whitespace, colon/double-colon (captured), macro end braces, or output modifier pipe.
exits(Tokens.Macro.BeforeEnd, modes.macro_identifier_end),
exits(Tokens.Filter.EndOfIdentifier, modes.macro_filter_modifer),
],
},
defaultMode: modes.plaintext,
};
/**
* The singleton instance of the MacroLexer.
*
* @type {MacroLexer}
*/
let instance;
export { instance as MacroLexer };
class MacroLexer extends Lexer {
/** @type {MacroLexer} */ static #instance;
/** @type {MacroLexer} */ static get instance() { return MacroLexer.#instance ?? (MacroLexer.#instance = new MacroLexer()); }
// Define the tokens
/** @readonly */ static tokens = Tokens;
/** @readonly */ static def = Def;
/** @readonly */ tokens = Tokens;
/** @readonly */ def = MacroLexer.def;
/** @private */
constructor() {
super(MacroLexer.def, {
traceInitPerf: true,
});
}
test(input) {
const result = this.tokenize(input);
return {
errors: result.errors,
groups: result.groups,
tokens: result.tokens.map(({ tokenType, ...rest }) => ({ type: tokenType.name, ...rest, tokenType: tokenType })),
};
}
}
instance = MacroLexer.instance;
/**
* [Utility]
* Set push mode on the token definition.
* Can be used inside the token mode definition block.
*
* Marks the token to **enter** the following lexer mode.
*
* Optionally, you can specify the modes to exit when entering this mode.
*
* @param {TokenType} token - The token to modify
* @param {string} mode - The mode to set
* @param {object} [options={}] - Additional options
* @param {string} [options.andExits] - The modes to exit when entering this mode
* @returns {TokenType} The token again
*/
function enter(token, mode, { andExits = undefined } = {}) {
if (!token) throw new Error('Token must not be undefined');
if (enterModesMap.has(token.name) && enterModesMap.get(token.name) !== mode) {
throw new Error(`Token ${token.name} already is set to enter mode ${enterModesMap.get(token.name)}. The token definition are global, so they cannot be used to lead to different modes.`);
}
if (andExits) exits(token, andExits);
token.PUSH_MODE = mode;
enterModesMap.set(token.name, mode);
return token;
}
/**
* [Utility]
* Set pop mode on the token definition.
* Can be used inside the token mode definition block.
*
* Marks the token to **exit** the following lexer mode.
*
* @param {TokenType} token - The token to modify
* @param {string} mode - The mode to leave
* @returns {TokenType} The token again
*/
function exits(token, mode) {
if (!token) throw new Error('Token must not be undefined');
token.POP_MODE = !!mode; // Always set to true. We just use the mode here, so the linter thinks it was used. We just pass it in for clarity in the definition
return token;
}
/**
* [Utility]
* Can be used inside the token mode definition block.
*
* Marks the token to to just be used/consumed, and not exit or enter a mode.
*
* @param {TokenType} token - The token to modify
* @returns {TokenType} The token again
*/
function using(token) {
if (!token) throw new Error('Token must not be undefined');
if (enterModesMap.has(token.name)) {
throw new Error(`Token ${token.name} is already marked to enter a mode (${enterModesMap.get(token.name)}). The token definition are global, so they cannot be used to lead or stay differently.`);
}
return token;
}
+149
View File
@@ -0,0 +1,149 @@
import { chevrotain } from '../../../lib.js';
import { MacroLexer } from './MacroLexer.js';
const { CstParser } = chevrotain;
/** @typedef {import('chevrotain').TokenType} TokenType */
/** @typedef {import('chevrotain').CstNode} CstNode */
/** @typedef {import('chevrotain').ILexingError} ILexingError */
/** @typedef {import('chevrotain').IRecognitionException} IRecognitionException */
/**
* The singleton instance of the MacroParser.
*
* @type {MacroParser}
*/
let instance;
export { instance as MacroParser };
class MacroParser extends CstParser {
/** @type {MacroParser} */ static #instance;
/** @type {MacroParser} */ static get instance() { return MacroParser.#instance ?? (MacroParser.#instance = new MacroParser()); }
/** @private */
constructor() {
super(MacroLexer.def, {
traceInitPerf: true,
nodeLocationTracking: 'full',
recoveryEnabled: true,
});
const Tokens = MacroLexer.tokens;
const $ = this;
// Top-level document rule that can handle both plaintext and macros
$.document = $.RULE('document', () => {
$.MANY(() => {
$.OR([
{ ALT: () => $.CONSUME(Tokens.Plaintext, { LABEL: 'plaintext' }) },
{ ALT: () => $.CONSUME(Tokens.PlaintextOpenBrace, { LABEL: 'plaintext' }) },
{ ALT: () => $.SUBRULE($.macro) },
{ ALT: () => $.CONSUME(Tokens.Macro.Start, { LABEL: 'plaintext' }) },
]);
});
});
// Basic Macro Structure
$.macro = $.RULE('macro', () => {
$.CONSUME(Tokens.Macro.Start);
$.OR([
{ ALT: () => $.CONSUME(Tokens.Macro.DoubleSlash, { LABEL: 'Macro.identifier' }) },
{ ALT: () => $.CONSUME(Tokens.Macro.Identifier, { LABEL: 'Macro.identifier' }) },
]);
$.OPTION(() => $.SUBRULE($.arguments));
$.CONSUME(Tokens.Macro.End);
});
// Arguments Parsing
$.arguments = $.RULE('arguments', () => {
$.OR([
{
ALT: () => {
$.CONSUME(Tokens.Args.DoubleColon, { LABEL: 'separator' });
$.AT_LEAST_ONE_SEP({
SEP: Tokens.Args.DoubleColon,
DEF: () => $.SUBRULE($.argument, { LABEL: 'argument' }),
});
},
},
{
ALT: () => {
$.OPTION(() => {
$.CONSUME(Tokens.Args.Colon, { LABEL: 'separator' });
});
$.SUBRULE($.argumentAllowingColons, { LABEL: 'argument' });
},
// So, this is a bit hacky. But implemented below, the argument capture does explicitly exclude double colons
// from being captured as the first token. The potential ambiguity chevrotain claims here is not possible.
// It says stuff like <Args.DoubleColon, Identifier/Macro/Unknown> is possible in both branches, but it is not.
IGNORE_AMBIGUITIES: true,
},
]);
});
// List the argument tokens here, as we need two rules, one to be able to parse with double colons and one without
const validArgumentTokens = [
{ ALT: () => $.SUBRULE($.macro) }, // Nested Macros
{ ALT: () => $.CONSUME(Tokens.Identifier) },
{ ALT: () => $.CONSUME(Tokens.Unknown) },
{ ALT: () => $.CONSUME(Tokens.Args.Colon) },
{ ALT: () => $.CONSUME(Tokens.Args.Equals) },
{ ALT: () => $.CONSUME(Tokens.Args.Quote) },
];
$.argument = $.RULE('argument', () => {
$.MANY(() => {
$.OR([...validArgumentTokens]);
});
});
$.argumentAllowingColons = $.RULE('argumentAllowingColons', () => {
$.AT_LEAST_ONE(() => {
$.OR([
...validArgumentTokens,
{ ALT: () => $.CONSUME(Tokens.Args.DoubleColon) },
]);
});
});
this.performSelfAnalysis();
}
/**
* Parses a document into a CST.
*
* @param {string} input
* @returns {{ cst: CstNode|null, errors: ({ message: string }|ILexingError|IRecognitionException)[] , lexingErrors: ILexingError[], parserErrors: IRecognitionException[] }}
*/
parseDocument(input) {
if (!input) {
return { cst: null, errors: [{ message: 'Input is empty' }], lexingErrors: [], parserErrors: [] };
}
const lexingResult = MacroLexer.tokenize(input);
this.input = lexingResult.tokens;
const cst = this.document();
const errors = [
...lexingResult.errors,
...this.errors,
];
return { cst, errors, lexingErrors: lexingResult.errors, parserErrors: this.errors };
}
test(input) {
const lexingResult = MacroLexer.tokenize(input);
// "input" is a setter which will reset the parser's state.
this.input = lexingResult.tokens;
const cst = this.macro();
// For testing purposes we need to actually persist the error messages in the object,
// otherwise the test cases cannot read those, as they don't have access to the exception object type.
const errors = this.errors.map(x => ({ message: x.message, ...x, stack: x.stack }));
return { cst, errors: errors };
}
}
instance = MacroParser.instance;
@@ -0,0 +1,672 @@
/** @typedef {import('chevrotain').CstNode} CstNode */
/** @typedef {import('./MacroEnv.types.js').MacroEnv} MacroEnv */
/** @typedef {import('./MacroCstWalker.js').MacroCall} MacroCall */
import { isFalseBoolean, isTrueBoolean } from '../../utils.js';
import { MacroEngine } from './MacroEngine.js';
import { createMacroRuntimeError, logMacroRegisterError, logMacroRegisterWarning, logMacroRuntimeWarning } from './MacroDiagnostics.js';
/**
* Enum of standard macro categories for grouping in documentation and autocomplete.
* Extensions may use these or define custom category strings.
*
* @readonly
* @enum {string}
*/
export const MacroCategory = Object.freeze({
/** Basic utilities and text manipulation (newline, noop, trim, reverse, comment) */
UTILITY: 'utility',
/** Randomization and dice rolling (random, pick, roll) */
RANDOM: 'random',
/** Participant names and name lists (user, char, group, notChar) */
NAMES: 'names',
/** Character card fields and persona (description, personality, scenario, mesExamples, persona) */
CHARACTER: 'character',
/** Chat history, messages, and swipes */
CHAT: 'chat',
/** Date, time, and duration macros */
TIME: 'time',
/** Local and global variable operations */
VARIABLE: 'variable',
/** Prompt templates for text completion (instruct sequences, system prompts, author's notes, context templates) */
PROMPTS: 'prompts',
/** Runtime application state (model, API, lastGenerationType, isMobile) */
STATE: 'state',
/** Macros that don't fit in any of the other categories, but don't really need/deserve their own */
MISC: 'misc',
});
/**
* Enum of standard macro value types for type checking and documentation.
* Used for both argument types and return types.
*
* @readonly
* @enum {string}
*/
export const MacroValueType = Object.freeze({
/** String value of any kind */
STRING: 'string',
/** Integer value (natural number, no decimal spaces) */
INTEGER: 'integer',
/** Number value (decimal spaces allowed, includes integers values) */
NUMBER: 'number',
/** Boolean value (true/false, 1/0, yes/no, on/off) */
BOOLEAN: 'boolean',
});
/**
* @typedef {Object} MacroDefinitionOptions
* @property {MacroAliasDef[]} [aliases] - Alternative names for this macro. Each alias creates a lookup entry pointing to the same definition.
* @property {MacroCategory|string} category - Category for grouping in documentation/autocomplete. Use MacroCategory enum values or a custom string.
* @property {number|MacroUnnamedArgDef[]} [unnamedArgs=0] - Specifies the macro's unnamed positional arguments. Can be a number (all required) or an array of definitions (supports optional args). Optional args must be a suffix.
* @property {boolean|MacroListSpec} [list] - Whether the macro allows a list of arguments (optional min and max values can be set). These arguments will be added AFTER the unnamed args.
* @property {boolean} [strictArgs=true] - Whether the macro should be strict about its arguments.
* @property {string} [description=''] - Add a description of what the macro does.
* @property {string} [returns] - Add a specific description of what the macro returns, if it is not obvious from the description.
* @property {MacroValueType|MacroValueType[]} [returnType=MacroValueType.STRING] - The type(s) this macro returns. Defaults to string.
* @property {string} [displayOverride] - Override the auto-generated macro signature for display (must include curly braces, e.g. "{{macro::arg}}").
* @property {string|string[]} [exampleUsage] - Example usage(s) shown in documentation (must include curly braces).
* @property {MacroHandler} handler - The handler function for the macro.
*/
/**
* @typedef {Object} MacroAliasDef
* @property {string} alias - The alias name.
* @property {boolean} [visible=true] - Whether this alias appears in documentation/autocomplete. Defaults to true.
*/
/**
* @typedef {Object} MacroUnnamedArgDef
* @property {string} name
* @property {boolean} [optional=false] - Whether this argument is optional. Optional args must form a contiguous suffix (no required args after an optional).
* @property {string} [defaultValue] - Default value for optional args. ONLY meaningful when optional is true. Shown in docs/autocomplete.
* @property {MacroValueType|MacroValueType[]} [type=MacroValueType.STRING] - Single type or array of accepted types.
* @property {string} [sampleValue]
* @property {string} [description]
*/
/**
* @typedef {Object} MacroListSpec
* @property {number} [min]
* @property {number} [max]
*/
/**
* @typedef {(context: MacroExecutionContext) => string} MacroHandler
*/
/**
* @typedef {Object} MacroExecutionContext
* @property {string} name
* @property {string[]} args - All unnamed arguments passed to the macro.
* @property {string[]} unnamedArgs - Unnamed positional arguments (both required and optional, up to the defined count).
* @property {string[]|null} list - List arguments (after unnamed args), or null if list is not enabled.
* @property {{ [key: string]: string }|null} namedArgs - Reserved for future named argument support.
* @property {string} raw
* @property {MacroEnv} env
* @property {CstNode|null} cstNode
* @property {{ startOffset: number, endOffset: number }|null} range
* @property {(value: any) => string} normalize - Normalize function to use on unsure macro results to make sure they return strings as expected.
*/
/**
* @typedef {Object} MacroDefinition
* @property {string} name - Primary macro name.
* @property {MacroResolvedAlias[]} aliases - Parsed alias definitions for this macro.
* @property {MacroCategory|string} category
* @property {number} minArgs - Minimum number of unnamed args required (excludes optional args).
* @property {number} maxArgs - Maximum number of unnamed args accepted (includes optional args).
* @property {MacroUnnamedArgDef[]} unnamedArgDefs - Definitions for all unnamed positional arguments (required + optional).
* @property {{ min: number, max: (number|null) }|null} list
* @property {boolean} strictArgs
* @property {string} description
* @property {string|null} returns
* @property {MacroValueType|MacroValueType[]} returnType - The type(s) this macro returns.
* @property {string|null} displayOverride - Override for the auto-generated macro signature display.
* @property {string[]} exampleUsage - Example usage strings for documentation.
* @property {MacroHandler} handler
* @property {MacroSource} source
* @property {string|null} aliasOf - If this is an alias, the primary macro name this is an alias of. Can also be used to check if this is an alias macro.
* @property {boolean|null} aliasVisible - If this is an alias, whether this alias is visible in docs/autocomplete.
*/
/**
* @typedef {Object} MacroResolvedAlias
* @property {string} alias - The alias name.
* @property {boolean} visible - Whether this alias is visible in documentation/autocomplete.
*/
/**
* @typedef {Object} MacroSource
* @property {string} name - Source identifier (extension name or script path)
* @property {boolean} isExtension - True if registered from an extension
* @property {boolean} isThirdParty - True if registered from a third-party extension
*/
/**
* The singleton instance of the MacroRegistry.
*
* @type {MacroRegistry}
*/
let instance;
export { instance as MacroRegistry };
class MacroRegistry {
/** @type {MacroRegistry} */ static #instance;
/** @type {MacroRegistry} */ static get instance() { return MacroRegistry.#instance ?? (MacroRegistry.#instance = new MacroRegistry()); }
/** @type {Map<string, MacroDefinition>} */
#macros;
/**
* @private
*/
constructor() {
/** @type {Map<string, MacroDefinition>} */
this.#macros = new Map();
}
/**
* Registers a macro with the registry.
* Errors during registration are caught and logged, the macro will not be registered, and the function returns null.
*
* @param {string} name - Macro name (identifier).
* @param {MacroDefinitionOptions} options - Macro registration options including handler and metadata.
* @returns {MacroDefinition|null} The registered definition, or null if registration failed.
*/
registerMacro(name, options) {
// Extract name early for error logging
name = typeof name === 'string' ? name.trim() : String(name);
try {
if (typeof name !== 'string' || !name) throw new Error('Macro name must be a non-empty string');
if (!options || typeof options !== 'object') throw new Error(`Macro "${name}" options must be a non-null object.`);
const {
aliases: rawAliases,
category: rawCategory,
unnamedArgs: rawUnnamedArgs,
list: rawList,
strictArgs: rawStrictArgs,
description: rawDescription,
returns: rawReturns,
returnType: rawReturnType,
displayOverride: rawDisplayOverride,
exampleUsage: rawExampleUsage,
handler,
} = options;
if (typeof handler !== 'function') throw new Error(`Macro "${name}" options.handler must be a function.`);
/** @type {MacroResolvedAlias[]} */
const aliases = [];
if (rawAliases !== undefined && rawAliases !== null) {
if (!Array.isArray(rawAliases)) throw new Error(`Macro "${name}" options.aliases must be an array.`);
for (const [i, aliasDef] of rawAliases.entries()) {
if (!aliasDef || typeof aliasDef !== 'object') throw new Error(`Macro "${name}" options.aliases[${i}] must be an object.`);
if (typeof aliasDef.alias !== 'string' || !aliasDef.alias.trim()) throw new Error(`Macro "${name}" options.aliases[${i}].alias must be a non-empty string.`);
const aliasName = aliasDef.alias.trim();
if (aliasName === name) throw new Error(`Macro "${name}" options.aliases[${i}].alias cannot be the same as the macro name.`);
const visible = aliasDef.visible !== false; // Default to true
aliases.push({ alias: aliasName, visible });
}
}
if (typeof rawCategory !== 'string' || !rawCategory.trim()) throw new Error(`Macro "${name}" options.category must be a non-empty string.`);
const category = rawCategory.trim();
let minArgs = 0;
let maxArgs = 0;
/** @type {MacroUnnamedArgDef[]} */
let unnamedArgDefs = [];
if (rawUnnamedArgs !== undefined) {
if (Array.isArray(rawUnnamedArgs)) {
// Parse array of argument definitions with optional support
let foundOptional = false;
unnamedArgDefs = rawUnnamedArgs.map((def, index) => {
if (!def || typeof def !== 'object') throw new Error(`Macro "${name}" options.unnamedArgs[${index}] must be an object when using argument definitions.`);
if (typeof def.name !== 'string' || !def.name.trim()) throw new Error(`Macro "${name}" options.unnamedArgs[${index}].name must be a non-empty string when using argument definitions.`);
// Validate: no required args after optional
if (foundOptional && !def.optional) {
throw new Error(`Macro "${name}" options.unnamedArgs[${index}] is required but follows an optional argument. Optional args must be a suffix.`);
}
if (def.optional) foundOptional = true;
/** @type {MacroUnnamedArgDef} */
const normalized = {
name: def.name.trim(),
optional: def.optional || false,
defaultValue: def.defaultValue?.trim(),
type: Array.isArray(def.type) && def.type.length === 0 ? 'string' : def.type ?? 'string',
sampleValue: def.sampleValue?.trim(),
description: typeof def.description === 'string' ? def.description : undefined,
};
const validTypes = ['string', 'integer', 'number', 'boolean'];
const type = Array.isArray(normalized.type) ? normalized.type : [normalized.type];
if (type.some(t => !validTypes.includes(t))) {
throw new Error(`Macro "${name}" options.unnamedArgs[${index}].type must be one of "string", "integer", "number", or "boolean" when provided.`);
}
return normalized;
});
// Compute minArgs (required count) and maxArgs (total count)
maxArgs = unnamedArgDefs.length;
minArgs = unnamedArgDefs.findIndex(d => d.optional);
if (minArgs === -1) minArgs = maxArgs; // No optional args, all are required
} else if (typeof rawUnnamedArgs === 'number') {
if (!Number.isInteger(rawUnnamedArgs) || rawUnnamedArgs < 0) {
throw new Error(`Macro "${name}" options.unnamedArgs must be a non-negative integer when provided.`);
}
minArgs = rawUnnamedArgs;
maxArgs = rawUnnamedArgs;
unnamedArgDefs = Array.from({ length: rawUnnamedArgs }, (_, i) => ({
name: `arg${i + 1}`,
optional: false,
type: 'string',
sampleValue: `arg${i + 1}`,
}));
} else {
throw new Error(`Macro "${name}" options.unnamedArgs must be a non-negative integer or an array of argument definitions when provided.`);
}
}
/** @type {{ min: number, max: (number|null) }|null} */
let list = null;
if (rawList !== undefined) {
if (typeof rawList === 'boolean') {
list = rawList ? { min: 0, max: null } : null;
} else if (typeof rawList === 'object' && rawList !== null) {
if (typeof rawList.min !== 'number' || rawList.min < 0) throw new Error(`Macro "${name}" options.list.min must be a non-negative integer when provided.`);
if (rawList.max !== undefined && typeof rawList.max !== 'number') throw new Error(`Macro "${name}" options.list.max must be a number when provided.`);
if (rawList.max !== undefined && rawList.max < rawList.min) throw new Error(`Macro "${name}" options.list.max must be greater than or equal to options.list.min.`);
list = { min: rawList.min, max: rawList.max ?? null };
} else {
throw new Error(`Macro "${name}" options.list must be a boolean or an object with numeric min/max when provided.`);
}
}
let strictArgs = true;
if (rawStrictArgs !== undefined) {
if (typeof rawStrictArgs !== 'boolean') throw new Error(`Macro "${name}" options.strictArgs must be a boolean when provided.`);
strictArgs = rawStrictArgs;
}
let description = '<no description>';
if (rawDescription !== undefined) {
if (typeof rawDescription !== 'string') throw new Error(`Macro "${name}" options.description must be a string when provided.`);
description = rawDescription;
}
let returns = null;
if (rawReturns !== undefined && rawReturns !== null) {
if (typeof rawReturns !== 'string') throw new Error(`Macro "${name}" options.returns must be a string when provided.`);
returns = rawReturns || '<empty string>';
}
// Process and validate returnType (defaults to 'string')
const validTypes = ['string', 'integer', 'number', 'boolean'];
let returnType = /** @type {MacroValueType|MacroValueType[]} */ ('string');
if (rawReturnType !== undefined && rawReturnType !== null) {
// Normalize to non-empty value or default
returnType = Array.isArray(rawReturnType) && rawReturnType.length === 0 ? 'string' : rawReturnType;
// Validate all types
const typesToValidate = Array.isArray(returnType) ? returnType : [returnType];
if (typesToValidate.some(t => !validTypes.includes(t))) {
throw new Error(`Macro "${name}" options.returnType must be one of "string", "integer", "number", or "boolean" (or an array of these) when provided.`);
}
}
let displayOverride = null;
if (rawDisplayOverride !== undefined && rawDisplayOverride !== null) {
if (typeof rawDisplayOverride !== 'string') throw new Error(`Macro "${name}" options.displayOverride must be a string when provided.`);
displayOverride = rawDisplayOverride.trim();
if (displayOverride && !displayOverride.startsWith('{{')) {
logMacroRegisterWarning({ macroName: name, message: `Macro "${name}" options.displayOverride should include curly braces. Auto-wrapping.` });
displayOverride = `{{${displayOverride}}}`;
}
}
/** @type {string[]} */
let exampleUsage = [];
if (rawExampleUsage !== undefined && rawExampleUsage !== null) {
const examples = Array.isArray(rawExampleUsage) ? rawExampleUsage : [rawExampleUsage];
for (const [i, ex] of examples.entries()) {
if (typeof ex !== 'string') throw new Error(`Macro "${name}" options.exampleUsage[${i}] must be a string.`);
let trimmed = ex.trim();
if (trimmed && !trimmed.startsWith('{{')) {
logMacroRegisterWarning({ macroName: name, message: `Macro "${name}" options.exampleUsage[${i}] should include curly braces. Auto-wrapping.` });
trimmed = `{{${trimmed}}}`;
}
if (trimmed) exampleUsage.push(trimmed);
}
}
if (this.#macros.has(name)) {
logMacroRegisterWarning({ macroName: name, message: `Macro "${name}" is already registered and will be overwritten.` });
}
// Detect extension/third-party status from call stack
const { isExtension, isThirdParty, source } = detectMacroSource();
/** @type {MacroDefinition} */
const definition = {
name: name,
aliases,
category,
minArgs,
maxArgs,
unnamedArgDefs,
list,
strictArgs,
description,
returns,
returnType,
displayOverride,
exampleUsage,
handler,
source: {
name: source,
isExtension,
isThirdParty,
},
aliasOf: null,
aliasVisible: null,
};
this.#macros.set(name, definition);
// Register alias entries pointing to the same definition
for (const { alias, visible } of aliases) {
if (this.#macros.has(alias)) {
logMacroRegisterWarning({ macroName: name, message: `Alias "${alias}" for macro "${name}" overwrites an existing macro.` });
}
/** @type {MacroDefinition} */
const aliasEntry = {
...definition,
name: alias, // The lookup name is the alias
aliasOf: name,
aliasVisible: visible,
};
this.#macros.set(alias, aliasEntry);
}
return definition;
} catch (error) {
logMacroRegisterError({
message: `Failed to register macro "${name}". The macro will not be available.`,
macroName: name,
error,
});
return null;
}
}
/**
* Unregisters a macro.
*
* @param {string} name - Macro name (identifier).
* @returns {boolean} True if a macro was removed.
*/
unregisterMacro(name) {
if (typeof name !== 'string' || !name.trim()) throw new Error('Macro name must be a non-empty string');
name = name.trim();
return this.#macros.delete(name);
}
/**
* Checks whether a macro with the given name is registered.
*
* @param {string} name - Macro name (identifier).
* @returns {boolean}
*/
hasMacro(name) {
if (typeof name !== 'string' || !name.trim()) return false;
name = name.trim();
return this.#macros.has(name);
}
/**
* Returns the macro definition for a given name.
*
* @param {string} name - Macro name (identifier).
* @returns {MacroDefinition|undefined}
*/
getMacro(name) {
if (typeof name !== 'string' || !name.trim()) return undefined;
name = name.trim();
return this.#macros.get(name);
}
/**
* Returns the primary (non-alias) definition for a macro.
* If given an alias name, returns the primary definition it points to.
*
* @param {string} name - Macro name or alias.
* @returns {MacroDefinition|undefined}
*/
getPrimaryMacro(name) {
const def = this.getMacro(name);
if (!def) return undefined;
return def.aliasOf ? this.getMacro(def.aliasOf) : def;
}
/**
* Returns an array of all registered macros.
*
* @param {Object} [options] - Filter options.
* @param {boolean} [options.excludeAliases=false] - If true, excludes alias entries (only returns primary definitions).
* @param {boolean} [options.excludeHiddenAliases=false] - If true, excludes alias entries where visible=false.
* @returns {MacroDefinition[]}
*/
getAllMacros({ excludeAliases = false, excludeHiddenAliases = false } = {}) {
let macros = Array.from(this.#macros.values());
if (excludeAliases) {
macros = macros.filter(m => !m.aliasOf);
} else if (excludeHiddenAliases) {
macros = macros.filter(m => !m.aliasOf || m.aliasVisible !== false);
}
return macros;
}
/**
* Executes a macro for a given call.
*
* @param {MacroCall} call - Macro call information.
* @param {Object} [options] - Additional options.
* @param {MacroDefinition} [options.defOverride] - Override the macro definition.
* @returns {string}
*/
executeMacro(call, { defOverride } = {}) {
const name = call.name;
const def = defOverride || this.getMacro(name);
if (!def) {
throw new Error(`Macro "${name}" is not registered`);
}
const args = Array.isArray(call.args) ? call.args : [];
if (!isArgsValid(def, args)) {
const expectedMin = def.list ? def.minArgs + def.list.min : def.minArgs;
const expectedMax = def.list && def.list.max !== null
? def.maxArgs + def.list.max
: (def.list ? null : def.maxArgs);
const expectation = (() => {
if (expectedMax !== null && expectedMax !== expectedMin) return `between ${expectedMin} and ${expectedMax}`;
if (expectedMax !== null && expectedMax === expectedMin) return `${expectedMin}`;
return `at least ${expectedMin}`;
})();
const message = `Macro "${def.name}" called with ${args.length} unnamed arguments but expects ${expectation}.`;
if (def.strictArgs) {
throw createMacroRuntimeError({ message, call, def });
}
logMacroRuntimeWarning({ message, call, def });
}
// Compute unnamed args (required + optional, up to maxArgs)
const unnamedArgsCount = Math.min(args.length, def.maxArgs);
const unnamedArgsValues = args.slice(0, unnamedArgsCount);
const listValues = !def.list ? null : args.length > def.maxArgs ? args.slice(def.maxArgs) : [];
// Perform best-effort type validation for documented positional arguments.
// This can throw an error if the arguments are invalid.
validateArgTypes(call, def, unnamedArgsValues);
const namedArgs = null;
/** @type {MacroExecutionContext} */
const executionContext = {
name: def.name,
args,
unnamedArgs: unnamedArgsValues,
list: listValues,
namedArgs,
raw: call.rawInner,
env: call.env,
cstNode: call.cstNode,
range: call.range,
normalize: MacroEngine.normalizeMacroResult.bind(MacroEngine),
};
const result = def.handler(executionContext);
return executionContext.normalize(result);
}
}
instance = MacroRegistry.instance;
/**
* Validates the arguments for a macro definition.
* Supports required args (minArgs), optional args (up to maxArgs), and list tail.
*
* @param {MacroDefinition} def - Macro definition.
* @param {any[]} args - Arguments to validate.
* @returns {boolean} True if the arguments are valid, false otherwise.
*/
function isArgsValid(def, args) {
const hasListArgs = def.list !== null;
// Without list: args must be between minArgs and maxArgs (inclusive)
if (!hasListArgs) {
return args.length >= def.minArgs && args.length <= def.maxArgs;
}
// With list: args must be at least minArgs + list.min
const minRequired = def.minArgs + def.list.min;
if (args.length < minRequired) return false;
// List items are everything after maxArgs positional slots
const listCount = Math.max(0, args.length - def.maxArgs);
if (def.list.max !== null && listCount > def.list.max) return false;
return true;
}
/**
* Performs type validation for unnamed positional arguments using the metadata
* defined on the macro definition. When strictArgs is true, invalid argument
* types cause an error to be thrown. When strictArgs is false, only warnings
* are logged and execution continues.
*
* @param {MacroCall} call
* @param {MacroDefinition} def
* @param {string[]} unnamedArgs
*/
function validateArgTypes(call, def, unnamedArgs) {
if (def.unnamedArgDefs.length === 0) return;
const defs = def.unnamedArgDefs;
const count = Math.min(defs.length, unnamedArgs.length);
for (let i = 0; i < count; i++) {
const argDef = defs[i];
const value = unnamedArgs[i];
if (!argDef || !argDef.type || typeof value !== 'string') {
// Misconfigured macro definition: always surface as an error.
throw new Error(`Macro "${call.name}" (position ${i + 1}) has invalid definition or type.`);
}
const types = Array.isArray(argDef.type) ? argDef.type : [argDef.type];
if (!types.some(type => isValueOfType(value, type))) {
const argName = argDef.name || `Argument ${i + 1}`;
const optionalLabel = argDef.optional ? ' (optional)' : '';
const message = `Macro "${call.name}" (position ${i + 1}${optionalLabel}) argument "${argName}" expected type ${argDef.type} but got value "${value}".`;
if (def.strictArgs) {
throw createMacroRuntimeError({ message, call, def: def });
}
logMacroRuntimeWarning({ message, call, def: def });
}
}
}
/**
* Checks whether a string value conforms to the given macro argument type.
*
* @param {string} value
* @param {MacroValueType} type
* @returns {boolean}
*/
function isValueOfType(value, type) {
const trimmed = value.trim();
if (type === 'string') {
return true;
}
if (type === 'integer') {
return /^-?\d+$/.test(trimmed);
}
if (type === 'number') {
const n = Number(trimmed);
return Number.isFinite(n);
}
if (type === 'boolean') {
return isTrueBoolean(trimmed) || isFalseBoolean(trimmed);
}
// Unknown type: treat it as invalid.
return false;
}
/**
* Detects the source of a macro registration from the call stack.
* Similar to how SlashCommandParser detects command sources.
*
* @returns {{ isExtension: boolean, isThirdParty: boolean, source: string }}
*/
function detectMacroSource() {
const stack = new Error().stack?.split('\n').map(line => line.trim()) ?? [];
const isExtension = stack.some(line => line.includes('/scripts/extensions/'));
const isThirdParty = stack.some(line => line.includes('/scripts/extensions/third-party/'));
let source = 'unknown';
if (isThirdParty) {
const match = stack.find(line => line.includes('/scripts/extensions/third-party/'));
if (match) {
source = match.replace(/^.*?\/scripts\/extensions\/third-party\/([^/]+)\/.*$/, '$1');
}
} else if (isExtension) {
const match = stack.find(line => line.includes('/scripts/extensions/'));
if (match) {
source = match.replace(/^.*?\/scripts\/extensions\/([^/]+)\/.*$/, '$1');
}
} else {
// Find the first meaningful caller outside MacroRegistry
const callerIdx = stack.findIndex(line =>
line.includes('registerMacro') && line.includes('MacroRegistry'),
);
if (callerIdx >= 0 && callerIdx + 1 < stack.length) {
const callerLine = stack[callerIdx + 1];
// Extract script path from stack frame
const scriptMatch = callerLine.match(/\/((?:scripts\/)?(?:macros\/)?[^/]+\.js)/);
if (scriptMatch) {
source = scriptMatch[1];
}
}
}
return { isExtension, isThirdParty, source };
}
+83
View File
@@ -0,0 +1,83 @@
/**
* Central entry point for the new macro system.
*
* Exposes the MacroEngine / MacroRegistry singletons and provides a
* single registerMacros() function that wires up all built-in macro
* definition sets (core, env, state, chat, time, variables, instruct).
*/
// Engine singletons and enums
import { MacroEngine } from './engine/MacroEngine.js';
import { MacroRegistry, MacroCategory, MacroValueType } from './engine/MacroRegistry.js';
import { MacroLexer } from './engine/MacroLexer.js';
import { MacroParser } from './engine/MacroParser.js';
import { MacroCstWalker } from './engine/MacroCstWalker.js';
import { MacroEnvBuilder } from './engine/MacroEnvBuilder.js';
// Macro definition groups
import { registerCoreMacros } from './definitions/core-macros.js';
import { registerEnvMacros } from './definitions/env-macros.js';
import { registerStateMacros } from './definitions/state-macros.js';
import { registerChatMacros } from './definitions/chat-macros.js';
import { registerTimeMacros } from './definitions/time-macros.js';
import { registerVariableMacros } from './definitions/variable-macros.js';
import { registerInstructMacros } from './definitions/instruct-macros.js';
// Re-export the category enum for external use
export { MacroCategory, MacroValueType };
// Re-export most-used jsdoc definitions
/** @typedef {import('./engine/MacroRegistry.js').MacroDefinitionOptions} MacroDefinitionOptions */
/** @typedef {import('./engine/MacroRegistry.js').MacroDefinition} MacroDefinition */
/** @typedef {import('./engine/MacroRegistry.js').MacroUnnamedArgDef} MacroUnnamedArgDef */
/** @typedef {import('./engine/MacroRegistry.js').MacroListSpec} MacroListSpec */
/** @typedef {import('./engine/MacroRegistry.js').MacroHandler} MacroHandler */
/** @typedef {import('./engine/MacroRegistry.js').MacroExecutionContext} MacroExecutionContext */
/** @typedef {import('chevrotain').CstNode} CstNode */
/** @typedef {import('./engine/MacroEnv.types.js').MacroEnv} MacroEnv */
/** @typedef {import('./engine/MacroEnv.types.js').MacroEnvNames} MacroEnvNames */
/** @typedef {import('./engine/MacroEnv.types.js').MacroEnvCharacter} MacroEnvCharacter */
/** @typedef {import('./engine/MacroEnv.types.js').MacroEnvSystem} MacroEnvSystem */
/** @typedef {import('./engine/MacroEnv.types.js').MacroEnvFunctions} MacroEnvFunctions */
export const macros = {
// engine singletons
engine: MacroEngine,
registry: MacroRegistry,
envBuilder: MacroEnvBuilder,
lexer: MacroLexer,
parser: MacroParser,
cstWalker: MacroCstWalker,
// enums
category: MacroCategory,
// shorthand functions
register: MacroRegistry.registerMacro.bind(MacroRegistry),
};
/**
* Registers all built-in macros in a well-defined order.
* Intended to be called once during app initialization.
*/
export function initRegisterMacros() {
// Core utilities and generic helpers
registerCoreMacros();
// Env / character / system / extras
registerEnvMacros();
// Runtime state tracking (eventSource etc.)
registerStateMacros();
// Chat/history inspection macros
registerChatMacros();
// Time / date / durations
registerTimeMacros();
// Variable and instruct macros
registerVariableMacros();
registerInstructMacros();
}
+7
View File
@@ -305,6 +305,7 @@ export const power_user = {
custom_stopping_strings_macro: true,
fuzzy_search: false,
encode_tags: false,
experimental_macro_engine: false,
servers: [],
bogus_folders: false,
zoomed_avatar_magnification: false,
@@ -1664,6 +1665,7 @@ export async function loadPowerUserSettings(settings, data) {
$('#persona_allow_multi_connections').prop('checked', power_user.persona_allow_multi_connections);
$('#persona_auto_lock').prop('checked', power_user.persona_auto_lock);
$('#encode_tags').prop('checked', power_user.encode_tags);
$('#experimental_macro_engine').prop('checked', power_user.experimental_macro_engine);
$('#example_messages_behavior').val(getExampleMessagesBehavior());
$(`#example_messages_behavior option[value="${getExampleMessagesBehavior()}"]`).prop('selected', true);
$('#instruct_derived').parent().find('i').toggleClass('toggleEnabled', !!power_user.instruct_derived);
@@ -4009,6 +4011,11 @@ jQuery(() => {
saveSettingsDebounced();
});
$('#experimental_macro_engine').on('input', function () {
power_user.experimental_macro_engine = !!$(this).prop('checked');
saveSettingsDebounced();
});
$('#disable_group_trimming').on('input', function () {
power_user.disable_group_trimming = !!$(this).prop('checked');
saveSettingsDebounced();
+16 -4
View File
@@ -4,7 +4,7 @@ import {
import { chat, closeMessageEditor, event_types, eventSource, main_api, messageFormatting, saveChatConditional, saveChatDebounced, saveSettingsDebounced, substituteParams, syncMesToSwipe, updateMessageBlock } from '../script.js';
import { getRegexedString, regex_placement } from './extensions/regex/engine.js';
import { getCurrentLocale, t, translate } from './i18n.js';
import { MacrosParser } from './macros.js';
import { macros, MacroCategory } from './macros/macro-system.js';
import { chat_completion_sources, getChatCompletionModel, oai_settings } from './openai.js';
import { Popup } from './popup.js';
import { performFuzzySearch, power_user } from './power-user.js';
@@ -1000,9 +1000,21 @@ function registerReasoningSlashCommands() {
}
function registerReasoningMacros() {
MacrosParser.registerMacro('reasoningPrefix', () => power_user.reasoning.prefix, t`Reasoning Prefix`);
MacrosParser.registerMacro('reasoningSuffix', () => power_user.reasoning.suffix, t`Reasoning Suffix`);
MacrosParser.registerMacro('reasoningSeparator', () => power_user.reasoning.separator, t`Reasoning Separator`);
macros.register('reasoningPrefix', {
category: MacroCategory.PROMPTS,
description: t`The prefix string used before reasoning blocks`,
handler: () => power_user.reasoning.prefix,
});
macros.register('reasoningSuffix', {
category: MacroCategory.PROMPTS,
description: t`The suffix string used after reasoning blocks`,
handler: () => power_user.reasoning.suffix,
});
macros.register('reasoningSeparator', {
category: MacroCategory.PROMPTS,
description: t`The separator between thinking content and response`,
handler: () => power_user.reasoning.separator,
});
}
function setReasoningEventHandlers() {
+12 -10
View File
@@ -5613,13 +5613,15 @@ export async function setSlashCommandAutoComplete(textarea, isFloating = false)
);
return ac;
}
/**@type {HTMLTextAreaElement} */
const sendTextarea = document.querySelector('#send_textarea');
setSlashCommandAutoComplete(sendTextarea);
sendTextarea.addEventListener('input', () => {
if (sendTextarea.value[0] == '/') {
sendTextarea.style.fontFamily = 'var(--monoFontFamily, monospace)';
} else {
sendTextarea.style.fontFamily = null;
}
});
export async function initSlashCommandAutoComplete() {
const sendTextarea = /** @type {HTMLTextAreaElement} */ (document.querySelector('#send_textarea'));
setSlashCommandAutoComplete(sendTextarea);
sendTextarea.addEventListener('input', () => {
if (sendTextarea.value && sendTextarea.value[0] == '/') {
sendTextarea.style.fontFamily = 'var(--monoFontFamily, monospace)';
} else {
sendTextarea.style.fontFamily = null;
}
});
}
@@ -15,13 +15,12 @@ import { SlashCommandAbortController } from './SlashCommandAbortController.js';
import { SlashCommandAutoCompleteNameResult } from './SlashCommandAutoCompleteNameResult.js';
import { SlashCommandUnnamedArgumentAssignment } from './SlashCommandUnnamedArgumentAssignment.js';
import { SlashCommandEnumValue } from './SlashCommandEnumValue.js';
import { MacroAutoCompleteOption } from '../autocomplete/MacroAutoCompleteOption.js';
import { EnhancedMacroAutoCompleteOption, parseMacroContext } from '../autocomplete/EnhancedMacroAutoCompleteOption.js';
import { SlashCommandBreakPoint } from './SlashCommandBreakPoint.js';
import { SlashCommandDebugController } from './SlashCommandDebugController.js';
import { commonEnumProviders } from './SlashCommandCommonEnumsProvider.js';
import { SlashCommandBreak } from './SlashCommandBreak.js';
import { MacrosParser } from '../macros.js';
import { t } from '../i18n.js';
import { macros as macroSystem } from '../macros/macro-system.js';
/** @typedef {import('./SlashCommand.js').NamedArgumentsCapture} NamedArgumentsCapture */
/** @typedef {import('./SlashCommand.js').NamedArguments} NamedArguments */
@@ -489,23 +488,21 @@ export class SlashCommandParser {
if (childClosure !== null) return null;
const macro = this.macroIndex.findLast(it=>it.start <= index && it.end >= index);
if (macro) {
const frag = document.createRange().createContextualFragment(await (await fetch('/scripts/templates/macros.html')).text());
const options = [...frag.querySelectorAll('ul:nth-of-type(2n+1) > li')].map(li=>new MacroAutoCompleteOption(
li.querySelector('tt').textContent.slice(2, -2).replace(/^([^\s:]+[\s:]+).*$/, '$1'),
li.querySelector('tt').textContent,
(li.querySelector('tt').remove(),li.innerHTML),
));
for (const macro of MacrosParser) {
if (options.find(it => it.name === macro.key)) continue;
options.push(new MacroAutoCompleteOption(macro.key, `{{${macro.key}}}`, macro.description || t`No description provided`));
}
// Calculate cursor position within the macro for argument context
const cursorInMacro = index - macro.start - 2; // -2 for {{
const macroContent = text.slice(macro.start + 2, macro.end - (text.slice(macro.end - 2, macro.end) === '}}' ? 2 : 0));
const context = parseMacroContext(macroContent, cursorInMacro);
// Extract just the identifier (strip trailing colons/whitespace/closing braces from macro.name)
const identifier = macro.name.replace(/[\s:}]+$/, '').trim();
// Use enhanced macro autocomplete when experimental engine is enabled
const options = this.#buildEnhancedMacroOptions(context);
const result = new AutoCompleteNameResult(
macro.name,
identifier,
macro.start + 2,
options,
false,
()=>`No matching macros for "{{${result.name}}}"`,
()=>'No macros found.',
);
return result;
}
@@ -541,6 +538,44 @@ export class SlashCommandParser {
return null;
}
/**
* Builds enhanced macro autocomplete options from the MacroRegistry.
* When typing arguments (after ::), prioritizes the exact macro match.
* @param {import('../autocomplete/EnhancedMacroAutoCompleteOption.js').MacroAutoCompleteContext} context
* @returns {EnhancedMacroAutoCompleteOption[]}
*/
#buildEnhancedMacroOptions(context) {
/** @type {EnhancedMacroAutoCompleteOption[]} */
const options = [];
// Get all macros from the registry (excluding hidden aliases)
const allMacros = macroSystem.registry.getAllMacros({ excludeHiddenAliases: true });
// If we're typing arguments (after ::), only show the context to the matching macro
const isTypingArgs = context.currentArgIndex >= 0;
for (const macro of allMacros) {
// Check if this macro matches the typed identifier
const isExactMatch = macro.name === context.identifier;
const isAliasMatch = macro.aliasOf === context.identifier;
// Only pass context to the macro that matches the identifier being typed
// This ensures argument hints only show for the relevant macro
const macroContext = (isExactMatch || isAliasMatch) ? context : null;
const option = new EnhancedMacroAutoCompleteOption(macro, macroContext);
// When typing arguments, prioritize exact matches by putting them first
if (isTypingArgs && (isExactMatch || isAliasMatch)) {
options.unshift(option);
} else {
options.push(option);
}
}
return options;
}
/**
* Moves the index <length> number of characters forward and returns the last character taken.
* @param {number} length Number of characters to take.
+13 -1
View File
@@ -95,12 +95,13 @@ import { tokenizers, getTextTokens, getTokenCount, getTokenCountAsync, getTokeni
import { ToolManager } from './tool-calling.js';
import { accountStorage } from './util/AccountStorage.js';
import { timestampToMoment, uuidv4 } from './utils.js';
import { getGlobalVariable, getLocalVariable, setGlobalVariable, setLocalVariable } from './variables.js';
import { addGlobalVariable, addLocalVariable, decrementGlobalVariable, decrementLocalVariable, deleteGlobalVariable, deleteLocalVariable, getGlobalVariable, getLocalVariable, incrementGlobalVariable, incrementLocalVariable, setGlobalVariable, setLocalVariable } from './variables.js';
import { convertCharacterBook, getWorldInfoPrompt, loadWorldInfo, reloadEditor, saveWorldInfo, updateWorldInfoList } from './world-info.js';
import { ChatCompletionService, TextCompletionService } from './custom-request.js';
import { ConnectionManagerRequestService } from './extensions/shared.js';
import { updateReasoningUI, parseReasoningFromString, getReasoningTemplateByName } from './reasoning.js';
import { IGNORE_SYMBOL } from './constants.js';
import { macros } from './macros/macro-system.js';
export function getContext() {
return {
@@ -165,7 +166,9 @@ export function getContext() {
timestampToMoment,
/** @deprecated Handlebars for extensions are no longer supported. */
registerHelper: () => { },
/** @deprecated Use `macros.register(name, { handler, description })` from scripts/macros/macro-system.js instead. */
registerMacro: MacrosParser.registerMacro.bind(MacrosParser),
/** @deprecated Use `macros.registry.unregisterMacro(name)` from scripts/macros/macro-system.js instead. */
unregisterMacro: MacrosParser.unregisterMacro.bind(MacrosParser),
registerFunctionTool: ToolManager.registerFunctionTool.bind(ToolManager),
unregisterFunctionTool: ToolManager.unregisterFunctionTool.bind(ToolManager),
@@ -221,6 +224,7 @@ export function getContext() {
getMediaIndex,
scrollChatToBottom,
scrollOnMediaLoad,
macros,
swipe: {
left: swipe_left,
right: swipe_right,
@@ -235,10 +239,18 @@ export function getContext() {
local: {
get: getLocalVariable,
set: setLocalVariable,
del: deleteLocalVariable,
add: addLocalVariable,
inc: incrementLocalVariable,
dec: decrementLocalVariable,
},
global: {
get: getGlobalVariable,
set: setGlobalVariable,
del: deleteGlobalVariable,
add: addGlobalVariable,
inc: incrementGlobalVariable,
dec: decrementGlobalVariable,
},
},
loadWorldInfo,
+21 -1
View File
@@ -4,6 +4,7 @@ import { t } from './i18n.js';
import { getMessageTimeStamp } from './RossAscends-mods.js';
import { getSlashCommandsHelp } from './slash-commands.js';
import { SlashCommandBrowser } from './slash-commands/SlashCommandBrowser.js';
import { MacroBrowser, getMacrosHelp } from './macros/MacroBrowser.js';
import { renderTemplateAsync } from './templates.js';
/** @type {Record<string, ChatMessage>} */
@@ -59,7 +60,7 @@ export async function initSystemMessages() {
}),
/** @type {ChatMessage} */
macros: lodash.merge(structuredClone(defaultMessage), {
mes: await renderTemplateAsync('macros'),
mes: '',
}),
/** @type {ChatMessage} */
welcome: lodash.merge(structuredClone(defaultMessage), {
@@ -135,6 +136,14 @@ export function getSystemMessageByType(type, text, extra = {}) {
newMessage.mes = getSlashCommandsHelp();
}
if (type === system_message_types.MACROS) {
newMessage.mes = getMacrosHelp();
}
if (!newMessage.extra || typeof newMessage.extra !== 'object') {
newMessage.extra = {};
}
newMessage.extra = Object.assign(newMessage.extra, extra);
newMessage.extra.type = type;
return newMessage;
@@ -159,4 +168,15 @@ export function sendSystemMessage(type, text, extra = {}) {
browser.renderInto(parent);
browser.search.focus();
}
if (type === system_message_types.MACROS) {
const browser = new MacroBrowser();
const spinner = document.querySelector('#chat .last_mes .custom-macroHelp');
if (spinner) {
const parent = spinner.parentElement;
spinner.remove();
browser.renderInto(parent);
browser.searchInput?.focus();
}
}
}
-104
View File
@@ -1,104 +0,0 @@
<div data-i18n="System-wide Replacement Macros (in order of evaluation):">
System-wide Replacement Macros (in order of evaluation):
</div>
<ul>
<li><tt>&lcub;&lcub;pipe&rcub;&rcub;</tt> <span data-i18n="help_macros_1">only for slash command batching. Replaced with the returned result of the previous command.</span></li>
<li><tt>&lcub;&lcub;newline&rcub;&rcub;</tt> <span data-i18n="help_macros_2">just inserts a newline.</span></li>
<li><tt>&lcub;&lcub;trim&rcub;&rcub;</tt> <span data-i18n="help_macros_3">trims newlines surrounding this macro.</span></li>
<li><tt>&lcub;&lcub;noop&rcub;&rcub;</tt> <span data-i18n="help_macros_4">no operation, just an empty string.</span></li>
<li><tt>&lcub;&lcub;original&rcub;&rcub;</tt> <span data-i18n="help_macros_5">global prompts defined in API settings. Only valid in Advanced Definitions prompt overrides.</span></li>
<li><tt>&lcub;&lcub;input&rcub;&rcub;</tt> <span data-i18n="help_macros_6">the user input</span></li>
<li><tt>&lcub;&lcub;lastGenerationType&rcub;&rcub;</tt> - <span>the type of the last queued generation request. Empty if no generations were performed yet or the active chat was switched. Possible values: "normal", "impersonate", "regenerate", "quiet", "swipe", "continue".</span></li>
<li><tt>&lcub;&lcub;charPrompt&rcub;&rcub;</tt> <span data-i18n="help_macros_7">the Character's Main Prompt override</span></li>
<li><tt>&lcub;&lcub;charInstruction&rcub;&rcub;</tt> <span data-i18n="help_macros_8">the Character's Post-History Instructions override</span></li>
<li><tt>&lcub;&lcub;description&rcub;&rcub;</tt> <span data-i18n="help_macros_9">the Character's Description</span></li>
<li><tt>&lcub;&lcub;personality&rcub;&rcub;</tt> <span data-i18n="help_macros_10">the Character's Personality</span></li>
<li><tt>&lcub;&lcub;scenario&rcub;&rcub;</tt> <span data-i18n="help_macros_11">the Character's Scenario</span></li>
<li><tt>&lcub;&lcub;persona&rcub;&rcub;</tt> <span data-i18n="help_macros_12">your current Persona Description</span></li>
<li><tt>&lcub;&lcub;mesExamples&rcub;&rcub;</tt> <span data-i18n="help_macros_13">the Character's Dialogue Examples</span></li>
<li><tt>&lcub;&lcub;mesExamplesRaw&rcub;&rcub;</tt> <span data-i18n="help_macros_14">unformatted Dialogue Examples</span></li>
<li><tt>&lcub;&lcub;summary&rcub;&rcub;</tt> <span data-i18n="help_macros_summary">the latest chat summary generated by the "Summarize" extension (if available).</span></li>
<li><tt>&lcub;&lcub;user&rcub;&rcub;</tt> <span data-i18n="help_macros_15">your current Persona username</span></li>
<li><tt>&lcub;&lcub;char&rcub;&rcub;</tt> <span data-i18n="help_macros_16">the Character's name</span></li>
<li><tt>&lcub;&lcub;version&rcub;&rcub;</tt> <span data-i18n="help_macros_17">the Character's version number</span></li>
<li><tt>&lcub;&lcub;charDepthPrompt&rcub;&rcub;</tt> <span data-i18n="help_macros_charDepthPrompt">the Character's @ Depth Note</span></li>
<li><tt>&lcub;&lcub;outlet::(name)&rcub;&rcub;</tt> <span data-i18n="help_macros_outletName">the WI entry content for the outlet with the specified name</span></li>
<li><tt>&lcub;&lcub;group&rcub;&rcub;</tt> <span data-i18n="help_macros_18">a comma-separated list of group member names (including muted) or the character name in solo chats. Alias: &lcub;&lcub;charIfNotGroup&rcub;&rcub;</span></li>
<li><tt>&lcub;&lcub;groupNotMuted&rcub;&rcub;</tt> <span data-i18n="help_groupNotMuted">the same as &lcub;&lcub;group&rcub;&rcub;, but excludes muted members</span></li>
<li><tt>&lcub;&lcub;notChar&rcub;&rcub;</tt> <span data-i18n="help_notChar">a comma-separated list of all participants in the conversation except for the current speaker (&lcub;&lcub;char&rcub;&rcub;). In group chats, this includes muted characters. When not in a generation, the list include all characters.</span></li>
<li><tt>&lcub;&lcub;model&rcub;&rcub;</tt> <span data-i18n="help_macros_19">a text generation model name for the currently selected API. </span><b data-i18n="Can be inaccurate!">Can be inaccurate!</b></li>
<li><tt>&lcub;&lcub;lastMessage&rcub;&rcub;</tt> <span data-i18n="help_macros_20">the text of the latest chat message.</span></li>
<li><tt>&lcub;&lcub;lastUserMessage&rcub;&rcub;</tt> <span data-i18n="help_macros_lastUser">the text of the latest user chat message.</span></li>
<li><tt>&lcub;&lcub;lastCharMessage&rcub;&rcub;</tt> <span data-i18n="help_macros_lastChar">the text of the latest character chat message.</span></li>
<li><tt>&lcub;&lcub;lastMessageId&rcub;&rcub;</tt> <span data-i18n="help_macros_21">index # of the latest chat message. Useful for slash command batching.</span></li>
<li><tt>&lcub;&lcub;firstIncludedMessageId&rcub;&rcub;</tt> <span data-i18n="help_macros_22">the ID of the first message included in the context. Requires generation to be run at least once in the current session. Will only be updated on generation.</span></li>
<li><tt>&lcub;&lcub;firstDisplayedMessageId&rcub;&rcub;</tt> <span data-i18n="help_macros_firstDisplayedMessageId">the ID of the first message loaded into the visible chat.</span></li>
<li><tt>&lcub;&lcub;currentSwipeId&rcub;&rcub;</tt> <span data-i18n="help_macros_23">the 1-based ID of the current swipe in the last chat message. Empty string if the last message is user or prompt-hidden.</span></li>
<li><tt>&lcub;&lcub;lastSwipeId&rcub;&rcub;</tt> <span data-i18n="help_macros_24">the number of swipes in the last chat message. Empty string if the last message is user or prompt-hidden.</span></li>
<li><tt>&lcub;&lcub;reverse:(content)&rcub;&rcub;</tt> <span data-i18n="help_macros_reverse">reverses the content of the macro.</span></li>
<li><tt>&lcub;&lcub;// (note)&rcub;&rcub;</tt> <span data-i18n="help_macros_25">you can leave a note here, and the macro will be replaced with blank content. Not visible for the AI.</span></li>
<li><tt>&lcub;&lcub;time&rcub;&rcub;</tt> <span data-i18n="help_macros_26">the current time</span></li>
<li><tt>&lcub;&lcub;date&rcub;&rcub;</tt> <span data-i18n="help_macros_27">the current date</span></li>
<li><tt>&lcub;&lcub;weekday&rcub;&rcub;</tt> <span data-i18n="help_macros_28">the current weekday</span></li>
<li><tt>&lcub;&lcub;isotime&rcub;&rcub;</tt> <span data-i18n="help_macros_29">the current ISO time (24-hour clock)</span></li>
<li><tt>&lcub;&lcub;isodate&rcub;&rcub;</tt> <span data-i18n="help_macros_30">the current ISO date (YYYY-MM-DD)</span></li>
<li><tt>&lcub;&lcub;datetimeformat &hellip;&rcub;&rcub;</tt> <span data-i18n="help_macros_31">the current date/time in the specified format, e. g. for German date/time: </span><tt>&lcub;&lcub;datetimeformat DD.MM.YYYY HH:mm&rcub;&rcub;</tt></li>
<li><tt>&lcub;&lcub;time_UTC±#&rcub;&rcub;</tt> <span data-i18n="help_macros_32">the current time in the specified UTC time zone offset, e.g. UTC-4 or UTC+2</span></li>
<li><tt>&lcub;&lcub;timeDiff::(time1)::(time2)&rcub;&rcub;</tt> <span data-i18n="help_macros_33">the time difference between time1 and time2. Accepts time and date macros. (Ex: &lcub;&lcub;timeDiff::&lcub;&lcub;isodate&rcub;&rcub; &lcub;&lcub;time&rcub;&rcub;::2024/5/11 12:30:00&rcub;&rcub;)</span></li>
<li><tt>&lcub;&lcub;idle_duration&rcub;&rcub;</tt> <span data-i18n="help_macros_34">the time since the last user message was sent</span></li>
<li><tt>&lcub;&lcub;bias "text here"&rcub;&rcub;</tt> <span data-i18n="help_macros_35">sets a behavioral bias for the AI until the next user input. Quotes around the text are important.</span></li>
<li><tt>&lcub;&lcub;roll:(formula)&rcub;&rcub;</tt> <span data-i18n="help_macros_36">rolls a dice. (ex: </span><tt>&lcub;&lcub;roll:1d6&rcub;&rcub;</tt><span data-i18n="space_ will roll a 6-sided dice and return a number between 1 and 6)"> will roll a 6-sided dice and return a number between 1 and 6)</span></li>
<li><tt>&lcub;&lcub;random:(args)&rcub;&rcub;</tt> <span data-i18n="help_macros_37">returns a random item from the list. (ex: </span><tt>&lcub;&lcub;random:1,2,3,4&rcub;&rcub;</tt><span data-i18n="space_ will return 1 of the 4 numbers at random. Works with text lists too."> will return 1 of the 4 numbers at random. Works with text lists too.</span></li>
<li><tt>&lcub;&lcub;random::(arg1)::(arg2)&rcub;&rcub;</tt> <span data-i18n="help_macros_38">alternative syntax for random that allows to use commas in the list items.</span></li>
<li><tt>&lcub;&lcub;pick::(args)&rcub;&rcub;</tt> <span data-i18n="help_macros_39">picks a random item from the list. Works the same as &lcub;&lcub;random&rcub;&rcub;, with the same possible syntax options, but the pick will stay consistent for this chat once picked and won't be re-rolled on consecutive messages and prompt processing.</span></li>
<li><tt>&lcub;&lcub;banned "text here"&rcub;&rcub;</tt> <span data-i18n="help_macros_40">dynamically add text in the quotes to banned words sequences, if Text Generation WebUI backend used. Do nothing for others backends. Can be used anywhere (Character description, WI, AN, etc.) Quotes around the text are important.</span></li>
<li><tt>&lcub;&lcub;isMobile&rcub;&rcub;</tt> <span data-i18n="help_macros_isMobile">"true" if currently running in a mobile environment, "false" otherwise</span></li>
</ul>
<div data-i18n="Instruct Mode and Context Template Macros:">
Instruct Mode and Context Template Macros:
</div>
<div>
<small data-i18n="(enabled in the Advanced Formatting settings)">(enabled in the Advanced Formatting settings)</small>
</div>
<ul>
<li><tt>&lcub;&lcub;maxPrompt&rcub;&rcub;</tt> <span data-i18n="help_macros_41">max allowed prompt length in tokens = (context size - response length)</span></li>
<li><tt>&lcub;&lcub;exampleSeparator&rcub;&rcub;</tt> <span data-i18n="help_macros_42">context template example dialogues separator</span></li>
<li><tt>&lcub;&lcub;chatStart&rcub;&rcub;</tt> <span data-i18n="help_macros_43">context template chat start line</span></li>
<li><tt>&lcub;&lcub;systemPrompt&rcub;&rcub;</tt> <span data-i18n="help_macros_44">system prompt content if enabled (either character prompt override if allowed, or defaultSystemPrompt)</span></li>
<li><tt>&lcub;&lcub;defaultSystemPrompt&rcub;&rcub;</tt> <span data-i18n="help_macros_45">system prompt content</span></li>
<li><tt>&lcub;&lcub;instructStoryStringPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_46">instruct story string prefix sequence</span></li>
<li><tt>&lcub;&lcub;instructStoryStringSuffix&rcub;&rcub;</tt> <span data-i18n="help_macros_47">instruct story string suffix sequence</span></li>
<li><tt>&lcub;&lcub;instructUserPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_48">instruct user prefix sequence</span></li>
<li><tt>&lcub;&lcub;instructUserSuffix&rcub;&rcub;</tt> <span data-i18n="help_macros_49">instruct user suffix sequence</span></li>
<li><tt>&lcub;&lcub;instructAssistantPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_50">instruct assistant prefix sequence</span></li>
<li><tt>&lcub;&lcub;instructAssistantSuffix&rcub;&rcub;</tt> <span data-i18n="help_macros_51">instruct assistant suffix sequence</span></li>
<li><tt>&lcub;&lcub;instructFirstAssistantPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_52">instruct assistant first output sequence</span></li>
<li><tt>&lcub;&lcub;instructLastAssistantPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_53">instruct assistant last output sequence</span></li>
<li><tt>&lcub;&lcub;instructSystemPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_54">instruct system message prefix sequence</span></li>
<li><tt>&lcub;&lcub;instructSystemSuffix&rcub;&rcub;</tt> <span data-i18n="help_macros_55">instruct system message suffix sequence</span></li>
<li><tt>&lcub;&lcub;instructSystemInstructionPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_56">instruct system instruction prefix</span></li>
<li><tt>&lcub;&lcub;instructUserFiller&rcub;&rcub;</tt> <span data-i18n="help_macros_57">instruct first user message filler</span></li>
<li><tt>&lcub;&lcub;instructStop&rcub;&rcub;</tt> <span data-i18n="help_macros_58">instruct stop sequence</span></li>
<li><tt>&lcub;&lcub;instructFirstUserPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_first_user">instruct user first input sequence</span></li>
<li><tt>&lcub;&lcub;instructLastUserPrefix&rcub;&rcub;</tt> <span data-i18n="help_macros_last_user">instruct user last input sequence</span></li>
</ul>
<div data-i18n="Chat variables Macros:">
Chat variables Macros:
</div>
<div><small data-i18n="Local variables = unique to the current chat">Local variables = unique to the current chat</small></div>
<div><small data-i18n="Global variables = works in any chat for any character">Global variables = works in any chat for any character</small></div>
<div><small data-i18n="Scoped variables = works in STscript">Scoped variables = works in STscript</small></div>
<ul>
<li><tt>&lcub;&lcub;getvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_59">replaced with the value of the local variable "name"</span></li>
<li><tt>&lcub;&lcub;setvar::name::value&rcub;&rcub;</tt> <span data-i18n="help_macros_60">replaced with empty string, sets the local variable "name" to "value"</span></li>
<li><tt>&lcub;&lcub;addvar::name::increment&rcub;&rcub;</tt> <span data-i18n="help_macros_61">replaced with empty strings, adds a numeric value of "increment" to the local variable "name"</span></li>
<li><tt>&lcub;&lcub;incvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_62">replaced with the result of the increment of value of the variable "name" by 1</span></li>
<li><tt>&lcub;&lcub;decvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_63">replaced with the result of the decrement of value of the variable "name" by 1</span></li>
<li><tt>&lcub;&lcub;getglobalvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_64">replaced with the value of the global variable "name"</span></li>
<li><tt>&lcub;&lcub;setglobalvar::name::value&rcub;&rcub;</tt> <span data-i18n="help_macros_65">replaced with empty string, sets the global variable "name" to "value"</span></li>
<li><tt>&lcub;&lcub;addglobalvar::name::value&rcub;&rcub;</tt> <span data-i18n="help_macros_66">replaced with empty string, adds a numeric value of "increment" to the global variable "name"</span></li>
<li><tt>&lcub;&lcub;incglobalvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_67">replaced with the result of the increment of value of the global variable "name" by 1</span></li>
<li><tt>&lcub;&lcub;decglobalvar::name&rcub;&rcub;</tt> <span data-i18n="help_macros_68">replaced with the result of the decrement of value of the global variable "name" by 1</span></li>
<li><tt>&lcub;&lcub;var::name&rcub;&rcub;</tt> <span data-i18n="help_macros_69">replaced with the value of the scoped variable "name"</span></li>
<li><tt>&lcub;&lcub;var::name::index&rcub;&rcub;</tt> <span data-i18n="help_macros_70">replaced with the value of item at index (for arrays / lists or objects / dictionaries) of the scoped variable "name"</span></li>
</ul>
+9 -9
View File
@@ -133,7 +133,7 @@ export function setGlobalVariable(name, value, args = {}) {
return value;
}
function addLocalVariable(name, value) {
export function addLocalVariable(name, value) {
const currentValue = getLocalVariable(name) || 0;
try {
const parsedValue = JSON.parse(currentValue);
@@ -163,7 +163,7 @@ function addLocalVariable(name, value) {
return newValue;
}
function addGlobalVariable(name, value) {
export function addGlobalVariable(name, value) {
const currentValue = getGlobalVariable(name) || 0;
try {
const parsedValue = JSON.parse(currentValue);
@@ -193,19 +193,19 @@ function addGlobalVariable(name, value) {
return newValue;
}
function incrementLocalVariable(name) {
export function incrementLocalVariable(name) {
return addLocalVariable(name, 1);
}
function incrementGlobalVariable(name) {
export function incrementGlobalVariable(name) {
return addGlobalVariable(name, 1);
}
function decrementLocalVariable(name) {
export function decrementLocalVariable(name) {
return addLocalVariable(name, -1);
}
function decrementGlobalVariable(name) {
export function decrementGlobalVariable(name) {
return addGlobalVariable(name, -1);
}
@@ -408,7 +408,7 @@ async function ifCallback(args, value) {
const { a, b, rule } = parseBooleanOperands(args);
const result = evalBoolean(rule, a, b);
/**@type {string|SlashCommandClosure} */
/** @type {string|SlashCommandClosure} */
let command;
if (value) {
if (value[0] instanceof SlashCommandClosure) {
@@ -608,7 +608,7 @@ async function executeSubCommands(command, scope = null, parserFlags = null, abo
* @param {string} name Variable name to delete
* @returns {string} Empty string
*/
function deleteLocalVariable(name) {
export function deleteLocalVariable(name) {
if (!existsLocalVariable(name)) {
console.warn(`The local variable "${name}" does not exist.`);
return '';
@@ -624,7 +624,7 @@ function deleteLocalVariable(name) {
* @param {string} name Variable name to delete
* @returns {string} Empty string
*/
function deleteGlobalVariable(name) {
export function deleteGlobalVariable(name) {
if (!existsGlobalVariable(name)) {
console.warn(`The global variable "${name}" does not exist.`);
return '';
+748
View File
@@ -0,0 +1,748 @@
import { test, expect } from '@playwright/test';
import { testSetup } from './frontent-test-utils.js';
test.describe('MacroEngine', () => {
test.beforeEach(testSetup.awaitST);
test.describe('Basic evaluation', () => {
test('should return input unchanged when there are no macros', async ({ page }) => {
const input = 'Hello world, no macros here.';
const output = await evaluateWithEngine(page, input);
expect(output).toBe(input);
});
test('should evaluate a simple macro without arguments', async ({ page }) => {
const input = 'Start {{newline}} end.';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Start \n end.');
});
test('should evaluate multiple macros in order', async ({ page }) => {
const input = 'A {{setvar::test::4}}{{getvar::test}} B {{setvar::test::2}}{{getvar::test}} C';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('A 4 B 2 C');
});
});
test.describe('Unnamed arguments', () => {
test('should handle normal double-colon separated unnamed argument', async ({ page }) => {
const input = 'Reversed: {{reverse::abc}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Reversed: cba!');
});
test('should handle (legacy) colon separated unnamed argument', async ({ page }) => {
const input = 'Reversed: {{reverse:abc}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Reversed: cba!');
});
test('should handle (legacy) colon separated argument as only one, even with more separators (double colon)', async ({ page }) => {
const input = 'Reversed: {{reverse:abc::def}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Reversed: fed::cba!');
});
test('should handle (legacy) colon separated argument as only one, even with more separators (single colon)', async ({ page }) => {
const input = 'Reversed: {{reverse:abc:def}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Reversed: fed:cba!');
});
test('should handle (legacy) whitespace separated unnamed argument', async ({ page }) => {
const input = 'Values: {{roll 1d1}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Values: 1!');
});
test('should handle (legacy) whitespace separated unnamed argument as only one, even with more separators (space)', async ({ page }) => {
const input = 'Values: {{reverse abc def}}!';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Values: fed cba!');
});
test('should support multi-line arguments for macros', async ({ page }) => {
const input = 'Result: {{reverse::first line\nsecond line}}'; // "\n" becomes a real newline in the macro argument
const output = await evaluateWithEngine(page, input);
const original = 'first line\nsecond line';
const expectedReversed = Array.from(original).reverse().join('');
expect(output).toBe(`Result: ${expectedReversed}`);
});
});
test.describe('Nested macros', () => {
test('should resolve nested macros inside arguments inside-out', async ({ page }) => {
const input = 'Result: {{setvar::test::0}}{{reverse::{{addvar::test::100}}{{getvar::test}}}}{{setvar::test::0}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Result: 001');
});
// {{wrap::{{upper::x}}::[::]}} -> '[X]'
test('should resolve nested macros across multiple arguments', async ({ page }) => {
const input = 'Result: {{setvar::addvname::test}}{{addvar::{{getvar::addvname}}::{{setvar::test::5}}{{getvar::test}}}}{{getvar::test}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Result: 10');
});
});
test.describe('Unknown macros', () => {
test('should keep unknown macro syntax but resolve nested macros inside it', async ({ page }) => {
const input = 'Test: {{unknown::{{newline}}}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Test: {{unknown::\n}}');
});
test('should keep surrounding text inside unknown macros intact', async ({ page }) => {
const input = 'Test: {{unknown::my {{newline}} example}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Test: {{unknown::my \n example}}');
});
});
test.describe('Comment macro', () => {
test('should remove single-line comments with simple body', async ({ page }) => {
const input = 'Hello{{// comment}}World';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('HelloWorld');
});
test('should accept non-word characters immediately after //', async ({ page }) => {
const input = 'A{{//!@#$%^&*()_+}}B';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('AB');
});
test('should ignore additional // sequences inside the comment body', async ({ page }) => {
const input = 'X{{//comment with // extra // slashes}}Y';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('XY');
});
test('should support multi-line comment bodies', async ({ page }) => {
const input = 'Start{{// line one\nline two\nline three}}End';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('StartEnd');
});
});
test.describe('Legacy compatibility', () => {
test('should strip trim macro and surrounding newlines (legacy behavior)', async ({ page }) => {
const input = 'foo\n\n{{trim}}\n\nbar';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('foobar');
});
test('should handle multiple trim macros in a single string', async ({ page }) => {
const input = 'A\n\n{{trim}}\n\nB\n\n{{trim}}\n\nC';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('ABC');
});
test('should support legacy time macro with positive offset via pre-processing', async ({ page }) => {
const input = 'Time: {{time_UTC+2}}';
const output = await evaluateWithEngine(page, input);
// After pre-processing, this should behave like {{time::UTC+2}} and be resolved by the time macro.
// We only assert that the placeholder was consumed and some non-empty value was produced.
expect(output).not.toBe(input);
expect(output.startsWith('Time: ')).toBeTruthy();
expect(output.length).toBeGreaterThan('Time: '.length);
});
test('should support legacy time macro with negative offset via pre-processing', async ({ page }) => {
const input = 'Time: {{time_UTC-10}}';
const output = await evaluateWithEngine(page, input);
expect(output).not.toBe(input);
expect(output.startsWith('Time: ')).toBeTruthy();
expect(output.length).toBeGreaterThan('Time: '.length);
});
test('should support legacy <USER> marker via pre-processing', async ({ page }) => {
const input = 'Hello <USER>!';
const output = await evaluateWithEngine(page, input);
// In the default test env, name1Override is "User".
expect(output).toBe('Hello User!');
});
test('should support legacy <BOT> and <CHAR> markers via pre-processing', async ({ page }) => {
const input = 'Bot: <BOT>, Char: <CHAR>.';
const output = await evaluateWithEngine(page, input);
// In the default test env, name2Override is "Character".
expect(output).toBe('Bot: Character, Char: Character.');
});
test('should support legacy <GROUP> and <CHARIFNOTGROUP> markers via pre-processing (non-group fallback)', async ({ page }) => {
const input = 'Group: <GROUP>, CharIfNotGroup: <CHARIFNOTGROUP>.';
const output = await evaluateWithEngine(page, input);
// Without an active group, both markers fall back to the current character name.
expect(output).toBe('Group: Character, CharIfNotGroup: Character.');
});
});
test.describe('Bracket handling around macros', () => {
test('should allow single opening brace inside macro arguments', async ({ page }) => {
const input = 'Test§ {{reverse::my { test}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// "my { test" reversed becomes "tset { ym"
expect(output).toBe('Test§ tset { ym');
const EXPECT_WARNINGS = false;
const EXPECT_ERRORS = false;
expect(hasMacroWarnings).toBe(EXPECT_WARNINGS);
expect(hasMacroErrors).toBe(EXPECT_ERRORS);
});
test('should allow single closing brace inside macro arguments', async ({ page }) => {
const input = 'Test§ {{reverse::my } test}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// "my } test" reversed becomes "tset } ym"
expect(output).toBe('Test§ tset } ym');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should treat unterminated macro with identifier at end of input as plain text', async ({ page }) => {
const input = 'Test {{ hehe';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe(input);
expect(hasMacroWarnings).toBe(true);
expect(hasMacroErrors).toBe(false);
});
test('should treat invalid macro start as plain text when followed by non-identifier characters', async ({ page }) => {
const input = 'Test {{§§ hehe';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe(input);
expect(hasMacroWarnings).toBe(false); // Doesn't even try to recognize this as a macro, doesn't look like one. No warning is fine
expect(hasMacroErrors).toBe(false);
});
test('should treat unterminated macro in the middle of the string as plain text', async ({ page }) => {
const input = 'Before {{ hehe After';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe(input);
expect(hasMacroWarnings).toBe(true);
expect(hasMacroErrors).toBe(false);
});
test('should treat dangling macro start as text and still evaluate subsequent macro', async ({ page }) => {
const input = 'Test {{ hehe {{user}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// Default test env uses name1Override = "User" and name2Override = "Character".
expect(output).toBe('Test {{ hehe User');
expect(hasMacroWarnings).toBe(true);
expect(hasMacroErrors).toBe(false);
});
test('should ignore invalid macro start but still evaluate following valid macro', async ({ page }) => {
const input = 'Test {{&& hehe {{user}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// Default test env uses name1Override = "User" and name2Override = "Character".
expect(output).toBe('Test {{&& hehe User');
expect(hasMacroWarnings).toBe(false); // Doesn't even try to recognize this as a macro, doesn't look like one. No warning is fine
expect(hasMacroErrors).toBe(false);
});
test('should allow single opening brace immediately before a macro', async ({ page }) => {
const input = '{{{char}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// One literal '{' plus the resolved character name.
expect(output).toBe('{Character');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should allow single closing brace immediately after a macro', async ({ page }) => {
const input = '{{char}}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Character}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should allow single braces around a macro', async ({ page }) => {
const input = '{{{char}}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('{Character}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should allow double opening braces immediately before a macro', async ({ page }) => {
const input = '{{{{char}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('{{Character');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should allow double closing braces immediately after a macro', async ({ page }) => {
const input = '{{char}}}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Character}}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should allow double braces around a macro', async ({ page }) => {
const input = '{{{{char}}}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('{{Character}}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should resolve nested macro inside argument with surrounding braces', async ({ page }) => {
const input = 'Result: {{reverse::pre-{ {{user}} }-post}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
// Argument "pre-{ User }-post" reversed becomes "tsop-} resU {-erp".
expect(output).toBe('Result: tsop-} resU {-erp');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should handle adjacent macros with no separator', async ({ page }) => {
const input = '{{char}}{{user}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('CharacterUser');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should handle macros separated only by surrounding braces', async ({ page }) => {
const input = '{{char}}{ {{user}} }';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Character{ User }');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should handle Windows newlines with braces near macros', async ({ page }) => {
const input = 'Line1 {{char}}\r\n{Line2}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Line1 Character\r\n{Line2}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should treat stray closing braces outside macros as plain text', async ({ page }) => {
const input = 'Foo }} bar';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe(input);
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should keep stray closing braces and still evaluate following macro', async ({ page }) => {
const input = 'Foo }} {{user}}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Foo }} User');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
test('should handle stray closing braces before macros as plain text', async ({ page }) => {
const input = 'Foo {{user}} }}';
const { output, hasMacroWarnings, hasMacroErrors } = await evaluateWithEngineAndCaptureMacroLogs(page, input);
expect(output).toBe('Foo User }}');
expect(hasMacroWarnings).toBe(false);
expect(hasMacroErrors).toBe(false);
});
});
test.describe('Arity errors', () => {
test('should not resolve newline when called with arguments', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
const input = 'Start {{newline::extra}} end.';
const output = await evaluateWithEngine(page, input);
// Macro text should remain unchanged
expect(output).toBe(input);
// Should have logged an arity warning for newline
expect(warnings.some(w => w.includes('Macro "newline"') && w.includes('unnamed arguments'))).toBeTruthy();
});
test('should not resolve reverse when called without arguments', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
const input = 'Result: {{reverse}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe(input);
expect(warnings.some(w => w.includes('Macro "reverse"') && w.includes('unnamed arguments'))).toBeTruthy();
});
test('should not resolve reverse when called with too many arguments', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
const input = 'Result: {{reverse::a::b}}';
const output = await evaluateWithEngine(page, input);
// Macro text should remain unchanged when extra unnamed args are provided
expect(output).toBe(input);
// Should have logged an arity warning for reverse
expect(warnings.some(w => w.includes('Macro "reverse"') && w.includes('unnamed arguments'))).toBeTruthy();
});
test('should not resolve list-bounded macro when called outside list bounds', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
// Register a temporary macro with explicit list bounds: exactly 1 required + 1-2 list args
await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
MacroRegistry.unregisterMacro('test-list-bounds');
MacroRegistry.registerMacro('test-list-bounds', {
unnamedArgs: 1,
list: { min: 1, max: 2 },
description: 'Test macro for list bounds.',
handler: ({ unnamedArgs, list }) => {
const all = [...unnamedArgs, ...(list ?? [])];
return all.join('|');
},
});
});
// First macro: too few list args (only required arg)
// Second macro: too many list args (required arg + 3 list entries)
const input = 'A {{test-list-bounds::base}} B {{test-list-bounds::base::x::y::z}}';
const output = await evaluateWithEngine(page, input);
// Both macros should remain unchanged in the output
expect(output).toBe(input);
const testWarnings = warnings.filter(w => w.includes('Macro "test-list-bounds"') && w.includes('unnamed arguments'));
// We expect one warning for each invalid invocation (too few and too many list args)
expect(testWarnings.length).toBe(2);
});
test('should resolve nested macros in arguments, even though the outer macro has wrong number of arguments', async ({ page }) => {
// Macro {{user ....}} will fail, because it has no args, but {{char}} should still resolve
const input = 'Result: {{user Something {{char}}}}';
const output = await evaluateWithEngine(page, input);
expect(output).toBe('Result: {{user Something Character}}');
});
});
test.describe('Type validation', () => {
test('should not resolve strict typed macro when argument type is invalid', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
MacroRegistry.unregisterMacro('test-int-strict');
MacroRegistry.registerMacro('test-int-strict', {
unnamedArgs: [
{ name: 'value', type: 'integer', description: 'Must be an integer.' },
],
strictArgs: true,
description: 'Strict integer macro for testing type validation.',
handler: ({ unnamedArgs: [value] }) => `#${value}#`,
});
});
const input = 'Value: {{test-int-strict::abc}}';
const output = await evaluateWithEngine(page, input);
// Strict typed macro should leave the text unchanged when the argument is invalid
expect(output).toBe(input);
// A runtime type validation warning should be logged
expect(warnings.some(w => w.includes('Macro "test-int-strict"') && w.includes('expected type integer'))).toBeTruthy();
});
test('should resolve non-strict typed macro when argument type is invalid but still log warning', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
MacroRegistry.unregisterMacro('test-int-nonstrict');
MacroRegistry.registerMacro('test-int-nonstrict', {
unnamedArgs: [
{ name: 'value', type: 'integer', description: 'Must be an integer.' },
],
strictArgs: false,
description: 'Non-strict integer macro for testing type validation.',
handler: ({ unnamedArgs: [value] }) => `#${value}#`,
});
});
const input = 'Value: {{test-int-nonstrict::abc}}';
const output = await evaluateWithEngine(page, input);
// Non-strict typed macro should still execute, even with invalid type
expect(output).toBe('Value: #abc#');
// A runtime type validation warning should still be logged
expect(warnings.some(w => w.includes('Macro "test-int-nonstrict"') && w.includes('expected type integer'))).toBeTruthy();
});
});
test.describe('Environment', () => {
test('should expose original content as env.content to macro handlers', async ({ page }) => {
const input = '{{env-content}}';
const originalContent = 'This is the full original input string.';
const output = await page.evaluate(async ({ input, originalContent }) => {
/** @type {import('../../public/scripts/macros/engine/MacroEngine.js')} */
const { MacroEngine } = await import('./scripts/macros/engine/MacroEngine.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
MacroRegistry.unregisterMacro('env-content');
MacroRegistry.registerMacro('env-content', {
description: 'Test macro that returns env.content.',
handler: ({ env }) => env.content,
});
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const rawEnv = {
content: originalContent,
};
const env = MacroEnvBuilder.buildFromRawEnv(rawEnv);
return MacroEngine.evaluate(input, env);
}, { input, originalContent });
expect(output).toBe(originalContent);
});
});
test.describe('Deterministic pick macro', () => {
test('should return stable results for the same chat and content', async ({ page }) => {
// Simulate a consistent chat id hash
let originalHash;
await page.evaluate(async ([originalHash]) => {
/** @type {import('../../public/script.js')} */
const { chat_metadata } = await import('./script.js');
originalHash = chat_metadata['chat_id_hash'];
chat_metadata['chat_id_hash'] = 123456;
}, [originalHash]);
const input = 'Choices: {{pick::red::green::blue}}, {{pick::red::green::blue}}.';
const output1 = await evaluateWithEngine(page, input);
const output2 = await evaluateWithEngine(page, input);
// Deterministic: same chat and same content should yield identical output.
expect(output1).toBe(output2);
// Sanity check: both picks should resolve to one of the provided options.
const match = output1.match(/Choices: ([^,]+), ([^.]+)\./);
expect(match).not.toBeNull();
if (!match) return;
const first = match[1].trim();
const second = match[2].trim();
const options = ['red', 'green', 'blue'];
expect(options.includes(first)).toBeTruthy();
expect(options.includes(second)).toBeTruthy();
// Restore original hash
await page.evaluate(async ([originalHash]) => {
/** @type {import('../../public/script.js')} */
const { chat_metadata } = await import('./script.js');
chat_metadata['chat_id_hash'] = originalHash;
}, [originalHash]);
});
});
test.describe('Dynamic macros', () => {
test('should not resolve dynamic macro when called with arguments due to strict arity', async ({ page }) => {
/** @type {string[]} */
const warnings = [];
page.on('console', msg => {
if (msg.type() === 'warning') {
warnings.push(msg.text());
}
});
const input = 'Dyn: {{dyn::extra}}';
const output = await page.evaluate(async (input) => {
/** @type {import('../../public/scripts/macros/engine/MacroEngine.js')} */
const { MacroEngine } = await import('./scripts/macros/engine/MacroEngine.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const rawEnv = {
content: input,
dynamicMacros: {
dyn: () => 'OK',
},
};
const env = MacroEnvBuilder.buildFromRawEnv(rawEnv);
return MacroEngine.evaluate(input, env);
}, input);
// Dynamic macro with arguments should not resolve because the
// temporary definition is strictArgs: true and minArgs/maxArgs: 0.
expect(output).toBe(input);
// A runtime arity warning for the dynamic macro should be logged
expect(warnings.some(w => w.includes('Macro "dyn"') && w.includes('unnamed arguments'))).toBeTruthy();
});
});
});
/**
* Evaluates the given input string using the MacroEngine inside the browser
* context, ensuring that the core macros are registered.
*
* @param {import('@playwright/test').Page} page
* @param {string} input
* @returns {Promise<string>}
*/
async function evaluateWithEngine(page, input) {
const result = await page.evaluate(async (input) => {
/** @type {import('../../public/scripts/macros/engine/MacroEngine.js')} */
const { MacroEngine } = await import('./scripts/macros/engine/MacroEngine.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const rawEnv = {
content: input,
name1Override: 'User',
name2Override: 'Character',
};
const env = MacroEnvBuilder.buildFromRawEnv(rawEnv);
const output = await MacroEngine.evaluate(input, env);
return output;
}, input);
return result;
}
/**
* Evaluates the given input string while capturing whether any macro-related
* warnings or errors were logged to the browser console.
*
* This is useful for tests that want to assert both the resolved output and
* whether the lexer/parser/engine reported issues (e.g. unterminated macros).
*
* @param {import('@playwright/test').Page} page
* @param {string} input
* @returns {Promise<{ output: string, hasMacroWarnings: boolean, hasMacroErrors: boolean }>}
*/
async function evaluateWithEngineAndCaptureMacroLogs(page, input) {
/** @type {boolean} */
let hasMacroWarnings = false;
/** @type {boolean} */
let hasMacroErrors = false;
/** @param {import('playwright').ConsoleMessage} msg */
const handler = (msg) => {
const text = msg.text();
if (text.includes('[Macro] Warning:')) {
hasMacroWarnings = true;
}
if (text.includes('[Macro] Error:')) {
hasMacroErrors = true;
}
};
page.on('console', handler);
try {
const output = await evaluateWithEngine(page, input);
return { output, hasMacroWarnings, hasMacroErrors };
} finally {
page.off('console', handler);
}
}
+312
View File
@@ -0,0 +1,312 @@
import { test, expect } from '@playwright/test';
import { testSetup } from './frontent-test-utils.js';
test.describe('MacroEnvBuilder', () => {
test.beforeEach(testSetup.awaitST);
test('builds names from overrides without relying on globals', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: 'ignored',
name1Override: 'UserOverride',
name2Override: 'CharOverride',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return {
user: env.names?.user,
char: env.names?.char,
};
});
expect(result).toEqual({
user: 'UserOverride',
char: 'CharOverride',
});
});
test('falls back to global name1/name2 when overrides are not provided', async ({ page }) => {
const result = await page.evaluate(async () => {
const script = await import('./script.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return {
globalUser: script.name1,
globalChar: script.name2,
envUser: env.names?.user,
envChar: env.names?.char,
};
});
expect(result.envUser).toBe(result.globalUser);
expect(result.envChar).toBe(result.globalChar);
});
test('does not populate character fields when replaceCharacterCard is false', async ({ page }) => {
const keys = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
replaceCharacterCard: false,
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return Object.keys(env.character || {});
});
expect(keys).toEqual([]);
});
test('populates character fields when replaceCharacterCard is true', async ({ page }) => {
const keys = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
replaceCharacterCard: true,
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return Object.keys(env.character || {});
});
// We do not assert on concrete values, only that the known keys exist
expect(keys).toEqual(expect.arrayContaining([
'charPrompt',
'charInstruction',
'description',
'personality',
'scenario',
'persona',
'mesExamplesRaw',
'version',
'charDepthPrompt',
'creatorNotes',
]));
});
test('wraps original string into a one-shot helper function', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
original: 'ORIGINAL_VALUE',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
const hasFn = typeof env.functions?.original === 'function';
const first = hasFn ? env.functions.original() : null;
const second = hasFn ? env.functions.original() : null;
return { hasFn, first, second };
});
expect(result).toEqual({
hasFn: true,
first: 'ORIGINAL_VALUE',
second: '',
});
});
test('does not expose original helper when original is not a string', async ({ page }) => {
const hasFn = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
original: undefined,
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return typeof env.functions?.original === 'function';
});
expect(hasFn).toBe(false);
});
test('uses groupOverride string for all group-related name fields', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
groupOverride: 'Group One, Group Two',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return {
group: env.names?.group,
groupNotMuted: env.names?.groupNotMuted,
notChar: env.names?.notChar,
};
});
expect(result).toEqual({
group: 'Group One, Group Two',
groupNotMuted: 'Group One, Group Two',
notChar: 'Group One, Group Two',
});
});
test('uses solo-chat semantics when no group is selected', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
const groupChats = await import('./scripts/group-chats.js');
// Ensure we are in a solo-chat like state for this test
if (typeof groupChats.resetSelectedGroup === 'function') {
groupChats.resetSelectedGroup();
}
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
name1Override: 'UserSolo',
name2Override: 'CharSolo',
groupOverride: undefined,
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return {
group: env.names?.group,
groupNotMuted: env.names?.groupNotMuted,
notChar: env.names?.notChar,
};
});
expect(result).toEqual({
group: 'CharSolo',
groupNotMuted: 'CharSolo',
notChar: 'UserSolo',
});
});
test('merges dynamicMacros properties into env.dynamicMacros', async ({ page }) => {
const dynamicMacros = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
dynamicMacros: {
simple: 'value',
number: 42,
nested: { foo: 'bar' },
},
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return env.dynamicMacros;
});
expect(dynamicMacros.simple).toBe('value');
expect(dynamicMacros.number).toBe(42);
expect(dynamicMacros.nested).toEqual({ foo: 'bar' });
});
test('sets system.model field from getGeneratingModel helper', async ({ page }) => {
const model = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return env.system?.model;
});
expect(typeof model === 'string' || model === undefined).toBe(true);
});
test('applies providers in the expected order buckets', async ({ page }) => {
const order = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder, env_provider_order } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
MacroEnvBuilder.registerProvider((env) => {
env.extra.order = [...(env.extra.order || []), 'EARLY'];
}, env_provider_order.EARLY);
MacroEnvBuilder.registerProvider((env) => {
env.extra.order = [...(env.extra.order || []), 'LATE'];
}, env_provider_order.LATE);
MacroEnvBuilder.registerProvider((env) => {
env.extra.order = [...(env.extra.order || []), 'NORMAL'];
}, env_provider_order.NORMAL);
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return env.extra.order;
});
// We only guarantee relative ordering between the buckets we added,
// not that there are no other entries from other providers.
const earlyIndex = order.indexOf('EARLY');
const normalIndex = order.indexOf('NORMAL');
const lateIndex = order.indexOf('LATE');
expect(earlyIndex).toBeGreaterThanOrEqual(0);
expect(normalIndex).toBeGreaterThan(earlyIndex);
expect(lateIndex).toBeGreaterThan(normalIndex);
});
test('ignores provider errors without breaking env construction', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js')} */
const { MacroEnvBuilder, env_provider_order } = await import('./scripts/macros/engine/MacroEnvBuilder.js');
MacroEnvBuilder.registerProvider(() => {
throw new Error('intentional test error');
}, env_provider_order.NORMAL);
/** @type {import('../../public/scripts/macros/engine/MacroEnvBuilder.js').MacroEnvRawContext} */
const ctx = {
content: '',
name1Override: 'User',
dynamicMacros: { marker: 'value' },
};
const env = MacroEnvBuilder.buildFromRawEnv(ctx);
return {
namesUser: env.names?.user,
hasDynamicMacro: env.dynamicMacros?.marker === 'value',
};
});
expect(result.hasDynamicMacro).toBe(true);
expect(result.namesUser).toBe('User');
});
});
File diff suppressed because it is too large Load Diff
+678
View File
@@ -0,0 +1,678 @@
import { test, expect } from '@playwright/test';
import { testSetup } from './frontent-test-utils.js';
/** @typedef {import('chevrotain').CstNode} CstNode */
/** @typedef {import('chevrotain').IRecognitionException} IRecognitionException */
/** @typedef {{[tokenName: string]: (string|string[]|TestableCstNode|TestableCstNode[])}} TestableCstNode */
/** @typedef {{name: string, message: string}} TestableRecognitionException */
const DEFAULT_FLATTEN_KEYS = [
'arguments.Args.DoubleColon',
];
const DEFAULT_IGNORE_KEYS = [
];
test.describe('MacroParser', () => {
// Currently this test suits runs without ST context. Enable, if ever needed
test.beforeEach(testSetup.goST);
test.describe('General Macro', () => {
// {{user}}
test('should parse a simple macro', async ({ page }) => {
const input = '{{user}}';
const macroCst = await runParser(page, input);
const expectedCst = {
'Macro.Start': '{{',
'Macro.identifier': 'user',
'Macro.End': '}}',
};
expect(macroCst).toEqual(expectedCst);
});
// {{ user }}
test('should generally handle whitespaces', async ({ page }) => {
const input = '{{ user }}';
const macroCst = await runParser(page, input);
const expectedCst = {
'Macro.Start': '{{',
'Macro.identifier': 'user',
'Macro.End': '}}',
};
expect(macroCst).toEqual(expectedCst);
});
test.describe('Error Cases (General Macro)', () => {
// {{}}
test('[Error] should throw an error for empty macro', async ({ page }) => {
const input = '{{}}';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
const expectedErrors = [
{ name: 'NoViableAltException' },
];
const expectedMessage = /Expecting: one of these possible Token sequences:(.*?)\[Macro\.Identifier\](.*?)but found: '}}'/gs;
expect(macroCst).toBeUndefined();
expect(errors).toMatchObject(expectedErrors);
expect(errors[0].message).toMatch(expectedMessage);
});
// {{§!#&blah}}
test('[Error] should throw an error for invalid identifier', async ({ page }) => {
const input = '{{§!#&blah}}';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
const expectedErrors = [
{ name: 'NoViableAltException' },
];
const expectedMessage = /Expecting: one of these possible Token sequences:(.*?)\[Macro\.Identifier\](.*?)but found: '!'/gs;
expect(macroCst).toBeUndefined();
expect(errors).toMatchObject(expectedErrors);
expect(errors[0].message).toMatch(expectedMessage);
});
// {{user
test('[Error] should throw an error for incomplete macro', async ({ page }) => {
const input = '{{user';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
const expectedErrors = [
{ name: 'MismatchedTokenException', message: 'Expecting token of type --> Macro.End <-- but found --> \'\' <--' },
];
expect(macroCst).toBeUndefined();
expect(errors).toEqual(expectedErrors);
});
// something{{user}}
test('[Error] for testing purposes, macros need to start at the beginning of the string', async ({ page }) => {
const input = 'something{{user}}';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
const expectedErrors = [
{ name: 'MismatchedTokenException', message: 'Expecting token of type --> Macro.Start <-- but found --> \'something\' <--' },
];
expect(macroCst).toBeUndefined();
expect(errors).toEqual(expectedErrors);
});
});
});
test.describe('Arguments Handling', () => {
// {{getvar::myvar}}
test('should parse macros with double-colon argument', async ({ page }) => {
const input = '{{getvar::myvar}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'getvar',
'arguments': {
'separator': '::',
'argument': 'myvar',
},
'Macro.End': '}}',
});
});
// {{roll:3d20}}
test('should parse macros with single colon argument', async ({ page }) => {
const input = '{{roll:3d20}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'roll',
'arguments': {
'separator': ':',
'argument': '3d20',
},
'Macro.End': '}}',
});
});
// {{setvar::myvar::value}}
test('should parse macros with multiple double-colon arguments', async ({ page }) => {
const input = '{{setvar::myvar::value}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
ignoreKeys: ['arguments.Args.DoubleColon'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'setvar',
'arguments': {
'separator': '::',
'argument': ['myvar', 'value'],
},
'Macro.End': '}}',
});
});
// {{something:: spaced }}
test('should strip spaces around arguments', async ({ page }) => {
const input = '{{something:: spaced }}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
ignoreKeys: ['arguments.separator', 'arguments.Args.DoubleColon'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'something',
'arguments': { 'argument': 'spaced' },
'Macro.End': '}}',
});
});
// {{something::with:single:colons}}
test('should treat single colons as part of the argument with double-colon separator', async ({ page }) => {
const input = '{{something::with:single:colons}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
ignoreKeys: ['arguments.Args.DoubleColon'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'something',
'arguments': {
'separator': '::',
'argument': 'with:single:colons',
},
'Macro.End': '}}',
});
});
// {{legacy:something:else}}
test('should treat single colons as part of the argument even with colon separator', async ({ page }) => {
const input = '{{legacy:something:else}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
ignoreKeys: ['arguments.separator', 'arguments.Args.Colon'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'legacy',
'arguments': { 'argument': 'something:else' },
'Macro.End': '}}',
});
});
// {{something::}}
test('should parse double-colon with an empty argument value', async ({ page }) => {
const input = '{{something::}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'something',
'arguments': {
'separator': '::',
'argument': '',
},
'Macro.End': '}}',
});
});
});
test.describe('Legacy Macros', () => {
// {{roll 1d5}}
test('should parse legacy roll macro with whitespace separator', async ({ page }) => {
const input = '{{roll 1d5}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'roll',
'arguments': { 'argument': '1d5' },
'Macro.End': '}}',
});
});
// {{roll:2d20}}
test('should parse legacy roll macro with explicit colon separator', async ({ page }) => {
const input = '{{roll:2d20}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'roll',
'arguments': {
'separator': ':',
'argument': '2d20',
},
'Macro.End': '}}',
});
});
// {{roll 20}}
test('should parse legacy roll macro with numeric argument', async ({ page }) => {
const input = '{{roll 20}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'roll',
'arguments': { 'argument': '20' },
'Macro.End': '}}',
});
});
// {{reverse:something}}
test('should parse reverse legacy macro with colon argument', async ({ page }) => {
const input = '{{reverse:something}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'reverse',
'arguments': {
'separator': ':',
'argument': 'something',
},
'Macro.End': '}}',
});
});
// {{reverse:this contains::double::colons}}
test('should parse legacy single colon argument that allows double colons inside the argument', async ({ page }) => {
const input = '{{reverse:this contains::double::colons}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'reverse',
'arguments': {
'separator': ':',
'argument': 'this contains::double::colons',
},
'Macro.End': '}}',
});
});
// {{//comment-style macro}}
// TODO: Comment like // is not a valid identifier, needs to be an exception (until we maybe add flags)
test('should parse legacy comment macro', async ({ page }) => {
const input = '{{//comment-style macro}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'arguments': { 'argument': 'comment-style macro' },
'Macro.End': '}}',
});
});
// {{datetimeformat HH:mm}}
test('should parse legacy datetime format macro', async ({ page }) => {
const input = '{{datetimeformat HH:mm}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'datetimeformat',
'arguments': { 'argument': 'HH:mm' },
'Macro.End': '}}',
});
});
// Note: Legacy time macros like {{time_UTC+2}} are now handled by the MacroEngine
// pre-processing pipeline instead of the parser. See MacroEngine.e2e tests for coverage.
// {{banned "abannedword"}}
test('should parse legacy banned macro with quoted argument', async ({ page }) => {
const input = '{{banned "abannedword"}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'banned',
'arguments': { 'argument': '"abannedword"' },
'Macro.End': '}}',
});
});
// {{banned ""}}
test('should parse legacy macro with empty quoted argument', async ({ page }) => {
const input = '{{banned ""}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'banned',
'arguments': { 'argument': '""' },
'Macro.End': '}}',
});
});
// {{setvar::myvar::}}
test('should allow legacy setvar with empty value argument', async ({ page }) => {
const input = '{{setvar::myvar::}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'setvar',
'arguments': {
'separator': '::',
'argument': ['myvar', ''],
},
'Macro.End': '}}',
});
});
});
test.describe('Comment Macros', () => {
// {{//comment}}
test('should parse comment macro without whitespace', async ({ page }) => {
const input = '{{//comment}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'Macro.End': '}}',
'arguments': {
'argument': 'comment',
},
});
});
// {{// comment}}
test('should parse comment macro with whitespace', async ({ page }) => {
const input = '{{// comment}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'Macro.End': '}}',
'arguments': {
'argument': 'comment',
},
});
});
// {{//!@#$%^&*()_+}}
test('should parse comment macro with special characters', async ({ page }) => {
const input = '{{//!@#$%^&*()_+}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'Macro.End': '}}',
'arguments': {
'argument': '!@#$%^&*()_+',
},
});
});
// {{//!@flags}}
test('should parse comment macro starting with flags', async ({ page }) => {
const input = '{{//!@flags}}';
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'Macro.End': '}}',
'arguments': {
'argument': '!@flags',
},
});
});
// {{// This is a multiline comment.
// This is the second line
// }}
test('should parse multiline comments', async ({ page }) => {
const input = `{{// This is a multiline comment.
This is the second line
}}`;
const macroCst = await runParser(page, input, {
flattenKeys: ['arguments.argument'],
});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': '//',
'Macro.End': '}}',
'arguments': {
'argument': 'This is a multiline comment.\nThis is the second line',
},
});
});
});
test.describe('Nested Macros', () => {
// {{outer::word {{inner}}}}
test('should parse nested macros inside arguments', async ({ page }) => {
const input = '{{outer::word {{inner}}}}';
const macroCst = await runParser(page, input, {});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'outer',
'arguments': {
'argument': {
'Identifier': 'word',
'macro': {
'Macro.Start': '{{',
'Macro.identifier': 'inner',
'Macro.End': '}}',
},
},
'separator': '::',
},
'Macro.End': '}}',
});
});
// {{outer::word {{inner1}}{{inner2}}}}
test('should parse two nested macros next to each other inside an argument', async ({ page }) => {
const input = '{{outer::word {{inner1}}{{inner2}}}}';
const macroCst = await runParser(page, input, {});
expect(macroCst).toEqual({
'Macro.Start': '{{',
'Macro.identifier': 'outer',
'arguments': {
'argument': {
'Identifier': 'word',
'macro': [
{
'Macro.Start': '{{',
'Macro.identifier': 'inner1',
'Macro.End': '}}',
},
{
'Macro.Start': '{{',
'Macro.identifier': 'inner2',
'Macro.End': '}}',
},
],
},
'separator': '::',
},
'Macro.End': '}}',
});
});
test.describe('Error Cases (Nested Macros)', () => {
// {{{{macroindentifier}}::value}}
test('[Error] should throw when there is a nested macro instead of an identifier', async ({ page }) => {
const input = '{{{{macroindentifier}}::value}}';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
expect(macroCst).toBeUndefined();
expect(errors).toHaveLength(1); // error doesn't really matter. Just don't parse it pls.
});
// {{inside{{macro}}me}}
test('[Error] should throw when there is a macro inside an identifier', async ({ page }) => {
const input = '{{inside{{macro}}me}}';
const { macroCst, errors } = await runParserAndGetErrors(page, input);
expect(macroCst).toBeUndefined();
expect(errors).toHaveLength(1); // error doesn't really matter. Just don't parse it pls.
});
});
});
});
/**
* Runs the input through the MacroParser and returns the result.
*
* @param {import('@playwright/test').Page} page - The Playwright page object.
* @param {string} input - The input string to be parsed.
* @param {Object} [options={}] Optional arguments
* @param {string[]} [options.flattenKeys=[]] Optional array of dot-separated keys to flatten
* @param {string[]} [options.ignoreKeys=[]] Optional array of dot-separated keys to ignore
* @returns {Promise<TestableCstNode>} A promise that resolves to the result of the MacroParser.
*/
async function runParser(page, input, options = {}) {
const { cst, errors } = await runParserAndGetErrors(page, input, options);
// Make sure that parser errors get correctly marked as errors during testing, even if the resulting structure might work.
// If we don't test for errors, the test should fail.
if (errors.length > 0) {
throw new Error('Parser errors found\n' + errors.map(x => x.message).join('\n'));
}
return cst;
}
/**
* Runs the input through the MacroParser and returns the syntax tree result and any parser errors.
*
* Use `runParser` if you don't want to explicitly test against parser errors.
*
* @param {import('@playwright/test').Page} page - The Playwright page object.
* @param {string} input - The input string to be parsed.
* @param {Object} [options={}] Optional arguments
* @param {string[]} [options.flattenKeys=[]] Optional array of dot-separated keys to flatten
* @param {string[]} [options.ignoreKeys=[]] Optional array of dot-separated keys to ignore
* @returns {Promise<{cst: TestableCstNode, errors: TestableRecognitionException[]}>} A promise that resolves to the result of the MacroParser and error list.
*/
async function runParserAndGetErrors(page, input, options = {}) {
const params = { input, options };
const { result } = await page.evaluate(async ({ input, options }) => {
/** @type {import('../../public/scripts/macros/engine/MacroParser.js')} */
const { MacroParser } = await import('./scripts/macros/engine/MacroParser.js');
const result = MacroParser.test(input);
return { result };
}, params);
return { cst: simplifyCstNode(result.cst, input, options), errors: simplifyErrors(result.errors) };
}
/**
* Simplify the parser syntax tree result into an easily testable format.
*
* @param {CstNode} result The result from the parser
* @param {Object} [options={}] Optional arguments
* @param {string[]} [options.flattenKeys=[]] Optional array of dot-separated keys to flatten
* @param {string[]} [options.ignoreKeys=[]] Optional array of dot-separated keys to ignore
* @returns {TestableCstNode} The testable syntax tree
*/
function simplifyCstNode(cst, input, { flattenKeys = [], ignoreKeys = [], ignoreDefaultFlattenKeys = false, ignoreDefaultIgnoreKeys = false } = {}) {
if (!ignoreDefaultFlattenKeys) flattenKeys = [...flattenKeys, ...DEFAULT_FLATTEN_KEYS];
if (!ignoreDefaultIgnoreKeys) ignoreKeys = [...ignoreKeys, ...DEFAULT_IGNORE_KEYS];
/** @returns {TestableCstNode} @param {CstNode} node @param {string[]} path */
function simplifyNode(node, path = []) {
if (!node) return node;
if (Array.isArray(node)) {
// Single-element arrays are converted to a single string
if (node.length === 1) {
return node[0].image || simplifyNode(node[0], path.concat('[]'));
}
// For multiple elements, return an array of simplified nodes
return node.map(child => simplifyNode(child, path.concat('[]')));
}
if (node.children) {
const simplifiedChildren = {};
for (const key in node.children) {
function simplifyChildNode(childNode, path) {
if (Array.isArray(childNode)) {
// Single-element arrays are converted to a single string
if (childNode.length === 1) {
return simplifyChildNode(childNode[0], path.concat('[]'));
}
return childNode.map(child => simplifyChildNode(child, path.concat('[]')));
}
const flattenKey = path.filter(x => x !== '[]').join('.');
if (ignoreKeys.includes(flattenKey)) {
return null;
} else if (flattenKeys.includes(flattenKey)) {
if (!childNode.location) return null;
const startOffset = childNode.location.startOffset;
const endOffset = childNode.location.endOffset;
return input.slice(startOffset, endOffset + 1);
} else {
return simplifyNode(childNode, path);
}
}
const simplifiedValue = simplifyChildNode(node.children[key], path.concat(key));
if (simplifiedValue !== null) simplifiedChildren[key] = simplifiedValue;
}
if (Object.values(simplifiedChildren).length === 0) return null;
return simplifiedChildren;
}
return node.image;
}
return simplifyNode(cst);
}
/**
* Simplifies a recognition exceptions into an easily testable format.
*
* @param {IRecognitionException[]} errors - The error list containing exceptions to be simplified.
* @return {TestableRecognitionException[]} - The simplified error list
*/
function simplifyErrors(errors) {
return errors.map(exception => ({
name: exception.name,
message: exception.message,
}));
}
+153
View File
@@ -0,0 +1,153 @@
import { test, expect } from '@playwright/test';
import { testSetup } from './frontent-test-utils.js';
test.describe('MacroRegistry', () => {
// Currently this test suits runs without ST context. Enable, if ever needed
test.beforeEach(testSetup.awaitST);
test.describe('valid', () => {
test('should register a macro with valid options', async ({ page }) => {
const result = await page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
MacroRegistry.unregisterMacro('test-valid');
MacroRegistry.registerMacro('test-valid', {
unnamedArgs: 2,
list: { min: 1, max: 3 },
strictArgs: false,
description: 'Test macro for validation.',
handler: ({ args }) => args.join(','),
});
const def = MacroRegistry.getMacro('test-valid');
return {
name: def?.name,
minArgs: def?.minArgs,
maxArgs: def?.maxArgs,
list: def?.list,
strictArgs: def?.strictArgs,
description: def?.description,
};
});
expect(result).toEqual({
name: 'test-valid',
minArgs: 2,
maxArgs: 2,
list: { min: 1, max: 3 },
strictArgs: false,
description: 'Test macro for validation.',
});
});
});
test.describe('reject', () => {
test('should reject invalid macro name', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// Empty name
MacroRegistry.registerMacro(' ', {
handler: () => '',
});
})).rejects.toThrow(/Macro name must be a non-empty string/);
});
test('should reject invalid options object', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// Options must be object
// @ts-expect-error intentionally wrong
MacroRegistry.registerMacro('invalid-options', null);
})).rejects.toThrow(/options must be a non-null object/);
});
test('should reject invalid handler', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// Handler must be function
// @ts-expect-error intentionally wrong
MacroRegistry.registerMacro('no-handler', { handler: null });
})).rejects.toThrow(/options\.handler must be a function/);
});
test('should reject invalid unnamedArgs', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// unnamedArgs must be non-negative integer
MacroRegistry.registerMacro('bad-required', {
// @ts-expect-error intentionally wrong
unnamedArgs: -1,
handler: () => '',
});
})).rejects.toThrow(/options\.unnamedArgs must be a non-negative integer/);
});
test('should reject invalid strictArgs', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// strictArgs must be boolean
MacroRegistry.registerMacro('bad-strict', {
// @ts-expect-error intentionally wrong
strictArgs: 'yes',
handler: () => '',
});
})).rejects.toThrow(/options\.strictArgs must be a boolean/);
});
test('should reject invalid list configuration', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// list must be boolean or object
MacroRegistry.registerMacro('bad-list-type', {
// @ts-expect-error intentionally wrong
list: 'invalid',
handler: () => '',
});
})).rejects.toThrow(/options\.list must be a boolean or an object/);
});
test('should reject invalid list.min', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// list.min must be non-negative
MacroRegistry.registerMacro('bad-list-min', {
list: { min: -1 },
handler: () => '',
});
})).rejects.toThrow(/options\.list\.min must be a non-negative integer/);
});
test('should reject invalid list.max', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// list.max must be >= min
MacroRegistry.registerMacro('bad-list-max', {
list: { min: 2, max: 1 },
handler: () => '',
});
})).rejects.toThrow(/options\.list\.max must be greater than or equal to options\.list\.min/);
});
test('should reject invalid description', async ({ page }) => {
await expect(page.evaluate(async () => {
/** @type {import('../../public/scripts/macros/engine/MacroRegistry.js')} */
const { MacroRegistry } = await import('./scripts/macros/engine/MacroRegistry.js');
// description must be string
MacroRegistry.registerMacro('bad-desc', {
// @ts-expect-error intentionally wrong
description: 123,
handler: () => '',
});
})).rejects.toThrow(/options\.description must be a string/);
});
});
});
+50
View File
@@ -0,0 +1,50 @@
import { test, expect } from '@playwright/test';
import { testSetup } from './frontent-test-utils.js';
// Tests for the deprecated MacrosParser shim to ensure it continues to work
// both with the legacy regex macro system (feature flag disabled) and with
// the new macro engine (feature flag enabled).
test.describe('MacrosParser (legacy shim)', () => {
test.beforeEach(testSetup.awaitST);
test('should resolve macros via legacy evaluateMacros when experimental engine is disabled', async ({ page }) => {
const output = await page.evaluate(async () => {
const { MacrosParser, evaluateMacros } = await import('./scripts/macros.js');
const { power_user } = await import('./scripts/power-user.js');
power_user.experimental_macro_engine = false;
MacrosParser.registerMacro('legacyParserTest', 'LEGACY_OK', 'Legacy parser test');
const env = {};
const result = evaluateMacros('Value: {{legacyParserTest}}.', env, (x) => x);
MacrosParser.unregisterMacro('legacyParserTest');
return result;
});
expect(output).toBe('Value: LEGACY_OK.');
});
test('should resolve macros via new engine when experimental engine is enabled', async ({ page }) => {
const output = await page.evaluate(async () => {
const { MacrosParser } = await import('./scripts/macros.js');
const { substituteParams } = await import('./script.js');
const { power_user } = await import('./scripts/power-user.js');
power_user.experimental_macro_engine = true;
MacrosParser.registerMacro('engineParserTest', 'ENGINE_OK', 'Engine parser test');
const result = substituteParams('Value: {{engineParserTest}}.', {});
MacrosParser.unregisterMacro('engineParserTest');
return result;
});
expect(output).toBe('Value: ENGINE_OK.');
});
});