Backport feat/chat-tree and fix #4709 (#4712)

* Performance improvements due to using chatElement instead of $('#chat').

* Reverts change `hideSwipeButtons()` change.

https://github.com/SillyTavern/SillyTavern/pull/4576#discussion_r2394996102

* I’ve been working on a PR for issue #1731: [Swipes on every AI message, not just the last one](https://github.com/SillyTavern/SillyTavern/issues/1731). I’d appreciate feedback to avoid unnecessary effort.

Currently, I have User and Assistant branches working.
I have not touched `/api/chats/save` or '/api/chats/get', so branches do not persist.

To keep `context.chat` unmodified (so extensions and the rest of the code remain unaffected), I’ve stored branches in `chatTree`:
```javascript
chatTree = {
  branch_id: 1,
  branch: [
    { mes: "Hi" },
    { mes: "Hello", branch_id: 0, branch: [...] },
  ]
}
```

When a message is swiped, chatTree is updated via `saveChatToTree()`, then the next branch is loaded using get`ChatFromTree()`.

Questions:
According to Cohee: [This requires reorganizations in the file format for chats, not viable in the short term.](https://github.com/SillyTavern/SillyTavern/issues/1731#issuecomment-1937845036)
I'm hesitant to proceed. May I store chatTree.json files alongside the existing .jsonl chats?
Should I create a new /save endpoint, modify the existing one, or discontinue using .jsonl for saving.

Known issues:
Branches do not persist on refresh.
Swiping multiple messages at once throws `Cannot read properties of undefined (reading 'mes')
`. This will be fixed in the UI.
Branches persist between chats/characters. This will be fixed when loading a chat.
The Swipe arrows overlap with messages.
No animation plays when swiping then editing a user message.

* Fixed bug.

Gemini:
The calculation of mesId is fragile and contains a bug. The fallback chat[chat.length - 1] is a message object, and Number(object) will result in NaN. This will break the swipe functionality if the preceding expressions are falsy. Additionally, chat.indexOf(message) will almost always point to the last message in the chat, which is incorrect when swiping on an earlier message.

A more robust approach is recommended to reliably get the message ID from the clicked element.

    const mesId = Number($(this).closest('.mes').attr('mesid'));

* Fixes swipes on long chats.

* chatTree is now persistent. Fixed bugs.

* Fixed bugs.
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2390042262
The logic to rename the chat tree file (.json) is currently in an unreachable else block. The if condition !fs.existsSync(pathToOriginalFile) || fs.existsSync(pathToRenamedFile) will almost always evaluate to true after the preceding copyFileSync and unlinkSync operations, because pathToOriginalFile will no longer exist. This prevents the chat tree file from being renamed, which breaks the persistence of message branches when a chat is renamed.

The scrollChatToMessage function is implemented incorrectly. Calling .scrollTop() without any arguments on a jQuery object retrieves the current vertical scroll position; it does not scroll the element into view. This function will not have the intended effect of scrolling the chat to the specified message.

* Fixed bugs. https://github.com/SillyTavern/SillyTavern/pull/4573#pullrequestreview-3282889120

There's a typo in the property name being deleted. It should be branch to match the data structure you've defined, not branches. This error will prevent the pruning logic from working correctly, potentially leading to corrupted or bloated chatTree data.

The modified treeData is not being saved here. Instead, the global chatTree is being sent in the request. This will cause any renames of group members within message branches to be lost upon saving. You should send treeData, which contains the modifications.

This function may have performance issues on large chats due to multiple structuredClone calls within a loop. swipelessMessage is created from a deep clone, and then it's deep-cloned again for every swipe. While this ensures data integrity, it is inefficient. Consider refactoring to reduce the number of deep-cloning operations, for instance, by creating swipelessMessage once per message and deep-cloning it only when creating a new branch for a swipe.

* Fix warning. https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2390183161

The logic for renaming the chat tree file seems to have a flaw. The condition !fs.existsSync(pathToOriginalTreeFile) || fs.existsSync(pathToRenamedTreeFile) will be true if the original tree file does not exist. In this case, a warning is logged. However, it's a valid scenario for a chat to not have a corresponding tree file, so no warning should be logged. The current logic could lead to confusing log messages.

* Refactored chatTree into `public/scripts/chat-tree.js`.

https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2390269514

* Displaying swipes on past messages and the entire chat tree functionality is now an opt-in toggle.

https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2390273599

* Only allow one concurrent swipe.
Fixed:
Swiping multiple messages at once throws Cannot read properties of undefined (reading 'mes') .

* Fixed bugs.
Now `swipe_id >= swipes.length` is set to swipes.length.

* Fixed bug.
Swiping a user message did not re-show swipe buttons after the generation finished.

* Moved `Show Swipes for All Messages` Toggle to `Chat/Message Handling`.
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2395008961

* `JSON.stringify doesn't add spaces by default`
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2395047031

* Fixed "Tree file left behind when renaming chats"

https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2395546142

* Fixed "Guard chatTree recursion in group member rename"
https://github.com/SillyTavern/SillyTavern/pull/4573#pullrequestreview-3290587129

* Re-implement: https://github.com/SillyTavern/SillyTavern/pull/4576#discussion_r2395506501

* Moved chat trees to a separate directory. https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2395045583
Enabled chat tree backups.

* Removed structuredClone() for improved performance.

* Fixed bugs.
Characters are now correctly renamed in the chatTree.
Directories are now recursively created.

* Reverts `hideSwipeButtons` to original functionality when `show_swipes_for_all_messages` is false.

https://github.com/SillyTavern/SillyTavern/pull/4576#discussion_r2395375718

* Added `refreshSwipeButtons`.
Updated `showSwipeButtons` and `hideSwipeButtons`.
Fixed bugs.

* Fixed.
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2400143193
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2400143202

* Added `clamp` to util.js.
Refactored `swipe`.

* Fixed bugs.
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2400685914
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2400685911

* Fixed bugs.

* Fixed swipe animations.

* Fixed bug by setting ids with `Number`.

* Delete swipes, and bugfixes.

* Merged `refactor/swipe`.

* Fixed bug created in `Delete swipes, and bugfixes.`.

* Merged from origin/staging.

* Merged changes from refactor/swipe.

* Fixed bug and refactored `syncWithSwipeId`.

* Merged from `origin/staging`.

* Fixed merge.

* Fixed overlapping message generations.

* Warn user, and refresh chat.

* Added metadata to chatTree file.

* Fixed "a sacrifice for the sake of simplicity." https://github.com/SillyTavern/SillyTavern/pull/2752#issuecomment-2323512022

Gemini:
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2464165990
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2464165991

* Improved swipeGenerate's animation.

* Added `SWIPE_SOURCE` constant.

* await `switchSwipesAllMessages`.

* Added `swipeState`.

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4573#issuecomment-3449690308

* Done:
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466884547
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466884547
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466894947
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466902075
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466911245
https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466938169

* Removed `NEUTRAL_CHAT_TREE_KEY`

* Moved backups

* Fixed `/send` and fixed a bug in `sendMessageAs`.

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466913570

* Fixes usage of `ENOENT`: https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466929980

* Removed LLM attribution.

* Updated link and warning.

* Removed debugging comments.

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4573#discussion_r2466905341

* Replaced `isSwipingAllowed` with `swipeState`.

* Backported many changes from feat/chat-tree.

* Refactored `showSwipeButtons` and `hideSwipeButtons` into `refreshSwipeButtons`

* Refactored `showSwipeButtons` and `hideSwipeButtons` into `refreshSwipeButtons`

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2472109888
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2472113323

* Merged from `refactor/backport-chat-tree`.

* Proposed fix for https://github.com/SillyTavern/SillyTavern/issues/4709
Reverts commit: https://github.com/SillyTavern/SillyTavern/commit/1b3db273891c1ba8c781d2f691ad2e41937ca2aa

On PR: https://github.com/SillyTavern/SillyTavern/pull/2940

* Proposed fix for https://github.com/SillyTavern/SillyTavern/issues/4709
Reverts commit: https://github.com/SillyTavern/SillyTavern/commit/1b3db273891c1ba8c781d2f691ad2e41937ca2aa

On PR: https://github.com/SillyTavern/SillyTavern/pull/2940

* Fixed regenerate and continue while editing the last message.

* Added swipesHidden.
Messages can now be designated as non-swipeable with `message.extra.swipeable`.

* Use `.hidden` and classes instead of `.css` to display swipe chevrons.

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3465749700
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2476136375
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2476140043
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3465776056
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3465794330

* Fixed bug.

* Fixed broken JQuery animation.

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2479003956
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2479011525
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2479014001

* Fixed active arrow transition.

* Fixed swipe shake direction.

* Fixed truthy check.

* Added temporary link to documentation PR.

* Moved `switchSwipesAllMessages`

* Fixed failed generations deleting the branch.
Fixed `forceSwipeId`.
Always call `saveChatConditional` if the `swipe_id` has changed.

* ESLint.

* Fixed bug.
Better `syncSwipeToMes` error handling.

* Backported changes from `feat/chat-tree`

Added failed swipe animation.
Fixed `newSwipeId`.

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#pullrequestreview-3406878337

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2483143387

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2483143387

* Added `showSwipes: false` to `showMoreMessages`.

And a few more performance improvements.

* Fixed animations.

* Added `'.mes'` to `.children`.

* Significantly improved `animateSwipeTransition` performance on large chats.

* Improved `syncSwipeToMes` error handling.

* Significantly improved `animateSwipeTransition` performance on large chats.

* Improved `syncSwipeToMes` error handling.

* Corrected Merge.

* Corrected Merge.

* Improved: `redisplayChat`
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2484177521

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2484215469

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2484217230

* Added a temporary implementation of `branchChat`.

* Fixed bug when generating a swipe.

* Fixed bug in `branchChat`.

* Fixed Off by one error due to `chatElement.children()` selecting `show_more_messages`.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3478331073

* Fixed Off by one error due to `chatElement.children()` selecting `show_more_messages`.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3478331073

* Cleaned system messages array and moved `swipeable`

* Removed CSS nesting.

* Fixed merge.

* Improved `syncSwipeToMes` error handling.

* Matched `swipes-counter` fade to chevrons.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3478192184

* Wrapped `transition-behavior: allow-discrete;` in `@supports`.

* Fixed types.

* Improved swipeability feedback and functions.

* Improved `refreshSwipeButtons` performance again.

* Swapped `.attr` to `.prop`.

* Fixed: `@supports (transition-behavior: allow-discrete)`

* Fixed and clarified typo.

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495919425
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495923036
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495927019
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495930080
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495943870
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495950927
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495954387
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495974324
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495974683

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495957181
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495960920
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495968137
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495968922
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495959430
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495924599

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495918468
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495931979
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495933092
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2495939511

* Improved `refreshSwipeButtons` performance by skipping 'swipes-counter' updates by default.

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2496382916
https://github.com/SillyTavern/SillyTavern/pull/4712#discussion_r2496379401

* Removed chevron fade-in on Cohee's request: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3493829978

* Set most system messages to `swipeable: false`.

* Implemented `OVERSWIPE_BEHAVIOR`.

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3499706714

* Fix formatting, make comments IDE friendly

* IDE friendly enum comments

* Move animation frames to a dedicated file

* Do not compare with -1

* overswipeBehavior => getOverswipeBehavior

* Formatting fix

* Don't let regenerate on is_system

* Fixed canceled generations in pristine chats.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3499819670

* Fixed `AUTO_SWIPE` when `animation_duration = 0`.

* Fixed mistake in `redisplayChat`.

* Holding the swipe button speeds up `swipeDuration`.

* Fixed 'animationend' never ending.
Altered resetTime.

* Swapped from `saveChatConditional` to `saveChatDebounced` in `swipe`.

* Skip the animation if it's faster than 50ms instead of 10ms.

* Typo.

* Fixed:
https://github.com/SillyTavern/SillyTavern/pull/4712#pullrequestreview-3457198109

* Adjust duration reset cooldown

* Add quotes to selector

* Specify type for message parameter in swipe function

* Add type for swipe UI event

* Disabled fade-in during printMessages.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3539315919

* Typo.

* Added quotes to selector .

* Reduce reset time 500 -> 350

* Loops do not cause a generation so their chevrons should not have increased opacity.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3543692019

* Revert reset time

* Renamed `heldSwipes` to `recentSwipes`.

* Autofix the swipes array during `updateSwipeCounter`.

* User messages should not have swipes.

* Chevrons should always be shown on pristine greetings: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3557893373

* Improve formatting

* Fixed: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3559617088

* Show `pristineGreetingSwipeNotice` once.

* `clearMessageData` when swipe-regenerating a message.

* accountStorage is already imported in the module

* Removed `await`.

* Removed pristine greeting notice.
https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3560758598

* Removed redundant functions in `StreamingProcessor` and fixed streamed replies missing counters.

* Moved `markUIGenStopped` after `eventSource.emit`.

Swapped to `saveChatDebounced` to fix: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3567014810.

* Save a structuredClone of `chat` to prevent an invalid chat from being saved.

* Only `structuredClone` `chat` on `saveChatDebounced`.

* Revert "Only `structuredClone` `chat` on `saveChatDebounced`."

This reverts commit 49498b7aa1410107b294555fb945d977e60bfebf.

* Revert "Save a structuredClone of `chat` to prevent an invalid chat from being saved."

This reverts commit 5f137ed1380107fde0765b951dc634081bdbf2ff.

* Prevent `saveChatDebounced` from saving while the swipe is in progress.
See: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3567077312

* Fixed animation never ending: https://github.com/SillyTavern/SillyTavern/pull/4712#issuecomment-3567106213

* `forceMesId` and `forceSwipeId` are not objects.

* Fixed Reduced Motion causing a warning when swiping back.

* Only  hide `.mes_buttons` when generating.

* Fix eslint

* Reset duration on switching direction

---------

Co-authored-by: user <user@exmaple.com>
Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>

public/style.css

@@ -677,11 +677,6 @@ small {
     margin: 0;
 }
 
-.mes.smallSysMes .swipe_right,
-.mes.smallSysMes .swipe_left {
-    display: none !important;
-}
-
 .mes.smallSysMes .mes_text {
     padding: 0 !important;
     text-align: center;
@@ -1230,6 +1225,14 @@ body .panelControlBar {
     --swipeCounterMargin: 5px;
 }
 
+/* https://stackoverflow.com/a/31819499 */
+/* This has much better performance than .transform on large chats*/
+.slide {
+    animation: slide calc(var(--slide-mes-x-duration, var(--animation-duration, 125ms))) ease-in-out forwards;
+    -webkit-animation: slide calc(var(--slide-mes-x-duration, var(--animation-duration, 125ms))) ease-in-out forwards;
+    will-change: transform;
+}
+
 .swipe_right,
 .swipe_left {
     width: 25px;
@@ -1257,6 +1260,7 @@ body .panelControlBar {
 }
 
 .swipes-counter {
+    opacity: 0.3;
     color: var(--SmartThemeBodyColor);
     font-size: 12px;
     padding: 0 5px;
@@ -1270,14 +1274,11 @@ body .panelControlBar {
     height: var(--swipeCounterHeight);
 }
 
-body:not(.swipeAllMessages) .mes:not(.last_mes) .swipes-counter {
-    visibility: hidden;
-}
 
 body.swipeAllMessages .mes:not(.last_mes) .swipes-counter {
     /* Avoid expensive DOM queries */
-    opacity: 0.3 !important;
-    display: flex !important;
+    opacity: 0.3;
+    display: flex;
 }
 
 .swipe_left {
@@ -1288,6 +1289,49 @@ body.swipeAllMessages .mes:not(.last_mes) .swipes-counter {
 .swipe_right {
     right: 5px;
     align-self: center;
+    display: flex;
+}
+
+/* The last swipe must be highlighted */
+.last_swipe :is(.swipes-counter, .swipe_right) {
+    opacity: 0.7;
+    /* Doubled opacity needs a longer transition to match a non active arrow. */
+    /* (Duration * Active Opacity / Inactive Opacity * 2) */
+    transition-duration: calc(var(--animation-duration-2x) * 0.7/0.3);
+}
+
+.fade :is(.swipes-counter, .swipe_left, .swipe_right){
+    /* Fade in and out: */
+    /* https://nerdy.dev/using-starting-style-and-transition-behavior-for-enter-and-exit-stage-effects */
+    /* https://developer.mozilla.org/en-US/docs/Web/CSS/transition-behavior#description */
+    transition: opacity calc(var(--animation-duration) * 2) ease-in;
+
+    /* Fade in. */
+    @supports (transition-behavior: allow-discrete) {
+        transition:
+            opacity var(--animation-duration-2x) ease-in,
+            display var(--animation-duration-2x) ease-in,
+            visibility var(--animation-duration-2x) ease-in;
+        transition-behavior: allow-discrete;
+    }
+}
+
+/* Hide all but the last message. */
+body:not(.swipeAllMessages) .mes:not(.last_mes) :is(.swipe_left, .swipe_right, .swipes-counter),
+/* Hide non-visible swipe_left. */
+.mes:not(.swipes_visible) .swipe_left,
+/* Only hide swipe_right if it's not visible, and not the last swipe. */
+.mes:not(.swipes_visible, .last_swipe) :is(.swipe_right),
+/* Hide all swipe buttons if .hideAllSwipeButtons. */
+body.hideAllSwipeButtons :is(.swipe_left, .swipe_right),
+.swipes-counter[hidden],
+.swipe_right[hidden],
+.swipe_left[hidden] {
+    /* Fade out */
+    opacity: 0 !important;
+    /* With display:none, the swipes-counter div wil collapse, shifting the chevron. */
+    visibility: hidden;
+    /* Interactivity has limited availability https://caniuse.com/?search=interactivity */
 }
 
 .ui-settings {
@@ -4445,11 +4489,12 @@ input[type="range"]::-webkit-slider-thumb {
     field-sizing: content;
 }
 
-body[data-generating="true"] #send_but,
-body[data-generating="true"] #mes_continue,
-body[data-generating="true"] #mes_impersonate,
-body[data-generating="true"] #chat .last_mes .mes_buttons,
-body[data-generating="true"] #chat .last_mes .mes_reasoning_actions {
+body:is([data-generating="true"], [data-swiping="true"]) :is(
+    #send_but,
+    #mes_continue,
+    #mes_impersonate,
+    #chat .last_mes .mes_buttons,
+    #chat .last_mes .mes_reasoning_actions) {
     display: none;
 }