I have a 5900X AMD CPU which might cause some weird issues with VRChat, but I've done the priority hack and tried the CPU Core assign masks and no improvement.

The GPU is GTX 3090.

The debugger show red frames, but the frame time is very low.

The jitter look like roll back between every other frame for my own avatar and my own movement. Other's avatars look fine.

Thoughts on where to go from here?

its the game. happens to me

It does not happen to me on my lower end device, ironically enough...

It worsen over time until i restart SteamVR. Feels memory-leaky >.<

for me vrchat works well i have i7 8700k

Try turning off CPU OC and CPU CO(current optimization or something like that). Might help a bit.