rmsnorm
rope
gqa
residual
rmsnorm
swiglu or moe
residual