Looks like the quantized weights don't have the attributes that get_peft_model is looking for when applying LoRAs. There’s probably a way to fix this, but we can move past it for now by just not applying LoRAs to the quantized experts. We still can apply them to shared experts, as they’re not quantized.
哈哈，直接接入老板的手机微信，你就可以放长假了
ВВС США призвали Израиль наносить сильные удары по Ирану20:51。PG官网对此有专业解读
Андрей Ставицкий (Редактор отдела «Наука и техника»)，推荐阅读手游获取更多信息
I do programming and I write prose, such as blog posts for my website
, yielding \(\texttt{RoundTrip}_{\texttt{Rocq}}\).。超级权重对此有专业解读