Great question. I asked the same thing.
Keep in mind that the anime is like 5 times faster than reading the light novel so conversations not central to the plot are cut out completely.
I think the anime leaves us to assume Sakura is in Ayanokouji's back pocket and their relationship isn't changing. And it moves on to Ayanokouji finding his next bitc--female follower, Karuizawa.
