I think you can start, but it's not a definitive answer because I don't really remember the first volumes.
So I know that the anime adapt the first 3 volumes, since in my country light novel is not licensed there are only amateur translation and they haven't translated the firsts 3 volumes, my question is: can I start reading the 4th volume or there are important things in the previous volumes not adapted in the anime?
I think you can start, but it's not a definitive answer because I don't really remember the first volumes.
The anime changed somethings and skipped a lot of smaller details so I suggest that you start from the beginning.
Highly recommend start reading from volume 1. The anime changed a lot of stuff that. U even get to read ayanokoji thought process unlike the anime which is what makes the LN great in the first place
You should read from the beginning, anime skipped a huge character story arc, and many fans are mad about that, and I think reading from the start also helps with understanding the later volumes