How low should I get my LDL with diet?

I am f43, always been normal weight, good blood pressure, non smoker, only risk factor is I have family history of heart disease (father passed at 55 from heart attack, but he was a life long smoker and alcoholic. Brother also has high cholestrol and is on statin). My total cholesterol has been high my whole life. But LDL has creeped up the last years so now I need to do something about it.

Here are my most recent numbers. ApoB was measured last year. All others taken this October. Range in the brackets. And i used a calculator to convert the numbers so easier to interpret.

Chol. 5.8 mmol/l (<5) = 224 mg/dL

LDL 4.5 mmol/l (<3) = 174 mg/dL

HDL 1.78 mmol/l (> 1.2) = 68 mg/dL

Trig 0.3 mmol/l (< 1.7) = 26 mg/dL

Apo B 0.98 g/l (<0.90) = 98 mg/dL

Lipo(a) 60mg/l (<250) = 6 mg/dl

So now I am doing a 2 month diet intervention. I already am a vegetarian, so meat is not an issue. I don’t eat fish, that’s a negative. I am going to check saturated fat in everything I eat, quit cheese and chocolate (hard!), ice cream, cream in cooking etc. I’ve already eaten fair amount of veggies, beans and legumes regularly but will increase, also more oats and chia. I’ve already started supplementing with psyllium husk daily.

I do not know if I should add a vegan omega supplement (since i don’t eat fish) on top of this or not, any ideas? If i add it, I fear I will then not know if it was the supplement or the diet that helped.

So my question is, how low should i try to get my LDL and total cholesterol? What would be the number where I honestly don’t need statin?

I am asking this so when I speak with a doctor I know a little where that threshold is. I’ve asked about a statin before and it has been refused due to my low overall risk. So I’m just worried that my numbers come down a little and the doctor will refuse it again. And I am left with still relatively high numbers but no medication.