AITA for wanting to make a food budget

Me (23f) and my partner (27) live together with their family (54m, 25f, 16f). Times are a bit tough right now with money and so we are on a tighter budget. Me and my partner eat very differently than the rest of the house (except for a couple of shared food items like eggs and flour), but are still expected to contribute to buying food that the rest of the family eats. We decided to budget out our money to make sure we would be able to afford things month to month, and our grocery budget is barely enough to feed the two of us. When trying to communicate this to the family, we got shut down and told that it was rude to limit our expenses on food because they might need us to buy food for the house (which is usually only food they eat). So we offered to pool our money into a household food budget but that also was not received well, since the family does not budget anything that they spend and think it's unreasonable. I think me and my partner should put our foot down, we only have a limited amount of money and we can't afford to spend extra any time someone asks, and we also offered an alternative where we could budget out everything together. AITA for wanting to make a budget and not spend extra money every time to buy food we don't eat?