Apartment smells like gas? Super says not to call 911 or ConEd?

So we had no heat and hot water since Thursday, but we got it back today. Now that it's been turned back on, the lobby next to the boiler smells like gas. We told the super and he came to inspect and couldn't find a gas leak, and said it was something to do with the hot water's gas valve.

Prior to this, the super said that ConEd inspected all the pipes and turned the gas back on the building - but we've seen no evidence of this, other than the gas and heat working again.

The super also said do not call ConEd because they will come and shut the gas off again, and it could take months (friends have been through this). Which is obviously going to suck.

So we're in a hard place - we (the tenants) are thinking of hiring an independent inspector to come and sign off.

Any thoughts?