We had our service technician out again friday. He has now come to the conclusion that it could be the gas pressure. I know he tested it, probably with the manometer, and found there was not enough pressure coming through, especially when both heaters are running. He said he adjusted the regulator and now the gas pressure should be sufficient. However, he also opened all of our water valves leading to the heater, which dropped my flow rate. I checked the heaters when I came in this morning and they were not running with both thermostats at max. I tweaked the valves to get the flow rate up and magically the heaters lit.
I was questioning the oxygen flow in that room too, but there are four rollout windows that we always have open. I am guessing this should be sufficient air flow.