as if to sayOften indicates that someone acts or reacts as though expressing the preceding words or idea. In そうとばかりも言えない, it is part of the expression meaning “not necessarily so.”
JA: 彼は待っていたとばかりに、すぐ話し始めた。
EN: As if he had been waiting, he immediately started talking.