If it's a diesel, vacuum pump should be supplying adequate vacuum while driving.
If gas, as you remove your foot from gas pedal vacuum will be created to supply vacuum needed.
However, the brake booster should hold enough vacuum for two full brake applications...between usages (period of driving). Do you notice a hard pedal, after a lengthy period of driving (expressway driving)? Or, does engine run rough while pedal is applied at idle? There is a one-way check valve at booster end of hose from engine. It could be getting weak or rubber grommet could be failing. If you lose all vacuum within a short amount of time and those checkout okay, might be internal failure inside booster.