Soon as you start up the vacuum pump does its job, presumably as with other modern cars (not needed on my old things..:-) you have to press the brake pedal to start the engine and you're feeling the vacuum assistance which usually causes the pedal to drop a little.

Pedal doesn't usually clunk though in my experience so might be worth getting someone to operate the pedal for you whislt you have a good poke nose round the pedal box inside, pedal hinges etc and under the bonnet the vaccum servo unit and brake master cyl, just to make sure there's nothing amiss....maybe it is indeed a ventilation flap or something, maybe try removing you're foot as soon as the engine catches to eliminate the brake assembly from the noise?

Doing this intermittently its probably losing vacuum over a period of a few hours, thats nothing to worry about, but might explain why it doesn't always perform for you.

Try pressing the brake pedal several times engine off to use up the vacuum, if it clunks then at least you will be able to demonstrate to the mechanic exactly whats happening, its probably nothing too alarming but i too would be checking it out.