Sounds very much like the vanes in the VNT ( Variable Nozzle Turbo ) are carboned up ?

Have you ever had the error memory read on a diagnostic computer ? The usual code is P0234 Turbo/supercharger "A" over boost condition. It could have been caused by a faulty EGR valve or short journeys and / or low engine speeds.

There is a possibility of a sticky/seized vacuum actuator or holed diaphragm or split vac hose ? Best get it on a diag computer to check what code(s) are stored and take the appropriate action.

I had sticky vanes in my Seat Leon 1.9 TDI when I bought it 2 years ago and used this guide here : www.audi-sport.net/xf/threads/un-sticking-your-tdi.../ to free them off. Never had it go into limp mode since but I suppose the way I drive it, as opposed to the previous owner, helps as it regularly gets driven in a 'spirited manner' so it burns off the soot/carbon and gives the actuator a workout too.