I do not see any problem for using set bouncemail. As per manual page:
set bouncemail Direct error mail to the sender (default) set no bouncemail Direct error mail to the local postmaster (as per the ’postmaster’ global option above). On my understanding comment "Avoid loss on 4xx errors." could mean following:
Errors starting on number 4 are temporary errors. Errors 4xx are not fatal errors and mean "I have a small problem at moment and can not process your message, you should try later". Example of this error would be 451 Temporary lookup failure. In this case sender server (if configured correctly per RFC) will try to deliver the message again later. If by the time of next attempt problem on receiving server will be solved - message will be delivered. If problem persists - than again it will give 451 Temporary lookup failure. This scenario will continue until message will expire (which is separate setting for mail server).
Now, how this can be applied to our situation. Let's say someone's server is not configured properly and thus on receiving 4xx error it will not try later - it will just delete the message. And he sends you e-mail while you have temporary lookup problem. So your postfix returns 451 error. Sender's server receives 451 error and stops any further attempts to deliver it (which is WRONG). At this point you lost your e-mail letter because you had temporary problem AND because of wrong configuration of another server.
So, fetchmail is trying to avoid the problem by not bouncing e-mail back in case of 4xx errors. Since it is already downloaded and it is already on your computer it does make some sense to just keep the message and not send it back.
Just enable set bouncemail and do not worry about anything, since majority of mail servers have proper (per RFC ) configuration and they will try to deliver messages again. You will have 1-5% chances to lost 1 e-mail in a while from wrong configured mail server.
On this:
On further thinking it seems to be silly to download such large messages, process and then bounce them. fetchmail has an option to limit the size of messages to be downloaded, but I cannot figure out how to send a bounce message in this case (and the message stays on server). Is there any way to do the trick?
Fetchmail must download the message if you want postfix to process the message. If fetchmail will not download the message - postfix will never see it. I doubt fetchmail will inform sender about that. If you want to inform sender about that - you have to have fetchmail to download the message and give it to postfix.
Another possible option - fetchmail downloads messages from somewhere (IMAP/POP3). If you have control over this POP3 account - set limit there and it will work as well.