If processing one payment per vendor (default behaviour), vendor credits may be marked and will be applied to the selected bills. If the overall balance for the vendor is in credit, no payment will be processed and no credits will be applied.
If making one payment per bill (check Separate Payment for Each Bill on Pay Bills screen), vendor credits cannot be marked and cannot be applied.