This HuffPo article is worth reading:
According to an Ernst & Young internal study, for "each additional 10 hours of vacation employees took, their year-end performance ratings improved 8 percent, and frequent vacationers also were significantly less likely to leave the firm".
By the way... using your vacation time one day here and a long weekend there is not a "vacation". Generally I find that you're brain doesn't relax until day 3 of a vacation, especially if there is travel involved. If you don't take a long break you never reach that point. If you take a lot of long weekends you end up spending that time doing laundry and you rob yourself of the opportunity to actually relax.
That's why my time management book has an entire chapter just on vacations and relaxation. Taking a vacation isn't wasting time... it is an investment in yourself.