"Labor" Day

Why is there a holiday called "Labor Day"?  Isn't it just a little assinine to call it a day of work when you give people a day off.  I'd understand it better if it were a day where women could deliver their babies, or people got to repair roads for a day with the chain gang.  But firing up the grill and relaxing is the exact opposite of what I consider "labor."  It's a royal waste of time.  There's only one holiday that's worth anything: Boss's Day.  Give thanks that you even have a job, and do some actual work this Labor Day.