Is a bottled water from a water cooler okay to drink after a couple of weeks?

In most cases yes. Don’t drink it if it has a different color, taste or smell.

Water from a water cooler is fine for months (sometimes even years).