Do coconut milk calories count since coconut oil is free?

Absolutely. Coconut milk is loaded with natural sugars so watch out for the sugar in coconut milk, and add those calories toward your limits.