Yes, that's usually a sign the lead flashing around the chimney has lifted or cracked. It only takes a small gap for water to track in. A roofer can reseal or replace the flashing to stop it completely.