That reminds me of a time I found a rock in a creek that looked totally dull. Turned it in the light and caught a tiny shimmer, just like garcia.kelly said. Your mileage may vary, but water can really hide that phyllite shine.
Slate is completely smooth and breaks in flat sheets. Phyllite has a faint satiny shine to it because its mica flakes are just a bit bigger. If your rock has any kind of shimmer when you turn it in the light, it’s phyllite. That change happens with a little more heat and pressure on the slate. Creek tumbling can polish it and make that shine harder to see, so check a fresh broken edge. Still just looks flat and dull gray? Probably slate.