I'm not going to advocate for completely abandoning semver, and I do think that people abuse it or try to use it in places where it might not have been originally intended.
That said, personally, my most hated anti-pattern (outside of not properly tagging broken interface versions), is people refusing to tag a 1.0. In theory, this allows folks to continue breaking the interface without calling it a broken interface. In practice, these packages are already widely deployed and NOT tagging a major version completely invalidates the versioning mechanism all together.

The versioning scheme cannot do its job unless if you use all three fields. So release a damn 1.0.0 already! Your software will never be perfect. it's ok.