Don't add useless labels which will bother changelog generation (#37267)

When generating release notes for v1.26, many pull requests haven't been
given correct labels so that I have to do many manual work. I think this
could be avoid to remove these useless modify labels.

.changelog.yml

@@ -37,10 +37,7 @@ groups:
     name: BUGFIXES
     labels:
       - type/bug
-  -
-    name: API
-    labels:
-      - modifies/api
+
   -
     name: TESTING
     labels: