Merge branch 'master' into feature/brawl-stars-api

.github/CONTRIBUTING.md

@@ -4,6 +4,8 @@
 opened to market company APIs that offer paid solutions. This API list is not a marketing tool, but a tool to help the
 community build applications and use free, public APIs quickly and easily. Pull requests that are identified as marketing attempts will not be accepted.
 >
+> Please make sure the API you want to add has a full free access or at least a free tier before submitting.
+>
 > Thanks for understanding! :)
 
 ## Formatting
@@ -12,7 +14,7 @@ Current API entry format:
 
 | API | Description | Auth | HTTPS | CORS |
 | --- | --- | --- | --- | --- |
-| API Title(Link to API webpage) | Description of API | Does this API require authentication? * | Does the API support HTTPS? | Does the API support [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)? * |
+| API Title(Link to API documentation) | Description of API | Does this API require authentication? * | Does the API support HTTPS? | Does the API support [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)? * |
 
 Example entry:
 

.github/PULL_REQUEST_TEMPLATE.md

-Thank you for taking the time to work on a Pull Request for this project!
-
-To ensure your PR is dealt with swiftly please check the following:
-
-- [ ] Your submissions are formatted according to the guidelines in the [contributing guide](CONTRIBUTING.md)
-- [ ] Your additions are ordered alphabetically
-- [ ] Your submission has a useful description
+<!-- Thank you for taking the time to work on a Pull Request for this project! -->
+<!-- To ensure your PR is dealt with swiftly please check the following: -->
+- [ ] My submission is formatted according to the guidelines in the [contributing guide](CONTRIBUTING.md)
+- [ ] My addition is ordered alphabetically
+- [ ] My submission has a useful description
 - [ ] The description does not end with punctuation
-- [ ] Each table column should be padded with one space on either side
-- [ ] You have searched the repository for any relevant issues or pull requests
-- [ ] Any category you are creating has the minimum requirement of 3 items
+- [ ] Each table column is padded with one space on either side
+- [ ] I have searched the repository for any relevant issues or pull requests
+- [ ] Any category I am creating has the minimum requirement of 3 items
 - [ ] All changes have been [squashed][squash-link] into a single commit
 
 [squash-link]: <https://github.com/todotxt/todo.txt-android/wiki/Squash-All-Commits-Related-to-a-Single-Issue-into-a-Single-Commit>

.github/workflows/test.yml

 name: "Run tests"
 
 on:
-  - push
-  - pull_request
+  schedule:
+    - cron:  '0 0 * * *'
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
 
 env:
   FORMAT_FILE: README.md

.github/workflows/validate_links.yml

+name: "Validate links"
+
+on:
+  schedule:
+    - cron:  '0 0 * * *'
+  push:
+    branches:
+      - master
+
+env:
+  FORMAT_FILE: README.md
+
+jobs:
+  test:
+    name: 'Validate links'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Validate all links from README.md
+        run: build/validate_links.py ${FORMAT_FILE}

build/validate_links.py

@@ -11,9 +11,9 @@ def parse_links(filename):
     with open(filename) as fp:
         data = fp.read()
     raw_links = re.findall(
-        'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+',
+        '((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:\'\".,<>?«»“”‘’]))',
         data)
-    links = [raw_link.replace(')', '') for raw_link in raw_links]
+    links = [raw_link[0] for raw_link in raw_links]
     return links
 
 
@@ -22,9 +22,9 @@ def validate_links(links):
     print('Validating {} links...'.format(len(links)))
     errors = []
     for link in links:
-        h = httplib2.Http(disable_ssl_certificate_validation=True, timeout=5)
+        h = httplib2.Http(disable_ssl_certificate_validation=True, timeout=25)
         try:
-            resp = h.request(link, 'HEAD')
+            resp = h.request(link, headers={'user-agent': 'python-httplib2/0.9.2'})
             code = int(resp[0]['status'])
             # check if status code is a client or server error
             if code >= 404:
@@ -33,9 +33,14 @@ def validate_links(links):
             errors.append("TMO: " + link)
         except socket.error as socketerror:
             errors.append("SOC: {} : {}".format(socketerror, link))
+        except Exception as e:
+            # Ignore some exceptions which are not actually errors.
+            # The list below should be extended with other exceptions in the future if needed
+            if ((-1 != str(e).find("Content purported to be compressed with gzip but failed to decompress.")) and 
+                (-1 != str(e).find("[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)"))) :
+                errors.append("ERR: {} : {}".format(e, link))
     return errors
 
-
 if __name__ == "__main__":
     num_args = len(sys.argv)
     if num_args < 2: