Local Assets (GitLab)

Site-specific asset files (e.g. css or JavaScript) known as CMS Local and Global Assets allow sites to customize their CSS and JavaScript which live in GitLab and are managed outside of AEM.
Assets are served from a local copy on each of the Dispatcher servers, as well as a copy on the Author and Publisher servers.

How to Set Up Local Assets Repo

Create a Local Assets Repository in GitLab

When someone wants to make custom changes to their AEM site, they usually submit a ticket requesting for the creation of a local asset for their site. Here is a step-by-step guide of the process:

Create a blank repository on GitLab under /cms/cms-asset.
The project's domain name is used as the project name, for example, www.example.vt.edu
Check Initialize repository with a README
OPTIONAL: Sometimes, the person requesting for the local assets may need the base files to be included in the repository. If that is the case, download the 3 base files from here and upload it to the repository.
When you are done creating the project, go to Webhooks under the project settings and add new webhook.
In the Webhook settings page:
- URL: https://apps.es.vt.edu/CmsAssetsAgent/sync
- Secret token: Details under "Generate Secret Token for GitLab Webhook." The secret token should be filled like this: Bearer eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Mjc3NDg3NTAuOTQ2ODQsImlhdCI6MTYyNTE1Njc1MC45NDY4NDksImlzcyI6InJmZW5uIiwic2NvcGVzIjpbInN0YXR1cyIsImFzc2V0czphZG1pbiJdLCJzdWIiOiJUZXN0IEFkbWluIFRva2VuIn0.ZaVazyDWZx80NsVxJuTWU_98naMx7uq8EaraN0Sn5HGZ3zvnYp-23JNxE-7vG73_sAPvYvQxM74zzKwZVKclAw. Take note of the Bearer word before the secret token.
- Trigger: Select Push Events > Wildcard pattern
- Finally, click on Add webhook.
Next, add a Deploy key. To do this, go to Repository under the project settings and expand Deploy keys. Under the Privately accessible deploy keys tab, find and enable CMS Assets Sync and that's all for enabling the Deploy keys.
When a project is created, the default branch is main but changes get pushed to the servers only based on a prod, pprd, dvlp or test branch. Under this project, create different branches to match the differernt tiers of our servers.

NOTE: This step should be done last in order to confirm that the steps #6 and #7 are working when the webhook is triggered as these branches are created. This way, additional step won't be necessary to test the webhook.

Generate Secret Token for GitLab Webhook (JWT)

Some useful Docker Notes

1) First, determine the container running asset sync agent, log into the "primary" (smarm manager) server, for example, dock55150.it.vt.edu and switch to docker user.

grep for cms: $ docker stack ls | grep '^cms' and you will locate cms-assets-agent-prod
Find the particular node running the agent with:$ docker stack ps cms-assets-agent-prod
Shell into this node, for example: dock57152.it.vt.edu and find the ID of the container running the agent, for example: docker ps -a | grep "cms-assets"
Finally, connect into the actual image/container running the application, insert the container id from above into the following command:

$ docker container exec -it 3a6e154f3259 bash
appsadm@3a6e154f3259:~$ python agent.py --help
usage: agent.py [-h] --subject SUBJECT --scopes {assets:sync,status,assets:admin,debug} --issued-by ISS [--weeks WEEKS] [--days DAYS] [--hours HOURS]
                [--keyfile KEYFILE]

optional arguments:
  -h, --help            show this help message and exit
  --subject SUBJECT     Token user (ex: code.vt.edu/cms-assets)
  --scopes {assets:sync,status,assets:admin,debug}
                        Actions the token is authorized to make
  --issued-by ISS       Your Name or PID (Token Issuer)
  --weeks WEEKS         Token lifetime in WEEKS. Specify one of WEEKS, DAYS, HOURS.
  --days DAYS           Token lifetime in DAYS. Specify one of WEEKS, DAYS, HOURS.
  --hours HOURS         Token lifetime in HOURS. Specify one of WEEKS, DAYS, HOURS.
  --keyfile KEYFILE     Ed25519 key file
appsadm@3a6e154f3259:~$

2) Generate a JWT with assets:sync scope and we want a JWT with a very long lifetime

appsadm@3a6e154f3259:~$ python agent.py --subject 'code.vt.edu' --issued-by 'rfenn' --scopes 'status' --scopes 'assets:sync' --days 730
---
{'sub': 'Test Admin Token', 'scopes': ['status', 'assets:admin'], 'iat': '2021-07-01T16:25:50.946849+00:00', 'exp': '2021-07-31T16:25:50.946840+00:00'}
Token Expires: 2021-07-31 16:25:50.946840
---
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Mjc3NDg3NTAuOTQ2ODQsImlhdCI6MTYyNTE1Njc1MC45NDY4NDksImlzcyI6InJmZW5uIiwic2NvcGVzIjpbInN0YXR1cyIsImFzc2V0czphZG1pbiJdLCJzdWIiOiJUZXN0IEFkbWluIFRva2VuIn0.ZaVazyDWZx80NsVxJuTWU_98naMx7uq8EaraN0Sn5HGZ3zvnYp-23JNxE-7vG73_sAPvYvQxM74zzKwZVKclAw

Update Ticket / Notify the Requestor

Before updating the ticket, add the requestor to the project (GitLab repository you creaded above) as a Developer using the person's email address or PID from the ticket.

Update the ticket with a note similar to:

Your repository for local assets has been set up. You can access your repository at: <link to the assets repo you just made>

Also, you can follow this documentation on how to configure your site to use local assets: https://ensemble.cms.vt.edu/docs/site-config.html. 
You can scroll down to the part that talks about Local Assets.

Please, let us know if you have any issues setting this up.

Manually Sync Local Assets

Manually Sync a Repository

We can manually sync a repository on all the target machines by sending a minimal JSON payload to the /manual-sync endpoint. This requires a JWT with assets:admin scope.

Parameter	Example Value	Content
Ref	"refs/heads/pprd"	Branch to update
Repo	"git@code.vt.edu:enterprise-systems/cms-assets-test/test.git"	Repository SSH URL

The ref parameter is typically refs/heads/<branch_name> as you would see it in code.vt.edu. (This is the format the Gitlab webhook payload uses). The repo is the value you would find in the "Clone with SSH" link for the repository. The target dispatcher, AEM author and AEM publisher machines use ssh to clone and/or update the repository locally. If you need one, generate a JWT as shown above. Python example - this can be run within the container.

appsadm@3a6e154f3259:~$ python
Python 3.9.5 (default, May 12 2021, 15:36:59) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json, httpx
>>> headers = {'Authorization': 'Bearer eyJhbGc...'}
>>> import json, httpx
>>> from pprint import pprint
>>> payload = {'ref':'refs/heads/pprd', 'repo': 'git@code.vt.edu:enterprise-systems/cms-assets-test/test.git'}
>>> pprint(httpx.post('http://localhost:8000/manual-sync', headers=headers, data=json.dumps(payload)).json())
{'_status': 'ok', 'targets': 5}

Manually Sync a Target

We can use curl to send a POST to the sync target which tells it to sync, exactly the same way the agent does. First, you will need the Bearer token, which can be found in the running CMSAssetsAgent container at /var/run/secrets/assets-agent-bearer-token, or in Vault at secrets/es.dbaa.app-delivery/CMS/cms-assets-agent-bearer-token.

Export your token as an environment varible (Note: be sure to put a space before the export command so it will not be saved in your shell history!)

$  export AUTH_TOKEN="Bearer abcdef0123456789"

The URL format is https://<host>:<port>/cgi-bin/notify-receiver/<git_ssh_url>/-/<branch> 1. is the same value used to clone the reposotory (from Gitlab), minus the suffix .git 2. will be one of dvlp, pprd, prod 3. This POST contains no body, but ModSecurity insists we be explicit about this, so we must specify Content-Length: 0

 curl -XPOST -H "Authorization: ${AUTH_TOKEN}" -H "Content-Length: 0" https://cmsw-pprd-01.hosting.vt.edu:8443/cgi-bin/notify-receiver/git@code.vt.edu:cms/cms-assets/test/-/pprd
{"_status":"Ok","detail":"No change"}

The "Ok" status tells us this worked.