Guide to set your own automated visual diff checker with GitHub Actions.
It is important to keep an eye on both your own and your competitor’s landing pages. You can use the visual diff checker to make sure everything is fine with your sites after each deployment or you can use it to keep track of your competitor's pages. Even you can track the changes on SERP :)
So here is a summary of how it will work
We will use jest/puppeteer/nodejs with git actions to do the structural similarity (SSIM) index test between the mobile/desktop version of snapshots and in case of any change, the app will send an HTML report with attached pdfs to your mail address.
Like below, I am able to get notifications when something changed on Serp related to my name.
Diff check for small changes inside the list element.
Example of email report
What are the changes from the forked project?
I started by removing everything that is redundant, such as mobile/desktop pixel comparison. So, in the end, we will have only a visual regression test.
We will force the puppeteer to wait till the load page fully as the original version was failing to load the lazy images and scrolling the page.
Changed the comparison library from pixel match to ssim
To send a report via mail, an HTML reporter is added.
Nodejs script that will be fired each time when one of the tests failed. This script will send a mail with HTML report/compress diff images and attach visual diffs as pngs.
Lastly, git actions will deal with cronjob so you can decide when it should be fired.
Let’s start with the basics.
Before explaining how to use/modify the project, you need to install a few things.
Install Node.js using the link: https://nodejs.org/en/download/
Install yarn https://classic.yarnpkg.com/en/docs/install/#mac-stable
Install git https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
Install any of the code editor (I prefer sublime) https://www.sublimetext.com/3
I was trying to code everything from the scratch, however, I come across a nice GitHub project https://github.com/jflagarde/website-visual-testing.So I forked the project and start to modify it. (If you like to skip everything and clone my project here is the link :) )
Let's start with cloning the project.
Go to https://github.com/emrecan-s/website-visual-testing and fork the project
Copy your project URL
Open a terminal and write
git clone “your project URL”
Then cd website-visual-testing
Then yarn install
After you install everything you need to change a few things inside the code to keep track of your desired pages.
Open the sublime and open the folder that the project in
Open the site.config.ts and change names, page URLs that you would like to follow. Also if you want to remove some dynamic elements/pop-ups which can cause false-positive you can add them in here to inside the elements to remove the part. For example, I removed the consent pop-up on Google search by adding the ‘.hide-focus-ring’ selector. (Well normally I should force the puppeteer to click on the consent button but as an example, I removed that :) ) You can do the same if you have animations etc. that you can ignore while comparison the images.
It is important that you changed the name as the folder will be the same as the name and snapshots will be under that folder.
After this, open the jest.setup.js — you can set the threshold which means the % of the change to send the mail. In my case, that is 1%.
Next, open the mailer.js and change the receiver mail address.
Next, you need to open a .env file to locally test the script before uploading it to GitHub.
save as .env
Open the .env file and enter your mail credentials
The important thing to do is to allow your google account to less secure app access (necessary to use nodemailer)
Go to https://myaccount.google.com/security and turn on the less secure app access
Next, in order to create the first snapshot that we are going to use for comparison, open the terminal inside the project and type yarn test
After the test, you will have a folder inside the __output__ folder with snapshots.
Now go to GitHub project and click on settings than secrets
Next, click on new repository secret and add PASSWORD AND USERNAME values as we have inside the .env file
Next, open the terminal inside the project folder and write
git add .
git commit -m “project settings are changed”
git push origin master
Now the action should be able the do the test every day at 00:00.
If you want to change the frequency you can change the on: cron settings inside the integration-tests.yml file.
With this site, you can validate your cron schedule expression.
What you can improve:
If you want to test too many pages git actions would not be enough for you so you can either use Gcloud/AWS or Heroku for the tests.
Sending mail might be an issue if the attachment size more than 25MB so instead of sending emails you can upload them to cloud storage or Gdrive.
You can check your title/description/content changes in case you want exact info about what is changed.
+The rest depends on your creativity :)
Feel free to reach me if you have any questions/suggestions or feedback(good or bad, all are welcomed :)) https://www.linkedin.com/in/emrecansanli/
Lastly, I would like to thank Jean-Francois Lagarde for his project that helped me a lot, Muhammed Tanrikulu for his guidance when I stuck on a problem, and lastly Anna Tumanova for testing the guide as a new user :).
